The targeted goal of natural language processing (NLP) is natural language understanding (NLU), the ability of the NLU agent (i.e. chatbot) to identify the user’s intent embedded in the human / machine dialogue. True NLU has been the dream and ambition of Artificial Intelligence (AI) researchers since A.M. Turing first proposed a test for an analogue to human intelligence over a half century ago [1]. It now appears that we might be getting close to realizing Turing’s vision with the advent of NLU agents incorporating machine learning (ML) within their architecture.

The improvements in NLP in AI agents is attributable to improvements in language parsing algorithms and the application of ML and recent advances in artificial neural network (ANN) paradigms.

This paper will examine modern language parsing techniques and applied ML to identify multiple intents within human / machine natural language discourse.

Linear symbolic NLP parsing algorithms

An NLP agent must be provided the syntax of the natural language from which it must infer meaning and a specific domain of inquiry presently. The NLP agent performs quite well within narrowly defined domains of discourse when the syntax of the target language has been defined well and provided to the NLP agent within a well-defined narrow domain.

Around 1968-1970, Terry Winograd, while associated with the MIT Artificial Intelligence Laboratory, incorporated sophisticated language syntax analysis with a well-defined domain of discourse, a block world, within an NLP algorithm that he named SHRDLU. SHRDLU was written in Lisp for the DEC PDP6 mini-computer [2].

In 1972, Winograd published a paper titled “Understanding Natural Language” in which he defines the NLP parsing agent of his SHRDUL program [3]. His model worked amazingly well for its time and handled multiple intents reasonably well.

Multiple intent identification by linear symbolic NLP parsing algorithms

One of the semantic decoding issues that must be addressed by an NLP agent is that of the meaning of specific words within the context of discourse in which they are found. For instance, in the author’s experimental model for his PhD dissertation, HAL (Heuristic Associative Linear-algorithm) is an artificial intelligence chess-playing engine containing an NLP parsing agent that derives word meaning from the context in which the words are found [4].

The word “Time,” for example, can be either a noun or a verb depending on the context in which it appears. In the request: “HAL, time the moves, please,” the NLP parsing agent recognizes “time” as a verb form. While in the request, “HAL, display the board each time a move is made,” the word “time” is recognized by the NLP agent within HAL, as a noun form based on the context of the word usage.

In the aforementioned examples, two separate intents are expressed, the first intent being a request for the automaton, to report the elapsed time required to produce a move response, and the second intent being a request to display the new chess-board after each move. HAL’s NLP parsing agent can easily isolate these two intents when each intent is given in a single input text expression. However, the isolation and identification of multiple conjoined intents within a single text expression requires the NLP parser to perform more than a context extrapolation.

One method for solving multiple intents embedded in a single expression is to separate each embedded expression at a conjunction. Typical conjunctions include: (and, but, or, nor), but may also include words and phrases such as: (also, additionally, in addition, but not) and so-forth. When the NLP parser encounters a conjunction in the input expression, the parser can separate the expression into two single subexpressions. Each subexpression can be enqueued and dequeued in order, one-subexpression-at-a-time for intent discovery and processing before the next subexpression is dequeued for NLP processing. This iterative process will be repeated until the expression-queue is empty.

For example, the following expression: “HAL, time the moves, please and display the board each time a move is made,” would be separated into two subexpressions at the conjunction “and.” Each resulting subexpression will be enqueued to be processed one-at-a-time, thus handling multiple intents occurring in a single conjoined text expression.

Machine learning as applied to NLP / NLU

Within the last few years, many advanced NLP and NLU agents have come to fruition, some of which are available within the Open Source community, such as the Rasa Core NLU paradigm. Rasa Core contains a machine learning component consisting of a Recurrent Neural Network (RNN) complemented with Long Short-Term Memory (LSTM) trained on intents within a specific domain.

The components of the Rasa Core consist of 1) an Interpreter that is the NLU parsing agent, 2) the Domain in which the chatbot is trained to interact, and 3) Stories that define the actions to be performed by the chatbot. The Domain consists of a file that is defined when the chatbot is implemented containing, Intents, Entities, Template, Actions, and Slots [5].

Conclusion

In the last several years, much advancement has been achieved toward more human-like conversational NLU paradigms. These advancements are largely due to the incorporation of Machine Learning algorithms in the Natural Language Understanding paradigms. However, the domains of influence are still quite narrow, making these systems brittle when the dialogue leaves the domains on which the NLU agent has been trained.

References

[1] Turing, A. M., “The Turing Test”, pub. (1950), in the Stanford Encyclopedia of Philosophy, First published Wed Apr 9, 2003; substantive revision Mon Feb 8, 2016, https://plato.stanford.edu/entries/turing-test/

[2] http://hci.stanford.edu/~winograd/shrdlu/

[3] Winograd, Terry, “Understanding Natural Language”, Massachusetts Institute of Technology Cambridge, Massachusetts, 1972, in Cognitive Psychology, vol. 3, l-191 (1972).

 [4] Wheeler, Stephen F., “HAL (Heuristic Associative Linear-algorithm): An Artificial Intelligence Chess-Playing Automaton with Natural Language Understanding”, No longer available on Chess.com. Source code, executable, and documentation available from author upon request by email at swheeler@ai-scientist.com.

[5] “Your Complete Guide to Building Stateless Bots Using Rasa Stack”, https://medium.com/velotio-perspectives/your-complete-guide-to-building-stateless-bots-using-rasa-stack-bc22a22e2fc7.