Natural language understanding - AbsoluteAstronomy.com

Natural language understanding is a subtopic of natural language processing

Natural language processing

Natural language processing is a field of computer science and linguistics concerned with the interactions between computers and human languages; it began as a branch of artificial intelligence....

in artificial intelligence

Artificial intelligence

Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...

that deals with machine reading comprehension

Reading comprehension

Reading comprehension is defined as the level of understanding of a text. This understanding comes from the interaction between the words that are written and how they trigger knowledge outside the text. ....

.

The process of disassembling and parsing input is more complex than the reverse process of assembling output in natural language generation

Natural language generation

Natural Language Generation is the natural language processing task of generating natural language from a machine representation system such as a knowledge base or a logical form...

because of the occurrence of unknown and unexpected features in the input and the need to determine the appropriate syntactic and semantic schemes to apply to it, factors which are pre-determined when outputting language.

There is considerable commercial interest in the field because of its application to news-gathering, text categorization, voice-activation, archiving and large-scale content-analysis.

History

The program STUDENT

STUDENT (computer program)

STUDENT is an early artificial intelligence program that solves algebra word problems. It is written in Lisp by Daniel G Bobrow as his PhD thesis in 1964 . It was designed to read and solve the kind of word problems found in high school algebra books...

, written in 1964 by Daniel Bobrow for his PhD dissertation at MIT is one of the earliest known attempts at natural language understanding by a computer. Eight years after John McCarthy

John McCarthy (computer scientist)

John McCarthy was an American computer scientist and cognitive scientist. He coined the term "artificial intelligence" , invented the Lisp programming language and was highly influential in the early development of AI.McCarthy also influenced other areas of computing such as time sharing systems...

coined the term artificial intelligence

Artificial intelligence

, Bobrow's dissertation (titled Natural Language Input for a Computer Problem Solving System) showed how a computer can understand simple natural language input to solve algebra word problems.

A year later, in 1965, Joseph Weizenbaum

Joseph Weizenbaum

Joseph Weizenbaum was a German-American author and professor emeritus of computer science at MIT.-Life and career:...

at MIT wrote ELIZA

ELIZA

ELIZA is a computer program and an early example of primitive natural language processing. ELIZA operated by processing users' responses to scripts, the most famous of which was DOCTOR, a simulation of a Rogerian psychotherapist. Using almost no information about human thought or emotion, DOCTOR...

, an interactive program that carried on a dialogue in English on any topic, the most popular being psychotherapy. ELIZA worked by simple parsing and substitution of key words into canned phrases and Weizenbaum sidestepped the problem of giving the program a database

Database

A database is an organized collection of data for one or more purposes, usually in digital form. The data are typically organized to model relevant aspects of reality , in a way that supports processes requiring this information...

of real-world knowledge or a rich lexicon

Lexicon

In linguistics, the lexicon of a language is its vocabulary, including its words and expressions. A lexicon is also a synonym of the word thesaurus. More formally, it is a language's inventory of lexemes. Coined in English 1603, the word "lexicon" derives from the Greek "λεξικόν" , neut...

. Yet ELIZA gained surprising popularity as a toy project and can be seen as a very early precursor to current commercial systems such as those used by Ask.com

Ask.com

Ask is a Q&A focused search engine founded in 1996 by Garrett Gruener and David Warthen in Berkeley, California. The original software was implemented by Gary Chevsky from his own design. Warthen, Chevsky, Justin Grant, and others built the early AskJeeves.com website around that core engine...

.

In 1969 Roger Schank

Roger Schank

Roger Schank is an American artificial intelligence theorist, cognitive psychologist, learning scientist, educational reformer, and entrepreneur.-Academic career:...

at Stanford University

Stanford University

The Leland Stanford Junior University, commonly referred to as Stanford University or Stanford, is a private research university on an campus located near Palo Alto, California. It is situated in the northwestern Santa Clara Valley on the San Francisco Peninsula, approximately northwest of San...

introduced the conceptual dependency theory

Conceptual dependency theory

Conceptual dependency theory is a model of natural language understanding used in artificial intelligence systems.Roger Schank at Stanford University introduced the model in 1969, in the early days of artificial intelligence...

for natural language understanding. This model, partially influenced by the work of Sydney Lamb

Sydney Lamb

Sydney MacDonald Lamb is an American linguist and professor at Rice University, whose stratificational grammar is a significant alternative theory to Chomsky's transformational grammar....

, was extensively used by Schank's students at Yale University

Yale University

Yale University is a private, Ivy League university located in New Haven, Connecticut, United States. Founded in 1701 in the Colony of Connecticut, the university is the third-oldest institution of higher education in the United States...

, such as Robert Wilensky, Wendy Lehnert, and Janet Kolodner.

In 1970, William A. Woods introduced the augmented transition network

Augmented transition network

An augmented transition network is a type of graph theoretic structure used in the operational definition of formal languages, used especially in parsing relatively complex natural languages, and having wide application in artificial intelligence...

(ATN) to represent natural language input. Instead of phrase structure rules
Phrase structure rules
Phrase-structure rules are a way to describe a given language's syntax. They are used to break down a natural language sentence into its constituent parts namely phrasal categories and lexical categories...

ATNs used an equivalent set of finite state automata that were called recursively. ATNs and their more general format called "generalized ATNs" continued to be used for a number of years.

In 1971 Terry Winograd

Terry Winograd

Terry Allen Winograd is an American professor of computer science at Stanford University, and co-director of the Stanford Human-Computer Interaction Group...

finished writing SHRDLU

SHRDLU

SHRDLU was an early natural language understanding computer program, developed by Terry Winograd at MIT from 1968-1970. In it, the user carries on a conversation with the computer, moving objects, naming collections and querying the state of a simplified "blocks world", essentially a virtual box...

for his PhD thesis at MIT. SHRDLU could understand simple English sentences in a restricted world of children's blocks to direct a robotic arm to move items. The successful demonstration of SHRDLU provided significant momentum for continued research in the field. Winograd continued to be a major influence in the field with the publication of his book Language as a Cognitive Process. At Stanford, Winograd was later the adviser for Larry Page

Larry Page

Lawrence "Larry" Page is an American computer scientist and internet entrepreneur who, with Sergey Brin, is best known as the co-founder of Google. As of April 4, 2011, he is also the chief executive of Google, as announced on January 20, 2011...

, who co-founded Google

Google

Google Inc. is an American multinational public corporation invested in Internet search, cloud computing, and advertising technologies. Google hosts and develops a number of Internet-based services and products, and generates profit primarily from advertising through its AdWords program...

.

In the 1970s and 1980s the natural language processing group at SRI International

SRI International

SRI International , founded as Stanford Research Institute, is one of the world's largest contract research institutes. Based in Menlo Park, California, the trustees of Stanford University established it in 1946 as a center of innovation to support economic development in the region. It was later...

continued research and development in the field. A number of commercial efforts based on the research were undertaken, e.g., in 1982 Gary Hendrix

Gary Hendrix

Gary Hendrix is a natural language analyst who founded Symantec Corporation, an international corporation which sells computer software, particularly in the fields of information management and antivirus software.-Personal life:...

formed Symantec Corporation originally as a company for developing a natural language interface for database queries on personal computers. However, with the advent of mouse driven, graphic user interfaces Symantec changed direction. A number of other commercial efforts were started around the same time, e.g., Larry R. Harris at the Artificial Intelligence Corporation and Roger Schank and his students at Cognitive Systems corp. In 1983, Michael Dyer developed the BORIS system at Yale which bore similarities to the work of Roger Schank and W. G. Lehnart.

Scope and context

The umbrella term "natural language understanding" can be applied to a diverse set of computer applications, ranging from small, relatively simple tasks such as short commands issued to robot

Robot

A robot is a mechanical or virtual intelligent agent that can perform tasks automatically or with guidance, typically by remote control. In practice a robot is usually an electro-mechanical machine that is guided by computer and electronic programming. Robots can be autonomous, semi-autonomous or...

s, to highly complex endeavors such as the full comprehension of newspaper articles or poetry passages. Many real world applications fall between the two extremes, for instance text classification for the automatic analysis of emails and their routing to a suitable department in a corporation does not require in depth understanding of the text, but is far more complex than the management of simple queries to database tables with fixed schemata.

Throughout the years various attempts at processing natural language or English-like sentences presented to computers have taken place at varying degrees of complexity. Some attempts have not resulted in systems with deep understanding, but have helped overall system usability. For example, Wayne Ratliff

Wayne Ratliff

Cecil Wayne Ratliff wrote the database program Vulcan. Raised in Ohio and Germany, he now resides in the Los Angeles area....

originally developed the Vulcan program with an English-like syntax to mimic the English speaking computer in Star Trek

Star Trek

Star Trek is an American science fiction entertainment franchise created by Gene Roddenberry. The core of Star Trek is its six television series: The Original Series, The Animated Series, The Next Generation, Deep Space Nine, Voyager, and Enterprise...

. Vulcan later became the dBase

DBASE

dBase II was the first widely used database management system for microcomputers. It was originally published by Ashton-Tate for CP/M, and later on ported to the Apple II and IBM PC under DOS...

system whose easy-to-use syntax effectively launched the personal computer database industry. Systems with an easy to use or English like syntax are, however, quite distinct from systems that use a rich lexicon

Lexicon

and include an internal representation (often as first order logic) of the semantics of natural language sentences.

Hence the breadth and depth of "understanding" aimed at by a system determine both the complexity of the system (and the implied challenges) and the types of applications it can deal with. The "breadth" of a system is measured by the sizes of its vocabulary and grammar. The "depth" is measured by the degree to which its understanding approximates that of a fluent native speaker. At the narrowest and shallowest, English-like command interpreters require minimal complexity, but have a small range of applications. Narrow but deep systems explore and model mechanisms of understanding, but they still have limited application. Systems that attempt to understand the contents of a document such as a news release beyond simple keyword matching and to judge its suitability for a user are broader and require significant complexity, but they are still somewhat shallow. Systems that are both very broad and very deep are beyond the current state of the art.

Components and architecture

Regardless of the approach used, some common components can be identified in most natural language understanding systems. The system needs a lexicon

Lexicon

of the language and a parser and grammar rules to break sentences into an internal representation. The construction of a rich lexicon with a suitable ontology requires significant effort, e.g., the Wordnet

WordNet

WordNet is a lexical database for the English language. It groups English words into sets of synonyms called synsets, provides short, general definitions, and records the various semantic relations between these synonym sets...

lexicon required many person-years of effort.

The system also needs a semantic theory
Semantics
Semantics is the study of meaning. It focuses on the relation between signifiers, such as words, phrases, signs and symbols, and what they stand for, their denotata....

to guide the comprehension. The interpretation capabilities of a language understanding system depend on the semantic theory it uses. Competing semantic theories of language have specific trade offs in their suitability
as the basis of computer automated semantic interpretation. These range from naive semantics
Naive semantics
Naive semantics is an approach used in computer science for representing basic knowledge about a specific domain, and has been used in applications such as the representation of the meaning of natural language sentences in artificial intelligence applications...

or stochastic semantic analysis
Stochastic semantic analysis
Stochastic semantic analysis is an approach used in computer science as a semantic component of natural language understanding.Stochastic models generally use the definition of segments of words as basic semantic units for the semantic models, and in some cases involve a two layered...

to the use of pragmatics
Pragmatics
Pragmatics is a subfield of linguistics which studies the ways in which context contributes to meaning. Pragmatics encompasses speech act theory, conversational implicature, talk in interaction and other approaches to language behavior in philosophy, sociology, and linguistics. It studies how the...

to derive meaning from context.

Advanced applications of natural language understanding also attempt to incorporate logical inference

Inference

Inference is the act or process of deriving logical conclusions from premises known or assumed to be true. The conclusion drawn is also called an idiomatic. The laws of valid inference are studied in the field of logic.Human inference Inference is the act or process of deriving logical conclusions...

within their framework. This is generally achieved by mapping the derived meaning into a set of assertions in predicate logic

Predicate logic

In mathematical logic, predicate logic is the generic term for symbolic formal systems like first-order logic, second-order logic, many-sorted logic or infinitary logic. This formal system is distinguished from other systems in that its formulae contain variables which can be quantified...

, then using logical deduction to arrive at conclusions. Systems based on functional languages such as Lisp hence need to include a subsystem for the representation of logical assertions, while logic oriented systems such as those using the language Prolog

Prolog

Prolog is a general purpose logic programming language associated with artificial intelligence and computational linguistics.Prolog has its roots in first-order logic, a formal logic, and unlike many other programming languages, Prolog is declarative: the program logic is expressed in terms of...

generally rely on an extension of the built in logical representation framework.

The management of context in natural language understanding can present special challenges. A large variety of examples and counter examples have resulted in multiple approaches to the formal modeling of context, each with specific strengths and weaknesses.

History

Scope and context

Components and architecture

See also