History of artificial intelligence
Encyclopedia
The history of artificial intelligence began in antiquity
, with myths, stories and rumors of artificial beings endowed with intelligence or consciousness by master craftsmen; as Pamela McCorduck
writes, AI began with "an ancient wish to forge the gods."
The seeds of modern AI were planted by classical philosophers who attempted to describe the process of human thinking as the mechanical manipulation of symbols. This work culminated in the invention of the programmable digital computer
in the 1940s, a machine based on the abstract essence of mathematical reasoning. This device and the ideas behind it inspired a handful of scientists to begin seriously discussing the possibility of building an electronic brain.
The field of artificial intelligence
research was founded at a conference on the campus of Dartmouth College
in the summer of 1956. Those who attended would become the leaders of AI research for decades. Many of them predicted that a machine as intelligent as a human being would exist in no more than a generation and they were given millions of dollars to make this vision come true. Eventually it became obvious that they had grossly underestimated the difficulty of the project. In 1973, in response to the criticism of Sir James Lighthill and ongoing pressure from congress, the U.S. and British Governments stopped funding undirected research into artificial intelligence. Seven years later, a visionary initiative by the Japanese Government inspired governments and industry to provide AI with billions of dollars, but by the late 80s the investors became disillusioned and withdrew funding again. This cycle of boom and bust, of "AI winter
s" and summers, continues to haunt the field. Undaunted, there are those who make extraordinary predictions even now.
Progress in AI has continued, despite the rise and fall of its reputation in the eyes of government bureaucrats and venture capitalists. Problems that had begun to seem impossible in 1970 have been solved and the solutions are now used in successful commercial products. However, no machine has been built with a human level of intelligence
, contrary to the optimistic predictions of the first generation of AI researchers. "We can only see a short distance ahead," admitted Alan Turing
, in a famous 1950 paper that catalyzed the modern search for machines that think. "But," he added, "we can see much that must be done."
in one form or another is an idea that has pervaded Western intellectual history, a dream in urgent need of being realized," expressed in humanity's myths, legends, stories, speculation and clockwork automaton
s.
and Pygmalion's
Galatea
.
In the Middle Ages, there were rumors of secret mystical or alchemical
means of placing mind into matter, such as Jābir ibn Hayyān's Takwin
, Paracelsus
' homunculus
and Rabbi Judah Loew's Golem
.
By the 19th century, ideas about artificial men and thinking machines were developed in fiction, as in Mary Shelley
's Frankenstein
or Karel Čapek
's R.U.R. (Rossum's Universal Robots)
,
and speculation, such as Samuel Butler
's "Darwin among the Machines
."
AI has continued to be an important element of science fiction
into the present.
Realistic humanoid automaton
s were built by craftsman from every civilization, including Yan Shi,
Hero of Alexandria
,
Al-Jazari
and Wolfgang von Kempelen
.
The oldest known automaton
s were the sacred statues
of ancient Egypt
and Greece
. The faithful believed that craftsman had imbued these figures with very real minds, capable of wisdom and emotion—Hermes Trismegistus
wrote that "by discovering the true nature of the gods, man has been able to reproduce it."
, Indian
and Greek
philosophers all developed structured methods of formal deduction in the first millennium BCE. Their ideas were developed over the centuries by philosophers such as Aristotle
(who gave a formal analysis of the syllogism
), Euclid
(whose Elements
was a model of formal reasoning), (who developed algebra
and gave his name to "algorithm
") and European scholastic
philosophers such as William of Ockham
and Duns Scotus
.
Majorcan philosopher Ramon Llull
(1232–1315) developed several logical machines devoted to the production of knowledge by logical means; Llull described his machines as mechanical entities that could combine basic and undeniable truths by simple logical operations, produced by the machine by mechanical meanings, in such ways as to produce all the possible knowledge. Llull's work had a great influence on Gottfried Leibniz
, who redeveloped his ideas.
In the 17th century, Leibniz
, Thomas Hobbes
and René Descartes
explored the possibility that all rational thought could be made as systematic as algebra or geometry.
Hobbes famously wrote in Leviathan
: "reason is nothing but reckoning".
Leibniz
envisioned a universal language of reasoning (his characteristica universalis
) which would reduce argumentation to calculation, so that "there would be no more need of disputation between two philosophers than between two accountants. For it would suffice to take their pencils in hand, down to their slates, and to say each other (with a friend as witness, if they liked): Let us calculate."
These philosophers had begun to articulate the physical symbol system
hypothesis that would become the guiding faith of AI research.
In the 20th century, the study of mathematical logic
provided the essential breakthrough that made artificial intelligence seem plausible. The foundations had been set by such works as Boole
's The Laws of Thought
and Frege's Begriffsschrift
. Building on Frege's system, Russell
and Whitehead
presented a formal treatment of the foundations of mathematics in their masterpiece, the Principia Mathematica
in 1913. Inspired by Russell
's success, David Hilbert
challenged mathematicians of the 1920s and 30s to answer this fundamental question: "can all of mathematical reasoning be formalized?"
His question was answered by Gödel
's incompleteness proof
, Turing
's machine
and Church
's Lambda calculus
.
Their answer was surprising in two ways. First, they proved that there were, in fact, limits to what mathematical logic could accomplish.
But second (and more important for AI) their work suggested that, within these limits, any form of mathematical reasoning could be mechanized. The Church-Turing thesis implied that a mechanical device, shuffling symbols as simple as 0 and 1, could imitate any conceivable process of mathematical deduction. The key insight was the Turing machine
—a simple theoretical construct that captured the essence of abstract symbol manipulation. This invention would inspire a handful of scientists to begin discussing the possibility of thinking machines.
Calculating machines were built in antiquity and improved throughout history by many mathematicians, including (once again) philosopher Gottfried Leibniz. In the early 19th century, Charles Babbage
designed a programmable computer (the Analytical Engine
), although it was never built. Ada Lovelace
speculated that the machine "might compose elaborate and scientific pieces of music of any degree of complexity or extent". (She is often credited as the first programmer because of a set of notes she wrote that completely detail a method for calculating Bernoulli numbers with the Engine.)
The first modern computers were the massive code breaking machines of the Second World War (such as Z3, ENIAC
and Colossus
). The latter two of these machines were based on the theoretical foundation laid by Alan Turing
and developed by John Von Neumann
.
A note on the sections in this article.
In the 1940s and 50s, a handful of scientists from a variety of fields (mathematics, psychology, engineering, economics and political science) began to discuss the possibility of creating an artificial brain. The field of artificial intelligence
research was founded as an academic discipline in 1956.
had shown that the brain was an electrical network of neuron
s that fired in all-or-nothing pulses. Norbert Wiener
's cybernetics described control and stability in electrical networks. Claude Shannon's information theory
described digital signals (i.e., all-or-nothing signals). Alan Turing
's theory of computation
showed any form computation could be described digitally. The close relationship between these ideas suggested that it might be possible to construct an electronic brain.
Examples of work in this vein includes robots such as W. Grey Walter's turtles
and the Johns Hopkins Beast
. These machines did not use computers, digital electronics or symbolic reasoning; they were controlled entirely by analog circuitry.
Walter Pitts
and Warren McCulloch
analyzed networks of idealized artificial neurons and showed how they might perform simple logical functions. They were the first to describe what later researchers would call a neural network
. One of the students inspired by Pitts
and McCulloch
was a young Marvin Minsky
, then a 24 year old graduate student. In 1951 (with Dean Edmonds) he built the first neural net machine, the SNARC
.
Minsky
was to become one of the most important leaders and innovators in AI for the next 50 years.
, Christopher Strachey
wrote a checkers program and Dietrich Prinz wrote one for chess. Arthur Samuel
's checkers program, developed in the middle 50s and early 60s, eventually achieved sufficient skill to challenge a respectable amateur. Game AI would continue to be used as a measure of progress in AI throughout its history.
published a landmark paper
in which he speculated about the possibility of creating machines with true intelligence.
He noted that "intelligence" is difficult to define and devised his famous Turing Test
. If a machine could carry on a conversation (over a teleprinter
) that was indistinguishable from a conversation with a human being, then the machine could be called "intelligent." This simplified version of the problem allowed Turing to argue convincingly that a "thinking machine" was at least plausible and the paper answered all the most common objections to the proposition. The Turing Test
was the first serious proposal in the philosophy of artificial intelligence
.
In 1955, Allen Newell
and (future Nobel Laureate) Herbert Simon
created the "Logic Theorist
" (with help from J. C. Shaw
). The program would eventually prove 38 of the first 52 theorems in Russell
and Whitehead's
Principia Mathematica
, and find new and more elegant proofs for some.
Simon said that they had "solved the venerable mind/body problem, explaining how a system composed of matter can have the properties of mind."
(This was an early statement of the philosophical position John Searle
would later call "Strong AI": that machines can contain minds just as human bodies do.)
of 1956
was organized by Marvin Minsky
, John McCarthy
and two senior scientists: Claude Shannon and Nathan Rochester
of IBM
. The proposal for the conference included this assertion: "every aspect of learning or any other feature of intelligence can be so precisely described that a machine can be made to simulate it".
The participants included Ray Solomonoff
, Oliver Selfridge
, Trenchard More
, Arthur Samuel
, Allen Newell
and Herbert Simon
, all of whom would create important programs during the first decades of AI research.
At the conference Newell and Simon debuted the "Logic Theorist
" and McCarthy persuaded the attendees to accept "Artificial Intelligence" as the name of the field.
The 1956 Dartmouth conference was the moment that AI gained its name, its mission, its first success and its major players, and is widely considered the birth of AI.
. To achieve some goal (like winning a game or proving a theorem), they proceeded step by step towards it (by making a move or a deduction) as if searching through a maze, backtracking
whenever they reached a dead end. This paradigm was called "reasoning as search".
The principal difficulty was that, for many problems, the number of possible paths through the "maze" was simply astronomical (a situation known as a "combinatorial explosion
"). Researchers would reduce the search space by using heuristics or "rules of thumb" that would eliminate those paths that were unlikely to lead to a solution.
Newell
and Simon
tried to capture a general version of this algorithm in a program called the "General Problem Solver
". Other "searching" programs were able to accomplish impressive tasks like solving problems in geometry and algebra, such as Herbert Gelernter
's Geometry Theorem Prover (1958) and SAINT, written by Minsky's
student James Slagle (1961). Other programs searched through goals and subgoals to plan actions, like the STRIPS
system developed at Stanford to control the behavior of their robot Shakey
.
like English. An early success was Daniel Bobrow's program STUDENT
, which could solve high school algebra word problems.
A semantic net represents concepts (e.g. "house","door") as nodes and relations among concepts (e.g. "has-a") as links between the nodes. The first AI program to use a semantic net was written by Ross Quillian and the most successful (and controversial) version was Roger Schank
's Conceptual dependency theory
.
Joseph Weizenbaum
's ELIZA
could carry out conversations that were so realistic that users occasionally were fooled into thinking they were communicating with a human being and not a program. But in fact, ELIZA had no idea what she was talking about. She simply gave a canned response
or repeated back what was said to her, rephrasing her response with a few grammar rules. ELIZA was the first chatterbot
.
and Seymour Papert
of the MIT AI Laboratory proposed that AI research should focus on artificially simple situations known as micro-worlds. They pointed out that in successful sciences like physics, basic principles were often best understood using simplified models like frictionless planes or perfectly rigid bodies. Much of the research focused on a "blocks world
," which consists of colored blocks of various shapes and sizes arrayed on a flat surface.
This paradigm led to innovative work in machine vision
by Gerald Sussman (who led the team), Adolfo Guzman, David Waltz
(who invented "constraint propagation"), and especially Patrick Winston
. At the same time, Minsky
and Papert
built a robot arm that could stack blocks, bringing the blocks world to life. The crowning achievement of the micro-world program was Terry Winograd
's SHRDLU
. It could communicate in ordinary English sentences, plan operations and execute them.
and McCarthy
five years earlier. ARPA continued to provide three million dollars a year until the 70s.
ARPA made similar grants to Newell
and Simon's
program at CMU
and to the Stanford AI Project (founded by John McCarthy
in 1963). Another important AI laboratory was established at Edinburgh University by Donald Michie
in 1965.
These four institutions would continue to be the main centers of AI research (and funding) in academia for many years.
The money was proffered with few strings attached: J. C. R. Licklider
, then the director of ARPA, believed that his organization should "fund people, not projects!" and allowed researchers to pursue whatever directions might interest them. This created a freewheeling atmosphere at MIT that gave birth to the hacker culture, but this "hands off" approach would not last.
(or neural nets) was shut down almost completely for 10 years by Marvin Minsky
's devastating criticism of perceptrons.
Despite the difficulties with public perception of AI in the late 70s, new ideas were explored in logic programming
, commonsense reasoning
and many other areas.
) became frustrated with the lack of progress and eventually cut off almost all funding for undirected research into AI. The pattern began as early as 1966 when the ALPAC
report appeared criticizing machine translation efforts. After spending 20 million dollars, the NRC
ended all support.
In 1973, the Lighthill report
on the state of AI research in England criticized the utter failure of AI to achieve its "grandiose objectives" and led to the dismantling of AI research in that country.
(The report specifically mentioned the combinatorial explosion
problem as a reason for AI's failings.)
DARPA was deeply disappointed with researchers working on the Speech Understanding Research
program at CMU
and canceled an annual grant of three million dollars.
By 1974, funding for AI projects was hard to find.
Hans Moravec
blamed the crisis on the unrealistic predictions of his colleagues. "Many researchers were caught up in a web of increasing exaggeration."
However, there was another issue: since the passage of the Mansfield Amendment in 1969, DARPA had been under increasing pressure to fund "mission-oriented direct research, rather than basic undirected research". Funding for the creative, freewheeling exploration that had gone on in the 60s would not come from DARPA. Instead, the money was directed at specific projects with clear objectives, such as autonomous tanks and battle management systems.
, who argued that Gödel's
incompleteness theorem showed that a formal system
(such as a computer program) could never see the truth of certain statements, while a human being could. Hubert Dreyfus
ridiculed the broken promises of the 60s and critiqued the assumptions of AI, arguing that human reasoning actually involved very little "symbol processing" and a great deal of embodied
, instinct
ive, unconscious "know how". John Searle
's Chinese Room
argument, presented in 1980, attempted to show that a program could not be said to "understand" the symbols that it uses (a quality called "intentionality
"). If the symbols have no meaning for the machine, Searle argued, then the machine can not be described as "thinking".
These critiques were not taken seriously by AI researchers, often because they seemed so far off the point. Problems like intractability and commonsense knowledge
seemed much more immediate and serious. It was unclear what difference "know how" or "intentionality
" made to an actual computer program
. Minsky
said of Dreyfus and Searle "they misunderstand, and should be ignored." Dreyfus, who taught at MIT, was given a cold shoulder: he later said that AI researchers "dared not be seen having lunch with me." Joseph Weizenbaum
, the author of ELIZA
, felt his colleagues' treatment of Dreyfus
was unprofessional and childish. Although he was an outspoken critic of Dreyfus' positions, he "deliberately made it plain that theirs was not the way to treat a human being."
Weizenbaum began to have serious ethical doubts about AI when Kenneth Colby
wrote DOCTOR
, a chatterbot
therapist. Weizenbaum was disturbed that Colby saw his mindless program as a serious therapeutic tool. A feud began, and the situation was not helped when Colby did not credit Weizenbaum for his contribution to the program. In 1976, Weizenbaum
published Computer Power and Human Reason
which argued that the misuse of artificial intelligence has the potential to devalue human life.
was a form of neural network
introduced in 1958 by Frank Rosenblatt
, who had been a schoolmate of Marvin Minsky
at the Bronx High School of Science
. Like most AI researchers, he was optimistic about their power, predicting that "perceptron may eventually be able to learn, make decisions, and translate languages." An active research program into the paradigm was carried out throughout the 60s but came to a sudden halt with the publication of Minsky
and Papert's
1969 book Perceptrons
. It suggested that there were severe limitations to what perceptrons could do and that Frank Rosenblatt
's predictions had been grossly exaggerated. The effect of the book was devastating: virtually no research at all was done in connectionism
for 10 years. Eventually, a new generation of researchers would revive the field and thereafter it would become a vital and useful part of artificial intelligence. Rosenblatt
would not live to see this, as he died in a boating accident shortly after the book was published.
in his Advice Taker
proposal.
In 1963, J. Alan Robinson
had discovered a simple method to implement deduction on computers, the resolution
and unification algorithm. However, straightforward implementations, like those attempted by McCarthy and his students in the late 60s, were especially intractable: the programs required astronomical numbers of steps to prove simple theorems. A more fruitful approach to logic was developed in the 70s by Robert Kowalski
at the University of Edinburgh
, and soon this led to the collaboration with French researchers Alain Colmerauer
and Phillipe Roussel who created the successful logic programming language Prolog
.
Prolog uses a subset of logic (Horn clause
s, closely related to "rules" and "production rules
") that permit tractable computation. Rules would continue to be influential, providing a foundation for Edward Feigenbaum
's expert systems and the continuing work by Allen Newell
and Herbert Simon
that would lead to Soar
and their unified theories of cognition
.
Critics of the logical approach noted, as Dreyfus
had, that human beings rarely used logic when they solved problems. Experiments by psychologists like Peter Wason
, Eleanor Rosch
, Amos Tversky
, Daniel Kahneman
and others provided proof.
McCarthy responded that what people do is irrelevant. He argued that what is really needed are machines that can solve problems—not machines that think as people do.
approach were his colleagues across the country at MIT. Marvin Minsky
, Seymour Papert
and Roger Schank
were trying to solve problems like "story understanding" and "object recognition" that required a machine to think like a person. In order to use ordinary concepts like "chair" or "restaurant" they had to make all the same illogical assumptions that people normally made. Unfortunately, imprecise concepts like these are hard to represent in logic. Gerald Sussman observed that "using precise language to describe essentially imprecise concepts doesn't make them any more precise." Schank
described their "anti-logic" approaches as "scruffy
", as opposed to the "neat
" paradigms used by McCarthy
, Kowalski
, Feigenbaum
, Newell
and Simon
.
In 1975, in a seminal paper, Minsky
noted that many of his fellow "scruffy" researchers were using the same kind of tool: a framework that captures all our common sense assumptions about something. For example, if we use the concept of a bird, there is a constellation of facts that immediately come to mind: we might assume that it flies, eats worms and so on. We know these facts are not always true and that deductions using these facts will not be "logical," but these structured sets of assumptions are part of the context of everything we say and think. He called these structures "frames
". Schank
used a version of frames he called "scripts
" to successfully answer questions about short stories in English. Many years later object-oriented programming
would adopt the essential idea of "inheritance
" from AI research on frames.
s" was adopted by corporations around the world and knowledge
became the focus of mainstream AI research. In those same years, the Japanese government aggressively funded AI with its fifth generation computer
project. Another encouraging event in the early 1980s was the revival of connectionism
in the work of John Hopfield and David Rumelhart
. Once again, AI had achieved success.
is a program that answers questions or solves problems about a specific domain of knowledge, using logical rules
that are derived from the knowledge of experts. The earliest examples were developed by Edward Feigenbaum
and his students. Dendral
, begun in 1965, identified compounds from spectrometer readings. MYCIN
, developed in 1972, diagnosed infectious blood diseases. They demonstrated the feasibility of the approach.
Expert systems restricted themselves to a small domain of specific knowledge (thus avoiding the commonsense knowledge problem) and their simple design made it relatively easy for programs to be built and then modified once they were in place. All in all, the programs proved to be useful: something that AI had not been able to achieve up to this point.
In 1980, an expert system called XCON
was completed at CMU
for the Digital Equipment Corporation
. It was an enormous success: it was saving the company 40 million dollars annually by 1986. Corporations around the world began to develop and deploy expert systems and by 1985 they were spending over a billion dollars on AI, most of it to in-house AI departments. An industry grew up to support them, including hardware companies like Symbolics
and Lisp Machines
and software companies such as IntelliCorp
and Aion
.
. "[T]he great lesson from the 1970s was that intelligent behavior depended very much on dealing with knowledge, sometimes quite detailed knowledge, of a domain where a given task lay". Knowledge based systems and knowledge engineering
became a major focus of AI research in the 1980s.
The 1980s also saw the birth of Cyc
, the first attempt to attack the commonsense knowledge problem
directly, by creating a massive database that would contain all the mundane facts that the average person knows. Douglas Lenat
, who started and led the project, argued that there is no shortcut ― the only way for machines to know the meaning of human concepts is to teach them, one concept at a time, by hand. The project was not expected to be completed for many decades.
set aside $850 million dollars for the Fifth generation computer
project. Their objectives were to write programs and build machines that could carry on conversations, translate languages, interpret pictures, and reason like human beings. Much to the chagrin of scruffies
, they chose Prolog
as the primary computer language for the project.
Other countries responded with new programs of their own. The UK began the ₤350 million Alvey
project. A consortium of American companies formed the Microelectronics and Computer Technology Corporation
(or "MCC") to fund large scale projects in AI and information technology. DARPA responded as well, founding the Strategic Computing Initiative
and tripling its investment in AI between 1984 and 1988.
") could learn and process information in a completely new way. Around the same time, David Rumelhart
popularized a new method for training neural networks called "backpropagation
" (discovered years earlier by Paul Werbos
). These two discoveries revived the field of connectionism
which had been largely abandoned since 1970.
The new field was unified and inspired by the appearance of Parallel Distributed Processing in 1986—a two volume collection of papers edited by Rumelhart
and psychologist James McClelland. Neural networks would become commercially successful in the 1990s, when they began to be used as the engines driving programs like optical character recognition
and speech recognition
.
. The collapse was in the perception of AI by government agencies and investors — the field continued to make advances despite the criticism. Rodney Brooks
and Hans Moravec
, researchers from the related field of robotics
, argued for an entirely new approach to artificial intelligence.
" was coined by researchers who had survived the funding cuts of 1974 when they became concerned that enthusiasm for expert systems had spiraled out of control and that disappointment would certainly follow. Their fears were well founded: in the late 80s and early 90s, AI suffered a series of financial setbacks.
The first indication of a change in weather was the sudden collapse of the market for specialized AI hardware in 1987. Desktop computers from Apple
and IBM
had been steadily gaining speed and power and in 1987 they became more powerful than the more expensive Lisp machines
made by Symbolics
and others. There was no longer a good reason to buy them. An entire industry worth half a billion dollars was demolished overnight.
Eventually the earliest successful expert systems, such as XCON
, proved too expensive to maintain. They were difficult to update, they could not learn, they were "brittle" (i.e., they could make grotesque mistakes when given unusual inputs), and they fell prey to problems (such as the qualification problem
) that had been identified years earlier. Expert systems proved useful, but only in a few special contexts.
In the late 80s, the Strategic Computing Initiative
cut funding to AI "deeply and brutally." New leadership at DARPA had decided that AI was not "the next wave" and directed funds towards projects that seemed more likely to produce immediate results.
By 1991, the impressive list of goals penned in 1981 for Japan's Fifth Generation Project
had not been met. Indeed, some of them, like "carry on a casual conversation" had not been met by 2010. As with other AI projects, expectations had run much higher than what was actually possible.
and that abstract reasoning was actually the least interesting or important human skill (see Moravec's paradox
). They advocated building intelligence "from the bottom up."
The approach revived ideas from cybernetics and control theory
that had been unpopular since the sixties. Another precursor was David Marr, who had come to MIT in the late 70s from a successful background in theoretical neuroscience to lead the group studying vision
. He rejected all symbolic approaches (both McCarthy's
logic and Minsky
's frames), arguing that AI needed to understand the physical machinery of vision from the bottom up before any symbolic processing took place. (Marr's work would be cut short by leukemia in 1980.)
In a 1990 paper Elephants Don't Play Chess, robotics researcher Rodney Brooks
took direct aim at the physical symbol system hypothesis
, arguing that symbols are not always necessary since "the world is its own best model. It is always exactly up to date. It always has every detail there is to be known. The trick is to sense it appropriately and often enough." In the 80s and 90s, many cognitive scientists
also rejected the symbol processing model of the mind and argued that the body was essential for reasoning, a theory called the embodied mind thesis.
. In 2005, a Stanford robot won the DARPA Grand Challenge
by driving autonomously for 131 miles along an unrehearsed desert trail.. Two years later, a team from CMU won the DARPA Urban Challenge by autonomously navigating 55 miles in an Urban environment while adhering to traffic hazards and all traffic laws. In February 2011, in a Jeopardy!
quiz show
exhibition match, IBM
's question answering system, Watson
, defeated the two greatest Jeopardy! champions, Brad Rutter
and Ken Jennings
, by a significant margin.
These successes were not due to some revolutionary new paradigm, but mostly on the tedious application of engineering skill and on the tremendous power of computers today. In fact, Deep Blue's computer was 10 million times faster than the Ferranti Mark 1 that Christopher Strachey
taught to play chess in 1951. This dramatic increase is measured by Moore's law
, which predicts that the speed and memory capacity of computers doubles every two years. The fundamental problem of "raw computer power" was slowly being overcome.
s" became widely accepted during the 90s. Although earlier researchers had proposed modular "divide and conquer" approaches to AI, the intelligent agent
did not reach its modern form until Judea Pearl
, Allen Newell
and others brought concepts from decision theory
and economics
into the study of AI. When the economist's
definition of a rational agent
was married to computer science
's definition of an object
or module, the intelligent agent
paradigm was complete.
An intelligent agent
is a system that perceives its environment and takes actions which maximize its chances of success. The simplest intelligent agents are programs that solve specific problems. The most complicated intelligent agents known are rational, thinking human beings. The intelligent agent paradigm
defines AI research as "the study of intelligent agents". This is a generalization of some earlier definitions of AI: it goes beyond studying human intelligence; it studies all kinds of intelligence.
The paradigm gave researchers license to study isolated problems and find solutions that were both verifiable and useful. It provided a common language to describe problems and share their solutions with each other, and with other fields that also used concepts of abstract agents, like economics
and control theory
. It was hoped that a complete agent architecture
(like Newell's
SOAR
) would one day allow researchers to build more versatile and intelligent systems out of interacting intelligent agents.
, economics
or operations research
. The shared mathematical language allowed both a higher level of collaboration with more established and successful fields and the achievement of results which were measurable and provable; AI had become a more rigorous "scientific" discipline. describe this as nothing less than a "revolution" and "the victory of the neats".
Judea Pearl
's highly influential 1988 book brought probability
and decision theory
into AI. Among the many new tools in use were Bayesian networks, hidden Markov models, information theory
, stochastic modeling and classical optimization
. Precise mathematical descriptions were also developed for "computational intelligence
" paradigms like neural networks
and evolutionary algorithm
s.
and their solutions proved to be useful throughout the technology industry, such as
data mining
,
industrial robotics,
logistics
,
speech recognition
,
banking software,
medical diagnosis
and Google
's search engine.
The field of AI receives little or no credit for these successes. Many of AI's greatest innovations have been reduced to the status of just another item in the tool chest of computer science. Nick Bostrom
explains "A lot of cutting edge AI has filtered into general applications, often without being called AI because once something becomes useful enough and common enough it's not labeled AI anymore."
Many researchers in AI today deliberately call their work by other names, such as informatics
, knowledge-based systems
, cognitive systems or computational intelligence
. In part, this may be because they considered their field to be fundamentally different from AI, but also the new names help to procure funding. In the commercial world at least, the failed promises of the AI Winter
continue to haunt AI research, as the New York Times reported in 2005: "Computer scientists and software engineers avoided the term artificial intelligence for fear of being viewed as wild-eyed dreamers."
and Stanley Kubrick
had imagined that by the year 2001, a machine would exist with an intelligence that matched or exceeded the capability of human beings. The character they created, HAL 9000
, was based on hard science: many leading AI researchers also believed that such a machine would exist by the year 2001.
Marvin Minsky
asks "So the question is why didn't we get HAL in 2001?" Minsky believes that the answer is that the central problems, like commonsense reasoning
, were being neglected, while most researchers pursued things like commercial applications of neural nets or genetic algorithms. John McCarthy
, on the other hand, still blames the qualification problem
. For Ray Kurzweil, the issue is computer power and, using Moore's Law
, he predicts that machines with human-level intelligence will appear by 2029. Jeff Hawkins
argues that neural net research ignores the essential properties of the human cortex
, preferring simple models that have been successful at solving simple problems. There are many other explanations and for each there is a corresponding research program underway.
Ancient history
Ancient history is the study of the written past from the beginning of recorded human history to the Early Middle Ages. The span of recorded history is roughly 5,000 years, with Cuneiform script, the oldest discovered form of coherent writing, from the protoliterate period around the 30th century BC...
, with myths, stories and rumors of artificial beings endowed with intelligence or consciousness by master craftsmen; as Pamela McCorduck
Pamela McCorduck
Pamela McCorduck is the author of a number of books concerning the history and philosophical significance of artificial intelligence, the future of engineering and the role of women and technology. She is also the author of three novels. She is a contributor to Omni, New York Times, Daedalus, the...
writes, AI began with "an ancient wish to forge the gods."
The seeds of modern AI were planted by classical philosophers who attempted to describe the process of human thinking as the mechanical manipulation of symbols. This work culminated in the invention of the programmable digital computer
Computer
A computer is a programmable machine designed to sequentially and automatically carry out a sequence of arithmetic or logical operations. The particular sequence of operations can be changed readily, allowing the computer to solve more than one kind of problem...
in the 1940s, a machine based on the abstract essence of mathematical reasoning. This device and the ideas behind it inspired a handful of scientists to begin seriously discussing the possibility of building an electronic brain.
The field of artificial intelligence
Artificial intelligence
Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...
research was founded at a conference on the campus of Dartmouth College
Dartmouth College
Dartmouth College is a private, Ivy League university in Hanover, New Hampshire, United States. The institution comprises a liberal arts college, Dartmouth Medical School, Thayer School of Engineering, and the Tuck School of Business, as well as 19 graduate programs in the arts and sciences...
in the summer of 1956. Those who attended would become the leaders of AI research for decades. Many of them predicted that a machine as intelligent as a human being would exist in no more than a generation and they were given millions of dollars to make this vision come true. Eventually it became obvious that they had grossly underestimated the difficulty of the project. In 1973, in response to the criticism of Sir James Lighthill and ongoing pressure from congress, the U.S. and British Governments stopped funding undirected research into artificial intelligence. Seven years later, a visionary initiative by the Japanese Government inspired governments and industry to provide AI with billions of dollars, but by the late 80s the investors became disillusioned and withdrew funding again. This cycle of boom and bust, of "AI winter
AI winter
In the history of artificial intelligence, an AI winter is a period of reduced funding and interest in artificial intelligence research. The process of hype, disappointment and funding cuts are common in many emerging technologies , but the problem has been particularly acute for AI...
s" and summers, continues to haunt the field. Undaunted, there are those who make extraordinary predictions even now.
Progress in AI has continued, despite the rise and fall of its reputation in the eyes of government bureaucrats and venture capitalists. Problems that had begun to seem impossible in 1970 have been solved and the solutions are now used in successful commercial products. However, no machine has been built with a human level of intelligence
Strong AI
Strong AI is artificial intelligence that matches or exceeds human intelligence — the intelligence of a machine that can successfully perform any intellectual task that a human being can. It is a primary goal of artificial intelligence research and an important topic for science fiction writers and...
, contrary to the optimistic predictions of the first generation of AI researchers. "We can only see a short distance ahead," admitted Alan Turing
Alan Turing
Alan Mathison Turing, OBE, FRS , was an English mathematician, logician, cryptanalyst, and computer scientist. He was highly influential in the development of computer science, providing a formalisation of the concepts of "algorithm" and "computation" with the Turing machine, which played a...
, in a famous 1950 paper that catalyzed the modern search for machines that think. "But," he added, "we can see much that must be done."
Precursors
writes "artificial intelligenceArtificial intelligence
Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...
in one form or another is an idea that has pervaded Western intellectual history, a dream in urgent need of being realized," expressed in humanity's myths, legends, stories, speculation and clockwork automaton
Automaton
An automaton is a self-operating machine. The word is sometimes used to describe a robot, more specifically an autonomous robot. An alternative spelling, now obsolete, is automation.-Etymology:...
s.
AI in myth, fiction and speculation
Mechanical men and artificial beings appear in Greek myths, such as the golden robots of HephaestusHephaestus
Hephaestus was a Greek god whose Roman equivalent was Vulcan. He is the son of Zeus and Hera, the King and Queen of the Gods - or else, according to some accounts, of Hera alone. He was the god of technology, blacksmiths, craftsmen, artisans, sculptors, metals, metallurgy, fire and volcanoes...
and Pygmalion's
Pygmalion (mythology)
Pygmalion is a legendary figure of Cyprus. Though Pygmalion is the Greek version of the Phoenician royal name Pumayyaton, he is most familiar from Ovid's Metamorphoses, X, in which Pygmalion was a sculptor who fell in love with a statue he had carved.-In Ovid:In Ovid's narrative, Pygmalion was a...
Galatea
Galatea (mythology)
-Name "Galatea":Though the name "Galatea" has become so firmly associated with Pygmalion's statue as to seem antique, its use in connection with Pygmalion originated with a post-classical writer. No extant ancient text mentions the statue's name...
.
In the Middle Ages, there were rumors of secret mystical or alchemical
Alchemy
Alchemy is an influential philosophical tradition whose early practitioners’ claims to profound powers were known from antiquity. The defining objectives of alchemy are varied; these include the creation of the fabled philosopher's stone possessing powers including the capability of turning base...
means of placing mind into matter, such as Jābir ibn Hayyān's Takwin
Takwin
Takwin was a goal of certain Ismaili Muslim alchemists, notably Jabir ibn Hayyan. In the alchemical context, takwin refers to the creation of synthetic life in the laboratory, up to and including human life...
, Paracelsus
Paracelsus
Paracelsus was a German-Swiss Renaissance physician, botanist, alchemist, astrologer, and general occultist....
' homunculus
Homunculus
Homunculus is a term used, generally, in various fields of study to refer to any representation of a human being. Historically, it referred specifically to the concept of a miniature though fully formed human body, for example, in the studies of alchemy and preformationism...
and Rabbi Judah Loew's Golem
Golem
In Jewish folklore, a golem is an animated anthropomorphic being, created entirely from inanimate matter. The word was used to mean an amorphous, unformed material in Psalms and medieval writing....
.
By the 19th century, ideas about artificial men and thinking machines were developed in fiction, as in Mary Shelley
Mary Shelley
Mary Shelley was a British novelist, short story writer, dramatist, essayist, biographer, and travel writer, best known for her Gothic novel Frankenstein: or, The Modern Prometheus . She also edited and promoted the works of her husband, the Romantic poet and philosopher Percy Bysshe Shelley...
's Frankenstein
Frankenstein
Frankenstein; or, The Modern Prometheus is a novel about a failed experiment that produced a monster, written by Mary Shelley, with inserts of poems by Percy Bysshe Shelley. Shelley started writing the story when she was eighteen, and the novel was published when she was twenty-one. The first...
or Karel Čapek
Karel Capek
Karel Čapek was Czech writer of the 20th century.-Biography:Born in 1890 in the Bohemian mountain village of Malé Svatoňovice to an overbearing, emotional mother and a distant yet adored father, Čapek was the youngest of three siblings...
's R.U.R. (Rossum's Universal Robots)
R.U.R. (Rossum's Universal Robots)
R.U.R. is a 1920 science fiction play in the Czech language by Karel Čapek. R.U.R. stands for Rossum's Universal Robots, an English phrase used as the subtitle in the Czech original. It premiered in 1921 and introduced the word "robot" to the English language and to science fiction as a whole.The...
,
and speculation, such as Samuel Butler
Samuel Butler (novelist)
Samuel Butler was an iconoclastic Victorian author who published a variety of works. Two of his most famous pieces are the Utopian satire Erewhon and a semi-autobiographical novel published posthumously, The Way of All Flesh...
's "Darwin among the Machines
Darwin Among the Machines
"Darwin among the Machines" appeared as the heading of an article published in The Press newspaper on 13 June 1863 in Christchurch, New Zealand...
."
AI has continued to be an important element of science fiction
Science fiction
Science fiction is a genre of fiction dealing with imaginary but more or less plausible content such as future settings, futuristic science and technology, space travel, aliens, and paranormal abilities...
into the present.
Automatons
Automaton
An automaton is a self-operating machine. The word is sometimes used to describe a robot, more specifically an autonomous robot. An alternative spelling, now obsolete, is automation.-Etymology:...
s were built by craftsman from every civilization, including Yan Shi,
Hero of Alexandria
Hero of Alexandria
Hero of Alexandria was an ancient Greek mathematician and engineerEnc. Britannica 2007, "Heron of Alexandria" who was active in his native city of Alexandria, Roman Egypt...
,
Al-Jazari
Al-Jazari
Abū al-'Iz Ibn Ismā'īl ibn al-Razāz al-Jazarī was a Muslim polymath: a scholar, inventor, mechanical engineer, craftsman, artist, mathematician and astronomer from Al-Jazira, Mesopotamia, who lived during the Islamic Golden Age...
and Wolfgang von Kempelen
Wolfgang von Kempelen
Johann Wolfgang Ritter von Kempelen de Pázmánd was a Hungarian author and inventor with Irish ancestors.-Life:...
.
The oldest known automaton
Automaton
An automaton is a self-operating machine. The word is sometimes used to describe a robot, more specifically an autonomous robot. An alternative spelling, now obsolete, is automation.-Etymology:...
s were the sacred statues
Cult image
In the practice of religion, a cult image is a human-made object that is venerated for the deity, spirit or daemon that it embodies or represents...
of ancient Egypt
Ancient Egypt
Ancient Egypt was an ancient civilization of Northeastern Africa, concentrated along the lower reaches of the Nile River in what is now the modern country of Egypt. Egyptian civilization coalesced around 3150 BC with the political unification of Upper and Lower Egypt under the first pharaoh...
and Greece
Ancient Greece
Ancient Greece is a civilization belonging to a period of Greek history that lasted from the Archaic period of the 8th to 6th centuries BC to the end of antiquity. Immediately following this period was the beginning of the Early Middle Ages and the Byzantine era. Included in Ancient Greece is the...
. The faithful believed that craftsman had imbued these figures with very real minds, capable of wisdom and emotion—Hermes Trismegistus
Hermes Trismegistus
Hermes Trismegistus is the eponymous author of the Hermetic Corpus, a sacred text belonging to the genre of divine revelation.-Origin and identity:...
wrote that "by discovering the true nature of the gods, man has been able to reproduce it."
Formal reasoning
Artificial intelligence is based on the assumption that the process of human thought can be mechanized. The study of mechanical—or "formal"—reasoning has a long history. ChineseChinese philosophy
Chinese philosophy is philosophy written in the Chinese tradition of thought. The majority of traditional Chinese philosophy originates in the Spring and Autumn and Warring States era, during a period known as the "Hundred Schools of Thought", which was characterized by significant intellectual and...
, Indian
Indian philosophy
India has a rich and diverse philosophical tradition dating back to ancient times. According to Radhakrishnan, the earlier Upanisads constitute "...the earliest philosophical compositions of the world."...
and Greek
Greek philosophy
Ancient Greek philosophy arose in the 6th century BCE and continued through the Hellenistic period, at which point Ancient Greece was incorporated in the Roman Empire...
philosophers all developed structured methods of formal deduction in the first millennium BCE. Their ideas were developed over the centuries by philosophers such as Aristotle
Aristotle
Aristotle was a Greek philosopher and polymath, a student of Plato and teacher of Alexander the Great. His writings cover many subjects, including physics, metaphysics, poetry, theater, music, logic, rhetoric, linguistics, politics, government, ethics, biology, and zoology...
(who gave a formal analysis of the syllogism
Syllogism
A syllogism is a kind of logical argument in which one proposition is inferred from two or more others of a certain form...
), Euclid
Euclid
Euclid , fl. 300 BC, also known as Euclid of Alexandria, was a Greek mathematician, often referred to as the "Father of Geometry". He was active in Alexandria during the reign of Ptolemy I...
(whose Elements
Euclid's Elements
Euclid's Elements is a mathematical and geometric treatise consisting of 13 books written by the Greek mathematician Euclid in Alexandria c. 300 BC. It is a collection of definitions, postulates , propositions , and mathematical proofs of the propositions...
was a model of formal reasoning), (who developed algebra
Algebra
Algebra is the branch of mathematics concerning the study of the rules of operations and relations, and the constructions and concepts arising from them, including terms, polynomials, equations and algebraic structures...
and gave his name to "algorithm
Algorithm
In mathematics and computer science, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function. Algorithms are used for calculation, data processing, and automated reasoning...
") and European scholastic
Scholasticism
Scholasticism is a method of critical thought which dominated teaching by the academics of medieval universities in Europe from about 1100–1500, and a program of employing that method in articulating and defending orthodoxy in an increasingly pluralistic context...
philosophers such as William of Ockham
William of Ockham
William of Ockham was an English Franciscan friar and scholastic philosopher, who is believed to have been born in Ockham, a small village in Surrey. He is considered to be one of the major figures of medieval thought and was at the centre of the major intellectual and political controversies of...
and Duns Scotus
Duns Scotus
Blessed John Duns Scotus, O.F.M. was one of the more important theologians and philosophers of the High Middle Ages. He was nicknamed Doctor Subtilis for his penetrating and subtle manner of thought....
.
Majorcan philosopher Ramon Llull
Ramon Llull
Ramon Llull was a Majorcan writer and philosopher, logician and tertiary Franciscan. He wrote the first major work of Catalan literature. Recently-surfaced manuscripts show him to have anticipated by several centuries prominent work on elections theory...
(1232–1315) developed several logical machines devoted to the production of knowledge by logical means; Llull described his machines as mechanical entities that could combine basic and undeniable truths by simple logical operations, produced by the machine by mechanical meanings, in such ways as to produce all the possible knowledge. Llull's work had a great influence on Gottfried Leibniz
Gottfried Leibniz
Gottfried Wilhelm Leibniz was a German philosopher and mathematician. He wrote in different languages, primarily in Latin , French and German ....
, who redeveloped his ideas.
Gottfried Leibniz
Gottfried Wilhelm Leibniz was a German philosopher and mathematician. He wrote in different languages, primarily in Latin , French and German ....
, Thomas Hobbes
Thomas Hobbes
Thomas Hobbes of Malmesbury , in some older texts Thomas Hobbs of Malmsbury, was an English philosopher, best known today for his work on political philosophy...
and René Descartes
René Descartes
René Descartes ; was a French philosopher and writer who spent most of his adult life in the Dutch Republic. He has been dubbed the 'Father of Modern Philosophy', and much subsequent Western philosophy is a response to his writings, which are studied closely to this day...
explored the possibility that all rational thought could be made as systematic as algebra or geometry.
Hobbes famously wrote in Leviathan
Leviathan (book)
Leviathan or The Matter, Forme and Power of a Common Wealth Ecclesiasticall and Civil — commonly called simply Leviathan — is a book written by Thomas Hobbes and published in 1651. Its name derives from the biblical Leviathan...
: "reason is nothing but reckoning".
Leibniz
Gottfried Leibniz
Gottfried Wilhelm Leibniz was a German philosopher and mathematician. He wrote in different languages, primarily in Latin , French and German ....
envisioned a universal language of reasoning (his characteristica universalis
Characteristica universalis
The Latin term characteristica universalis, commonly interpreted as universal characteristic, or universal character in English, is a universal and formal language imagined by the German philosopher Gottfried Leibniz able to express mathematical, scientific, and metaphysical concepts...
) which would reduce argumentation to calculation, so that "there would be no more need of disputation between two philosophers than between two accountants. For it would suffice to take their pencils in hand, down to their slates, and to say each other (with a friend as witness, if they liked): Let us calculate."
These philosophers had begun to articulate the physical symbol system
Physical symbol system
A physical symbol system takes physical patterns , combining them into structures and manipulating them to produce new expressions....
hypothesis that would become the guiding faith of AI research.
In the 20th century, the study of mathematical logic
Mathematical logic
Mathematical logic is a subfield of mathematics with close connections to foundations of mathematics, theoretical computer science and philosophical logic. The field includes both the mathematical study of logic and the applications of formal logic to other areas of mathematics...
provided the essential breakthrough that made artificial intelligence seem plausible. The foundations had been set by such works as Boole
George Boole
George Boole was an English mathematician and philosopher.As the inventor of Boolean logic—the basis of modern digital computer logic—Boole is regarded in hindsight as a founder of the field of computer science. Boole said,...
's The Laws of Thought
The Laws of Thought
The Laws of Thought, more precisely, An Investigation of the Laws of Thought on Which are Founded the Mathematical Theories of Logic and Probabilities, is a very influential 19th century book on logic by George Boole, the second of his two monographs on algebraic logic...
and Frege's Begriffsschrift
Begriffsschrift
Begriffsschrift is a book on logic by Gottlob Frege, published in 1879, and the formal system set out in that book...
. Building on Frege's system, Russell
Bertrand Russell
Bertrand Arthur William Russell, 3rd Earl Russell, OM, FRS was a British philosopher, logician, mathematician, historian, and social critic. At various points in his life he considered himself a liberal, a socialist, and a pacifist, but he also admitted that he had never been any of these things...
and Whitehead
Alfred North Whitehead
Alfred North Whitehead, OM FRS was an English mathematician who became a philosopher. He wrote on algebra, logic, foundations of mathematics, philosophy of science, physics, metaphysics, and education...
presented a formal treatment of the foundations of mathematics in their masterpiece, the Principia Mathematica
Principia Mathematica
The Principia Mathematica is a three-volume work on the foundations of mathematics, written by Alfred North Whitehead and Bertrand Russell and published in 1910, 1912, and 1913...
in 1913. Inspired by Russell
Bertrand Russell
Bertrand Arthur William Russell, 3rd Earl Russell, OM, FRS was a British philosopher, logician, mathematician, historian, and social critic. At various points in his life he considered himself a liberal, a socialist, and a pacifist, but he also admitted that he had never been any of these things...
's success, David Hilbert
Hilbert's program
In mathematics, Hilbert's program, formulated by German mathematician David Hilbert, was a proposed solution to the foundational crisis of mathematics, when early attempts to clarify the foundations of mathematics were found to suffer from paradoxes and inconsistencies...
challenged mathematicians of the 1920s and 30s to answer this fundamental question: "can all of mathematical reasoning be formalized?"
His question was answered by Gödel
Kurt Gödel
Kurt Friedrich Gödel was an Austrian logician, mathematician and philosopher. Later in his life he emigrated to the United States to escape the effects of World War II. One of the most significant logicians of all time, Gödel made an immense impact upon scientific and philosophical thinking in the...
's incompleteness proof
Gödel's incompleteness theorems
Gödel's incompleteness theorems are two theorems of mathematical logic that establish inherent limitations of all but the most trivial axiomatic systems capable of doing arithmetic. The theorems, proven by Kurt Gödel in 1931, are important both in mathematical logic and in the philosophy of...
, Turing
Alan Turing
Alan Mathison Turing, OBE, FRS , was an English mathematician, logician, cryptanalyst, and computer scientist. He was highly influential in the development of computer science, providing a formalisation of the concepts of "algorithm" and "computation" with the Turing machine, which played a...
's machine
Turing machine
A Turing machine is a theoretical device that manipulates symbols on a strip of tape according to a table of rules. Despite its simplicity, a Turing machine can be adapted to simulate the logic of any computer algorithm, and is particularly useful in explaining the functions of a CPU inside a...
and Church
Alonzo Church
Alonzo Church was an American mathematician and logician who made major contributions to mathematical logic and the foundations of theoretical computer science. He is best known for the lambda calculus, Church–Turing thesis, Frege–Church ontology, and the Church–Rosser theorem.-Life:Alonzo Church...
's Lambda calculus
Lambda calculus
In mathematical logic and computer science, lambda calculus, also written as λ-calculus, is a formal system for function definition, function application and recursion. The portion of lambda calculus relevant to computation is now called the untyped lambda calculus...
.
Their answer was surprising in two ways. First, they proved that there were, in fact, limits to what mathematical logic could accomplish.
Turing machine
A Turing machine is a theoretical device that manipulates symbols on a strip of tape according to a table of rules. Despite its simplicity, a Turing machine can be adapted to simulate the logic of any computer algorithm, and is particularly useful in explaining the functions of a CPU inside a...
—a simple theoretical construct that captured the essence of abstract symbol manipulation. This invention would inspire a handful of scientists to begin discussing the possibility of thinking machines.
Computer science
Calculating machines were built in antiquity and improved throughout history by many mathematicians, including (once again) philosopher Gottfried Leibniz. In the early 19th century, Charles Babbage
Charles Babbage
Charles Babbage, FRS was an English mathematician, philosopher, inventor and mechanical engineer who originated the concept of a programmable computer...
designed a programmable computer (the Analytical Engine
Analytical engine
The Analytical Engine was a proposed mechanical general-purpose computer designed by English mathematician Charles Babbage. It was first described in 1837 as the successor to Babbage's difference engine, a design for a mechanical calculator...
), although it was never built. Ada Lovelace
Ada Lovelace
Augusta Ada King, Countess of Lovelace , born Augusta Ada Byron, was an English writer chiefly known for her work on Charles Babbage's early mechanical general-purpose computer, the analytical engine...
speculated that the machine "might compose elaborate and scientific pieces of music of any degree of complexity or extent". (She is often credited as the first programmer because of a set of notes she wrote that completely detail a method for calculating Bernoulli numbers with the Engine.)
The first modern computers were the massive code breaking machines of the Second World War (such as Z3, ENIAC
ENIAC
ENIAC was the first general-purpose electronic computer. It was a Turing-complete digital computer capable of being reprogrammed to solve a full range of computing problems....
and Colossus
Colossus computer
Not to be confused with the fictional computer of the same name in the movie Colossus: The Forbin Project.Colossus was the world's first electronic, digital, programmable computer. Colossus and its successors were used by British codebreakers to help read encrypted German messages during World War II...
). The latter two of these machines were based on the theoretical foundation laid by Alan Turing
Alan Turing
Alan Mathison Turing, OBE, FRS , was an English mathematician, logician, cryptanalyst, and computer scientist. He was highly influential in the development of computer science, providing a formalisation of the concepts of "algorithm" and "computation" with the Turing machine, which played a...
and developed by John Von Neumann
John von Neumann
John von Neumann was a Hungarian-American mathematician and polymath who made major contributions to a vast number of fields, including set theory, functional analysis, quantum mechanics, ergodic theory, geometry, fluid dynamics, economics and game theory, computer science, numerical analysis,...
.
The birth of artificial intelligence 1943−1956
In the 1940s and 50s, a handful of scientists from a variety of fields (mathematics, psychology, engineering, economics and political science) began to discuss the possibility of creating an artificial brain. The field of artificial intelligence
Artificial intelligence
Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...
research was founded as an academic discipline in 1956.
Cybernetics and early neural networks
The earliest research into thinking machines was inspired by a confluence of ideas that became prevalent in the late 30s, 40s and early 50s. Recent research in neurologyNeurology
Neurology is a medical specialty dealing with disorders of the nervous system. Specifically, it deals with the diagnosis and treatment of all categories of disease involving the central, peripheral, and autonomic nervous systems, including their coverings, blood vessels, and all effector tissue,...
had shown that the brain was an electrical network of neuron
Neuron
A neuron is an electrically excitable cell that processes and transmits information by electrical and chemical signaling. Chemical signaling occurs via synapses, specialized connections with other cells. Neurons connect to each other to form networks. Neurons are the core components of the nervous...
s that fired in all-or-nothing pulses. Norbert Wiener
Norbert Wiener
Norbert Wiener was an American mathematician.A famous child prodigy, Wiener later became an early researcher in stochastic and noise processes, contributing work relevant to electronic engineering, electronic communication, and control systems.Wiener is regarded as the originator of cybernetics, a...
's cybernetics described control and stability in electrical networks. Claude Shannon's information theory
Information theory
Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and...
described digital signals (i.e., all-or-nothing signals). Alan Turing
Alan Turing
Alan Mathison Turing, OBE, FRS , was an English mathematician, logician, cryptanalyst, and computer scientist. He was highly influential in the development of computer science, providing a formalisation of the concepts of "algorithm" and "computation" with the Turing machine, which played a...
's theory of computation
Theory of computation
In theoretical computer science, the theory of computation is the branch that deals with whether and how efficiently problems can be solved on a model of computation, using an algorithm...
showed any form computation could be described digitally. The close relationship between these ideas suggested that it might be possible to construct an electronic brain.
Examples of work in this vein includes robots such as W. Grey Walter's turtles
Turtle (robot)
Turtles are a class of educational robots designed originally in the late 1940s and used in computer science and mechanical engineering training. These devices are traditionally built low to the ground with a roughly hemispheric shell and a power train capable of a very small turning radius...
and the Johns Hopkins Beast
Johns Hopkins Beast
The Johns Hopkins Beast was an early robot built in the 1960s at the Johns Hopkins University Applied Physics Laboratory. The machine had a rudimentary intelligence and the ability to survive on its own. As it wandered through the white halls of the laboratory, it would seek black wall outlets....
. These machines did not use computers, digital electronics or symbolic reasoning; they were controlled entirely by analog circuitry.
Walter Pitts
Walter Pitts
Walter Harry Pitts, Jr. was a logician who worked in the field of cognitive psychology.He proposed landmark theoretical formulations of neural activity and emergent processes that influenced diverse fields such as cognitive sciences and psychology, philosophy, neurosciences, computer science,...
and Warren McCulloch
Warren Sturgis McCulloch
Warren Sturgis McCulloch was an American neurophysiologist and cybernetician, known for his work on the foundation for certain brain theories and his contribution to the cybernetics movement.- Biography :...
analyzed networks of idealized artificial neurons and showed how they might perform simple logical functions. They were the first to describe what later researchers would call a neural network
Neural network
The term neural network was traditionally used to refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks, which are composed of artificial neurons or nodes...
. One of the students inspired by Pitts
Walter Pitts
Walter Harry Pitts, Jr. was a logician who worked in the field of cognitive psychology.He proposed landmark theoretical formulations of neural activity and emergent processes that influenced diverse fields such as cognitive sciences and psychology, philosophy, neurosciences, computer science,...
and McCulloch
Warren Sturgis McCulloch
Warren Sturgis McCulloch was an American neurophysiologist and cybernetician, known for his work on the foundation for certain brain theories and his contribution to the cybernetics movement.- Biography :...
was a young Marvin Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
, then a 24 year old graduate student. In 1951 (with Dean Edmonds) he built the first neural net machine, the SNARC
SNARC
SNARC is a neural net machine designed by Marvin Lee Minsky. It is a randomly connected network of Hebb synapses....
.
Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
was to become one of the most important leaders and innovators in AI for the next 50 years.
Game AI
In 1951, using the Ferranti Mark 1 machine of the University of ManchesterUniversity of Manchester
The University of Manchester is a public research university located in Manchester, United Kingdom. It is a "red brick" university and a member of the Russell Group of research-intensive British universities and the N8 Group...
, Christopher Strachey
Christopher Strachey
Christopher Strachey was a British computer scientist. He was one of the founders of denotational semantics, and a pioneer in programming language design...
wrote a checkers program and Dietrich Prinz wrote one for chess. Arthur Samuel
Arthur Samuel
Arthur Lee Samuel was an American pioneer in the field of computer gaming and artificial intelligence. The Samuel Checkers-playing Program appears to be the world's first self-learning program, and as such a very early demonstration of the fundamental concept of artificial intelligence...
's checkers program, developed in the middle 50s and early 60s, eventually achieved sufficient skill to challenge a respectable amateur. Game AI would continue to be used as a measure of progress in AI throughout its history.
Turing's test
In 1950 Alan TuringAlan Turing
Alan Mathison Turing, OBE, FRS , was an English mathematician, logician, cryptanalyst, and computer scientist. He was highly influential in the development of computer science, providing a formalisation of the concepts of "algorithm" and "computation" with the Turing machine, which played a...
published a landmark paper
Computing machinery and intelligence
Computing Machinery and Intelligence, written by Alan Turing and published in 1950 in Mind, is a seminal paper on the topic of artificial intelligence in which the concept of what is now known as the Turing test was introduced to a wide audience....
in which he speculated about the possibility of creating machines with true intelligence.
He noted that "intelligence" is difficult to define and devised his famous Turing Test
Turing test
The Turing test is a test of a machine's ability to exhibit intelligent behaviour. In Turing's original illustrative example, a human judge engages in a natural language conversation with a human and a machine designed to generate performance indistinguishable from that of a human being. All...
. If a machine could carry on a conversation (over a teleprinter
Teleprinter
A teleprinter is a electromechanical typewriter that can be used to communicate typed messages from point to point and point to multipoint over a variety of communication channels that range from a simple electrical connection, such as a pair of wires, to the use of radio and microwave as the...
) that was indistinguishable from a conversation with a human being, then the machine could be called "intelligent." This simplified version of the problem allowed Turing to argue convincingly that a "thinking machine" was at least plausible and the paper answered all the most common objections to the proposition. The Turing Test
Turing test
The Turing test is a test of a machine's ability to exhibit intelligent behaviour. In Turing's original illustrative example, a human judge engages in a natural language conversation with a human and a machine designed to generate performance indistinguishable from that of a human being. All...
was the first serious proposal in the philosophy of artificial intelligence
Philosophy of artificial intelligence
The philosophy of artificial intelligence attempts to answer such questions as:* Can a machine act intelligently? Can it solve any problem that a person would solve by thinking?...
.
Symbolic reasoning and the Logic Theorist
When access to digital computers became possible in the middle fifties, a few scientists instinctively recognized that a machine that could manipulate numbers could also manipulate symbols and that the manipulation of symbols could well be the essence of human thought. This was a new approach to creating thinking machines.In 1955, Allen Newell
Allen Newell
Allen Newell was a researcher in computer science and cognitive psychology at the RAND corporation and at Carnegie Mellon University’s School of Computer Science, Tepper School of Business, and Department of Psychology...
and (future Nobel Laureate) Herbert Simon
Herbert Simon
Herbert Alexander Simon was an American political scientist, economist, sociologist, and psychologist, and professor—most notably at Carnegie Mellon University—whose research ranged across the fields of cognitive psychology, cognitive science, computer science, public administration, economics,...
created the "Logic Theorist
Logic Theorist
Logic Theorist is a computer program written in 1955 and 1956 by Allen Newell, Herbert Simon and J. C. Shaw. It was the first program deliberately engineered to mimic the problem solving skills of a human being and is called "the first artificial intelligence program." It would eventually prove 38...
" (with help from J. C. Shaw
Cliff Shaw
J.C. Shaw was a systems programmer at the RAND Corporation. He is a coauthor of the first artificial intelligence program, the Logic Theorist, and was one of the developers of Information Processing Language, a programming language of the 1950s. It is considered the true "father" of the JOSS...
). The program would eventually prove 38 of the first 52 theorems in Russell
Bertrand Russell
Bertrand Arthur William Russell, 3rd Earl Russell, OM, FRS was a British philosopher, logician, mathematician, historian, and social critic. At various points in his life he considered himself a liberal, a socialist, and a pacifist, but he also admitted that he had never been any of these things...
and Whitehead's
Alfred North Whitehead
Alfred North Whitehead, OM FRS was an English mathematician who became a philosopher. He wrote on algebra, logic, foundations of mathematics, philosophy of science, physics, metaphysics, and education...
Principia Mathematica
Principia Mathematica
The Principia Mathematica is a three-volume work on the foundations of mathematics, written by Alfred North Whitehead and Bertrand Russell and published in 1910, 1912, and 1913...
, and find new and more elegant proofs for some.
Simon said that they had "solved the venerable mind/body problem, explaining how a system composed of matter can have the properties of mind."
(This was an early statement of the philosophical position John Searle
John Searle
John Rogers Searle is an American philosopher and currently the Slusser Professor of Philosophy at the University of California, Berkeley.-Biography:...
would later call "Strong AI": that machines can contain minds just as human bodies do.)
Dartmouth Conference 1956: the birth of AI
The Dartmouth ConferenceDartmouth Conference
The Dartmouth Summer Research Conference on Artificial Intelligence was the name of a 1956 conference now considered the seminal event for artificial intelligence as a field.-People:...
of 1956
was organized by Marvin Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
, John McCarthy
John McCarthy (computer scientist)
John McCarthy was an American computer scientist and cognitive scientist. He coined the term "artificial intelligence" , invented the Lisp programming language and was highly influential in the early development of AI.McCarthy also influenced other areas of computing such as time sharing systems...
and two senior scientists: Claude Shannon and Nathan Rochester
Nathaniel Rochester (computer scientist)
Nathan Rochester designed the IBM 701, wrote the first assembler and participated in the founding of the field of artificial intelligence.- Early work :...
of IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
. The proposal for the conference included this assertion: "every aspect of learning or any other feature of intelligence can be so precisely described that a machine can be made to simulate it".
The participants included Ray Solomonoff
Ray Solomonoff
Ray Solomonoff was the inventor of algorithmic probability, and founder of algorithmic information theory, He was an originator of the branch of artificial intelligence based on machine learning, prediction and probability...
, Oliver Selfridge
Oliver Selfridge
Oliver Gordon Selfridge , grandson of Harry Gordon Selfridge, the founder of Selfridges' department stores, was a pioneer of artificial intelligence. He has been called the "Father of Machine Perception."...
, Trenchard More
Trenchard More
Trenchard More is a professor at Dartmouth College who participated in the 1956 Dartmouth Summer Research Project on Artificial Intelligence....
, Arthur Samuel
Arthur Samuel
Arthur Lee Samuel was an American pioneer in the field of computer gaming and artificial intelligence. The Samuel Checkers-playing Program appears to be the world's first self-learning program, and as such a very early demonstration of the fundamental concept of artificial intelligence...
, Allen Newell
Allen Newell
Allen Newell was a researcher in computer science and cognitive psychology at the RAND corporation and at Carnegie Mellon University’s School of Computer Science, Tepper School of Business, and Department of Psychology...
and Herbert Simon
Herbert Simon
Herbert Alexander Simon was an American political scientist, economist, sociologist, and psychologist, and professor—most notably at Carnegie Mellon University—whose research ranged across the fields of cognitive psychology, cognitive science, computer science, public administration, economics,...
, all of whom would create important programs during the first decades of AI research.
At the conference Newell and Simon debuted the "Logic Theorist
Logic Theorist
Logic Theorist is a computer program written in 1955 and 1956 by Allen Newell, Herbert Simon and J. C. Shaw. It was the first program deliberately engineered to mimic the problem solving skills of a human being and is called "the first artificial intelligence program." It would eventually prove 38...
" and McCarthy persuaded the attendees to accept "Artificial Intelligence" as the name of the field.
The 1956 Dartmouth conference was the moment that AI gained its name, its mission, its first success and its major players, and is widely considered the birth of AI.
The golden years 1956−1974
The years after the Dartmouth conference were an era of discovery, of sprinting across new ground. The programs that were developed during this time were, to most people, simply "astonishing": computers were solving algebra word problems, proving theorems in geometry and learning to speak English. Few at the time would have believed that such "intelligent" behavior by machines was possible at all. Researchers expressed an intense optimism in private and in print, predicting that a fully intelligent machine would be built in less than 20 years. Government agencies like ARPA poured money into the new field.The work
There were many successful programs and new directions in the late 50s and 1960s. Among the most influential were these:Reasoning as search
Many early AI programs used the same basic algorithmAlgorithm
In mathematics and computer science, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function. Algorithms are used for calculation, data processing, and automated reasoning...
. To achieve some goal (like winning a game or proving a theorem), they proceeded step by step towards it (by making a move or a deduction) as if searching through a maze, backtracking
Backtracking
Backtracking is a general algorithm for finding all solutions to some computational problem, that incrementally builds candidates to the solutions, and abandons each partial candidate c as soon as it determines that c cannot possibly be completed to a valid solution.The classic textbook example...
whenever they reached a dead end. This paradigm was called "reasoning as search".
The principal difficulty was that, for many problems, the number of possible paths through the "maze" was simply astronomical (a situation known as a "combinatorial explosion
Combinatorial explosion
In administration and computing, a combinatorial explosion is the rapidly accelerating increase in lines of communication as organizations are added in a process...
"). Researchers would reduce the search space by using heuristics or "rules of thumb" that would eliminate those paths that were unlikely to lead to a solution.
Newell
Allen Newell
Allen Newell was a researcher in computer science and cognitive psychology at the RAND corporation and at Carnegie Mellon University’s School of Computer Science, Tepper School of Business, and Department of Psychology...
and Simon
Herbert Simon
Herbert Alexander Simon was an American political scientist, economist, sociologist, and psychologist, and professor—most notably at Carnegie Mellon University—whose research ranged across the fields of cognitive psychology, cognitive science, computer science, public administration, economics,...
tried to capture a general version of this algorithm in a program called the "General Problem Solver
General Problem Solver
General Problem Solver was a computer program created in 1959 by Herbert Simon, J.C. Shaw, and Allen Newell intended to work as a universal problem solver machine. Any formalized symbolic problem can be solved, in principle, by GPS. For instance: theorems proof, geometric problems and chess...
". Other "searching" programs were able to accomplish impressive tasks like solving problems in geometry and algebra, such as Herbert Gelernter
Herbert Gelernter
Herbert Gelernter is professor Emeritus of in the Computer department of Stony Brook university -Short biography:Studied for his Ph.D at Rochester university in 1957...
's Geometry Theorem Prover (1958) and SAINT, written by Minsky's
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
student James Slagle (1961). Other programs searched through goals and subgoals to plan actions, like the STRIPS
STRIPS
In artificial intelligence, STRIPS is an automated planner developed by Richard Fikes and Nils Nilsson in 1971. The same name was later used to refer to the formal language of the inputs to this planner...
system developed at Stanford to control the behavior of their robot Shakey
Shakey the Robot
Shakey the Robot was the first general-purpose mobile robot to be able to reason about its own actions. While other robots would have to be instructed on each individual step of completing a larger task, Shakey could analyze the command and break it down into basic chunks by itself...
.
Natural language
An important goal of AI research is to allow computers to communicate in natural languagesNatural language processing
Natural language processing is a field of computer science and linguistics concerned with the interactions between computers and human languages; it began as a branch of artificial intelligence....
like English. An early success was Daniel Bobrow's program STUDENT
STUDENT (computer program)
STUDENT is an early artificial intelligence program that solves algebra word problems. It is written in Lisp by Daniel G Bobrow as his PhD thesis in 1964 . It was designed to read and solve the kind of word problems found in high school algebra books...
, which could solve high school algebra word problems.
A semantic net represents concepts (e.g. "house","door") as nodes and relations among concepts (e.g. "has-a") as links between the nodes. The first AI program to use a semantic net was written by Ross Quillian and the most successful (and controversial) version was Roger Schank
Roger Schank
Roger Schank is an American artificial intelligence theorist, cognitive psychologist, learning scientist, educational reformer, and entrepreneur.-Academic career:...
's Conceptual dependency theory
Conceptual dependency theory
Conceptual dependency theory is a model of natural language understanding used in artificial intelligence systems.Roger Schank at Stanford University introduced the model in 1969, in the early days of artificial intelligence...
.
Joseph Weizenbaum
Joseph Weizenbaum
Joseph Weizenbaum was a German-American author and professor emeritus of computer science at MIT.-Life and career:...
's ELIZA
ELIZA
ELIZA is a computer program and an early example of primitive natural language processing. ELIZA operated by processing users' responses to scripts, the most famous of which was DOCTOR, a simulation of a Rogerian psychotherapist. Using almost no information about human thought or emotion, DOCTOR...
could carry out conversations that were so realistic that users occasionally were fooled into thinking they were communicating with a human being and not a program. But in fact, ELIZA had no idea what she was talking about. She simply gave a canned response
Canned response
Canned responses are predetermined responses to common questions. Rather than typing the same answer repeatedly or pasting from some other resource, the operator can insert a canned response triggered by keystrokes or from a drop-down menu...
or repeated back what was said to her, rephrasing her response with a few grammar rules. ELIZA was the first chatterbot
Chatterbot
A chatter robot, chatterbot, chatbot, or chat bot is a computer program designed to simulate an intelligent conversation with one or more human users via auditory or textual methods, primarily for engaging in small talk. The primary aim of such simulation has been to fool the user into thinking...
.
Micro-worlds
In the late 60s, Marvin MinskyMarvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
and Seymour Papert
Seymour Papert
Seymour Papert is an MIT mathematician, computer scientist, and educator. He is one of the pioneers of artificial intelligence, as well as an inventor of the Logo programming language....
of the MIT AI Laboratory proposed that AI research should focus on artificially simple situations known as micro-worlds. They pointed out that in successful sciences like physics, basic principles were often best understood using simplified models like frictionless planes or perfectly rigid bodies. Much of the research focused on a "blocks world
Blocks world
The blocks world is one of the most famous planning domains in artificial intelligence. The program was created by Terry Winograd and is a limited-domain natural-language system that can understand typed commands and move blocks around on a surface....
," which consists of colored blocks of various shapes and sizes arrayed on a flat surface.
This paradigm led to innovative work in machine vision
Machine vision
Machine vision is the process of applying a range of technologies and methods to provide imaging-based automatic inspection, process control and robot guidance in industrial applications. While the scope of MV is broad and a comprehensive definition is difficult to distil, a "generally accepted...
by Gerald Sussman (who led the team), Adolfo Guzman, David Waltz
David Waltz
David L. Waltz is computer scientist who has made significant contributions in several areas of Artificial Intelligence,including constraint satisfaction, case-based reasoning and the application of massively parallel computation to AI problems...
(who invented "constraint propagation"), and especially Patrick Winston
Patrick Winston
Patrick Henry Winston is an American computer scientist, and is a professor at the Massachusetts Institute of Technology. Winston was director of the MIT Artificial Intelligence Laboratory from 1972 to 1997, succeeding Marvin Minsky, who left to found the MIT Media Lab and succeeded by Rodney Brooks...
. At the same time, Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
and Papert
Seymour Papert
Seymour Papert is an MIT mathematician, computer scientist, and educator. He is one of the pioneers of artificial intelligence, as well as an inventor of the Logo programming language....
built a robot arm that could stack blocks, bringing the blocks world to life. The crowning achievement of the micro-world program was Terry Winograd
Terry Winograd
Terry Allen Winograd is an American professor of computer science at Stanford University, and co-director of the Stanford Human-Computer Interaction Group...
's SHRDLU
SHRDLU
SHRDLU was an early natural language understanding computer program, developed by Terry Winograd at MIT from 1968-1970. In it, the user carries on a conversation with the computer, moving objects, naming collections and querying the state of a simplified "blocks world", essentially a virtual box...
. It could communicate in ordinary English sentences, plan operations and execute them.
The optimism
The first generation of AI researchers made these predictions about their work:- 1958, H. A. Simon and Allen NewellAllen NewellAllen Newell was a researcher in computer science and cognitive psychology at the RAND corporation and at Carnegie Mellon University’s School of Computer Science, Tepper School of Business, and Department of Psychology...
: "within ten years a digital computer will be the world's chess champion" and "within ten years a digital computer will discover and prove an important new mathematical theorem." - 1965, H. A. Simon: "machines will be capable, within twenty years, of doing any work a man can do."
- 1967, Marvin MinskyMarvin MinskyMarvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
: "Within a generation ... the problem of creating 'artificial intelligence' will substantially be solved." - 1970, Marvin MinskyMarvin MinskyMarvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
(in Life Magazine): "In from three to eight years we will have a machine with the general intelligence of an average human being."
The money
In June 1963 MIT received a $2.2 million grant from the newly created Advanced Research Projects Agency (later known as DARPA). The money was used to fund project MAC which subsumed the "AI Group" founded by MinskyMarvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
and McCarthy
John McCarthy (computer scientist)
John McCarthy was an American computer scientist and cognitive scientist. He coined the term "artificial intelligence" , invented the Lisp programming language and was highly influential in the early development of AI.McCarthy also influenced other areas of computing such as time sharing systems...
five years earlier. ARPA continued to provide three million dollars a year until the 70s.
ARPA made similar grants to Newell
Allen Newell
Allen Newell was a researcher in computer science and cognitive psychology at the RAND corporation and at Carnegie Mellon University’s School of Computer Science, Tepper School of Business, and Department of Psychology...
and Simon's
Herbert Simon
Herbert Alexander Simon was an American political scientist, economist, sociologist, and psychologist, and professor—most notably at Carnegie Mellon University—whose research ranged across the fields of cognitive psychology, cognitive science, computer science, public administration, economics,...
program at CMU
Carnegie Mellon University
Carnegie Mellon University is a private research university in Pittsburgh, Pennsylvania, United States....
and to the Stanford AI Project (founded by John McCarthy
John McCarthy (computer scientist)
John McCarthy was an American computer scientist and cognitive scientist. He coined the term "artificial intelligence" , invented the Lisp programming language and was highly influential in the early development of AI.McCarthy also influenced other areas of computing such as time sharing systems...
in 1963). Another important AI laboratory was established at Edinburgh University by Donald Michie
Donald Michie
Donald Michie was a British researcher in artificial intelligence. During World War II, Michie worked for the Government Code and Cypher School at Bletchley Park, contributing to the effort to solve "Tunny," a German teleprinter cipher.-Early life and career:Michie was born in Rangoon, Burma...
in 1965.
These four institutions would continue to be the main centers of AI research (and funding) in academia for many years.
The money was proffered with few strings attached: J. C. R. Licklider
J. C. R. Licklider
Joseph Carl Robnett Licklider , known simply as J.C.R. or "Lick" was an American computer scientist, considered one of the most important figures in computer science and general computing history...
, then the director of ARPA, believed that his organization should "fund people, not projects!" and allowed researchers to pursue whatever directions might interest them. This created a freewheeling atmosphere at MIT that gave birth to the hacker culture, but this "hands off" approach would not last.
The first AI winter 1974−1980
In the 70s, AI was subject to critiques and financial setbacks. AI researchers had failed to appreciate the difficulty of the problems they faced. Their tremendous optimism had raised expectations impossibly high, and when the promised results failed to materialize, funding for AI disappeared. At the same time, the field of connectionismConnectionism
Connectionism is a set of approaches in the fields of artificial intelligence, cognitive psychology, cognitive science, neuroscience and philosophy of mind, that models mental or behavioral phenomena as the emergent processes of interconnected networks of simple units...
(or neural nets) was shut down almost completely for 10 years by Marvin Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
's devastating criticism of perceptrons.
Despite the difficulties with public perception of AI in the late 70s, new ideas were explored in logic programming
Logic programming
Logic programming is, in its broadest sense, the use of mathematical logic for computer programming. In this view of logic programming, which can be traced at least as far back as John McCarthy's [1958] advice-taker proposal, logic is used as a purely declarative representation language, and a...
, commonsense reasoning
Commonsense reasoning
Commonsense reasoning is the branch of Artificial intelligence concerned with replicating human thinking. There are several components to this problem, including:* Developing adequately broad and deep commonsense knowledge bases....
and many other areas.
The problems
In the early seventies, the capabilities of AI programs were limited. Even the most impressive could only handle trivial versions of the problems they were supposed to solve; all the programs were, in some sense, "toys". AI researchers had begun to run into several fundamental limits that could not be overcome in the 1970s. Although some of these limits would be conquered in later decades, others still stymie the field to this day.- Limited computer power: There was not enough memory or processing speed to accomplish anything truly useful. For example, Ross Quillian's successful work on natural language was demonstrated with a vocabulary of only twenty words, because that was all that would fit in memory. Hans MoravecHans MoravecHans Moravec is an adjunct faculty member at the Robotics Institute of Carnegie Mellon University. He is known for his work on robotics, artificial intelligence, and writings on the impact of technology. Moravec also is a futurist with many of his publications and predictions focusing on...
argued in 1976 that computers were still millions of times too weak to exhibit intelligence. He suggested an analogy: artificial intelligence requires computer power in the same way that aircraft require power. Below a certain threshold, it's impossible, but, as power increasesMoore's LawMoore's law describes a long-term trend in the history of computing hardware: the number of transistors that can be placed inexpensively on an integrated circuit doubles approximately every two years....
, eventually it could become easy. With regard to computer vision, Moravec estimated that simply matching the edge and motion detection capabilities of human retina in real time would require a general-purpose computer capable of 109 operations/second (1000 MIPS). As of 2011, practical computer vision applications require 10,000 to 1,000,000 MIPS. By comparison, the fastest supercomputer in 1976, Cray-1Cray-1The Cray-1 was a supercomputer designed, manufactured, and marketed by Cray Research. The first Cray-1 system was installed at Los Alamos National Laboratory in 1976, and it went on to become one of the best known and most successful supercomputers in history...
(retailing at $5 million to $8 million), was only capable of around 80 to 130 MIPS, and a typical desktop computer at the time achieved less than 1 MIPS.
- Intractability and the combinatorial explosionCombinatorial explosionIn administration and computing, a combinatorial explosion is the rapidly accelerating increase in lines of communication as organizations are added in a process...
. In 1972 Richard KarpRichard KarpRichard Manning Karp is a computer scientist and computational theorist at the University of California, Berkeley, notable for research in the theory of algorithms, for which he received a Turing Award in 1985, The Benjamin Franklin Medal in Computer and Cognitive Science in 2004, and the Kyoto...
(building on Stephen CookStephen CookStephen Arthur Cook is a renowned American-Canadian computer scientist and mathematician who has made major contributions to the fields of complexity theory and proof complexity...
's 1971 theoremCook's theoremIn computational complexity theory, the Cook–Levin theorem, also known as Cook's theorem, states that the Boolean satisfiability problem is NP-complete...
) showed there are many problemsKarp's 21 NP-complete problemsOne of the most important results in computational complexity theory was Stephen Cook's 1971 demonstration of the first NP-complete problem, the boolean satisfiability problem...
that can probably only be solved in exponential time (in the size of the inputs). Finding optimal solutions to these problems requires unimaginable amounts of computer time except when the problems are trivial. This almost certainly meant that many of the "toy" solutions used by AI would probably never scale up into useful systems. - Commonsense knowledge and reasoningCommonsense reasoningCommonsense reasoning is the branch of Artificial intelligence concerned with replicating human thinking. There are several components to this problem, including:* Developing adequately broad and deep commonsense knowledge bases....
. Many important artificial intelligence applications like visionComputer visionComputer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
or natural languageNatural languageIn the philosophy of language, a natural language is any language which arises in an unpremeditated fashion as the result of the innate facility for language possessed by the human intellect. A natural language is typically used for communication, and may be spoken, signed, or written...
require simply enormous amounts of information about the world: the program needs to have some idea of what it might be looking at or what it is talking about. This requires that the program know most of the same things about the world that a child does. Researchers soon discovered that this was a truly vast amount of information. No one in 1970 could build a database so large and no one knew how a program might learn so much information. - Moravec's paradoxMoravec's paradoxMoravec's paradox is the discovery by artificial intelligence and robotics researchers that, contrary to traditional assumptions, high-level reasoning requires very little computation, but low-level sensorimotor skills require enormous computational resources. The principle was articulated by Hans...
: Proving theorems and solving geometry problems is comparatively easy for computers, but a supposedly simple task like recognizing a face or crossing a room without bumping into anything is extremely difficult. This helps explain why research into visionMachine visionMachine vision is the process of applying a range of technologies and methods to provide imaging-based automatic inspection, process control and robot guidance in industrial applications. While the scope of MV is broad and a comprehensive definition is difficult to distil, a "generally accepted...
and roboticsRoboticsRobotics is the branch of technology that deals with the design, construction, operation, structural disposition, manufacture and application of robots...
had made so little progress by the middle 1970s. - The frameFrame problemIn artificial intelligence, the frame problem was initially formulated as the problem of expressing a dynamical domain in logic without explicitly specifying which conditions are not affected by an action. John McCarthy and Patrick J. Hayes defined this problem in their 1969 article, Some...
and qualification problemQualification problemIn philosophy and AI , the qualification problem is concerned with the impossibility of listing all the preconditions required for a real-world action to have its intended effect. It might be posed as how to deal with the things that prevent me from achieving my intended result...
s. AI researchers (like John McCarthyJohn McCarthy (computer scientist)John McCarthy was an American computer scientist and cognitive scientist. He coined the term "artificial intelligence" , invented the Lisp programming language and was highly influential in the early development of AI.McCarthy also influenced other areas of computing such as time sharing systems...
) who used logicLogicIn philosophy, Logic is the formal systematic study of the principles of valid inference and correct reasoning. Logic is used in most intellectual activities, but is studied primarily in the disciplines of philosophy, mathematics, semantics, and computer science...
discovered that they could not represent ordinary deductions that involved planningAutomated planning and schedulingAutomated planning and scheduling is a branch of artificial intelligence that concerns the realization of strategies or action sequences, typically for execution by intelligent agents, autonomous robots and unmanned vehicles. Unlike classical control and classification problems, the solutions are...
or default reasoning without making changes to the structure of logic itself. They developed new logics (like non-monotonic logicNon-monotonic logicA non-monotonic logic is a formal logic whose consequence relation is not monotonic. Most studied formal logics have a monotonic consequence relation, meaning that adding a formula to a theory never produces a reduction of its set of consequences. Intuitively, monotonicity indicates that learning a...
s and modal logicModal logicModal logic is a type of formal logic that extends classical propositional and predicate logic to include operators expressing modality. Modals — words that express modalities — qualify a statement. For example, the statement "John is happy" might be qualified by saying that John is...
s) to try to solve the problems.
The end of funding
The agencies which funded AI research (such as the British government, DARPA and NRCUnited States National Research Council
The National Research Council of the USA is the working arm of the United States National Academies, carrying out most of the studies done in their names.The National Academies include:* National Academy of Sciences...
) became frustrated with the lack of progress and eventually cut off almost all funding for undirected research into AI. The pattern began as early as 1966 when the ALPAC
ALPAC
ALPAC was a committee of seven scientists led by John R. Pierce, established in 1964 by the U. S. Government in order to evaluate the progress in computational linguistics in general and machine translation in particular...
report appeared criticizing machine translation efforts. After spending 20 million dollars, the NRC
United States National Research Council
The National Research Council of the USA is the working arm of the United States National Academies, carrying out most of the studies done in their names.The National Academies include:* National Academy of Sciences...
ended all support.
In 1973, the Lighthill report
Lighthill report
The Lighthill report is the name commonly used for the paper "Artificial Intelligence: A General Survey" by James Lighthill, published in Artificial Intelligence: a paper symposium in 1973....
on the state of AI research in England criticized the utter failure of AI to achieve its "grandiose objectives" and led to the dismantling of AI research in that country.
(The report specifically mentioned the combinatorial explosion
Combinatorial explosion
In administration and computing, a combinatorial explosion is the rapidly accelerating increase in lines of communication as organizations are added in a process...
problem as a reason for AI's failings.)
DARPA was deeply disappointed with researchers working on the Speech Understanding Research
Speech recognition
Speech recognition converts spoken words to text. The term "voice recognition" is sometimes used to refer to recognition systems that must be trained to a particular speaker—as is the case for most desktop recognition software...
program at CMU
Carnegie Mellon University
Carnegie Mellon University is a private research university in Pittsburgh, Pennsylvania, United States....
and canceled an annual grant of three million dollars.
By 1974, funding for AI projects was hard to find.
Hans Moravec
Hans Moravec
Hans Moravec is an adjunct faculty member at the Robotics Institute of Carnegie Mellon University. He is known for his work on robotics, artificial intelligence, and writings on the impact of technology. Moravec also is a futurist with many of his publications and predictions focusing on...
blamed the crisis on the unrealistic predictions of his colleagues. "Many researchers were caught up in a web of increasing exaggeration."
However, there was another issue: since the passage of the Mansfield Amendment in 1969, DARPA had been under increasing pressure to fund "mission-oriented direct research, rather than basic undirected research". Funding for the creative, freewheeling exploration that had gone on in the 60s would not come from DARPA. Instead, the money was directed at specific projects with clear objectives, such as autonomous tanks and battle management systems.
Critiques from across campus
Several philosophers had strong objections to the claims being made by AI researchers. One of the earliest was John LucasJohn Lucas (philosopher)
- Overview :John Lucas was educated at Winchester College and Balliol College, Oxford, where he studied first mathematics, then Greats , obtaining first class honors, and proceeding to an MA in Philosophy in 1954. He spent the 1957-58 academic year at Princeton University, deepening his...
, who argued that Gödel's
Kurt Gödel
Kurt Friedrich Gödel was an Austrian logician, mathematician and philosopher. Later in his life he emigrated to the United States to escape the effects of World War II. One of the most significant logicians of all time, Gödel made an immense impact upon scientific and philosophical thinking in the...
incompleteness theorem showed that a formal system
Formal system
In formal logic, a formal system consists of a formal language and a set of inference rules, used to derive an expression from one or more other premises that are antecedently supposed or derived . The axioms and rules may be called a deductive apparatus...
(such as a computer program) could never see the truth of certain statements, while a human being could. Hubert Dreyfus
Hubert Dreyfus
Hubert Lederer Dreyfus is an American philosopher. He is a professor of philosophy at the University of California, Berkeley....
ridiculed the broken promises of the 60s and critiqued the assumptions of AI, arguing that human reasoning actually involved very little "symbol processing" and a great deal of embodied
Embodied cognition
Philosophers, psychologists, cognitive scientists and artificial intelligence researchers who study embodied cognition and the embodied mind believe that the nature of the human mind is largely determined by the form of the human body. They argue that all aspects of cognition, such as ideas,...
, instinct
Instinct
Instinct or innate behavior is the inherent inclination of a living organism toward a particular behavior.The simplest example of an instinctive behavior is a fixed action pattern, in which a very short to medium length sequence of actions, without variation, are carried out in response to a...
ive, unconscious "know how". John Searle
John Searle
John Rogers Searle is an American philosopher and currently the Slusser Professor of Philosophy at the University of California, Berkeley.-Biography:...
's Chinese Room
Chinese room
The Chinese room is a thought experiment by John Searle, which first appeared in his paper "Minds, Brains, and Programs", published in Behavioral and Brain Sciences in 1980...
argument, presented in 1980, attempted to show that a program could not be said to "understand" the symbols that it uses (a quality called "intentionality
Intentionality
The term intentionality was introduced by Jeremy Bentham as a principle of utility in his doctrine of consciousness for the purpose of distinguishing acts that are intentional and acts that are not...
"). If the symbols have no meaning for the machine, Searle argued, then the machine can not be described as "thinking".
These critiques were not taken seriously by AI researchers, often because they seemed so far off the point. Problems like intractability and commonsense knowledge
Commonsense reasoning
Commonsense reasoning is the branch of Artificial intelligence concerned with replicating human thinking. There are several components to this problem, including:* Developing adequately broad and deep commonsense knowledge bases....
seemed much more immediate and serious. It was unclear what difference "know how" or "intentionality
Intentionality
The term intentionality was introduced by Jeremy Bentham as a principle of utility in his doctrine of consciousness for the purpose of distinguishing acts that are intentional and acts that are not...
" made to an actual computer program
Computer program
A computer program is a sequence of instructions written to perform a specified task with a computer. A computer requires programs to function, typically executing the program's instructions in a central processor. The program has an executable form that the computer can use directly to execute...
. Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
said of Dreyfus and Searle "they misunderstand, and should be ignored." Dreyfus, who taught at MIT, was given a cold shoulder: he later said that AI researchers "dared not be seen having lunch with me." Joseph Weizenbaum
Joseph Weizenbaum
Joseph Weizenbaum was a German-American author and professor emeritus of computer science at MIT.-Life and career:...
, the author of ELIZA
ELIZA
ELIZA is a computer program and an early example of primitive natural language processing. ELIZA operated by processing users' responses to scripts, the most famous of which was DOCTOR, a simulation of a Rogerian psychotherapist. Using almost no information about human thought or emotion, DOCTOR...
, felt his colleagues' treatment of Dreyfus
Hubert Dreyfus
Hubert Lederer Dreyfus is an American philosopher. He is a professor of philosophy at the University of California, Berkeley....
was unprofessional and childish. Although he was an outspoken critic of Dreyfus' positions, he "deliberately made it plain that theirs was not the way to treat a human being."
Weizenbaum began to have serious ethical doubts about AI when Kenneth Colby
Kenneth Colby
Kenneth Mark Colby, M.D. was an American psychiatrist dedicated to the theory and application of computer science and artificial intelligence to psychiatry. Colby was a pioneer in the development of computer technology as a tool to try to understand cognitive functions and to assist both patients...
wrote DOCTOR
ELIZA
ELIZA is a computer program and an early example of primitive natural language processing. ELIZA operated by processing users' responses to scripts, the most famous of which was DOCTOR, a simulation of a Rogerian psychotherapist. Using almost no information about human thought or emotion, DOCTOR...
, a chatterbot
Chatterbot
A chatter robot, chatterbot, chatbot, or chat bot is a computer program designed to simulate an intelligent conversation with one or more human users via auditory or textual methods, primarily for engaging in small talk. The primary aim of such simulation has been to fool the user into thinking...
therapist. Weizenbaum was disturbed that Colby saw his mindless program as a serious therapeutic tool. A feud began, and the situation was not helped when Colby did not credit Weizenbaum for his contribution to the program. In 1976, Weizenbaum
Joseph Weizenbaum
Joseph Weizenbaum was a German-American author and professor emeritus of computer science at MIT.-Life and career:...
published Computer Power and Human Reason
Computer Power and Human Reason
Joseph Weizenbaum's influential 1976 book Computer Power and Human Reason: From Judgment To Calculation displays his ambivalence towards computer technology and lays out his case: while artificial intelligence may be possible, we should never allow computers to make important decisions because...
which argued that the misuse of artificial intelligence has the potential to devalue human life.
Perceptrons and the dark age of connectionism
A perceptronPerceptron
The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. It can be seen as the simplest kind of feedforward neural network: a linear classifier.- Definition :...
was a form of neural network
Neural network
The term neural network was traditionally used to refer to a network or circuit of biological neurons. The modern usage of the term often refers to artificial neural networks, which are composed of artificial neurons or nodes...
introduced in 1958 by Frank Rosenblatt
Frank Rosenblatt
Frank Rosenblatt was a New York City born computer scientist who completed the Perceptron, or MARK 1, computer at Cornell University in 1960...
, who had been a schoolmate of Marvin Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
at the Bronx High School of Science
Bronx High School of Science
The Bronx High School of Science is a specialized New York City public high school often considered the premier science magnet school in the United States. Founded in 1938, it is now located in the Bedford Park section of the Bronx...
. Like most AI researchers, he was optimistic about their power, predicting that "perceptron may eventually be able to learn, make decisions, and translate languages." An active research program into the paradigm was carried out throughout the 60s but came to a sudden halt with the publication of Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
and Papert's
Seymour Papert
Seymour Papert is an MIT mathematician, computer scientist, and educator. He is one of the pioneers of artificial intelligence, as well as an inventor of the Logo programming language....
1969 book Perceptrons
Perceptrons (book)
Perceptrons: an introduction to computational geometry is a book authored by Marvin Minsky and Seymour Papert, published in 1969. An edition with handwritten corrections and additions was released in the early 1970s...
. It suggested that there were severe limitations to what perceptrons could do and that Frank Rosenblatt
Frank Rosenblatt
Frank Rosenblatt was a New York City born computer scientist who completed the Perceptron, or MARK 1, computer at Cornell University in 1960...
's predictions had been grossly exaggerated. The effect of the book was devastating: virtually no research at all was done in connectionism
Connectionism
Connectionism is a set of approaches in the fields of artificial intelligence, cognitive psychology, cognitive science, neuroscience and philosophy of mind, that models mental or behavioral phenomena as the emergent processes of interconnected networks of simple units...
for 10 years. Eventually, a new generation of researchers would revive the field and thereafter it would become a vital and useful part of artificial intelligence. Rosenblatt
Frank Rosenblatt
Frank Rosenblatt was a New York City born computer scientist who completed the Perceptron, or MARK 1, computer at Cornell University in 1960...
would not live to see this, as he died in a boating accident shortly after the book was published.
The neats: logic, Prolog and expert systems
Logic was introduced into AI research as early as 1958, by John McCarthyJohn McCarthy (computer scientist)
John McCarthy was an American computer scientist and cognitive scientist. He coined the term "artificial intelligence" , invented the Lisp programming language and was highly influential in the early development of AI.McCarthy also influenced other areas of computing such as time sharing systems...
in his Advice Taker
Advice taker
The advice taker was a hypothetical computer program, proposed by John McCarthy in his 1958 paper "Programs with Common Sense" . It was probably the first proposal to use logic to represent information in a computer and not just as the subject matter of another program. It may also have been the...
proposal.
In 1963, J. Alan Robinson
J. Alan Robinson
John Alan Robinson is a philosopher , mathematician and computer scientist. He is University Professor Emeritus at Syracuse University, United States....
had discovered a simple method to implement deduction on computers, the resolution
Resolution (logic)
In mathematical logic and automated theorem proving, resolution is a rule of inference leading to a refutation theorem-proving technique for sentences in propositional logic and first-order logic...
and unification algorithm. However, straightforward implementations, like those attempted by McCarthy and his students in the late 60s, were especially intractable: the programs required astronomical numbers of steps to prove simple theorems. A more fruitful approach to logic was developed in the 70s by Robert Kowalski
Robert Kowalski
Robert "Bob" Anthony Kowalski is a British logician and computer scientist, who has spent most of his career in the United Kingdom....
at the University of Edinburgh
University of Edinburgh
The University of Edinburgh, founded in 1583, is a public research university located in Edinburgh, the capital of Scotland, and a UNESCO World Heritage Site. The university is deeply embedded in the fabric of the city, with many of the buildings in the historic Old Town belonging to the university...
, and soon this led to the collaboration with French researchers Alain Colmerauer
Alain Colmerauer
Alain Colmerauer is a French computer scientist.After completing his Ph.D. at the University of Grenoble, he spent 1967–1970 as Assistant Professor at the University of Montreal, where he created Q-Systems, one of the earliest linguistic formalisms used in the development of the TAUM-METEO machine...
and Phillipe Roussel who created the successful logic programming language Prolog
Prolog
Prolog is a general purpose logic programming language associated with artificial intelligence and computational linguistics.Prolog has its roots in first-order logic, a formal logic, and unlike many other programming languages, Prolog is declarative: the program logic is expressed in terms of...
.
Prolog uses a subset of logic (Horn clause
Horn clause
In mathematical logic, a Horn clause is a clause with at most one positive literal. They are named after the logician Alfred Horn, who first pointed out the significance of such clauses in 1951...
s, closely related to "rules" and "production rules
Production system
A production system is a computer program typically used to provide some form of artificial intelligence, which consists primarily of a set of rules about behavior. These rules, termed productions, are a basic representation found useful in automated planning, expert systems and action selection...
") that permit tractable computation. Rules would continue to be influential, providing a foundation for Edward Feigenbaum
Edward Feigenbaum
Edward Albert Feigenbaum is a computer scientist working in the field of artificial intelligence. He is often called the "father of expert systems."...
's expert systems and the continuing work by Allen Newell
Allen Newell
Allen Newell was a researcher in computer science and cognitive psychology at the RAND corporation and at Carnegie Mellon University’s School of Computer Science, Tepper School of Business, and Department of Psychology...
and Herbert Simon
Herbert Simon
Herbert Alexander Simon was an American political scientist, economist, sociologist, and psychologist, and professor—most notably at Carnegie Mellon University—whose research ranged across the fields of cognitive psychology, cognitive science, computer science, public administration, economics,...
that would lead to Soar
Soar (cognitive architecture)
Soar is a symbolic cognitive architecture, created by John Laird, Allen Newell, and Paul Rosenbloom at Carnegie Mellon University, now maintained by John Laird's research group at the University of Michigan. It is both a view of what cognition is and an implementation of that view through a...
and their unified theories of cognition
Unified theory of cognition
Unified Theories of Cognition is a 1990 book by Allen Newell. Newell argues for the need of a set of general assumptions for cognitive models that account for all of cognition: a unified theory of cognition ....
.
Critics of the logical approach noted, as Dreyfus
Hubert Dreyfus
Hubert Lederer Dreyfus is an American philosopher. He is a professor of philosophy at the University of California, Berkeley....
had, that human beings rarely used logic when they solved problems. Experiments by psychologists like Peter Wason
Peter Cathcart Wason
Peter Cathcart Wason was a cognitive psychologist, who worked on the psychology of reason. He made great progress in explaining why people make certain consistent mistakes in logical reasoning...
, Eleanor Rosch
Eleanor Rosch
Eleanor Rosch is a professor of psychology at the University of California, Berkeley, specializing in cognitive psychology and primarily known for her work on categorization, in particular her prototype theory, which has profoundly influenced the field of cognitive psychology...
, Amos Tversky
Amos Tversky
Amos Nathan Tversky, was a cognitive and mathematical psychologist, a pioneer of cognitive science, a longtime collaborator of Daniel Kahneman, and a key figure in the discovery of systematic human cognitive bias and handling of risk. Much of his early work concerned the foundations of measurement...
, Daniel Kahneman
Daniel Kahneman
Daniel Kahneman is an Israeli-American psychologist and Nobel laureate. He is notable for his work on the psychology of judgment and decision-making, behavioral economics and hedonic psychology....
and others provided proof.
McCarthy responded that what people do is irrelevant. He argued that what is really needed are machines that can solve problems—not machines that think as people do.
The scruffies: frames and scripts
Among the critics of McCarthy'sJohn McCarthy (computer scientist)
John McCarthy was an American computer scientist and cognitive scientist. He coined the term "artificial intelligence" , invented the Lisp programming language and was highly influential in the early development of AI.McCarthy also influenced other areas of computing such as time sharing systems...
approach were his colleagues across the country at MIT. Marvin Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
, Seymour Papert
Seymour Papert
Seymour Papert is an MIT mathematician, computer scientist, and educator. He is one of the pioneers of artificial intelligence, as well as an inventor of the Logo programming language....
and Roger Schank
Roger Schank
Roger Schank is an American artificial intelligence theorist, cognitive psychologist, learning scientist, educational reformer, and entrepreneur.-Academic career:...
were trying to solve problems like "story understanding" and "object recognition" that required a machine to think like a person. In order to use ordinary concepts like "chair" or "restaurant" they had to make all the same illogical assumptions that people normally made. Unfortunately, imprecise concepts like these are hard to represent in logic. Gerald Sussman observed that "using precise language to describe essentially imprecise concepts doesn't make them any more precise." Schank
Roger Schank
Roger Schank is an American artificial intelligence theorist, cognitive psychologist, learning scientist, educational reformer, and entrepreneur.-Academic career:...
described their "anti-logic" approaches as "scruffy
Neats vs. scruffies
Neat and scruffy are labels for two different types of artificial intelligence research. Neats consider that solutions should be elegant, clear and provably correct...
", as opposed to the "neat
Neats vs. scruffies
Neat and scruffy are labels for two different types of artificial intelligence research. Neats consider that solutions should be elegant, clear and provably correct...
" paradigms used by McCarthy
John McCarthy (computer scientist)
John McCarthy was an American computer scientist and cognitive scientist. He coined the term "artificial intelligence" , invented the Lisp programming language and was highly influential in the early development of AI.McCarthy also influenced other areas of computing such as time sharing systems...
, Kowalski
Robert Kowalski
Robert "Bob" Anthony Kowalski is a British logician and computer scientist, who has spent most of his career in the United Kingdom....
, Feigenbaum
Edward Feigenbaum
Edward Albert Feigenbaum is a computer scientist working in the field of artificial intelligence. He is often called the "father of expert systems."...
, Newell
Allen Newell
Allen Newell was a researcher in computer science and cognitive psychology at the RAND corporation and at Carnegie Mellon University’s School of Computer Science, Tepper School of Business, and Department of Psychology...
and Simon
Herbert Simon
Herbert Alexander Simon was an American political scientist, economist, sociologist, and psychologist, and professor—most notably at Carnegie Mellon University—whose research ranged across the fields of cognitive psychology, cognitive science, computer science, public administration, economics,...
.
In 1975, in a seminal paper, Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
noted that many of his fellow "scruffy" researchers were using the same kind of tool: a framework that captures all our common sense assumptions about something. For example, if we use the concept of a bird, there is a constellation of facts that immediately come to mind: we might assume that it flies, eats worms and so on. We know these facts are not always true and that deductions using these facts will not be "logical," but these structured sets of assumptions are part of the context of everything we say and think. He called these structures "frames
Frame (artificial intelligence)
Frames were proposed by Marvin Minsky in his 1974 article "A Framework for Representing Knowledge." A frame is an artificial intelligence data structure used to divide knowledge into substructures by representing "stereotyped situations." Frames are connected together to form a complete idea.Frames...
". Schank
Roger Schank
Roger Schank is an American artificial intelligence theorist, cognitive psychologist, learning scientist, educational reformer, and entrepreneur.-Academic career:...
used a version of frames he called "scripts
Scripts (artificial intelligence)
Scripts were developed in the early AI work by Roger Schank, Robert P. Abelson and their research group, and are a method of representing procedural knowledge...
" to successfully answer questions about short stories in English. Many years later object-oriented programming
Object-oriented programming
Object-oriented programming is a programming paradigm using "objects" – data structures consisting of data fields and methods together with their interactions – to design applications and computer programs. Programming techniques may include features such as data abstraction,...
would adopt the essential idea of "inheritance
Inheritance (computer science)
In object-oriented programming , inheritance is a way to reuse code of existing objects, establish a subtype from an existing object, or both, depending upon programming language support...
" from AI research on frames.
Boom 1980–1987
In the 1980s a form of AI program called "expert systemExpert system
In artificial intelligence, an expert system is a computer system that emulates the decision-making ability of a human expert. Expert systems are designed to solve complex problems by reasoning about knowledge, like an expert, and not by following the procedure of a developer as is the case in...
s" was adopted by corporations around the world and knowledge
Knowledge representation
Knowledge representation is an area of artificial intelligence research aimed at representing knowledge in symbols to facilitate inferencing from those knowledge elements, creating new elements of knowledge...
became the focus of mainstream AI research. In those same years, the Japanese government aggressively funded AI with its fifth generation computer
Fifth generation computer
The Fifth Generation Computer Systems project was an initiative by Japan'sMinistry of International Trade and Industry, begun in 1982, to create a "fifth generation computer" which was supposed to perform much calculation using massive parallel processing...
project. Another encouraging event in the early 1980s was the revival of connectionism
Connectionism
Connectionism is a set of approaches in the fields of artificial intelligence, cognitive psychology, cognitive science, neuroscience and philosophy of mind, that models mental or behavioral phenomena as the emergent processes of interconnected networks of simple units...
in the work of John Hopfield and David Rumelhart
David Rumelhart
David Everett Rumelhart was an American psychologist who made many contributions to the formal analysis of human cognition, working primarily within the frameworks of mathematical psychology, symbolic artificial intelligence, and parallel distributed processing...
. Once again, AI had achieved success.
The rise of expert systems
An expert systemExpert system
In artificial intelligence, an expert system is a computer system that emulates the decision-making ability of a human expert. Expert systems are designed to solve complex problems by reasoning about knowledge, like an expert, and not by following the procedure of a developer as is the case in...
is a program that answers questions or solves problems about a specific domain of knowledge, using logical rules
Production system
A production system is a computer program typically used to provide some form of artificial intelligence, which consists primarily of a set of rules about behavior. These rules, termed productions, are a basic representation found useful in automated planning, expert systems and action selection...
that are derived from the knowledge of experts. The earliest examples were developed by Edward Feigenbaum
Edward Feigenbaum
Edward Albert Feigenbaum is a computer scientist working in the field of artificial intelligence. He is often called the "father of expert systems."...
and his students. Dendral
Dendral
Dendral was an influential pioneer project in artificial intelligence of the 1960s, and the computer software expert system that it produced. Its primary aim was to study hypothesis formation and discovery in science...
, begun in 1965, identified compounds from spectrometer readings. MYCIN
Mycin
In artificial intelligence, MYCIN was an early expert system designed to identify bacteria causing severe infections, such as bacteremia and meningitis, and to recommend antibiotics, with the dosage adjusted for patient's body weight — the name derived from the antibiotics themselves, as many...
, developed in 1972, diagnosed infectious blood diseases. They demonstrated the feasibility of the approach.
Expert systems restricted themselves to a small domain of specific knowledge (thus avoiding the commonsense knowledge problem) and their simple design made it relatively easy for programs to be built and then modified once they were in place. All in all, the programs proved to be useful: something that AI had not been able to achieve up to this point.
In 1980, an expert system called XCON
Xcon
The R1 program was a production-rule-based system written in OPS5 by John P. McDermott of CMU in 1978 to assist in the ordering of DEC's VAX computer systems by automatically selecting the computer system components based on the customer's requirements...
was completed at CMU
Carnegie Mellon University
Carnegie Mellon University is a private research university in Pittsburgh, Pennsylvania, United States....
for the Digital Equipment Corporation
Digital Equipment Corporation
Digital Equipment Corporation was a major American company in the computer industry and a leading vendor of computer systems, software and peripherals from the 1960s to the 1990s...
. It was an enormous success: it was saving the company 40 million dollars annually by 1986. Corporations around the world began to develop and deploy expert systems and by 1985 they were spending over a billion dollars on AI, most of it to in-house AI departments. An industry grew up to support them, including hardware companies like Symbolics
Symbolics
Symbolics refers to two companies: now-defunct computer manufacturer Symbolics, Inc., and a privately held company that acquired the assets of the former company and continues to sell and maintain the Open Genera Lisp system and the Macsyma computer algebra system.The symbolics.com domain was...
and Lisp Machines
Lisp Machines
Lisp Machines, Inc. was a company formed in 1979 by Richard Greenblatt of MIT's Artificial Intelligence Laboratory to build Lisp machines. It was based in Cambridge, Massachusetts....
and software companies such as IntelliCorp
IntelliCorp (Software)
IntelliCorp is a software company that provides, develops, and markets SAP application lifecycle management, business process management and data management software for SAP customers and partners...
and Aion
Cleverpath AION Business Rules Expert
Cleverpath AION Business Rules Expert is an expert system and Business rules engine owned by Computer Associates by 2005....
.
The knowledge revolution
The power of expert systems came from the expert knowledge they contained. They were part of a new direction in AI research that had been gaining ground throughout the 70s. "AI researchers were beginning to suspect—reluctantly, for it violated the scientific canon of parsimony—that intelligence might very well be based on the ability to use large amounts of diverse knowledge in different ways," writes Pamela McCorduckPamela McCorduck
Pamela McCorduck is the author of a number of books concerning the history and philosophical significance of artificial intelligence, the future of engineering and the role of women and technology. She is also the author of three novels. She is a contributor to Omni, New York Times, Daedalus, the...
. "[T]he great lesson from the 1970s was that intelligent behavior depended very much on dealing with knowledge, sometimes quite detailed knowledge, of a domain where a given task lay". Knowledge based systems and knowledge engineering
Knowledge engineering
Knowledge engineering was defined in 1983 by Edward Feigenbaum, and Pamela McCorduck as follows:At present, it refers to the building, maintaining and development of knowledge-based systems...
became a major focus of AI research in the 1980s.
The 1980s also saw the birth of Cyc
Cyc
Cyc is an artificial intelligence project that attempts to assemble a comprehensive ontology and knowledge base of everyday common sense knowledge, with the goal of enabling AI applications to perform human-like reasoning....
, the first attempt to attack the commonsense knowledge problem
Commonsense reasoning
Commonsense reasoning is the branch of Artificial intelligence concerned with replicating human thinking. There are several components to this problem, including:* Developing adequately broad and deep commonsense knowledge bases....
directly, by creating a massive database that would contain all the mundane facts that the average person knows. Douglas Lenat
Douglas Lenat
Douglas B. Lenat is the CEO of Cycorp, Inc. of Austin, Texas, and has been a prominent researcher in artificial intelligence, especially machine learning , knowledge representation, blackboard systems, and "ontological engineering"...
, who started and led the project, argued that there is no shortcut ― the only way for machines to know the meaning of human concepts is to teach them, one concept at a time, by hand. The project was not expected to be completed for many decades.
The money returns: the fifth generation project
In 1981, the Japanese Ministry of International Trade and IndustryMinistry of International Trade and Industry
The Ministry of International Trade and Industry was one of the most powerful agencies of the Government of Japan. At the height of its influence, it effectively ran much of Japanese industrial policy, funding research and directing investment...
set aside $850 million dollars for the Fifth generation computer
Fifth generation computer
The Fifth Generation Computer Systems project was an initiative by Japan'sMinistry of International Trade and Industry, begun in 1982, to create a "fifth generation computer" which was supposed to perform much calculation using massive parallel processing...
project. Their objectives were to write programs and build machines that could carry on conversations, translate languages, interpret pictures, and reason like human beings. Much to the chagrin of scruffies
Neats vs. scruffies
Neat and scruffy are labels for two different types of artificial intelligence research. Neats consider that solutions should be elegant, clear and provably correct...
, they chose Prolog
Prolog
Prolog is a general purpose logic programming language associated with artificial intelligence and computational linguistics.Prolog has its roots in first-order logic, a formal logic, and unlike many other programming languages, Prolog is declarative: the program logic is expressed in terms of...
as the primary computer language for the project.
Other countries responded with new programs of their own. The UK began the ₤350 million Alvey
Alvey
The Alvey Programme was a British government sponsored research program in information technology that ran from 1983 to 1987. The program was a reaction to the Japanese Fifth generation computer project.Focus areas for the Alvey Programme included:...
project. A consortium of American companies formed the Microelectronics and Computer Technology Corporation
Microelectronics and Computer Technology Corporation
Microelectronics and Computer Technology Corporation was the first, and - at one time - one of the largest, computer industry research and development consortia in the United States....
(or "MCC") to fund large scale projects in AI and information technology. DARPA responded as well, founding the Strategic Computing Initiative
Strategic Computing Initiative
The United States government's Strategic Computing Initiative funded research into advanced computer hardware and artificial intelligence from 1983 to 1993...
and tripling its investment in AI between 1984 and 1988.
The revival of connectionism
In 1982, physicist John Hopfield was able to prove that a form of neural network (now called a "Hopfield netHopfield net
A Hopfield network is a form of recurrent artificial neural network invented by John Hopfield. Hopfield nets serve as content-addressable memory systems with binary threshold units. They are guaranteed to converge to a local minimum, but convergence to one of the stored patterns is not guaranteed...
") could learn and process information in a completely new way. Around the same time, David Rumelhart
David Rumelhart
David Everett Rumelhart was an American psychologist who made many contributions to the formal analysis of human cognition, working primarily within the frameworks of mathematical psychology, symbolic artificial intelligence, and parallel distributed processing...
popularized a new method for training neural networks called "backpropagation
Backpropagation
Backpropagation is a common method of teaching artificial neural networks how to perform a given task. Arthur E. Bryson and Yu-Chi Ho described it as a multi-stage dynamic system optimization method in 1969 . It wasn't until 1974 and later, when applied in the context of neural networks and...
" (discovered years earlier by Paul Werbos
Paul Werbos
Paul J. Werbos is a scientist best known for his 1974 Harvard University Ph.D. thesis, which first described the process of training artificial neural networks through backpropagation of errors. The thesis, and some supplementary information, can be found in his book, The Roots of Backpropagation...
). These two discoveries revived the field of connectionism
Connectionism
Connectionism is a set of approaches in the fields of artificial intelligence, cognitive psychology, cognitive science, neuroscience and philosophy of mind, that models mental or behavioral phenomena as the emergent processes of interconnected networks of simple units...
which had been largely abandoned since 1970.
The new field was unified and inspired by the appearance of Parallel Distributed Processing in 1986—a two volume collection of papers edited by Rumelhart
David Rumelhart
David Everett Rumelhart was an American psychologist who made many contributions to the formal analysis of human cognition, working primarily within the frameworks of mathematical psychology, symbolic artificial intelligence, and parallel distributed processing...
and psychologist James McClelland. Neural networks would become commercially successful in the 1990s, when they began to be used as the engines driving programs like optical character recognition
Optical character recognition
Optical character recognition, usually abbreviated to OCR, is the mechanical or electronic translation of scanned images of handwritten, typewritten or printed text into machine-encoded text. It is widely used to convert books and documents into electronic files, to computerize a record-keeping...
and speech recognition
Speech recognition
Speech recognition converts spoken words to text. The term "voice recognition" is sometimes used to refer to recognition systems that must be trained to a particular speaker—as is the case for most desktop recognition software...
.
Bust: the second AI winter 1987−1993
The business community's fascination with AI rose and fell in the 80s in the classic pattern of an economic bubbleEconomic bubble
An economic bubble is "trade in high volumes at prices that are considerably at variance with intrinsic values"...
. The collapse was in the perception of AI by government agencies and investors — the field continued to make advances despite the criticism. Rodney Brooks
Rodney Brooks
Rodney Allen Brooks is the former Panasonic professor of robotics at the Massachusetts Institute of Technology. Since 1986 he has authored a series of highly influential papers which have inaugurated a fundamental shift in artificial intelligence research...
and Hans Moravec
Hans Moravec
Hans Moravec is an adjunct faculty member at the Robotics Institute of Carnegie Mellon University. He is known for his work on robotics, artificial intelligence, and writings on the impact of technology. Moravec also is a futurist with many of his publications and predictions focusing on...
, researchers from the related field of robotics
Robotics
Robotics is the branch of technology that deals with the design, construction, operation, structural disposition, manufacture and application of robots...
, argued for an entirely new approach to artificial intelligence.
AI winter
The term "AI winterAI winter
In the history of artificial intelligence, an AI winter is a period of reduced funding and interest in artificial intelligence research. The process of hype, disappointment and funding cuts are common in many emerging technologies , but the problem has been particularly acute for AI...
" was coined by researchers who had survived the funding cuts of 1974 when they became concerned that enthusiasm for expert systems had spiraled out of control and that disappointment would certainly follow. Their fears were well founded: in the late 80s and early 90s, AI suffered a series of financial setbacks.
The first indication of a change in weather was the sudden collapse of the market for specialized AI hardware in 1987. Desktop computers from Apple
Apple Computer
Apple Inc. is an American multinational corporation that designs and markets consumer electronics, computer software, and personal computers. The company's best-known hardware products include the Macintosh line of computers, the iPod, the iPhone and the iPad...
and IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
had been steadily gaining speed and power and in 1987 they became more powerful than the more expensive Lisp machines
Lisp Machines
Lisp Machines, Inc. was a company formed in 1979 by Richard Greenblatt of MIT's Artificial Intelligence Laboratory to build Lisp machines. It was based in Cambridge, Massachusetts....
made by Symbolics
Symbolics
Symbolics refers to two companies: now-defunct computer manufacturer Symbolics, Inc., and a privately held company that acquired the assets of the former company and continues to sell and maintain the Open Genera Lisp system and the Macsyma computer algebra system.The symbolics.com domain was...
and others. There was no longer a good reason to buy them. An entire industry worth half a billion dollars was demolished overnight.
Eventually the earliest successful expert systems, such as XCON
Xcon
The R1 program was a production-rule-based system written in OPS5 by John P. McDermott of CMU in 1978 to assist in the ordering of DEC's VAX computer systems by automatically selecting the computer system components based on the customer's requirements...
, proved too expensive to maintain. They were difficult to update, they could not learn, they were "brittle" (i.e., they could make grotesque mistakes when given unusual inputs), and they fell prey to problems (such as the qualification problem
Qualification problem
In philosophy and AI , the qualification problem is concerned with the impossibility of listing all the preconditions required for a real-world action to have its intended effect. It might be posed as how to deal with the things that prevent me from achieving my intended result...
) that had been identified years earlier. Expert systems proved useful, but only in a few special contexts.
In the late 80s, the Strategic Computing Initiative
Strategic Computing Initiative
The United States government's Strategic Computing Initiative funded research into advanced computer hardware and artificial intelligence from 1983 to 1993...
cut funding to AI "deeply and brutally." New leadership at DARPA had decided that AI was not "the next wave" and directed funds towards projects that seemed more likely to produce immediate results.
By 1991, the impressive list of goals penned in 1981 for Japan's Fifth Generation Project
Fifth generation computer
The Fifth Generation Computer Systems project was an initiative by Japan'sMinistry of International Trade and Industry, begun in 1982, to create a "fifth generation computer" which was supposed to perform much calculation using massive parallel processing...
had not been met. Indeed, some of them, like "carry on a casual conversation" had not been met by 2010. As with other AI projects, expectations had run much higher than what was actually possible.
The importance of having a body: Nouvelle AI and embodied reason
In the late 80s, several researchers advocated a completely new approach to artificial intelligence, based on robotics. They believed that, to show real intelligence, a machine needs to have a body — it needs to perceive, move, survive and deal with the world. They argued that these sensorimotor skills are essential to higher level skills like commonsense reasoningCommonsense reasoning
Commonsense reasoning is the branch of Artificial intelligence concerned with replicating human thinking. There are several components to this problem, including:* Developing adequately broad and deep commonsense knowledge bases....
and that abstract reasoning was actually the least interesting or important human skill (see Moravec's paradox
Moravec's paradox
Moravec's paradox is the discovery by artificial intelligence and robotics researchers that, contrary to traditional assumptions, high-level reasoning requires very little computation, but low-level sensorimotor skills require enormous computational resources. The principle was articulated by Hans...
). They advocated building intelligence "from the bottom up."
The approach revived ideas from cybernetics and control theory
Control theory
Control theory is an interdisciplinary branch of engineering and mathematics that deals with the behavior of dynamical systems. The desired output of a system is called the reference...
that had been unpopular since the sixties. Another precursor was David Marr, who had come to MIT in the late 70s from a successful background in theoretical neuroscience to lead the group studying vision
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
. He rejected all symbolic approaches (both McCarthy's
John McCarthy (computer scientist)
John McCarthy was an American computer scientist and cognitive scientist. He coined the term "artificial intelligence" , invented the Lisp programming language and was highly influential in the early development of AI.McCarthy also influenced other areas of computing such as time sharing systems...
logic and Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
's frames), arguing that AI needed to understand the physical machinery of vision from the bottom up before any symbolic processing took place. (Marr's work would be cut short by leukemia in 1980.)
In a 1990 paper Elephants Don't Play Chess, robotics researcher Rodney Brooks
Rodney Brooks
Rodney Allen Brooks is the former Panasonic professor of robotics at the Massachusetts Institute of Technology. Since 1986 he has authored a series of highly influential papers which have inaugurated a fundamental shift in artificial intelligence research...
took direct aim at the physical symbol system hypothesis
Physical symbol system
A physical symbol system takes physical patterns , combining them into structures and manipulating them to produce new expressions....
, arguing that symbols are not always necessary since "the world is its own best model. It is always exactly up to date. It always has every detail there is to be known. The trick is to sense it appropriately and often enough." In the 80s and 90s, many cognitive scientists
Cognitive science
Cognitive science is the interdisciplinary scientific study of mind and its processes. It examines what cognition is, what it does and how it works. It includes research on how information is processed , represented, and transformed in behaviour, nervous system or machine...
also rejected the symbol processing model of the mind and argued that the body was essential for reasoning, a theory called the embodied mind thesis.
AI 1993−present
The field of AI, now more than a half a century old, finally achieved some of its oldest goals. It began to be used successfully throughout the technology industry, although somewhat behind the scenes. Some of the success was due to increasing computer power and some was achieved by focusing on specific isolated problems and pursuing them with the highest standards of scientific accountability. Still, the reputation of AI, in the business world at least, was less than pristine. Inside the field there was little agreement on the reasons for AI's failure to fulfill the dream of human level intelligence that had captured the imagination of the world in the 1960s. Together, all these factors helped to fragment AI into competing subfields focused on particular problems or approaches, sometimes even under new names that disguised the tarnished pedigree of "artificial intelligence". AI was both more cautious and more successful than it had ever been.Milestones and Moore's Law
On 11 May 1997, Deep Blue became the first computer chess-playing system to beat a reigning world chess champion, Garry KasparovGarry Kasparov
Garry Kimovich Kasparov is a Russian chess grandmaster, a former World Chess Champion, writer, political activist, and one of the greatest chess players of all time....
. In 2005, a Stanford robot won the DARPA Grand Challenge
DARPA Grand Challenge
The DARPA Grand Challenge is a prize competition for driverless vehicles, funded by the Defense Advanced Research Projects Agency, the most prominent research organization of the United States Department of Defense...
by driving autonomously for 131 miles along an unrehearsed desert trail.. Two years later, a team from CMU won the DARPA Urban Challenge by autonomously navigating 55 miles in an Urban environment while adhering to traffic hazards and all traffic laws. In February 2011, in a Jeopardy!
Jeopardy!
Griffin's first conception of the game used a board comprising ten categories with ten clues each, but after finding that this board could not be shown on camera easily, he reduced it to two rounds of thirty clues each, with five clues in each of six categories...
quiz show
Quiz Show
Quiz Show is a 1994 American historical drama film produced and directed by Robert Redford. Adapted by Paul Attanasio from Richard Goodwin's memoir Remembering America, the film is based upon the Twenty One quiz show scandal of the 1950s...
exhibition match, IBM
IBM
International Business Machines Corporation or IBM is an American multinational technology and consulting corporation headquartered in Armonk, New York, United States. IBM manufactures and sells computer hardware and software, and it offers infrastructure, hosting and consulting services in areas...
's question answering system, Watson
Watson (artificial intelligence software)
Watson is an artificial intelligence computer system capable of answering questions posed in natural language, developed in IBM's DeepQA project by a research team led by principal investigator David Ferrucci. Watson was named after IBM's first president, Thomas J...
, defeated the two greatest Jeopardy! champions, Brad Rutter
Brad Rutter
Bradford Gates "Brad" Rutter is the biggest all-time money winner on the U.S. syndicated game show Jeopardy! and the second biggest all-time money winner on a game show....
and Ken Jennings
Ken Jennings
Kenneth Wayne "Ken" Jennings III is an American game show contestant and author. Jennings is noted for holding the record for the longest winning streak on the U.S. syndicated game show Jeopardy! and as being the all-time leading money winner on American game shows...
, by a significant margin.
These successes were not due to some revolutionary new paradigm, but mostly on the tedious application of engineering skill and on the tremendous power of computers today. In fact, Deep Blue's computer was 10 million times faster than the Ferranti Mark 1 that Christopher Strachey
Christopher Strachey
Christopher Strachey was a British computer scientist. He was one of the founders of denotational semantics, and a pioneer in programming language design...
taught to play chess in 1951. This dramatic increase is measured by Moore's law
Moore's Law
Moore's law describes a long-term trend in the history of computing hardware: the number of transistors that can be placed inexpensively on an integrated circuit doubles approximately every two years....
, which predicts that the speed and memory capacity of computers doubles every two years. The fundamental problem of "raw computer power" was slowly being overcome.
Intelligent agents
A new paradigm called "intelligent agentIntelligent agent
In artificial intelligence, an intelligent agent is an autonomous entity which observes through sensors and acts upon an environment using actuators and directs its activity towards achieving goals . Intelligent agents may also learn or use knowledge to achieve their goals...
s" became widely accepted during the 90s. Although earlier researchers had proposed modular "divide and conquer" approaches to AI, the intelligent agent
Intelligent agent
In artificial intelligence, an intelligent agent is an autonomous entity which observes through sensors and acts upon an environment using actuators and directs its activity towards achieving goals . Intelligent agents may also learn or use knowledge to achieve their goals...
did not reach its modern form until Judea Pearl
Judea Pearl
Judea Pearl is a computer scientist and philosopher, best known for developing the probabilistic approach to artificial intelligence and the development of Bayesian networks ....
, Allen Newell
Allen Newell
Allen Newell was a researcher in computer science and cognitive psychology at the RAND corporation and at Carnegie Mellon University’s School of Computer Science, Tepper School of Business, and Department of Psychology...
and others brought concepts from decision theory
Decision theory
Decision theory in economics, psychology, philosophy, mathematics, and statistics is concerned with identifying the values, uncertainties and other issues relevant in a given decision, its rationality, and the resulting optimal decision...
and economics
Economics
Economics is the social science that analyzes the production, distribution, and consumption of goods and services. The term economics comes from the Ancient Greek from + , hence "rules of the house"...
into the study of AI. When the economist's
Economics
Economics is the social science that analyzes the production, distribution, and consumption of goods and services. The term economics comes from the Ancient Greek from + , hence "rules of the house"...
definition of a rational agent
Rational agent
In economics, game theory, decision theory, and artificial intelligence, a rational agent is an agent which has clear preferences, models uncertainty via expected values, and always chooses to perform the action that results in the optimal outcome for itself from among all feasible actions...
was married to computer science
Computer science
Computer science or computing science is the study of the theoretical foundations of information and computation and of practical techniques for their implementation and application in computer systems...
's definition of an object
Object-oriented programming
Object-oriented programming is a programming paradigm using "objects" – data structures consisting of data fields and methods together with their interactions – to design applications and computer programs. Programming techniques may include features such as data abstraction,...
or module, the intelligent agent
Intelligent agent
In artificial intelligence, an intelligent agent is an autonomous entity which observes through sensors and acts upon an environment using actuators and directs its activity towards achieving goals . Intelligent agents may also learn or use knowledge to achieve their goals...
paradigm was complete.
An intelligent agent
Intelligent agent
In artificial intelligence, an intelligent agent is an autonomous entity which observes through sensors and acts upon an environment using actuators and directs its activity towards achieving goals . Intelligent agents may also learn or use knowledge to achieve their goals...
is a system that perceives its environment and takes actions which maximize its chances of success. The simplest intelligent agents are programs that solve specific problems. The most complicated intelligent agents known are rational, thinking human beings. The intelligent agent paradigm
Intelligent agent
In artificial intelligence, an intelligent agent is an autonomous entity which observes through sensors and acts upon an environment using actuators and directs its activity towards achieving goals . Intelligent agents may also learn or use knowledge to achieve their goals...
defines AI research as "the study of intelligent agents". This is a generalization of some earlier definitions of AI: it goes beyond studying human intelligence; it studies all kinds of intelligence.
The paradigm gave researchers license to study isolated problems and find solutions that were both verifiable and useful. It provided a common language to describe problems and share their solutions with each other, and with other fields that also used concepts of abstract agents, like economics
Economics
Economics is the social science that analyzes the production, distribution, and consumption of goods and services. The term economics comes from the Ancient Greek from + , hence "rules of the house"...
and control theory
Control theory
Control theory is an interdisciplinary branch of engineering and mathematics that deals with the behavior of dynamical systems. The desired output of a system is called the reference...
. It was hoped that a complete agent architecture
Agent architecture
Agent architecture in computer science is a blueprint for software agents and intelligent control systems, depicting the arrangement of components...
(like Newell's
Allen Newell
Allen Newell was a researcher in computer science and cognitive psychology at the RAND corporation and at Carnegie Mellon University’s School of Computer Science, Tepper School of Business, and Department of Psychology...
SOAR
Soar (cognitive architecture)
Soar is a symbolic cognitive architecture, created by John Laird, Allen Newell, and Paul Rosenbloom at Carnegie Mellon University, now maintained by John Laird's research group at the University of Michigan. It is both a view of what cognition is and an implementation of that view through a...
) would one day allow researchers to build more versatile and intelligent systems out of interacting intelligent agents.
"Victory of the neats"
AI researchers began to develop and use sophisticated mathematical tools more than they ever had in the past. There was a widespread realization that many of the problems that AI needed to solve were already being worked on by researchers in fields like mathematicsMathematics
Mathematics is the study of quantity, space, structure, and change. Mathematicians seek out patterns and formulate new conjectures. Mathematicians resolve the truth or falsity of conjectures by mathematical proofs, which are arguments sufficient to convince other mathematicians of their validity...
, economics
Economics
Economics is the social science that analyzes the production, distribution, and consumption of goods and services. The term economics comes from the Ancient Greek from + , hence "rules of the house"...
or operations research
Operations research
Operations research is an interdisciplinary mathematical science that focuses on the effective use of technology by organizations...
. The shared mathematical language allowed both a higher level of collaboration with more established and successful fields and the achievement of results which were measurable and provable; AI had become a more rigorous "scientific" discipline. describe this as nothing less than a "revolution" and "the victory of the neats".
Judea Pearl
Judea Pearl
Judea Pearl is a computer scientist and philosopher, best known for developing the probabilistic approach to artificial intelligence and the development of Bayesian networks ....
's highly influential 1988 book brought probability
Probability
Probability is ordinarily used to describe an attitude of mind towards some proposition of whose truth we arenot certain. The proposition of interest is usually of the form "Will a specific event occur?" The attitude of mind is of the form "How certain are we that the event will occur?" The...
and decision theory
Decision theory
Decision theory in economics, psychology, philosophy, mathematics, and statistics is concerned with identifying the values, uncertainties and other issues relevant in a given decision, its rationality, and the resulting optimal decision...
into AI. Among the many new tools in use were Bayesian networks, hidden Markov models, information theory
Information theory
Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and...
, stochastic modeling and classical optimization
Optimization (mathematics)
In mathematics, computational science, or management science, mathematical optimization refers to the selection of a best element from some set of available alternatives....
. Precise mathematical descriptions were also developed for "computational intelligence
Computational intelligence
Computational intelligence is a set of Nature-inspired computational methodologies and approaches to address complex problems of the real world applications to which traditional methodologies and approaches are ineffective or infeasible. It primarily includes Fuzzy logic systems, Neural Networks...
" paradigms like neural networks
Neural Networks
Neural Networks is the official journal of the three oldest societies dedicated to research in neural networks: International Neural Network Society, European Neural Network Society and Japanese Neural Network Society, published by Elsevier...
and evolutionary algorithm
Evolutionary algorithm
In artificial intelligence, an evolutionary algorithm is a subset of evolutionary computation, a generic population-based metaheuristic optimization algorithm. An EA uses some mechanisms inspired by biological evolution: reproduction, mutation, recombination, and selection...
s.
AI behind the scenes
Algorithms originally developed by AI researchers began to appear as parts of larger systems. AI had solved a lot of very difficult problemsand their solutions proved to be useful throughout the technology industry, such as
data mining
Data mining
Data mining , a relatively young and interdisciplinary field of computer science is the process of discovering new patterns from large data sets involving methods at the intersection of artificial intelligence, machine learning, statistics and database systems...
,
industrial robotics,
logistics
Logistics
Logistics is the management of the flow of goods between the point of origin and the point of destination in order to meet the requirements of customers or corporations. Logistics involves the integration of information, transportation, inventory, warehousing, material handling, and packaging, and...
,
speech recognition
Speech recognition
Speech recognition converts spoken words to text. The term "voice recognition" is sometimes used to refer to recognition systems that must be trained to a particular speaker—as is the case for most desktop recognition software...
,
banking software,
medical diagnosis
and Google
Google
Google Inc. is an American multinational public corporation invested in Internet search, cloud computing, and advertising technologies. Google hosts and develops a number of Internet-based services and products, and generates profit primarily from advertising through its AdWords program...
's search engine.
The field of AI receives little or no credit for these successes. Many of AI's greatest innovations have been reduced to the status of just another item in the tool chest of computer science. Nick Bostrom
Nick Bostrom
Nick Bostrom is a Swedish philosopher at the University of Oxford known for his work on existential risk and the anthropic principle. He holds a PhD from the London School of Economics...
explains "A lot of cutting edge AI has filtered into general applications, often without being called AI because once something becomes useful enough and common enough it's not labeled AI anymore."
Many researchers in AI today deliberately call their work by other names, such as informatics
Informatics (academic field)
Informatics is the science of information, the practice of information processing, and the engineering of information systems. Informatics studies the structure, algorithms, behavior, and interactions of natural and artificial systems that store, process, access and communicate information...
, knowledge-based systems
Knowledge-based systems
Knowledge based systems are artificial intelligent tools working in a narrow domain to provide intelligent decisions with justification. Knowledge is acquired and represented using various knowledge representation techniques rules, frames and scripts...
, cognitive systems or computational intelligence
Computational intelligence
Computational intelligence is a set of Nature-inspired computational methodologies and approaches to address complex problems of the real world applications to which traditional methodologies and approaches are ineffective or infeasible. It primarily includes Fuzzy logic systems, Neural Networks...
. In part, this may be because they considered their field to be fundamentally different from AI, but also the new names help to procure funding. In the commercial world at least, the failed promises of the AI Winter
AI winter
In the history of artificial intelligence, an AI winter is a period of reduced funding and interest in artificial intelligence research. The process of hype, disappointment and funding cuts are common in many emerging technologies , but the problem has been particularly acute for AI...
continue to haunt AI research, as the New York Times reported in 2005: "Computer scientists and software engineers avoided the term artificial intelligence for fear of being viewed as wild-eyed dreamers."
Where is HAL 9000?
In 1968, Arthur C. ClarkeArthur C. Clarke
Sir Arthur Charles Clarke, CBE, FRAS was a British science fiction author, inventor, and futurist, famous for his short stories and novels, among them 2001: A Space Odyssey, and as a host and commentator in the British television series Mysterious World. For many years, Robert A. Heinlein,...
and Stanley Kubrick
Stanley Kubrick
Stanley Kubrick was an American film director, writer, producer, and photographer who lived in England during most of the last four decades of his career...
had imagined that by the year 2001, a machine would exist with an intelligence that matched or exceeded the capability of human beings. The character they created, HAL 9000
HAL 9000
HAL 9000 is the antagonist in Arthur C. Clarke's science fiction Space Odyssey saga. HAL is an artificial intelligence that interacts with the astronaut crew of the Discovery One spacecraft, usually represented as a red television-camera eye found throughout the ship...
, was based on hard science: many leading AI researchers also believed that such a machine would exist by the year 2001.
Marvin Minsky
Marvin Minsky
Marvin Lee Minsky is an American cognitive scientist in the field of artificial intelligence , co-founder of Massachusetts Institute of Technology's AI laboratory, and author of several texts on AI and philosophy.-Biography:...
asks "So the question is why didn't we get HAL in 2001?" Minsky believes that the answer is that the central problems, like commonsense reasoning
Commonsense reasoning
Commonsense reasoning is the branch of Artificial intelligence concerned with replicating human thinking. There are several components to this problem, including:* Developing adequately broad and deep commonsense knowledge bases....
, were being neglected, while most researchers pursued things like commercial applications of neural nets or genetic algorithms. John McCarthy
John McCarthy (computer scientist)
John McCarthy was an American computer scientist and cognitive scientist. He coined the term "artificial intelligence" , invented the Lisp programming language and was highly influential in the early development of AI.McCarthy also influenced other areas of computing such as time sharing systems...
, on the other hand, still blames the qualification problem
Qualification problem
In philosophy and AI , the qualification problem is concerned with the impossibility of listing all the preconditions required for a real-world action to have its intended effect. It might be posed as how to deal with the things that prevent me from achieving my intended result...
. For Ray Kurzweil, the issue is computer power and, using Moore's Law
Moore's Law
Moore's law describes a long-term trend in the history of computing hardware: the number of transistors that can be placed inexpensively on an integrated circuit doubles approximately every two years....
, he predicts that machines with human-level intelligence will appear by 2029. Jeff Hawkins
Jeff Hawkins
Jeffrey Hawkins is the founder of Palm Computing and Handspring...
argues that neural net research ignores the essential properties of the human cortex
Cerebral cortex
The cerebral cortex is a sheet of neural tissue that is outermost to the cerebrum of the mammalian brain. It plays a key role in memory, attention, perceptual awareness, thought, language, and consciousness. It is constituted of up to six horizontal layers, each of which has a different...
, preferring simple models that have been successful at solving simple problems. There are many other explanations and for each there is a corresponding research program underway.
See also
- History of natural language processingHistory of natural language processingThe history of natural language processing describes the advances of natural language processing. There is some overlap with the history of machine translation and the history of artificial intelligence.-Theoretical history:...