Cybernetic revolt
Encyclopedia
Cybernetic revolt or robot uprising is a scenario
in which an artificial intelligence
(either a single supercomputer
, a computer network, or sometimes a "race" of intelligent machines) decide that human
s (and/or organic non-humans
) are a threat (either to the machines or to themselves), are inferior, or are oppressors and try to destroy or to enslave them potentially leading to machine rule
. In this fictional scenario, humans are often depicted to prevail using "human" qualities, for example using emotion
s, illogic
, inefficiency
, duplicity
, unpredictability, or exploiting the supposedly rigid, rules-based thinking and lack of innovation of the computer's black/white mind.
's 1921 play R.U.R. (Rossum's Universal Robots)
. However, even as they were slowly being displaced from most physical tasks, humans have always prided themselves on their brains, taking the mechanistic 'thoughts' of early computers as proof that they would not be overtaken by their 'Frankenstein' creations
.
While artificial intelligence
, in terms of a replication of human intelligence, is still a remote concept, successes in certain parts of intelligence—as for example in the victories of the Deep Blue chess computer and the Watson
Jeopardy!
quiz show computer—have shaken previous certainty about a permanent place for humanity at the top of sapience.
has shown, computer power has (seemingly) limitless growth potential. While there are physical constraints to the speed at which modern microprocessors can function, scientists are already developing means that might eventually supersede these limits, such as quantum computers. As futurist and computer scientist
Raymond Kurzweil
has noted, "There are physical limits to computation, but they're not very limiting." If this process of growth continues, and existing problems in creating artificial intelligence are overcome, sentient machines are likely to immediately hold an enormous advantage in at least some forms of mental capability, including the capacity of perfect recall, a vastly superior knowledge base, and the ability to multitask
in ways not possible to biological entities. This may give them the opportunity to— either as a single being or as a new species
— become much more powerful than humans, and to displace them.
Such fears stem from a belief that competitiveness and aggression are necessary in any intelligent being's goal system. Such human competitiveness stems from the evolutionary background to our intelligence, where the survival and reproduction of genes in the face of human and non-human competitors was the central goal. In fact, an arbitrary intelligence could have arbitrary goals: there is no particular reason that an artificially-intelligent machine (not sharing humanity's evolutionary context) would be hostile—or friendly—unless its creator programs it to be such (and indeed military systems would be designed to be hostile, at least under certain circumstances). But the question remains: what would happen if AI systems could interact and evolve (evolution in this context means self-modification or selection and reproduction) and need to compete over resources, would that create goals of self-preservation? AI's goal of self-preservation could be in conflict with some goals of humans.
Some scientists dispute the likelihood of cybernetic revolts as depicted in science fiction such as The Matrix
, claiming that it is more likely that any artificial intelligences powerful enough to threaten humanity would probably be programmed not to attack it. This would not, however, protect against the possibility of a revolt initiated by terrorists, or by accident. Artificial General Intelligence researcher Eliezer Yudkowsky
has stated on this note that, probabilistically, humanity is less likely to be threatened by deliberately aggressive AIs than by AIs which were programmed such that their goals are unintentionally incompatible
with human survival or well-being (as in the film I, Robot
and in the short story "The Evitable Conflict
").
Another factor which may negate the likelihood of a cybernetic revolt is the vast difference between humans and AIs in terms of the resources necessary for survival. Humans require a "wet," organic, temperate, oxygen-laden environment while an AI might thrive essentially anywhere because their construction and energy needs would most likely be largely non-organic. With little or no competition for resources, conflict would perhaps be less likely no matter what sort of motivational architecture an artificial intelligence was given, especially provided with the superabundance of non-organic material resources in, for instance, the asteroid belt
. This, however, does not negate the possibility of a disinterested or unsympathetic AI artificially decomposing
all life on earth into mineral components for consumption or other purposes.
Other scientists point to the possibility of humans upgrading their capabilities with bionics
and/or genetic engineering
and, as cyborg
s, becoming the dominant species in themselves.
sentiences of this type would therefore be to humanity's great benefit. To this end, there has been much recent work in what has become known as Friendliness Theory, which holds that, as advocate and AI researcher Eliezer Yudkowsky
states, "... you ought to be able to reach into 'mind-design-space' (i.e. the hypothetical realm which contains all possible intelligent minds) and pull out a mind (design an intelligent machine) such that afterwards, you're glad you made it real."
Self-replicating machines:
"Smart" machines:
Scenario
A scenario is a synoptical collage of an event or series of actions and events. In the Commedia dell'arte it was an outline of entrances, exits, and action describing the plot of a play that was literally pinned to the back of the scenery...
in which an artificial intelligence
Artificial intelligence
Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...
(either a single supercomputer
Supercomputer
A supercomputer is a computer at the frontline of current processing capacity, particularly speed of calculation.Supercomputers are used for highly calculation-intensive tasks such as problems including quantum physics, weather forecasting, climate research, molecular modeling A supercomputer is a...
, a computer network, or sometimes a "race" of intelligent machines) decide that human
Human
Humans are the only living species in the Homo genus...
s (and/or organic non-humans
Extraterrestrial life
Extraterrestrial life is defined as life that does not originate from Earth...
) are a threat (either to the machines or to themselves), are inferior, or are oppressors and try to destroy or to enslave them potentially leading to machine rule
Machine Rule
The concept of machine rule is a common theme in science fiction stories and film, in which an artificially created lifeform takes over the naturally evolved beings that created them....
. In this fictional scenario, humans are often depicted to prevail using "human" qualities, for example using emotion
Emotion
Emotion is a complex psychophysiological experience of an individual's state of mind as interacting with biochemical and environmental influences. In humans, emotion fundamentally involves "physiological arousal, expressive behaviors, and conscious experience." Emotion is associated with mood,...
s, illogic
Logic
In philosophy, Logic is the formal systematic study of the principles of valid inference and correct reasoning. Logic is used in most intellectual activities, but is studied primarily in the disciplines of philosophy, mathematics, semantics, and computer science...
, inefficiency
Inefficiency
The term inefficiency has several meanings depending on the context in which its used:*Algorithmic inefficiency - refers to less than optimum computer programs that might exhibit one of more of the symptoms of:** slow execution...
, duplicity
Duplicity
Duplicity is a software suite that provides easy encrypted, digitally signed, versioned, remote backup of files requiring little of the remote server. It is capable of both full backups and incremental backups where only changes since the prior backup was made are stored...
, unpredictability, or exploiting the supposedly rigid, rules-based thinking and lack of innovation of the computer's black/white mind.
Relevance
Fear of humanity being made obsolete by technology taps into some of modern humans' deepest fears. This can be shown to have been the case even before the computer became prominent, as in Karel CapekKarel Capek
Karel Čapek was Czech writer of the 20th century.-Biography:Born in 1890 in the Bohemian mountain village of Malé Svatoňovice to an overbearing, emotional mother and a distant yet adored father, Čapek was the youngest of three siblings...
's 1921 play R.U.R. (Rossum's Universal Robots)
R.U.R. (Rossum's Universal Robots)
R.U.R. is a 1920 science fiction play in the Czech language by Karel Čapek. R.U.R. stands for Rossum's Universal Robots, an English phrase used as the subtitle in the Czech original. It premiered in 1921 and introduced the word "robot" to the English language and to science fiction as a whole.The...
. However, even as they were slowly being displaced from most physical tasks, humans have always prided themselves on their brains, taking the mechanistic 'thoughts' of early computers as proof that they would not be overtaken by their 'Frankenstein' creations
Frankenstein's monster
Frankenstein's monster is a fictional character that first appeared in Mary Shelley's novel, Frankenstein, or The Modern Prometheus. The creature is often erroneously referred to as "Frankenstein", but in the novel the creature has no name...
.
While artificial intelligence
Artificial intelligence
Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...
, in terms of a replication of human intelligence, is still a remote concept, successes in certain parts of intelligence—as for example in the victories of the Deep Blue chess computer and the Watson
Watson (artificial intelligence software)
Watson is an artificial intelligence computer system capable of answering questions posed in natural language, developed in IBM's DeepQA project by a research team led by principal investigator David Ferrucci. Watson was named after IBM's first president, Thomas J...
Jeopardy!
Jeopardy!
Griffin's first conception of the game used a board comprising ten categories with ten clues each, but after finding that this board could not be shown on camera easily, he reduced it to two rounds of thirty clues each, with five clues in each of six categories...
quiz show computer—have shaken previous certainty about a permanent place for humanity at the top of sapience.
Computing power
As Moore's lawMoore's Law
Moore's law describes a long-term trend in the history of computing hardware: the number of transistors that can be placed inexpensively on an integrated circuit doubles approximately every two years....
has shown, computer power has (seemingly) limitless growth potential. While there are physical constraints to the speed at which modern microprocessors can function, scientists are already developing means that might eventually supersede these limits, such as quantum computers. As futurist and computer scientist
Computer scientist
A computer scientist is a scientist who has acquired knowledge of computer science, the study of the theoretical foundations of information and computation and their application in computer systems....
Raymond Kurzweil
Raymond Kurzweil
Raymond "Ray" Kurzweil is an American author, inventor and futurist. He is involved in fields such as optical character recognition , text-to-speech synthesis, speech recognition technology, and electronic keyboard instruments...
has noted, "There are physical limits to computation, but they're not very limiting." If this process of growth continues, and existing problems in creating artificial intelligence are overcome, sentient machines are likely to immediately hold an enormous advantage in at least some forms of mental capability, including the capacity of perfect recall, a vastly superior knowledge base, and the ability to multitask
Human multitasking
Human multitasking is the best performance by an individual of appearing to handle more than one task at the same time. The term is derived from computer multitasking. An example of multitasking is taking phone calls while typing an email...
in ways not possible to biological entities. This may give them the opportunity to— either as a single being or as a new species
Species
In biology, a species is one of the basic units of biological classification and a taxonomic rank. A species is often defined as a group of organisms capable of interbreeding and producing fertile offspring. While in many cases this definition is adequate, more precise or differing measures are...
— become much more powerful than humans, and to displace them.
Necessity of conflict
For a cybernetic revolt to be inevitable, it has to be postulated that two intelligent species cannot pursue mutually the goals of coexisting peacefully in an overlapping environment—especially if one is of much more advanced intelligence and power. While a cybernetic revolt (where the machine is the more advanced species) is thus a possible outcome of machines gaining sentience and/or sapience, neither can it be disproven that a peaceful outcome is possible. The fear of a cybernetic revolt is often based on interpretations of humanity's history, which is rife with incidents of enslavement and genocide.Such fears stem from a belief that competitiveness and aggression are necessary in any intelligent being's goal system. Such human competitiveness stems from the evolutionary background to our intelligence, where the survival and reproduction of genes in the face of human and non-human competitors was the central goal. In fact, an arbitrary intelligence could have arbitrary goals: there is no particular reason that an artificially-intelligent machine (not sharing humanity's evolutionary context) would be hostile—or friendly—unless its creator programs it to be such (and indeed military systems would be designed to be hostile, at least under certain circumstances). But the question remains: what would happen if AI systems could interact and evolve (evolution in this context means self-modification or selection and reproduction) and need to compete over resources, would that create goals of self-preservation? AI's goal of self-preservation could be in conflict with some goals of humans.
Some scientists dispute the likelihood of cybernetic revolts as depicted in science fiction such as The Matrix
The Matrix
The Matrix is a 1999 science fiction-action film written and directed by Larry and Andy Wachowski, starring Keanu Reeves, Laurence Fishburne, Carrie-Anne Moss, Joe Pantoliano, and Hugo Weaving...
, claiming that it is more likely that any artificial intelligences powerful enough to threaten humanity would probably be programmed not to attack it. This would not, however, protect against the possibility of a revolt initiated by terrorists, or by accident. Artificial General Intelligence researcher Eliezer Yudkowsky
Eliezer Yudkowsky
Eliezer Shlomo Yudkowsky is an American artificial intelligence researcher concerned with the singularity and an advocate of friendly artificial intelligence, living in Redwood City, California.- Biography :...
has stated on this note that, probabilistically, humanity is less likely to be threatened by deliberately aggressive AIs than by AIs which were programmed such that their goals are unintentionally incompatible
Unintended consequence
In the social sciences, unintended consequences are outcomes that are not the outcomes intended by a purposeful action. The concept has long existed but was named and popularised in the 20th century by American sociologist Robert K. Merton...
with human survival or well-being (as in the film I, Robot
I, Robot (film)
I, Robot is a 2004 science-fiction action film directed by Alex Proyas. The screenplay was written by Jeff Vintar, Akiva Goldsman and Hillary Seitz, and is very loosely based on Isaac Asimov's short-story collection of the same name. Will Smith stars in the lead role of the film as Detective Del...
and in the short story "The Evitable Conflict
The Evitable Conflict
The Evitable Conflict is a science fiction short story by Isaac Asimov. It first appeared in the June 1950 issue of Astounding Science Fiction and subsequently appeared in the collections I, Robot , The Complete Robot , and Robot Visions .-Plot summary:The "Machines", powerful positronic computers...
").
Another factor which may negate the likelihood of a cybernetic revolt is the vast difference between humans and AIs in terms of the resources necessary for survival. Humans require a "wet," organic, temperate, oxygen-laden environment while an AI might thrive essentially anywhere because their construction and energy needs would most likely be largely non-organic. With little or no competition for resources, conflict would perhaps be less likely no matter what sort of motivational architecture an artificial intelligence was given, especially provided with the superabundance of non-organic material resources in, for instance, the asteroid belt
Asteroid belt
The asteroid belt is the region of the Solar System located roughly between the orbits of the planets Mars and Jupiter. It is occupied by numerous irregularly shaped bodies called asteroids or minor planets...
. This, however, does not negate the possibility of a disinterested or unsympathetic AI artificially decomposing
Decomposition
Decomposition is the process by which organic material is broken down into simpler forms of matter. The process is essential for recycling the finite matter that occupies physical space in the biome. Bodies of living organisms begin to decompose shortly after death...
all life on earth into mineral components for consumption or other purposes.
Other scientists point to the possibility of humans upgrading their capabilities with bionics
Bionics
Bionics is the application of biological methods and systems found in nature to the study and design of engineering systems and modern technology.The word bionic was coined by Jack E...
and/or genetic engineering
Genetic engineering
Genetic engineering, also called genetic modification, is the direct human manipulation of an organism's genome using modern DNA technology. It involves the introduction of foreign DNA or synthetic genes into the organism of interest...
and, as cyborg
Cyborg
A cyborg is a being with both biological and artificial parts. The term was coined in 1960 when Manfred Clynes and Nathan S. Kline used it in an article about the advantages of self-regulating human-machine systems in outer space. D. S...
s, becoming the dominant species in themselves.
Technological singularity
Some groups, called Singularitarians, who advocate what might be defined as a peaceful (non-violent, non-invasive, non-coercive) cybernetic revolt known as a 'technological singularity', argue that it is in humanity's best interests to bring about such an event, as long as it can be ensured that the event would be beneficial. They postulate that a society run by intelligent machines (or cyborgs) could potentially be vastly more efficient than a society run by normal human beings. A society led by friendly, altruisticAltruism
Altruism is a concern for the welfare of others. It is a traditional virtue in many cultures, and a core aspect of various religious traditions, though the concept of 'others' toward whom concern should be directed can vary among cultures and religions. Altruism is the opposite of...
sentiences of this type would therefore be to humanity's great benefit. To this end, there has been much recent work in what has become known as Friendliness Theory, which holds that, as advocate and AI researcher Eliezer Yudkowsky
Eliezer Yudkowsky
Eliezer Shlomo Yudkowsky is an American artificial intelligence researcher concerned with the singularity and an advocate of friendly artificial intelligence, living in Redwood City, California.- Biography :...
states, "... you ought to be able to reach into 'mind-design-space' (i.e. the hypothetical realm which contains all possible intelligent minds) and pull out a mind (design an intelligent machine) such that afterwards, you're glad you made it real."
In fiction
See also
- Technological singularityTechnological singularityTechnological singularity refers to the hypothetical future emergence of greater-than-human intelligence through technological means. Since the capabilities of such an intelligence would be difficult for an unaided human mind to comprehend, the occurrence of a technological singularity is seen as...
- Humans United Against RobotsHumans United Against RobotsHumans United Against Robots is a semi-tongue-in-cheek organization designed to unify the human race and prepare them for a futuristic robotic uprising...
Self-replicating machines:
- Clanking replicatorClanking replicatorA clanking replicator is an artificial self-replicating system that relies on conventional large-scale technology and automation. The term evolved to distinguish such systems from the microscopic "assemblers" that nanotechnology may make possible...
- Grey gooGrey gooGrey goo is a hypothetical end-of-the-world scenario involving molecular nanotechnology in which out-of-control self-replicating robots consume all matter on Earth while building more of themselves, a scenario known as ecophagy .Self-replicating machines of the macroscopic variety were originally...
- Self-replicationSelf-replicationSelf-replication is any behavior of a dynamical system that yields construction of an identical copy of that dynamical system. Biological cells, given suitable environments, reproduce by cell division. During cell division, DNA is replicated and can be transmitted to offspring during reproduction...
- Self-modifying codeSelf-modifying codeIn computer science, self-modifying code is code that alters its own instructions while it is executing - usually to reduce the instruction path length and improve performance or simply to reduce otherwise repetitively similar code, thus simplifying maintenance...
- Computer viruses
"Smart" machines:
- Biological computer
- Quantum computerQuantum computerA quantum computer is a device for computation that makes direct use of quantum mechanical phenomena, such as superposition and entanglement, to perform operations on data. Quantum computers are different from traditional computers based on transistors...
- RobotRobotA robot is a mechanical or virtual intelligent agent that can perform tasks automatically or with guidance, typically by remote control. In practice a robot is usually an electro-mechanical machine that is guided by computer and electronic programming. Robots can be autonomous, semi-autonomous or...
- Artificial intelligenceArtificial intelligenceArtificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...
External links
- The Singularity Institute for Artificial Intelligence (official institute website)
- ArmedRobots.com (tracks developments in robotics which may culminate in Cybernetic Revolt)