Imagined speech
Encyclopedia
Imagined speech is thinking
in the form of sound – “hearing” one’s own voice silently to oneself, without the intentional movement of any extremities such as the lips, tongue, or hands. Logically, imagined speech has been possible since the emergence of language, however, the phenomenon is most associated with the signal processing
and detection within electroencephalograph (EEG)
data as well as data obtained using alternative non-invasive, brain–computer interface (BCI) devices.
, with the intent of providing a foundation for synthetic telepathy. According to DARPA, the project “will allow user-to-user communication on the battlefield without the use of vocalized speech through neural signals analysis. The brain generates word-specific signals prior to sending electrical impulses to the vocal cords. These imagined speech signals would be analyzed and translated into distinct words allowing covert person-to-person communication.”
DARPA's program outline has three major goals:
than that of functional magnetic resonance imaging (fMRI’s), another commonly used non-invasive BCI.
s is used to translate raw data into the imagined speech content. Processing is also intended to occur in real-time—the information is processed as it is recorded, which allows for near-simultaneous viewing of the content as the subject imagines it.
Each session consisted of 20 trials per syllable, multiplied by 6 sessions for a total of 120 trials per syllable per subject.
This experiment resulted in a 98.96% ± 0.05% success rate (with 95% confidence
) in identifying the imagined speech content.
, demonstrates a significant challenge in analyzing neural signals of this type.
Another unique challenge is a relatively low signal-to-noise ratio (SNR)
in the recorded data. A SNR represents the amount of meaningful signals found in a data set, compared to the amount of arbitrary or useless signals present in the same set. Artifacts present in EEG data are just one of many significant sources of noise.
To further complicate matters, the relative placement of EEG electrodes will vary amongst subjects. This is because the anatomical details of people's heads will differ; therefore, the signals recorded will vary in each subject, regardless of individuals-specific imagined speech characteristics.
)
In addition, current detection methods cannot distinguish between more than two signals (i.e. /ba/ or /ku/, yes or no). Therefore, a significant advancement in EEG processing algorithms is still required. This may suggest that an overall understanding of human-information-processing patterns must be better understood first, as it would offer insight into classifying word-specific neural-patterns common to all people.
Thought
"Thought" generally refers to any mental or intellectual activity involving an individual's subjective consciousness. It can refer either to the act of thinking or the resulting ideas or arrangements of ideas. Similar concepts include cognition, sentience, consciousness, and imagination...
in the form of sound – “hearing” one’s own voice silently to oneself, without the intentional movement of any extremities such as the lips, tongue, or hands. Logically, imagined speech has been possible since the emergence of language, however, the phenomenon is most associated with the signal processing
Signal processing
Signal processing is an area of systems engineering, electrical engineering and applied mathematics that deals with operations on or analysis of signals, in either discrete or continuous time...
and detection within electroencephalograph (EEG)
EEG
EEG commonly refers to electroencephalography, a measurement of the electrical activity of the brain.EEG may also refer to:* Emperor Entertainment Group, a Hong Kong-based entertainment company...
data as well as data obtained using alternative non-invasive, brain–computer interface (BCI) devices.
History
In 2008, the US Defense Advanced Research Projects Agency (DARPA) provided a $4 million grant to the University of California (Irvine)University of California, Irvine
The University of California, Irvine , founded in 1965, is one of the ten campuses of the University of California, located in Irvine, California, USA...
, with the intent of providing a foundation for synthetic telepathy. According to DARPA, the project “will allow user-to-user communication on the battlefield without the use of vocalized speech through neural signals analysis. The brain generates word-specific signals prior to sending electrical impulses to the vocal cords. These imagined speech signals would be analyzed and translated into distinct words allowing covert person-to-person communication.”
DARPA's program outline has three major goals:
- To attempt to identify EEG patterns unique to individual words
- To ensure these patterns are common to different users to avoid extensive device training
- To construct a prototypePrototypeA prototype is an early sample or model built to test a concept or process or to act as a thing to be replicated or learned from.The word prototype derives from the Greek πρωτότυπον , "primitive form", neutral of πρωτότυπος , "original, primitive", from πρῶτος , "first" and τύπος ,...
that would decode the signals and transmit them over a limited range
Methods for detection
The process for analyzing subjects' silent speech is composed of recording subjects’ brain waves, and then using a computer to process the data and determine the content of the subjects' covert speech.Recording
Subject neural patterns (brain waves) can be recorded using BCI devices; currently, use of non-invasive devices, specifically the EEG, is of greater interest to researchers than invasive and partially invasive types. This is because non-invasive types, pose the least risk to subject health; EEG's have attracted the greatest interest because they offer the most user-friendly approach in addition to having far less complex instrumentationInstrumentation
Instrumentation is defined as the art and science of measurement and control of process variables within a production, or manufacturing area....
than that of functional magnetic resonance imaging (fMRI’s), another commonly used non-invasive BCI.
Processing
The first step in processing non-invasive data is to remove artifacts such as eye movement and blinking, as well as well as other electromyographic activity. After artifact-removal, a series of algorithmAlgorithm
In mathematics and computer science, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function. Algorithms are used for calculation, data processing, and automated reasoning...
s is used to translate raw data into the imagined speech content. Processing is also intended to occur in real-time—the information is processed as it is recorded, which allows for near-simultaneous viewing of the content as the subject imagines it.
Scientific trials and results
In one experiment performed at the University of California (Irvine), EEG data was collected while subjects imagined the syllables /ba/ and /ku/. The syllables were imagined once every 1.5 seconds over a 6 second period, resulting in three imagined syllables per trial.Each session consisted of 20 trials per syllable, multiplied by 6 sessions for a total of 120 trials per syllable per subject.
This experiment resulted in a 98.96% ± 0.05% success rate (with 95% confidence
Confidence interval
In statistics, a confidence interval is a particular kind of interval estimate of a population parameter and is used to indicate the reliability of an estimate. It is an observed interval , in principle different from sample to sample, that frequently includes the parameter of interest, if the...
) in identifying the imagined speech content.
Challenges
In detection of other imagined actions, such as imagined physical movements, greater brain activity occurs in one hemisphere over the other. This presence of asymmetrical activity acts as a major aid in identifying the subject's imagined action. In imagined speech detection however, equal levels of activity commonly occur in both the left and right hemispheres simultaneously. This lack of lateralizationLateralization of brain function
A longitudinal fissure separates the human brain into two distinct cerebral hemispheres, connected by the corpus callosum. The sides resemble each other and each hemisphere's structure is generally mirrored by the other side. Yet despite the strong anatomical similarities, the functions of each...
, demonstrates a significant challenge in analyzing neural signals of this type.
Another unique challenge is a relatively low signal-to-noise ratio (SNR)
Signal-to-noise ratio
Signal-to-noise ratio is a measure used in science and engineering that compares the level of a desired signal to the level of background noise. It is defined as the ratio of signal power to the noise power. A ratio higher than 1:1 indicates more signal than noise...
in the recorded data. A SNR represents the amount of meaningful signals found in a data set, compared to the amount of arbitrary or useless signals present in the same set. Artifacts present in EEG data are just one of many significant sources of noise.
To further complicate matters, the relative placement of EEG electrodes will vary amongst subjects. This is because the anatomical details of people's heads will differ; therefore, the signals recorded will vary in each subject, regardless of individuals-specific imagined speech characteristics.
Limitations for practical communication
Foremost, EEG use requires meticulously securing electrodes onto a subject’s head; the electrodes are connected through a web of wires tethered to a CPU. So, creating an everyday, user-friendly communicator requires a further development of compacting EEGs and their signal-processors into an easy-to-use, lightweight, and fashionable device. (I.e. a headband with Wi-FiWi-Fi
Wi-Fi or Wifi, is a mechanism for wirelessly connecting electronic devices. A device enabled with Wi-Fi, such as a personal computer, video game console, smartphone, or digital audio player, can connect to the Internet via a wireless network access point. An access point has a range of about 20...
)
In addition, current detection methods cannot distinguish between more than two signals (i.e. /ba/ or /ku/, yes or no). Therefore, a significant advancement in EEG processing algorithms is still required. This may suggest that an overall understanding of human-information-processing patterns must be better understood first, as it would offer insight into classifying word-specific neural-patterns common to all people.