Viseme
Encyclopedia
A viseme is a representational unit used to classify speech sounds in the visual domain. The term viseme was introduced based on the interpretation of the phoneme
as a basic unit of speech in the acoustic/auditory domain, (Fisher, 1968). This is, however, at variance with the accepted definition of the phoneme
as the smallest structural unit that distinguishes meaning within a given language - as a cognitive abstraction that is not bound to any sensory modality.
A "viseme" describes the particular facial
and oral
positions and movements that occur alongside the voicing of phonemes. The analogous term for the acoustic reflection of a phoneme would be "audieme", but this is not in use.
Phonemes and visemes do not always share a one-to-one correspondence; often, several phonemes share the same viseme. In other words, several phonemes look the same on the face when produced, such as /k/, /ɡ/, /ŋ/, (viseme: /k/), or /t͡ʃ/, /ʃ/, /d͡ʒ/, /ʒ/ (viseme: /ch/). However, there could be differences in timing and duration during actual speech in terms of the visual 'signature' of a given gesture that can not be captured with a single photograph. Conversely, some sounds which are hard to distinguish acoustically are clearly distinguished by the face (Chen 2001). For example, acoustically speaking English /l/ and /r/ could be quite similar (especially in clusters, such as 'grass' vs. 'glass'). Yet visual information can show a clear contrast. This is demonstrated by the more frequent mishearing of words on the telephone than in person. Some linguists
have argued that speech is best understood as bimodal (aural and visual), and comprehension can be compromised if one of these two domains is absent (McGurk and MacDonald 1976
). The comprehension of speech by visemes alone is known as speechreading or "lip reading".
Applications for the study of visemes includes speech processing
, speech recognition
and computer facial animation
.
Phoneme
In a language or dialect, a phoneme is the smallest segmental unit of sound employed to form meaningful contrasts between utterances....
as a basic unit of speech in the acoustic/auditory domain, (Fisher, 1968). This is, however, at variance with the accepted definition of the phoneme
Phoneme
In a language or dialect, a phoneme is the smallest segmental unit of sound employed to form meaningful contrasts between utterances....
as the smallest structural unit that distinguishes meaning within a given language - as a cognitive abstraction that is not bound to any sensory modality.
A "viseme" describes the particular facial
Face
The face is a central sense organ complex, for those animals that have one, normally on the ventral surface of the head, and can, depending on the definition in the human case, include the hair, forehead, eyebrow, eyelashes, eyes, nose, ears, cheeks, mouth, lips, philtrum, temple, teeth, skin, and...
and oral
Mouth
The mouth is the first portion of the alimentary canal that receives food andsaliva. The oral mucosa is the mucous membrane epithelium lining the inside of the mouth....
positions and movements that occur alongside the voicing of phonemes. The analogous term for the acoustic reflection of a phoneme would be "audieme", but this is not in use.
Phonemes and visemes do not always share a one-to-one correspondence; often, several phonemes share the same viseme. In other words, several phonemes look the same on the face when produced, such as /k/, /ɡ/, /ŋ/, (viseme: /k/), or /t͡ʃ/, /ʃ/, /d͡ʒ/, /ʒ/ (viseme: /ch/). However, there could be differences in timing and duration during actual speech in terms of the visual 'signature' of a given gesture that can not be captured with a single photograph. Conversely, some sounds which are hard to distinguish acoustically are clearly distinguished by the face (Chen 2001). For example, acoustically speaking English /l/ and /r/ could be quite similar (especially in clusters, such as 'grass' vs. 'glass'). Yet visual information can show a clear contrast. This is demonstrated by the more frequent mishearing of words on the telephone than in person. Some linguists
Linguistics
Linguistics is the scientific study of human language. Linguistics can be broadly broken into three categories or subfields of study: language form, language meaning, and language in context....
have argued that speech is best understood as bimodal (aural and visual), and comprehension can be compromised if one of these two domains is absent (McGurk and MacDonald 1976
McGurk effect
The McGurk effect is a perceptual phenomenon which demonstrates an interaction between hearing and vision in speech perception. "It is a compelling illusion in which humans perceive mismatched audiovisual speech as a completely different syllable". The visual information a person gets from seeing a...
). The comprehension of speech by visemes alone is known as speechreading or "lip reading".
Applications for the study of visemes includes speech processing
Speech processing
Speech processing is the study of speech signals and the processing methods of these signals.The signals are usually processed in a digital representation, so speech processing can be regarded as a special case of digital signal processing, applied to speech signal.It is also closely tied to...
, speech recognition
Speech recognition
Speech recognition converts spoken words to text. The term "voice recognition" is sometimes used to refer to recognition systems that must be trained to a particular speaker—as is the case for most desktop recognition software...
and computer facial animation
Computer facial animation
Computer facial animation is primarily an area of computer graphics that encapsulates models and techniques for generating and animating images of the human head and face. Due to its subject and output type, it is also related to many other scientific and artistic fields from psychology to...
.