Structural information theory
Encyclopedia
Structural information theory (SIT) is a theory about human perception
and in particular about perceptual organization: the way the human visual system
organizes a raw visual stimulus into objects and object parts. SIT was initiated, in the 1960s, by Emanuel Leeuwenberg and has been developed further by Hans Buffart, Peter van der Helm, and Rob van Lier. It has been applied to a wide range of research topics, mostly in visual form perception but also in, for instance, visual ergonomics, data visualization
, and music perception
.
SIT began as a quantitative model of visual pattern classification. Nowadays, it also includes quantitative models of symmetry perception
and amodal completion
, and it is theoretically founded in formalizations of visual regularity and viewpoint dependency. SIT has been argued to be the best defined and most successful extension of Gestalt ideas
. It is the only Gestalt approach providing a formal calculus
that generates plausible perceptual interpretations.
. A simplest code is a code with minimum information load, that is, a code that enables a reconstruction of the stimulus using a minimum number of descriptive parameters. Such a code is obtained by capturing a maximum amount of visual regularity and yields a hierarchical organization of the stimulus in terms of wholes and parts.
The assumption that the visual system prefers simplest interpretations is called the simplicity principle. Historically, the simplicity principle is an information-theoretical descendant of the Gestalt law of Prägnanz, which was based on the natural tendency of physical systems to settle into stable minimum-energy states. Furthermore, just as the later-proposed minimum description length principle
in algorithmic information theory
(AIT), it can be seen as a formalization of Occam's Razor
in which the best hypothesis for a given set of data is the one that leads to the largest compression of the data.
which had been developed in communication theory. In Shannon's approach, things are assigned codes with lengths based on their probability in terms of frequencies of occurrence (as, e.g., in the Morse code
). In many domains, including perception, such probabilities are hardly quantifiable if at all, however. Both SIT and AIT circumvent this problem by turning to descriptive complexities of individual things.
Although SIT and AIT share many starting points and objectives, there are also several relevant differences:
Crucial to the latter finding is the distinction between, and integration of, viewpoint-independent and viewpoint-dependent factors in vision, as proposed in SIT's empirically successful model of amodal completion. In the Bayesian framework, these factors correspond to prior probabilities and conditional probabilities, respectively. In SIT's model, however, both factors are quantified in terms of complexities, that is, complexities of objects and spatial relationships, respectively. This approach is consistent with neuroscientific
ideas about the distinction and interaction between the ventral ("what") and dorsal ("where") streams in the brain.
can be seen as something in between, that is, it flirts with DST when it comes to the usage of differential equations and it flirts with theories like SIT when it comes to the representation of information. In fact, the analyses provided by SIT, connectionism, and DST, correspond to what Marr called the computational, the algorithmic, and the implementational levels of description, respectively. According to Marr, such analyses are complementary rather than opposite.
What SIT, connectionism, and DST have in common is that they describe nonlinear system behavior, that is, a minor change in the input may yield a major change in the output. Their complementarity expresses itself in that they focus on different aspects:
, in which identical symbols refer to identical perceptual primitives (e.g., blobs or edges). Every substring of such a string represents a spatially contiguous part of an interpretation, so that the entire string can be read as a reconstruction recipe for the interpretation and, thereby, for the stimulus. These strings then are encoded (i.e., they are searched for visual regularities) to find the interpretation with the simplest code.
In SIT's formal coding model, this encoding is modelled by way of symbol manipulation. In psychology, this has led to critical statements of the sort of "SIT assumes that the brain performs symbol manipulation". Such statements, however, fall in the same category as statements such as "physics assumes that nature applies formulas such as Einstein's
E=mc2 or Newton's
F=ma" and "DST models assume that dynamic systems apply differential equations". That is, these statements ignore that the very concept of formalization
means that things are represented by symbols and that relationships between these things are captured by formulas or, in the case of SIT, by simplest codes.
A crucial difference with respect to the traditional, so-called transformational, formalization of visual regularity is that, holographically, mirror symmetry is composed of many relationships between symmetry pairs rather than one relationship between symmetry halfs. Whereas the transformational characterization may be better suited for object recognition
, the holographic characterization seems more consistent with the build up of mental representations in object perception.
The perceptual relevance of the criteria of holography and transparency has been verified in the so-called holographic approach to visual regularity. This approach provides an empirically successful model of the detectability of single and combined visual regularities, whether or not perturbed by noise. Furthermore, the transparent holographic regularities have been shown to lend themselves for transparallel processing
which means that, in the process of selecting a simplest code from among all possible codes, O(2N) codes can be taken into account as if only one code of length N were concerned. This supports the computational tractability
of simplest codes and, thereby, the feasibility of the simplicity principle in perceptual organization.
Perception
Perception is the process of attaining awareness or understanding of the environment by organizing and interpreting sensory information. All perception involves signals in the nervous system, which in turn result from physical stimulation of the sense organs...
and in particular about perceptual organization: the way the human visual system
Visual system
The visual system is the part of the central nervous system which enables organisms to process visual detail, as well as enabling several non-image forming photoresponse functions. It interprets information from visible light to build a representation of the surrounding world...
organizes a raw visual stimulus into objects and object parts. SIT was initiated, in the 1960s, by Emanuel Leeuwenberg and has been developed further by Hans Buffart, Peter van der Helm, and Rob van Lier. It has been applied to a wide range of research topics, mostly in visual form perception but also in, for instance, visual ergonomics, data visualization
Data visualization
Data visualization is the study of the visual representation of data, meaning "information that has been abstracted in some schematic form, including attributes or variables for the units of information"....
, and music perception
Music Perception
Music Perception is a music journal published by University of California Press, in Berkeley, California. Published five times a year, Music Perception publishes empirical and theoretical papers on such topics as psychology, psychophysics, linguistics, neurology, neurophysiology, artificial...
.
SIT began as a quantitative model of visual pattern classification. Nowadays, it also includes quantitative models of symmetry perception
Symmetry
Symmetry generally conveys two primary meanings. The first is an imprecise sense of harmonious or aesthetically pleasing proportionality and balance; such that it reflects beauty or perfection...
and amodal completion
Amodal perception
Amodal perception is the term used to describe the perception of the whole of a physical structure when only parts of it affect the sensory receptors...
, and it is theoretically founded in formalizations of visual regularity and viewpoint dependency. SIT has been argued to be the best defined and most successful extension of Gestalt ideas
Gestalt psychology
Gestalt psychology or gestaltism is a theory of mind and brain of the Berlin School; the operational principle of gestalt psychology is that the brain is holistic, parallel, and analog, with self-organizing tendencies...
. It is the only Gestalt approach providing a formal calculus
Calculus
Calculus is a branch of mathematics focused on limits, functions, derivatives, integrals, and infinite series. This subject constitutes a major part of modern mathematics education. It has two major branches, differential calculus and integral calculus, which are related by the fundamental theorem...
that generates plausible perceptual interpretations.
The simplicity principle
Although visual stimuli are fundamentally multi-interpretable, the human visual system usually has a clear preference for only one interpretation. To explain this preference, SIT introduced a formal coding model starting from the assumption that the perceptually preferred interpretation of a stimulus is the one with the simplest codeSimplicity
Simplicity is the state or quality of being simple. It usually relates to the burden which a thing puts on someone trying to explain or understand it. Something which is easy to understand or explain is simple, in contrast to something complicated...
. A simplest code is a code with minimum information load, that is, a code that enables a reconstruction of the stimulus using a minimum number of descriptive parameters. Such a code is obtained by capturing a maximum amount of visual regularity and yields a hierarchical organization of the stimulus in terms of wholes and parts.
The assumption that the visual system prefers simplest interpretations is called the simplicity principle. Historically, the simplicity principle is an information-theoretical descendant of the Gestalt law of Prägnanz, which was based on the natural tendency of physical systems to settle into stable minimum-energy states. Furthermore, just as the later-proposed minimum description length principle
Minimum description length
The minimum description length principle is a formalization of Occam's Razor in which the best hypothesis for a given set of data is the one that leads to the best compression of the data. MDL was introduced by Jorma Rissanen in 1978...
in algorithmic information theory
Algorithmic information theory
Algorithmic information theory is a subfield of information theory and computer science that concerns itself with the relationship between computation and information...
(AIT), it can be seen as a formalization of Occam's Razor
Occam's razor
Occam's razor, also known as Ockham's razor, and sometimes expressed in Latin as lex parsimoniae , is a principle that generally recommends from among competing hypotheses selecting the one that makes the fewest new assumptions.-Overview:The principle is often summarized as "simpler explanations...
in which the best hypothesis for a given set of data is the one that leads to the largest compression of the data.
Structural versus algorithmic information theory
Since the 1960s, SIT (in psychology) and AIT (in computer science) evolved independently as viable alternatives for Shannon's classical information theoryInformation theory
Information theory is a branch of applied mathematics and electrical engineering involving the quantification of information. Information theory was developed by Claude E. Shannon to find fundamental limits on signal processing operations such as compressing data and on reliably storing and...
which had been developed in communication theory. In Shannon's approach, things are assigned codes with lengths based on their probability in terms of frequencies of occurrence (as, e.g., in the Morse code
Morse code
Morse code is a method of transmitting textual information as a series of on-off tones, lights, or clicks that can be directly understood by a skilled listener or observer without special equipment...
). In many domains, including perception, such probabilities are hardly quantifiable if at all, however. Both SIT and AIT circumvent this problem by turning to descriptive complexities of individual things.
Although SIT and AIT share many starting points and objectives, there are also several relevant differences:
- First, SIT makes the perceptually relevant distinction between structural and metrical information, whereas AIT does not;
- Second, SIT encodes for a restricted set of perceptually relevant kinds of regularities, whereas AIT encodes for any imaginable regularity;
- Third, in SIT, the relevant outcome of an encoding is a hierarchical organization, whereas in AIT, it is a complexity value.
Simplicity versus likelihood
In visual perception research, the simplicity principle contrasts with the Helmholtzian likelihood principle, which assumes that the preferred interpretation of a stimulus is the one with the highest probability of being correct in this world. As shown within a Bayesian framework using AIT findings, the simplicity principle would imply that perceptual interpretations are fairly veridical (i.e., truthful) in many worlds rather than, as assumed by the likelihood principle, highly veridical in only one world. In other words, whereas the likelihood principle suggests that the visual system is a special-purpose system (i.e., dedicated to one world), the simplicity principle suggests that it is a general-purpose system (i.e., suited in many worlds).Crucial to the latter finding is the distinction between, and integration of, viewpoint-independent and viewpoint-dependent factors in vision, as proposed in SIT's empirically successful model of amodal completion. In the Bayesian framework, these factors correspond to prior probabilities and conditional probabilities, respectively. In SIT's model, however, both factors are quantified in terms of complexities, that is, complexities of objects and spatial relationships, respectively. This approach is consistent with neuroscientific
Cognitive neuroscience
Cognitive neuroscience is an academic field concerned with the scientific study of biological substrates underlying cognition, with a specific focus on the neural substrates of mental processes. It addresses the questions of how psychological/cognitive functions are produced by the brain...
ideas about the distinction and interaction between the ventral ("what") and dorsal ("where") streams in the brain.
SIT versus connectionism and dynamic systems theory
On the one hand, a representational theory like SIT seems opposite to dynamic systems theory (DST). On the other hand, connectionismConnectionism
Connectionism is a set of approaches in the fields of artificial intelligence, cognitive psychology, cognitive science, neuroscience and philosophy of mind, that models mental or behavioral phenomena as the emergent processes of interconnected networks of simple units...
can be seen as something in between, that is, it flirts with DST when it comes to the usage of differential equations and it flirts with theories like SIT when it comes to the representation of information. In fact, the analyses provided by SIT, connectionism, and DST, correspond to what Marr called the computational, the algorithmic, and the implementational levels of description, respectively. According to Marr, such analyses are complementary rather than opposite.
What SIT, connectionism, and DST have in common is that they describe nonlinear system behavior, that is, a minor change in the input may yield a major change in the output. Their complementarity expresses itself in that they focus on different aspects:
- First, DST focuses primarily on how the state of a physical system as a whole (in this case, the brainBrainThe brain is the center of the nervous system in all vertebrate and most invertebrate animals—only a few primitive invertebrates such as sponges, jellyfish, sea squirts and starfishes do not have one. It is located in the head, usually close to primary sensory apparatus such as vision, hearing,...
) develops over time, whereas both SIT and connectionism focus primarily on what a system does in terms of information processing; according to both SIT and connectionism, this information processing (which, in this case, can be said to constitute cognitionCognitive scienceCognitive science is the interdisciplinary scientific study of mind and its processes. It examines what cognition is, what it does and how it works. It includes research on how information is processed , represented, and transformed in behaviour, nervous system or machine...
) thrives on interactions between bits of information. - Second, regarding these interactions between bits of information, connectionism focuses primarily on the nature of concrete interaction mechanisms (assuming existing bits of information suited for any input), whereas SIT focuses primarily on the nature of the (assumed to be transient, i.e., input-dependent) bits of information involved and on the nature of the outcome of the interaction between them (modelling the interaction itself in a more abstract way).
Modelling principles
In SIT, candidate interpretations of a stimulus are represented by symbol stringsString (computer science)
In formal languages, which are used in mathematical logic and theoretical computer science, a string is a finite sequence of symbols that are chosen from a set or alphabet....
, in which identical symbols refer to identical perceptual primitives (e.g., blobs or edges). Every substring of such a string represents a spatially contiguous part of an interpretation, so that the entire string can be read as a reconstruction recipe for the interpretation and, thereby, for the stimulus. These strings then are encoded (i.e., they are searched for visual regularities) to find the interpretation with the simplest code.
In SIT's formal coding model, this encoding is modelled by way of symbol manipulation. In psychology, this has led to critical statements of the sort of "SIT assumes that the brain performs symbol manipulation". Such statements, however, fall in the same category as statements such as "physics assumes that nature applies formulas such as Einstein's
Albert Einstein
Albert Einstein was a German-born theoretical physicist who developed the theory of general relativity, effecting a revolution in physics. For this achievement, Einstein is often regarded as the father of modern physics and one of the most prolific intellects in human history...
E=mc2 or Newton's
Isaac Newton
Sir Isaac Newton PRS was an English physicist, mathematician, astronomer, natural philosopher, alchemist, and theologian, who has been "considered by many to be the greatest and most influential scientist who ever lived."...
F=ma" and "DST models assume that dynamic systems apply differential equations". That is, these statements ignore that the very concept of formalization
Formalization
Formalization may refer to* formal system in formal logic* a process enhancing bureaucracy in sociology...
means that things are represented by symbols and that relationships between these things are captured by formulas or, in the case of SIT, by simplest codes.
Visual regularity
To obtain simplest codes, SIT applies coding rules that capture the kinds of regularity called iteration, symmetry, and alternation. These have been shown to be the only regularities that satisfy the formal accessibility criteria of- (a) being so-called holographic regularities that
- (b) allow for so-called hierarchically transparent codes.
A crucial difference with respect to the traditional, so-called transformational, formalization of visual regularity is that, holographically, mirror symmetry is composed of many relationships between symmetry pairs rather than one relationship between symmetry halfs. Whereas the transformational characterization may be better suited for object recognition
Object recognition
Object recognition in computer vision is the task of finding a given object in an image or video sequence. Humans recognize a multitude of objects in images with little effort, despite the fact that the image of the objects may vary somewhat in different view points, in many different sizes / scale...
, the holographic characterization seems more consistent with the build up of mental representations in object perception.
The perceptual relevance of the criteria of holography and transparency has been verified in the so-called holographic approach to visual regularity. This approach provides an empirically successful model of the detectability of single and combined visual regularities, whether or not perturbed by noise. Furthermore, the transparent holographic regularities have been shown to lend themselves for transparallel processing
Transparallel processing
Transparallel processing is a form of processing, in computing or otherwise, in which items are processed simultaneously by one processor.Transparallel processing complements the three forms of processing called:...
which means that, in the process of selecting a simplest code from among all possible codes, O(2N) codes can be taken into account as if only one code of length N were concerned. This supports the computational tractability
Computational complexity theory
Computational complexity theory is a branch of the theory of computation in theoretical computer science and mathematics that focuses on classifying computational problems according to their inherent difficulty, and relating those classes to each other...
of simplest codes and, thereby, the feasibility of the simplicity principle in perceptual organization.
See also
- Neural processing for individual categories of objectsNeural processing for individual categories of objectsDiscrete categories of objects such as faces, body parts, tools, animals and buildings have been associated with preferential activation in specialised areas of the cerebral cortex, leading to the suggestion that they may be produced separately in discrete neural regions.Several such regions have...
- Principles of groupingPrinciples of groupingThe Principles of grouping are a set of principles in psychology, first proposed by Gestalt psychologists to account for the observation that humans naturally perceive objects as organized patterns and objects...
- Theory of indispensable attributesTheory of indispensable attributesThe Theory of indispensable attributes is a theory in the context of perceptual organisation which asks for the functional units and elementary features that are relevant for a perceptual system in the constitution of perceptual objects...