Sonification
Encyclopedia
Sonification, a form of auditory display
, is the use of non-speech audio
to convey information
or perceptualize data. Auditory perception
has high temporal and pressure resolution, which opens up possibilities for it as an alternative or complement to visualization
techniques.
For example, the Geiger counter
plays a sonification to help users know how much radiation is present. The number and frequency of audio clicks are directly dependent on the radiation level in the immediate vicinity of the device.
Though many experiments with data sonification are explored at forums such as the International Community for Auditory Display
(ICAD), sonification faces many challenges to widespread use for presenting and analyzing data. It is difficult to provide adequate context for interpreting sonifications of data. Also, there is not yet a flexible tool for sonification research and actual data exploration, meaning many sonification attempts are coded from scratch.
, amplitude
or tempo
, but could also be indicated by varying other less commonly used components. For example, a stock market price could be portrayed by rising pitch as the stock price rose, and lowering pitch as it fell. To allow the user to determine that more than one stock was being portrayed, different timbres or brightnesses might be used for the different stocks, or they may be played to the user from different points in space, for example, through different sides of their headphones.
Many studies have been undertaken to try to find the best techniques for various types of information to be presented, and as yet, no conclusive set of techniques to be used has been formulated. As the area of sonification is still considered to be in its infancy, current studies are working towards determining the best set of sound components to vary in different situations.
Several different techniques for rendering auditory data representations can be categorized:
Auditory display
Auditory display is the use of sound to communicate information from a computer to the user..The primary forum for exploring these techniques is the International Community for Auditory Display .-Types of auditory display:...
, is the use of non-speech audio
Sound
Sound is a mechanical wave that is an oscillation of pressure transmitted through a solid, liquid, or gas, composed of frequencies within the range of hearing and of a level sufficiently strong to be heard, or the sensation stimulated in organs of hearing by such vibrations.-Propagation of...
to convey information
Information
Information in its most restricted technical sense is a message or collection of messages that consists of an ordered sequence of symbols, or it is the meaning that can be interpreted from such a message or collection of messages. Information can be recorded or transmitted. It can be recorded as...
or perceptualize data. Auditory perception
Auditory system
The auditory system is the sensory system for the sense of hearing.- Outer ear :The folds of cartilage surrounding the ear canal are called the pinna...
has high temporal and pressure resolution, which opens up possibilities for it as an alternative or complement to visualization
Data visualization
Data visualization is the study of the visual representation of data, meaning "information that has been abstracted in some schematic form, including attributes or variables for the units of information"....
techniques.
For example, the Geiger counter
Geiger counter
A Geiger counter, also called a Geiger–Müller counter, is a type of particle detector that measures ionizing radiation. They detect the emission of nuclear radiation: alpha particles, beta particles or gamma rays. A Geiger counter detects radiation by ionization produced in a low-pressure gas in a...
plays a sonification to help users know how much radiation is present. The number and frequency of audio clicks are directly dependent on the radiation level in the immediate vicinity of the device.
Though many experiments with data sonification are explored at forums such as the International Community for Auditory Display
International Community for Auditory Display
The International Community for Auditory Display , founded in 1992, provides an annual conference for all research related to auditory display, the use of sound to display information. ICAD has given a home to auditory display researchers, who come from many different disciplines, through its...
(ICAD), sonification faces many challenges to widespread use for presenting and analyzing data. It is difficult to provide adequate context for interpreting sonifications of data. Also, there is not yet a flexible tool for sonification research and actual data exploration, meaning many sonification attempts are coded from scratch.
Some existing applications and projects
- auditory altimeterAltimeterAn altimeter is an instrument used to measure the altitude of an object above a fixed level. The measurement of altitude is called altimetry, which is related to the term bathymetry, the measurement of depth underwater.-Pressure altimeter:...
http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6VGB-3SWK140-7&_user=10&_rdoc=1&_fmt=&_orig=search&_sort=d&view=c&_acct=C000050221&_version=1&_urlVersion=0&_userid=10&md5=80b8dbced0d8112373fb2206a5aea704, also used in skydiving - auditory thermometer http://adsabs.harvard.edu/abs/1897AnP...299...66Q
- cockpit auditory displayAuditory displayAuditory display is the use of sound to communicate information from a computer to the user..The primary forum for exploring these techniques is the International Community for Auditory Display .-Types of auditory display:...
s - Geiger counterGeiger counterA Geiger counter, also called a Geiger–Müller counter, is a type of particle detector that measures ionizing radiation. They detect the emission of nuclear radiation: alpha particles, beta particles or gamma rays. A Geiger counter detects radiation by ionization produced in a low-pressure gas in a...
- interactive sonification
- medical auditory displayAuditory displayAuditory display is the use of sound to communicate information from a computer to the user..The primary forum for exploring these techniques is the International Community for Auditory Display .-Types of auditory display:...
s - multimodal (combined sense) displays to minimize visual overload and fatigue
- sonarSonarSonar is a technique that uses sound propagation to navigate, communicate with or detect other vessels...
- speed alarm in motor vehicles
- space physicsSpace physicsSpace physics, also known as space plasma physics, is the study of plasmas as they occur naturally in the universe. As such, it encompasses a far-ranging number of topics, including the sun, solar wind, planetary magnetospheres and ionospheres, auroras, cosmic rays, and synchrotron radiation...
http://spdf.gsfc.nasa.gov/research/sonification/sonification.html - storm and weather sonification http://www.icad.org/websiteV2.0/Conferences/ICAD2004/papers/polli.pdf
- volcanic activity detection
Sonification techniques
Many different components can be altered to change the user's perception of the sound, and in turn, their perception of the underlying information being portrayed. Often, an increase or decrease in some level in this information is indicated by an increase or decrease in pitchPitch (music)
Pitch is an auditory perceptual property that allows the ordering of sounds on a frequency-related scale.Pitches are compared as "higher" and "lower" in the sense associated with musical melodies,...
, amplitude
Amplitude
Amplitude is the magnitude of change in the oscillating variable with each oscillation within an oscillating system. For example, sound waves in air are oscillations in atmospheric pressure and their amplitudes are proportional to the change in pressure during one oscillation...
or tempo
Tempo
In musical terminology, tempo is the speed or pace of a given piece. Tempo is a crucial element of any musical composition, as it can affect the mood and difficulty of a piece.-Measuring tempo:...
, but could also be indicated by varying other less commonly used components. For example, a stock market price could be portrayed by rising pitch as the stock price rose, and lowering pitch as it fell. To allow the user to determine that more than one stock was being portrayed, different timbres or brightnesses might be used for the different stocks, or they may be played to the user from different points in space, for example, through different sides of their headphones.
Many studies have been undertaken to try to find the best techniques for various types of information to be presented, and as yet, no conclusive set of techniques to be used has been formulated. As the area of sonification is still considered to be in its infancy, current studies are working towards determining the best set of sound components to vary in different situations.
Several different techniques for rendering auditory data representations can be categorized:
- Model-Based Sonification
- Parameter Mapping Sonification
- Stream-Based Sonification
External links
- International Community for Auditory Display
- Using Sound to Extract Meaning from Complex Data, C. Scaletti and A. Craig, 1991.
- Preliminary report on design rationale, syntax, and semantics of LSL: A specification language for program auralization, D. Boardman and AP Mathur, 1993.
- A specification language for program auralization, D. Boardman, V. Khandelwal, and AP Mathur, 1994.
- Sonification Report (1997) provides an introduction to the status of the field and current research agendas.
- SonEnvir general sonification environment
- Sonification.de provides information about Sonification and Auditory Display, links to interesting event and related projects
- Auditory Information Design, PhD Thesis by Stephen Barrass 1998, User Centred Approach to Designing Sonifications.
- Sonification for Exploratory Data Analysis, PhD Thesis by Thomas Hermann 2002, developing Model Based Sonfication.
- Sonification of Mobile and Wireless Communications
- Interactive Sonification a hub to news and upcoming events in the field of interactive sonification
- zero-th space-time association
- CodeSounding — an open source sonification framework which makes possible to hear how any existing JavaJava (programming language)Java is a programming language originally developed by James Gosling at Sun Microsystems and released in 1995 as a core component of Sun Microsystems' Java platform. The language derives much of its syntax from C and C++ but has a simpler object model and fewer low-level facilities...
program "sounds like", by assigning instruments and pitches to code statements (if, for, etc.) and playing them as they are executed at runtime. In this way the flowing of execution is played as a flow of music and its rhythm changes depending on user interaction. - LYCAY, a Java library for sonification of Java source code
- WebMelody, a system for sonification of activity of web servers.
- Sonification of a Cantor setCantor setIn mathematics, the Cantor set is a set of points lying on a single line segment that has a number of remarkable and deep properties. It was discovered in 1875 by Henry John Stephen Smith and introduced by German mathematician Georg Cantor in 1883....
http://swiki.hfbk-hamburg.de:8888/MusicTechnology/799 - Volcanoes may reveal secrets through 'song' (New ScientistNew ScientistNew Scientist is a weekly non-peer-reviewed English-language international science magazine, which since 1996 has also run a website, covering recent developments in science and technology for a general audience. Founded in 1956, it is published by Reed Business Information Ltd, a subsidiary of...
) - Sound of science (New Scientist) discusses many applications of sonification
- Sonification Sandbox v.3.0, a Java program to convert datasets to sounds, GT Sonification Lab, School of Psychology, Georgia Institute of TechnologyGeorgia Institute of TechnologyThe Georgia Institute of Technology is a public research university in Atlanta, Georgia, in the United States...
. - Infrasound Laboratory of Hawaii (sounds)
- xSonify a Java application to display numerical data as sound, Goddard Space Flight CenterGoddard Space Flight CenterThe Goddard Space Flight Center is a major NASA space research laboratory established on May 1, 1959 as NASA's first space flight center. GSFC employs approximately 10,000 civil servants and contractors, and is located approximately northeast of Washington, D.C. in Greenbelt, Maryland, USA. GSFC,...
, NASANASAThe National Aeronautics and Space Administration is the agency of the United States government that is responsible for the nation's civilian space program and for aeronautics and aerospace research... - Program Sonification using Java, an online chapter (with code) explaining how to implement sonification using speech synthesis, MIDI note generation, and audio clips.
- Earthquake Sonification using USGS XML feed and sound synthesis software.
- New York City Taxi Real-Time Sonification using live video feeds from New York’s Traffic Management Center Max/MSP and Symbolic Sound's Kyma Software.