Interactive visualization
Encyclopedia
Interactive visualization is a branch of graphic visualization in computer science
that involves studying how humans interact with computers to create graphic illustrations of information and how this process can be made more efficient.
For a visualization to be considered interactive it must satisfy two criteria:
One particular type of interactive visualization is virtual reality
(VR), where the visual representation of information is presented using an immersive display device such as a stereo projector (see stereoscopy
). VR is also characterized by the use of a spatial metaphor, where some aspect of the information is represented in three dimensions so that humans can explore the information as if it were present (where instead it was remote), sized appropriately (where instead it was on a much smaller or larger scale than humans can sense directly), or had shape (where instead it might be completely abstract).
Another type of interactive visualization is collaborative visualization, in which multiple people interact with the same computer visualization to communicate their ideas to each other or to explore information cooperatively. Frequently, collaborative visualization is used when people are physically separated. Using several networked computers, the same visualization can be presented to each person simultaneously. The people then make annotations to the visualization as well as communicate via audio (i.e., telephone), video (i.e., a video-conference), or text (i.e., IRC) messages.
) was one of the first programmatic efforts at interactive visualization and provided an enumeration of the types of input humans provide. People can:
All of these actions require a physical device. Input devices range from the common – keyboards, mice
, graphics tablet
s, trackball
s, and touchpad
s – to the esoteric – wired glove
s, boom arms, and even omnidirectional treadmill
s.
These input actions can be used to control both the information being represented or the way that the information is presented. When the information being presented is altered, the visualization is usually part of a feedback loop. For example, consider an aircraft avionics system where the pilot inputs roll, pitch, and yaw and the visualization system provides a rendering of the aircraft's new attitude. Another example would be a scientist who changes a simulation while it is running in response to a visualization (see Visulation
) of its current progress. This is called computational steering.
More frequently, the representation of the information is changed rather than the information itself (see Visualization (graphic)
).
between when input is provided and a visual representation is updated is noticeable by most people. Thus it is desirable for an interactive visualization to provide a render
ing based on human input within this time frame. However, when large amounts of data must be processed to create a visualization, this becomes hard or even impossible with current technology. Thus the term “interactive visualization” is usually applied to systems that provide feedback to users within several seconds of input. The term interactive framerate is often used to measure how interactive a visualization is. Framerates measure the frequency with which an image (a frame) can be generated by a visualization system. A framerate of 50 frames per second (frame/s) is considered good while 0.1 frame/s would be considered poor. The use of framerates to characterize interactivity is slightly misleading however, since framerate is a measure of bandwidth
while humans are more sensitive to latency
. Specifically, it is possible to achieve a good framerate of 50 frame/s but if the images generated refer to changes to the visualization that a person made more than 1 second ago, it will not feel interactive to a person.
The rapid response time required for interactive visualization is a difficult constraint to meet and there are several approaches that have been explored to provide people with rapid visual feedback based on their input. Some include
and freely available. Some are commercial.
Computer science
Computer science or computing science is the study of the theoretical foundations of information and computation and of practical techniques for their implementation and application in computer systems...
that involves studying how humans interact with computers to create graphic illustrations of information and how this process can be made more efficient.
For a visualization to be considered interactive it must satisfy two criteria:
- Human input: control of some aspect of the visual representationKnowledge representationKnowledge representation is an area of artificial intelligence research aimed at representing knowledge in symbols to facilitate inferencing from those knowledge elements, creating new elements of knowledge...
of information, or of the information being represented, must be available to a human, and - Response time: changes made by the human must be incorporated into the visualization in a timely manner. In general, interactive visualization is considered a soft real-time task.
One particular type of interactive visualization is virtual reality
Virtual reality
Virtual reality , also known as virtuality, is a term that applies to computer-simulated environments that can simulate physical presence in places in the real world, as well as in imaginary worlds...
(VR), where the visual representation of information is presented using an immersive display device such as a stereo projector (see stereoscopy
Stereoscopy
Stereoscopy refers to a technique for creating or enhancing the illusion of depth in an image by presenting two offset images separately to the left and right eye of the viewer. Both of these 2-D offset images are then combined in the brain to give the perception of 3-D depth...
). VR is also characterized by the use of a spatial metaphor, where some aspect of the information is represented in three dimensions so that humans can explore the information as if it were present (where instead it was remote), sized appropriately (where instead it was on a much smaller or larger scale than humans can sense directly), or had shape (where instead it might be completely abstract).
Another type of interactive visualization is collaborative visualization, in which multiple people interact with the same computer visualization to communicate their ideas to each other or to explore information cooperatively. Frequently, collaborative visualization is used when people are physically separated. Using several networked computers, the same visualization can be presented to each person simultaneously. The people then make annotations to the visualization as well as communicate via audio (i.e., telephone), video (i.e., a video-conference), or text (i.e., IRC) messages.
Human control of visualization
The Programmer's Hierarchical Interactive Graphics System (PHIGSPHIGS
PHIGS is an API standard for rendering 3D computer graphics, at one time considered to be the 3D graphics standard for the 1990s. Instead a combination of features and power led to the rise of OpenGL, which became the most popular professional 3D API of the 1990s...
) was one of the first programmatic efforts at interactive visualization and provided an enumeration of the types of input humans provide. People can:
- Pick some part of an existing visual representation;
- Locate a point of interest (which may not have an existing representation);
- Stroke a path;
- Choose an option from a list of options;
- Valuate by inputting a number; and
- Write by inputting text.
All of these actions require a physical device. Input devices range from the common – keyboards, mice
Mouse (computing)
In computing, a mouse is a pointing device that functions by detecting two-dimensional motion relative to its supporting surface. Physically, a mouse consists of an object held under one of the user's hands, with one or more buttons...
, graphics tablet
Graphics tablet
A graphics tablet is a computer input device that enables a user to hand-draw images and graphics, similar to the way a person draws images with a pencil and paper. These tablets may also be used to capture data or handwritten signatures...
s, trackball
Trackball
A trackball is a pointing device consisting of a ball held by a socket containing sensors to detect a rotation of the ball about two axes—like an upside-down mouse with an exposed protruding ball. The user rolls the ball with the thumb, fingers, or the palm of the hand to move a cursor...
s, and touchpad
Touchpad
A touchpad is a pointing device featuring a tactile sensor, a specialized surface that can translate the motion and position of a user's fingers to a relative position on screen. Touch pads are a common feature of laptop computers, and they are also used as a substitute for a mouse where desk...
s – to the esoteric – wired glove
Wired glove
A wired glove is an input device for human–computer interaction worn like a glove.Various sensor technologies are used to capture physical data such as bending of fingers. Often a motion tracker, such as a magnetic tracking device or inertial tracking device, is attached to capture the global...
s, boom arms, and even omnidirectional treadmill
Omnidirectional treadmill
An omnidirectional treadmill, or ODT, is a device that allows a person to perform locomotive motion in any direction. The ability to move in any direction is how these treadmills differ from their basic counterparts...
s.
These input actions can be used to control both the information being represented or the way that the information is presented. When the information being presented is altered, the visualization is usually part of a feedback loop. For example, consider an aircraft avionics system where the pilot inputs roll, pitch, and yaw and the visualization system provides a rendering of the aircraft's new attitude. Another example would be a scientist who changes a simulation while it is running in response to a visualization (see Visulation
Visulation
Visulation is a portmanteau describing a coupled system where graphic visualization and computer simulation occur simultaneously. Visulation is conducted in many GPGPU applications and commonly employs interactive visualization techniques....
) of its current progress. This is called computational steering.
More frequently, the representation of the information is changed rather than the information itself (see Visualization (graphic)
Visualization (graphic)
Visualization is any technique for creating images, diagrams, or animations to communicate a message. Visualization through visual imagery has been an effective way to communicate both abstract and concrete ideas since the dawn of man...
).
Rapid response to human input
Experiments have shown that a delay of more than 20 msMillisecond
A millisecond is a thousandth of a second.10 milliseconds are called a centisecond....
between when input is provided and a visual representation is updated is noticeable by most people. Thus it is desirable for an interactive visualization to provide a render
Rendering (computer graphics)
Rendering is the process of generating an image from a model , by means of computer programs. A scene file contains objects in a strictly defined language or data structure; it would contain geometry, viewpoint, texture, lighting, and shading information as a description of the virtual scene...
ing based on human input within this time frame. However, when large amounts of data must be processed to create a visualization, this becomes hard or even impossible with current technology. Thus the term “interactive visualization” is usually applied to systems that provide feedback to users within several seconds of input. The term interactive framerate is often used to measure how interactive a visualization is. Framerates measure the frequency with which an image (a frame) can be generated by a visualization system. A framerate of 50 frames per second (frame/s) is considered good while 0.1 frame/s would be considered poor. The use of framerates to characterize interactivity is slightly misleading however, since framerate is a measure of bandwidth
Bandwidth (computing)
In computer networking and computer science, bandwidth, network bandwidth, data bandwidth, or digital bandwidth is a measure of available or consumed data communication resources expressed in bits/second or multiples of it .Note that in textbooks on wireless communications, modem data transmission,...
while humans are more sensitive to latency
Lag
Lag is a common word meaning to fail to keep up or to fall behind. In real-time applications, the term is used when the application fails to respond in a timely fashion to inputs...
. Specifically, it is possible to achieve a good framerate of 50 frame/s but if the images generated refer to changes to the visualization that a person made more than 1 second ago, it will not feel interactive to a person.
The rapid response time required for interactive visualization is a difficult constraint to meet and there are several approaches that have been explored to provide people with rapid visual feedback based on their input. Some include
- Parallel renderingParallel renderingParallel rendering is the application of parallel programming to the computational domain of computer graphics. Rendering graphics can require massive computational resources for complex scenes that arise in scientific visualization, medical visualization, CAD applications, and virtual reality...
– where more than one computer or video card is used simultaneously to render an image. Multiple frames can be rendered at the same time by different computers and the results transferred over the network for display on a single monitor. This requires each computer to hold a copy of all the information to be rendered and increases bandwidth, but also increases latency. Also, each computer can render a different region of a single frame and send the results over a network for display. This again requires each computer to hold all of the data and can lead to a load imbalance when one computer is responsible for rendering a region of the screen with more information than other computers. Finally, each computer can render an entire frame containing a subset of the information. The resulting images plus the associated depth buffer can then be sent across the network and merged with the images from other computers. The result is a single frame containing all the information to be rendered, even though no single computer's memory held all of the information. This is called parallel depth compositing and is used when large amounts of information must be rendered interactively. - Progressive rendering – where a framerate is guaranteed by rendering some subset of the information to be presented and providing incremental (progressive) improvements to the rendering once the visualization is no longer changing.
- Level-of-detail (LODLevel of detailIn computer graphics, accounting for level of detail involves decreasing the complexity of a 3D object representation as it moves away from the viewer or according other metrics such as object importance, eye-space speed or position....
) rendering – where simplified representations of information are rendered to achieve a desired framerate while a person is providing input and then the full representation is used to generate a still image once the person is through manipulating the visualization. One common variant of LOD rendering is subsampling. When the information being represented is stored in a topologicallyTopologyTopology is a major area of mathematics concerned with properties that are preserved under continuous deformations of objects, such as deformations that involve stretching, but no tearing or gluing...
rectangular array (as is common with digital photos, MRI scans, and finite differenceFinite differenceA finite difference is a mathematical expression of the form f − f. If a finite difference is divided by b − a, one gets a difference quotient...
simulations), a lower resolution version can easily be generated by skipping n points for each 1 point rendered. Subsampling can also be used to accelerate rendering techniques such as volume visualization that require more than twice the computations for an image twice the size. By rendering a smaller image and then scaling the image to fill the requested screen space, much less time is required to render the same data. - Frameless rendering – where the visualization is no longer presented as a time series of images, but as a single image where different regions are updated over time.
Programs
Many interactive visualization programs exist. Some are open sourceOpen source
The term open source describes practices in production and development that promote access to the end product's source materials. Some consider open source a philosophy, others consider it a pragmatic methodology...
and freely available. Some are commercial.
- Avizo
- Bunkspeed
- DataTank
- DrishtiDrishtiDrishti is a multi-platform, open-source Volume Exploration and Presentation Tool. It was written for visualizing tomography data, electron-microscopy data and so forth. It aims to ease understanding of the data set and to assist with conveying that understanding to the research community or a...
- EnSight
- Eye-SysEye-SysEye-Sys is a general-purpose commercial visualization application. It was developed by Interactive Data Visualization, Inc. , developers of the SpeedTree video game middleware. Eye-Sys development was partially funded by the U.S...
- Grapheur
- InfoScope
- KeyShot
- KMviz
- Lattice 3D
- LightWorks
- Lumiscaphe Patchwork3D
- MayaVi
- MedINRIA
- Miner3D
- NovoSpark Visualizer
- Opticore
- OsiriX
- ParaViewParaViewParaView is an open source, freely available program for parallel, interactive, scientific visualization. It has a client–server architecture to facilitate remote visualization of datasets, and generates level of detail models to maintain interactive framerates for large datasets. It is an...
- RTT
- ScienceGL
- SpinFire
- StatPlanet Thematic Maps & Graphs
- Teamcenter Lifecycle Visualization
- Tecplot
- V3D
- vcollab
- Visalix
- VisIt
Libraries
Many software libraries exist that can be used by developers to create interactive visualization programs.- APEX Toolkit
- Insight Segmentation and Registration Toolkit (ITK)
- JT Open
- Open Inventor 3D graphics toolkit
- OpenRM Scene Graph
- OpenSG
- Prefuse Information Visualization Toolkit
- Unigine Engine
- VisualEyes Visualization Authoring Tool
- Visualization Tool Kit (VTK)
Research
Many conferences occur where interactive visualization academic papers are presented and published.- Amer. Soc. of Information Science and Technology (ASIS&T SIGVIS) Special Interest Group in Visualization Information and Sound
- ACM SIGCHI
- ACM SIGGRAPH
- ACM VRST
- Eurographics
- IEEE Visualization
- ACM Transactions on Graphics
- IEEE Transactions on Visualization and Computer Graphics