Tangible User Interface
Encyclopedia
A tangible user interface (TUI) is a user interface
in which a person interacts with digital information through the physical environment
. The initial name was Graspable User Interface, which no longer is used.
One of the pioneers in tangible user interfaces is Hiroshi Ishii
, a professor in the MIT
Media Laboratory who heads the Tangible Media Group. His particular vision for tangible UIs, called Tangible Bits, is to give physical form to digital information, making bits directly manipulable and perceptible. Tangible bits pursues seamless coupling between these two very different worlds of bits and atoms.
represents a single message left on the answering machine
. Dropping a marble into a dish plays back the associated message or calls back the caller.
Another example is the Topobo system. The blocks in Topobo are like LEGO
blocks which can be snapped together, but can also move by themselves using motorized components. A person can push, pull, and twist these blocks, and the blocks can memorize these movements and replay them.
Another implementation allows the user to sketch a picture on the system's table top with a real tangible pen. Using hand gestures, the user can clone the image and stretch it in the X and Y axes just as one would in a paint program. This system would integrate a video camera with a gesture recognition system.
Another example is jive, the implementation of a TUI helped make this product more accessible to elderly users of the product. The 'friend' passes can also be used to activate different interactions with the product.
Several approaches have been made to establish a generic middleware for TUIs. They target toward the independence of application domains as well as flexibility in terms of the deployed sensor technology. For example, Siftables provides an application platform in which small gesture sensitive displays act together to form a human-computer interface.
For collaboration support TUIs have to allow the spatial distribution, asynchronous activities, and the dynamic modification of the TUI infrastructure, to name the most prominent ones. This approach presents a framework based on the LINDA tuple space concept to meet these requirements. The implemented TUIpist framework deploys arbitrary sensor technology for any type of application and actuators in distributed environments.
A further example of a type of TUI is a Projection Augmented model
.
The MIT Tangible Media Group, headed by Hiroshi Ishi is continuously developing and experimenting with TUIs including many tabletop applications.
The Urp and the more advanced Augmented Urban Planning Workbench allows digital simulations of air flow, shadows, reflections, and other data based on the positions and orientations of physical models of buildings, on the table surface.
Newer developments go even one step further and incorporate the third dimension by allowing to form landscapes with clay (Illuminating Clay) or sand (Sand Scape). Again different simulations allow the analysis of shadows, height maps, slopes and other characteristics of the interactively formable landmasses.
InfrActables is a back projection collaborative table that allows interaction by using TUIs that incorporate state recognition. Adding different buttons to the TUIs enables additional functions associated to the TUIs. Newer Versions of the technology can even be integrated into LC-displays by using infrared sensors behind the LC matrix.
The Tangible Disaster allows the user to analyze disaster measures and simulate different kinds of disasters (fire, flood, tsunami,.) and evacuation scenarios during collaborative planning sessions. Physical objects ‚gpuckss’ allow positioning disasters by placing them on the interactive map and additionally tuning parameters (i.e. scale) using dials attached to them.
Apparently the commercial potential of TUIs has been identified recently. The repeatadly awarded Reactable, an interactive tangible tabletop instrument, is now distributed commercially by Reactable Systems, a spinoff company of the Pompeu Fabra University, where it was developed. With the Reactable users can set up their own instrument interactively, by physically placing different objects (representing oscillators, filters, modulators... ) and parametrise them by rotating and using touch-input.
Microsoft is distributing its novel Windows-based platform Microsoft Surface since last year. Beside Multitouch tracking of fingers the platform supports the recognition of physical objects by their footprints. Several applications, mainly for the use in commercial space, have been presented. Examples reach from designing an own individual graphical layout for a snowboard or skateboard to studying the details of a wine in a restaurant by placing it on the table and navigating through menus via touch input. Also interactions like the collaborative browsing of photographs from a handycam or cell phone that connects seamlessly once placed on the table are supported.
Another notable interactive installation is instant city that combines gaming, music, architecture and collaborative aspects. It allows the user to build three dimensional structures and set up a city with rectangular building blocks, which simultaneously results in the interactive assembly of musical fragments of different composers.
The development of the Reactable
and the subsequent release of its tracking technology reacTIVision[12] under the GNU/GPL as well as the open specifications of the TUIO protocol have triggered an enormous amount of developments based on this technology.
In the last few years also many amateur and semi-professional projects, beside academia and commerce have been started. Thanks to open source tracking technologies (reacTIVision) and the ever increasing computational power available to end-consumers, the required infrastructure is nowadays accessible to almost everyone. A standard PC, a web-cam, and some handicraft work allow to setup tangible systems with a minimal programming and material effort. This opens doors to novel ways of perception of human-computer interaction and gives room for new forms of creativity for the broad public, to experiment and play with.
It is difficult to keep track and overlook the rapidly growing amount of all these systems and tools, but while many of them seem only to utilize the available technologies and are limited to some initial experiments and tests with some basic ideas or just reproduce existing systems, a few of them open out into novel interfaces and interactions and are deployed in public space or embedded in art installations.
The Tangible Factory Planning is a tangible table based on reacTIVision that allows to collaboratively plan and visualize production processes in combination with plans of new factory buildings and was developed within a diploma thesis.
Another of the many reacTIVision-based tabletops is ImpulsBauhaus-Interactive Table and was on exhibition at the Bauhaus-University in Weimar marking the 90th anniversary of the establishment of Bauhaus. Visitors could browse and explore the biographies, complex relations and social networks between members of the movement.
User interface
The user interface, in the industrial design field of human–machine interaction, is the space where interaction between humans and machines occurs. The goal of interaction between a human and a machine at the user interface is effective operation and control of the machine, and feedback from the...
in which a person interacts with digital information through the physical environment
Built environment
The term built environment refers to the human-made surroundings that provide the setting for human activity, ranging in scale from personal shelter and buildings to neighborhoods and cities that can often include their supporting infrastructure, such as water supply or energy networks.The built...
. The initial name was Graspable User Interface, which no longer is used.
One of the pioneers in tangible user interfaces is Hiroshi Ishii
Hiroshi Ishii
is a Japanese computer scientist. He is a professor at the Massachusetts Institute of Technology. Ishii pioneered the Tangible User Interface in the field of Human-computer interaction.-Biography:Ishii was born in Tokyo and raised in Sapporo....
, a professor in the MIT
Massachusetts Institute of Technology
The Massachusetts Institute of Technology is a private research university located in Cambridge, Massachusetts. MIT has five schools and one college, containing a total of 32 academic departments, with a strong emphasis on scientific and technological education and research.Founded in 1861 in...
Media Laboratory who heads the Tangible Media Group. His particular vision for tangible UIs, called Tangible Bits, is to give physical form to digital information, making bits directly manipulable and perceptible. Tangible bits pursues seamless coupling between these two very different worlds of bits and atoms.
Characteristics of tangible user interfaces
- Physical representations are computationally coupled to underlying digital information.
- Physical representations embody mechanisms for interactive control.
- Physical representations are perceptually coupled to actively mediated digital representations.
- Physical state of tangibles embodies key aspects of the digital state of a system
Examples
An example of a tangible UI is the Marble Answering Machine by Durrell Bishop (1992). A marbleMarbles
A marble is a small spherical toy usually made from glass, clay, steel, or agate. These balls vary in size. Most commonly, they are about ½ inch in diameter, but they may range from less than ¼ inch to over 3 inches , while some art glass marbles fordisplay purposes are over 12 inches ...
represents a single message left on the answering machine
Answering machine
The answering machine or message machine, also known as the telephone answering machine in the UK and some Commonwealth countries) and previously known as an ansaphone, ansafone, or telephone answering device is a device for answering telephones and recording callers' messages.Unlike voicemail,...
. Dropping a marble into a dish plays back the associated message or calls back the caller.
Another example is the Topobo system. The blocks in Topobo are like LEGO
Lego
Lego is a line of construction toys manufactured by the Lego Group, a privately held company based in Billund, Denmark. The company's flagship product, Lego, consists of colorful interlocking plastic bricks and an accompanying array of gears, minifigures and various other parts...
blocks which can be snapped together, but can also move by themselves using motorized components. A person can push, pull, and twist these blocks, and the blocks can memorize these movements and replay them.
Another implementation allows the user to sketch a picture on the system's table top with a real tangible pen. Using hand gestures, the user can clone the image and stretch it in the X and Y axes just as one would in a paint program. This system would integrate a video camera with a gesture recognition system.
Another example is jive, the implementation of a TUI helped make this product more accessible to elderly users of the product. The 'friend' passes can also be used to activate different interactions with the product.
Several approaches have been made to establish a generic middleware for TUIs. They target toward the independence of application domains as well as flexibility in terms of the deployed sensor technology. For example, Siftables provides an application platform in which small gesture sensitive displays act together to form a human-computer interface.
For collaboration support TUIs have to allow the spatial distribution, asynchronous activities, and the dynamic modification of the TUI infrastructure, to name the most prominent ones. This approach presents a framework based on the LINDA tuple space concept to meet these requirements. The implemented TUIpist framework deploys arbitrary sensor technology for any type of application and actuators in distributed environments.
A further example of a type of TUI is a Projection Augmented model
Projection augmented model
A Projection Augmented model consists of a physical three-dimensional model onto which a computer image is projected to create a realistic looking object...
.
State of the art
Since the invention of Durell Bishop's Marble Answering Machine (1992) almost two decades ago, the interest in Tangible User Interfaces (TUIs) has grown constantly and with every year more tangible systems are showing up. In 1999 Gary Zalewski patented a system of moveable children's blocks containing sensors and displays for teaching spelling and sentence composition. A similar system is being marketed as "Siftables".The MIT Tangible Media Group, headed by Hiroshi Ishi is continuously developing and experimenting with TUIs including many tabletop applications.
The Urp and the more advanced Augmented Urban Planning Workbench allows digital simulations of air flow, shadows, reflections, and other data based on the positions and orientations of physical models of buildings, on the table surface.
Newer developments go even one step further and incorporate the third dimension by allowing to form landscapes with clay (Illuminating Clay) or sand (Sand Scape). Again different simulations allow the analysis of shadows, height maps, slopes and other characteristics of the interactively formable landmasses.
InfrActables is a back projection collaborative table that allows interaction by using TUIs that incorporate state recognition. Adding different buttons to the TUIs enables additional functions associated to the TUIs. Newer Versions of the technology can even be integrated into LC-displays by using infrared sensors behind the LC matrix.
The Tangible Disaster allows the user to analyze disaster measures and simulate different kinds of disasters (fire, flood, tsunami,.) and evacuation scenarios during collaborative planning sessions. Physical objects ‚gpuckss’ allow positioning disasters by placing them on the interactive map and additionally tuning parameters (i.e. scale) using dials attached to them.
Apparently the commercial potential of TUIs has been identified recently. The repeatadly awarded Reactable, an interactive tangible tabletop instrument, is now distributed commercially by Reactable Systems, a spinoff company of the Pompeu Fabra University, where it was developed. With the Reactable users can set up their own instrument interactively, by physically placing different objects (representing oscillators, filters, modulators... ) and parametrise them by rotating and using touch-input.
Microsoft is distributing its novel Windows-based platform Microsoft Surface since last year. Beside Multitouch tracking of fingers the platform supports the recognition of physical objects by their footprints. Several applications, mainly for the use in commercial space, have been presented. Examples reach from designing an own individual graphical layout for a snowboard or skateboard to studying the details of a wine in a restaurant by placing it on the table and navigating through menus via touch input. Also interactions like the collaborative browsing of photographs from a handycam or cell phone that connects seamlessly once placed on the table are supported.
Another notable interactive installation is instant city that combines gaming, music, architecture and collaborative aspects. It allows the user to build three dimensional structures and set up a city with rectangular building blocks, which simultaneously results in the interactive assembly of musical fragments of different composers.
The development of the Reactable
Reactable
The Reactable is an electronic musical instrument with a tabletop Tangible User Interface that has been developed within the at the Universitat Pompeu Fabra in Barcelona, Spain by Sergi Jordà, Marcos Alonso, Martin Kaltenbrunner and Günter Geiger....
and the subsequent release of its tracking technology reacTIVision[12] under the GNU/GPL as well as the open specifications of the TUIO protocol have triggered an enormous amount of developments based on this technology.
In the last few years also many amateur and semi-professional projects, beside academia and commerce have been started. Thanks to open source tracking technologies (reacTIVision) and the ever increasing computational power available to end-consumers, the required infrastructure is nowadays accessible to almost everyone. A standard PC, a web-cam, and some handicraft work allow to setup tangible systems with a minimal programming and material effort. This opens doors to novel ways of perception of human-computer interaction and gives room for new forms of creativity for the broad public, to experiment and play with.
It is difficult to keep track and overlook the rapidly growing amount of all these systems and tools, but while many of them seem only to utilize the available technologies and are limited to some initial experiments and tests with some basic ideas or just reproduce existing systems, a few of them open out into novel interfaces and interactions and are deployed in public space or embedded in art installations.
The Tangible Factory Planning is a tangible table based on reacTIVision that allows to collaboratively plan and visualize production processes in combination with plans of new factory buildings and was developed within a diploma thesis.
Another of the many reacTIVision-based tabletops is ImpulsBauhaus-Interactive Table and was on exhibition at the Bauhaus-University in Weimar marking the 90th anniversary of the establishment of Bauhaus. Visitors could browse and explore the biographies, complex relations and social networks between members of the movement.
External links
- uniTUIO : MultiTouch integration for the Unity game engine using the TUIO framework
- Proceedings of the 16th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises
- Tangible and Embedded Interaction Conference
- MIT Media Lab Tangible Media Group
- Encyclopedia entry on the history of Tangible Interaction and Tangible User Interfaces
- Hiroshi Ishii and Brygg Ullmer, Tangible Bits: Towards Seamless Interfaces between People, Bits and Atoms. Published in the Proceedings of Human Factors in Computing Systems: CHI 97, Denver, CO.
- jive A Magnetic Tangible user interface that uses friend passes to load information on the screen.
- Tangint: Wiki for drawing together content and discussion related to research on tangible interfaces/interaction.
- Papier-Mâché: a toolkit for building tangible UIs
- Percussa AudioCubes: Tangible interface for exploring sound/music
- reacTIVision: a framework for creating tangible UIs
- Topobo project
- Interactive Paper: Integrating paper and digital information
- The Moving Pictures Project: a tangible video composition platform
- Spatial Augmented Reality
- Audiopad: a composition and performance instrument
- Memodules Project : Memodules as tangible shortcuts to multimedia information
- TANGerINE Project : A Tangible Interactive Natural Environment which merges tabletop interfaces and smart objects
- reactable : A collaborative electronic music instrument with a tabletop tangible multi-touchMulti-touchIn computing, multi-touch refers to a touch sensing surface's ability to recognize the presence of two or more points of contact with the surface...
interface. - Trackmate : An open source initiative to create an inexpensive, do-it-yourself tangible tracking system via the LusidOSC protocol.
- Tangible Web : Tangible Web Project
- TUIO : A community standard for the design and creation of tangible multi-touch surfaces.
- d-touch : A visual markers recognition systems that enables the development of low-cost tangible user interfaces and mixed reality applications.
- d-touch drum machine A drum machine with a tangible user interface.
- Bubblegum Sequencer A step sequencer using colored balls to represent sounds.
- d-touch sequencer A sequencer able to record live from microphone with a tangible user interface.
- From Turtles to Tangible Programming Bricks A survey of tangible user interfaces in education research at MIT.