Natural user interface
Encyclopedia
In computing
, a natural user interface, or NUI, is the common parlance used by designers and developers of computer interfaces to refer to a user interface
that is effectively invisible, or becomes invisible with successive learned interactions, to its users. The word natural is used because most computer interfaces use artificial control devices whose operation has to be learned. A NUI relies on a user being able to quickly transition from novice to expert. While the interface requires learning, that learning is eased through design which gives the user the feeling that they are instantly and continuously successful. This can be aided by technology which allows users to carry out relatively natural motions, movements or gestures that they quickly discover control the computer application or manipulate the on-screen content. A common misunderstanding of the "Natural User Interface" is that it is somehow mimicry of nature or that some inputs to a computer are somehow more 'natural' than others. In truth, the goal is to make the user feel like a natural.
Mann referred to this work as "Natural User Interfaces", "Direct User Interfaces", and "Metaphor-Free Computing". Mann's EyeTap
technology typically embodies an example of a Natural User Interface. Another example of an NUI is the hydraulophone
when it is used as an input device, in which touching a natural element (water) becomes a way of inputting data.
In 2006 Christian Moore established an open research community with the goal to expand discussion and development related to NUI technologies. In a 2008 conference presentation "Predicting the Past," August de los Reyes, a Principal User Experience Director of Surface Computing at Microsoft described the NUI as the next evolutionary phase following the shift from the command-line interface
(CLI) to the graphical user interface
(GUI).
In the CLI, users had to learn an artificial means of input, the keyboard, and a series of codified inputs, that had a limited range of responses, where the syntax of those commands was strict.
Then, when the mouse enabled the GUI, users could more easily learn the mouse movements and actions, and were able to explore the interface much more. The GUI relied on metaphors for interacting with on-screen content or objects. The 'desktop' and 'drag' for example, being metaphors for a visual interface that ultimately was translated back into the strict codified language of the computer.
An example of the misunderstanding of the term NUI was demonstrated by demonstrations at the Consumer Electronics Show
in 2010. "Now a new wave of products is poised to bring "natural user interfaces"—as these methods of controlling electronics devices are called—to an even broader audience."
In 2010 Microsoft's Bill Buxton reiterated the importance of the NUI within Microsoft Corporation with a video discussing the technologies and its future potential.
When Bill Buxton
was asked about the iPhone's interface, he responded "Multi-touch technologies have a long history. To put it in perspective, the original work undertaken by my team was done in 1984, the same year that the first Macintosh computer was released, and we were not the first."
Multi-Touch is a technology which could enable a natural user interface. However, most UI toolkits used to construct interfaces executed with such technology are traditional GUI interfaces.
One example is the work done by Jefferson Han on multi-touch
interfaces. In a demonstration at TED in 2006, he showed a variety of means of interacting with on-screen content using both direct manipulations and gestures. For example, to shape an on-screen glutinous mass, Jeff literally 'pinches' and prods and pokes it with his fingers. In a GUI interface for a design application for example, a user would use the metaphor of 'tools' to do this, for example, selecting a prod tool, or selecting two parts of the mass that they then wanted to apply a 'pinch' action to. Han showed that user interaction could be much more intuitive by doing away with the interaction devices that we are used to and replacing them with a screen that was capable of detecting a much wider range of human actions and gestures.
Microsoft Surface
Microsoft Surface
takes similar ideas on how users interact with content, but adds in the ability for the device to optically recognise objects placed on top of it. In this way, users can trigger actions on the computer through the same gestures and motions as Jeff Han's touchscreen allowed, but also objects become a part of the control mechanisms. So for example, when you place a wine glass on the table, the computer recognises it as such and displays content associated with that wine glass. Placing a wine glass on a table is a natural thing for a person to do, hence it fits as part of a natural user interface.
3D Immersive Touch
3D Immersive Touch is defined as the direct manipulation of 3D virtual environment objects using single or multi-touch surface hardware in multi-user 3D virtual environments. Coined first in 2007 to describe and define the 3D natural user interface learning principles associated with Edusim. Immersive Touch natural user interface now appears to be taking on a broader focus and meaning with the broader adaption of surface and touch driven hardware such as the iPhone, iPod touch, iPad, and a growing list of other hardware. Apple also seems to be taking a keen interest in “Immersive Touch” 3D natural user interfaces over the past few years.
Xbox Kinect
Xbox
Kinect is a product from Xbox
that uses spatial gestures for interaction instead of a game controller
. According to Microsoft
's page, Kinect is designed for a "a revolutionary new way to play: no controller required."
Academic research
Computing
Computing is usually defined as the activity of using and improving computer hardware and software. It is the computer-specific part of information technology...
, a natural user interface, or NUI, is the common parlance used by designers and developers of computer interfaces to refer to a user interface
User interface
The user interface, in the industrial design field of human–machine interaction, is the space where interaction between humans and machines occurs. The goal of interaction between a human and a machine at the user interface is effective operation and control of the machine, and feedback from the...
that is effectively invisible, or becomes invisible with successive learned interactions, to its users. The word natural is used because most computer interfaces use artificial control devices whose operation has to be learned. A NUI relies on a user being able to quickly transition from novice to expert. While the interface requires learning, that learning is eased through design which gives the user the feeling that they are instantly and continuously successful. This can be aided by technology which allows users to carry out relatively natural motions, movements or gestures that they quickly discover control the computer application or manipulate the on-screen content. A common misunderstanding of the "Natural User Interface" is that it is somehow mimicry of nature or that some inputs to a computer are somehow more 'natural' than others. In truth, the goal is to make the user feel like a natural.
History
In the 1970s, 80's and 90's Steve Mann developed a number of user-interface strategies using natural interaction with the real world as an alternative to command line user interfaces or graphical user interfaces.Mann referred to this work as "Natural User Interfaces", "Direct User Interfaces", and "Metaphor-Free Computing". Mann's EyeTap
Eyetap
An EyeTap is a device that is worn in front of the eye that acts as a camera to record the scene available to the eye as well as a display to superimpose a computer-generated imagery on the original scene available to the eye....
technology typically embodies an example of a Natural User Interface. Another example of an NUI is the hydraulophone
Hydraulophone
A hydraulophone is a tonal acoustic musical instrument played by direct physical contact with water where sound is generated or affected hydraulically. Typically sound is produced by the same hydraulic fluid in contact with the player's fingers...
when it is used as an input device, in which touching a natural element (water) becomes a way of inputting data.
In 2006 Christian Moore established an open research community with the goal to expand discussion and development related to NUI technologies. In a 2008 conference presentation "Predicting the Past," August de los Reyes, a Principal User Experience Director of Surface Computing at Microsoft described the NUI as the next evolutionary phase following the shift from the command-line interface
Command-line interface
A command-line interface is a mechanism for interacting with a computer operating system or software by typing commands to perform specific tasks...
(CLI) to the graphical user interface
Graphical user interface
In computing, a graphical user interface is a type of user interface that allows users to interact with electronic devices with images rather than text commands. GUIs can be used in computers, hand-held devices such as MP3 players, portable media players or gaming devices, household appliances and...
(GUI).
In the CLI, users had to learn an artificial means of input, the keyboard, and a series of codified inputs, that had a limited range of responses, where the syntax of those commands was strict.
Then, when the mouse enabled the GUI, users could more easily learn the mouse movements and actions, and were able to explore the interface much more. The GUI relied on metaphors for interacting with on-screen content or objects. The 'desktop' and 'drag' for example, being metaphors for a visual interface that ultimately was translated back into the strict codified language of the computer.
An example of the misunderstanding of the term NUI was demonstrated by demonstrations at the Consumer Electronics Show
Consumer Electronics Show
The International Consumer Electronics Show is a major technology-related trade show held each January in the Las Vegas Convention Center, Las Vegas, Nevada, United States. Not open to the public, the Consumer Electronics Association-sponsored show typically hosts previews of products and new...
in 2010. "Now a new wave of products is poised to bring "natural user interfaces"—as these methods of controlling electronics devices are called—to an even broader audience."
In 2010 Microsoft's Bill Buxton reiterated the importance of the NUI within Microsoft Corporation with a video discussing the technologies and its future potential.
Early examples
Multi-TouchWhen Bill Buxton
Bill Buxton
William Arthur Stewart "Bill" Buxton is a Canadian computer scientist and designer. He is currently a Principal researcher at Microsoft Research...
was asked about the iPhone's interface, he responded "Multi-touch technologies have a long history. To put it in perspective, the original work undertaken by my team was done in 1984, the same year that the first Macintosh computer was released, and we were not the first."
Multi-Touch is a technology which could enable a natural user interface. However, most UI toolkits used to construct interfaces executed with such technology are traditional GUI interfaces.
Examples of interfaces commonly referred to as NUI
Perceptive PixelOne example is the work done by Jefferson Han on multi-touch
Multi-touch
In computing, multi-touch refers to a touch sensing surface's ability to recognize the presence of two or more points of contact with the surface...
interfaces. In a demonstration at TED in 2006, he showed a variety of means of interacting with on-screen content using both direct manipulations and gestures. For example, to shape an on-screen glutinous mass, Jeff literally 'pinches' and prods and pokes it with his fingers. In a GUI interface for a design application for example, a user would use the metaphor of 'tools' to do this, for example, selecting a prod tool, or selecting two parts of the mass that they then wanted to apply a 'pinch' action to. Han showed that user interaction could be much more intuitive by doing away with the interaction devices that we are used to and replacing them with a screen that was capable of detecting a much wider range of human actions and gestures.
Microsoft Surface
Microsoft Surface
Microsoft Surface
Microsoft Surface is a multi-touch product from Microsoft which is developed as a software and hardware combination technology that allows a user, or multiple users, to manipulate digital content by the use of gesture recognition. This could involve the motion of hands or physical objects. It was...
takes similar ideas on how users interact with content, but adds in the ability for the device to optically recognise objects placed on top of it. In this way, users can trigger actions on the computer through the same gestures and motions as Jeff Han's touchscreen allowed, but also objects become a part of the control mechanisms. So for example, when you place a wine glass on the table, the computer recognises it as such and displays content associated with that wine glass. Placing a wine glass on a table is a natural thing for a person to do, hence it fits as part of a natural user interface.
3D Immersive Touch
3D Immersive Touch is defined as the direct manipulation of 3D virtual environment objects using single or multi-touch surface hardware in multi-user 3D virtual environments. Coined first in 2007 to describe and define the 3D natural user interface learning principles associated with Edusim. Immersive Touch natural user interface now appears to be taking on a broader focus and meaning with the broader adaption of surface and touch driven hardware such as the iPhone, iPod touch, iPad, and a growing list of other hardware. Apple also seems to be taking a keen interest in “Immersive Touch” 3D natural user interfaces over the past few years.
Xbox Kinect
Xbox
Xbox
The Xbox is a sixth-generation video game console manufactured by Microsoft. It was released on November 15, 2001 in North America, February 22, 2002 in Japan, and March 14, 2002 in Australia and Europe and is the predecessor to the Xbox 360. It was Microsoft's first foray into the gaming console...
Kinect is a product from Xbox
Xbox
The Xbox is a sixth-generation video game console manufactured by Microsoft. It was released on November 15, 2001 in North America, February 22, 2002 in Japan, and March 14, 2002 in Australia and Europe and is the predecessor to the Xbox 360. It was Microsoft's first foray into the gaming console...
that uses spatial gestures for interaction instead of a game controller
Game controller
A game controller is a device used with games or entertainment systems used to control a playable character or object, or otherwise provide input in a computer game. A controller is typically connected to a game console or computer by means of a wire, cord or nowadays, by means of wireless connection...
. According to Microsoft
Microsoft
Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...
's page, Kinect is designed for a "a revolutionary new way to play: no controller required."
See also
- Kinetic user interface
- Organic user interfaceOrganic user interfaceIn human–computer interaction, an organic user interface is a user interface with interaction techniques that use the physical shape and position of a device to control it...
- Tangible user interfaceTangible User InterfaceA tangible user interface is a user interface in which a person interacts with digital information through the physical environment. The initial name was Graspable User Interface, which no longer is used....
- Post-WIMPPost-WIMPIn computing post-WIMP comprises work on user interfaces, mostly graphical user interfaces, which attempt to go beyond the paradigm of windows, icons, menus and a pointing device, i.e. WIMP interfaces....
- Spatial navigationSpatial navigationIn computing, spatial navigation is the ability to navigate between focusable elements, such as hyperlinks and form controls, within a structured document or user interface according to the spatial location....
- Touch user interfaceTouch user interfaceA touch user interface is a computer-pointing technology based upon the sense of touch . Whereas a graphical user interface relies upon the sense of sight, the TUI enables not only the sense of touch to innervate and activate computer-based functions, it also allows the user, particularly those...
- Eye trackingEye trackingEye tracking is the process of measuring either the point of gaze or the motion of an eye relative to the head. An eye tracker is a device for measuring eye positions and eye movement. Eye trackers are used in research on the visual system, in psychology, in cognitive linguistics and in product...
- EdusimEdusimEdusim is a Cave Automatic Virtual Environment "Immersive Touch" 3D Natural User Interface based concept of lesson driven 3D virtual worlds on the classroom interactive whiteboard or classroom interactive surface...
- Graphical User Interface (GUI)Graphical user interfaceIn computing, a graphical user interface is a type of user interface that allows users to interact with electronic devices with images rather than text commands. GUIs can be used in computers, hand-held devices such as MP3 players, portable media players or gaming devices, household appliances and...
External links
Examples and Further Reading- http://www.vimeo.com/channels/nui - Vimeo's NUI Channel showcasing NUI Interfaces
- http://nuigroup.com/faq/ - NUI Group FAQ
- http://wiki.nuigroup.com/Natural_User_Interface NUI Group's Wiki Entry on Natural User Interface
Academic research
- NiCE (Natural User Interfaces for Collaborative Environments) - NUI project at the Media Interaction Lab of the Upper Austria University of Applied Sciences
- Natural user interaction in virtual/augmented reality - NUI research at the University of North Carolina at Chapel HillUniversity of North Carolina at Chapel HillThe University of North Carolina at Chapel Hill is a public research university located in Chapel Hill, North Carolina, United States...
Department of Computer Science - Multimodal Human-Computer Interaction: a constructive and empirical study - Research from the University of TampereUniversity of TampereThe University of Tampere is a university in Tampere, Finland. It has 15,400 degree students and 2,100 employees. It was originally founded in 1925 in Helsinki as a Civic College , and from 1930 onwards it was known as a School of Social Sciences...