FreeTrack
Encyclopedia
FreeTrack is a general purpose optical motion tracking
Optical motion tracking
Motion Tracking can differ from Motion Capture because in applications such as sports and missiles the object of interest is tracked optically instead of a person...

 application for Microsoft Windows
Microsoft Windows
Microsoft Windows is a series of operating systems produced by Microsoft.Microsoft introduced an operating environment named Windows on November 20, 1985 as an add-on to MS-DOS in response to the growing interest in graphical user interfaces . Microsoft Windows came to dominate the world's personal...

, released under the GNU General Public License
GNU General Public License
The GNU General Public License is the most widely used free software license, originally written by Richard Stallman for the GNU Project....

, that can be used with common inexpensive cameras. Its primary focus is head tracking with uses in virtual reality
Virtual reality
Virtual reality , also known as virtuality, is a term that applies to computer-simulated environments that can simulate physical presence in places in the real world, as well as in imaginary worlds...

, simulation
Simulation
Simulation is the imitation of some real thing available, state of affairs, or process. The act of simulating something generally entails representing certain key characteristics or behaviours of a selected physical or abstract system....

, video games, 3D modeling
3D modeling
In 3D computer graphics, 3D modeling is the process of developing a mathematical representation of any three-dimensional surface of object via specialized software. The product is called a 3D model...

, computer aided design and general hands-free computing
Hands-free computing
Hands-free computing is a term used to describe a configuration of computers so that they can be used by persons without the use of the hands interfacing with commonly used human interface devices such as the mouse and keyboard. Hands-free computing is important because it is useful to both able...

 to improve computer accessibility
Computer accessibility
In human-computer interaction, computer accessibility refers to the accessibility of a computer system to all people, regardless of disability or severity of impairment...

. Tracking can be made sensitive enough that only small head movements are required so that the user's eyes never leave the screen.

A camera is positioned to observe a rigid point model worn by the user, the points of which need to be isolated from background light by means of physical and software filtering. Motion is tracked with up to six degrees of freedom
Six degrees of freedom
Six degrees of freedom refers to motion of a rigid body in three-dimensional space, namely the ability to move forward/backward, up/down, left/right combined with rotation about three perpendicular axes...

 (6DOF), specifically; yaw, pitch, roll, left/right, up/down and forward/back. Windows compatible video devices like webcams are supported, as well as special support for the Nintendo
Nintendo
is a multinational corporation located in Kyoto, Japan. Founded on September 23, 1889 by Fusajiro Yamauchi, it produced handmade hanafuda cards. By 1963, the company had tried several small niche businesses, such as a cab company and a love hotel....

 Wii Remote
Wii Remote
The , also known as the Wiimote, is the primary controller for Nintendo's Wii console. A main feature of the Wii Remote is its motion sensing capability, which allows the user to interact with and manipulate items on screen via gesture recognition and pointing through the use of accelerometer and...

 camera and NaturalPoint cameras (TrackIR
TrackIR
TrackIR is an optical motion tracking game controller for Microsoft Windows, created by NaturalPoint Inc. that tracks head motions with up to six degrees of freedom in later models, allowing handsfree view control for improved game immersiveness and situational awareness...

, SmartNav and OptiTrack).

FreeTrack can output head tracking data to programs directly using its own open interface, as well as TrackIR, SimConnect and FSUIPC interfaces. Programs that support these interfaces are regarded as being FreeTrack-compatible. FreeTrack can also emulate mouse
Mouse (computing)
In computing, a mouse is a pointing device that functions by detecting two-dimensional motion relative to its supporting surface. Physically, a mouse consists of an object held under one of the user's hands, with one or more buttons...

, keyboard
Computer keyboard
In computing, a keyboard is a typewriter-style keyboard, which uses an arrangement of buttons or keys, to act as mechanical levers or electronic switches...

, and joystick
Joystick
A joystick is an input device consisting of a stick that pivots on a base and reports its angle or direction to the device it is controlling. Joysticks, also known as 'control columns', are the principal control in the cockpit of many civilian and military aircraft, either as a center stick or...

 (via PPJoy) if a program does not support a direct interface.

FreeTrack is coded in Delphi 7
Borland Delphi
Embarcadero Delphi is an integrated development environment for console, desktop graphical, web, and mobile applications.Delphi's compilers use its own Object Pascal dialect of Pascal and generate native code for 32- and 64-bit Windows operating systems, as well as 32-bit Mac OS X and iOS...

 and uses DirectShow
DirectShow
DirectShow , codename Quartz, is a multimedia framework and API produced by Microsoft for software developers to perform various operations with media files or streams. It is the replacement for Microsoft's earlier Video for Windows technology...

 and DirectX
DirectX
Microsoft DirectX is a collection of application programming interfaces for handling tasks related to multimedia, especially game programming and video, on Microsoft platforms. Originally, the names of these APIs all began with Direct, such as Direct3D, DirectDraw, DirectMusic, DirectPlay,...

. Head tracking is achieved using implementations of DeMenthon's four point iterative pose
Pose (computer vision)
In computer vision and in robotics, a typical task is to identify specific objects in an image and to determine each object's position and orientation relative to some coordinate system. This information can then be used, for example, to allow a robot to manipulate an object or to avoid moving...

 estimation algorithm (POSIT) and Alter's three point geometric algorithm.

Software

FreeTrack uses a camera to collect real-time information on the point model worn by the user. Specifically the image coordinates of the model points, which are either received directly from the camera or extracted from a video stream. These coordinates are used to generate an estimate of the real head pose, which can be transformed by the user in a number of ways to create a virtual pose. One of the most fundamental transformations involves amplifying rotation so that only small head movements are required. Finally, the virtual pose is sent to the user's choice of outputs. This is all done in the background, with tracking status displayed in the system tray.

A 3D preview is available that shows the virtual head position and orientation for a given real head pose and can be viewed from multiple perspectives, including first-person. This greatly assists with testing and makes it easier to experiment with different settings.

Each degree of freedom
Degrees of freedom (engineering)
In mechanics, degrees of freedom are the set of independent displacements and/or rotations that specify completely the displaced or deformed position and orientation of the body or system...

 (axis) has a response curve that can be modified to change the way the virtual head moves for a given real head movement. This is commonly used to create a central deadzone region so that the user’s head can be more relaxed there.

Keyboard, mouse and joystick buttons can be used to toggle tracking settings, including the virtual centre location (like adjusting the seat position in a car) and individually toggle axes and outputs.

For NaturalPoint cameras, FreeTrack can provide advanced features and a level of customization that is not available with official software.

Camera

Comparison of some cameras compatible with FreeTrack
Camera Sensor resolution
Image resolution
Image resolution is an umbrella term that describes the detail an image holds. The term applies to raster digital images, film images, and other types of images. Higher resolution means more image detail....

FPS
Frame rate
Frame rate is the frequency at which an imaging device produces unique consecutive images called frames. The term applies equally well to computer graphics, video cameras, film cameras, and motion capture systems...

Sensor
Image sensor
An image sensor is a device that converts an optical image into an electronic signal. It is used mostly in digital cameras and other imaging devices...

Angle(°)
Angle of view
In photography, angle of view describes the angular extent of a given scene that is imaged by a camera. It is used interchangeably with the more general term field of view....

Output CPU usage Subpixel precision IR
Infrared
Infrared light is electromagnetic radiation with a wavelength longer than that of visible light, measured from the nominal edge of visible red light at 0.74 micrometres , and extending conventionally to 300 µm...

 LEDs
LEDS
LEDS can be initials for:* Law Enforcement Data System* Link Eleven Display System* Low Energy Dislocation Structure* Land Electronic Defence System * LEDs * Life-Events and Difficulties Schedule...

Approx. price (USD)
Ideal webcam 640x480 60+ monochrome
Monochrome
Monochrome describes paintings, drawings, design, or photographs in one color or shades of one color. A monochromatic object or image has colors in shades of limited colors or hues. Images using only shades of grey are called grayscale or black-and-white...

42 highly compressed small Software dependent Yes ?
OEM
OEM
OEM means the original manufacturer of a component for a product, which may be resold by another company.OEM may also refer to:-Computing:* OEM font, or OEM-US, the original character set of the IBM PC, circa 1981...

 IR
Infrared
Infrared light is electromagnetic radiation with a wavelength longer than that of visible light, measured from the nominal edge of visible red light at 0.74 micrometres , and extending conventionally to 300 µm...

 webcam
640x480 30 color
Bayer filter
A Bayer filter mosaic is a color filter array for arranging RGB color filters on a square grid of photosensors. Its particular arrangement of color filters is used in most single-chip digital image sensors used in digital cameras, camcorders, and scanners to create a color image...

42 JPEG compressed small Software dependent Yes $5
Sony
Sony
, commonly referred to as Sony, is a Japanese multinational conglomerate corporation headquartered in Minato, Tokyo, Japan and the world's fifth largest media conglomerate measured by revenues....

 PlayStation EyeToy
EyeToy
The EyeToy is a color digital camera device, similar to a webcam, for the PlayStation 2. The technology uses computer vision and Gesture recognition to process images taken by the camera...

640x480 30 color 56 JPEG compressed small Software dependent No $16
Sony
Sony
, commonly referred to as Sony, is a Japanese multinational conglomerate corporation headquartered in Minato, Tokyo, Japan and the world's fifth largest media conglomerate measured by revenues....

 PlayStation 3 Eye
PlayStation Eye
The PlayStation Eye is a digital camera device, similar to a webcam, for the PlayStation 3. The technology uses computer vision and gesture recognition to process images taken by the camera. This allows players to interact with games using motion and color detection as well as sound through its...

640x480 187@320x240(CLEye), 125@320x240(Directshow), 75@640x480(DirectShow) color 75, 56 JPEG compressed, raw small Software dependent No $24
Microsoft
Microsoft
Microsoft Corporation is an American public multinational corporation headquartered in Redmond, Washington, USA that develops, manufactures, licenses, and supports a wide range of products and services predominantly related to computing through its various product divisions...

 Xbox Live Vision
640x480 60@320x240, 30@640x480 color ? JPEG compressed, raw small Software dependent No $14
Nintendo
Nintendo
is a multinational corporation located in Kyoto, Japan. Founded on September 23, 1889 by Fusajiro Yamauchi, it produced handmade hanafuda cards. By 1963, the company had tried several small niche businesses, such as a cab company and a love hotel....

 Wii Remote
Wii Remote
The , also known as the Wiimote, is the primary controller for Nintendo's Wii console. A main feature of the Wii Remote is its motion sensing capability, which allows the user to interact with and manipulate items on screen via gesture recognition and pointing through the use of accelerometer and...

128x96 100(Bluetooth
Bluetooth
Bluetooth is a proprietary open wireless technology standard for exchanging data over short distances from fixed and mobile devices, creating personal area networks with high levels of security...

), 250(I2C)
color 41 point coordinates
Cartesian coordinate system
A Cartesian coordinate system specifies each point uniquely in a plane by a pair of numerical coordinates, which are the signed distances from the point to two fixed perpendicular directed lines, measured in the same unit of length...

none 1/8 No $23
NaturalPoint TrackIR 1 60k pixels (e.g. 300x200) 60 monochrome 33 binary threshold
Thresholding (image processing)
Thresholding is the simplest method of image segmentation. From a grayscale image, thresholding can be used to create binary images Thresholding is the simplest method of image segmentation. From a grayscale image, thresholding can be used to create binary images Thresholding is the simplest method...

minimal Yes retired
NaturalPoint TrackIR 2 60k pixels (e.g. 300x200) 100 monochrome 33 binary threshold minimal Yes retired
NaturalPoint TrackIR 3 355x288 80 monochrome 33 binary threshold minimal Yes retired
NaturalPoint TrackIR 3 Pro 355x288 120 monochrome 33 binary threshold minimal Yes retired
NaturalPoint TrackIR 4 Pro 355x288 (subsampled at 710x480) 120 monochrome 46 binary threshold minimal 1/20th Yes $99.95
NaturalPoint TrackIR 5 640x480 120 monochrome 51.7 grayscale threshold minimal 1/150th Yes $149.95
NaturalPoint SmartNav 1/2 60k pixels (e.g. 300x200) 60 monochrome 33 binary threshold minimal Yes retired
NaturalPoint SmartNav 3 355x288 120 monochrome 33 binary threshold minimal 1/20th Yes retired
NaturalPoint SmartNav 4 640x480 (subsampled at 1280x480 ) 100 monochrome 41 grayscale threshold minimal 1/150th Yes $400 to $500

Resolution

In most cases a resolution of 320x240 is sufficient, this is capable of producing a much higher sub-pixel resolution
Sub-pixel resolution
In Digital Image Processing, Sub-pixel resolution can be obtained in digital images containing well defined lines, points or edges that can be processed by an algorithm to reliably measure the position of the line, point or edge in the image with an accuracy exceeding the nominal pixel resolution...

, enough to allow accurate cursor control on a high resolution monitor. Resolutions 640x480 and above have diminishing returns and correspond with an exponential increase in CPU usage when not sufficiently compressed before reaching the computer. Higher resolutions become more important at greater distances from the camera. The Wii utilizes a low resolution 128x96 sensor which is found by some to produce jittery tracking and may require smoothing to improve stability at the cost of decreased responsiveness.

Sensor

For the same resolution, monochrome
Monochrome
Monochrome describes paintings, drawings, design, or photographs in one color or shades of one color. A monochromatic object or image has colors in shades of limited colors or hues. Images using only shades of grey are called grayscale or black-and-white...

 sensors can resolve finer detail much better than color sensors
Image sensor
An image sensor is a device that converts an optical image into an electronic signal. It is used mostly in digital cameras and other imaging devices...

 due to the lack of a color filter array
Color filter array
In photography, a color filter array , or color filter mosaic , is a mosaic of tiny color filters placed over the pixel sensors of an image sensor to capture color information....

.

Frame rate

FreeTrack uses interpolation
Interpolation
In the mathematical field of numerical analysis, interpolation is a method of constructing new data points within the range of a discrete set of known data points....

 with low frame rate video devices to improve panning smoothness. However, responsiveness is fundamentally limited to the frame rate; a 30frame/s webcam has a maximum response delay of 33.3 milliseconds compared with 8.33 milliseconds for a 120frame/s camera. To put this into perspective, a human’s reaction time to visual stimulus (finger reflex) is typically around 200ms, 30ms can be regarded as a competitive ping
Ping
Ping is a computer network administration utility used to test the reachability of a host on an Internet Protocol network and to measure the round-trip time for messages sent from the originating host to a destination computer...

 in online reflex-based games and an LCD monitor refresh rate is typically 17ms.

Higher responsiveness gives a greater feeling of control but since virtual head motion is amplified it can also cause it to move unrealistically fast. For this reason, some programs limit head movement speed, wasting some of the responsiveness of higher frame-rate cameras.

Angle

A wider viewing angle allows a larger tracking region when in close proximity to the camera. At further distances a wide angle is not desirable, more of the frame is unused and the effective resolution drops more rapidly. More peripheral light can also be seen which can interfere with tracking. Viewing angle can be reduced by using digital zoom
Digital zoom
Digital zoom is a method of decreasing the apparent angle of view of a digital photographic or video image. Digital zoom is accomplished by cropping an image down to a centered area with the same aspect ratio as the original, and usually also interpolating the result back up to the pixel...

 at the cost of resolution.

CPU usage

The Nintendo Wii Remote effectively uses no CPU, NaturalPoint cameras use a small amount and general video devices can use a significant amount depending on the brand and the specific camera settings in use. A PlayStation Eye running at the same resolution and frame rate as a TrackIR 4 would be very demanding on a single-core CPU. However, modern multi-core CPUs are making this less of an issue. Resolution and frame rate can always be reduced to conserve CPU resources.

Filters

For reasons of efficiency, FreeTrack requires the tracking points to be isolated from all other light, this is best done using infrared LEDs and a visible-light blocking filter in front of the camera. Photographic film
Photographic film
Photographic film is a sheet of plastic coated with an emulsion containing light-sensitive silver halide salts with variable crystal sizes that determine the sensitivity, contrast and resolution of the film...

 or the magnetic storage
Magnetic storage
Magnetic storage and magnetic recording are terms from engineering referring to the storage of data on a magnetized medium. Magnetic storage uses different patterns of magnetization in a magnetizable material to store data and is a form of non-volatile memory. The information is accessed using...

 medium inside floppy disk
Floppy disk
A floppy disk is a disk storage medium composed of a disk of thin and flexible magnetic storage medium, sealed in a rectangular plastic carrier lined with fabric that removes dust particles...

s can be used as inexpensive visible-light filters. Further filtering can be done in software by adjusting exposure and threshold.

All video devices like webcams have a built-in infrared blocking filter which can be removed to improve sensitivity to infrared light, allowing better point isolation and the possibility of retroreflective tracking. This is normally a straightforward and reversible procedure for most webcams.

Wii Remotes and NaturalPoint cameras are designed for infrared point tracking so already have visible-light blocking filters.

Model configurations

  • Single point: a single tracking point, for example attached on a baseball cap or wire that extends forward of the head. Can be easily achieved with a small light source like a penlight. Unlike other configurations, this is limited to two degrees of freedom (yaw and pitch) and can be more sensitive to translation than rotation.
  • Three point clip: asymmetrical
    Asymmetry
    Asymmetry is the absence of, or a violation of, symmetry.-In organisms:Due to how cells divide in organisms, asymmetry in organisms is fairly usual in at least one dimension, with biological symmetry also being common in at least one dimension....

     triangular arrangement of points that can be clipped to headphones
    Headphones
    Headphones are a pair of small loudspeakers, or less commonly a single speaker, held close to a user's ears and connected to a signal source such as an audio amplifier, radio, CD player or portable Media Player. They are also known as stereophones, headsets or, colloquially, cans. The in-ear...

     and positioned to the side of the head.
  • Three point cap: symmetrical triangular arrangement of points, typically attached to the rim of a baseball cap.
  • Four point cap: symmetrical non-coplanar
    Coplanarity
    In geometry, a set of points in space is coplanar if all the points lie in the same geometric plane. For example, three distinct points are always coplanar; but a fourth point or more added in space can exist in another plane, incoplanarly....

     arrangement of points, also attached to the rim of a baseball cap.


Models can be made in a DIY fashion at minimal expense using readily available electronic components. Component kits and fully constructed models are also available for purchase from some members of the FreeTrack community.

Active points

An active point model uses visible or infrared LEDs (5 mm or larger) to represent the tracking points, powered by battery, transformer (plug pack) or USB
Universal Serial Bus
USB is an industry standard developed in the mid-1990s that defines the cables, connectors and protocols used in a bus for connection, communication and power supply between computers and electronic devices....

. The electric circuit is very basic and can be made by someone with little or no experience with electronics.

Common LEDs, like those found in remote control
Remote control
A remote control is a component of an electronics device, most commonly a television set, used for operating the television device wirelessly from a short line-of-sight distance.The remote control is usually contracted to remote...

s, have a narrow, highly focused beam which is not suitable for optical motion tracking. They can be easily turned into wide angle LEDs by filing their lens tips down flat. Alternatively, wide angle LEDs can be purchased from specialist electronics retailers, like the infrared Siemens/Osram SFH485P, with a half-angle of 40 degrees.

Reflective points

Retroreflective material can be used to represent the tracking points by illumination with an infrared light source. This configuration doesn’t require wires or batteries connected to the user but is more susceptible to interference by background light. In most cases a webcam’s internal infrared blocking filter needs to be removed to increase sensitivity enough that the infrared light reflected by the tracking points can be seen.

FreeTrack interface

FreeTrack has a simple interface that can be freely used by third party programs to access 6DOF tracking data, both real raw measurements and virtual. It is hardware agnostic, so is not dependent on a specific brand or version of hardware and can be used without restriction. Bohemia Interactive's ARMA 2
ArmA 2
ARMA 2 is a tactical shooter video game for the PC developed by Bohemia Interactive Studio. It is considered the spiritual successor in the Operation Flashpoint-series following its predecessor, ArmA: Armed Assault . ARMA 2 was released in June and July 2009...

 is the first game to support the FreeTrack interface and GP Bikes is the first to have exclusive support.

TrackIR interface

FreeTrack is compatible with the unencrypted version of NaturalPoint's head tracking TrackIR interface that has widespread support in simulation games. NaturalPoint have been supplying game developers with an encrypted version of the interface for more popular titles since late 2008, these can be identified as requiring TrackIR software version 4.1.036 or higher and are incompatible with FreeTrack.
The developers of the first game affected, DCS: Black Shark
Digital Combat Simulator
Digital Combat Simulator is a series of study combat flight simulator video games by Eagle Dynamics, and the descendent of the popular "Su-27 Flanker" series....

, tried to release their own head tracking interface but soon after canceled it at NaturalPoint's request. FreeTrack compatibility is still possible using TrackIRFixer to remove the encryption requirement in games.

TIRViews.dll is a dynamic-link library
Dynamic-link library
Dynamic-link library , or DLL, is Microsoft's implementation of the shared library concept in the Microsoft Windows and OS/2 operating systems...

 file distributed with TrackIR software that provides tailored support for a small number of mostly older games, using special interfaces or memory hacks
Kludge
A kludge is a workaround, a quick-and-dirty solution, a clumsy or inelegant, yet effective, solution to a problem, typically using parts that are cobbled together...

 to facilitate view control.
Though a violation of the TrackIR software's EULA,
it is possible to use it with FreeTrack.

NaturalPoint's TrackIR interface SDK is only available under a signed license agreement
and is covered by a NDA
Non-disclosure agreement
A non-disclosure agreement , also known as a confidentiality agreement , confidential disclosure agreement , proprietary information agreement , or secrecy agreement, is a legal contract between at least two parties that outlines confidential material, knowledge, or information that the parties...

, so while FreeTrack is open source, the TrackIR interface component is required to be closed source.

External links

The source of this article is wikipedia, the free encyclopedia.  The text of this article is licensed under the GFDL.
 
x
OK