Video Content Analysis is the capability of automatically analyzing video

Video

Video is the technology of electronically capturing, recording, processing, storing, transmitting, and reconstructing a sequence of still images representing scenes in motion.- History :...

 to detect and determine temporal events not based on a single image

Image

An image is an artifact, for example a two-dimensional picture, that has a similar appearance to some subject—usually a physical object or a person.-Characteristics:...

. As such, it can be seen as the automated equivalent of the biological visual cortex

Visual cortex

The visual cortex of the brain is the part of the cerebral cortex responsible for processing visual information. It is located in the occipital lobe, in the back of the brain....

.

This technical capability is used in a wide range of domains including entertainment

Entertainment

Entertainment consists of any activity which provides a diversion or permits people to amuse themselves in their leisure time. Entertainment is generally passive, such as watching opera or a movie. Active forms of amusement, such as sports, are more often considered to be recreation...

, health care

Health care

Health care is the diagnosis, treatment, and prevention of disease, illness, injury, and other physical and mental impairments in humans. Health care is delivered by practitioners in medicine, chiropractic, dentistry, nursing, pharmacy, allied health, and other care providers...

, retail

Retail

Retail consists of the sale of physical goods or merchandise from a fixed location, such as a department store, boutique or kiosk, or by mail, in small or individual lots for direct consumption by the purchaser. Retailing may include subordinated services, such as delivery. Purchasers may be...

, automotive, transport

Transport

Transport or transportation is the movement of people, cattle, animals and goods from one location to another. Modes of transport include air, rail, road, water, cable, pipeline, and space. The field can be divided into infrastructure, vehicles, and operations...

, home automation

Home automation

Home automation is the residential extension of "building automation". It is automation of the home, housework or household activity. Home automation may include centralized control of lighting, HVAC , appliances, and other systems, to provide improved convenience, comfort, energy efficiency and...

, safety

Safety

Safety is the state of being "safe" , the condition of being protected against physical, social, spiritual, financial, political, emotional, occupational, psychological, educational or other types or consequences of failure, damage, error, accidents, harm or any other event which could be...

 and security

Security

Security is the degree of protection against danger, damage, loss, and crime. Security as a form of protection are structures and processes that provide or improve security as a condition. The Institute for Security and Open Methodologies in the OSSTMM 3 defines security as "a form of protection...

. The algorithm

Algorithm

In mathematics and computer science, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function. Algorithms are used for calculation, data processing, and automated reasoning...

s can be implemented as software on general purpose machines, or as hardware in specialized video processing units.

Many different functionalities can be implemented in VCA. Video Motion Detection is one of the simpler forms where motion is detected with regard to a fixed background scene. More advanced functionalities include video tracking

Video tracking

Video tracking is the process of locating a moving object over time using a camera. It has a variety of uses, some of which are: human-computer interaction, security and surveillance, video communication and compression, augmented reality, traffic control, medical imaging and video editing...

 and egomotion

Egomotion

Egomotion is defined as the 3D motion of a camera within an environment. In the field of computer vision, egomotion refers to estimating a camera's motion relative to a rigid scene. An example of egomotion estimation would be estimating a car's moving position relative to lines on the road or...

 estimation.

Based on the internal representation that VCA generates in the machine, it is possible to build other functionalities, such as identification

Identification

Identification or Identify may refer to:* Body identification* Combat Identification* Eyewitness identification* Forensic identification* Gender identity* Hazard Identification...

, behavior

Behavior

Behavior or behaviour refers to the actions and mannerisms made by organisms, systems, or artificial entities in conjunction with its environment, which includes the other systems or organisms around as well as the physical environment...

 analysis or other forms of situation awareness

Situation awareness

Situation awareness, situational awareness, or SA, is the perception of environmental elements with respect to time and/or space, the comprehension of their meaning, and the projection of their status after some variable has changed, such as time...

.

VCA relies on good input video, so it is often combined with video enhancement technologies such as video denoising

Video denoising

Video denoising is the process of removing noise from a video signal. Video denoising methods can be divided into:* Spatial video denoising methods, where image noise reduction is applied to each frame individually....

, image stabilization

Image stabilization

Image stabilization is a family of techniques used to reduce blurring associated with the motion of a camera during exposure. Specifically, it compensates for pan and tilt of a camera or other imaging device. It is used in image-stabilized binoculars, still and video cameras, and astronomical...

, unsharp masking

Unsharp masking

Unsharp masking is an image manipulation technique, often available in digital image processing software.The "unsharp" of the name derives from the fact that the technique uses a blurred, or "unsharp," positive to create a "mask" of the original image...

 and super-resolution

Super-resolution

Super-resolution are techniques that enhance the resolution of an imaging system. Some SR techniques break the diffraction-limit of systems, while other SR techniques improve over the resolution of digital imaging sensor....

.

Functionalities

Several articles provide an overview of the modules involved in the development of video analytic applications.
This is a list of known functionalities and a short description.

Function	Description
Dynamic Masking	Blocking a part of the video signal based on the signal itself, for example because of privacy concerns.
Egomotion estimation Egomotion Egomotion is defined as the 3D motion of a camera within an environment. In the field of computer vision, egomotion refers to estimating a camera's motion relative to a rigid scene. An example of egomotion estimation would be estimating a car's moving position relative to lines on the road or...	Egomotion estimation is used to determine the location of a camera by analyzing its output signal.
Motion detection	Motion detection is used to determine the presence of relevant motion in the observed scene.
Object detection	Object detection is used to determine the presence of a type of object or entity, for example a person or car. Other examples include fire and smoke detection.
Recognition	Face recognition Facial recognition system A facial recognition system is a computer application for automatically identifying or verifying a person from a digital image or a video frame from a video source...  and Automatic Number Plate Recognition Automatic number plate recognition Automatic number plate recognition is a mass surveillance method that uses optical character recognition on images to read the license plates on vehicles. They can use existing closed-circuit television or road-rule enforcement cameras, or ones specifically designed for the task...  are used to recognize, and therefore possibly identify persons or cars.
Style detection	Style detection is used in settings where the video signal has been produced, for example for television broadcast. Style detection detects the style of the production process.
Tamper detection	Tamper detection is used to determine whether the camera or output signal is tampered with.
Video tracking Video tracking Video tracking is the process of locating a moving object over time using a camera. It has a variety of uses, some of which are: human-computer interaction, security and surveillance, video communication and compression, augmented reality, traffic control, medical imaging and video editing...	Video tracking is used to determine the location of persons or objects in the video signal, possibly with regard to an external reference grid.

Commercial applications

VCA is a new technology. New applications are frequently found, however the track record of different types of VCA differs widely. Functionalities such as motion detection and people counting are believed to be available as Commercial off-the-shelf

Commercial off-the-shelf

In the United States, Commercially available Off-The-Shelf is a Federal Acquisition Regulation term defining a nondevelopmental item of supply that is both commercial and sold in substantial quantities in the commercial marketplace, and that can be procured or utilized under government contract...

 products with a decent track-record.

In many domains VCA is implemented on CCTV

Closed-circuit television

Closed-circuit television is the use of video cameras to transmit a signal to a specific place, on a limited set of monitors....

 systems, either distributed on the cameras (at-the-edge) or centralized on dedicated processing systems. Video Analytics and Smart CCTV are commercial terms for VCA in the security domain. In the UK the BSIA has developed an introduction guide for VCA in the security domain. In addition to video analytics and to complement it audio analytics can also be used .

Kinect is an add-on peripheral for the Xbox 360 gaming console that uses VCA for part of the user input.

The quality of VCA in the commercial setting is difficult to determine. It depends on many variables such as use case

Use case

In software engineering and systems engineering, a use case is a description of steps or actions between a user and a software system which leads the user towards something useful...

, implementation

Implementation

Implementation is the realization of an application, or execution of a plan, idea, model, design, specification, standard, algorithm, or policy.-Computer Science:...

, system configuration

Computer configuration

In communications or computer systems, a configuration is an arrangement of functional units according to their nature, number, and chief characteristics. Often, configuration pertains to the choice of hardware, software, firmware, and documentation...

 and computing platform. Typical methods to get an objective idea of the quality in commercial settings include independent benchmarking

Benchmarking

Benchmarking is the process of comparing one's business processes and performance metrics to industry bests and/or best practices from other industries. Dimensions typically measured are quality, time and cost...

 and designated test-locations.

Academic research

Video Content Analysis is a subset of Computer Vision

Computer vision

Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...

 and thereby of Artificial Intelligence

Artificial intelligence

Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...

. Two major academic benchmark initiatives are TRECVID

TRECVID

The TRECVID evaluation meetings are on-going series of workshops focusing on a list of different information retrieval research areas in content based retrieval of video. It is co-sponsored by the National Institute of Standards and Technology and the Intelligence Advanced Projects Activity of...

 and the PETS Benchmark Data. They focus on functionalities such as tracking, left luggage detection and virtual fencing.
Significant academic research into the field is ongoing at the LIVS, University of Calgary

University of Calgary

The University of Calgary is a public research university located in Calgary, Alberta, Canada. Founded in 1966 the U of C is composed of 14 faculties and more than 85 research institutes and centres.More than 25,000 undergraduate and 5,500 graduate students are currently...

, University of Waterloo

University of Waterloo

The University of Waterloo is a comprehensive public university in the city of Waterloo, Ontario, Canada. The school was founded in 1957 by Drs. Gerry Hagey and Ira G. Needles, and has since grown to an institution of more than 30,000 students, faculty, and staff...

, University of Kingston, Georgia Institute of Technology

Georgia Institute of Technology

The Georgia Institute of Technology is a public research university in Atlanta, Georgia, in the United States...

, Carnegie Mellon University

Carnegie Mellon University

Carnegie Mellon University is a private research university in Pittsburgh, Pennsylvania, United States....

, West Virginia University

West Virginia University

West Virginia University is a public research university in Morgantown, West Virginia, USA. Other campuses include: West Virginia University at Parkersburg in Parkersburg; West Virginia University Institute of Technology in Montgomery; Potomac State College of West Virginia University in Keyser;...

, and The British Columbia Institute of Technology

British Columbia Institute of Technology

The British Columbia Institute of Technology , is a public, coeducational, academic institution of higher education in Burnaby, British Columbia, Canada. The polytechnic has five campuses located in the Metro Vancouver region, with the main campus in Burnaby...

.