Automatic image annotation
Encyclopedia
Automatic image annotation (also known as automatic image tagging) is the process by which a computer system automatically assigns metadata
in the form of caption
ing or keywords to a digital image. This application of computer vision
techniques is used in image retrieval
systems to organize and locate images of interest from a database
.
This method can be regarded as a type of multi-class
image classification with a very large number of classes - as large as the vocabulary size. Typically, image analysis
in the form of extracted feature vectors and the training annotation words are used by machine learning
techniques to attempt to automatically apply annotations to new images. The first methods learned the correlations between image features and training annotations, then techniques were developed using machine translation to try and translate the textual vocabulary with the 'visual vocabulary', or clustered regions known as blobs. Work following these efforts have included classification approaches, relevance models and so on.
The advantages of automatic image annotation versus content-based image retrieval
are that queries can be more naturally specified by the user http://research.nii.ac.jp/~m-inoue/paper/inoue04irix.pdf. CBIR generally (at present) requires users to search by image concepts such as color and texture, or finding example queries. Certain image features in example images may override the concept that the user is really focusing on. The traditional methods of image retrieval such as those used by libraries have relied on manually annotated images, which is expensive and time-consuming, especially given the large and constantly-growing image databases in existence.
Some annotation engines are online, including the ALIPR.com real-time tagging engine developed by Penn State researchers, and Behold - an image search engine that indexes over 1 million Flickr images using automatically generated tags.
Metadata
The term metadata is an ambiguous term which is used for two fundamentally different concepts . Although the expression "data about data" is often used, it does not apply to both in the same way. Structural metadata, the design and specification of data structures, cannot be about data, because at...
in the form of caption
Caption
Caption may refer to:*Caption , a small press and independent comic convention held annually in Oxford, England*Closed captioning, used to provide the text of a show's audio portion to those who may have trouble hearing it...
ing or keywords to a digital image. This application of computer vision
Computer vision
Computer vision is a field that includes methods for acquiring, processing, analysing, and understanding images and, in general, high-dimensional data from the real world in order to produce numerical or symbolic information, e.g., in the forms of decisions...
techniques is used in image retrieval
Image retrieval
An image retrieval system is a computer system for browsing, searching and retrieving images from a large database of digital images. Most traditional and common methods of image retrieval utilize some method of adding metadata such as captioning, keywords, or descriptions to the images so that...
systems to organize and locate images of interest from a database
Database
A database is an organized collection of data for one or more purposes, usually in digital form. The data are typically organized to model relevant aspects of reality , in a way that supports processes requiring this information...
.
This method can be regarded as a type of multi-class
Multiclass classification
In machine learning, multiclass or multinomial classification is the problem of classifying instances into more than two classes.While some classification algorithms naturally permit the use of more than two classes, others are by nature binary algorithms; these can, however, be turned into...
image classification with a very large number of classes - as large as the vocabulary size. Typically, image analysis
Image analysis
Image analysis is the extraction of meaningful information from images; mainly from digital images by means of digital image processing techniques...
in the form of extracted feature vectors and the training annotation words are used by machine learning
Machine learning
Machine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases...
techniques to attempt to automatically apply annotations to new images. The first methods learned the correlations between image features and training annotations, then techniques were developed using machine translation to try and translate the textual vocabulary with the 'visual vocabulary', or clustered regions known as blobs. Work following these efforts have included classification approaches, relevance models and so on.
The advantages of automatic image annotation versus content-based image retrieval
Content-based image retrieval
Content-based image retrieval , also known as query by image content and content-based visual information retrieval is the application of computer vision techniques to the image retrieval problem, that is, the problem of searching for digital images in large databases....
are that queries can be more naturally specified by the user http://research.nii.ac.jp/~m-inoue/paper/inoue04irix.pdf. CBIR generally (at present) requires users to search by image concepts such as color and texture, or finding example queries. Certain image features in example images may override the concept that the user is really focusing on. The traditional methods of image retrieval such as those used by libraries have relied on manually annotated images, which is expensive and time-consuming, especially given the large and constantly-growing image databases in existence.
Some annotation engines are online, including the ALIPR.com real-time tagging engine developed by Penn State researchers, and Behold - an image search engine that indexes over 1 million Flickr images using automatically generated tags.
Some major work
- Word co-occurrence model
-
- Annotation as machine translation
-
- Statistical models
-
- Automatic linguistic indexing of pictures
- Hierarchical Aspect Cluster Model
-
- Latent Dirichlet Allocation model
-
- SupervisedSupervised learningSupervised learning is the machine learning task of inferring a function from supervised training data. The training data consist of a set of training examples. In supervised learning, each example is a pair consisting of an input object and a desired output value...
multiclass labeling
- Supervised
-
- Texture similarity
-
- Support Vector Machines
-
- Ensemble of Decision Trees and Random Subwindows
-
- Maximum Entropy
-
- Relevance models
-
- Relevance models using continuous probability density functions
-
- Coherent Language Model
-
- Inference networks
-
- Multiple Bernoulli distribution
-
- Multiple design alternatives
-
- Natural scene annotation
-
- Relevant low-level global filters
-
- Global image features and nonparametric density estimation
-
- Video semantics
-
- Image Annotation Refinement
-
-
- Automatic Image Annotation by Ensemble of Visual Descriptors
-
- Simultaneous Image Classification and Annotation
External links
- ALIPR.com - Real-time automatic tagging engine developed by Penn State researchers.
- Behold Image Search - An image search engine that indexes over 1 million Flickr images using automatically generated tags.
- SpiritTagger Global Photograph Annotation - Annotation system from UCSB on 1.4 million images that predicts where a photo was taken and suggests tags.
-