Instance-based learning
Encyclopedia
In machine learning
, instance-based learning or memory-based learning is a family of learning algorithms that, instead of performing explicit generalization, compare new problem instances with instances seen in training, which have been stored in memory. Instance-based learning is a kind of lazy learning
.
It is called instance-based because it constructs hypotheses directly from the training instances themselves.
This means that the hypothesis complexity can grow with the data: in the worst case, a hypothesis is a list of n training items and classification takes O
(n). One advantage that instance-based learning has over other methods of machine learning is its ability to adapt its model to previously unseen data. Where other methods generally require the entire set of training data to be re-examined when one instance is changed, instance-based learners may simply store a new instance or throw an old instance away.
A simple example of an instance-based learning algorithm is the k-nearest neighbor algorithm
. Daelemans and Van den Bosch describe variations of this algorithm for use in natural language processing
(NLP), claiming that memory-based learning is both more psychologically realistic than other machine-learning schemes and more effective in practice.
Machine learning
Machine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases...
, instance-based learning or memory-based learning is a family of learning algorithms that, instead of performing explicit generalization, compare new problem instances with instances seen in training, which have been stored in memory. Instance-based learning is a kind of lazy learning
Lazy learning
In artificial intelligence, lazy learning is a learning method in which generalization beyond the training data is delayed until a query is made to the system, as opposed to in eager learning, where the system tries to generalize the training data before receiving queries.The main advantage gained...
.
It is called instance-based because it constructs hypotheses directly from the training instances themselves.
This means that the hypothesis complexity can grow with the data: in the worst case, a hypothesis is a list of n training items and classification takes O
Big O notation
In mathematics, big O notation is used to describe the limiting behavior of a function when the argument tends towards a particular value or infinity, usually in terms of simpler functions. It is a member of a larger family of notations that is called Landau notation, Bachmann-Landau notation, or...
(n). One advantage that instance-based learning has over other methods of machine learning is its ability to adapt its model to previously unseen data. Where other methods generally require the entire set of training data to be re-examined when one instance is changed, instance-based learners may simply store a new instance or throw an old instance away.
A simple example of an instance-based learning algorithm is the k-nearest neighbor algorithm
K-nearest neighbor algorithm
In pattern recognition, the k-nearest neighbor algorithm is a method for classifying objects based on closest training examples in the feature space. k-NN is a type of instance-based learning, or lazy learning where the function is only approximated locally and all computation is deferred until...
. Daelemans and Van den Bosch describe variations of this algorithm for use in natural language processing
Natural language processing
Natural language processing is a field of computer science and linguistics concerned with the interactions between computers and human languages; it began as a branch of artificial intelligence....
(NLP), claiming that memory-based learning is both more psychologically realistic than other machine-learning schemes and more effective in practice.
External links
- TiMBL, the Tilburg Memory Based Learner is an instance-based learning package geared toward NLPNatural language processingNatural language processing is a field of computer science and linguistics concerned with the interactions between computers and human languages; it began as a branch of artificial intelligence....