Generalization error
Encyclopedia
The generalization error of a machine learning
model is a function
that measures how far the student machine is from the teacher machine in average over the entire set of possible data that can be generated by the teacher after each iteration of the learning process. It has this name because this function indicates the capacity of a machine that learns with the specified algorithm to infer a rule (or generalize) that is used by the teacher machine to generate data based only on a few examples.
The theoretical model assumes a probabilistic distribution of the examples, and a function giving the exact target.
The model can also include noise in the example (in the input and/or target output).
The generalization error is usually defined as the expected value of the square of the difference between the learned function and the exact target (mean-square error).
In practical cases, the distribution and target are unknown; statistical estimates are used.
The performance of a machine learning
algorithm
is measured by plots of the generalization error values through the learning process and are called learning curves.
The generalization error of a perceptron
is the probability of the student perceptron to classify an example differently from the teacher and is given by the overlap of
the student and teacher synaptic vectors and is a function of their scalar product.
Machine learning
Machine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases...
model is a function
Function (mathematics)
In mathematics, a function associates one quantity, the argument of the function, also known as the input, with another quantity, the value of the function, also known as the output. A function assigns exactly one output to each input. The argument and the value may be real numbers, but they can...
that measures how far the student machine is from the teacher machine in average over the entire set of possible data that can be generated by the teacher after each iteration of the learning process. It has this name because this function indicates the capacity of a machine that learns with the specified algorithm to infer a rule (or generalize) that is used by the teacher machine to generate data based only on a few examples.
The theoretical model assumes a probabilistic distribution of the examples, and a function giving the exact target.
The model can also include noise in the example (in the input and/or target output).
The generalization error is usually defined as the expected value of the square of the difference between the learned function and the exact target (mean-square error).
In practical cases, the distribution and target are unknown; statistical estimates are used.
The performance of a machine learning
Machine learning
Machine learning, a branch of artificial intelligence, is a scientific discipline concerned with the design and development of algorithms that allow computers to evolve behaviors based on empirical data, such as from sensor data or databases...
algorithm
Algorithm
In mathematics and computer science, an algorithm is an effective method expressed as a finite list of well-defined instructions for calculating a function. Algorithms are used for calculation, data processing, and automated reasoning...
is measured by plots of the generalization error values through the learning process and are called learning curves.
The generalization error of a perceptron
Perceptron
The perceptron is a type of artificial neural network invented in 1957 at the Cornell Aeronautical Laboratory by Frank Rosenblatt. It can be seen as the simplest kind of feedforward neural network: a linear classifier.- Definition :...
is the probability of the student perceptron to classify an example differently from the teacher and is given by the overlap of
the student and teacher synaptic vectors and is a function of their scalar product.
See also
- Generalization (logic)Generalization (logic)In mathematical logic, generalization is an inference rule of predicate calculus. It states that if \vdash P has been derived, then \vdash \forall x \, P can be derived....
- Hasty generalizationHasty generalizationHasty generalization is a logical fallacy of faulty generalization by reaching an inductive generalization based on insufficient evidence essentially making a hasty conclusion without considering all of the variables...
- Categorical imperativeCategorical imperativeThe Categorical Imperative is the central philosophical concept in the moral philosophy of Immanuel Kant, as well as modern deontological ethics...
redirected from "Generalization in Ethics" - Generalizations of Fibonacci numbersGeneralizations of Fibonacci numbersIn mathematics, the Fibonacci numbers form a sequence defined recursively by:The Fibonacci sequence has been studied extensively and generalized in many ways, for example, by starting with other numbers than 0 and 1, by adding more than two numbers to generate the next number, or by adding objects...
- Generalizations of Pauli matricesGeneralizations of Pauli matricesIn mathematics and physics, in particular quantum information, the term generalized Pauli matrices refers to families of matrices which generalize the properties of the Pauli matrices. In this article we give a few classes of such matrices....
- Derivative (generalizations)Derivative (generalizations)The derivative is a fundamental construction of differential calculus and admits many possible generalizations within the fields of mathematical analysis, combinatorics, algebra, and geometry.- Derivatives in analysis :...
- RandomnessRandomnessRandomness has somewhat differing meanings as used in various fields. It also has common meanings which are connected to the notion of predictability of events....