Additive model
Encyclopedia
In statistics
, an additive model (AM) is a nonparametric regression
method. It was suggested by Jerome H. Friedman and Werner Stuetzle (1981) and is an essential part of the ACE
algorithm. The AM uses a one dimensional smoother
to build a restricted class of nonparametric regression models. Because of this, it is less affected by the curse of dimensionality
than e.g. a p-dimensional smoother. Furthermore, the AM is more flexible than a standard linear model
, while being more interpretable than a general regression surface at the cost of approximation errors. Problems with AM include model selection
, overfitting
, and multicollinearity
.
set
of n statistical unit
s, where
represent predictors and 
is the outcome, the additive model takes the form
or
Where
, 
and 
. The functions 
are unknown smooth functions
fit from the data. Fitting the AM (i.e. the functions
) can be done using the backfitting algorithm
proposed by Andreas Buja, Trevor Hastie and Robert Tibshirani (1989).
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
, an additive model (AM) is a nonparametric regression
Nonparametric regression
Nonparametric regression is a form of regression analysis in which the predictor does not take a predetermined form but is constructed according to information derived from the data...
method. It was suggested by Jerome H. Friedman and Werner Stuetzle (1981) and is an essential part of the ACE
Ace
An ace is a playing card. In the standard French deck, an ace has a single suit symbol located in the middle of the card, sometimes large and decorated, especially in the case of the Ace of Spades...
algorithm. The AM uses a one dimensional smoother
Smoothing
In statistics and image processing, to smooth a data set is to create an approximating function that attempts to capture important patterns in the data, while leaving out noise or other fine-scale structures/rapid phenomena. Many different algorithms are used in smoothing...
to build a restricted class of nonparametric regression models. Because of this, it is less affected by the curse of dimensionality
Curse of dimensionality
The curse of dimensionality refers to various phenomena that arise when analyzing and organizing high-dimensional spaces that do not occur in low-dimensional settings such as the physical space commonly modeled with just three dimensions.There are multiple phenomena referred to by this name in...
than e.g. a p-dimensional smoother. Furthermore, the AM is more flexible than a standard linear model
Linear regression
In statistics, linear regression is an approach to modeling the relationship between a scalar variable y and one or more explanatory variables denoted X. The case of one explanatory variable is called simple regression...
, while being more interpretable than a general regression surface at the cost of approximation errors. Problems with AM include model selection
Model selection
Model selection is the task of selecting a statistical model from a set of candidate models, given data. In the simplest cases, a pre-existing set of data is considered...
, overfitting
Overfitting
In statistics, overfitting occurs when a statistical model describes random error or noise instead of the underlying relationship. Overfitting generally occurs when a model is excessively complex, such as having too many parameters relative to the number of observations...
, and multicollinearity
Multicollinearity
Multicollinearity is a statistical phenomenon in which two or more predictor variables in a multiple regression model are highly correlated. In this situation the coefficient estimates may change erratically in response to small changes in the model or the data...
.
Description
Given a dataData
The term data refers to qualitative or quantitative attributes of a variable or set of variables. Data are typically the results of measurements and can be the basis of graphs, images, or observations of a set of variables. Data are often viewed as the lowest level of abstraction from which...
set
of n statistical unitStatistical unit
A unit in a statistical analysis refers to one member of a set of entities being studied. It is the material source for the mathematical abstraction of a "random variable"...
s, where
represent predictors and is the outcome, the additive model takes the form
or
Where
, and . The functions are unknown smooth functionsSmooth function
In mathematical analysis, a differentiability class is a classification of functions according to the properties of their derivatives. Higher order differentiability classes correspond to the existence of more derivatives. Functions that have derivatives of all orders are called smooth.Most of...
fit from the data. Fitting the AM (i.e. the functions
) can be done using the backfitting algorithmBackfitting algorithm
In statistics, the backfitting algorithm is a simple iterative procedure used to fit a generalized additive model. It was introduced in 1985 by Leo Breiman and Jerome Friedman along with generalized additive models...
proposed by Andreas Buja, Trevor Hastie and Robert Tibshirani (1989).
See also
- Generalized additive modelGeneralized additive modelIn statistics, the generalized additive model is a statistical model developed by Trevor Hastie and Rob Tibshirani for blending properties of generalized linear models with additive models....
- Backfitting algorithmBackfitting algorithmIn statistics, the backfitting algorithm is a simple iterative procedure used to fit a generalized additive model. It was introduced in 1985 by Leo Breiman and Jerome Friedman along with generalized additive models...
- Alternating conditional expectation model
- Projection pursuit regressionProjection pursuit regressionIn statistics, projection pursuit regression is a statistical model developed by Jerome H. Friedman and Werner Stuetzle which is an extension of additive models...
- Generalized additive model for location, scale, and shape (GAMLSS)