LogitBoost is a boosting

Boosting

Boosting is a machine learning meta-algorithm for performing supervised learning. Boosting is based on the question posed by Kearns: can a set of weak learners create a single strong learner? A weak learner is defined to be a classifier which is only slightly correlated with the true classification...

 algorithm formulated by Jerome Friedman, Trevor Hastie, and Robert Tibshirani

Robert Tibshirani

Robert Tibshirani is a Professor in the Departments of Statistics and Health Research and Policy at Stanford University. He was a Professor at the University of Toronto from 1985 to 1998. In his work, he develops statistical tools for the analysis of complex datasets, most recently in genomics and...

. The original paper casts the AdaBoost

AdaBoost

AdaBoost, short for Adaptive Boosting, is a machine learning algorithm, formulated by Yoav Freund and Robert Schapire. It is a meta-algorithm, and can be used in conjunction with many other learning algorithms to improve their performance. AdaBoost is adaptive in the sense that subsequent...

 algorithm into a statistics framework. Specifically, if one considers AdaBoost

AdaBoost

AdaBoost, short for Adaptive Boosting, is a machine learning algorithm, formulated by Yoav Freund and Robert Schapire. It is a meta-algorithm, and can be used in conjunction with many other learning algorithms to improve their performance. AdaBoost is adaptive in the sense that subsequent...

 as a generalized additive model

Generalized additive model

In statistics, the generalized additive model is a statistical model developed by Trevor Hastie and Rob Tibshirani for blending properties of generalized linear models with additive models....

 and then applies the cost functional of logistic regression

Logistic regression

In statistics, logistic regression is used for prediction of the probability of occurrence of an event by fitting data to a logit function logistic curve. It is a generalized linear model used for binomial regression...

, one can derive the LogitBoost algorithm.

Minimizing the LogitBoost cost functional

LogitBoost can be seen as a convex optimization. Specifically, given that we seek an additive model of the form


the LogitBoost algorithm minimizes the logistic loss: