Residual sum of squares
Encyclopedia
In statistics
, the residual sum of squares (RSS) is the sum
of squares of residuals
. It is also known as the sum of squared residuals (SSR) or the sum of squared errors of prediction (SSE). It is a measure of the discrepancy between the data and an estimation model. A small RSS indicates a tight fit of the model to the data.
In general, total sum of squares
= explained sum of squares
+ residual sum of squares. For a proof of this in the multivariate ordinary least squares
(OLS) case, see partitioning in the general OLS model.
where yi is the i th value of the variable to be predicted, xi is the i th value of the explanatory variable, and
is the predicted value of yi.
In a standard linear simple regression model,
, where a and b are coefficient
s, y and x are the regressand and the regressor, respectively, and ε is the error term
. The sum of squares of residuals is the sum of squares of estimates
of εi; that is
where
is the estimated value of the constant term 
and 
is the estimated value of the slope coefficient b.
where y is an n × 1 vector of dependent variable observations, each column of the n × k matrix X is a vector of observations on one of the k explanators,
is a k × 1 vector of true coefficients, and e is an n× 1 vector of the true underlying errors. The ordinary least squares
estimator for
is
The residual vector
is 
, so the residual sum of squares 
is, after simplification,
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
, the residual sum of squares (RSS) is the sum
SUM
SUM can refer to:* The State University of Management* Soccer United Marketing* Society for the Establishment of Useful Manufactures* StartUp-Manager* Software User’s Manual,as from DOD-STD-2 167A, and MIL-STD-498...
of squares of residuals
Errors and residuals in statistics
In statistics and optimization, statistical errors and residuals are two closely related and easily confused measures of the deviation of a sample from its "theoretical value"...
. It is also known as the sum of squared residuals (SSR) or the sum of squared errors of prediction (SSE). It is a measure of the discrepancy between the data and an estimation model. A small RSS indicates a tight fit of the model to the data.
In general, total sum of squares
Total sum of squares
In statistical data analysis the total sum of squares is a quantity that appears as part of a standard way of presenting results of such analyses...
= explained sum of squares
Explained sum of squares
In statistics, the explained sum of squares is a quantity used in describing how well a model, often a regression model, represents the data being modelled...
+ residual sum of squares. For a proof of this in the multivariate ordinary least squares
Ordinary least squares
In statistics, ordinary least squares or linear least squares is a method for estimating the unknown parameters in a linear regression model. This method minimizes the sum of squared vertical distances between the observed responses in the dataset and the responses predicted by the linear...
(OLS) case, see partitioning in the general OLS model.
One explanatory variable
In a model with a single explanatory variable, RSS is given bywhere yi is the i th value of the variable to be predicted, xi is the i th value of the explanatory variable, and
is the predicted value of yi.In a standard linear simple regression model,
, where a and b are coefficientCoefficient
In mathematics, a coefficient is a multiplicative factor in some term of an expression ; it is usually a number, but in any case does not involve any variables of the expression...
s, y and x are the regressand and the regressor, respectively, and ε is the error term
Errors and residuals in statistics
In statistics and optimization, statistical errors and residuals are two closely related and easily confused measures of the deviation of a sample from its "theoretical value"...
. The sum of squares of residuals is the sum of squares of estimates
Estimator
In statistics, an estimator is a rule for calculating an estimate of a given quantity based on observed data: thus the rule and its result are distinguished....
of εi; that is
where
is the estimated value of the constant term and is the estimated value of the slope coefficient b.Matrix expression for the OLS residual sum of squares
The general regression model with n observations and k explanators, the first of which is a constant unit vector whose coefficient is the regression intercept, iswhere y is an n × 1 vector of dependent variable observations, each column of the n × k matrix X is a vector of observations on one of the k explanators,
is a k × 1 vector of true coefficients, and e is an n× 1 vector of the true underlying errors. The ordinary least squaresOrdinary least squares
In statistics, ordinary least squares or linear least squares is a method for estimating the unknown parameters in a linear regression model. This method minimizes the sum of squared vertical distances between the observed responses in the dataset and the responses predicted by the linear...
estimator for
isThe residual vector
is , so the residual sum of squares is, after simplification,See also
- Sum of squares (statistics)
- Squared deviationsSquared deviationsIn probability theory and statistics, the definition of variance is either the expected value , or average value , of squared deviations from the mean. Computations for analysis of variance involve the partitioning of a sum of squared deviations...
- Errors and residuals in statisticsErrors and residuals in statisticsIn statistics and optimization, statistical errors and residuals are two closely related and easily confused measures of the deviation of a sample from its "theoretical value"...
- Lack-of-fit sum of squaresLack-of-fit sum of squaresIn statistics, a sum of squares due to lack of fit, or more tersely a lack-of-fit sum of squares, is one of the components of a partition of the sum of squares in an analysis of variance, used in the numerator in an F-test of the null hypothesis that says that a proposed model fits well.- Sketch of...