Heteroskedasticity
Encyclopedia
In statistics
, a collection of random variable
s is heteroscedastic, or heteroskedastic, if there are sub-populations that have different variabilities than others. Here "variability" could be quantified by the variance
or any other measure of statistical dispersion
. Thus heteroscedasticity is the absence of homoscedasticity
.
The possible existence of heteroscedasticity is a major concern in the application of regression analysis
, including the analysis of variance
, because the presence of heteroscedasticity can invalidate statistical tests of significance
that assume the effect and residual (error) variances are uncorrelated and normally distributed.
The term means "differing variance" and comes from the Greek "hetero" ('different') and "skedasis" ('dispersion').
s of random variables, {Xt}t=1n. In dealing with conditional expectation
s of Yt given Xt, the sequence {Yt}t=1n is said to be heteroskedastic if the conditional variance
of Yt given Xt, changes with t. Some authors refer to this as conditional heteroscedasticity to emphasize the fact that it is the sequence of conditional variance that changes and not the unconditional variance. In fact it is possible to observe conditional heteroscedasticity even when dealing with a sequence of unconditional homoscedastic random variables, however, the opposite does not hold.
When using some statistical techniques, such as ordinary least squares
(OLS), a number of assumptions are typically made. One of these is that the error term has a constant
variance
. This might not be true even if the error term is assumed to be drawn from identical distributions.
For example, the error term could vary or increase with each observation, something that is often the case with cross-sectional
or time series
measurements. Heteroscedasticity is often studied as part of econometrics
, which frequently deals with data exhibiting it. White's influential paper used "heteroskedasticity" instead of "heteroscedasticity" whereas the latter has been used in later works.
, when that null hypothesis was actually uncharacteristic of the actual population (i.e., make a type II error
).
It is widely known that, under certain assumptions, the OLS estimator has a normal asymptotic distribution
when properly normalized and centered (even when the data does not come from a normal distribution). This result is used to justify using a normal distribution, or a chi square distribution (depending on how the test statistic
is calculated), when conducting a hypothesis test. This holds even under heteroscedasticity. More precisely, the OLS estimator in the presence of heteroscedasticity is asymptotically normal, when properly normalized and centered, with a variance-covariance matrix
that differs from the case of homoscedasticity. In 1980, White proposed a consistent estimator
for the variance-covariance matrix of the asymptotic distribution of the OLS estimator. This validates the use of hypothesis testing using OLS estimators and White's variance-covariance estimator under heteroscedasticity.
Heteroscedasticity is also a major practical issue encountered in ANOVA problems.
The F test can still be used in some circumstances.
However, it has been said that students in econometrics
should not overreact to heteroskedasticity. One author wrote, "unequal error variance is worth correcting only when the problem is severe." And another word of caution was in the form, "heteroscedasticity has never been a reason to throw out an otherwise good model."
With the advent of robust standard errors allowing for inference without specifying the conditional second moment of error term, testing conditional homoscedasticity is not as important as in the past.
The econometrician Robert Engle won the 2003 Nobel Memorial Prize for Economics for his studies on regression analysis
in the presence of heteroscedasticity, which led to his formulation of the autoregressive conditional heteroscedasticity (ARCH) modeling technique.
These methods consist, in general, in performing hypothesis tests. These tests consist of a statistic (a mathematical expression), a hypothesis that is going to be tested (the null hypothesis), an alternative hypothesis, and a distributional statement about the statistic (the mathematical expression).
Many introductory statistics and econometrics books, for pedagogical reasons, present these tests under the assumption that the data set in hand comes from a normal distribution. A great misconception is the thought that this assumption is necessary. Most of the methods of detecting heteroscedasticity outlined above can be used even when the data do not come from a normal distribution. In most cases, this assumption can be relaxed by using asymptotic distribution
s which can be obtained from asymptotic theory
.
Statistics
Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....
, a collection of random variable
Random variable
In probability and statistics, a random variable or stochastic variable is, roughly speaking, a variable whose value results from a measurement on some type of random process. Formally, it is a function from a probability space, typically to the real numbers, which is measurable functionmeasurable...
s is heteroscedastic, or heteroskedastic, if there are sub-populations that have different variabilities than others. Here "variability" could be quantified by the variance
Variance
In probability theory and statistics, the variance is a measure of how far a set of numbers is spread out. It is one of several descriptors of a probability distribution, describing how far the numbers lie from the mean . In particular, the variance is one of the moments of a distribution...
or any other measure of statistical dispersion
Statistical dispersion
In statistics, statistical dispersion is variability or spread in a variable or a probability distribution...
. Thus heteroscedasticity is the absence of homoscedasticity
Homoscedasticity
In statistics, a sequence or a vector of random variables is homoscedastic if all random variables in the sequence or vector have the same finite variance. This is also known as homogeneity of variance. The complementary notion is called heteroscedasticity...
.
The possible existence of heteroscedasticity is a major concern in the application of regression analysis
Regression analysis
In statistics, regression analysis includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables...
, including the analysis of variance
Analysis of variance
In statistics, analysis of variance is a collection of statistical models, and their associated procedures, in which the observed variance in a particular variable is partitioned into components attributable to different sources of variation...
, because the presence of heteroscedasticity can invalidate statistical tests of significance
Statistical hypothesis testing
A statistical hypothesis test is a method of making decisions using data, whether from a controlled experiment or an observational study . In statistics, a result is called statistically significant if it is unlikely to have occurred by chance alone, according to a pre-determined threshold...
that assume the effect and residual (error) variances are uncorrelated and normally distributed.
The term means "differing variance" and comes from the Greek "hetero" ('different') and "skedasis" ('dispersion').
Definition
Suppose there is a sequence of random variables {Yt}t=1n and a sequence of vectorTuple
In mathematics and computer science, a tuple is an ordered list of elements. In set theory, an n-tuple is a sequence of n elements, where n is a positive integer. There is also one 0-tuple, an empty sequence. An n-tuple is defined inductively using the construction of an ordered pair...
s of random variables, {Xt}t=1n. In dealing with conditional expectation
Conditional expectation
In probability theory, a conditional expectation is the expected value of a real random variable with respect to a conditional probability distribution....
s of Yt given Xt, the sequence {Yt}t=1n is said to be heteroskedastic if the conditional variance
Conditional variance
In probability theory and statistics, a conditional variance is the variance of a conditional probability distribution. Particularly in econometrics, the conditional variance is also known as the scedastic function or skedastic function...
of Yt given Xt, changes with t. Some authors refer to this as conditional heteroscedasticity to emphasize the fact that it is the sequence of conditional variance that changes and not the unconditional variance. In fact it is possible to observe conditional heteroscedasticity even when dealing with a sequence of unconditional homoscedastic random variables, however, the opposite does not hold.
When using some statistical techniques, such as ordinary least squares
Ordinary least squares
In statistics, ordinary least squares or linear least squares is a method for estimating the unknown parameters in a linear regression model. This method minimizes the sum of squared vertical distances between the observed responses in the dataset and the responses predicted by the linear...
(OLS), a number of assumptions are typically made. One of these is that the error term has a constant
Mathematical constant
A mathematical constant is a special number, usually a real number, that is "significantly interesting in some way". Constants arise in many different areas of mathematics, with constants such as and occurring in such diverse contexts as geometry, number theory and calculus.What it means for a...
variance
Variance
In probability theory and statistics, the variance is a measure of how far a set of numbers is spread out. It is one of several descriptors of a probability distribution, describing how far the numbers lie from the mean . In particular, the variance is one of the moments of a distribution...
. This might not be true even if the error term is assumed to be drawn from identical distributions.
For example, the error term could vary or increase with each observation, something that is often the case with cross-sectional
Cross-sectional study
Cross-sectional studies form a class of research methods that involve observation of all of a population, or a representative subset, at one specific point in time...
or time series
Time series
In statistics, signal processing, econometrics and mathematical finance, a time series is a sequence of data points, measured typically at successive times spaced at uniform time intervals. Examples of time series are the daily closing value of the Dow Jones index or the annual flow volume of the...
measurements. Heteroscedasticity is often studied as part of econometrics
Econometrics
Econometrics has been defined as "the application of mathematics and statistical methods to economic data" and described as the branch of economics "that aims to give empirical content to economic relations." More precisely, it is "the quantitative analysis of actual economic phenomena based on...
, which frequently deals with data exhibiting it. White's influential paper used "heteroskedasticity" instead of "heteroscedasticity" whereas the latter has been used in later works.
Consequences
Heteroscedasticity does not cause ordinary least squares coefficient estimates to be biased, although it can cause ordinary least squares estimates of the variance (and, thus, standard errors) of the coefficients to be biased, possibly above or below the true or population variance. Thus, regression analysis using heteroscedastic data will still provide an unbiased estimate for the relationship between the predictor variable and the outcome, but standard errors and therefore inferences obtained from data analysis are suspect. Biased standard errors lead to biased inference, so results of hypothesis tests are possibly wrong. As an example of the consequence of biased standard error estimation which OLS will produce if heteroskedasticity is present, a researcher might find compelling results against the rejection of a null hypothesis at a given significance level as statistically significantStatistical significance
In statistics, a result is called statistically significant if it is unlikely to have occurred by chance. The phrase test of significance was coined by Ronald Fisher....
, when that null hypothesis was actually uncharacteristic of the actual population (i.e., make a type II error
Type I and type II errors
In statistical test theory the notion of statistical error is an integral part of hypothesis testing. The test requires an unambiguous statement of a null hypothesis, which usually corresponds to a default "state of nature", for example "this person is healthy", "this accused is not guilty" or...
).
It is widely known that, under certain assumptions, the OLS estimator has a normal asymptotic distribution
Asymptotic distribution
In mathematics and statistics, an asymptotic distribution is a hypothetical distribution that is in a sense the "limiting" distribution of a sequence of distributions...
when properly normalized and centered (even when the data does not come from a normal distribution). This result is used to justify using a normal distribution, or a chi square distribution (depending on how the test statistic
Test statistic
In statistical hypothesis testing, a hypothesis test is typically specified in terms of a test statistic, which is a function of the sample; it is considered as a numerical summary of a set of data that...
is calculated), when conducting a hypothesis test. This holds even under heteroscedasticity. More precisely, the OLS estimator in the presence of heteroscedasticity is asymptotically normal, when properly normalized and centered, with a variance-covariance matrix
Matrix (mathematics)
In mathematics, a matrix is a rectangular array of numbers, symbols, or expressions. The individual items in a matrix are called its elements or entries. An example of a matrix with six elements isMatrices of the same size can be added or subtracted element by element...
that differs from the case of homoscedasticity. In 1980, White proposed a consistent estimator
Consistent estimator
In statistics, a sequence of estimators for parameter θ0 is said to be consistent if this sequence converges in probability to θ0...
for the variance-covariance matrix of the asymptotic distribution of the OLS estimator. This validates the use of hypothesis testing using OLS estimators and White's variance-covariance estimator under heteroscedasticity.
Heteroscedasticity is also a major practical issue encountered in ANOVA problems.
The F test can still be used in some circumstances.
However, it has been said that students in econometrics
Econometrics
Econometrics has been defined as "the application of mathematics and statistical methods to economic data" and described as the branch of economics "that aims to give empirical content to economic relations." More precisely, it is "the quantitative analysis of actual economic phenomena based on...
should not overreact to heteroskedasticity. One author wrote, "unequal error variance is worth correcting only when the problem is severe." And another word of caution was in the form, "heteroscedasticity has never been a reason to throw out an otherwise good model."
With the advent of robust standard errors allowing for inference without specifying the conditional second moment of error term, testing conditional homoscedasticity is not as important as in the past.
The econometrician Robert Engle won the 2003 Nobel Memorial Prize for Economics for his studies on regression analysis
Regression analysis
In statistics, regression analysis includes many techniques for modeling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables...
in the presence of heteroscedasticity, which led to his formulation of the autoregressive conditional heteroscedasticity (ARCH) modeling technique.
Detection
There are several methods to test for the presence of heteroscedasticity:- Park test (1966)
- Glejser test (1969)
- White testWhite testIn statistics, the White test is a statistical test that establishes whether the residual variance of a variable in a regression model is constant: that is for homoscedasticity....
(1980) - Breusch–Pagan test
- Goldfeld–Quandt test
- Cook–Weisberg test
- Harrison–McCabe test
- Brown–Forsythe test
- Levene test
These methods consist, in general, in performing hypothesis tests. These tests consist of a statistic (a mathematical expression), a hypothesis that is going to be tested (the null hypothesis), an alternative hypothesis, and a distributional statement about the statistic (the mathematical expression).
Many introductory statistics and econometrics books, for pedagogical reasons, present these tests under the assumption that the data set in hand comes from a normal distribution. A great misconception is the thought that this assumption is necessary. Most of the methods of detecting heteroscedasticity outlined above can be used even when the data do not come from a normal distribution. In most cases, this assumption can be relaxed by using asymptotic distribution
Asymptotic distribution
In mathematics and statistics, an asymptotic distribution is a hypothetical distribution that is in a sense the "limiting" distribution of a sequence of distributions...
s which can be obtained from asymptotic theory
Asymptotic theory (statistics)
In statistics, asymptotic theory, or large sample theory, is a generic framework for assessment of properties of estimators and statistical tests...
.
Fixes
There are three common corrections for heteroscedasticity:- View Logged data. Unlogged series that are growing exponentially often appear to have increasing variability as the series rises over time. The variability in percentage terms may, however, be rather stable.
- Use a different specification for the model (different X variables, or perhaps non-linear transformations of the X variables).
- Apply a weighted least squares estimation method, in which OLS is applied to transformed or weighted values of X and Y. The weights vary over observations, usually depending on the changing error variances. In one variation the weights are directly related to the magnitude of the dependent variable, and this corresponds to least squares percentage regression.
- Heteroscedasticity-consistent standard errorsHeteroscedasticity-consistent standard errorsThe topic of heteroscedasticity-consistent standard errors arises in statistics and econometrics in the context of linear regression and also time series analysis...
(HCSE), while still biased, improve upon OLS estimates (White 1980). HCSE is a consistent estimator of standard errors in regression models with heteroskedasticity. The White method corrects for heteroscedasticity without altering the values of the coefficients. This method may be superior to regular OLS because if heteroscedasticity is present it corrects for it, however, if the data is homoscedastistic, the standard errors are equivalent to conventional standard errors estimated by ols. Several modifications of the White method of computing heteroscedasticity-consistent standard errors have been proposed as corrections with superior finite sample properties.
Examples
Heteroscedasticity often occurs when there is a large difference among the sizes of the observations.- A classic example of heteroscedasticity is that of income versus expenditure on meals. As one's income increases, the variability of food consumption will increase. A poorer person will spend a rather constant amount by always eating less expensive food; a wealthier person may occasionally buy inexpensive food and at other times eat expensive meals. Those with higher incomes display a greater variability of food consumption.
- Imagine you are watching a rocket take off nearby and measuring the distance it has traveled once each second. In the first couple of seconds your measurements may be accurate to the nearest centimeter, say. However, 5 minutes later as the rocket recedes into space, the accuracy of your measurements may only be good to 100 m, because of the increased distance, atmospheric distortion and a variety of other factors. The data you collect would exhibit heteroscedasticity.
Further reading
Most statistics textbooks will include at least some material on heteroscedasticity. Some examples are: (devotes a chapter to heteroscedasticity)- Verbeek, Marno (2004) A Guide to Modern Econometrics, 2. ed., Chichester: John Wiley & Sons
- Greene, W.H. (1993) Econometric Analysis, Prentice–Hall, ISBN 0-13-013297-7, an introductory but thorough general text, considered the standard for a pre-doctorate university Econometrics course;
- Hamilton, J.D. (1994), Time Series Analysis, Princeton University Press ISBN 0-691-04289-6, the text of reference for historical series analysis; it contains an introduction to ARCHArchAn arch is a structure that spans a space and supports a load. Arches appeared as early as the 2nd millennium BC in Mesopotamian brick architecture and their systematic use started with the Ancient Romans who were the first to apply the technique to a wide range of structures.-Technical aspects:The...
models. - Vinod, H.D. (2008) Hands On Intermediate Econometrics Using R: Templates for Extending Dozens of Practical Examples.World Scientific Publishers: Hackensack, NJ. ISBN 978-9812818850 (Section 2.8 provides R snippets)