Wald test
Encyclopedia
The Wald test is a parametric
statistical test named after Abraham Wald
with a great variety of uses. Whenever a relationship within or between data items can be expressed as a statistical model with parameters to be estimated from a sample, the Wald test can be used to test the true value of the parameter based on the sample estimate.
For example, suppose an economist, who has data on social class and shoe size, wonders whether social class is associated with shoe size. Say θ is the average increase in shoe size for upper class people compared to middle class people: then the Wald test can be used to test whether θ is 0 (in which case social class has no association with shoe size) or non-zero (shoe size varies between social classes). Here, θ, the hypothetical difference in shoe sizes between upper and middle class people in the whole population, is a parameter. An estimate of θ might be the difference in shoe size between upper and middle class people in the sample. In the Wald test the economist uses the estimate and an estimate of variability (see below) to draw conclusions about the unobserved true θ. Or, for a medical example, suppose smoking multiplies the risk of lung cancer by some number R: then the Wald test can be used to test whether R = 1 (i.e. there is no effect of smoking) or is greater (or less) than 1 (i.e. smoking alters risk).
A Wald test can be used in a great variety of different models including models for dichotomous variables and models for continuous variable
s.
estimate
of the parameter(s) of interest 
is compared with the proposed value 
, with the assumption that the difference between the two will be approximately normal. Typically the square of the difference is compared to a chi-squared distribution. In the univariate case, the Wald statistic
is
which is compared against a chi-squared distribution.
Alternatively, the difference can be compared to a normal distribution. In this case the test statistic is
where
is the standard error
of the maximum likelihood estimate. A reasonable estimate of the standard error for the MLE can be given by
, where 
is the Fisher information
of the parameter.
In the multivariate case, a test about several parameters at once is carried out using a variance matrix. A common use for this is to carry out a Wald test on a categorical variable by recoding it as several dichotomous variables.
can also be used to test whether an effect exists or not. Usually the Wald test and the likelihood ratio test give very similar conclusions (as they are asymptotically equivalent), but very rarely, they disagree enough to lead to different conclusions: the researcher finds him/herself asking, or being asked, why the p-value
is significant when the confidence interval includes 0, or why the p-value is not significant when the confidence interval excludes 0. In this situation, first remember that statistical significance
is always somewhat arbitrary, as it depends on an arbitrarily chosen significance level.
There are several reasons to prefer the likelihood ratio test to the Wald test.
One is that the Wald test can give different answers to the same question, according to how the question is phrased. For example, asking whether R = 1 is the same as asking whether log R = 0; but the Wald statistic for R = 1 is not the same as the Wald statistic for log R = 0 (because there is in general no neat relationship between the standard errors of R and log R). Likelihood ratio tests will give exactly the same answer whether we work with R, log R or any other monotonic transformation of R. The other reason is that the Wald test uses two approximations (that we know the standard error, and that the distribution is chi-squared
), whereas the likelihood ratio test uses one approximation (that the distribution is chi-squared).
Yet another alternative is the score test
, which has the advantage that it can be formulated in situations where the variability is difficult to estimate; e.g. the Cochran–Mantel–Haenzel test is a score test.
showed that the Wald test, the likelihood-ratio test
and the Lagrange multiplier test
are asymptotically equivalent
.
Parametric
Parametric may refer to:*Parametric equation*Parametric statistics*Parametric derivative*Parametric plot*Parametric model*Parametric oscillator *Parametric contract*Parametric insurance*Parametric feature based modeler...
statistical test named after Abraham Wald
Abraham Wald
- See also :* Sequential probability ratio test * Wald distribution* Wald–Wolfowitz runs test...
with a great variety of uses. Whenever a relationship within or between data items can be expressed as a statistical model with parameters to be estimated from a sample, the Wald test can be used to test the true value of the parameter based on the sample estimate.
For example, suppose an economist, who has data on social class and shoe size, wonders whether social class is associated with shoe size. Say θ is the average increase in shoe size for upper class people compared to middle class people: then the Wald test can be used to test whether θ is 0 (in which case social class has no association with shoe size) or non-zero (shoe size varies between social classes). Here, θ, the hypothetical difference in shoe sizes between upper and middle class people in the whole population, is a parameter. An estimate of θ might be the difference in shoe size between upper and middle class people in the sample. In the Wald test the economist uses the estimate and an estimate of variability (see below) to draw conclusions about the unobserved true θ. Or, for a medical example, suppose smoking multiplies the risk of lung cancer by some number R: then the Wald test can be used to test whether R = 1 (i.e. there is no effect of smoking) or is greater (or less) than 1 (i.e. smoking alters risk).
A Wald test can be used in a great variety of different models including models for dichotomous variables and models for continuous variable
Variable (mathematics)
In mathematics, a variable is a value that may change within the scope of a given problem or set of operations. In contrast, a constant is a value that remains unchanged, though often unknown or undetermined. The concepts of constants and variables are fundamental to many areas of mathematics and...
s.
Mathematical details
Under the Wald statistical test, the maximum likelihoodMaximum likelihood
In statistics, maximum-likelihood estimation is a method of estimating the parameters of a statistical model. When applied to a data set and given a statistical model, maximum-likelihood estimation provides estimates for the model's parameters....
estimate
of the parameter(s) of interest is compared with the proposed value , with the assumption that the difference between the two will be approximately normal. Typically the square of the difference is compared to a chi-squared distribution. In the univariate case, the Wald statisticStatistic
A statistic is a single measure of some attribute of a sample . It is calculated by applying a function to the values of the items comprising the sample which are known together as a set of data.More formally, statistical theory defines a statistic as a function of a sample where the function...
is
which is compared against a chi-squared distribution.
Alternatively, the difference can be compared to a normal distribution. In this case the test statistic is
where
is the standard errorStandard error (statistics)
The standard error is the standard deviation of the sampling distribution of a statistic. The term may also be used to refer to an estimate of that standard deviation, derived from a particular sample used to compute the estimate....
of the maximum likelihood estimate. A reasonable estimate of the standard error for the MLE can be given by
, where is the Fisher informationFisher information
In mathematical statistics and information theory, the Fisher information is the variance of the score. In Bayesian statistics, the asymptotic distribution of the posterior mode depends on the Fisher information and not on the prior...
of the parameter.
In the multivariate case, a test about several parameters at once is carried out using a variance matrix. A common use for this is to carry out a Wald test on a categorical variable by recoding it as several dichotomous variables.
Alternatives to the Wald test
The likelihood-ratio testLikelihood-ratio test
In statistics, a likelihood ratio test is a statistical test used to compare the fit of two models, one of which is a special case of the other . The test is based on the likelihood ratio, which expresses how many times more likely the data are under one model than the other...
can also be used to test whether an effect exists or not. Usually the Wald test and the likelihood ratio test give very similar conclusions (as they are asymptotically equivalent), but very rarely, they disagree enough to lead to different conclusions: the researcher finds him/herself asking, or being asked, why the p-value
P-value
In statistical significance testing, the p-value is the probability of obtaining a test statistic at least as extreme as the one that was actually observed, assuming that the null hypothesis is true. One often "rejects the null hypothesis" when the p-value is less than the significance level α ,...
is significant when the confidence interval includes 0, or why the p-value is not significant when the confidence interval excludes 0. In this situation, first remember that statistical significance
Statistical significance
In statistics, a result is called statistically significant if it is unlikely to have occurred by chance. The phrase test of significance was coined by Ronald Fisher....
is always somewhat arbitrary, as it depends on an arbitrarily chosen significance level.
There are several reasons to prefer the likelihood ratio test to the Wald test.
One is that the Wald test can give different answers to the same question, according to how the question is phrased. For example, asking whether R = 1 is the same as asking whether log R = 0; but the Wald statistic for R = 1 is not the same as the Wald statistic for log R = 0 (because there is in general no neat relationship between the standard errors of R and log R). Likelihood ratio tests will give exactly the same answer whether we work with R, log R or any other monotonic transformation of R. The other reason is that the Wald test uses two approximations (that we know the standard error, and that the distribution is chi-squared
Chi-squared
In statistics, the term chi-squared has different uses:*chi-squared distribution, a continuous probability distribution;*chi-squared statistic, a statistic used in some statistical tests;...
), whereas the likelihood ratio test uses one approximation (that the distribution is chi-squared).
Yet another alternative is the score test
Score test
A score test is a statistical test of a simple null hypothesis that a parameter of interest \theta isequal to some particular value \theta_0. It is the most powerful test when the true value of \theta is close to \theta_0. The main advantage of the Score-test is that it does not require an...
, which has the advantage that it can be formulated in situations where the variability is difficult to estimate; e.g. the Cochran–Mantel–Haenzel test is a score test.
Asymptotic properties
Robert EngleRobert F. Engle
Robert Fry Engle III is an American economist and the winner of the 2003 Nobel Memorial Prize in Economic Sciences, sharing the award with Clive Granger, "for methods of analyzing economic time series with time-varying volatility ".-Biography:Engle was born in Syracuse, New York and went on to...
showed that the Wald test, the likelihood-ratio test
Likelihood-ratio test
In statistics, a likelihood ratio test is a statistical test used to compare the fit of two models, one of which is a special case of the other . The test is based on the likelihood ratio, which expresses how many times more likely the data are under one model than the other...
and the Lagrange multiplier test
Score test
A score test is a statistical test of a simple null hypothesis that a parameter of interest \theta isequal to some particular value \theta_0. It is the most powerful test when the true value of \theta is close to \theta_0. The main advantage of the Score-test is that it does not require an...
are asymptotically equivalent
Asymptotic distribution
In mathematics and statistics, an asymptotic distribution is a hypothetical distribution that is in a sense the "limiting" distribution of a sequence of distributions...
.