In statistics

Statistics

Statistics is the study of the collection, organization, analysis, and interpretation of data. It deals with all aspects of this, including the planning of data collection in terms of the design of surveys and experiments....

, the closed testing procedure is a general method for performing more than one hypothesis test

Statistical hypothesis testing

A statistical hypothesis test is a method of making decisions using data, whether from a controlled experiment or an observational study . In statistics, a result is called statistically significant if it is unlikely to have occurred by chance alone, according to a pre-determined threshold...

 simultaneously.

The closed testing principle

Suppose there are k hypotheses H₁,..., H_k to be tested and the overall type I error rate is α. The closed testing principle allows the rejection of any one of these elementary hypotheses, say H_i, if all possible intersection hypotheses involving H_i can be rejected by using valid local level α tests. It controls the familywise error rate

Familywise error rate

In statistics, familywise error rate is the probability of making one or more false discoveries, or type I errors among all the hypotheses when performing multiple pairwise tests.-Classification of m hypothesis tests:...

 for all the k hypotheses at level α in the strong sense.

Example

Suppose there are three hypotheses H₁,H₂, and H₃ are to be tested and the overall type I error rate is 0.05. Then H₁ can be rejected at level α if H₁ ∩ H₂ ∩ H₃, H₁ ∩ H₂, H₁ ∩ H₃ and H₁ can all be rejected using valid tests with level 0.05.

Special cases

The Holm–Bonferroni method is a special case of a closed test procedure for which each intersection null hypothesis is tested using the simple Bonferroni test. As such, it controls the familywise error rate

Familywise error rate

In statistics, familywise error rate is the probability of making one or more false discoveries, or type I errors among all the hypotheses when performing multiple pairwise tests.-Classification of m hypothesis tests:...

 for all the k hypotheses at level α in the strong sense.

See also

• Multiple comparisons
  Multiple comparisons
  In statistics, the multiple comparisons or multiple testing problem occurs when one considers a set of statistical inferences simultaneously. Errors in inference, including confidence intervals that fail to include their corresponding population parameters or hypothesis tests that incorrectly...
  
  
• Holm–Bonferroni method
• Bonferroni correction
  Bonferroni correction
  In statistics, the Bonferroni correction is a method used to counteract the problem of multiple comparisons. It was developed and introduced by Italian mathematician Carlo Emilio Bonferroni...