Quasi-experiment
Encyclopedia
A quasi-experiment is an empirical
study used to estimate the causal impact of an intervention on its target population. Quasi-experimental research designs share many similarities with the traditional experimental design or randomized controlled trial
, but they specifically lack the element of random assignment to treatment or control. Instead, quasi-experimental designs typically allow the researcher to control the assignment to the treatment condition, but using some criterion other than random assignment (e.g., an eligibility cutoff score). In some cases, the researcher may have no control over assignment to treatment condition.
Quasi-experiments are subject to concerns regarding internal validity
, because the treatment and control groups may not be comparable at baseline. With random assignment
, study participants have the same chance of being assigned to the intervention group or the comparison group. As a result, the treatment group will be statistically identical to the control group, on both observed and unobserved characteristics, at baseline (provided that the study has adequate sample size). Any change in characteristics post-intervention is due, therefore, to the intervention alone. With quasi-experimental studies, it may not be possible to convincingly demonstrate a causal link between the treatment condition and observed outcomes. This is particularly true if there are confounding variables that cannot be controlled or accounted for.
There are several types of quasi-experimental designs, each with different strengths, weaknesses and applications. These designs include (but are not limited to):
Of all of these designs, the regression discontinuity design comes the closest to the experimental design, as the experimenter maintains control of the treatment assignment and it is known to “yield an unbiased estimate of the treatment effects”. It does, however, require large numbers of study participants and precise modeling of the functional form between the assignment and the outcome variable, in order to yield the same power as a traditional experimental design
would randomly assign children to a scholarship, in order to control for all other variables. Quasi-experiments are commonly used in social sciences
, public health
, education
, and policy analysis
, especially when it is not practical or reasonable to randomize study participants to the treatment condition.
As an example, suppose we divide households into two categories: Households in which the parents spank their children, and households in which the parents do not spank their children. We can run a linear regression to determine if there is a positive correlation between parents' spanking and their children's aggressive behavior. However, to simply randomize parents to spank or to not spank their children may not be practical or ethical.
Some authors distinguish between a natural experiment
and a "quasi-experiment". The difference is that in a quasi-experiment the criterion for assignment is selected by the researcher, while in a natural experiment the assignment occurs 'naturally,' without the researcher's intervention.
variables. In the example above, a variation in the children's response to spanking is plausibly influenced by factors that cannot be easily measured and controlled, for example the child's intrinsic wildness or the parent's irritability.
Empirical
The word empirical denotes information gained by means of observation or experimentation. Empirical data are data produced by an experiment or observation....
study used to estimate the causal impact of an intervention on its target population. Quasi-experimental research designs share many similarities with the traditional experimental design or randomized controlled trial
Randomized controlled trial
A randomized controlled trial is a type of scientific experiment - a form of clinical trial - most commonly used in testing the safety and efficacy or effectiveness of healthcare services or health technologies A randomized controlled trial (RCT) is a type of scientific experiment - a form of...
, but they specifically lack the element of random assignment to treatment or control. Instead, quasi-experimental designs typically allow the researcher to control the assignment to the treatment condition, but using some criterion other than random assignment (e.g., an eligibility cutoff score). In some cases, the researcher may have no control over assignment to treatment condition.
Quasi-experiments are subject to concerns regarding internal validity
Internal validity
Internal validity is the validity of inferences in scientific studies, usually based on experiments as experimental validity.- Details :...
, because the treatment and control groups may not be comparable at baseline. With random assignment
Random assignment
Random assignment or random placement is an experimental technique for assigning subjects to different treatments . The thinking behind random assignment is that by randomizing treatment assignment, then the group attributes for the different treatments will be roughly equivalent and therefore any...
, study participants have the same chance of being assigned to the intervention group or the comparison group. As a result, the treatment group will be statistically identical to the control group, on both observed and unobserved characteristics, at baseline (provided that the study has adequate sample size). Any change in characteristics post-intervention is due, therefore, to the intervention alone. With quasi-experimental studies, it may not be possible to convincingly demonstrate a causal link between the treatment condition and observed outcomes. This is particularly true if there are confounding variables that cannot be controlled or accounted for.
There are several types of quasi-experimental designs, each with different strengths, weaknesses and applications. These designs include (but are not limited to):
- Difference-in-difference (pre-post with-without comparison)
- Regression discontinuity design
- case-control design
- Interrupted time-series design
- Propensity score matchingPropensity score matchingIn the statistical analysis of observational data, propensity score matching is a methodology attempting to provide unbiased estimation of treatment-effects...
- Instrumental variables
- Panel analysisPanel analysisPanel analysis is statistical method, widely used in social science, epidemiology, and econometrics, which deals with two-dimensional panel data. The data are usually collected over time and over the same individuals and then a regression is run over these two dimensions...
Of all of these designs, the regression discontinuity design comes the closest to the experimental design, as the experimenter maintains control of the treatment assignment and it is known to “yield an unbiased estimate of the treatment effects”. It does, however, require large numbers of study participants and precise modeling of the functional form between the assignment and the outcome variable, in order to yield the same power as a traditional experimental design
Ethics
A true experimentTrue experiment
A true experiment is a method of social research in which there are two kinds of variables. The independent variable is manipulated by the experimenter, and the dependent variable is measured...
would randomly assign children to a scholarship, in order to control for all other variables. Quasi-experiments are commonly used in social sciences
Social sciences
Social science is the field of study concerned with society. "Social science" is commonly used as an umbrella term to refer to a plurality of fields outside of the natural sciences usually exclusive of the administrative or managerial sciences...
, public health
Public health
Public health is "the science and art of preventing disease, prolonging life and promoting health through the organized efforts and informed choices of society, organizations, public and private, communities and individuals" . It is concerned with threats to health based on population health...
, education
Education
Education in its broadest, general sense is the means through which the aims and habits of a group of people lives on from one generation to the next. Generally, it occurs through any experience that has a formative effect on the way one thinks, feels, or acts...
, and policy analysis
Policy analysis
Policy analysis is "determining which of various alternative policies will most achieve a given set of goals in light of the relations between the policies and the goals". However, policy analysis can be divided into two major fields. Analysis of policy is analytical and descriptive—i.e., it...
, especially when it is not practical or reasonable to randomize study participants to the treatment condition.
As an example, suppose we divide households into two categories: Households in which the parents spank their children, and households in which the parents do not spank their children. We can run a linear regression to determine if there is a positive correlation between parents' spanking and their children's aggressive behavior. However, to simply randomize parents to spank or to not spank their children may not be practical or ethical.
Some authors distinguish between a natural experiment
Natural experiment
A natural experiment is an observational study in which the assignment of treatments to subjects has been haphazard: That is, the assignment of treatments has been made "by nature", but not by experimenters. Thus, a natural experiment is not a controlled experiment...
and a "quasi-experiment". The difference is that in a quasi-experiment the criterion for assignment is selected by the researcher, while in a natural experiment the assignment occurs 'naturally,' without the researcher's intervention.
Disadvantages
Quasi-experimental estimates of impact are subject to contamination by confoundingConfounding
In statistics, a confounding variable is an extraneous variable in a statistical model that correlates with both the dependent variable and the independent variable...
variables. In the example above, a variation in the children's response to spanking is plausibly influenced by factors that cannot be easily measured and controlled, for example the child's intrinsic wildness or the parent's irritability.
External links
- Quasi-Experimental Design at the Research Methods Knowledge Base