NATURAL AND QUASI-EXPERIMENTS IN ECONOMICS

NATURAL AND QUASI-EXPERIMENTS IN ECONOMICS

December 1994 | Bruce D. Meyer
This paper discusses the use of natural and quasi-experiments in economics to study the effects of treatments on outcomes. It highlights the advantages of these studies, which use variation in explanatory variables generated by changes in state laws, government draft mechanisms, or other means. The paper suggests ways to improve these studies and provides aids in judging the validity of their inferences. It also describes design complications such as multiple treatment and comparison groups and multiple pre- or post-intervention observations. The paper argues that natural experiments can be improved through more complicated research designs. Multiple treatment and comparison groups allow further checks of hypotheses and may allow hypotheses to be refined and alternative explanations to be ruled out. Similarly, multiple pre-intervention or post-intervention observations can be used to examine the comparability of comparison groups and the influences of omitted factors. These and other design features can increase the validity of inferences that can be drawn from natural experiments. Good natural experiments are studies where there is a transparent exogenous source of variation in the explanatory variables that determine the treatment assignment. A natural experiment induced by policy changes, government randomization or other events may allow a researcher to obtain exogenous variation in the main explanatory variables. This occurrence is especially useful in situations where estimates are ordinarily biased because of endogenous variation due to omitted variables or selection. Such approaches have been recently used to analyze a wide range of issues. The paper outlines several threats to validity, including omitted variables, trends in outcomes, misspecified variances, mismeasurement, political economy, simultaneity, selection, attrition, and omitted interactions. It also discusses threats to external validity, such as interaction of selection and treatment, interaction of setting and treatment, and interaction of history and treatment. The paper describes several research designs, including the one group before and after design and the before and after design with an untreated comparison group. It also discusses extensions to these methods, such as studies without a time dimension, controls for individual characteristics, and treatments that are higher order interactions. The paper emphasizes the importance of examining the comparability of comparison groups and the validity of inferences drawn from natural experiments. It concludes with a discussion of the sources of exogenous variation in natural experiments and other studies.This paper discusses the use of natural and quasi-experiments in economics to study the effects of treatments on outcomes. It highlights the advantages of these studies, which use variation in explanatory variables generated by changes in state laws, government draft mechanisms, or other means. The paper suggests ways to improve these studies and provides aids in judging the validity of their inferences. It also describes design complications such as multiple treatment and comparison groups and multiple pre- or post-intervention observations. The paper argues that natural experiments can be improved through more complicated research designs. Multiple treatment and comparison groups allow further checks of hypotheses and may allow hypotheses to be refined and alternative explanations to be ruled out. Similarly, multiple pre-intervention or post-intervention observations can be used to examine the comparability of comparison groups and the influences of omitted factors. These and other design features can increase the validity of inferences that can be drawn from natural experiments. Good natural experiments are studies where there is a transparent exogenous source of variation in the explanatory variables that determine the treatment assignment. A natural experiment induced by policy changes, government randomization or other events may allow a researcher to obtain exogenous variation in the main explanatory variables. This occurrence is especially useful in situations where estimates are ordinarily biased because of endogenous variation due to omitted variables or selection. Such approaches have been recently used to analyze a wide range of issues. The paper outlines several threats to validity, including omitted variables, trends in outcomes, misspecified variances, mismeasurement, political economy, simultaneity, selection, attrition, and omitted interactions. It also discusses threats to external validity, such as interaction of selection and treatment, interaction of setting and treatment, and interaction of history and treatment. The paper describes several research designs, including the one group before and after design and the before and after design with an untreated comparison group. It also discusses extensions to these methods, such as studies without a time dimension, controls for individual characteristics, and treatments that are higher order interactions. The paper emphasizes the importance of examining the comparability of comparison groups and the validity of inferences drawn from natural experiments. It concludes with a discussion of the sources of exogenous variation in natural experiments and other studies.
Reach us at info@study.space
[slides and audio] Natural and Quasi- Experiments in Economics