Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?

Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators?

June 2003 | Jeffrey Smith, Petra Todd
This paper evaluates the effectiveness of propensity score matching in estimating the impact of the National Supported Work (NSW) Demonstration program, using data previously analyzed by LaLonde (1986) and Dehejia and Wahba (1999, 2002). The study finds that propensity score matching estimates are highly sensitive to the variables included in the scores and the analysis sample used. Among the estimators studied, the difference-in-differences matching estimator performs best, as it eliminates potential sources of temporally-invariant bias in the NSW data, such as geographic mismatch and differences in measurement of the dependent variable. The paper concludes that while propensity score matching is a potentially useful econometric tool, it does not represent a general solution to the evaluation problem. The study also highlights the importance of data quality and the need for careful selection of variables and samples in nonexperimental evaluations. The findings suggest that matching estimators can be sensitive to the choice of subsample and the set of variables used to estimate propensity scores, and that difference-in-differences matching estimators may perform better in certain contexts. The paper emphasizes the importance of using experimental data as a benchmark to evaluate nonexperimental estimators and the need for careful consideration of the assumptions underlying different estimation methods.This paper evaluates the effectiveness of propensity score matching in estimating the impact of the National Supported Work (NSW) Demonstration program, using data previously analyzed by LaLonde (1986) and Dehejia and Wahba (1999, 2002). The study finds that propensity score matching estimates are highly sensitive to the variables included in the scores and the analysis sample used. Among the estimators studied, the difference-in-differences matching estimator performs best, as it eliminates potential sources of temporally-invariant bias in the NSW data, such as geographic mismatch and differences in measurement of the dependent variable. The paper concludes that while propensity score matching is a potentially useful econometric tool, it does not represent a general solution to the evaluation problem. The study also highlights the importance of data quality and the need for careful selection of variables and samples in nonexperimental evaluations. The findings suggest that matching estimators can be sensitive to the choice of subsample and the set of variables used to estimate propensity scores, and that difference-in-differences matching estimators may perform better in certain contexts. The paper emphasizes the importance of using experimental data as a benchmark to evaluate nonexperimental estimators and the need for careful consideration of the assumptions underlying different estimation methods.
Reach us at info@study.space
[slides and audio] Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators%3F