October 2005, Vol. 23, No. 4 | Peter Reinhard HANSEN
This paper proposes a new test for superior predictive ability (SPA) that outperforms the reality check (RC) test for data snooping. The new test is more powerful and less sensitive to poor and irrelevant alternatives. It achieves this by using a studentized test statistic and a sample-dependent null distribution. The studentized test statistic reduces the influence of erratic forecasts, while the sample-dependent null distribution incorporates additional sample information to identify relevant alternatives. The new test is validated through Monte Carlo experiments and an empirical analysis comparing regression-based forecasts of U.S. inflation with a simple random-walk forecast. The results show that regression-based forecasts outperform the random-walk forecast, with the best performance achieved by models with a Phillips curve structure. The paper also discusses the theoretical foundations of the test, including the use of a bootstrap implementation and the importance of handling nuisance parameters. The new test is shown to be more robust and effective in detecting superior predictive ability compared to the RC test.This paper proposes a new test for superior predictive ability (SPA) that outperforms the reality check (RC) test for data snooping. The new test is more powerful and less sensitive to poor and irrelevant alternatives. It achieves this by using a studentized test statistic and a sample-dependent null distribution. The studentized test statistic reduces the influence of erratic forecasts, while the sample-dependent null distribution incorporates additional sample information to identify relevant alternatives. The new test is validated through Monte Carlo experiments and an empirical analysis comparing regression-based forecasts of U.S. inflation with a simple random-walk forecast. The results show that regression-based forecasts outperform the random-walk forecast, with the best performance achieved by models with a Phillips curve structure. The paper also discusses the theoretical foundations of the test, including the use of a bootstrap implementation and the importance of handling nuisance parameters. The new test is shown to be more robust and effective in detecting superior predictive ability compared to the RC test.