COMPARING PREDICTIVE ACCURACY

COMPARING PREDICTIVE ACCURACY

November 1994 | Francis X. Diebold, Roberto S. Mariano
This paper proposes and evaluates tests for the null hypothesis of no difference in the accuracy of two competing forecasts. Unlike previous tests, the proposed tests allow for a wide variety of accuracy measures, including non-Gaussian, non-zero mean, serially and contemporaneously correlated forecast errors. Both asymptotic and exact finite sample tests are developed, evaluated, and illustrated. The paper introduces an asymptotic test based on the sample mean of the loss differential, which is asymptotically normally distributed. The test uses a consistent estimate of the spectral density of the loss differential to account for serial correlation. An exact finite-sample test is also presented, based on the sign test and Wilcoxon's signed-rank test, which are robust to non-normality and serial correlation. The paper also reviews existing tests of forecast accuracy, including the simple F test, the Morgan-Granger-Newbold test, and the Meese-Rogoff test. These tests make different assumptions about the distribution of forecast errors and are evaluated in the context of the paper's framework. A Monte Carlo analysis is conducted to evaluate the finite-sample size of the tests under various assumptions. The results show that the proposed tests perform well in both Gaussian and non-Gaussian settings, while traditional tests such as the F test and the Morgan-Granger-Newbold test are often missized in the presence of serial correlation or non-normality. The paper concludes that the proposed tests are more robust and flexible than existing tests, and that they provide a more accurate assessment of forecast accuracy in a wide range of economic applications. The tests are applicable under a variety of loss functions and are particularly useful for evaluating forecasts in the presence of non-normality and serial correlation. The paper also highlights the importance of considering forecast accuracy in model comparison and emphasizes the need for further research in this area.This paper proposes and evaluates tests for the null hypothesis of no difference in the accuracy of two competing forecasts. Unlike previous tests, the proposed tests allow for a wide variety of accuracy measures, including non-Gaussian, non-zero mean, serially and contemporaneously correlated forecast errors. Both asymptotic and exact finite sample tests are developed, evaluated, and illustrated. The paper introduces an asymptotic test based on the sample mean of the loss differential, which is asymptotically normally distributed. The test uses a consistent estimate of the spectral density of the loss differential to account for serial correlation. An exact finite-sample test is also presented, based on the sign test and Wilcoxon's signed-rank test, which are robust to non-normality and serial correlation. The paper also reviews existing tests of forecast accuracy, including the simple F test, the Morgan-Granger-Newbold test, and the Meese-Rogoff test. These tests make different assumptions about the distribution of forecast errors and are evaluated in the context of the paper's framework. A Monte Carlo analysis is conducted to evaluate the finite-sample size of the tests under various assumptions. The results show that the proposed tests perform well in both Gaussian and non-Gaussian settings, while traditional tests such as the F test and the Morgan-Granger-Newbold test are often missized in the presence of serial correlation or non-normality. The paper concludes that the proposed tests are more robust and flexible than existing tests, and that they provide a more accurate assessment of forecast accuracy in a wide range of economic applications. The tests are applicable under a variety of loss functions and are particularly useful for evaluating forecasts in the presence of non-normality and serial correlation. The paper also highlights the importance of considering forecast accuracy in model comparison and emphasizes the need for further research in this area.
Reach us at info@study.space
[slides] Comparing Predictive Accuracy | StudySpace