This paper investigates the prevalence of structural instability in macroeconomic time series relations and evaluates the effectiveness of adaptive forecasting techniques in handling such instability. Using a sample of 76 monthly U.S. postwar macroeconomic time series, comprising 5,700 bivariate forecasting relations, the authors perform formal tests for instability and compute out-of-sample forecasts from sixteen different models. The results indicate widespread instability in both univariate and bivariate autoregressive models. However, adaptive forecasting models, particularly time-varying parameter models, show limited success in exploiting this instability to improve upon fixed-parameter or recursive autoregressive forecasts.
The study finds that while some adaptive models perform well in certain cases, they generally do not outperform traditional fixed-parameter or recursive least squares models. For example, in 57% of the 5,700 pairs, fixed-coefficient or recursive least squares forecasts have the lowest out-of-sample mean square error (MSE) among the sixteen competing models, while in only 10% of the pairs do bivariate time-varying parameter (TVP) models have the lowest MSE. When they perform best, the gains associated with TVP models are typically small.
The paper also examines the robustness of different forecasting models by identifying those that successfully guard against severe out-of-sample forecasting failures. The results suggest that while some adaptive models may perform well in specific cases, they are not consistently superior to traditional models. Overall, the TVP models fail to effectively exploit the time variation identified by the stability tests.
The study concludes that the class of models most widely studied in adaptive forecasting, including TVP models, are largely unsuccessful in modeling and exploiting the instability found in typical macroeconomic applications. The findings highlight the importance of systematic stability analysis in structural VAR modeling and suggest that traditional fixed-parameter models may be more robust in many cases.This paper investigates the prevalence of structural instability in macroeconomic time series relations and evaluates the effectiveness of adaptive forecasting techniques in handling such instability. Using a sample of 76 monthly U.S. postwar macroeconomic time series, comprising 5,700 bivariate forecasting relations, the authors perform formal tests for instability and compute out-of-sample forecasts from sixteen different models. The results indicate widespread instability in both univariate and bivariate autoregressive models. However, adaptive forecasting models, particularly time-varying parameter models, show limited success in exploiting this instability to improve upon fixed-parameter or recursive autoregressive forecasts.
The study finds that while some adaptive models perform well in certain cases, they generally do not outperform traditional fixed-parameter or recursive least squares models. For example, in 57% of the 5,700 pairs, fixed-coefficient or recursive least squares forecasts have the lowest out-of-sample mean square error (MSE) among the sixteen competing models, while in only 10% of the pairs do bivariate time-varying parameter (TVP) models have the lowest MSE. When they perform best, the gains associated with TVP models are typically small.
The paper also examines the robustness of different forecasting models by identifying those that successfully guard against severe out-of-sample forecasting failures. The results suggest that while some adaptive models may perform well in specific cases, they are not consistently superior to traditional models. Overall, the TVP models fail to effectively exploit the time variation identified by the stability tests.
The study concludes that the class of models most widely studied in adaptive forecasting, including TVP models, are largely unsuccessful in modeling and exploiting the instability found in typical macroeconomic applications. The findings highlight the importance of systematic stability analysis in structural VAR modeling and suggest that traditional fixed-parameter models may be more robust in many cases.