The Effective Number of Spatial Degrees of Freedom of a Time-Varying Field

The Effective Number of Spatial Degrees of Freedom of a Time-Varying Field

1990 | CHRISTOPHER S. BRETHERTON, MARTIN WIDMANN, VALENTIN P. DYMNIKOV, JOHN M. WALLACE, AND ILEANA BLADE
The paper by Bretherton et al. (1) investigates two measures of the effective number of spatial degrees of freedom (ESDOF) for time-varying fields, which are crucial for understanding the complexity and variability of such fields. The first measure is based on matching the mean and variance of the chi-squared distribution of the spatially integrated squared anomaly of the field, while the second measure is based on partitioning the variance between EOFs. These measures were proposed nearly 30 years ago but have not been widely used. The authors provide a comprehensive discussion of these measures, including their theoretical basis and practical considerations for estimating them from limited time samples or non-normally distributed data. The paper demonstrates that standard statistical significance tests for comparing two realizations of a field (e.g., forecast and observation) are approximately valid when the number of degrees of freedom is chosen using these ESDOF measures. It also introduces a method for deciding whether two time-varying fields are significantly correlated using ESDOF. Additionally, the authors explore the concept of effective sample size for autocorrelated time series and its application in statistical significance testing. The introduction of ESDOF is motivated by its importance in various applications, such as estimating the number of stations required to represent a field, assessing field significance, characterizing and comparing observed and simulated fields, and estimating the dimension of an underlying attractor. The paper reviews the theoretical foundations of the two ESDOF measures and provides formulas for their estimation, considering both normal and non-normal data. It also discusses the impact of sample length on the accuracy of ESDOF estimates and the bias and scatter in these estimates. Two meteorological datasets are used to illustrate the application of ESDOF: Northern Hemisphere wintertime 500-mb heights and daily precipitation in Switzerland. The results show that the two ESDOF measures are useful in different contexts, with the moment-matching estimate being more suitable for variance distributions and the eigenvalue formula for covariance and correlation distributions. The paper also discusses the robustness of ESDOF estimates from limited data and provides guidelines for interpreting and using ESDOF in practical scenarios. Finally, the paper explores the use of ESDOF for significance testing, including comparing a realization to its mean and comparing two realizations of a field. It shows that the chi-squared distribution with ESDOF degrees of freedom can be used to test the significance of departures from the mean and the similarity between two realizations, even when the principal components are not normally distributed. The paper concludes with a summary of the key findings and the practical implications of ESDOF in geophysical data analysis.The paper by Bretherton et al. (1) investigates two measures of the effective number of spatial degrees of freedom (ESDOF) for time-varying fields, which are crucial for understanding the complexity and variability of such fields. The first measure is based on matching the mean and variance of the chi-squared distribution of the spatially integrated squared anomaly of the field, while the second measure is based on partitioning the variance between EOFs. These measures were proposed nearly 30 years ago but have not been widely used. The authors provide a comprehensive discussion of these measures, including their theoretical basis and practical considerations for estimating them from limited time samples or non-normally distributed data. The paper demonstrates that standard statistical significance tests for comparing two realizations of a field (e.g., forecast and observation) are approximately valid when the number of degrees of freedom is chosen using these ESDOF measures. It also introduces a method for deciding whether two time-varying fields are significantly correlated using ESDOF. Additionally, the authors explore the concept of effective sample size for autocorrelated time series and its application in statistical significance testing. The introduction of ESDOF is motivated by its importance in various applications, such as estimating the number of stations required to represent a field, assessing field significance, characterizing and comparing observed and simulated fields, and estimating the dimension of an underlying attractor. The paper reviews the theoretical foundations of the two ESDOF measures and provides formulas for their estimation, considering both normal and non-normal data. It also discusses the impact of sample length on the accuracy of ESDOF estimates and the bias and scatter in these estimates. Two meteorological datasets are used to illustrate the application of ESDOF: Northern Hemisphere wintertime 500-mb heights and daily precipitation in Switzerland. The results show that the two ESDOF measures are useful in different contexts, with the moment-matching estimate being more suitable for variance distributions and the eigenvalue formula for covariance and correlation distributions. The paper also discusses the robustness of ESDOF estimates from limited data and provides guidelines for interpreting and using ESDOF in practical scenarios. Finally, the paper explores the use of ESDOF for significance testing, including comparing a realization to its mean and comparing two realizations of a field. It shows that the chi-squared distribution with ESDOF degrees of freedom can be used to test the significance of departures from the mean and the similarity between two realizations, even when the principal components are not normally distributed. The paper concludes with a summary of the key findings and the practical implications of ESDOF in geophysical data analysis.
Reach us at info@study.space
[slides and audio] The Effective Number of Spatial Degrees of Freedom of a Time-Varying Field