29 March 2024 | Lizao Li, Robert Carvert, Ignacio Lopez-Gomez, Fei Sha, John Anderson
This paper introduces a method for generating weather forecasts using diffusion models to emulate ensemble forecasts, reducing computational costs. The approach leverages historical data to train deep generative models that can produce thousands of realistic forecasts at low cost. These models, called Scalable Ensemble Envelope Diffusion Sampler (SEEDS), can generate ensembles with a large number of members, often exceeding those of physics-based systems. When designed to correct biases in operational forecasts, the generated ensembles show improved probabilistic forecast metrics and better reliability, especially for extreme weather events. The method is applicable not only to weather forecasting but also to climate risk assessment by enabling large climate projection ensembles.
The study focuses on the limitations of traditional ensemble forecasting, which is computationally expensive and often insufficient for rare, high-impact weather events. The proposed method uses generative artificial intelligence (GAI) to reduce the cost of ensemble forecasting by learning statistical priors from historical data and enabling efficient sampling from the learned probability distributions. This approach allows for the generation of ensembles with high spatial coherence, multivariate correlation structures, and wave number spectra consistent with actual weather states.
The SEEDS method was tested against operational ensembles and reanalysis data, showing that the generated ensembles match or exceed physics-based ensembles in skill metrics such as rank histogram, root mean squared error (RMSE), and continuous ranked probability score (CRPS). The generated ensembles also assign more accurate likelihoods to extreme weather events. The computational cost of the model is low, with a throughput of 256 ensemble members per 3 minutes on Google Cloud TPUv3-32 instances.
The study also evaluates the reliability and predictive skill of the generated ensembles, finding that they are more reliable and better at predicting extreme weather events than traditional ensembles. The results show that the generated ensembles have higher reliability and better predictive skill, especially for long lead times. The method is also compared to statistical postprocessing techniques, showing that it can match or outperform them in predicting extreme events.
The study highlights the potential of generative AI in weather forecasting, offering a scalable and efficient alternative to traditional ensemble forecasting methods. The approach can be applied to a wide range of weather and climate applications, including nowcasting and upsampling. The results demonstrate that the generated ensembles can provide more accurate and reliable forecasts, particularly for extreme weather events, and can be used to improve climate risk assessment by enabling large climate projection ensembles.This paper introduces a method for generating weather forecasts using diffusion models to emulate ensemble forecasts, reducing computational costs. The approach leverages historical data to train deep generative models that can produce thousands of realistic forecasts at low cost. These models, called Scalable Ensemble Envelope Diffusion Sampler (SEEDS), can generate ensembles with a large number of members, often exceeding those of physics-based systems. When designed to correct biases in operational forecasts, the generated ensembles show improved probabilistic forecast metrics and better reliability, especially for extreme weather events. The method is applicable not only to weather forecasting but also to climate risk assessment by enabling large climate projection ensembles.
The study focuses on the limitations of traditional ensemble forecasting, which is computationally expensive and often insufficient for rare, high-impact weather events. The proposed method uses generative artificial intelligence (GAI) to reduce the cost of ensemble forecasting by learning statistical priors from historical data and enabling efficient sampling from the learned probability distributions. This approach allows for the generation of ensembles with high spatial coherence, multivariate correlation structures, and wave number spectra consistent with actual weather states.
The SEEDS method was tested against operational ensembles and reanalysis data, showing that the generated ensembles match or exceed physics-based ensembles in skill metrics such as rank histogram, root mean squared error (RMSE), and continuous ranked probability score (CRPS). The generated ensembles also assign more accurate likelihoods to extreme weather events. The computational cost of the model is low, with a throughput of 256 ensemble members per 3 minutes on Google Cloud TPUv3-32 instances.
The study also evaluates the reliability and predictive skill of the generated ensembles, finding that they are more reliable and better at predicting extreme weather events than traditional ensembles. The results show that the generated ensembles have higher reliability and better predictive skill, especially for long lead times. The method is also compared to statistical postprocessing techniques, showing that it can match or outperform them in predicting extreme events.
The study highlights the potential of generative AI in weather forecasting, offering a scalable and efficient alternative to traditional ensemble forecasting methods. The approach can be applied to a wide range of weather and climate applications, including nowcasting and upsampling. The results demonstrate that the generated ensembles can provide more accurate and reliable forecasts, particularly for extreme weather events, and can be used to improve climate risk assessment by enabling large climate projection ensembles.