The paper introduces SLAMS (Score-based Latent Assimilation in Multimodal Setting), a novel framework for data assimilation that integrates physical knowledge and data to improve computational simulations, particularly in Earth system models. Traditional data assimilation methods, such as Kalman filters and variational approaches, rely on linear and Gaussian assumptions, which can be computationally expensive and less accurate in complex systems. SLAMS leverages deep generative models, specifically diffusion-based probabilistic frameworks, to calibrate model outputs with observations, including remote sensing imagery and ground station measurements. Key contributions of SLAMS include: 1. **Unified Latent Space**: SLAMS projects heterogeneous, multimodal datasets into a common latent subspace, eliminating the need for complex observation operators. 2. **Score-based Diffusion Model**: The framework uses a score-based diffusion process to generate samples conditioned on observations, allowing for uncertainty quantification. 3. **Robustness to Noisy and Sparse Data**: Extensive ablation studies demonstrate that SLAMS is robust even in low-resolution, noisy, and sparse data settings. The authors validate SLAMS using in-situ weather station data and ex-situ satellite imagery to calibrate vertical temperature profiles globally. The results show that SLAMS produces consistent and physically consistent states, outperforming traditional pixel-based approaches in various scenarios. The framework's adaptability and robustness make it a promising tool for real-world Earth system modeling and other scientific disciplines requiring accurate state estimation.The paper introduces SLAMS (Score-based Latent Assimilation in Multimodal Setting), a novel framework for data assimilation that integrates physical knowledge and data to improve computational simulations, particularly in Earth system models. Traditional data assimilation methods, such as Kalman filters and variational approaches, rely on linear and Gaussian assumptions, which can be computationally expensive and less accurate in complex systems. SLAMS leverages deep generative models, specifically diffusion-based probabilistic frameworks, to calibrate model outputs with observations, including remote sensing imagery and ground station measurements. Key contributions of SLAMS include: 1. **Unified Latent Space**: SLAMS projects heterogeneous, multimodal datasets into a common latent subspace, eliminating the need for complex observation operators. 2. **Score-based Diffusion Model**: The framework uses a score-based diffusion process to generate samples conditioned on observations, allowing for uncertainty quantification. 3. **Robustness to Noisy and Sparse Data**: Extensive ablation studies demonstrate that SLAMS is robust even in low-resolution, noisy, and sparse data settings. The authors validate SLAMS using in-situ weather station data and ex-situ satellite imagery to calibrate vertical temperature profiles globally. The results show that SLAMS produces consistent and physically consistent states, outperforming traditional pixel-based approaches in various scenarios. The framework's adaptability and robustness make it a promising tool for real-world Earth system modeling and other scientific disciplines requiring accurate state estimation.