The paper introduces the Diffusion World Model (DWM), a conditional diffusion model designed to predict multistep future states and rewards simultaneously. Unlike traditional one-step dynamics models, DWM offers long-horizon predictions in a single forward pass, reducing the need for recursive queries and mitigating compounding errors. The authors integrate DWM into a model-based value estimation framework, where the short-term return is simulated using future trajectories sampled from DWM. In the context of offline reinforcement learning (RL), DWM can be seen as a conservative value regularization through generative modeling or a data source for offline Q-learning with synthetic data. Experiments on the D4RL dataset confirm DWM's robustness to long-horizon simulation, achieving a 44% performance gain over one-step dynamics models and comparable or slightly better performance compared to model-free methods. The paper also discusses key differences between DWM and other diffusion-based offline RL methods, highlighting its advantages in reducing compounding errors and improving performance.The paper introduces the Diffusion World Model (DWM), a conditional diffusion model designed to predict multistep future states and rewards simultaneously. Unlike traditional one-step dynamics models, DWM offers long-horizon predictions in a single forward pass, reducing the need for recursive queries and mitigating compounding errors. The authors integrate DWM into a model-based value estimation framework, where the short-term return is simulated using future trajectories sampled from DWM. In the context of offline reinforcement learning (RL), DWM can be seen as a conservative value regularization through generative modeling or a data source for offline Q-learning with synthetic data. Experiments on the D4RL dataset confirm DWM's robustness to long-horizon simulation, achieving a 44% performance gain over one-step dynamics models and comparable or slightly better performance compared to model-free methods. The paper also discusses key differences between DWM and other diffusion-based offline RL methods, highlighting its advantages in reducing compounding errors and improving performance.