April 12, 2024 | Minshuo Chen¹ Song Mei² Jianqing Fan¹ Mengdi Wang¹
Diffusion models are powerful generative AI technologies with significant success in computer vision, audio, reinforcement learning, and computational biology. They provide flexible high-dimensional data modeling and generate new samples under active guidance towards task-desired properties. Despite empirical success, theoretical understanding of diffusion models is limited, hindering principled methodological innovations. This paper reviews emerging applications, existing theories, and future directions of diffusion models. It discusses their statistical properties, sampling capabilities, and high-dimensional structured optimization through conditional diffusion models. The paper also explores the theoretical foundations of diffusion models, including score function learning, score approximation, and distribution estimation. It highlights the importance of understanding the sample complexity of score estimation and the impact of guidance on sample generation. The paper also discusses the application of diffusion models in black-box optimization and their connection to stochastic control, adversarial robustness, and discrete diffusion models. The paper concludes with a discussion of future research directions in diffusion models.Diffusion models are powerful generative AI technologies with significant success in computer vision, audio, reinforcement learning, and computational biology. They provide flexible high-dimensional data modeling and generate new samples under active guidance towards task-desired properties. Despite empirical success, theoretical understanding of diffusion models is limited, hindering principled methodological innovations. This paper reviews emerging applications, existing theories, and future directions of diffusion models. It discusses their statistical properties, sampling capabilities, and high-dimensional structured optimization through conditional diffusion models. The paper also explores the theoretical foundations of diffusion models, including score function learning, score approximation, and distribution estimation. It highlights the importance of understanding the sample complexity of score estimation and the impact of guidance on sample generation. The paper also discusses the application of diffusion models in black-box optimization and their connection to stochastic control, adversarial robustness, and discrete diffusion models. The paper concludes with a discussion of future research directions in diffusion models.