14 Apr 2023 | Diederik P. Kingma*, Tim Salimans*, Ben Poole, Jonathan Ho
The paper introduces a family of diffusion-based generative models called Variational Diffusion Models (VDMs) that achieve state-of-the-art likelihoods on standard image density estimation benchmarks. Unlike other diffusion models, VDMs allow for efficient optimization of the noise schedule, which is a key innovation. The authors provide theoretical insights into the variational lower bound (VLB) of these models, showing that it simplifies to a short expression in terms of the signal-to-noise ratio (SNR) of the diffused data. This simplification improves the understanding of the model class and leads to new theoretical results, such as the invariance of the generative model and its VLB to the specification of the diffusion process. The authors also demonstrate how to use the model for lossless compression and show that it outperforms autoregressive models in terms of likelihood and optimization speed. The code for VDMs is available at <https://github.com/google-research/vdm>.The paper introduces a family of diffusion-based generative models called Variational Diffusion Models (VDMs) that achieve state-of-the-art likelihoods on standard image density estimation benchmarks. Unlike other diffusion models, VDMs allow for efficient optimization of the noise schedule, which is a key innovation. The authors provide theoretical insights into the variational lower bound (VLB) of these models, showing that it simplifies to a short expression in terms of the signal-to-noise ratio (SNR) of the diffused data. This simplification improves the understanding of the model class and leads to new theoretical results, such as the invariance of the generative model and its VLB to the specification of the diffusion process. The authors also demonstrate how to use the model for lossless compression and show that it outperforms autoregressive models in terms of likelihood and optimization speed. The code for VDMs is available at <https://github.com/google-research/vdm>.