10 Feb 2021 | Yang Song*, Jascha Sohl-Dickstein, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, Ben Poole
The paper presents a novel framework for score-based generative modeling using stochastic differential equations (SDEs). The authors propose a continuous-time SDE that smoothly transforms a complex data distribution into a known prior distribution by gradually injecting noise. They also introduce a reverse-time SDE that transforms the prior distribution back into the data distribution by removing the noise. The reverse-time SDE depends only on the time-dependent gradient field (score) of the perturbed data distribution. By leveraging advances in score-based generative modeling, the authors can accurately estimate these scores using neural networks and use numerical SDE solvers to generate samples. The framework encapsulates previous approaches in score-based generative modeling and diffusion probabilistic modeling, enabling new sampling procedures and modeling capabilities. Key contributions include a predictor-corrector framework for correcting errors in the discretized reverse-time SDE, an equivalent neural ordinary differential equation (ODE) for exact likelihood computation and improved sampling efficiency, and a method for solving inverse problems with score-based models. The authors demonstrate state-of-the-art performance on unconditional image generation on CIFAR-10, achieving an Inception score of 9.89 and FID score of 2.20, as well as high-fidelity generation of 1024 × 1024 images for the first time from a score-based generative model.The paper presents a novel framework for score-based generative modeling using stochastic differential equations (SDEs). The authors propose a continuous-time SDE that smoothly transforms a complex data distribution into a known prior distribution by gradually injecting noise. They also introduce a reverse-time SDE that transforms the prior distribution back into the data distribution by removing the noise. The reverse-time SDE depends only on the time-dependent gradient field (score) of the perturbed data distribution. By leveraging advances in score-based generative modeling, the authors can accurately estimate these scores using neural networks and use numerical SDE solvers to generate samples. The framework encapsulates previous approaches in score-based generative modeling and diffusion probabilistic modeling, enabling new sampling procedures and modeling capabilities. Key contributions include a predictor-corrector framework for correcting errors in the discretized reverse-time SDE, an equivalent neural ordinary differential equation (ODE) for exact likelihood computation and improved sampling efficiency, and a method for solving inverse problems with score-based models. The authors demonstrate state-of-the-art performance on unconditional image generation on CIFAR-10, achieving an Inception score of 9.89 and FID score of 2.20, as well as high-fidelity generation of 1024 × 1024 images for the first time from a score-based generative model.