CONTRACTIVE DIFFUSION PROBABILISTIC MODELS

CONTRACTIVE DIFFUSION PROBABILISTIC MODELS

23 May 2024 | WENPIN TANG AND HANYANG ZHAO
Diffusion probabilistic models (DPMs) have emerged as a promising technique in generative modeling, relying on two key ingredients: time reversal of diffusion processes and score matching. However, the assumption that score matching is nearly perfect is questionable. To address this, the authors propose a new criterion—contracting backward sampling, leading to a novel class of contractive DPMs (CDPMs). The key insight is that contraction in the backward process can reduce score matching errors and discretization errors, making CDPMs more robust. The authors demonstrate that CDPMs can leverage pre-trained DPMs with a simple transformation and do not require retraining. Experiments on synthetic and real datasets, including 1D examples, Swiss Roll, MNIST, CIFAR-10, and AFHQ, show that CDPMs outperform other SDE-based DPMs, particularly in terms of Wasserstein distance and FID scores. The contributions of this work include a new methodology, theoretical guarantees, and empirical validation.Diffusion probabilistic models (DPMs) have emerged as a promising technique in generative modeling, relying on two key ingredients: time reversal of diffusion processes and score matching. However, the assumption that score matching is nearly perfect is questionable. To address this, the authors propose a new criterion—contracting backward sampling, leading to a novel class of contractive DPMs (CDPMs). The key insight is that contraction in the backward process can reduce score matching errors and discretization errors, making CDPMs more robust. The authors demonstrate that CDPMs can leverage pre-trained DPMs with a simple transformation and do not require retraining. Experiments on synthetic and real datasets, including 1D examples, Swiss Roll, MNIST, CIFAR-10, and AFHQ, show that CDPMs outperform other SDE-based DPMs, particularly in terms of Wasserstein distance and FID scores. The contributions of this work include a new methodology, theoretical guarantees, and empirical validation.
Reach us at info@study.space