EXPLORING DIFFUSION TIME-STEPS FOR UNSUPERVISED REPRESENTATION LEARNING

EXPLORING DIFFUSION TIME-STEPS FOR UNSUPERVISED REPRESENTATION LEARNING

21 Jan 2024 | Zhongqi Yue1, Jiankun Wang1, Qianru Sun2, Lei Ji3, Eric I-Chao Chang3, Hanwang Zhang1
The paper explores the potential of Denoising Diffusion Probabilistic Models (DDPMs) in unsupervised learning of modular attributes. It introduces a theoretical framework that connects diffusion time-steps with hidden attributes, serving as an inductive bias for disentangled representation learning. The forward diffusion process gradually adds Gaussian noise to samples, collapsing different samples into similar ones by losing attributes. The authors propose DiTi, a method that learns a set of time-step-specific features to compensate for the lost attributes at each step. This approach improves attribute classification and enables faithful counterfactual generation, validating the disentanglement quality on datasets like CelebA, FFHQ, and Bedroom. The method leverages the intrinsic connection between diffusion time-steps and hidden attributes, providing a simple and effective approach to disentangle modular attributes.The paper explores the potential of Denoising Diffusion Probabilistic Models (DDPMs) in unsupervised learning of modular attributes. It introduces a theoretical framework that connects diffusion time-steps with hidden attributes, serving as an inductive bias for disentangled representation learning. The forward diffusion process gradually adds Gaussian noise to samples, collapsing different samples into similar ones by losing attributes. The authors propose DiTi, a method that learns a set of time-step-specific features to compensate for the lost attributes at each step. This approach improves attribute classification and enables faithful counterfactual generation, validating the disentanglement quality on datasets like CelebA, FFHQ, and Bedroom. The method leverages the intrinsic connection between diffusion time-steps and hidden attributes, providing a simple and effective approach to disentangle modular attributes.
Reach us at info@study.space
Understanding Exploring Diffusion Time-steps for Unsupervised Representation Learning