D⁴M: Dataset Distillation via Disentangled Diffusion Model

D⁴M: Dataset Distillation via Disentangled Diffusion Model

21 Jul 2024 | Duo Su, Junjie Hou, Weizhi Gao, Yingjie Tian, Bowen Tang
D⁴M: Dataset Distillation via Disentangled Diffusion Model D⁴M is a novel dataset distillation framework that addresses the limitations of existing methods by employing a disentangled diffusion model. Traditional dataset distillation methods often rely on bi-level optimization and specific architectures, leading to high computational costs and poor cross-architecture generalization. D⁴M introduces a training-time matching (TTM) strategy that eliminates the need for architecture-specific matching, enabling efficient and versatile dataset distillation. By leveraging latent diffusion models, D⁴M ensures consistency between input and output spaces and incorporates label information into category prototypes, resulting in high-quality, high-resolution synthetic images. The method demonstrates superior performance across various datasets, including ImageNet-1K, CIFAR-10, and Tiny-ImageNet, with significant improvements in accuracy and computational efficiency. D⁴M also exhibits strong cross-architecture generalization and reduces the need for repeated dataset generation for different architectures. The framework is architecture-free, allowing for efficient synthesis of large-scale datasets with limited computational resources. Through extensive experiments, D⁴M outperforms state-of-the-art methods in most aspects, showcasing its effectiveness in dataset distillation.D⁴M: Dataset Distillation via Disentangled Diffusion Model D⁴M is a novel dataset distillation framework that addresses the limitations of existing methods by employing a disentangled diffusion model. Traditional dataset distillation methods often rely on bi-level optimization and specific architectures, leading to high computational costs and poor cross-architecture generalization. D⁴M introduces a training-time matching (TTM) strategy that eliminates the need for architecture-specific matching, enabling efficient and versatile dataset distillation. By leveraging latent diffusion models, D⁴M ensures consistency between input and output spaces and incorporates label information into category prototypes, resulting in high-quality, high-resolution synthetic images. The method demonstrates superior performance across various datasets, including ImageNet-1K, CIFAR-10, and Tiny-ImageNet, with significant improvements in accuracy and computational efficiency. D⁴M also exhibits strong cross-architecture generalization and reduces the need for repeated dataset generation for different architectures. The framework is architecture-free, allowing for efficient synthesis of large-scale datasets with limited computational resources. Through extensive experiments, D⁴M outperforms state-of-the-art methods in most aspects, showcasing its effectiveness in dataset distillation.
Reach us at info@study.space