This paper investigates the trajectory regularity in ODE-based diffusion sampling. Diffusion models use stochastic differential equations (SDEs) and their equivalent ordinary differential equations (ODEs) to connect complex data distributions with tractable prior distributions. The study identifies key trajectory properties in ODE-based sampling, including an implicit denoising trajectory that plays a crucial role in forming trajectories with strong shape regularity. A dynamic programming-based approach is proposed to optimize the time schedule for sampling, which requires minimal modification to existing ODE solvers and incurs negligible computational cost, while significantly improving image generation performance.
The research reveals that sampling trajectories in diffusion models exhibit a consistent linear-nonlinear-linear structure, regardless of the generated content. This regularity is attributed to the underlying geometric structure of the sampling trajectories, which can be characterized using kernel density estimation (KDE). The study also demonstrates that the sampling trajectory can be approximated using a few orthogonal bases, enabling efficient and effective sampling strategies.
The paper introduces a geometry-inspired time scheduling (GITS) method that dynamically allocates time steps based on the curvature of the sampling trajectory. This approach leverages the observed regularity to optimize the time schedule, resulting in improved sample quality and reduced computational cost. Experiments show that GITS significantly outperforms existing methods in terms of sample quality, especially with a small number of function evaluations.
The findings contribute to a deeper understanding of diffusion models and provide a theoretical basis for developing more efficient sampling strategies. The study also highlights the importance of trajectory regularity in diffusion models and its potential applications in improving the performance of generative models. The results demonstrate that the trajectory regularity can be effectively utilized to enhance the efficiency and quality of diffusion-based generative models.This paper investigates the trajectory regularity in ODE-based diffusion sampling. Diffusion models use stochastic differential equations (SDEs) and their equivalent ordinary differential equations (ODEs) to connect complex data distributions with tractable prior distributions. The study identifies key trajectory properties in ODE-based sampling, including an implicit denoising trajectory that plays a crucial role in forming trajectories with strong shape regularity. A dynamic programming-based approach is proposed to optimize the time schedule for sampling, which requires minimal modification to existing ODE solvers and incurs negligible computational cost, while significantly improving image generation performance.
The research reveals that sampling trajectories in diffusion models exhibit a consistent linear-nonlinear-linear structure, regardless of the generated content. This regularity is attributed to the underlying geometric structure of the sampling trajectories, which can be characterized using kernel density estimation (KDE). The study also demonstrates that the sampling trajectory can be approximated using a few orthogonal bases, enabling efficient and effective sampling strategies.
The paper introduces a geometry-inspired time scheduling (GITS) method that dynamically allocates time steps based on the curvature of the sampling trajectory. This approach leverages the observed regularity to optimize the time schedule, resulting in improved sample quality and reduced computational cost. Experiments show that GITS significantly outperforms existing methods in terms of sample quality, especially with a small number of function evaluations.
The findings contribute to a deeper understanding of diffusion models and provide a theoretical basis for developing more efficient sampling strategies. The study also highlights the importance of trajectory regularity in diffusion models and its potential applications in improving the performance of generative models. The results demonstrate that the trajectory regularity can be effectively utilized to enhance the efficiency and quality of diffusion-based generative models.