The paper explores the hierarchical nature of data and its implications for diffusion models. It argues that diffusion models can capture the hierarchical structure of data, particularly in generating high-quality images. The study focuses on a hierarchical generative model of data, where features are organized in a hierarchical and combinatorial manner. Key findings include: 1. **Phase Transition in Reconstruction**: The backward diffusion process after a certain time \( t \) exhibits a phase transition where the probability of reconstructing high-level features (e.g., the class of an image) suddenly drops. In contrast, low-level features (e.g., specific details of an image) are reconstructed smoothly throughout the diffusion process. 2. ** retention of Low-Level Features**: Even after the class transition, some low-level features from the original image are retained and used to compose the new image. This is observed both in synthetic data and real-world ImageNet images. 3. **Theoretical Insights**: The paper provides theoretical insights into the denoising dynamics of diffusion models using hierarchical generative models. It shows that the denoising process can be solved exactly for these models using belief propagation (BP), revealing a phase transition at a critical noise value. 4. **Empirical Validation**: Numerical experiments on ImageNet confirm the presence of a sharp transition in the class at a given time or noise level, supporting the theoretical predictions. 5. **Hierarchical Generative Models**: The study uses hierarchical generative models to understand the denoising dynamics of diffusion models, providing a theoretical framework for interpreting the behavior of these models. 6. **Implications**: The findings suggest that diffusion models act at different hierarchical levels of the data at different time scales, highlighting the importance of hierarchical generative models in understanding the structure of data and the success of diffusion models in generating high-quality samples. Overall, the paper advances the understanding of how diffusion models capture the hierarchical structure of data and provides a theoretical foundation for further research in machine learning.The paper explores the hierarchical nature of data and its implications for diffusion models. It argues that diffusion models can capture the hierarchical structure of data, particularly in generating high-quality images. The study focuses on a hierarchical generative model of data, where features are organized in a hierarchical and combinatorial manner. Key findings include: 1. **Phase Transition in Reconstruction**: The backward diffusion process after a certain time \( t \) exhibits a phase transition where the probability of reconstructing high-level features (e.g., the class of an image) suddenly drops. In contrast, low-level features (e.g., specific details of an image) are reconstructed smoothly throughout the diffusion process. 2. ** retention of Low-Level Features**: Even after the class transition, some low-level features from the original image are retained and used to compose the new image. This is observed both in synthetic data and real-world ImageNet images. 3. **Theoretical Insights**: The paper provides theoretical insights into the denoising dynamics of diffusion models using hierarchical generative models. It shows that the denoising process can be solved exactly for these models using belief propagation (BP), revealing a phase transition at a critical noise value. 4. **Empirical Validation**: Numerical experiments on ImageNet confirm the presence of a sharp transition in the class at a given time or noise level, supporting the theoretical predictions. 5. **Hierarchical Generative Models**: The study uses hierarchical generative models to understand the denoising dynamics of diffusion models, providing a theoretical framework for interpreting the behavior of these models. 6. **Implications**: The findings suggest that diffusion models act at different hierarchical levels of the data at different time scales, highlighting the importance of hierarchical generative models in understanding the structure of data and the success of diffusion models in generating high-quality samples. Overall, the paper advances the understanding of how diffusion models capture the hierarchical structure of data and provides a theoretical foundation for further research in machine learning.