Factorized Diffusion: Perceptual Illusions by Noise Decomposition

Factorized Diffusion: Perceptual Illusions by Noise Decomposition

17 Apr 2024 | Daniel Geng*, Inbum Park*, and Andrew Owens
Factorized Diffusion is a method that enables control over individual components of an image through diffusion model sampling. By decomposing an image into components such as spatial frequencies, color, and motion blur, the method allows for generating hybrid images that change appearance based on viewing conditions. For example, images can be conditioned on different text prompts to produce hybrid images that change appearance when viewed from different distances or under different lighting conditions. The method uses a composite noise estimate built from components conditioned on different prompts to achieve this effect. It also allows for generating hybrid images from real images by fixing one component and generating the rest. The approach is compared to traditional methods and shown to produce higher quality results. The method is also related to prior work on compositional generation and spatial control. The paper presents results for various decompositions, including spatial frequencies, color spaces, and motion blur, and shows how the method can be applied to generate perceptual illusions. The method is also shown to solve inverse problems by fixing one component and generating the rest. The paper concludes that the method provides a new approach to generating hybrid images and perceptual illusions through diffusion models.Factorized Diffusion is a method that enables control over individual components of an image through diffusion model sampling. By decomposing an image into components such as spatial frequencies, color, and motion blur, the method allows for generating hybrid images that change appearance based on viewing conditions. For example, images can be conditioned on different text prompts to produce hybrid images that change appearance when viewed from different distances or under different lighting conditions. The method uses a composite noise estimate built from components conditioned on different prompts to achieve this effect. It also allows for generating hybrid images from real images by fixing one component and generating the rest. The approach is compared to traditional methods and shown to produce higher quality results. The method is also related to prior work on compositional generation and spatial control. The paper presents results for various decompositions, including spatial frequencies, color spaces, and motion blur, and shows how the method can be applied to generate perceptual illusions. The method is also shown to solve inverse problems by fixing one component and generating the rest. The paper concludes that the method provides a new approach to generating hybrid images and perceptual illusions through diffusion models.
Reach us at info@study.space
Understanding Factorized Diffusion%3A Perceptual Illusions by Noise Decomposition