SDEdit: GUIDED IMAGE SYNTHESIS AND EDITING WITH STOCHASTIC DIFFERENTIAL EQUATIONS

SDEdit: GUIDED IMAGE SYNTHESIS AND EDITING WITH STOCHASTIC DIFFERENTIAL EQUATIONS

5 Jan 2022 | Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, Stefano Ermon
SDEdit is a novel method for guided image synthesis and editing using stochastic differential equations (SDEs). It enables users to create and edit realistic images with minimal effort by iteratively denoising an input image through an SDE prior. Given a user-provided guide in the form of RGB pixel manipulation, SDEdit adds noise to the input and then denoises it using the SDE to enhance realism while preserving faithfulness to the guide. Unlike existing GAN-based methods, SDEdit does not require task-specific training or inversions, making it more efficient and versatile. It outperforms state-of-the-art GAN-based methods in realism and overall satisfaction scores, achieving up to 98.09% improvement in realism and 91.72% in overall satisfaction on stroke-based image synthesis and editing tasks. SDEdit also excels in image compositing, with up to 83.73% improvement in overall satisfaction scores. The method is based on diffusion models and leverages the SDE framework to balance realism and faithfulness, allowing for flexible and controllable image editing. SDEdit is applicable to various tasks, including stroke-based image synthesis, editing, and compositing, and does not require paired datasets or specific loss functions. The method is evaluated on multiple datasets, including LSUN and CelebA-HQ, and demonstrates superior performance in both quantitative and qualitative assessments. SDEdit provides a unified framework for image synthesis and editing, offering a more efficient and effective alternative to existing methods.SDEdit is a novel method for guided image synthesis and editing using stochastic differential equations (SDEs). It enables users to create and edit realistic images with minimal effort by iteratively denoising an input image through an SDE prior. Given a user-provided guide in the form of RGB pixel manipulation, SDEdit adds noise to the input and then denoises it using the SDE to enhance realism while preserving faithfulness to the guide. Unlike existing GAN-based methods, SDEdit does not require task-specific training or inversions, making it more efficient and versatile. It outperforms state-of-the-art GAN-based methods in realism and overall satisfaction scores, achieving up to 98.09% improvement in realism and 91.72% in overall satisfaction on stroke-based image synthesis and editing tasks. SDEdit also excels in image compositing, with up to 83.73% improvement in overall satisfaction scores. The method is based on diffusion models and leverages the SDE framework to balance realism and faithfulness, allowing for flexible and controllable image editing. SDEdit is applicable to various tasks, including stroke-based image synthesis, editing, and compositing, and does not require paired datasets or specific loss functions. The method is evaluated on multiple datasets, including LSUN and CelebA-HQ, and demonstrates superior performance in both quantitative and qualitative assessments. SDEdit provides a unified framework for image synthesis and editing, offering a more efficient and effective alternative to existing methods.
Reach us at info@study.space
Understanding SDEdit%3A Guided Image Synthesis and Editing with Stochastic Differential Equations