Diffusion Model-Based Image Editing: A Survey

Diffusion Model-Based Image Editing: A Survey

16 Mar 2024 | Yi Huang, Jiancheng Huang, Yifan Liu, Mingfu Yan, Jiaxi Lv, Jianzhuang Liu, Wei Xiong, He Zhang, Shifeng Chen, and Liangliang Cao
This survey provides an overview of diffusion model-based image editing, covering both theoretical and practical aspects. Diffusion models, which work by gradually adding noise to data and then learning to reverse this process, have become powerful tools for image generation and editing. They enable the synthesis of visual content in an unconditional or input-conditional manner, allowing for high-quality samples from complex distributions. The survey analyzes and categorizes existing methods from multiple perspectives, including learning strategies, user-input conditions, and specific editing tasks. It pays special attention to image inpainting and outpainting, exploring both traditional context-driven and current multimodal conditional methods. A benchmark called EditEval is introduced to evaluate text-guided image editing algorithms, featuring an innovative metric, LMM Score. The survey also addresses current limitations and potential future research directions. The accompanying repository is available at <https://github.com/SiatMMLab/Awesome-Diffusion-Model-Based-Image-Editing-Methods>. Index Terms—Diffusion Model, Image Editing, AIGC.This survey provides an overview of diffusion model-based image editing, covering both theoretical and practical aspects. Diffusion models, which work by gradually adding noise to data and then learning to reverse this process, have become powerful tools for image generation and editing. They enable the synthesis of visual content in an unconditional or input-conditional manner, allowing for high-quality samples from complex distributions. The survey analyzes and categorizes existing methods from multiple perspectives, including learning strategies, user-input conditions, and specific editing tasks. It pays special attention to image inpainting and outpainting, exploring both traditional context-driven and current multimodal conditional methods. A benchmark called EditEval is introduced to evaluate text-guided image editing algorithms, featuring an innovative metric, LMM Score. The survey also addresses current limitations and potential future research directions. The accompanying repository is available at <https://github.com/SiatMMLab/Awesome-Diffusion-Model-Based-Image-Editing-Methods>. Index Terms—Diffusion Model, Image Editing, AIGC.
Reach us at info@study.space