Understanding the Impact of Negative Prompts: When and How Do They Take Effect?

Understanding the Impact of Negative Prompts: When and How Do They Take Effect?

5 Jun 2024 | Yuanhao Ban, Ruochen Wang, Tianyi Zhou, Minhao Cheng, Boqing Gong, Cho-Jui Hsieh
This paper explores the mechanisms and effects of negative prompts in text-to-image diffusion models, a concept that allows users to specify what should not be generated in the output images. The study identifies two primary behaviors of negative prompts: *Delayed Effect*, where their impact is observed after positive prompts have rendered corresponding content, and *Deletion Through Neutralization*, where negative prompts delete concepts from the image through mutual cancellation with positive prompts in the latent space. The research reveals significant potential applications, such as object inpainting with minimal background alterations. The authors introduce a novel controllable inpainting approach that strategically applies negative prompts during the reverse-diffusion process, demonstrating its effectiveness in removing undesired elements while preserving the image's integrity. The findings offer valuable insights for enhancing the use of negative prompts in image generation tasks.This paper explores the mechanisms and effects of negative prompts in text-to-image diffusion models, a concept that allows users to specify what should not be generated in the output images. The study identifies two primary behaviors of negative prompts: *Delayed Effect*, where their impact is observed after positive prompts have rendered corresponding content, and *Deletion Through Neutralization*, where negative prompts delete concepts from the image through mutual cancellation with positive prompts in the latent space. The research reveals significant potential applications, such as object inpainting with minimal background alterations. The authors introduce a novel controllable inpainting approach that strategically applies negative prompts during the reverse-diffusion process, demonstrating its effectiveness in removing undesired elements while preserving the image's integrity. The findings offer valuable insights for enhancing the use of negative prompts in image generation tasks.
Reach us at info@study.space