Tuning-Free Image Customization with Image and Text Guidance

Tuning-Free Image Customization with Image and Text Guidance

19 Mar 2024 | Pengzhi Li1*, Qiang Nie2*, Ying Chen2, Xi Jiang3, Kai Wu2, Yuhuan Lin2, Yong Liu2, Jinlong Peng2, Chengjie Wang2, Feng Zheng3†
The paper introduces a tuning-free framework for image customization that leverages both text prompts and reference images. The method aims to enable precise editing of specific image regions within seconds, preserving the semantic features of the reference image subject while allowing detailed attribute modifications based on text descriptions. The key innovation is an attention blending strategy that integrates self-attention features in the UNet decoder during the denoising process. This approach addresses the limitations of existing methods, such as unintended changes in non-target areas and time-consuming fine-tuning. The proposed method outperforms previous approaches in both human and quantitative evaluations, making it suitable for various practical applications like image synthesis, design, and creative photography. The paper also includes a detailed experimental setup, comparison with previous works, user studies, and ablation studies to validate the effectiveness of the proposed method.The paper introduces a tuning-free framework for image customization that leverages both text prompts and reference images. The method aims to enable precise editing of specific image regions within seconds, preserving the semantic features of the reference image subject while allowing detailed attribute modifications based on text descriptions. The key innovation is an attention blending strategy that integrates self-attention features in the UNet decoder during the denoising process. This approach addresses the limitations of existing methods, such as unintended changes in non-target areas and time-consuming fine-tuning. The proposed method outperforms previous approaches in both human and quantitative evaluations, making it suitable for various practical applications like image synthesis, design, and creative photography. The paper also includes a detailed experimental setup, comparison with previous works, user studies, and ablation studies to validate the effectiveness of the proposed method.
Reach us at info@study.space
[slides and audio] Tuning-Free Image Customization with Image and Text Guidance