RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customization

RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customization

1 Mar 2024 | Mengqi Huang, Zhendong Mao, Mingcong Liu, Qian He, Yongdong Zhang
RealCustom is a novel paradigm for text-to-image customization that addresses the dual-optimum paradox in existing methods, which struggle to balance the similarity of the given subject and the controllability of the given text. By disentangling the similarity and controllability components, RealCustom achieves both high-quality similarity and controllability in real-time open-domain scenarios. The key innovation lies in progressively narrowing a real text word from its general connotation to the specific subject, using cross-attention to distinguish relevance. During training, an adaptive scoring module learns to modulate the influence quantity based on text and generated features. During inference, an adaptive mask guidance strategy updates the influence scope and quantity of the given subjects iteratively. Extensive experiments demonstrate that RealCustom outperforms existing methods in terms of controllability and similarity, achieving superior results in both qualitative and quantitative evaluations.RealCustom is a novel paradigm for text-to-image customization that addresses the dual-optimum paradox in existing methods, which struggle to balance the similarity of the given subject and the controllability of the given text. By disentangling the similarity and controllability components, RealCustom achieves both high-quality similarity and controllability in real-time open-domain scenarios. The key innovation lies in progressively narrowing a real text word from its general connotation to the specific subject, using cross-attention to distinguish relevance. During training, an adaptive scoring module learns to modulate the influence quantity based on text and generated features. During inference, an adaptive mask guidance strategy updates the influence scope and quantity of the given subjects iteratively. Extensive experiments demonstrate that RealCustom outperforms existing methods in terms of controllability and similarity, achieving superior results in both qualitative and quantitative evaluations.
Reach us at info@study.space