29 Jan 2024 | Qinghe Wang, Xu Jia, Xiaomin Li, Taiqing Li, Liqian Ma, Yunzhi Zhuge, Huchuan Lu
StableIdentity is a novel framework designed to enable identity-consistent recontextualization with just one face image. The method leverages a face encoder with an identity prior to encode the input face and then projects the face representation into a space with an editable prior, constructed from celebrity names. By incorporating both identity and editability priors, StableIdentity can inject the learned identity into various contexts, ensuring stable identity preservation and flexible editability. The framework also includes a masked two-phase diffusion loss to enhance pixel-level perception and maintain generation diversity. Extensive experiments demonstrate that StableIdentity outperforms existing customization methods, achieving superior results in identity preservation, editability, and image quality. Notably, StableIdentity can be seamlessly integrated with off-the-shelf modules like ControlNet and even enable zero-shot identity-driven video and 3D generation without fine-tuning. The method's effectiveness is further validated through comprehensive ablation studies and comparisons with various baselines, showcasing its robustness and generalization capabilities.StableIdentity is a novel framework designed to enable identity-consistent recontextualization with just one face image. The method leverages a face encoder with an identity prior to encode the input face and then projects the face representation into a space with an editable prior, constructed from celebrity names. By incorporating both identity and editability priors, StableIdentity can inject the learned identity into various contexts, ensuring stable identity preservation and flexible editability. The framework also includes a masked two-phase diffusion loss to enhance pixel-level perception and maintain generation diversity. Extensive experiments demonstrate that StableIdentity outperforms existing customization methods, achieving superior results in identity preservation, editability, and image quality. Notably, StableIdentity can be seamlessly integrated with off-the-shelf modules like ControlNet and even enable zero-shot identity-driven video and 3D generation without fine-tuning. The method's effectiveness is further validated through comprehensive ablation studies and comparisons with various baselines, showcasing its robustness and generalization capabilities.