Face2Diffusion for Fast and Editable Face Personalization

Face2Diffusion for Fast and Editable Face Personalization

8 Mar 2024 | Kaede Shiohara Toshihiko Yamasaki
Face2Diffusion (F2D) is a novel method for fast and editable face personalization in text-to-image (T2I) diffusion models. The core idea of F2D is to remove identity-irrelevant information from the training pipeline to prevent overfitting and improve the editability of encoded faces. F2D consists of three main components: 1) Multi-scale identity encoder (MSID) which provides well-disentangled identity features while maintaining multi-scale information, enhancing camera pose diversity. 2) Expression guidance which disentangles face expressions from identities, improving the controllability of face expressions. 3) Class-guided denoising regularization (CGDR) which encourages the model to learn how faces should be denoised, improving text-alignment of backgrounds. Extensive experiments on the FaceForensics++ dataset and diverse prompts demonstrate that F2D significantly improves the trade-off between identity and text fidelity compared to previous state-of-the-art methods. The code for F2D is available at <https://github.com/mapoon/Face2Diffusion>.Face2Diffusion (F2D) is a novel method for fast and editable face personalization in text-to-image (T2I) diffusion models. The core idea of F2D is to remove identity-irrelevant information from the training pipeline to prevent overfitting and improve the editability of encoded faces. F2D consists of three main components: 1) Multi-scale identity encoder (MSID) which provides well-disentangled identity features while maintaining multi-scale information, enhancing camera pose diversity. 2) Expression guidance which disentangles face expressions from identities, improving the controllability of face expressions. 3) Class-guided denoising regularization (CGDR) which encourages the model to learn how faces should be denoised, improving text-alignment of backgrounds. Extensive experiments on the FaceForensics++ dataset and diverse prompts demonstrate that F2D significantly improves the trade-off between identity and text fidelity compared to previous state-of-the-art methods. The code for F2D is available at <https://github.com/mapoon/Face2Diffusion>.
Reach us at info@study.space
[slides and audio] Face2Diffusion for Fast and Editable Face Personalization