2024-06-12 | George Cazenavette, Avneesh Sud, Thomas Leung, Ben Usman
FakeInversion is a novel method for detecting synthetic images generated by unseen text-to-image models. The method leverages text-conditioned inversion maps derived from a pre-trained Stable Diffusion model to enhance detection performance. By incorporating additional signals such as inverted latent noise maps and reconstructed input images, the detector achieves state-of-the-art generalization across various text-to-image models, including both open-source and closed-source generators. The method is trained using fake images generated by Stable Diffusion and real LAION images, and it is evaluated using a new benchmark called SynRIS, which uses reverse image search to ensure that the evaluation is not biased towards specific themes or styles. This evaluation protocol is more reliable in assessing the detector's ability to distinguish between real and fake images, especially for closed-source models. The method outperforms existing approaches on both academic benchmarks and the new RIS-based evaluation. The key contributions include the introduction of FakeInversion, the development of SynRIS, and the demonstration of the effectiveness of text-conditioned inversion features in improving generalization. The method is robust to common image degradations and provides interpretable results, highlighting the importance of internal representations in detecting synthetic images. The results show that FakeInversion achieves state-of-the-art performance in detecting images from unseen text-to-image models, and the evaluation protocol ensures that the detector is not biased towards any particular style or theme.FakeInversion is a novel method for detecting synthetic images generated by unseen text-to-image models. The method leverages text-conditioned inversion maps derived from a pre-trained Stable Diffusion model to enhance detection performance. By incorporating additional signals such as inverted latent noise maps and reconstructed input images, the detector achieves state-of-the-art generalization across various text-to-image models, including both open-source and closed-source generators. The method is trained using fake images generated by Stable Diffusion and real LAION images, and it is evaluated using a new benchmark called SynRIS, which uses reverse image search to ensure that the evaluation is not biased towards specific themes or styles. This evaluation protocol is more reliable in assessing the detector's ability to distinguish between real and fake images, especially for closed-source models. The method outperforms existing approaches on both academic benchmarks and the new RIS-based evaluation. The key contributions include the introduction of FakeInversion, the development of SynRIS, and the demonstration of the effectiveness of text-conditioned inversion features in improving generalization. The method is robust to common image degradations and provides interpretable results, highlighting the importance of internal representations in detecting synthetic images. The results show that FakeInversion achieves state-of-the-art performance in detecting images from unseen text-to-image models, and the evaluation protocol ensures that the detector is not biased towards any particular style or theme.