2024-06-12 | Hai Ci, Yiren Song, Pei Yang, Jinheng Xie, Mike Zheng Shou
WMAdapter is a lightweight watermark plugin for latent diffusion models that enables seamless watermark imprinting during image generation. It is efficient, robust, and maintains high image quality. The key design elements include a contextual adapter structure that facilitates knowledge transfer from pretrained models and a hybrid fine-tuning strategy to enhance image quality and eliminate artifacts. WMAdapter accepts user-specified watermark information and imprints it on-the-fly without requiring individual fine-tuning for each watermark. This makes it highly flexible and scalable. Compared to existing methods, WMAdapter achieves strong watermark robustness, high image quality, and competitive performance. It is designed to be plug-and-play, preserving the original VAE decoder and allowing for minimal artifacts. Experimental results show that WMAdapter outperforms other methods in terms of image quality, robustness, and scalability. It is also effective in various scenarios, including different VAE architectures and image generation tasks. The method is evaluated on multiple datasets and demonstrates strong performance under various attacks and distortions. WMAdapter provides a simple yet effective solution for diffusion model watermarking, balancing image quality, robustness, and flexibility.WMAdapter is a lightweight watermark plugin for latent diffusion models that enables seamless watermark imprinting during image generation. It is efficient, robust, and maintains high image quality. The key design elements include a contextual adapter structure that facilitates knowledge transfer from pretrained models and a hybrid fine-tuning strategy to enhance image quality and eliminate artifacts. WMAdapter accepts user-specified watermark information and imprints it on-the-fly without requiring individual fine-tuning for each watermark. This makes it highly flexible and scalable. Compared to existing methods, WMAdapter achieves strong watermark robustness, high image quality, and competitive performance. It is designed to be plug-and-play, preserving the original VAE decoder and allowing for minimal artifacts. Experimental results show that WMAdapter outperforms other methods in terms of image quality, robustness, and scalability. It is also effective in various scenarios, including different VAE architectures and image generation tasks. The method is evaluated on multiple datasets and demonstrates strong performance under various attacks and distortions. WMAdapter provides a simple yet effective solution for diffusion model watermarking, balancing image quality, robustness, and flexibility.