ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models

ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models

4 Mar 2024 | Jiaxiang Cheng, Pan Xie*, Xin Xia, Jiashi Li, Jie Wu, Yuxi Ren, Huixia Li, Xuefeng Xiao, Min Zheng, Lean Fu
ResAdapter is a domain-consistent resolution adapter for diffusion models that enables generation of images with unrestricted resolutions and aspect ratios without altering the original style domain. Unlike existing methods that rely on complex post-processing, ResAdapter directly generates images with dynamic resolutions. It consists of two key components: ResCLoRA and ResENorm. ResCLoRA is designed to learn resolution priors by inserting low-rank adaptation into convolution layers, while ResENorm improves resolution extrapolation by optimizing normalization layers. Together, they allow diffusion models to generate high-quality images across various resolutions and aspect ratios. ResAdapter is lightweight, efficient, and compatible with other modules such as ControlNet, IP-Adapter, and LCM-LoRA. It can be integrated into other multi-resolution models like ElasticDiffusion to enhance generation efficiency. Experiments show that ResAdapter with only 0.5M parameters can generate images with resolutions ranging from 128x128 to 1536x1536 for different diffusion models. It outperforms existing methods in terms of image quality and inference efficiency, and is compatible with various modules for flexible resolution generation. The method is effective in preserving the original style domain while enabling resolution interpolation and extrapolation.ResAdapter is a domain-consistent resolution adapter for diffusion models that enables generation of images with unrestricted resolutions and aspect ratios without altering the original style domain. Unlike existing methods that rely on complex post-processing, ResAdapter directly generates images with dynamic resolutions. It consists of two key components: ResCLoRA and ResENorm. ResCLoRA is designed to learn resolution priors by inserting low-rank adaptation into convolution layers, while ResENorm improves resolution extrapolation by optimizing normalization layers. Together, they allow diffusion models to generate high-quality images across various resolutions and aspect ratios. ResAdapter is lightweight, efficient, and compatible with other modules such as ControlNet, IP-Adapter, and LCM-LoRA. It can be integrated into other multi-resolution models like ElasticDiffusion to enhance generation efficiency. Experiments show that ResAdapter with only 0.5M parameters can generate images with resolutions ranging from 128x128 to 1536x1536 for different diffusion models. It outperforms existing methods in terms of image quality and inference efficiency, and is compatible with various modules for flexible resolution generation. The method is effective in preserving the original style domain while enabling resolution interpolation and extrapolation.
Reach us at info@study.space