AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation

AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation

21 Mar 2024 | Yuning Cui¹, Syed Waqas Zamir², Salman Khan³,⁴, Alois Knoll¹, Mubarak Shah⁵, and Fahad Shahbaz Khan³,⁶
AdaIR is an adaptive all-in-one image restoration framework that leverages both spatial and frequency domain information to effectively decouple degradations from the desired clean image content. The framework introduces the Adaptive Frequency Learning Block (AFLB), which is a plugin block designed for easy integration into existing image restoration architectures. The AFLB performs two sequential tasks: first, through its Frequency Mining Module (FMiM), it generates low- and high-frequency feature maps via guidance obtained from the spectra decomposition of the original degraded image; second, the Frequency Modulation Module (FMoM) within the AFLB calibrates these features by enabling the exchange of information across different frequency bands to effectively handle diverse types of image degradations. The proposed method achieves adaptive reconstruction by accentuating the informative frequency subbands according to different input degradations. Extensive experiments demonstrate that the proposed method, named AdaIR, achieves state-of-the-art performance on different image restoration tasks, including image denoising, dehazing, deraining, motion deblurring, and low-light image enhancement. The model's code and pre-trained models are available at https://github.com/c-yn/AdaIR. The framework is evaluated on multiple degradation types and shows superior performance compared to existing all-in-one and single-task restoration methods. The results indicate that AdaIR is effective in removing various types of image degradations and generates images that are visually closer to the ground truth than other approaches. The method is also tested on out-of-distribution degradations and shows strong generalization ability. The proposed framework is capable of handling multiple degradations within a single model, making it a versatile solution for image restoration tasks.AdaIR is an adaptive all-in-one image restoration framework that leverages both spatial and frequency domain information to effectively decouple degradations from the desired clean image content. The framework introduces the Adaptive Frequency Learning Block (AFLB), which is a plugin block designed for easy integration into existing image restoration architectures. The AFLB performs two sequential tasks: first, through its Frequency Mining Module (FMiM), it generates low- and high-frequency feature maps via guidance obtained from the spectra decomposition of the original degraded image; second, the Frequency Modulation Module (FMoM) within the AFLB calibrates these features by enabling the exchange of information across different frequency bands to effectively handle diverse types of image degradations. The proposed method achieves adaptive reconstruction by accentuating the informative frequency subbands according to different input degradations. Extensive experiments demonstrate that the proposed method, named AdaIR, achieves state-of-the-art performance on different image restoration tasks, including image denoising, dehazing, deraining, motion deblurring, and low-light image enhancement. The model's code and pre-trained models are available at https://github.com/c-yn/AdaIR. The framework is evaluated on multiple degradation types and shows superior performance compared to existing all-in-one and single-task restoration methods. The results indicate that AdaIR is effective in removing various types of image degradations and generates images that are visually closer to the ground truth than other approaches. The method is also tested on out-of-distribution degradations and shows strong generalization ability. The proposed framework is capable of handling multiple degradations within a single model, making it a versatile solution for image restoration tasks.
Reach us at info@study.space