10 Jul 2024 | Yu Guo, Yuan Gao, Yuxu Lu, Huilin Zhu, Ryan Wen Liu, Shengfeng He
OneRestore is a novel, transformer-based framework designed for adaptive and controllable image restoration in complex, composite degradation scenarios. The framework addresses the limitations of existing methods that typically target isolated degradation types, such as low light, haze, rain, and snow. OneRestore introduces a versatile imaging model that consolidates four physical corruption paradigms to accurately represent complex degradation scenarios. It employs a unique cross-attention mechanism that merges degraded scene descriptors with image features, allowing for nuanced restoration. The model supports versatile input scene descriptors, including manual text embeddings and automatic extractions based on visual attributes. To enhance the model's robustness, a composite degradation restoration loss is used, utilizing additional degraded images as negative samples to establish stringent lower-bound constraints. Comparative results on synthetic and real-world datasets demonstrate OneRestore's superior performance in addressing complex, composite degradations, outperforming state-of-the-art methods in both synthetic and real-world datasets. The framework's effectiveness is further validated through an extensive ablation study, which highlights the importance of different network modules and loss functions. OneRestore's controllability is demonstrated through experiments on intricate synthetic and real-world scenarios, where the model can selectively focus on different degradation factors, achieving targeted restorations. The framework's limitations and future research directions are also discussed, emphasizing the need for further refinement to handle extremely high-density corruption and high-complex corruption scenarios.OneRestore is a novel, transformer-based framework designed for adaptive and controllable image restoration in complex, composite degradation scenarios. The framework addresses the limitations of existing methods that typically target isolated degradation types, such as low light, haze, rain, and snow. OneRestore introduces a versatile imaging model that consolidates four physical corruption paradigms to accurately represent complex degradation scenarios. It employs a unique cross-attention mechanism that merges degraded scene descriptors with image features, allowing for nuanced restoration. The model supports versatile input scene descriptors, including manual text embeddings and automatic extractions based on visual attributes. To enhance the model's robustness, a composite degradation restoration loss is used, utilizing additional degraded images as negative samples to establish stringent lower-bound constraints. Comparative results on synthetic and real-world datasets demonstrate OneRestore's superior performance in addressing complex, composite degradations, outperforming state-of-the-art methods in both synthetic and real-world datasets. The framework's effectiveness is further validated through an extensive ablation study, which highlights the importance of different network modules and loss functions. OneRestore's controllability is demonstrated through experiments on intricate synthetic and real-world scenarios, where the model can selectively focus on different degradation factors, achieving targeted restorations. The framework's limitations and future research directions are also discussed, emphasizing the need for further refinement to handle extremely high-density corruption and high-complex corruption scenarios.