Towards Effective Multiple-in-One Image Restoration: A Sequential and Prompt Learning Strategy

Towards Effective Multiple-in-One Image Restoration: A Sequential and Prompt Learning Strategy

20 Mar 2024 | Xiangtao Kong1,2, Chao Dong3,4, Lei Zhang1,2 *
This paper addresses the challenge of training a single model to handle multiple image restoration (IR) tasks, known as the Multiple-in-One (MiO) IR problem. The authors identify two main challenges: optimizing diverse objectives and adapting to different tasks. To tackle these challenges, they propose two strategies: sequential learning and prompt learning. Sequential learning involves training the model incrementally, focusing on tasks that reconstruct high-frequency details first, followed by tasks that adjust global luminance. Prompt learning uses explicit or adaptive prompts to help the network understand and adapt to specific tasks. The effectiveness of these strategies is evaluated on 19 test sets, demonstrating significant improvements in performance for both CNN and Transformer backbones. The proposed strategies also enhance the state-of-the-art method PromptIR with fewer parameters. The paper concludes by highlighting the potential of these strategies in advancing the field of MiO IR.This paper addresses the challenge of training a single model to handle multiple image restoration (IR) tasks, known as the Multiple-in-One (MiO) IR problem. The authors identify two main challenges: optimizing diverse objectives and adapting to different tasks. To tackle these challenges, they propose two strategies: sequential learning and prompt learning. Sequential learning involves training the model incrementally, focusing on tasks that reconstruct high-frequency details first, followed by tasks that adjust global luminance. Prompt learning uses explicit or adaptive prompts to help the network understand and adapt to specific tasks. The effectiveness of these strategies is evaluated on 19 test sets, demonstrating significant improvements in performance for both CNN and Transformer backbones. The proposed strategies also enhance the state-of-the-art method PromptIR with fewer parameters. The paper concludes by highlighting the potential of these strategies in advancing the field of MiO IR.
Reach us at info@study.space