24 Mar 2024 | Pengfei Zhu, Yang Sun, Bing Cao*, Qinghua Hu
The paper "Task-Customized Mixture of Adapters for General Image Fusion" addresses the challenge of integrating information from multi-source images in various fusion tasks, such as multi-modal, multi-exposure, and multi-focus image fusion. The authors propose a novel method called Task-Customized Mixture of Adapters (TC-MoA) to adaptively prompt different fusion tasks within a unified model. Inspired by the mixture of experts (MoE) architecture, TC-MoA uses a set of task-specific routers and shared adapters to generate task-specific prompts, which are then used for fusion. Mutual information regularization ensures that the adapters complement each other while retaining task-specific information. The proposed method is evaluated on three datasets: LLVIP for multi-modal fusion, MEFB for multi-exposure fusion, and RealMFF for multi-focus fusion. Extensive experiments demonstrate that TC-MoA outperforms existing methods in terms of both quantitative metrics and visual quality, showing strong controllability and generalizability to unseen tasks. The code for the method is available at <https://github.com/YangSun22/TC-MoA>.The paper "Task-Customized Mixture of Adapters for General Image Fusion" addresses the challenge of integrating information from multi-source images in various fusion tasks, such as multi-modal, multi-exposure, and multi-focus image fusion. The authors propose a novel method called Task-Customized Mixture of Adapters (TC-MoA) to adaptively prompt different fusion tasks within a unified model. Inspired by the mixture of experts (MoE) architecture, TC-MoA uses a set of task-specific routers and shared adapters to generate task-specific prompts, which are then used for fusion. Mutual information regularization ensures that the adapters complement each other while retaining task-specific information. The proposed method is evaluated on three datasets: LLVIP for multi-modal fusion, MEFB for multi-exposure fusion, and RealMFF for multi-focus fusion. Extensive experiments demonstrate that TC-MoA outperforms existing methods in terms of both quantitative metrics and visual quality, showing strong controllability and generalizability to unseen tasks. The code for the method is available at <https://github.com/YangSun22/TC-MoA>.