Frequency-Adaptive Pan-Sharpening with Mixture of Experts

Frequency-Adaptive Pan-Sharpening with Mixture of Experts

4 Jan 2024 | Xuanhua He1,2*, Keyu Yan1,2*, Rui Li1, Chengjun Xie1, Jie Zhang1†, Man Zhou3†
The paper introduces a novel method called Frequency Adaptive Mixture of Experts (FAME) for pansharpening, which aims to reconstruct high-frequency information in multi-spectral images using a higher-resolution panchromatic image as guidance. The FAME framework consists of three key components: the Adaptive Frequency Separation Prediction Module, the Sub-Frequency Learning Expert Module, and the Expert Mixture Module. The first module uses the Discrete Cosine Transform (DCT) to predict frequency masks, while the second module processes low- and high-frequency information separately using low-frequency MOE and high-frequency MOE, respectively. The final module dynamically fuses high- and low-frequency features to adapt to remote sensing images with significant content variations. Experiments on multiple datasets demonstrate that the proposed method outperforms state-of-the-art methods in terms of both quantitative and qualitative metrics, showing strong generalization ability for real-world scenes. The code for the method will be made publicly available.The paper introduces a novel method called Frequency Adaptive Mixture of Experts (FAME) for pansharpening, which aims to reconstruct high-frequency information in multi-spectral images using a higher-resolution panchromatic image as guidance. The FAME framework consists of three key components: the Adaptive Frequency Separation Prediction Module, the Sub-Frequency Learning Expert Module, and the Expert Mixture Module. The first module uses the Discrete Cosine Transform (DCT) to predict frequency masks, while the second module processes low- and high-frequency information separately using low-frequency MOE and high-frequency MOE, respectively. The final module dynamically fuses high- and low-frequency features to adapt to remote sensing images with significant content variations. Experiments on multiple datasets demonstrate that the proposed method outperforms state-of-the-art methods in terms of both quantitative and qualitative metrics, showing strong generalization ability for real-world scenes. The code for the method will be made publicly available.
Reach us at info@study.space
[slides and audio] Frequency-Adaptive Pan-Sharpening with Mixture of Experts