Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion

Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion

25 Mar 2024 | Xunpeng Yi, Han Xu, Hao Zhang, Linfeng Tang, Jiayi Ma
Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion This paper proposes a novel text-guided image fusion framework, Text-IF, to address the challenges of degradation-aware and interactive image fusion. Existing fusion methods struggle with degradations in low-quality source images and lack interactivity with user needs. Text-IF introduces a semantic text guidance approach to enhance image fusion performance and handle degradation issues. It integrates text and image fusion to achieve degradation-aware processing and interactive fusion. The framework includes a text semantic encoder and a semantic interaction guidance module. The image fusion pipeline consists of a Transformer-based image extraction module and a cross fusion layer for high-quality fusion. The text semantic encoder aggregates text semantic features from pre-trained vision-language models. The semantic interaction guidance module couples text and image features to achieve text-guided image fusion. Text-IF solves the problem of existing methods being unable to adapt to complex scenes with degradations and can only output fixed results. It provides a feasible direction for future text-guided image fusion tasks. The proposed method achieves multi-modal image fusion and multi-modal information fusion. Extensive experiments show that Text-IF outperforms state-of-the-art methods in image fusion performance and degradation treatment. The code is available at https://github.com/XunpengYi/Text-IF.Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion This paper proposes a novel text-guided image fusion framework, Text-IF, to address the challenges of degradation-aware and interactive image fusion. Existing fusion methods struggle with degradations in low-quality source images and lack interactivity with user needs. Text-IF introduces a semantic text guidance approach to enhance image fusion performance and handle degradation issues. It integrates text and image fusion to achieve degradation-aware processing and interactive fusion. The framework includes a text semantic encoder and a semantic interaction guidance module. The image fusion pipeline consists of a Transformer-based image extraction module and a cross fusion layer for high-quality fusion. The text semantic encoder aggregates text semantic features from pre-trained vision-language models. The semantic interaction guidance module couples text and image features to achieve text-guided image fusion. Text-IF solves the problem of existing methods being unable to adapt to complex scenes with degradations and can only output fixed results. It provides a feasible direction for future text-guided image fusion tasks. The proposed method achieves multi-modal image fusion and multi-modal information fusion. Extensive experiments show that Text-IF outperforms state-of-the-art methods in image fusion performance and degradation treatment. The code is available at https://github.com/XunpengYi/Text-IF.
Reach us at info@study.space
Understanding Text-IF%3A Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion