Context-Aware Interaction Network for RGB-T Semantic Segmentation

Context-Aware Interaction Network for RGB-T Semantic Segmentation

3 Jan 2024 | Ying Lv, Zhi Liu, Senior Member, IEEE, Gongyang Li
The paper introduces the Context-Aware Interaction Network (CAINet) for RGB-T semantic segmentation, addressing the issue of ineffective exploration of the complementary relationship between different modalities in existing methods. CAINet constructs an interaction space to exploit auxiliary tasks and global context for explicit guided learning. It includes the Context-Aware Complementary Reasoning (CACR) module to establish complementary relationships between multimodal features, the Global Context Modeling (GCM) module to provide global context guidance, and the Detail Aggregation (DA) module to refine segmentation results. Auxiliary supervision is introduced to guide multi-level feature interactions, enhancing the representation of multi-level fused features. Extensive experiments on the MFNet and PST900 datasets demonstrate that CAINet achieves state-of-the-art performance. The code is available at <https://github.com/YingLv1106/CAINet>.The paper introduces the Context-Aware Interaction Network (CAINet) for RGB-T semantic segmentation, addressing the issue of ineffective exploration of the complementary relationship between different modalities in existing methods. CAINet constructs an interaction space to exploit auxiliary tasks and global context for explicit guided learning. It includes the Context-Aware Complementary Reasoning (CACR) module to establish complementary relationships between multimodal features, the Global Context Modeling (GCM) module to provide global context guidance, and the Detail Aggregation (DA) module to refine segmentation results. Auxiliary supervision is introduced to guide multi-level feature interactions, enhancing the representation of multi-level fused features. Extensive experiments on the MFNet and PST900 datasets demonstrate that CAINet achieves state-of-the-art performance. The code is available at <https://github.com/YingLv1106/CAINet>.
Reach us at info@study.space
[slides and audio] Context-Aware Interaction Network for RGB-T Semantic Segmentation