Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification

Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification

29 Jul 2024 | Jiangming Shi, Xiangbo Yin, Yeyun Chen, Yachao Zhang, Zhizhong Zhang, Yuan Xie, and Yanyun Qu
This paper proposes a Multi-Memory Matching (MMM) framework for unsupervised visible-infrared person re-identification (USL-VI-ReID). The key challenges in USL-VI-ReID are accurately generating pseudo-labels and establishing cross-modality correspondences without prior annotations. Existing methods often rely on a single memory to represent identities, leading to noisy cross-modality correspondences. To address this, MMM introduces a Cross-Modality Clustering (CMC) module to generate pseudo-labels by clustering both visible and infrared samples. A Multi-Memory Learning and Matching (MMLM) module is then designed to establish reliable cross-modality correspondences by using multiple memories to capture individual nuances. Finally, a Soft Cluster-level Alignment (SCA) loss is introduced to narrow the modality gap and mitigate the effect of noisy pseudo-labels through soft cluster-level alignment. Extensive experiments on the SYSU-MM01 and RegDB datasets demonstrate that MMM achieves reliable cross-modality correspondences and outperforms existing USL-VI-ReID methods. The proposed framework effectively addresses the challenges of USL-VI-ReID by leveraging multi-memory representations and cluster-level alignment to improve cross-modality correspondence.This paper proposes a Multi-Memory Matching (MMM) framework for unsupervised visible-infrared person re-identification (USL-VI-ReID). The key challenges in USL-VI-ReID are accurately generating pseudo-labels and establishing cross-modality correspondences without prior annotations. Existing methods often rely on a single memory to represent identities, leading to noisy cross-modality correspondences. To address this, MMM introduces a Cross-Modality Clustering (CMC) module to generate pseudo-labels by clustering both visible and infrared samples. A Multi-Memory Learning and Matching (MMLM) module is then designed to establish reliable cross-modality correspondences by using multiple memories to capture individual nuances. Finally, a Soft Cluster-level Alignment (SCA) loss is introduced to narrow the modality gap and mitigate the effect of noisy pseudo-labels through soft cluster-level alignment. Extensive experiments on the SYSU-MM01 and RegDB datasets demonstrate that MMM achieves reliable cross-modality correspondences and outperforms existing USL-VI-ReID methods. The proposed framework effectively addresses the challenges of USL-VI-ReID by leveraging multi-memory representations and cluster-level alignment to improve cross-modality correspondence.
Reach us at info@study.space
[slides and audio] Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification