Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification

Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification

29 Jul 2024 | Jiangming Shi, Xiangbo Yin, Yeyun Chen, Yachao Zhang, Zhizhong Zhang, Yuan Xie, Yanyun Qu
The paper "Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification" addresses the challenge of unsupervised visible-infrared (VI) person re-identification (USL-VI-ReID), focusing on accurately generating pseudo-labels and establishing reliable cross-modality correspondences without prior annotations. The key contributions include: 1. **Introduction of ARI Metric**: The authors introduce the Adjusted Rand Index (ARI) to evaluate the quality of pseudo-labels and cross-modality correspondences, revealing that previous methods' cross-modality correspondences are not as reliable as their performance suggests. 2. **Multi-Memory Matching (MMM) Framework**: To improve the reliability of cross-modality correspondences, the MMM framework is proposed. It consists of three main components: - **Cross-Modality Clustering (CMC)**: Generates pseudo-labels by clustering both visible and infrared samples. - **Multi-Memory Learning and Matching (MMLM)**: Establishes reliable cross-modality correspondences by sub-dividing a single memory into multiple memories to capture individual nuances. - **Soft Cluster-level Alignment (SCA) Loss**: Reduces the modality gap and mitigates the impact of noisy pseudo-labels through soft cluster-level alignment. 3. **Experimental Results**: Extensive experiments on the SYSU-MM01 and RegDB datasets demonstrate the effectiveness of MMM. It outperforms several state-of-the-art methods in supervised, semi-supervised, and unsupervised settings, showing improved performance in Rank-1 and mAP metrics. 4. **Ablation Studies**: The effectiveness of each component of MMM is evaluated through ablation studies, confirming the benefits of MMLM and SCA. 5. **Hyper-parameter Analysis**: The impact of key hyper-parameters, such as the number of memories and the balancing weights in SCA, is analyzed, showing that MMM is robust to these settings. 6. **Qualitative Analysis**: Visualizations of pseudo-labels and intra-identity distances further illustrate the effectiveness of MMM in reducing cross-modality distances and establishing reliable correspondences. The paper concludes that MMM is a promising approach for USL-VI-ReID, addressing the critical challenge of reliable cross-modality correspondences.The paper "Multi-Memory Matching for Unsupervised Visible-Infrared Person Re-Identification" addresses the challenge of unsupervised visible-infrared (VI) person re-identification (USL-VI-ReID), focusing on accurately generating pseudo-labels and establishing reliable cross-modality correspondences without prior annotations. The key contributions include: 1. **Introduction of ARI Metric**: The authors introduce the Adjusted Rand Index (ARI) to evaluate the quality of pseudo-labels and cross-modality correspondences, revealing that previous methods' cross-modality correspondences are not as reliable as their performance suggests. 2. **Multi-Memory Matching (MMM) Framework**: To improve the reliability of cross-modality correspondences, the MMM framework is proposed. It consists of three main components: - **Cross-Modality Clustering (CMC)**: Generates pseudo-labels by clustering both visible and infrared samples. - **Multi-Memory Learning and Matching (MMLM)**: Establishes reliable cross-modality correspondences by sub-dividing a single memory into multiple memories to capture individual nuances. - **Soft Cluster-level Alignment (SCA) Loss**: Reduces the modality gap and mitigates the impact of noisy pseudo-labels through soft cluster-level alignment. 3. **Experimental Results**: Extensive experiments on the SYSU-MM01 and RegDB datasets demonstrate the effectiveness of MMM. It outperforms several state-of-the-art methods in supervised, semi-supervised, and unsupervised settings, showing improved performance in Rank-1 and mAP metrics. 4. **Ablation Studies**: The effectiveness of each component of MMM is evaluated through ablation studies, confirming the benefits of MMLM and SCA. 5. **Hyper-parameter Analysis**: The impact of key hyper-parameters, such as the number of memories and the balancing weights in SCA, is analyzed, showing that MMM is robust to these settings. 6. **Qualitative Analysis**: Visualizations of pseudo-labels and intra-identity distances further illustrate the effectiveness of MMM in reducing cross-modality distances and establishing reliable correspondences. The paper concludes that MMM is a promising approach for USL-VI-ReID, addressing the critical challenge of reliable cross-modality correspondences.
Reach us at info@study.space