TOWARDS SEAMLESS ADAPTATION OF PRE-TRAINED MODELS FOR VISUAL PLACE RECOGNITION

TOWARDS SEAMLESS ADAPTATION OF PRE-TRAINED MODELS FOR VISUAL PLACE RECOGNITION

3 Apr 2024 | Feng Lu, Lijun Zhang, Xiangyuan Lan, Shuting Dong, Yaowei Wang, Chun Yuan
This paper proposes a novel method, SelaVPR, for seamless adaptation of pre-trained foundation models in visual place recognition (VPR). The method combines global and local adaptation to produce both global and local features for VPR tasks. The global adaptation is achieved by adding lightweight adapters to the pre-trained model, while the local adaptation is implemented by upsampling the feature map to generate dense local features. A mutual nearest neighbor local feature loss is introduced to guide effective adaptation and optimize the network. The proposed method outperforms state-of-the-art methods on several VPR benchmarks, achieving the best performance on the MSLS challenge leaderboard. It also significantly reduces retrieval runtime compared to two-stage VPR methods with RANSAC-based spatial verification. The method is efficient, requiring only 3% of the runtime of such methods. The results show that the proposed SelaVPR method provides a promising way to address the VPR task in real-world large-scale applications.This paper proposes a novel method, SelaVPR, for seamless adaptation of pre-trained foundation models in visual place recognition (VPR). The method combines global and local adaptation to produce both global and local features for VPR tasks. The global adaptation is achieved by adding lightweight adapters to the pre-trained model, while the local adaptation is implemented by upsampling the feature map to generate dense local features. A mutual nearest neighbor local feature loss is introduced to guide effective adaptation and optimize the network. The proposed method outperforms state-of-the-art methods on several VPR benchmarks, achieving the best performance on the MSLS challenge leaderboard. It also significantly reduces retrieval runtime compared to two-stage VPR methods with RANSAC-based spatial verification. The method is efficient, requiring only 3% of the runtime of such methods. The results show that the proposed SelaVPR method provides a promising way to address the VPR task in real-world large-scale applications.
Reach us at info@study.space