Grounding Image Matching in 3D with MAST3R

Grounding Image Matching in 3D with MAST3R

June 2024 | Vincent Leroy, Yohann Cabon, Jerome Revaud
MASt3R is a 3D-aware image matching method that improves upon DUSt3R by adding a new head that outputs dense local features and training with an additional matching loss. This approach enables highly accurate and robust matching, even in challenging scenarios with extreme viewpoint changes. MASt3R also introduces a fast reciprocal matching scheme that significantly accelerates the matching process while maintaining theoretical guarantees and improving performance. The method is evaluated on multiple benchmarks, including the Map-free localization dataset, where it outperforms the state of the art by 30% in VCRE AUC. MASt3R is capable of handling high-resolution images through a coarse-to-fine matching scheme, and it achieves state-of-the-art results in both camera pose estimation and 3D scene reconstruction. The method is also effective in dense multi-view stereo (MVS) reconstruction, where it outperforms existing approaches in terms of accuracy and completeness. Overall, MASt3R demonstrates the effectiveness of grounding image matching in 3D, leading to improved performance in a variety of vision tasks.MASt3R is a 3D-aware image matching method that improves upon DUSt3R by adding a new head that outputs dense local features and training with an additional matching loss. This approach enables highly accurate and robust matching, even in challenging scenarios with extreme viewpoint changes. MASt3R also introduces a fast reciprocal matching scheme that significantly accelerates the matching process while maintaining theoretical guarantees and improving performance. The method is evaluated on multiple benchmarks, including the Map-free localization dataset, where it outperforms the state of the art by 30% in VCRE AUC. MASt3R is capable of handling high-resolution images through a coarse-to-fine matching scheme, and it achieves state-of-the-art results in both camera pose estimation and 3D scene reconstruction. The method is also effective in dense multi-view stereo (MVS) reconstruction, where it outperforms existing approaches in terms of accuracy and completeness. Overall, MASt3R demonstrates the effectiveness of grounding image matching in 3D, leading to improved performance in a variety of vision tasks.
Reach us at info@study.space
Understanding Grounding Image Matching in 3D with MASt3R