The paper introduces a novel approach called Map-Relative Pose Regression (marepo) for visual relocalization, which combines a scene-agnostic pose regression network with a scene-specific metric representation. This method addresses the limitations of traditional Absolute Pose Regression (APR) methods, which require extensive training data and struggle with generalization to new scenes. Marepo uses a fast-training scene coordinate regression model to encode the geometric priors of each scene, allowing the pose regression network to learn the generic relationship between scene-specific map representations and camera poses. The proposed approach is evaluated on two public datasets, indoor and outdoor, demonstrating superior performance in terms of accuracy and scalability compared to existing methods. The code for marepo is available at <https://nianticlabs.github.io/marepo>.The paper introduces a novel approach called Map-Relative Pose Regression (marepo) for visual relocalization, which combines a scene-agnostic pose regression network with a scene-specific metric representation. This method addresses the limitations of traditional Absolute Pose Regression (APR) methods, which require extensive training data and struggle with generalization to new scenes. Marepo uses a fast-training scene coordinate regression model to encode the geometric priors of each scene, allowing the pose regression network to learn the generic relationship between scene-specific map representations and camera poses. The proposed approach is evaluated on two public datasets, indoor and outdoor, demonstrating superior performance in terms of accuracy and scalability compared to existing methods. The code for marepo is available at <https://nianticlabs.github.io/marepo>.