This paper presents a novel cross-modal 6-DoF tracking approach for event cameras, leveraging semi-dense 3D point cloud priors from depth cameras or regular camera-based mapping algorithms. The method uses geometric 3D-2D registration of semi-dense maps and events, achieving highly reliable and accurate tracking results in challenging conditions. The approach includes a novel polarity-aware registration using Signed Time Surface Maps (STSMs) and a culling strategy for occluded points, enhancing speed and robustness. The framework is validated on various real datasets, demonstrating superior performance compared to similar solutions using regular cameras. The paper also introduces an open-source framework that supports both depth camera-supported tracking and map-based localization with semi-dense maps created by monocular visual SLAM or structure-from-motion systems. The experimental results show that the proposed method outperforms alternative methods in terms of accuracy and efficiency, making it suitable for applications such as AR, autonomous parking, and campus navigation.This paper presents a novel cross-modal 6-DoF tracking approach for event cameras, leveraging semi-dense 3D point cloud priors from depth cameras or regular camera-based mapping algorithms. The method uses geometric 3D-2D registration of semi-dense maps and events, achieving highly reliable and accurate tracking results in challenging conditions. The approach includes a novel polarity-aware registration using Signed Time Surface Maps (STSMs) and a culling strategy for occluded points, enhancing speed and robustness. The framework is validated on various real datasets, demonstrating superior performance compared to similar solutions using regular cameras. The paper also introduces an open-source framework that supports both depth camera-supported tracking and map-based localization with semi-dense maps created by monocular visual SLAM or structure-from-motion systems. The experimental results show that the proposed method outperforms alternative methods in terms of accuracy and efficiency, making it suitable for applications such as AR, autonomous parking, and campus navigation.