**SpatialTracker: Tracking Any 2D Pixels in 3D Space** This paper addresses the challenging problem of dense and long-range pixel motion estimation in videos, particularly in complex scenarios involving occlusions and intricate 3D motion. The authors propose SpatialTracker, a method that lifts 2D pixels into 3D space using monocular depth estimators, representing each frame efficiently with a triplane representation, and performing iterative updates using a transformer to estimate 3D trajectories. Tracking in 3D allows for leveraging geometric priors, such as the as-rigid-as-possible (ARAP) constraint, to improve tracking accuracy and handle occlusions. The method is evaluated on various benchmarks, including TAP-Vid, BADIA, and PointOdyssey, demonstrating state-of-the-art performance in both qualitative and quantitative metrics, especially in challenging scenarios like out-of-plane rotation. The project page is available at <https://henry123-boy.github.io/SpaTracker/>. The paper introduces SpatialTracker, a novel framework for tracking any 2D pixels in 3D space. It leverages monocular depth estimators to lift 2D pixels into 3D, representing each frame with a triplane feature map, and uses a transformer to iteratively update 3D trajectories. The ARAP constraint is enforced during training to ensure spatial consistency, and a rigidity embedding is learned to group pixels into rigid parts. Extensive experiments show that SpatialTracker outperforms existing methods in various benchmarks, particularly in handling complex and occluded motions. The method's effectiveness is further demonstrated through qualitative comparisons and ablation studies.**SpatialTracker: Tracking Any 2D Pixels in 3D Space** This paper addresses the challenging problem of dense and long-range pixel motion estimation in videos, particularly in complex scenarios involving occlusions and intricate 3D motion. The authors propose SpatialTracker, a method that lifts 2D pixels into 3D space using monocular depth estimators, representing each frame efficiently with a triplane representation, and performing iterative updates using a transformer to estimate 3D trajectories. Tracking in 3D allows for leveraging geometric priors, such as the as-rigid-as-possible (ARAP) constraint, to improve tracking accuracy and handle occlusions. The method is evaluated on various benchmarks, including TAP-Vid, BADIA, and PointOdyssey, demonstrating state-of-the-art performance in both qualitative and quantitative metrics, especially in challenging scenarios like out-of-plane rotation. The project page is available at <https://henry123-boy.github.io/SpaTracker/>. The paper introduces SpatialTracker, a novel framework for tracking any 2D pixels in 3D space. It leverages monocular depth estimators to lift 2D pixels into 3D, representing each frame with a triplane feature map, and uses a transformer to iteratively update 3D trajectories. The ARAP constraint is enforced during training to ensure spatial consistency, and a rigidity embedding is learned to group pixels into rigid parts. Extensive experiments show that SpatialTracker outperforms existing methods in various benchmarks, particularly in handling complex and occluded motions. The method's effectiveness is further demonstrated through qualitative comparisons and ablation studies.