Cameras as Rays: Pose Estimation via Ray Diffusion

Cameras as Rays: Pose Estimation via Ray Diffusion

4 Apr 2024 | Jason Y. Zhang, Amy Lin*, Moneish Kumar, Tzu-Hsuan Yang, Deva Ramanan, Shubham Tulsiani
The paper "Cameras as Rays: Pose Estimation via Ray Diffusion" by Jason Y. Zhang, Amy Lin, Moneish Kumar, Tzu-Hsuan Yang, Deva Ramanan, and Shubham Tulsiani from Carnegie Mellon University proposes a novel approach to camera pose estimation using a distributed ray representation. Unlike traditional methods that predict global camera extrinsics, this work treats a camera as a bundle of rays, allowing for better coupling with spatial image features and improved pose precision. The authors develop a regression-based approach to map image patches to corresponding rays and extend it to a denoising diffusion model to handle uncertainties in sparse-view pose inference. Their methods, both regression- and diffusion-based, achieve state-of-the-art performance on the CO3D dataset while generalizing to unseen object categories and in-the-wild captures. The paper also includes a detailed experimental setup, evaluation metrics, and ablation studies to demonstrate the effectiveness of their approach.The paper "Cameras as Rays: Pose Estimation via Ray Diffusion" by Jason Y. Zhang, Amy Lin, Moneish Kumar, Tzu-Hsuan Yang, Deva Ramanan, and Shubham Tulsiani from Carnegie Mellon University proposes a novel approach to camera pose estimation using a distributed ray representation. Unlike traditional methods that predict global camera extrinsics, this work treats a camera as a bundle of rays, allowing for better coupling with spatial image features and improved pose precision. The authors develop a regression-based approach to map image patches to corresponding rays and extend it to a denoising diffusion model to handle uncertainties in sparse-view pose inference. Their methods, both regression- and diffusion-based, achieve state-of-the-art performance on the CO3D dataset while generalizing to unseen object categories and in-the-wild captures. The paper also includes a detailed experimental setup, evaluation metrics, and ablation studies to demonstrate the effectiveness of their approach.
Reach us at info@study.space
Understanding Cameras as Rays%3A Pose Estimation via Ray Diffusion