| Richard A. Newcombe, Steven J. Lovegrove and Andrew J. Davison
DTAM is a real-time system for camera tracking and reconstruction that uses dense, pixel-wise methods instead of feature extraction. As a hand-held RGB camera moves over a static scene, DTAM generates detailed textured depth maps at keyframes to create a surface with millions of vertices. It uses hundreds of images from a video stream to improve a photometric data term and minimizes a global spatially regularized energy function in a non-convex optimization framework. The system tracks the camera's 6DOF motion using whole image alignment against the dense model, achieving real-time performance with current GPU hardware. DTAM outperforms state-of-the-art feature-based methods in tracking under rapid motion and enables real-time scene interaction in augmented reality. The system creates a dense 3D model and uses it for dense camera tracking via whole image registration. It is highly parallelizable and uses a novel non-convex optimization framework with accelerated exact exhaustive search and an interleaved Newton step for fine accuracy. The dense model provides robustness to occlusions and motion blur, making it more accurate than feature-based methods. The system is self-supporting once initialized and uses a dense scene model for simplifying issues with point-based systems. DTAM's dense mapping method uses a global energy minimization framework to estimate inverse depth maps from multiple images, combining photometric error and spatial regularization. The system uses a regularized cost function with a weighted Huber norm over the gradient of the inverse depth map and a convex regularizer. The solution is optimized using a primal-dual approach, enabling efficient computation on GPU hardware. DTAM's dense tracking method uses a dense model to estimate camera pose by finding the parameters of motion that generate a synthetic view matching the live video image. The system is robust to unmodelled objects and can track densely while ignoring pixels with high photometric error. DTAM has been evaluated in a desktop setting with a Point Grey Flea2 camera and shows superior tracking performance compared to PTAM, especially in high acceleration scenarios. DTAM represents a significant advance in real-time geometrical vision with potential applications in augmented reality, robotics, and other fields.DTAM is a real-time system for camera tracking and reconstruction that uses dense, pixel-wise methods instead of feature extraction. As a hand-held RGB camera moves over a static scene, DTAM generates detailed textured depth maps at keyframes to create a surface with millions of vertices. It uses hundreds of images from a video stream to improve a photometric data term and minimizes a global spatially regularized energy function in a non-convex optimization framework. The system tracks the camera's 6DOF motion using whole image alignment against the dense model, achieving real-time performance with current GPU hardware. DTAM outperforms state-of-the-art feature-based methods in tracking under rapid motion and enables real-time scene interaction in augmented reality. The system creates a dense 3D model and uses it for dense camera tracking via whole image registration. It is highly parallelizable and uses a novel non-convex optimization framework with accelerated exact exhaustive search and an interleaved Newton step for fine accuracy. The dense model provides robustness to occlusions and motion blur, making it more accurate than feature-based methods. The system is self-supporting once initialized and uses a dense scene model for simplifying issues with point-based systems. DTAM's dense mapping method uses a global energy minimization framework to estimate inverse depth maps from multiple images, combining photometric error and spatial regularization. The system uses a regularized cost function with a weighted Huber norm over the gradient of the inverse depth map and a convex regularizer. The solution is optimized using a primal-dual approach, enabling efficient computation on GPU hardware. DTAM's dense tracking method uses a dense model to estimate camera pose by finding the parameters of motion that generate a synthetic view matching the live video image. The system is robust to unmodelled objects and can track densely while ignoring pixels with high photometric error. DTAM has been evaluated in a desktop setting with a Point Grey Flea2 camera and shows superior tracking performance compared to PTAM, especially in high acceleration scenarios. DTAM represents a significant advance in real-time geometrical vision with potential applications in augmented reality, robotics, and other fields.