22 Mar 2024 | Zhiqiang Yan, Yuankai Lin, Kun Wang, Yupeng Zheng, Yufei Wang, Zhenyu Zhang, Jun Li, and Jian Yang
This paper proposes a novel framework called Tri-Perspective View Decomposition (TPVD) for geometry-aware depth completion. TPVD decomposes the original 3D point cloud into three 2D views (top, front, and side) to densify sparse depth measurements while preserving 3D geometry. The framework includes TPV Fusion, which uses recurrent 2D-3D-2D aggregation with a Distance-Aware Spherical Convolution (DASC) to learn 3D geometric priors. Additionally, a Geometric Spatial Propagation Network (GSPN) is introduced to further improve geometric consistency. TPVD outperforms existing methods on KITTI, NYUv2, and SUN RGBD datasets. A new depth completion dataset named TOFDC is also introduced, collected using a smartphone with a TOF sensor and color camera. The proposed method achieves state-of-the-art performance on four benchmarks, including the newly collected TOFDC dataset.This paper proposes a novel framework called Tri-Perspective View Decomposition (TPVD) for geometry-aware depth completion. TPVD decomposes the original 3D point cloud into three 2D views (top, front, and side) to densify sparse depth measurements while preserving 3D geometry. The framework includes TPV Fusion, which uses recurrent 2D-3D-2D aggregation with a Distance-Aware Spherical Convolution (DASC) to learn 3D geometric priors. Additionally, a Geometric Spatial Propagation Network (GSPN) is introduced to further improve geometric consistency. TPVD outperforms existing methods on KITTI, NYUv2, and SUN RGBD datasets. A new depth completion dataset named TOFDC is also introduced, collected using a smartphone with a TOF sensor and color camera. The proposed method achieves state-of-the-art performance on four benchmarks, including the newly collected TOFDC dataset.