NeSLAM: Neural Implicit Mapping and Self-Supervised Feature Tracking With Depth Completion and Denoising

NeSLAM: Neural Implicit Mapping and Self-Supervised Feature Tracking With Depth Completion and Denoising

29 Mar 2024 | Tianchen Deng, Yanbo Wang, Hongle Xie, Hesheng Wang, Senior Member, IEEE, Jingchuan Wang, Danwei Wang, Fellow, IEEE, Weidong Chen, Member, IEEE
NeSLAM is an advanced framework that integrates Neural Radiance Fields (NeRF) with dense RGB-D SLAM systems to achieve accurate and dense depth estimation, robust camera tracking, and realistic synthesis of novel views. The system addresses the challenges posed by sparse and noisy depth images from consumer-grade RGB-D sensors and the limitations of existing methods in real-world indoor environments. Key contributions include: 1. **Depth Completion and Denoising Network**: This network generates dense and precise depth images with depth uncertainty, improving geometry representation and guiding neural point sampling. 2. **Signed Distance Field (SDF) Hierarchical Scene Representation**: Replacing the occupancy value with SDF values enhances scene geometry representation and view synthesis. 3. **NeRF-Based Self-Supervised Feature Tracking**: A novel algorithm for accurate and robust camera tracking in complex indoor scenes, leveraging the strengths of NeRF and feature tracking. Experiments on various indoor datasets demonstrate the effectiveness and accuracy of NeSLAM in reconstruction, tracking quality, and novel view synthesis, outperforming existing methods in terms of accuracy and robustness. The system is scalable, predictive, and robust to complex indoor scenes, offering high-fidelity novel views and accurate 3D meshes.NeSLAM is an advanced framework that integrates Neural Radiance Fields (NeRF) with dense RGB-D SLAM systems to achieve accurate and dense depth estimation, robust camera tracking, and realistic synthesis of novel views. The system addresses the challenges posed by sparse and noisy depth images from consumer-grade RGB-D sensors and the limitations of existing methods in real-world indoor environments. Key contributions include: 1. **Depth Completion and Denoising Network**: This network generates dense and precise depth images with depth uncertainty, improving geometry representation and guiding neural point sampling. 2. **Signed Distance Field (SDF) Hierarchical Scene Representation**: Replacing the occupancy value with SDF values enhances scene geometry representation and view synthesis. 3. **NeRF-Based Self-Supervised Feature Tracking**: A novel algorithm for accurate and robust camera tracking in complex indoor scenes, leveraging the strengths of NeRF and feature tracking. Experiments on various indoor datasets demonstrate the effectiveness and accuracy of NeSLAM in reconstruction, tracking quality, and novel view synthesis, outperforming existing methods in terms of accuracy and robustness. The system is scalable, predictive, and robust to complex indoor scenes, offering high-fidelity novel views and accurate 3D meshes.
Reach us at info@study.space