NeSLAM: Neural Implicit Mapping and Self-Supervised Feature Tracking With Depth Completion and Denoising

NeSLAM: Neural Implicit Mapping and Self-Supervised Feature Tracking With Depth Completion and Denoising

29 Mar 2024 | Tianchen Deng, Yanbo Wang, Hongle Xie, Hesheng Wang, Senior Member, IEEE, Jingchuan Wang, Danwei Wang, Fellow, IEEE, Weidong Chen, Member, IEEE
NeSLAM is a novel dense RGB-D SLAM system that combines neural implicit scene representation with robust camera tracking and novel view synthesis. The system addresses the challenges of sparse and noisy depth images from consumer-grade RGB-D sensors and the limitations of existing tracking methods in real-world indoor environments. To improve depth estimation, a depth completion and denoising network is designed to generate dense and accurate depth images with depth uncertainty, which guides neural point sampling and enhances geometric consistency. The system replaces the occupancy value with Signed Distance Field (SDF) values for better scene geometry representation. A NeRF-based self-supervised feature tracking network is also proposed for accurate and robust camera tracking in complex environments. The system is evaluated on various indoor datasets, demonstrating superior performance in scene reconstruction, tracking accuracy, and view synthesis compared to existing methods. NeSLAM achieves accurate and dense depth estimation, robust camera tracking, and realistic novel view synthesis, making it suitable for real-world applications such as autonomous navigation and virtual reality. The system's hierarchical scene representation and self-supervised feature tracking enable efficient and accurate mapping and tracking in complex indoor environments.NeSLAM is a novel dense RGB-D SLAM system that combines neural implicit scene representation with robust camera tracking and novel view synthesis. The system addresses the challenges of sparse and noisy depth images from consumer-grade RGB-D sensors and the limitations of existing tracking methods in real-world indoor environments. To improve depth estimation, a depth completion and denoising network is designed to generate dense and accurate depth images with depth uncertainty, which guides neural point sampling and enhances geometric consistency. The system replaces the occupancy value with Signed Distance Field (SDF) values for better scene geometry representation. A NeRF-based self-supervised feature tracking network is also proposed for accurate and robust camera tracking in complex environments. The system is evaluated on various indoor datasets, demonstrating superior performance in scene reconstruction, tracking accuracy, and view synthesis compared to existing methods. NeSLAM achieves accurate and dense depth estimation, robust camera tracking, and realistic novel view synthesis, making it suitable for real-world applications such as autonomous navigation and virtual reality. The system's hierarchical scene representation and self-supervised feature tracking enable efficient and accurate mapping and tracking in complex indoor environments.
Reach us at info@study.space
[slides and audio] NeSLAM%3A Neural Implicit Mapping and Self-Supervised Feature Tracking With Depth Completion and Denoising