MAY 2024 | Guangming Wang, Lei Pan, Songyou Peng, Shaohui Liu, Chenfeng Xu, Yanzi Miao, Wei Zhan, Masayoshi Tomizuka, Life Fellow, IEEE, Marc Pollefeys, Fellow, IEEE, and Hesheng Wang, Senior Member, IEEE
This survey provides a comprehensive overview of the application of Neural Radiance Fields (NeRF) in robotics. NeRF is a neural implicit representation that enables the creation of continuous 3D scene representations from 2D images. It has shown great potential in robotics, particularly in scene perception, interaction, and navigation. The survey discusses the advantages and limitations of NeRF, as well as its current applications and future potential in robotics. It is divided into two main sections: the application of NeRF in robotics and the advancement of NeRF in robotics. The first section introduces and analyzes works that have been or could be used in robotics from the perception and interaction perspectives. The second section shows works related to improving NeRF's own properties, which are essential for deploying NeRF in robotics. The survey also discusses existing challenges and provides future research directions. The survey is structured into sections covering background knowledge, application in robotics, and challenges and future directions. It highlights the use of NeRF in scene reconstruction, segmentation, editing, navigation, and manipulation. The survey also discusses the integration of NeRF with other techniques such as reinforcement learning and tactile perception. Overall, the survey provides a detailed understanding of NeRF in robotics and its potential for future research and applications.This survey provides a comprehensive overview of the application of Neural Radiance Fields (NeRF) in robotics. NeRF is a neural implicit representation that enables the creation of continuous 3D scene representations from 2D images. It has shown great potential in robotics, particularly in scene perception, interaction, and navigation. The survey discusses the advantages and limitations of NeRF, as well as its current applications and future potential in robotics. It is divided into two main sections: the application of NeRF in robotics and the advancement of NeRF in robotics. The first section introduces and analyzes works that have been or could be used in robotics from the perception and interaction perspectives. The second section shows works related to improving NeRF's own properties, which are essential for deploying NeRF in robotics. The survey also discusses existing challenges and provides future research directions. The survey is structured into sections covering background knowledge, application in robotics, and challenges and future directions. It highlights the use of NeRF in scene reconstruction, segmentation, editing, navigation, and manipulation. The survey also discusses the integration of NeRF with other techniques such as reinforcement learning and tactile perception. Overall, the survey provides a detailed understanding of NeRF in robotics and its potential for future research and applications.