SPIN: Simultaneous Perception, Interaction and Navigation

SPIN: Simultaneous Perception, Interaction and Navigation

13 May 2024 | Shagun Uppal, Ananye Agarwal, Haoyu Xiong, Kenneth Shaw, Deepak Pathak
The paper presents SPIN, a reactive mobile manipulation framework that enables robots to simultaneously perceive, interact, and navigate in cluttered environments. The robot uses an actuated camera with a limited field of view to gather information about its surroundings, and its motion and perception are tightly coupled. The authors train a single model using reinforcement learning (RL) to control the robot's body and arm while predicting where to look at each time step to avoid obstacles. The approach is evaluated in both simulated and real-world environments, demonstrating the robot's ability to navigate and manipulate objects while avoiding dynamic and static obstacles. The method outperforms classical planning-based approaches and baselines that do not use active vision, showing robustness to dynamic obstacles and improved success rates in cluttered scenarios. The paper also discusses the design choices and limitations of the approach, highlighting the importance of active vision and whole-body coordination in mobile manipulation tasks.The paper presents SPIN, a reactive mobile manipulation framework that enables robots to simultaneously perceive, interact, and navigate in cluttered environments. The robot uses an actuated camera with a limited field of view to gather information about its surroundings, and its motion and perception are tightly coupled. The authors train a single model using reinforcement learning (RL) to control the robot's body and arm while predicting where to look at each time step to avoid obstacles. The approach is evaluated in both simulated and real-world environments, demonstrating the robot's ability to navigate and manipulate objects while avoiding dynamic and static obstacles. The method outperforms classical planning-based approaches and baselines that do not use active vision, showing robustness to dynamic obstacles and improved success rates in cluttered scenarios. The paper also discusses the design choices and limitations of the approach, highlighting the importance of active vision and whole-body coordination in mobile manipulation tasks.
Reach us at info@study.space
Understanding SPIN%3A Simultaneous Perception%2C Interaction and Navigation