GRUtopia: Dream General Robots in a City at Scale

GRUtopia: Dream General Robots in a City at Scale

15 Jul 2024 | Hanhqin Wang, Jiahe Chen, Wensi Huang, Qingwei Ben, Tai Wang, Boyu Mi, Tao Huang, Siheng Zhao, Yilun Chen, Sizhe Yang, Peizhou Cao, Wenye Yu, Zichao Ye, Jialun Li, Jiafeng Long, Zirui Wang, Huiling Wang, Ying Zhao, Zhongying Tu, Yu Qiao, Dahua Lin, Jiangmiao Pang
GRUtopia is a novel platform designed to stimulate and benchmark advanced embodied AI research. It features a city-scale 3D environment with diverse functional scenarios, learning-based control APIs for various robots, and multi-modal NPCs capable of social interaction, task generation, and assignment. The key contributions of GRUtopia include: 1. **GRScenes**: A large-scale scene dataset with 100k interactive, finely annotated scenes covering 89 diverse categories, enabling the construction of city-scale environments. 2. **GRResidents**: An NPC system driven by Large Language Models (LLMs) for social interaction, task generation, and assignment, enhancing the realism and complexity of the simulated environment. 3. **GRBench**: A benchmark that supports various robots, primarily legged robots, with tasks such as Object Loco-Navigation, Social Loco-Navigation, and Loco-Manipulation, providing a comprehensive evaluation of embodied AI capabilities. The platform aims to address the challenges of data collection and generalization in embodied AI by leveraging simulation, thereby facilitating the development of more robust and versatile robots. Extensive experiments demonstrate the effectiveness of GRUtopia in evaluating and improving the performance of embodied agents.GRUtopia is a novel platform designed to stimulate and benchmark advanced embodied AI research. It features a city-scale 3D environment with diverse functional scenarios, learning-based control APIs for various robots, and multi-modal NPCs capable of social interaction, task generation, and assignment. The key contributions of GRUtopia include: 1. **GRScenes**: A large-scale scene dataset with 100k interactive, finely annotated scenes covering 89 diverse categories, enabling the construction of city-scale environments. 2. **GRResidents**: An NPC system driven by Large Language Models (LLMs) for social interaction, task generation, and assignment, enhancing the realism and complexity of the simulated environment. 3. **GRBench**: A benchmark that supports various robots, primarily legged robots, with tasks such as Object Loco-Navigation, Social Loco-Navigation, and Loco-Manipulation, providing a comprehensive evaluation of embodied AI capabilities. The platform aims to address the challenges of data collection and generalization in embodied AI by leveraging simulation, thereby facilitating the development of more robust and versatile robots. Extensive experiments demonstrate the effectiveness of GRUtopia in evaluating and improving the performance of embodied agents.
Reach us at info@study.space