AI2-THOR: An Interactive 3D Environment for Visual AI

AI2-THOR: An Interactive 3D Environment for Visual AI

26 Aug 2022 | Eric Kolve, Roozbeh Mottaghi, Winson Han, Eli VanderBilt, Luca Weihs, Alvaro Herrasti, Matt Deitke, Kiana Ehsani, Daniel Gordon, Yuke Zhu, Aniruddha Kembhavi, Abhinav Gupta, Ali Farhadi
AI2-THOR is an interactive 3D environment designed for visual AI research, offering near photo-realistic indoor scenes where AI agents can navigate and interact with objects to perform tasks. It supports a wide range of applications including deep reinforcement learning, imitation learning, planning, visual question answering, and object detection. AI2-THOR provides a Python API to interact with the Unity 3D game engine, enabling navigation, object interaction, and physics modeling. It includes multiple scene datasets such as iTHOR, RoboTHOR, ProcTHOR-10K, and ArchitecTHOR, each with different interactive features and purposes. The environment supports various agents with different physical embodiments and actions, including ManipulaTHOR, StretchRE1, LoCoBot, Abstract, and Drone. AI2-THOR also supports multiple image modalities such as RGB, depth, semantic segmentation, and instance segmentation. It includes a large number of interactive objects and provides environment metadata for tasks requiring detailed information about the scene. AI2-THOR has been used in over 150 publications for various tasks including visual navigation, audio-visual navigation, vision-and-language, human-robot interaction, sim2real transfer, and multi-agent interaction. It is a scalable and efficient platform for training AI models in various scenarios, providing a proxy for real-world experiments. AI2-THOR is continuously updated and has been used to advance research in embodied AI.AI2-THOR is an interactive 3D environment designed for visual AI research, offering near photo-realistic indoor scenes where AI agents can navigate and interact with objects to perform tasks. It supports a wide range of applications including deep reinforcement learning, imitation learning, planning, visual question answering, and object detection. AI2-THOR provides a Python API to interact with the Unity 3D game engine, enabling navigation, object interaction, and physics modeling. It includes multiple scene datasets such as iTHOR, RoboTHOR, ProcTHOR-10K, and ArchitecTHOR, each with different interactive features and purposes. The environment supports various agents with different physical embodiments and actions, including ManipulaTHOR, StretchRE1, LoCoBot, Abstract, and Drone. AI2-THOR also supports multiple image modalities such as RGB, depth, semantic segmentation, and instance segmentation. It includes a large number of interactive objects and provides environment metadata for tasks requiring detailed information about the scene. AI2-THOR has been used in over 150 publications for various tasks including visual navigation, audio-visual navigation, vision-and-language, human-robot interaction, sim2real transfer, and multi-agent interaction. It is a scalable and efficient platform for training AI models in various scenarios, providing a proxy for real-world experiments. AI2-THOR is continuously updated and has been used to advance research in embodied AI.
Reach us at info@study.space
[slides] AI2-THOR%3A An Interactive 3D Environment for Visual AI | StudySpace