15 Aug 2024 | Vineet Bhat†, Ali Umut Kaypak†, Prashanth Krishnamurthy, Ramesh Karri, Farshad Khorrami
This paper introduces a novel approach to robotic task planning using two separate Large Language Models (LLMs) for high-level planning and low-level control, enhancing task success rates and goal condition recall. The proposed algorithm, *BrainBody-LLM*, is inspired by the human brain-body architecture and employs a closed-loop feedback mechanism to learn from simulator errors and resolve execution issues in complex environments. The authors demonstrate the effectiveness of BrainBody-LLM in the VirtualHome simulation environment, achieving a 29% improvement in task-oriented success rates over competitive baselines with GPT-4. Additionally, the algorithm is evaluated on seven complex tasks using a realistic physics simulator and the Franka Research 3 robotic arm, showing advancements in reasoning capabilities and error resolution. The key contributions of the paper include a novel planning algorithm, improved task success rates, and successful deployment on real-world robotic tasks. The authors also discuss the limitations and future directions, emphasizing the need for further research to address oscillatory behaviors and hallucinations in LLM-generated plans.This paper introduces a novel approach to robotic task planning using two separate Large Language Models (LLMs) for high-level planning and low-level control, enhancing task success rates and goal condition recall. The proposed algorithm, *BrainBody-LLM*, is inspired by the human brain-body architecture and employs a closed-loop feedback mechanism to learn from simulator errors and resolve execution issues in complex environments. The authors demonstrate the effectiveness of BrainBody-LLM in the VirtualHome simulation environment, achieving a 29% improvement in task-oriented success rates over competitive baselines with GPT-4. Additionally, the algorithm is evaluated on seven complex tasks using a realistic physics simulator and the Franka Research 3 robotic arm, showing advancements in reasoning capabilities and error resolution. The key contributions of the paper include a novel planning algorithm, improved task success rates, and successful deployment on real-world robotic tasks. The authors also discuss the limitations and future directions, emphasizing the need for further research to address oscillatory behaviors and hallucinations in LLM-generated plans.