March 11-14, 2024 | Ulas Berk Karli, Jiu-Tung Chen, Victor Nikhil Antony, Chien-Ming Huang
Alchemist is an end-to-end system that leverages large language models (LLMs) to enable natural language-based robot programming for end-users. It allows users to create, test, and run robot programs using a chat interface, with a 3D visualization panel, terminal panel, and chat panel. The system is designed to be platform-agnostic and supports both physical and simulated robots. Alchemist aims to reduce the technical barriers for end-users by enabling intuitive, collaborative programming through natural language interactions. The system includes a function library, LLM initialization prompts, and code safety mechanisms to ensure code quality and reliability. It also features grounded prompting to enhance code safety and execution accuracy. The system was tested with end-users, revealing that while it is effective for novices, there are still challenges in code reliability and user interaction. The study highlights the potential of LLMs in democratizing robot programming but also identifies the need for further improvements in code generation and user training. Alchemist's design emphasizes modularity, allowing for easy adaptation to different robotic platforms and LLMs. The system's success in enabling end-users to program robots with minimal technical knowledge demonstrates the potential of LLMs in advancing end-user development of robot applications.Alchemist is an end-to-end system that leverages large language models (LLMs) to enable natural language-based robot programming for end-users. It allows users to create, test, and run robot programs using a chat interface, with a 3D visualization panel, terminal panel, and chat panel. The system is designed to be platform-agnostic and supports both physical and simulated robots. Alchemist aims to reduce the technical barriers for end-users by enabling intuitive, collaborative programming through natural language interactions. The system includes a function library, LLM initialization prompts, and code safety mechanisms to ensure code quality and reliability. It also features grounded prompting to enhance code safety and execution accuracy. The system was tested with end-users, revealing that while it is effective for novices, there are still challenges in code reliability and user interaction. The study highlights the potential of LLMs in democratizing robot programming but also identifies the need for further improvements in code generation and user training. Alchemist's design emphasizes modularity, allowing for easy adaptation to different robotic platforms and LLMs. The system's success in enabling end-users to program robots with minimal technical knowledge demonstrates the potential of LLMs in advancing end-user development of robot applications.