PProgramAR: Augmented Reality End-User Robot Programming

PProgramAR: Augmented Reality End-User Robot Programming

March 2024 | BRYCE IKEDA and DANIEL SZAFIR
PRogramAR is a novel end-user robot programming system that combines augmented reality (AR) with trigger-action programming (TAP) to enable non-expert users to program reactive robot behaviors. The system provides intuitive visual feedback through AR, allowing users to author task rules and reactive behaviors while observing program feedback in the real world. Users can program the robot within the entire 3D workspace, verify their programs by visualizing a simulated version of the robot's behavior, and benefit from a cohesive system that reduces context switching. PRogramAR also allows users to freely position the AR interface without physically holding it, enabling them to monitor both the physical workspace and TAP rules simultaneously. PRogramAR supports two types of TAP rules: If-Then and While-Do. These rules allow users to define triggers and actions that dictate when and how a robot should perform a task. The system includes a Rule Manager and Evaluator that continuously check the conditions of rules and provide feedback to users. This feedback helps users debug their programs by indicating whether a rule should or should not be executed. The system also includes a Motion Planner and Robot Digital Twin, which enable users to simulate and execute robot programs. The Digital Twin provides a simulated version of the robot's behavior, allowing users to preview the robot's actions before execution. In a system validation study, 17 end users (ages 18 to 83) used PRogramAR to program a robot to assist with three collaborative tasks. The results demonstrated that merging AR and TAP can enhance the robot programming process for non-expert users. Participants found the system user-friendly, with color-coded TAP rules helping them understand and debug their programs. The system also supported different levels of expression, allowing users to apply their own strategies for programming. Participants appreciated the ability to visualize the entire workspace and the simulation component, which helped them build confidence in their programs. However, some participants struggled with interpreting error messages, highlighting the need for improved error feedback in future research. Overall, PRogramAR provides a promising solution for end-user robot programming, combining the benefits of AR and TAP to enable non-expert users to program reactive robot behaviors. The system has been validated through a study with 17 participants, demonstrating its effectiveness in enhancing the robot programming process for non-expert users. The system is open-source and can be generalized to various robots and AR headsets, fostering reproducibility and encouraging future research and extensions.PRogramAR is a novel end-user robot programming system that combines augmented reality (AR) with trigger-action programming (TAP) to enable non-expert users to program reactive robot behaviors. The system provides intuitive visual feedback through AR, allowing users to author task rules and reactive behaviors while observing program feedback in the real world. Users can program the robot within the entire 3D workspace, verify their programs by visualizing a simulated version of the robot's behavior, and benefit from a cohesive system that reduces context switching. PRogramAR also allows users to freely position the AR interface without physically holding it, enabling them to monitor both the physical workspace and TAP rules simultaneously. PRogramAR supports two types of TAP rules: If-Then and While-Do. These rules allow users to define triggers and actions that dictate when and how a robot should perform a task. The system includes a Rule Manager and Evaluator that continuously check the conditions of rules and provide feedback to users. This feedback helps users debug their programs by indicating whether a rule should or should not be executed. The system also includes a Motion Planner and Robot Digital Twin, which enable users to simulate and execute robot programs. The Digital Twin provides a simulated version of the robot's behavior, allowing users to preview the robot's actions before execution. In a system validation study, 17 end users (ages 18 to 83) used PRogramAR to program a robot to assist with three collaborative tasks. The results demonstrated that merging AR and TAP can enhance the robot programming process for non-expert users. Participants found the system user-friendly, with color-coded TAP rules helping them understand and debug their programs. The system also supported different levels of expression, allowing users to apply their own strategies for programming. Participants appreciated the ability to visualize the entire workspace and the simulation component, which helped them build confidence in their programs. However, some participants struggled with interpreting error messages, highlighting the need for improved error feedback in future research. Overall, PRogramAR provides a promising solution for end-user robot programming, combining the benefits of AR and TAP to enable non-expert users to program reactive robot behaviors. The system has been validated through a study with 17 participants, demonstrating its effectiveness in enhancing the robot programming process for non-expert users. The system is open-source and can be generalized to various robots and AR headsets, fostering reproducibility and encouraging future research and extensions.
Reach us at info@study.space