GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment

GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment

17 Mar 2024 | Lance Ying, Kunal Jha, Shivam Aarya, Joshua B. Tenenbaum, Antonio Torralba, Tianmin Shu
This paper introduces a novel cooperative communication framework called Goal-Oriented Mental Alignment (GOMA), which enables an embodied AI assistant to proactively initiate verbal communication with humans to achieve better cooperation. GOMA formulates verbal communication as a planning problem that minimizes the misalignment between the mental states of the agents relevant to their goals. The framework uses a proxy reward to optimize communication, focusing on sharing and requesting information that aligns the joint plans of both agents. The authors evaluate GOMA in two challenging environments: Overcooked, a multiplayer game, and VirtualHome, a household simulator. Experimental results show that GOMA outperforms strong baselines, including a recent LLM-based baseline, in terms of speedup and total plan costs. Additionally, human participants rated GOMA as more helpful and provided more useful information compared to other models. The study highlights the effectiveness of GOMA in improving cooperation and the importance of goal-relevant communication in embodied AI assistance.This paper introduces a novel cooperative communication framework called Goal-Oriented Mental Alignment (GOMA), which enables an embodied AI assistant to proactively initiate verbal communication with humans to achieve better cooperation. GOMA formulates verbal communication as a planning problem that minimizes the misalignment between the mental states of the agents relevant to their goals. The framework uses a proxy reward to optimize communication, focusing on sharing and requesting information that aligns the joint plans of both agents. The authors evaluate GOMA in two challenging environments: Overcooked, a multiplayer game, and VirtualHome, a household simulator. Experimental results show that GOMA outperforms strong baselines, including a recent LLM-based baseline, in terms of speedup and total plan costs. Additionally, human participants rated GOMA as more helpful and provided more useful information compared to other models. The study highlights the effectiveness of GOMA in improving cooperation and the importance of goal-relevant communication in embodied AI assistance.
Reach us at info@study.space
Understanding GOMA%3A Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment