Preference-Conditioned Language-Guided Abstraction

Preference-Conditioned Language-Guided Abstraction

March 11–14, 2024, Boulder, CO, USA | Andi Peng, Andreea Bobu, Belinda Z. Li, Theodore R. Sumers, Ilia Sucholutsky, Nishanth Kumar, Thomas L. Griffiths, Julie A. Shah
The paper "Preference-Conditioned Language-Guided Abstraction" by Andi Peng, Andreea Bobu, Belinda Z. Li, Theodore R. Sumers, Ilia Sucholutsky, Nishanth Kumar, Thomas L. Griffiths, and Julie A. Shah introduces a framework called Preference-Conditioned Language-Guided Abstraction (PLGA) to improve the construction of state abstractions for robots. The authors address the challenge of learning from demonstrations, where the language specification may not fully capture all relevant features for a task, leading to spurious feature correlations. PLGA leverages language models (LMs) to query for hidden preferences directly from human demonstrations, given that changes in behavior indicate differences in preferences. The framework uses LMs in two ways: first, to query for possible hidden preferences based on a task description and observed behavioral changes; second, to construct state abstractions using the most likely preference. The LM can also actively query humans when uncertain about its estimates. The paper demonstrates the effectiveness of PLGA through simulated experiments, a user study, and real-world robotics experiments with a Spot robot performing mobile manipulation tasks. The results show that PLGA can successfully capture human preferences, leading to more effective state abstractions and improved task performance.The paper "Preference-Conditioned Language-Guided Abstraction" by Andi Peng, Andreea Bobu, Belinda Z. Li, Theodore R. Sumers, Ilia Sucholutsky, Nishanth Kumar, Thomas L. Griffiths, and Julie A. Shah introduces a framework called Preference-Conditioned Language-Guided Abstraction (PLGA) to improve the construction of state abstractions for robots. The authors address the challenge of learning from demonstrations, where the language specification may not fully capture all relevant features for a task, leading to spurious feature correlations. PLGA leverages language models (LMs) to query for hidden preferences directly from human demonstrations, given that changes in behavior indicate differences in preferences. The framework uses LMs in two ways: first, to query for possible hidden preferences based on a task description and observed behavioral changes; second, to construct state abstractions using the most likely preference. The LM can also actively query humans when uncertain about its estimates. The paper demonstrates the effectiveness of PLGA through simulated experiments, a user study, and real-world robotics experiments with a Spot robot performing mobile manipulation tasks. The results show that PLGA can successfully capture human preferences, leading to more effective state abstractions and improved task performance.
Reach us at info@study.space
[slides] Preference-Conditioned Language-Guided Abstraction | StudySpace