Bayesian Preference Elicitation with Language Models

Bayesian Preference Elicitation with Language Models

8 Mar 2024 | Kunal Handa, Yarin Gal, Ellie Pavlick, Noah Goodman, Jacob Andreas, Alex Tamkin, Belinda Z. Li
The paper introduces OPEN (Optimal Preference Elicitation with Natural language), a framework that combines Bayesian Optimal Experimental Design (BOED) and language models (LMs) to optimize the elicitation of user preferences. OPEN aims to address the challenges of quantifying uncertainty, modeling human mental states, and asking informative questions in preference learning. By using BOED to guide the selection of informative questions and LMs to extract features and translate abstract queries into natural language, OPEN can optimize query informativeness while remaining adaptable to real-world domains. The framework is evaluated through user studies, demonstrating superior performance compared to existing LM- and BOED-based methods for preference elicitation. The paper also discusses the importance of feature weightings and provides qualitative analysis of user feedback, highlighting the benefits and limitations of the proposed approach.The paper introduces OPEN (Optimal Preference Elicitation with Natural language), a framework that combines Bayesian Optimal Experimental Design (BOED) and language models (LMs) to optimize the elicitation of user preferences. OPEN aims to address the challenges of quantifying uncertainty, modeling human mental states, and asking informative questions in preference learning. By using BOED to guide the selection of informative questions and LMs to extract features and translate abstract queries into natural language, OPEN can optimize query informativeness while remaining adaptable to real-world domains. The framework is evaluated through user studies, demonstrating superior performance compared to existing LM- and BOED-based methods for preference elicitation. The paper also discusses the importance of feature weightings and provides qualitative analysis of user feedback, highlighting the benefits and limitations of the proposed approach.
Reach us at info@study.space
Understanding Bayesian Preference Elicitation with Language Models