This paper introduces OPEN (Optimal Preference Elicitation with Natural language), a framework that combines language models (LMs) with Bayesian Optimal Experimental Design (BOED) to efficiently elicit user preferences. The framework uses an LM to identify relevant features and translate abstract BOED queries into natural language questions, while a Bayesian model selects optimal pairwise comparison queries. By integrating the flexibility of LMs with the rigor of BOED, OPEN optimizes the informativity of queries while remaining adaptable to real-world domains. In user studies, OPEN outperforms existing LM- and BOED-based methods for preference elicitation. The paper discusses the challenges of preference learning, including quantifying uncertainty, modeling human mental states, and asking informative questions. These challenges are addressed by BOED, which focuses on designing informative queries within a well-defined feature space. However, BOED methods are difficult to scale and apply to real-world problems where identifying relevant features can be challenging. OPEN addresses these limitations by leveraging LMs to extract features and interface with users, while using BOED to track feature weightings and select informative questions. The paper presents the OPEN framework, which follows several steps: featurization, initializing user preferences, selecting an optimal question, verbalizing the query, updating user preferences, and prediction. The framework uses a Bayesian particle filter to approximate the posterior distribution of user preferences, enabling efficient inference. The framework is tested in a content recommendation domain, where it outperforms both LM- and BOED-based preference elicitation approaches. The paper also discusses related work, including classical preference learning techniques and LM preference learning. It highlights the importance of feature weightings in preference learning and the limitations of LM-only approaches. The paper concludes with a discussion of the framework's advantages, including improved transparency and reduced computational costs, as well as future directions for research.This paper introduces OPEN (Optimal Preference Elicitation with Natural language), a framework that combines language models (LMs) with Bayesian Optimal Experimental Design (BOED) to efficiently elicit user preferences. The framework uses an LM to identify relevant features and translate abstract BOED queries into natural language questions, while a Bayesian model selects optimal pairwise comparison queries. By integrating the flexibility of LMs with the rigor of BOED, OPEN optimizes the informativity of queries while remaining adaptable to real-world domains. In user studies, OPEN outperforms existing LM- and BOED-based methods for preference elicitation. The paper discusses the challenges of preference learning, including quantifying uncertainty, modeling human mental states, and asking informative questions. These challenges are addressed by BOED, which focuses on designing informative queries within a well-defined feature space. However, BOED methods are difficult to scale and apply to real-world problems where identifying relevant features can be challenging. OPEN addresses these limitations by leveraging LMs to extract features and interface with users, while using BOED to track feature weightings and select informative questions. The paper presents the OPEN framework, which follows several steps: featurization, initializing user preferences, selecting an optimal question, verbalizing the query, updating user preferences, and prediction. The framework uses a Bayesian particle filter to approximate the posterior distribution of user preferences, enabling efficient inference. The framework is tested in a content recommendation domain, where it outperforms both LM- and BOED-based preference elicitation approaches. The paper also discusses related work, including classical preference learning techniques and LM preference learning. It highlights the importance of feature weightings in preference learning and the limitations of LM-only approaches. The paper concludes with a discussion of the framework's advantages, including improved transparency and reduced computational costs, as well as future directions for research.