Recipes for Building an Open-Domain Chatbot

Recipes for Building an Open-Domain Chatbot

April 19 - 23, 2021 | Stephen Roller, Emily Dinan, Naman Goyal, Da Ju, Mary Williamson, Yinhan Liu*, Jing Xu, Myle Ott, Kurt Shuster, Eric M. Smith, Y-Lan Boureau, Jason Weston
This paper presents a comprehensive approach to building open-domain chatbots, focusing on blending various skills such as engagingness, knowledge, empathy, and personality. The authors highlight the importance of fine-tuning on data that emphasizes these conversational skills and the impact of decoding strategies on model performance. They demonstrate that large-scale models can learn these skills effectively when provided with appropriate training data and generation strategies. The study involves three types of architectures: retrieval, generative, and retrieve-and-refine models, all based on Transformers. The models are fine-tuned on datasets focusing on specific conversational traits, and their performance is evaluated using human evaluations and automatic metrics. The best models outperform existing approaches in multi-turn dialogue on engagingness and humanness measurements. However, the authors also discuss limitations, such as the tendency to repeat phrases and lack of in-depth knowledge, and propose potential solutions like unlikelihood training and retrieve-and-refine mechanisms. The paper concludes by emphasizing the importance of releasing models to enable full insight into their capabilities and reproducibility.This paper presents a comprehensive approach to building open-domain chatbots, focusing on blending various skills such as engagingness, knowledge, empathy, and personality. The authors highlight the importance of fine-tuning on data that emphasizes these conversational skills and the impact of decoding strategies on model performance. They demonstrate that large-scale models can learn these skills effectively when provided with appropriate training data and generation strategies. The study involves three types of architectures: retrieval, generative, and retrieve-and-refine models, all based on Transformers. The models are fine-tuned on datasets focusing on specific conversational traits, and their performance is evaluated using human evaluations and automatic metrics. The best models outperform existing approaches in multi-turn dialogue on engagingness and humanness measurements. However, the authors also discuss limitations, such as the tendency to repeat phrases and lack of in-depth knowledge, and propose potential solutions like unlikelihood training and retrieve-and-refine mechanisms. The paper concludes by emphasizing the importance of releasing models to enable full insight into their capabilities and reproducibility.
Reach us at info@study.space