24 Jan 2024 | HONGRU WANG, The Chinese University of Hong Kong, China; WENYU HUANG, The University of Edinburgh, United Kingdom; YANG DENG, National University of Singapore, Singapore; RUI WANG, Harbin Institute of Technology, China; ZEZHONG WANG, The Chinese University of Hong Kong, China; YUFEI WANG, Huawei Noah Ark Lab, China; FEI MI, Huawei Noah Ark Lab, China; JEFF Z. PAN, The University of Edinburgh, United Kingdom; KAM-FAI WONG, MoE Key Laboratory of High Confidence Software Technologies, The Chinese University of Hong Kong, China
**UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems**
This paper addresses the challenge of personalizing dialogue systems by integrating multiple knowledge sources. It proposes a novel framework called UniMS-RAG, which unifies three sub-tasks—Knowledge Source Selection, Knowledge Retrieval, and Response Generation—into a single sequence-to-sequence model. UniMS-RAG uses special tokens, such as acting tokens and evaluation tokens, to guide the model in selecting knowledge sources, retrieving relevant evidence, and generating personalized responses. The framework is trained using a combination of DPR and LLMs prompting, and it includes a self-refinement mechanism to iteratively refine generated responses based on their relevance and consistency with the dialogue context.
The contributions of this work include:
1. Formally defining a multi-source personalized knowledge-grounded dialogue task.
2. Proposing UniMS-RAG, a unified model that handles all sub-tasks in a personalized dialogue system.
3. Investigating different strategies for obtaining soft labels of evaluation tokens during training.
4. Introducing a self-refinement mechanism to refine generated responses using updated evidence.
Experimental results on two personalized datasets (DuLeMon and KBP) show that UniMS-RAG achieves state-of-the-art performance in knowledge source selection and response generation, demonstrating its effectiveness and flexibility in handling multiple sources of knowledge in personalized dialogue systems. Extensive analyses provide insights into the future directions of multi-source retrieval-augmented generation tasks.**UniMS-RAG: A Unified Multi-source Retrieval-Augmented Generation for Personalized Dialogue Systems**
This paper addresses the challenge of personalizing dialogue systems by integrating multiple knowledge sources. It proposes a novel framework called UniMS-RAG, which unifies three sub-tasks—Knowledge Source Selection, Knowledge Retrieval, and Response Generation—into a single sequence-to-sequence model. UniMS-RAG uses special tokens, such as acting tokens and evaluation tokens, to guide the model in selecting knowledge sources, retrieving relevant evidence, and generating personalized responses. The framework is trained using a combination of DPR and LLMs prompting, and it includes a self-refinement mechanism to iteratively refine generated responses based on their relevance and consistency with the dialogue context.
The contributions of this work include:
1. Formally defining a multi-source personalized knowledge-grounded dialogue task.
2. Proposing UniMS-RAG, a unified model that handles all sub-tasks in a personalized dialogue system.
3. Investigating different strategies for obtaining soft labels of evaluation tokens during training.
4. Introducing a self-refinement mechanism to refine generated responses using updated evidence.
Experimental results on two personalized datasets (DuLeMon and KBP) show that UniMS-RAG achieves state-of-the-art performance in knowledge source selection and response generation, demonstrating its effectiveness and flexibility in handling multiple sources of knowledge in personalized dialogue systems. Extensive analyses provide insights into the future directions of multi-source retrieval-augmented generation tasks.