Citation-Enhanced Generation for LLM-based Chatbots

Citation-Enhanced Generation for LLM-based Chatbots

4 Mar 2024 | Weitao Li, Junkai Li, Weizhi Ma, Yang Liu
This paper introduces a novel post-hoc Citation-Enhanced Generation (CEG) approach to address the hallucination problem in Large Language Models (LLMs) used in chatbots. The CEG framework combines retrieval augmentation and natural language inference (NLI) technologies to verify and regenerate responses. Unlike previous methods that focus on preventing hallucinations during generation, CEG operates post-hoc, using a retrieval module to search for supporting documents and an NLI module to generate citations. If statements in the generated content lack references, the model regenerates responses until all statements are supported by citations. The method is a training-free, plug-and-play plugin that can be applied to various LLMs. Experiments on multiple hallucination-related datasets show that the CEG framework outperforms state-of-the-art methods in both hallucination detection and response regeneration. The paper also discusses the effectiveness of each module and the limitations of the approach, such as the restricted retriever and corpus.This paper introduces a novel post-hoc Citation-Enhanced Generation (CEG) approach to address the hallucination problem in Large Language Models (LLMs) used in chatbots. The CEG framework combines retrieval augmentation and natural language inference (NLI) technologies to verify and regenerate responses. Unlike previous methods that focus on preventing hallucinations during generation, CEG operates post-hoc, using a retrieval module to search for supporting documents and an NLI module to generate citations. If statements in the generated content lack references, the model regenerates responses until all statements are supported by citations. The method is a training-free, plug-and-play plugin that can be applied to various LLMs. Experiments on multiple hallucination-related datasets show that the CEG framework outperforms state-of-the-art methods in both hallucination detection and response regeneration. The paper also discusses the effectiveness of each module and the limitations of the approach, such as the restricted retriever and corpus.
Reach us at info@study.space
[slides and audio] Citation-Enhanced Generation for LLM-based Chatbots