Corrective Retrieval Augmented Generation

Corrective Retrieval Augmented Generation

16 Feb 2024 | Shi-Qi Yan1*, Jia-Chen Gu2*, Yun Zhu3, Zhen-Hua Ling1
The paper introduces Corrective Retrieval Augmented Generation (CRAG), a method designed to enhance the robustness of retrieval-augmented generation (RAG) by improving the quality and relevance of retrieved documents. CRAG addresses the issue of hallucinations in large language models (LLMs) by incorporating a lightweight retrieval evaluator that assesses the quality of retrieved documents and triggers different actions based on the confidence score. If the retrieved documents are deemed incorrect or ambiguous, CRAG uses web searches to gather complementary knowledge, while if they are correct, a decompose-then-recompose algorithm refines the documents to extract key information. Experiments on four datasets (PopQA, Biography, PubHealth, and Arc-Challenge) demonstrate that CRAG significantly improves the performance of RAG-based approaches, showing adaptability and generalizability across short- and long-form generation tasks. The method is plug-and-play, allowing seamless integration with various RAG-based models, and its effectiveness is validated through ablation studies and comparisons with state-of-the-art methods.The paper introduces Corrective Retrieval Augmented Generation (CRAG), a method designed to enhance the robustness of retrieval-augmented generation (RAG) by improving the quality and relevance of retrieved documents. CRAG addresses the issue of hallucinations in large language models (LLMs) by incorporating a lightweight retrieval evaluator that assesses the quality of retrieved documents and triggers different actions based on the confidence score. If the retrieved documents are deemed incorrect or ambiguous, CRAG uses web searches to gather complementary knowledge, while if they are correct, a decompose-then-recompose algorithm refines the documents to extract key information. Experiments on four datasets (PopQA, Biography, PubHealth, and Arc-Challenge) demonstrate that CRAG significantly improves the performance of RAG-based approaches, showing adaptability and generalizability across short- and long-form generation tasks. The method is plug-and-play, allowing seamless integration with various RAG-based models, and its effectiveness is validated through ablation studies and comparisons with state-of-the-art methods.
Reach us at info@study.space
[slides and audio] Corrective Retrieval Augmented Generation