Retrieval-Augmented Generation for Large Language Models: A Survey

Retrieval-Augmented Generation for Large Language Models: A Survey

27 Mar 2024 | Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang
This survey provides a comprehensive overview of Retrieval-Augmented Generation (RAG) for Large Language Models (LLMs), highlighting its role in enhancing the accuracy and credibility of LLMs by integrating external knowledge. RAG combines the intrinsic knowledge of LLMs with external databases, enabling continuous knowledge updates and domain-specific information integration. The paper reviews three main RAG paradigms: Naive RAG, Advanced RAG, and Modular RAG, detailing their components, challenges, and advancements. It emphasizes the importance of retrieval, generation, and augmentation techniques in RAG systems, and presents current evaluation frameworks and benchmarks. The paper also discusses the evolution of RAG, its integration with LLMs, and future research directions. Key contributions include a systematic review of RAG methods, analysis of central technologies, and comprehensive evaluation of RAG's downstream tasks and performance metrics. The paper identifies challenges in RAG, such as retrieval precision, generation hallucination, and augmentation complexity, and proposes solutions through advanced techniques like pre-retrieval optimization, post-retrieval processing, and modular RAG. It also compares RAG with fine-tuning and prompt engineering, highlighting their respective strengths and limitations. The survey concludes with a detailed discussion of RAG's retrieval, generation, and augmentation processes, emphasizing the importance of efficient retrieval, context curation, and LLM fine-tuning in achieving high-quality results. The paper aims to provide a structured understanding of RAG and its role in advancing LLM applications.This survey provides a comprehensive overview of Retrieval-Augmented Generation (RAG) for Large Language Models (LLMs), highlighting its role in enhancing the accuracy and credibility of LLMs by integrating external knowledge. RAG combines the intrinsic knowledge of LLMs with external databases, enabling continuous knowledge updates and domain-specific information integration. The paper reviews three main RAG paradigms: Naive RAG, Advanced RAG, and Modular RAG, detailing their components, challenges, and advancements. It emphasizes the importance of retrieval, generation, and augmentation techniques in RAG systems, and presents current evaluation frameworks and benchmarks. The paper also discusses the evolution of RAG, its integration with LLMs, and future research directions. Key contributions include a systematic review of RAG methods, analysis of central technologies, and comprehensive evaluation of RAG's downstream tasks and performance metrics. The paper identifies challenges in RAG, such as retrieval precision, generation hallucination, and augmentation complexity, and proposes solutions through advanced techniques like pre-retrieval optimization, post-retrieval processing, and modular RAG. It also compares RAG with fine-tuning and prompt engineering, highlighting their respective strengths and limitations. The survey concludes with a detailed discussion of RAG's retrieval, generation, and augmentation processes, emphasizing the importance of efficient retrieval, context curation, and LLM fine-tuning in achieving high-quality results. The paper aims to provide a structured understanding of RAG and its role in advancing LLM applications.
Reach us at info@study.space
[slides and audio] Retrieval-Augmented Generation for Large Language Models%3A A Survey