HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models

23 May 2024 | Bernal Jiménez Gutiérrez, Yiheng Shu, Yu Gu, Michihiro Yasunaga, Yu Su
HippoRAG is a novel retrieval framework inspired by the hippocampal indexing theory of human long-term memory, designed to enable deeper and more efficient knowledge integration for large language models (LLMs). The framework synergistically integrates LLMs, knowledge graphs, and the Personalized PageRank (PPR) algorithm to mimic the roles of the neocortex and hippocampus in human memory. HippoRAG outperforms existing retrieval-augmented generation (RAG) methods on multi-hop question answering tasks, achieving up to 20% better performance. It also offers significant cost and speed advantages over iterative retrieval methods like IRCoT, being 10-30 times cheaper and 6-13 times faster. HippoRAG can handle new types of scenarios that are out of reach of existing methods. The framework uses an LLM to process passages into a schemaless knowledge graph (KG) for offline indexing, and then applies PPR to integrate information across passages for retrieval. This single-step retrieval process enables efficient multi-hop reasoning. HippoRAG's performance is further enhanced by incorporating node specificity, which improves retrieval by considering the frequency of noun phrases in passages. The framework is evaluated on multi-hop QA benchmarks, including MuSiQue and 2WikiMultiHopQA, and shows strong results. HippoRAG's ability to perform single-step multi-hop retrieval makes it a powerful solution for long-term memory in LLMs, offering a balance between efficiency and effectiveness. The method is also shown to handle complex tasks like path-finding multi-hop QA, which require identifying multiple paths between entities. HippoRAG's design is inspired by the hippocampal memory indexing theory, which posits that human long-term memory involves interactions between the neocortex and hippocampus. The framework's offline indexing phase involves extracting knowledge graph triples from passages, while the online retrieval phase uses PPR to integrate information across passages. The method's efficiency and effectiveness make it a promising solution for improving LLMs' ability to integrate new knowledge.HippoRAG is a novel retrieval framework inspired by the hippocampal indexing theory of human long-term memory, designed to enable deeper and more efficient knowledge integration for large language models (LLMs). The framework synergistically integrates LLMs, knowledge graphs, and the Personalized PageRank (PPR) algorithm to mimic the roles of the neocortex and hippocampus in human memory. HippoRAG outperforms existing retrieval-augmented generation (RAG) methods on multi-hop question answering tasks, achieving up to 20% better performance. It also offers significant cost and speed advantages over iterative retrieval methods like IRCoT, being 10-30 times cheaper and 6-13 times faster. HippoRAG can handle new types of scenarios that are out of reach of existing methods. The framework uses an LLM to process passages into a schemaless knowledge graph (KG) for offline indexing, and then applies PPR to integrate information across passages for retrieval. This single-step retrieval process enables efficient multi-hop reasoning. HippoRAG's performance is further enhanced by incorporating node specificity, which improves retrieval by considering the frequency of noun phrases in passages. The framework is evaluated on multi-hop QA benchmarks, including MuSiQue and 2WikiMultiHopQA, and shows strong results. HippoRAG's ability to perform single-step multi-hop retrieval makes it a powerful solution for long-term memory in LLMs, offering a balance between efficiency and effectiveness. The method is also shown to handle complex tasks like path-finding multi-hop QA, which require identifying multiple paths between entities. HippoRAG's design is inspired by the hippocampal memory indexing theory, which posits that human long-term memory involves interactions between the neocortex and hippocampus. The framework's offline indexing phase involves extracting knowledge graph triples from passages, while the online retrieval phase uses PPR to integrate information across passages. The method's efficiency and effectiveness make it a promising solution for improving LLMs' ability to integrate new knowledge.
Reach us at info@study.space