**GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning** **Authors:** Costas Mavromatis **Abstract:** Knowledge Graphs (KGs) represent factual knowledge in the form of triplets, and Question Answering over KGs (KGQA) involves answering natural questions grounded in this information. Large Language Models (LLMs) excel at understanding natural language, while Graph Neural Networks (GNNs) are effective at handling complex graph information. This work introduces GNN-RAG, a novel method that combines the strengths of both GNNs and LLMs in a retrieval-augmented generation (RAG) style. GNN-RAG first uses a GNN to reason over a dense subgraph of the KG to retrieve answer candidates. It then extracts shortest paths in the KG that connect question entities and answer candidates, representing useful reasoning paths. These paths are verbalized and input into an LLM for further reasoning. The framework leverages GNNs for dense subgraph reasoning and LLMs for natural language processing, enhancing KGQA performance. Additionally, a retrieval augmentation (RA) technique is developed to further improve performance. Experimental results show that GNN-RAG achieves state-of-the-art performance on two widely used KGQA benchmarks (WebQSP and CWQ), outperforming or matching GPT-4 with a 7B tuned LLM. GNN-RAG also excels on multi-hop and multi-entity questions, improving performance by 8.9–15.5% points at answer F1. The code and results are available at https://github.com/cmavro/GNN-RAG.**GNN-RAG: Graph Neural Retrieval for Large Language Model Reasoning** **Authors:** Costas Mavromatis **Abstract:** Knowledge Graphs (KGs) represent factual knowledge in the form of triplets, and Question Answering over KGs (KGQA) involves answering natural questions grounded in this information. Large Language Models (LLMs) excel at understanding natural language, while Graph Neural Networks (GNNs) are effective at handling complex graph information. This work introduces GNN-RAG, a novel method that combines the strengths of both GNNs and LLMs in a retrieval-augmented generation (RAG) style. GNN-RAG first uses a GNN to reason over a dense subgraph of the KG to retrieve answer candidates. It then extracts shortest paths in the KG that connect question entities and answer candidates, representing useful reasoning paths. These paths are verbalized and input into an LLM for further reasoning. The framework leverages GNNs for dense subgraph reasoning and LLMs for natural language processing, enhancing KGQA performance. Additionally, a retrieval augmentation (RA) technique is developed to further improve performance. Experimental results show that GNN-RAG achieves state-of-the-art performance on two widely used KGQA benchmarks (WebQSP and CWQ), outperforming or matching GPT-4 with a 7B tuned LLM. GNN-RAG also excels on multi-hop and multi-entity questions, improving performance by 8.9–15.5% points at answer F1. The code and results are available at https://github.com/cmavro/GNN-RAG.