Enhancing Complex Question Answering over Knowledge Graphs through Evidence Pattern Retrieval

Enhancing Complex Question Answering over Knowledge Graphs through Evidence Pattern Retrieval

May 13-17, 2024 | Wentao Ding, Jinmao Li, Liangchuan Luo, Yuzhong Qu
This paper proposes Evidence Pattern Retrieval (EPR) to enhance complex question answering over knowledge graphs (KGQA). Current subgraph extraction methods in KGQA often overlook the importance of structural dependencies among evidence facts. EPR explicitly models these dependencies during subgraph extraction by indexing atomic adjacency patterns of resource pairs. Given a question, EPR performs dense retrieval to obtain atomic patterns formed by resource pairs, then enumerates their combinations to construct candidate evidence patterns. These patterns are scored using a neural model, and the best one is selected to extract a subgraph for downstream answer reasoning. Experimental results show that EPR significantly improves the F1 scores of IR-KGQA methods on ComplexWebQuestions by over 10 points and achieves competitive performance on WebQuestionsSP. The paper introduces the concept of evidence patterns, which represent how necessary resources (entities and relations) are connected to support a knowledge graph node as an answer to a question. EPR is implemented by building a vector index for fast retrieval of atomic patterns and proposing an algorithm to construct evidence patterns. The system is evaluated on two widely used benchmarks, ComplexWebQuestions (CWQ) and WebQuestionsSP (WebQSP). Results show that EPR achieves new state-of-the-art performance on CWQ, with a +4.9 increase in H@1 and a +13.2 increase in F1 score compared to previous methods. On WebQSP, EPR shows competitive performance compared to the state-of-the-art method UniKGQA, with a slight decrease in H@1 but similar F1 scores. EPR also outperforms other NSM-based methods. The paper also discusses the impact of the number of retrieved atomic patterns on performance and efficiency. Results show that the number of APs significantly affects performance on complex questions but has less impact on simpler questions. Additionally, the paper analyzes error sources, finding that EPR effectively reduces the impact of noisy extraction. However, there are still challenges, such as handling numerical information and unseen relations, which require further exploration. The paper concludes that EPR improves subgraph extraction by reducing noisy facts and enhances the ability of IR-KGQA methods to handle complex questions.This paper proposes Evidence Pattern Retrieval (EPR) to enhance complex question answering over knowledge graphs (KGQA). Current subgraph extraction methods in KGQA often overlook the importance of structural dependencies among evidence facts. EPR explicitly models these dependencies during subgraph extraction by indexing atomic adjacency patterns of resource pairs. Given a question, EPR performs dense retrieval to obtain atomic patterns formed by resource pairs, then enumerates their combinations to construct candidate evidence patterns. These patterns are scored using a neural model, and the best one is selected to extract a subgraph for downstream answer reasoning. Experimental results show that EPR significantly improves the F1 scores of IR-KGQA methods on ComplexWebQuestions by over 10 points and achieves competitive performance on WebQuestionsSP. The paper introduces the concept of evidence patterns, which represent how necessary resources (entities and relations) are connected to support a knowledge graph node as an answer to a question. EPR is implemented by building a vector index for fast retrieval of atomic patterns and proposing an algorithm to construct evidence patterns. The system is evaluated on two widely used benchmarks, ComplexWebQuestions (CWQ) and WebQuestionsSP (WebQSP). Results show that EPR achieves new state-of-the-art performance on CWQ, with a +4.9 increase in H@1 and a +13.2 increase in F1 score compared to previous methods. On WebQSP, EPR shows competitive performance compared to the state-of-the-art method UniKGQA, with a slight decrease in H@1 but similar F1 scores. EPR also outperforms other NSM-based methods. The paper also discusses the impact of the number of retrieved atomic patterns on performance and efficiency. Results show that the number of APs significantly affects performance on complex questions but has less impact on simpler questions. Additionally, the paper analyzes error sources, finding that EPR effectively reduces the impact of noisy extraction. However, there are still challenges, such as handling numerical information and unseen relations, which require further exploration. The paper concludes that EPR improves subgraph extraction by reducing noisy facts and enhances the ability of IR-KGQA methods to handle complex questions.
Reach us at info@study.space
[slides and audio] Enhancing Complex Question Answering over Knowledge Graphs through Evidence Pattern Retrieval