2024 | JIAJIA WANG, JIMMY X. HUANG, XINHUI TU, JUNMEI WANG, ANGELA J. HUANG, MD TAHMID RAHMAN LASKAR, AMRAN BHUIYAN
This paper presents a survey on the application of BERT in information retrieval (IR), including its applications, resources, and challenges. BERT, a bidirectional encoder representation from transformers, has demonstrated superior performance in various NLP tasks due to its ability to capture contextual relationships. The paper categorizes BERT-based IR methods into six high-level categories: handling long documents, integrating semantic information, balancing effectiveness and efficiency, predicting term weights, query expansion, and document expansion. It also provides links to resources such as datasets and toolkits for BERT-based IR systems. A key highlight is the comparison between BERT's encoder-based models and the latest generative large language models (LLMs), such as ChatGPT, which rely on decoders. Despite the popularity of LLMs, the paper finds that for specific tasks, finely tuned BERT encoders still outperform and are less costly to deploy. The paper also summarizes the comprehensive outcomes of the survey and suggests directions for future research in the area. The survey discusses the background of BERT, traditional IR models, neural ranking models, and improvements to pretrained language models. It then presents BERT-based approaches for ad-hoc IR, including handling long documents, integrating semantic information, balancing effectiveness and efficiency, predicting term weights, query expansion, and document expansion. The paper also provides resources for BERT-based IR systems and concludes with a discussion of the challenges and future directions of BERT-based IR.This paper presents a survey on the application of BERT in information retrieval (IR), including its applications, resources, and challenges. BERT, a bidirectional encoder representation from transformers, has demonstrated superior performance in various NLP tasks due to its ability to capture contextual relationships. The paper categorizes BERT-based IR methods into six high-level categories: handling long documents, integrating semantic information, balancing effectiveness and efficiency, predicting term weights, query expansion, and document expansion. It also provides links to resources such as datasets and toolkits for BERT-based IR systems. A key highlight is the comparison between BERT's encoder-based models and the latest generative large language models (LLMs), such as ChatGPT, which rely on decoders. Despite the popularity of LLMs, the paper finds that for specific tasks, finely tuned BERT encoders still outperform and are less costly to deploy. The paper also summarizes the comprehensive outcomes of the survey and suggests directions for future research in the area. The survey discusses the background of BERT, traditional IR models, neural ranking models, and improvements to pretrained language models. It then presents BERT-based approaches for ad-hoc IR, including handling long documents, integrating semantic information, balancing effectiveness and efficiency, predicting term weights, query expansion, and document expansion. The paper also provides resources for BERT-based IR systems and concludes with a discussion of the challenges and future directions of BERT-based IR.