May 13-17, 2024 | Zhen Zhang, Yuhua Zhao, Hang Gao, Mengting Hu
LinkNER is a framework that combines local fine-tuned named entity recognition (NER) models with large language models (LLMs) using uncertainty-based linking to enhance NER performance. The framework, called RDC (Recognition-Detection-Classification), enables the local model to identify uncertain entities and pass them to the LLM for classification. This approach improves the robustness of NER models, especially in handling out-of-vocabulary (OOV) and out-of-domain (OOD) entities. The local model, based on SpanNER, is fine-tuned with four uncertainty estimation methods (confidence, entropy, Monte Carlo Dropout, and evidential learning). During inference, the LLM is used to classify uncertain entities, enhancing the overall performance. LinkNER was tested on standard NER datasets and noisy social media data, showing significant improvements in robustness compared to state-of-the-art models. The framework also analyzes the impact of uncertainty estimation methods, LLMs, and in-context learning on NER tasks, providing specific recommendations for web-related applications. The results demonstrate that LinkNER outperforms existing methods in robustness tests, particularly in handling unseen entities. The study highlights the effectiveness of combining local models with LLMs through uncertainty-based linking to address the limitations of both models.LinkNER is a framework that combines local fine-tuned named entity recognition (NER) models with large language models (LLMs) using uncertainty-based linking to enhance NER performance. The framework, called RDC (Recognition-Detection-Classification), enables the local model to identify uncertain entities and pass them to the LLM for classification. This approach improves the robustness of NER models, especially in handling out-of-vocabulary (OOV) and out-of-domain (OOD) entities. The local model, based on SpanNER, is fine-tuned with four uncertainty estimation methods (confidence, entropy, Monte Carlo Dropout, and evidential learning). During inference, the LLM is used to classify uncertain entities, enhancing the overall performance. LinkNER was tested on standard NER datasets and noisy social media data, showing significant improvements in robustness compared to state-of-the-art models. The framework also analyzes the impact of uncertainty estimation methods, LLMs, and in-context learning on NER tasks, providing specific recommendations for web-related applications. The results demonstrate that LinkNER outperforms existing methods in robustness tests, particularly in handling unseen entities. The study highlights the effectiveness of combining local models with LLMs through uncertainty-based linking to address the limitations of both models.