28 Apr 2024 | Xinwei Chen, Kun Li, Jiangjian Guo, Tianyou Song
This paper presents a few-shot named entity recognition (NER) method called RoBERTa+MAML for the StackOverflow NER corpus, which contains 27 entity types. The method leverages meta-learning to improve performance with minimal annotated data. The approach achieves a 5% improvement in F1 score over the baseline model. The study also explores domain-specific phrase processing and knowledge-based pattern extraction to further enhance accuracy.
The paper introduces a few-shot entity typing task, where the goal is to identify entity types in sentences with limited training examples. The method uses prompt-based fine-tuning and meta-learning to adapt to new tasks. The prompt-based approach involves generating templates with unfilled slots, while meta-learning enables the model to quickly adapt to new domains.
The experiments show that RoBERTa+MAML outperforms the baseline RoBERTa model in both micro-F1 and macro-F1 scores. The model performs particularly well in recognizing categories such as data structure, user interface element, operating system, user name, and data types. However, some categories, such as OS, library class, function name, keyboard IP, language, variable name, and algorithm, show lower performance due to ambiguity in the training data.
The study also investigates the impact of domain-specific phrase processing and knowledge-based pattern extraction. These techniques significantly improve the performance of the model, especially for categories like file types and OS. The results show that manually selected training data outperforms randomized data, highlighting the importance of high-quality training examples.
The paper concludes that meta-learning, domain-specific phrase processing, and knowledge-based pattern extraction are effective for few-shot NER in the software domain. Future work aims to expand the dataset and explore additional sample support sets and query sets to further enhance the impact of meta-learning.This paper presents a few-shot named entity recognition (NER) method called RoBERTa+MAML for the StackOverflow NER corpus, which contains 27 entity types. The method leverages meta-learning to improve performance with minimal annotated data. The approach achieves a 5% improvement in F1 score over the baseline model. The study also explores domain-specific phrase processing and knowledge-based pattern extraction to further enhance accuracy.
The paper introduces a few-shot entity typing task, where the goal is to identify entity types in sentences with limited training examples. The method uses prompt-based fine-tuning and meta-learning to adapt to new tasks. The prompt-based approach involves generating templates with unfilled slots, while meta-learning enables the model to quickly adapt to new domains.
The experiments show that RoBERTa+MAML outperforms the baseline RoBERTa model in both micro-F1 and macro-F1 scores. The model performs particularly well in recognizing categories such as data structure, user interface element, operating system, user name, and data types. However, some categories, such as OS, library class, function name, keyboard IP, language, variable name, and algorithm, show lower performance due to ambiguity in the training data.
The study also investigates the impact of domain-specific phrase processing and knowledge-based pattern extraction. These techniques significantly improve the performance of the model, especially for categories like file types and OS. The results show that manually selected training data outperforms randomized data, highlighting the importance of high-quality training examples.
The paper concludes that meta-learning, domain-specific phrase processing, and knowledge-based pattern extraction are effective for few-shot NER in the software domain. Future work aims to expand the dataset and explore additional sample support sets and query sets to further enhance the impact of meta-learning.