Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study

Leveraging ChatGPT in Pharmacovigilance Event Extraction: An Empirical Study

24 Feb 2024 | Zhaoyue Sun, Gabriele Pergola, Byron C. Wallace, Yulan He
This paper explores the application of large language models (LLMs), specifically ChatGPT, in pharmacovigilance event extraction, which involves identifying and extracting adverse events or potential therapeutic events from medical text. The study assesses ChatGPT's performance through various prompting and demonstration selection strategies, finding that while it performs reasonably well with appropriate strategies, it still lags behind fully fine-tuned small models. The research also investigates the use of ChatGPT for data augmentation, but results show that incorporating synthesized data can lead to a performance decline due to noise in the generated labels. To mitigate this, filtering strategies are introduced, which help improve stability but do not fully restore performance to that of fully fine-tuned models. The study concludes that while ChatGPT shows promise in few-shot learning, fine-tuned models remain superior, and future work should focus on enhancing the quality and diversity of synthetic data to improve performance.This paper explores the application of large language models (LLMs), specifically ChatGPT, in pharmacovigilance event extraction, which involves identifying and extracting adverse events or potential therapeutic events from medical text. The study assesses ChatGPT's performance through various prompting and demonstration selection strategies, finding that while it performs reasonably well with appropriate strategies, it still lags behind fully fine-tuned small models. The research also investigates the use of ChatGPT for data augmentation, but results show that incorporating synthesized data can lead to a performance decline due to noise in the generated labels. To mitigate this, filtering strategies are introduced, which help improve stability but do not fully restore performance to that of fully fine-tuned models. The study concludes that while ChatGPT shows promise in few-shot learning, fine-tuned models remain superior, and future work should focus on enhancing the quality and diversity of synthetic data to improve performance.
Reach us at info@study.space
[slides] Leveraging ChatGPT in Pharmacovigilance Event Extraction%3A An Empirical Study | StudySpace