2024 | Siru Liu, PhD.*, Allison B. McCoy, PhD, Josh F. Peterson, MD, MPH, Thomas A. Lasko, MD, PhD, Dean F. Sittig, PhD, Scott D. Nelson, PharmD, MS, Jennifer Andrews, MD, Lorraine Patterson, MSN, Cheryl M. Cobb, MD, David Mulherin, PharmD, Colleen T. Morton, MD, Adam Wright, PhD
This study aims to develop and evaluate a data-driven process to generate suggestions for improving alert criteria using explainable artificial intelligence (XAI) techniques. The researchers extracted data on alerts from Vanderbilt University Medical Center between January 1, 2019, and December 31, 2020, and developed machine learning models to predict user responses to alerts. They applied XAI techniques to generate global and local explanations and evaluated the suggestions by comparing them with historical change logs and stakeholder interviews. The LightGBM model achieved the highest Area Under the ROC Curve (AUC) of 0.919. The study identified 96 helpful suggestions, which could eliminate 278,807 firings (9.3%) of alerts. The approach not only improved clinical decision support (CDS) but also revealed workflow and education issues that were not previously identified. The study concludes that XAI techniques can effectively identify and address overlooked or delayed improvements in CDS, enhancing the quality and efficiency of healthcare.This study aims to develop and evaluate a data-driven process to generate suggestions for improving alert criteria using explainable artificial intelligence (XAI) techniques. The researchers extracted data on alerts from Vanderbilt University Medical Center between January 1, 2019, and December 31, 2020, and developed machine learning models to predict user responses to alerts. They applied XAI techniques to generate global and local explanations and evaluated the suggestions by comparing them with historical change logs and stakeholder interviews. The LightGBM model achieved the highest Area Under the ROC Curve (AUC) of 0.919. The study identified 96 helpful suggestions, which could eliminate 278,807 firings (9.3%) of alerts. The approach not only improved clinical decision support (CDS) but also revealed workflow and education issues that were not previously identified. The study concludes that XAI techniques can effectively identify and address overlooked or delayed improvements in CDS, enhancing the quality and efficiency of healthcare.