Large Language Model-Based Responses to Patients' In-Basket Messages

Large Language Model-Based Responses to Patients' In-Basket Messages

2024-07-16 | William R. Small, MD, MBA; Batia Wiesenfeld, PhD; Beatrix Brandfield-Harvey, BS; Zoe Jonassen, PhD; Soumik Mandal, PhD; Elizabeth R. Stevens, PhD; Vincent J. Major, PhD; Erin Lostraglio, BA; Adam Szercensy, DO; Simon Jones, PhD; Yindalon Aphinyanaphongs, MD, PhD; Stephen B. Johnson, PhD; Oded Nov, PhD; Devin Mann, MD
This study evaluates the perceptions of primary care physicians (PCPs) regarding the quality of responses generated by generative artificial intelligence (GenAI) compared to those generated by healthcare professionals (HCPs). The study was conducted at NYU Langone Health, where 16 PCPs reviewed 344 messages (175 GenAI-drafted and 169 HCP-drafted). Both GenAI and HCP responses were rated favorably, with GenAI responses being rated higher for communication style but similar in information content and usability to HCP responses. GenAI responses were perceived as more empathetic, containing more subjective and positive language, and being more personalized. However, they were also found to be longer and more linguistically complex, which may pose challenges for patients with lower health or English literacy. The study suggests that GenAI can help mitigate the burden of in-basket messages, a significant contributor to physician burnout, by improving communication quality and empathy. However, further research is needed to optimize the perceived quality of GenAI responses and address potential biases and readability issues.This study evaluates the perceptions of primary care physicians (PCPs) regarding the quality of responses generated by generative artificial intelligence (GenAI) compared to those generated by healthcare professionals (HCPs). The study was conducted at NYU Langone Health, where 16 PCPs reviewed 344 messages (175 GenAI-drafted and 169 HCP-drafted). Both GenAI and HCP responses were rated favorably, with GenAI responses being rated higher for communication style but similar in information content and usability to HCP responses. GenAI responses were perceived as more empathetic, containing more subjective and positive language, and being more personalized. However, they were also found to be longer and more linguistically complex, which may pose challenges for patients with lower health or English literacy. The study suggests that GenAI can help mitigate the burden of in-basket messages, a significant contributor to physician burnout, by improving communication quality and empathy. However, further research is needed to optimize the perceived quality of GenAI responses and address potential biases and readability issues.
Reach us at info@study.space
[slides] Large Language Model%E2%80%93Based Responses to Patients%E2%80%99 In-Basket Messages | StudySpace