Large Language Model-Based Responses to Patients' In-Basket Messages

Large Language Model-Based Responses to Patients' In-Basket Messages

July 16, 2024 | William R. Small, MD, MBA; Batia Wiesenfeld, PhD; Beatrix Brandfield-Harvey, BS; Zoe Jonassen, PhD; Soumik Mandal, PhD; Elizabeth R. Stevens, PhD; Vincent J. Major, PhD; Erin Lostraglio, BA; Adam Szerencsy, DO; Simon Jones, PhD; Yindalon Aphinyanaphongs, MD, PhD; Stephen B. Johnson, PhD; Oded Nov, PhD; Devin Mann, MD
This study evaluated primary care physicians' (PCPs) perceptions of generative artificial intelligence (GenAI) draft responses to patient messages compared to responses generated by health care professionals (HCPs). The study involved 16 PCPs reviewing 344 messages, including 175 GenAI drafts and 169 HCP drafts. PCPs rated GenAI responses as having better communication style than HCP responses, though both were rated similarly on information content quality and usability. GenAI responses were found to be more empathetic, with more subjective and positive language, and were longer and more linguistically complex than HCP responses. However, GenAI responses were less readable, potentially causing issues for patients with lower health or English literacy. The study highlights the potential of GenAI to enhance patient-HCP communication but also raises concerns about readability and bias. The findings suggest that while GenAI responses may be perceived as more empathetic and effective in communication, they may not be suitable for all patients. The study also emphasizes the need for further research to optimize GenAI responses and address potential biases and health inequities.This study evaluated primary care physicians' (PCPs) perceptions of generative artificial intelligence (GenAI) draft responses to patient messages compared to responses generated by health care professionals (HCPs). The study involved 16 PCPs reviewing 344 messages, including 175 GenAI drafts and 169 HCP drafts. PCPs rated GenAI responses as having better communication style than HCP responses, though both were rated similarly on information content quality and usability. GenAI responses were found to be more empathetic, with more subjective and positive language, and were longer and more linguistically complex than HCP responses. However, GenAI responses were less readable, potentially causing issues for patients with lower health or English literacy. The study highlights the potential of GenAI to enhance patient-HCP communication but also raises concerns about readability and bias. The findings suggest that while GenAI responses may be perceived as more empathetic and effective in communication, they may not be suitable for all patients. The study also emphasizes the need for further research to optimize GenAI responses and address potential biases and health inequities.
Reach us at info@futurestudyspace.com