Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology

Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology

May 11–16, 2024, Honolulu, HI, USA | Nur Yildirim, Hannah Richardson, Maria T. Wetscherek, Junaid Bajwa, Joseph Jacob, Mark A. Pinnock, Stephen Harris, Daniel Coelho de Castro, Shruthi Bannur, Stephanie L. Hyland, Pratik Ghosh, Mercy Ranjit, Kenza Bouzid, Anton Schwaighofer, Fernando Pérez-García, Harshita Sharma, Ozan Oktay, Matthew Lungren, Javier Alvarez-Valle, Aditya Nori, Anja Thieme
Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology This paper explores the application of vision-language models (VLMs) in radiology to identify and design clinically relevant use cases. The authors conducted an iterative, multidisciplinary design process involving radiologists and clinicians to explore VLM interactions. Four VLM use concepts were co-designed: Draft Report Generation, Augmented Report Review, Visual Search and Querying, and Patient Imaging History Highlights. These concepts were studied with 13 radiologists and clinicians, who found them valuable but articulated design considerations. The study highlights the need for human-centered, participatory approaches in AI design for healthcare. The authors discuss implications for integrating VLM capabilities in radiology and healthcare more broadly. The paper also addresses challenges in AI implementation, including uncertainty about clinical value, skepticism due to inconsistent performance, and the need for clinical effectiveness trials. The study emphasizes the importance of understanding clinical workflows and user needs in AI design. The authors propose a reflective account of their design process as a case study of early phase AI innovation with clinical stakeholders. The paper concludes with a discussion of design implications and future research directions for integrating VLM capabilities into radiology and healthcare.Multimodal Healthcare AI: Identifying and Designing Clinically Relevant Vision-Language Applications for Radiology This paper explores the application of vision-language models (VLMs) in radiology to identify and design clinically relevant use cases. The authors conducted an iterative, multidisciplinary design process involving radiologists and clinicians to explore VLM interactions. Four VLM use concepts were co-designed: Draft Report Generation, Augmented Report Review, Visual Search and Querying, and Patient Imaging History Highlights. These concepts were studied with 13 radiologists and clinicians, who found them valuable but articulated design considerations. The study highlights the need for human-centered, participatory approaches in AI design for healthcare. The authors discuss implications for integrating VLM capabilities in radiology and healthcare more broadly. The paper also addresses challenges in AI implementation, including uncertainty about clinical value, skepticism due to inconsistent performance, and the need for clinical effectiveness trials. The study emphasizes the importance of understanding clinical workflows and user needs in AI design. The authors propose a reflective account of their design process as a case study of early phase AI innovation with clinical stakeholders. The paper concludes with a discussion of design implications and future research directions for integrating VLM capabilities into radiology and healthcare.
Reach us at info@study.space