2024 | Hugo C Temperley, Niall J O'Sullivan, Benjamin M Mac Curtain, Alison Corr, James F Meaney, Michael E Kelly, Ian Brennan
This systematic review evaluates the current applications and future potential of ChatGPT in radiology. ChatGPT, an AI-based chat model, has the potential to enhance decision-making, optimize workflow efficiency, and support interdisciplinary collaboration and teaching in healthcare. A systematic search in PubMed, EMBASE, and Web of Science identified six studies meeting inclusion criteria, with 551 ChatGPT assessment events. ChatGPT was found to output inaccurate data 80% of the time, with 45% of responses to interventional radiology questions being entirely incorrect. However, ChatGPT performed better on lower-order thinking tasks and showed improved accuracy in imaging questions between versions 3.5 and 4.0 (61% vs. 85%). ChatGPT had an average translational ability score of 4.27/5 on the Likert scale for CT and MRI findings.
ChatGPT's performance in generating academic articles was assessed, revealing significant inaccuracies, including fabricated references. In clinical validation, ChatGPT's responses to interventional radiology questions were evaluated by two radiologists, with 45% of answers deemed entirely incorrect. In trainee examinations, ChatGPT answered 69% of questions correctly, performing better on lower-order thinking tasks. GPT-4 outperformed GPT-3.5 in higher-order thinking tasks but showed no significant improvement in lower-order tasks. ChatGPT's references were often inaccurate, with only 36% of references accessible through internet searches.
ChatGPT's ability to translate radiology reports into plain English was assessed, with an average score of 4.27/5. It provided relevant suggestions for 37% of cases. Despite its potential, ChatGPT has limitations, including lack of strong evidence for image processing and potential inaccuracies due to training data biases. Ethical and legal implications, such as data privacy and liability, require careful consideration. ChatGPT should be seen as a complement to human expertise rather than a replacement. The review highlights the need for thorough evaluation and validation before widespread adoption in radiology.This systematic review evaluates the current applications and future potential of ChatGPT in radiology. ChatGPT, an AI-based chat model, has the potential to enhance decision-making, optimize workflow efficiency, and support interdisciplinary collaboration and teaching in healthcare. A systematic search in PubMed, EMBASE, and Web of Science identified six studies meeting inclusion criteria, with 551 ChatGPT assessment events. ChatGPT was found to output inaccurate data 80% of the time, with 45% of responses to interventional radiology questions being entirely incorrect. However, ChatGPT performed better on lower-order thinking tasks and showed improved accuracy in imaging questions between versions 3.5 and 4.0 (61% vs. 85%). ChatGPT had an average translational ability score of 4.27/5 on the Likert scale for CT and MRI findings.
ChatGPT's performance in generating academic articles was assessed, revealing significant inaccuracies, including fabricated references. In clinical validation, ChatGPT's responses to interventional radiology questions were evaluated by two radiologists, with 45% of answers deemed entirely incorrect. In trainee examinations, ChatGPT answered 69% of questions correctly, performing better on lower-order thinking tasks. GPT-4 outperformed GPT-3.5 in higher-order thinking tasks but showed no significant improvement in lower-order tasks. ChatGPT's references were often inaccurate, with only 36% of references accessible through internet searches.
ChatGPT's ability to translate radiology reports into plain English was assessed, with an average score of 4.27/5. It provided relevant suggestions for 37% of cases. Despite its potential, ChatGPT has limitations, including lack of strong evidence for image processing and potential inaccuracies due to training data biases. Ethical and legal implications, such as data privacy and liability, require careful consideration. ChatGPT should be seen as a complement to human expertise rather than a replacement. The review highlights the need for thorough evaluation and validation before widespread adoption in radiology.