{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T03:25:31Z","timestamp":1787887531936,"version":"build-2784847793"},"reference-count":34,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2025,6,5]],"date-time":"2025-06-05T00:00:00Z","timestamp":1749081600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JPM"],"abstract":"<jats:p>Background\/Objectives: This study compares the accuracy of responses from state-of-the-art large language models (LLMs) to patient questions before CT and MRI imaging. We aim to demonstrate the potential of LLMs in improving workflow efficiency, while also highlighting risks such as misinformation. Methods: There were 57 CT-related and 64 MRI-related patient questions displayed to ChatGPT-4o, Claude 3.5 Sonnet, Google Gemini, and Mistral Large 2. Each answer was evaluated by two board-certified radiologists and scored for accuracy\/correctness\/likelihood to mislead using a 5-point Likert scale. Statistics compared LLM performance across question categories. Results: ChatGPT-4o achieved the highest average scores for CT-related questions and tied with Claude 3.5 Sonnet for MRI-related questions, with higher scores across all models for MRI (ChatGPT-4o: CT [4.52 (\u00b1 0.46)], MRI: [4.79 (\u00b1 0.37)]; Google Gemini: CT [4.44 (\u00b1 0.58)]; MRI [4.68 (\u00b1 0.58)]; Claude 3.5 Sonnet: CT [4.40 (\u00b1 0.59)]; MRI [4.79 (\u00b1 0.37)]; Mistral Large 2: CT [4.25 (\u00b1 0.54)]; MRI [4.74 (\u00b1 0.47)]). At least one response per LLM was rated as inaccurate, with Google Gemini answering most often potentially misleading (in 5.26% for CT and 2.34% for MRI). Mistral Large 2 was outperformed by ChatGPT-4o for all CT-related questions (p &lt; 0.001) and by ChatGPT-4o (p = 0.003), Google Gemini (p = 0.022), and Claude 3.5 Sonnet (p = 0.004) for all CT Contrast media information questions. Conclusions: Even though all LLMs performed well overall and showed great potential for patient education, each model occasionally displayed potentially misleading information, highlighting the clinical application risk.<\/jats:p>","DOI":"10.3390\/jpm15060235","type":"journal-article","created":{"date-parts":[[2025,6,5]],"date-time":"2025-06-05T05:49:22Z","timestamp":1749102562000},"page":"235","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Comparison of Multiple State-of-the-Art Large Language Models for Patient Education Prior to CT and MRI Examinations"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-1871-2976","authenticated-orcid":false,"given":"Semil","family":"Eminovic","sequence":"first","affiliation":[{"name":"Department of Radiology, Charit\u00e9\u2013Universit\u00e4tsmedizin Berlin, Corporate Member of Freie Universit\u00e4t Berlin and Humboldt-Universit\u00e4t zu Berlin, 13353 Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bogdan","family":"Levita","sequence":"additional","affiliation":[{"name":"Department of Radiology, Charit\u00e9\u2013Universit\u00e4tsmedizin Berlin, Corporate Member of Freie Universit\u00e4t Berlin and Humboldt-Universit\u00e4t zu Berlin, 13353 Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3964-8360","authenticated-orcid":false,"given":"Andrea","family":"Dell\u2019Orco","sequence":"additional","affiliation":[{"name":"Department of Neuroradiology, Charit\u00e9\u2013Universit\u00e4tsmedizin Berlin, Corporate Member of Freie Universit\u00e4t Berlin and Humboldt-Universit\u00e4t zu Berlin, 13353 Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonas Alexander","family":"Leppig","sequence":"additional","affiliation":[{"name":"Department of Radiology, Immanuel Klinik R\u00fcdersdorf\u2014Universit\u00e4tsklinikum der Medizinischen Hochschule Brandenburg, 15562 R\u00fcdersdorf bei Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1137-0643","authenticated-orcid":false,"given":"Jawed","family":"Nawabi","sequence":"additional","affiliation":[{"name":"Department of Neuroradiology, Charit\u00e9\u2013Universit\u00e4tsmedizin Berlin, Corporate Member of Freie Universit\u00e4t Berlin and Humboldt-Universit\u00e4t zu Berlin, 13353 Berlin, Germany"},{"name":"Berlin Institute of Health, 10178 Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tobias","family":"Penzkofer","sequence":"additional","affiliation":[{"name":"Department of Radiology, Charit\u00e9\u2013Universit\u00e4tsmedizin Berlin, Corporate Member of Freie Universit\u00e4t Berlin and Humboldt-Universit\u00e4t zu Berlin, 13353 Berlin, Germany"},{"name":"Berlin Institute of Health, 10178 Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,6,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1001\/jama.2019.11456","article-title":"Trends in Use of Medical Imaging in US Health Care Systems and in Ontario, Canada, 2000\u20132016","volume":"322","author":"Kwan","year":"2019","journal-title":"JAMA"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1053\/crad.2002.0939","article-title":"What Do Patients Know About Ultrasound, CT and MRI?","volume":"57","author":"Chesson","year":"2002","journal-title":"Clin. Radiol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1016\/j.pec.2009.03.030","article-title":"The information needs of people attending for computed tomography (CT): What are they and how can they be met?","volume":"77","author":"Mathers","year":"2009","journal-title":"Patient Educ. Couns."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"ojae062","DOI":"10.1093\/asjof\/ojae062","article-title":"Preoperative Patient Guidance and Education in Aesthetic Breast Plastic Surgery: A Novel Proposed Application of Artificial Intelligence Large Language Models","volume":"6","author":"Kazan","year":"2024","journal-title":"Aesthetic Surg. J. Open Forum"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1136\/ard-2024-226202","article-title":"Assessing the performance of AI chatbots in answering patients\u2019 common questions about low back pain","volume":"84","author":"Scaff","year":"2024","journal-title":"Ann. Rheum. Dis."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1136\/bjo-2024-325599","article-title":"Evaluating the effectiveness of large language models in patient education for conjunctivitis","volume":"109","author":"Wang","year":"2024","journal-title":"Br. J. Ophthalmol."},{"key":"ref_7","unstructured":"Thieme (2025, April 14). Aufkl\u00e4rungsbogen Computertomografie (CT). Available online: https:\/\/www.thieme-compliance.de\/de\/shop\/Artikel\/Aufkl%C3%A4rungsb%C3%B6gen\/Computertomografie\/p\/AE62125001."},{"key":"ref_8","unstructured":"Thieme (2025, April 14). Aufkl\u00e4rungsbogen Kernspintomografie (MRT\/MRS\/MRA). Available online: https:\/\/www.thieme-compliance.de\/de\/shop\/Artikel\/Aufkl%C3%A4rungsb%C3%B6gen\/Kernspintomografie-%28MRT-MRS-MRA%29\/p\/AE62124201."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"396","DOI":"10.9734\/BJAST\/2015\/14975","article-title":"Likert Scale: Explored and Explained","volume":"7","author":"Joshi","year":"2015","journal-title":"Br. J. Appl. Sci. Technol."},{"key":"ref_10","unstructured":"ESUR ESoUR (2018). ESUR Guidelines on Contrast Media, ESUR."},{"key":"ref_11","unstructured":"ACR ACoR (2023). ACR Manual on Contrast Media, ACR."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1007\/s00405-024-08859-8","article-title":"Accuracy of ChatGPT responses on tracheotomy for patient education","volume":"281","author":"Khaldi","year":"2024","journal-title":"Eur. Arch. Oto-Rhino-Laryngol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s41747-024-00526-1","article-title":"Probing clarity: AI-generated simplified breast imaging reports for enhanced patient comprehension powered by ChatGPT-4o","volume":"8","author":"Maroncelli","year":"2024","journal-title":"Eur. Radiol. Exp."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1007\/s40123-024-01066-y","article-title":"Assessment of Large Language Models in Cataract Care Information Provision: A Quantitative Comparison","volume":"14","author":"Su","year":"2024","journal-title":"Ophthalmol. Ther."},{"key":"ref_15","unstructured":"OpenAI (2024, October 04). Color Health\u2014Color Health uses the reasoning capabilities of GPT-4o to help doctors transform cancer care. Available online: https:\/\/openai.com\/index\/color-health\/."},{"key":"ref_16","unstructured":"OpenAI (2024, October 04). Summer Health\u2014Summer Health reimagines pediatric doctor\u2019s visits with OpenAI. Available online: https:\/\/openai.com\/index\/summer-health\/."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1016\/j.diii.2023.02.003","article-title":"Revolutionizing radiology with GPT-based models: Current applications, future possibilities and limitations of ChatGPT","volume":"104","author":"Lecler","year":"2023","journal-title":"Diagn. Interv. Imaging"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Sallam, M. (2023). ChatGPT utility in healthcare education, research, and practice: Systematic review on the promising perspectives and valid concerns. Healthcare, 11.","DOI":"10.3390\/healthcare11060887"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"e58831","DOI":"10.2196\/58831","article-title":"\u201cDoctor ChatGPT, Can You Help Me?\u201d The Patient\u2019s Perspective: Cross-Sectional Study","volume":"26","author":"Armbruster","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"407","DOI":"10.59249\/NKOY5498","article-title":"Artificial Intelligence to Improve Patient Understanding of Radiology Reports","volume":"96","author":"Amin","year":"2023","journal-title":"Yale J. Biol. Med."},{"key":"ref_21","first-page":"1","article-title":"Patient-centered radiology reports with generative artificial intelligence: Adding value to radiology reporting","volume":"14","author":"Park","year":"2024","journal-title":"Sci. Rep."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"e2330320","DOI":"10.1001\/jamanetworkopen.2023.30320","article-title":"Comparison of Ophthalmologist and AI Chatbot Responses to Online Patient Eye Care Questions","volume":"6","author":"Bernstein","year":"2023","journal-title":"JAMA Netw. Open"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10916-024-02100-z","article-title":"Comparing ChatGPT and a Single Anesthesiologist\u2019s Responses to Common Patient Questions: An Exploratory Cross-Sectional Survey of a Panel of Anesthesiologists","volume":"48","author":"Kuo","year":"2024","journal-title":"J. Med. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"715","DOI":"10.1093\/ced\/llad456","article-title":"Comparing the quality of ChatGPT- and physician-generated responses to patients\u2019 dermatology questions in the electronic medical record","volume":"49","author":"Reynolds","year":"2024","journal-title":"Clin. Exp. Dermatol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"658","DOI":"10.1177\/00034894241249621","article-title":"Assessing ChatGPT\u2019s Responses to Otolaryngology Patient Questions","volume":"133","author":"Carnino","year":"2024","journal-title":"Ann. Otol. Rhinol. Laryngol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1571","DOI":"10.1007\/s00266-023-03660-0","article-title":"Exploring the Potential of ChatGPT-4 in Responding to Common Questions About Abdominoplasty: An AI-Based Case Study of a Plastic Surgery Consultation","volume":"48","author":"Li","year":"2023","journal-title":"Aesthetic Plast. Surg."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"e53297","DOI":"10.2196\/53297","article-title":"Triage Performance Across Large Language Models, ChatGPT, and Untrained Doctors in Emergency Medicine: Comparative Study","volume":"26","author":"Masanneck","year":"2024","journal-title":"J. Med. Internet Res."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Chen, L., Zaharia, M., and Zou, J. (2023). How is ChatGPT\u2019s behavior changing over time?. arXiv.","DOI":"10.1162\/99608f92.5317da47"},{"key":"ref_29","unstructured":"Manchanda, J., Boettcher, L., Westphalen, M., and Jasser, J. (2024). The Open Source Advantage in Large Language Models (LLMs). arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1038\/d41586-023-01295-4","article-title":"Why open-source generative AI models are an ethical way forward for science","volume":"616","author":"Spirling","year":"2023","journal-title":"Nature"},{"key":"ref_31","first-page":"e43262","article-title":"Unraveling the Ethical Enigma: Artificial Intelligence in Healthcare","volume":"15","author":"Jeyaraman","year":"2023","journal-title":"Cureus"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"e48009","DOI":"10.2196\/48009","article-title":"Ethical Considerations of Using ChatGPT in Health Care","volume":"25","author":"Wang","year":"2023","journal-title":"J. Med. Internet Res."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"e428","DOI":"10.1016\/S2589-7500(24)00061-X","article-title":"Ethical and regulatory challenges of large language models in medicine","volume":"6","author":"Ong","year":"2024","journal-title":"Lancet Digit. Health"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1136\/jme-2023-109347","article-title":"Consent-GPT: Is it ethical to delegate procedural consent to conversational AI?","volume":"50","author":"Allen","year":"2024","journal-title":"J. Med. Ethics"}],"container-title":["Journal of Personalized Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4426\/15\/6\/235\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T17:46:52Z","timestamp":1760032012000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4426\/15\/6\/235"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,5]]},"references-count":34,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2025,6]]}},"alternative-id":["jpm15060235"],"URL":"https:\/\/doi.org\/10.3390\/jpm15060235","relation":{},"ISSN":["2075-4426"],"issn-type":[{"value":"2075-4426","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,5]]}}}