{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T01:47:07Z","timestamp":1792720027865,"version":"4.5.0"},"reference-count":34,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T00:00:00Z","timestamp":1772496000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003042","name":"Else Kr\u00f6ner-Fresenius-Stiftung","doi-asserted-by":"publisher","award":["2023_EKEA.77"],"award-info":[{"award-number":["2023_EKEA.77"]}],"id":[{"id":"10.13039\/501100003042","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: To evaluate the diagnostic accuracy of two visual large language models (vLLMs), GPT-4o (OpenAI) and Claude Sonnet 3.5 (Anthropic), for detecting brain metastases in routine MRI using combined imaging and textual input. Methods: This retrospective study included 31 patients with and 46 without brain metastases with underlying melanoma (n = 24), lung cancer (n = 23), breast cancer (n = 17), or renal cell carcinoma (n = 13). In total, 100 MRI examinations (50 with, 50 without metastases) were provided to both vLLMs using a single representative slice per sequence, together with clinical history and the referring question. The generated free-text reports were evaluated for detection accuracy, overdiagnosis, correct sequence recognition, anatomical localization, lesion laterality, and lesion size estimation. Results: Both vLLMs showed perfect sensitivity (100% for both) but very low specificity (GPT-4o: 8%, Sonnet 3.5: 4%; p = 0.625), resulting in low diagnostic accuracy (GPT-4o: 54%, Sonnet 3.5: 52%; p = 0.625). Sequence identification was highly accurate in both models, with GPT-4o performing significantly better (100% vs. 93%; p &lt; 0.05). Identification of the anatomical brain region (70% vs. 72%; p = 1.00) and lesion laterality (62% vs. 76%; p = 0.189) was comparable. Both models hallucinated additional lesions in 12% of cases. Lesion size measurements showed no significant differences between the models or in comparison with the radiologist. Conclusions: GPT-4o and Claude Sonnet 3.5 can generate radiological reports and detect brain metastases with excellent sensitivity, but their very low specificity, frequent hallucinations, and limited spatial reliability currently preclude clinical application. Future work should address how the balance between visual and textual input influences diagnostic behavior in vLLMs.<\/jats:p>","DOI":"10.3390\/diagnostics16050749","type":"journal-article","created":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T15:22:21Z","timestamp":1772551341000},"page":"749","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Opportunities and Challenges of Visual Large Language Models in Imaging Diagnostics: Lessons from Brain Metastasis Detection in Clinical MRI"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7351-7516","authenticated-orcid":false,"given":"Christian","family":"Nelles","sequence":"first","affiliation":[{"name":"Institute for Diagnostic and Interventional Radiology, Faculty of Medicine, University Hospital Cologne, University of Cologne, 50937 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nour","family":"Abou Zeid","sequence":"additional","affiliation":[{"name":"Institute for Diagnostic and Interventional Radiology, Faculty of Medicine, University Hospital Cologne, University of Cologne, 50937 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-1068-8477","authenticated-orcid":false,"given":"Robert","family":"Terzis","sequence":"additional","affiliation":[{"name":"Institute for Diagnostic and Interventional Radiology, Faculty of Medicine, University Hospital Cologne, University of Cologne, 50937 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3694-0235","authenticated-orcid":false,"given":"Andra-Iza","family":"Iuga","sequence":"additional","affiliation":[{"name":"Institute for Diagnostic and Interventional Radiology, Faculty of Medicine, University Hospital Cologne, University of Cologne, 50937 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lukas","family":"G\u00f6rtz","sequence":"additional","affiliation":[{"name":"Institute for Diagnostic and Interventional Radiology, Faculty of Medicine, University Hospital Cologne, University of Cologne, 50937 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6129-9267","authenticated-orcid":false,"given":"Marvin A.","family":"Spurek","sequence":"additional","affiliation":[{"name":"Institute for Diagnostic and Interventional Radiology, Faculty of Medicine, University Hospital Cologne, University of Cologne, 50937 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Maintz","sequence":"additional","affiliation":[{"name":"Institute for Diagnostic and Interventional Radiology, Faculty of Medicine, University Hospital Cologne, University of Cologne, 50937 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3254-4809","authenticated-orcid":false,"given":"Simon","family":"Lennartz","sequence":"additional","affiliation":[{"name":"Institute for Diagnostic and Interventional Radiology, Faculty of Medicine, University Hospital Cologne, University of Cologne, 50937 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5021-6895","authenticated-orcid":false,"given":"Jonathan","family":"Kottlors","sequence":"additional","affiliation":[{"name":"Institute for Diagnostic and Interventional Radiology, Faculty of Medicine, University Hospital Cologne, University of Cologne, 50937 Cologne, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,3,3]]},"reference":[{"key":"ref_1","first-page":"941","article-title":"Management of Brain Metastases","volume":"13","author":"Wen","year":"1999","journal-title":"Oncology"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1007\/s11912-011-0203-y","article-title":"Epidemiology of Brain Metastases","volume":"14","author":"Nayak","year":"2012","journal-title":"Curr. Oncol. Rep."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1200\/JCO.2011.38.0527","article-title":"Summary Report on the Graded Prognostic Assessment: An Accurate and Facile Diagnosis-Specific Tool to Estimate Survival for Patients with Brain Metastases","volume":"30","author":"Sperduto","year":"2012","journal-title":"J. Clin. Oncol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1001\/jama.2016.9839","article-title":"Effect of Radiosurgery Alone vs Radiosurgery with Whole Brain Radiation Therapy on Cognitive Function in Patients with 1 to 3 Brain Metastases: A Randomized Clinical Trial","volume":"316","author":"Brown","year":"2016","journal-title":"JAMA"},{"key":"ref_5","first-page":"293","article-title":"Diagnosis of Cerebral Metastases: Double-Dose Delayed CT vs Contrast-Enhanced MR Imaging","volume":"12","author":"Davis","year":"1991","journal-title":"AJNR Am. J. Neuroradiol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1016\/S1042-3680(18)30369-3","article-title":"Imaging of Cerebral Metastases","volume":"7","author":"Schaefer","year":"1996","journal-title":"Neurosurg. Clin. N. Am."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1023\/A:1011132609055","article-title":"High Incidence of Central Nervous System Involvement in Patients with Metastatic or Locally Advanced Breast Cancer Treated with Epirubicin and Docetaxel","volume":"12","author":"Crivellari","year":"2001","journal-title":"Ann. Oncol."},{"key":"ref_8","first-page":"108","article-title":"Changing Patterns of Bone and Brain Metastases in Patients with Colorectal Cancer","volume":"5","author":"Sundermeyer","year":"2005","journal-title":"Clin. Color. Cancer"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1530","DOI":"10.1200\/JCO.2005.04.123","article-title":"High Risk of Brain Metastases in Surgically Staged IIIA Non-Small-Cell Lung Cancer Patients Treated with Surgery, Chemotherapy, and Radiation","volume":"23","author":"Mamon","year":"2005","journal-title":"J. Clin. Oncol."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Rahman, T., Chowdhury, M.E.H., Khandakar, A., Islam, K.R., Islam, K.F., Mahbub, Z.B., Kadir, M.A., and Kashem, S. (2020). Transfer Learning with Deep Convolutional Neural Network (CNN) for Pneumonia Detection Using Chest X-Ray. Appl. Sci., 10.","DOI":"10.3390\/app10093233"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"111798","DOI":"10.1016\/j.ejrad.2024.111798","article-title":"AI in Radiology: From Promise to Practice\u2014A Guide to Effective Integration","volume":"181","author":"Katal","year":"2024","journal-title":"Eur. J. Radiol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1097\/RTI.0000000000000613","article-title":"Diagnostic Accuracy and Performance of Artificial Intelligence in Detecting Lung Nodules in Patients with Complex Lung Disease: A Noninferiority Study","volume":"37","author":"Abadia","year":"2022","journal-title":"J. Thorac. Imaging"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1148\/radiol.2017162326","article-title":"Deep Learning at Chest Radiography: Automated Classification of Pulmonary Tuberculosis by Using Convolutional Neural Networks","volume":"284","author":"Lakhani","year":"2017","journal-title":"Radiology"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e232756","DOI":"10.1148\/radiol.232756","article-title":"Chatbots and Large Language Models in Radiology: A Practical Primer for Clinical and Research Applications","volume":"310","author":"Bhayana","year":"2024","journal-title":"Radiology"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"e231259","DOI":"10.1148\/radiol.231259","article-title":"Evaluating GPT4 on Impressions Generation in Radiology Reports","volume":"307","author":"Sun","year":"2023","journal-title":"Radiology"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"e231167","DOI":"10.1148\/radiol.231167","article-title":"Feasibility of Differential Diagnosis Based on Imaging Patterns Using a Large Language Model","volume":"308","author":"Kottlors","year":"2023","journal-title":"Radiology"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1007\/s10916-024-02105-8","article-title":"Comparison of Vision Transformers and Convolutional Neural Networks in Medical Image Analysis: A Systematic Review","volume":"48","author":"Takahashi","year":"2024","journal-title":"J. Med. Syst."},{"key":"ref_18","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An Image Is Worth 16 \u00d7 16 Words: Transformers for Image Recognition at Scale 2021. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.ajem.2024.07.043","article-title":"The Accuracy of Gemini, GPT-4, and GPT-4o in ECG Analysis: A Comparison with Cardiologists and Emergency Medicine Specialists","volume":"84","year":"2024","journal-title":"Am. J. Emerg. Med."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e240273","DOI":"10.1148\/radiol.240273","article-title":"Comparing Diagnostic Accuracy of Radiologists versus GPT-4V and Gemini Pro Vision Using Image Inputs from Diagnosis Please Cases","volume":"312","author":"Suh","year":"2024","journal-title":"Radiology"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e240153","DOI":"10.1148\/radiol.240153","article-title":"Performance of GPT-4 with Vision on Text- and Image-Based ACR Diagnostic Radiology In-Training Examination Questions","volume":"312","author":"Hayden","year":"2024","journal-title":"Radiology"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1959","DOI":"10.1007\/s00330-024-11035-5","article-title":"Assessing GPT-4 Multimodal Performance in Radiological Image Analysis","volume":"35","author":"Brin","year":"2025","journal-title":"Eur. Radiol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1320","DOI":"10.1001\/jama.2023.27861","article-title":"Comparative Analysis of Multimodal Large Language Model Performance on Clinical Vignette Questions","volume":"331","author":"Han","year":"2024","journal-title":"JAMA"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1111","DOI":"10.1007\/s00330-024-11115-6","article-title":"Revolution or Risk?\u2014Assessing the Potential and Challenges of GPT-4V in Radiologic Image Interpretation","volume":"35","author":"Huppertz","year":"2025","journal-title":"Eur. Radiol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4096","DOI":"10.1097\/JS9.0000000000001359","article-title":"Step into the Era of Large Multimodal Models: A Pilot Study on ChatGPT-4V(Ision)\u2019s Ability to Interpret Radiological Images","volume":"110","author":"Zhu","year":"2024","journal-title":"Int. J. Surg."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"110101","DOI":"10.1016\/j.clinimag.2024.110101","article-title":"When Vision Meets Reality: Exploring the Clinical Applicability of GPT-4 with Vision","volume":"108","author":"Deng","year":"2024","journal-title":"Clin. Imaging"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"276","DOI":"10.1055\/s-0043-1777746","article-title":"Assessing the Capability of ChatGPT, Google Bard, and Microsoft Bing in Solving Radiology Case Vignettes","volume":"34","author":"Sarangi","year":"2024","journal-title":"Indian J. Radiol. Imaging"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"184","DOI":"10.1016\/j.csbj.2024.12.019","article-title":"Visual-Textual Integration in LLMs for Medical Diagnosis: A Preliminary Quantitative Analysis","volume":"27","author":"Agbareia","year":"2025","journal-title":"Comput. Struct. Biotechnol. J."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ozkara, B.B., Chen, M.M., Federau, C., Karabacak, M., Briere, T.M., Li, J., and Wintermark, M. (2023). Deep Learning for Detecting Brain Metastases on MRI: A Systematic Review and Meta-Analysis. Cancers, 15.","DOI":"10.3390\/cancers15020334"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"655","DOI":"10.3174\/ajnr.A6982","article-title":"Automated Detection and Segmentation of Brain Metastases in Malignant Melanoma: Evaluation of a Dedicated Deep Learning Model","volume":"42","author":"Pennig","year":"2021","journal-title":"AJNR Am. J. Neuroradiol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1559","DOI":"10.1093\/neuonc\/noac025","article-title":"Development and Validation of a Deep-Learning Model for Detecting Brain Metastases on 3D Post-Contrast MRI: A Multi-Center Multi-Reader Evaluation Study","volume":"24","author":"Yin","year":"2022","journal-title":"Neuro-Oncology"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"e200204","DOI":"10.1148\/ryai.2021200204","article-title":"Three-Dimensional U-Net Convolutional Neural Network for Detection and Segmentation of Intracranial Metastases","volume":"3","author":"Rudie","year":"2021","journal-title":"Radiol. Artif. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"e428","DOI":"10.1016\/S2589-7500(24)00061-X","article-title":"Ethical and Regulatory Challenges of Large Language Models in Medicine","volume":"6","author":"Ong","year":"2024","journal-title":"Lancet Digit. Health"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Elbattah, M., Arnaud, E., Ghazali, D.A., and Dequen, G. (2024, January 3\u20136). Exploring the Ethical Challenges of Large Language Models in Emergency Medicine: A Comparative International Review. Proceedings of the 2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Lisbon, Portugal.","DOI":"10.1109\/BIBM62325.2024.10822376"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/5\/749\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,3]],"date-time":"2026-03-03T15:48:14Z","timestamp":1772552894000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/5\/749"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,3]]},"references-count":34,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["diagnostics16050749"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16050749","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,3]]}}}