{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T04:01:03Z","timestamp":1793073663995,"version":"4.5.0"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T00:00:00Z","timestamp":1736899200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T00:00:00Z","timestamp":1736899200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010269","name":"Wellcome Trust","doi-asserted-by":"publisher","award":["203930\/B\/16\/Z"],"award-info":[{"award-number":["203930\/B\/16\/Z"]}],"id":[{"id":"10.13039\/100010269","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Eur Radiol Exp"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Background<\/jats:title>\n            <jats:p>The large language model ChatGPT can now accept image input with the GPT4-vision (GPT4V) version. We aimed to compare the performance of GPT4V to pretrained U-Net and vision transformer (ViT) models for the identification of the progression of multiple sclerosis (MS) on magnetic resonance imaging (MRI).<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>Paired coregistered MR images with and without progression were provided as input to ChatGPT4V in a zero-shot experiment to identify radiologic progression. Its performance was compared to pretrained U-Net and ViT models. Accuracy was the primary evaluation metric and 95% confidence interval (CIs) were calculated by bootstrapping. We included 170 patients with MS (50 males, 120 females), aged 21\u201374 years (mean 42.3), imaged at a single institution from 2019 to 2021, each with 2\u20135 MRI studies (496 in total).<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>One hundred seventy patients were included, 110 for training, 30 for tuning, and 30 for testing; 100 unseen paired images were randomly selected from the test set for evaluation. Both U-Net and ViT had 94% (95% CI: 89\u201398%) accuracy while GPT4V had 85% (77\u201391%). GPT4V gave cautious nonanswers in six cases. GPT4V had precision (specificity), recall (sensitivity), and F1 score of 89% (75\u201393%), 92% (82\u201398%), 91 (82\u201397%) compared to 100% (100\u2013100%), 88 (78\u201396%), and 0.94 (88\u201398%) for U-Net and 94% (87\u2013100%), 94 (88\u2013100%), and 94 (89\u201398%) for ViT.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion<\/jats:title>\n            <jats:p>The performance of GPT4V combined with its accessibility suggests has the potential to impact AI radiology research. However, misclassified cases and overly cautious non-answers confirm that it is not yet ready for clinical use.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Relevance statement<\/jats:title>\n            <jats:p>GPT4V can identify the radiologic progression of MS in a simplified experimental setting. However, GPT4V is not a medical device, and its widespread availability highlights the need for caution and education for lay users, especially those with limited access to expert healthcare.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Key Points<\/jats:title>\n            <jats:p>\n              <jats:list list-type=\"bullet\">\n                <jats:list-item>\n                  <jats:p>Without fine-tuning or the need for prior coding experience, GPT4V can perform a zero-shot radiologic change detection task with reasonable accuracy.<\/jats:p>\n                <\/jats:list-item>\n                <jats:list-item>\n                  <jats:p>However, in absolute terms, in a simplified \u201cspot the difference\u201d medical imaging task, GPT4V was inferior to state-of-the-art computer vision methods.<\/jats:p>\n                <\/jats:list-item>\n                <jats:list-item>\n                  <jats:p>GPT4V\u2019s performance metrics were more similar to the ViT than the U-net.<\/jats:p>\n                <\/jats:list-item>\n                <jats:list-item>\n                  <jats:p>This is an exploratory experimental study and GPT4V is not intended for use as a medical device.<\/jats:p>\n                <\/jats:list-item>\n              <\/jats:list>\n            <\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Graphical Abstract<\/jats:title>\n          <\/jats:sec>","DOI":"10.1186\/s41747-024-00547-w","type":"journal-article","created":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T14:36:55Z","timestamp":1736951815000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Can ChatGPT4-vision identify radiologic progression of multiple sclerosis on brain MRI?"],"prefix":"10.1186","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3449-8017","authenticated-orcid":false,"given":"Brendan S.","family":"Kelly","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sophie","family":"Duignan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Prateek","family":"Mathur","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Henry","family":"Dillon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edward H.","family":"Lee","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristen W.","family":"Yeom","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pearse A.","family":"Keane","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aonghus","family":"Lawlor","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ronan P.","family":"Killeen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,1,15]]},"reference":[{"key":"547_CR1","doi-asserted-by":"publisher","first-page":"169","DOI":"10.1056\/nejmra1401483","volume":"378","author":"DS Reich","year":"2018","unstructured":"Reich DS, Lucchinetti CF, Calabresi PA (2018) Multiple sclerosis. N Engl J Med 378:169\u2013180. https:\/\/doi.org\/10.1056\/nejmra1401483","journal-title":"N Engl J Med"},{"key":"547_CR2","doi-asserted-by":"publisher","first-page":"1664","DOI":"10.3174\/ajnr.a5147","volume":"38","author":"C McNamara","year":"2017","unstructured":"McNamara C, Sugrue G, Murray B, MacMahon PJ (2017) Current and emerging therapies in multiple sclerosis: implications for the radiologist, part 1\u2014mechanisms, efficacy, and safety. AJNR Am J Neuroradiol 38:1664\u20131671. https:\/\/doi.org\/10.3174\/ajnr.a5147","journal-title":"AJNR Am J Neuroradiol"},{"key":"547_CR3","doi-asserted-by":"publisher","first-page":"5877","DOI":"10.3390\/ijms23115877","volume":"23","author":"J Yang","year":"2022","unstructured":"Yang J, Hamade M, Wu Q et al (2022) Current and future biomarkers in multiple sclerosis. Int J Mol Sci 23:5877. https:\/\/doi.org\/10.3390\/ijms23115877","journal-title":"Int J Mol Sci"},{"key":"547_CR4","doi-asserted-by":"publisher","first-page":"1222","DOI":"10.1016\/j.jacr.2018.05.037","volume":"15","author":"AB Rosenkrantz","year":"2018","unstructured":"Rosenkrantz AB, Duszak R, Babb JS et al (2018) Discrepancy rates and clinical impact of imaging secondary interpretations: a systematic review and meta-analysis. J Am Coll Radiol 15:1222\u20131231. https:\/\/doi.org\/10.1016\/j.jacr.2018.05.037","journal-title":"J Am Coll Radiol"},{"key":"547_CR5","doi-asserted-by":"publisher","first-page":"7349","DOI":"10.1007\/s00521-022-07099-3","volume":"34","author":"M Sah","year":"2022","unstructured":"Sah M, Direkoglu C (2022) A survey of deep learning methods for multiple sclerosis identification using brain MRI images. Neural Comput Appl 34:7349\u20137373. https:\/\/doi.org\/10.1007\/s00521-022-07099-3","journal-title":"Neural Comput Appl"},{"key":"547_CR6","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A et al (2021) An image is worth 16\u2009\u00d7\u200916 words: transformers for image recognition at scale. Preprint at https:\/\/arxiv.org\/abs\/2010.11929. Accessed 6 Nov 2023"},{"key":"547_CR7","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2023.102802","volume":"88","author":"F Shamshad","year":"2023","unstructured":"Shamshad F, Khan S, Zamir SW et al (2023) Transformers in medical imaging: a survey. Med Image Anal 88:102802. https:\/\/doi.org\/10.1016\/j.media.2023.102802","journal-title":"Med Image Anal"},{"key":"547_CR8","doi-asserted-by":"publisher","unstructured":"Yang Z, Li L, Lin K et al (2023) The dawn of LMMs: preliminary explorations with GPT-4V(ision). Preprint at https:\/\/doi.org\/10.48550\/arxiv.2309.17421","DOI":"10.48550\/arxiv.2309.17421"},{"key":"547_CR9","doi-asserted-by":"publisher","first-page":"7","DOI":"10.1186\/s41747-023-00420-2","volume":"8","author":"B Hamm","year":"2024","unstructured":"Hamm B, Marti-Bonmati L, Sardanelli F (2024) ESR journals editors\u2019 joint statement on guidelines for the use of large language models by authors, reviewers, and editors. Eur Radiol Exp 8:7. https:\/\/doi.org\/10.1186\/s41747-023-00420-2","journal-title":"Eur Radiol Exp"},{"key":"547_CR10","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1186\/s13244-024-01769-7","volume":"15","author":"J Zhong","year":"2024","unstructured":"Zhong J, Xing Y, Hu Y et al (2024) The policies on the use of large language models in radiological journals are lacking: a meta-research study. Insights Imaging 15:186. https:\/\/doi.org\/10.1186\/s13244-024-01769-7","journal-title":"Insights Imaging"},{"key":"547_CR11","doi-asserted-by":"publisher","unstructured":"Horiuchi D, Tatekawa H, Oura T et al (2024) ChatGPT\u2019s diagnostic performance based on textual vs. visual information compared to radiologists\u2019 diagnostic performance in musculoskeletal radiology. Eur Radiol. https:\/\/doi.org\/10.1007\/s00330-024-10902-5","DOI":"10.1007\/s00330-024-10902-5"},{"key":"547_CR12","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1186\/s41747-024-00457-x","volume":"8","author":"S Rau","year":"2024","unstructured":"Rau S, Rau A, Nattenm\u00fcller J et al (2024) A retrieval-augmented chatbot based on GPT-4 provides appropriate differential diagnosis in gastrointestinal radiology: a proof of concept study. Eur Radiol Exp 8:60. https:\/\/doi.org\/10.1186\/s41747-024-00457-x","journal-title":"Eur Radiol Exp"},{"key":"547_CR13","doi-asserted-by":"publisher","unstructured":"Sun L, Yan Z, Zhang K, Zhou R, He L, Li X (2023) Multimodal ChatGPT for medical applications: an experimental study of GPT-4V. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2310.19061","DOI":"10.48550\/arXiv.2310.19061"},{"key":"547_CR14","doi-asserted-by":"publisher","first-page":"990","DOI":"10.1016\/j.jacr.2023.05.003","volume":"20","author":"A Rao","year":"2023","unstructured":"Rao A, Kim J, Kamineni M et al (2023) Evaluating GPT as an adjunct for radiologic decision making: GPT-4 versus GPT-3.5 in a breast imaging pilot. J Am Coll Radiol 20:990\u2013997. https:\/\/doi.org\/10.1016\/j.jacr.2023.05.003","journal-title":"J Am Coll Radiol"},{"key":"547_CR15","doi-asserted-by":"publisher","unstructured":"Kelly BS, Judge C, Bollard SM et al (2022) Radiology artificial intelligence: a systematic review and evaluation of methods (RAISE). Eur Radiol. https:\/\/doi.org\/10.1007\/s00330-022-08784-6","DOI":"10.1007\/s00330-022-08784-6"},{"key":"547_CR16","unstructured":"Bandara WGC, Patel VM (2023) A transformer-based siamese network for change detection. Preprint at https:\/\/arxiv.org\/pdf\/2201.01293.pdf. Accessed 6 Nov 2023"},{"key":"547_CR17","doi-asserted-by":"publisher","unstructured":"Acosta JN, Falcone GJ, Rajpurkar P (2022) The need for medical artificial intelligence that incorporates prior Images. Radiology. https:\/\/doi.org\/10.1148\/radiol.212830","DOI":"10.1148\/radiol.212830"},{"key":"547_CR18","doi-asserted-by":"publisher","first-page":"2251","DOI":"10.1109\/tpami.2018.2857768","volume":"41","author":"Y Xian","year":"2017","unstructured":"Xian Y, Lampert CH, Schiele B, Akata Z (2017) Zero-shot learning\u2014a comprehensive evaluation of the good, the bad and the ugly. IEEE Trans Pattern Anal Mach Intell 41:2251\u20132265. https:\/\/doi.org\/10.1109\/tpami.2018.2857768","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"547_CR19","doi-asserted-by":"crossref","unstructured":"Daudt RC, Saux BL, Boulch A (2018) Fully convolutional siamese networks for change detection. In: 25th IEEE international conference on image processing. IEEE, pp 4063\u20134067","DOI":"10.1109\/ICIP.2018.8451652"},{"key":"547_CR20","doi-asserted-by":"publisher","first-page":"e200029","DOI":"10.1148\/ryai.2020200029","volume":"2","author":"J Mongan","year":"2020","unstructured":"Mongan J, Moy L, Kahn Jr CE (2020) Checklist for artificial intelligence in medical imaging (CLAIM): a guide for authors and reviewers. Radiol Artif Intell 2:e200029. https:\/\/doi.org\/10.1148\/ryai.2020200029","journal-title":"Radiol Artif Intell"},{"key":"547_CR21","doi-asserted-by":"publisher","first-page":"162","DOI":"10.1016\/s1474-4422(17)30470-2","volume":"17","author":"AJ Thompson","year":"2018","unstructured":"Thompson AJ, Banwell BL, Barkhof F et al (2018) Diagnosis of multiple sclerosis: 2017 revisions of the McDonald criteria. Lancet Neurol 17:162\u2013173. https:\/\/doi.org\/10.1016\/s1474-4422(17)30470-2","journal-title":"Lancet Neurol"},{"key":"547_CR22","doi-asserted-by":"publisher","first-page":"471","DOI":"10.1038\/nrneurol.2015.106","volume":"11","author":"\u00c0 Rovira","year":"2015","unstructured":"Rovira \u00c0, Wattjes MP, Tintor\u00e9 M et al (2015) MAGNIMS consensus guidelines on the use of MRI in multiple sclerosis\u2014clinical implementation in the diagnostic process. Nat Rev Neurol 11:471\u2013482. https:\/\/doi.org\/10.1038\/nrneurol.2015.106","journal-title":"Nat Rev Neurol"},{"key":"547_CR23","doi-asserted-by":"publisher","first-page":"782","DOI":"10.1016\/j.neuroimage.2011.09.015","volume":"62","author":"M Jenkinson","year":"2012","unstructured":"Jenkinson M, Beckmann CF, Behrens TEJ et al (2012) FSL. NeuroImage 62:782\u2013790. https:\/\/doi.org\/10.1016\/j.neuroimage.2011.09.015","journal-title":"NeuroImage"},{"key":"547_CR24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/tgrs.2021.3095166","volume":"60","author":"H Chen","year":"2022","unstructured":"Chen H, Qi Z, Shi Z (2022) Remote sensing image change detection with transformers. IEEE Trans Geosci Remote Sens 60:1\u201314. https:\/\/doi.org\/10.1109\/tgrs.2021.3095166","journal-title":"IEEE Trans Geosci Remote Sens"},{"key":"547_CR25","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2023.102918","volume":"89","author":"MA Mazurowski","year":"2023","unstructured":"Mazurowski MA, Dong H, Gu H et al (2023) Segment anything model for medical image analysis: an experimental study. Med Image Anal 89:102918. https:\/\/doi.org\/10.1016\/j.media.2023.102918","journal-title":"Med Image Anal"},{"key":"547_CR26","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1016\/j.media.2016.10.004","volume":"36","author":"K Kamnitsas","year":"2017","unstructured":"Kamnitsas K, Ledig C, Newcombe VFJ et al (2017) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36:61\u201378. https:\/\/doi.org\/10.1016\/j.media.2016.10.004","journal-title":"Med Image Anal"},{"key":"547_CR27","doi-asserted-by":"publisher","first-page":"3342","DOI":"10.1109\/embc.2016.7591443","volume":"2016","author":"PA Yushkevich","year":"2016","unstructured":"Yushkevich PA, Gao Y, Gerig G et al (2016) ITK-SNAP: an interactive tool for semi-automatic segmentation of multi-modality biomedical images. Annu Int Conf IEEE Eng Med Biol Soc 2016:3342\u20133345. https:\/\/doi.org\/10.1109\/embc.2016.7591443","journal-title":"Annu Int Conf IEEE Eng Med Biol Soc"},{"key":"547_CR28","doi-asserted-by":"publisher","unstructured":"Maier-Hein L, Reinke A, Christodoulou E et al (2022) Metrics reloaded: pitfalls and recommendations for image analysis validation. Preprint at https:\/\/doi.org\/10.48550\/arXiv.2206.01653","DOI":"10.48550\/arXiv.2206.01653"},{"key":"547_CR29","doi-asserted-by":"publisher","first-page":"e323","DOI":"10.2196\/jmir.5870","volume":"18","author":"W Luo","year":"2016","unstructured":"Luo W, Phung D, Tran T et al (2016) Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J Med Internet Res 18:e323. https:\/\/doi.org\/10.2196\/jmir.5870","journal-title":"J Med Internet Res"},{"key":"547_CR30","doi-asserted-by":"publisher","first-page":"240","DOI":"10.1186\/s13244-024-01801-w","volume":"15","author":"M Zanardo","year":"2024","unstructured":"Zanardo M, Visser JJ, Colarieti A et al (2024) Impact of AI on radiology: a EuroAIM\/EuSoMII 2024 survey among members of the European Society of Radiology. Insights Imaging 15:240. https:\/\/doi.org\/10.1186\/s13244-024-01801-w","journal-title":"Insights Imaging"},{"key":"547_CR31","doi-asserted-by":"publisher","first-page":"e232","DOI":"10.1016\/s2589-7500(19)30108-6","volume":"1","author":"L Faes","year":"2019","unstructured":"Faes L, Wagner SK, Fu DJ et al (2019) Automated deep learning design for medical image classification by health-care professionals with no coding experience: a feasibility study. Lancet Digit Health 1:e232\u2013e242. https:\/\/doi.org\/10.1016\/s2589-7500(19)30108-6","journal-title":"Lancet Digit Health"},{"key":"547_CR32","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1186\/s41747-024-00485-7","volume":"8","author":"F Pesapane","year":"2024","unstructured":"Pesapane F, Cuocolo R, Sardanelli F (2024) The Picasso\u2019s skepticism on computer science and the dawn of generative AI: questions after the answers to keep \u201cmachines-in-the-loop.\u201d. Eur Radiol Exp 8:81. https:\/\/doi.org\/10.1186\/s41747-024-00485-7","journal-title":"Eur Radiol Exp"}],"container-title":["European Radiology Experimental"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s41747-024-00547-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s41747-024-00547-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s41747-024-00547-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T14:37:00Z","timestamp":1736951820000},"score":1,"resource":{"primary":{"URL":"https:\/\/eurradiolexp.springeropen.com\/articles\/10.1186\/s41747-024-00547-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,15]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["547"],"URL":"https:\/\/doi.org\/10.1186\/s41747-024-00547-w","relation":{},"ISSN":["2509-9280"],"issn-type":[{"value":"2509-9280","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,15]]},"assertion":[{"value":"25 July 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 December 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 January 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This retrospective study was granted IRB approval by the SVUH ethics committee SVUH_RS_020-061_3. The requirement for written informed consent was waived by the IRB.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"Not applicable.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}}],"article-number":"9"}}