{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T15:46:41Z","timestamp":1790610401526,"version":"4.1.0"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T00:00:00Z","timestamp":1781654400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T00:00:00Z","timestamp":1781654400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100019559","name":"Carl von Ossietzky Universit\u00e4t Oldenburg","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100019559","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Multimodal large language models (LLMs) that process text and images are increasingly discussed for medical imaging, yet their diagnostic performance on radiographs remains poorly characterized. We evaluated three commercially available multimodal LLMs (GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro) in a strict zero-shot setting for pediatric wrist fracture detection and compared them with a domain-trained Inception v3 convolutional neural network (CNN) on the same GRAZPEDWRI-DX dataset. We constructed a balanced patient-level test cohort of 1,000 children (2,298 radiographs; 500 fracture, 500 non-fracture). The CNN achieved high diagnostic performance (AUROC 0.905, AUPRC 0.920), whereas all LLMs performed close to chance (accuracies\u2009&lt;\u20090.55, Matthews correlation coefficients\u2009\u2248\u20090) and produced bounding boxes often inconsistent with expert annotations. These findings indicate that, in a strict zero-shot setting, the three commercial multimodal LLMs evaluated here lack reliable diagnostic ability for pediatric wrist fracture detection and should therefore be regarded as exploratory research tools rather than clinically dependable systems for pediatric radiograph interpretation.<\/jats:p>","DOI":"10.1038\/s41598-026-58763-w","type":"journal-article","created":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T20:35:19Z","timestamp":1781728519000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Zero-shot multimodal large language models underperform a domain-trained CNN baseline in pediatric wrist fracture detection"],"prefix":"10.1038","volume":"16","author":[{"given":"Matteo","family":"Haupt","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Wei\u00df","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tim","family":"Bellersen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Martin H.","family":"Maurer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,6,17]]},"reference":[{"key":"58763_CR1","doi-asserted-by":"publisher","first-page":"100021","DOI":"10.1016\/j.ejrai.2025.100021","volume":"2","author":"JP Kao","year":"2025","unstructured":"Kao, J. P. & Kao, H. T. Large Language Models in radiology: A technical and clinical perspective. Eur. J. Radiol. Artif. Intell. 2, 100021 (2025).","journal-title":"Eur. J. Radiol. Artif. Intell."},{"key":"58763_CR2","doi-asserted-by":"publisher","first-page":"900","DOI":"10.3348\/kjr.2025.0599","volume":"26","author":"Y Nam","year":"2025","unstructured":"Nam, Y. et al. Multimodal Large Language Models in Medical Imaging: Current State and Future Directions. Korean J. Radiol. 26, 900\u2013923 (2025).","journal-title":"Korean J. Radiol."},{"key":"58763_CR3","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1007\/s12178-012-9146-7","volume":"6","author":"AA Williams","year":"2012","unstructured":"Williams, A. A. & Lochner, H. V. Pediatric hand and wrist injuries. Curr. Rev. Musculoskelet. Med. 6, 18\u201325 (2012).","journal-title":"Curr. Rev. Musculoskelet. Med."},{"key":"58763_CR4","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1097\/01202412-199704000-00002","volume":"6","author":"LA Landin","year":"1997","unstructured":"Landin, L. A. Epidemiology of children\u2019s fractures. J. Pediatr. Orthop. Part. B. 6, 79\u201383 (1997).","journal-title":"J. Pediatr. Orthop. Part. B"},{"key":"58763_CR5","doi-asserted-by":"publisher","first-page":"11591","DOI":"10.1073\/pnas.1806905115","volume":"115","author":"R Lindsey","year":"2018","unstructured":"Lindsey, R. et al. Deep neural network improves fracture detection by clinicians. Proc. Natl. Acad. Sci. U S A. 115, 11591\u201311596 (2018).","journal-title":"Proc. Natl. Acad. Sci. U S A"},{"key":"58763_CR6","doi-asserted-by":"crossref","unstructured":"Sun, Z. et al. Evaluating GPT-4 on Impressions Generation in Radiology Reports. Radiology 307(5), e231259 (2023).","DOI":"10.1148\/radiol.231259"},{"key":"58763_CR7","doi-asserted-by":"publisher","first-page":"3566","DOI":"10.1007\/s00330-023-10384-x","volume":"34","author":"AM Hasani","year":"2024","unstructured":"Hasani, A. M. et al. Evaluating the performance of Generative Pre-trained Transformer-4 (GPT-4) in standardizing radiology reports. Eur. Radiol. 34, 3566\u20133574 (2024).","journal-title":"Eur. Radiol."},{"key":"58763_CR8","doi-asserted-by":"publisher","first-page":"1231","DOI":"10.1007\/s11604-024-01619-y","volume":"42","author":"Y Sonoda","year":"2024","unstructured":"Sonoda, Y. et al. Diagnostic performances of GPT-4o, Claude 3 Opus, and Gemini 1.5 Pro in \u2018Diagnosis Please\u2019 cases. Jpn J. Radiol. 42, 1231\u20131235 (2024).","journal-title":"Jpn J. Radiol."},{"key":"58763_CR9","doi-asserted-by":"publisher","first-page":"2882","DOI":"10.3390\/diagnostics15222882","volume":"15","author":"JKS Phua","year":"2025","unstructured":"Phua, J. K. S. & Tan, T. S. E. Flexing ChatGPT-4o\u2019s diagnostic muscle: detection of fractures in the ossifying pediatric elbow on radiographs. Diagnostics 15, 2882 (2025).","journal-title":"Diagnostics"},{"key":"58763_CR10","doi-asserted-by":"crossref","unstructured":"Ju, R. Y. & Cai, W. Fracture detection in pediatric wrist trauma X-ray images using YOLOv8 algorithm. Sci. Rep. 2023 13, 1\u201313 (2023).","DOI":"10.1038\/s41598-023-47460-7"},{"key":"58763_CR11","doi-asserted-by":"publisher","first-page":"25095","DOI":"10.1007\/s00521-025-11539-1","volume":"37","author":"P Basak","year":"2025","unstructured":"Basak, P. et al. Improving pediatric trauma care: an automated system for wrist trauma detection using GELAN. Neural Comput. Appl. 37, 25095\u201325121 (2025).","journal-title":"Neural Comput. Appl."},{"key":"58763_CR12","doi-asserted-by":"publisher","unstructured":"Li, Y. et al. A Systematic Evaluation of GPT-4V\u2019s Multimodal Capability for Medical Image Analysis. Preprint at (2024). https:\/\/doi.org\/10.48550\/arXiv.2310.20381","DOI":"10.48550\/arXiv.2310.20381"},{"key":"58763_CR13","doi-asserted-by":"crossref","unstructured":"Hartsock, I. & Rasool, G. Vision-language models for medical report generation and visual question answering: a review. Front. Artif. Intell. 7, 1430984 (2024).","DOI":"10.3389\/frai.2024.1430984"},{"key":"58763_CR14","doi-asserted-by":"publisher","first-page":"e54948","DOI":"10.2196\/54948","volume":"26","author":"F Busch","year":"2024","unstructured":"Busch, F. et al. Integrating Text and Image Analysis: Exploring GPT-4V\u2019s Capabilities in Advanced Radiological Applications Across Subspecialties. J. Med. Internet Res. 26, e54948 (2024).","journal-title":"J. Med. Internet Res."},{"key":"58763_CR15","doi-asserted-by":"publisher","first-page":"108013","DOI":"10.1016\/j.cmpb.2024.108013","volume":"245","author":"J Li","year":"2024","unstructured":"Li, J., Dada, A., Puladi, B., Kleesiek, J. & Egger, J. ChatGPT in healthcare: A taxonomy and systematic review. Comput. Methods Programs Biomed. 245, 108013 (2024).","journal-title":"Comput. Methods Programs Biomed."},{"key":"58763_CR16","doi-asserted-by":"publisher","first-page":"104620","DOI":"10.1016\/j.jbi.2024.104620","volume":"151","author":"Q Wei","year":"2024","unstructured":"Wei, Q. et al. Evaluation of ChatGPT-generated medical responses: A systematic review and meta-analysis. J. Biomed. Inf. 151, 104620 (2024).","journal-title":"J. Biomed. Inf."},{"key":"58763_CR17","doi-asserted-by":"publisher","first-page":"842","DOI":"10.1001\/jama.2023.1044","volume":"329","author":"A Sarraju","year":"2023","unstructured":"Sarraju, A. et al. Appropriateness of Cardiovascular Disease Prevention Recommendations Obtained From a Popular Online Chat-Based Artificial Intelligence Model. JAMA 329, 842\u2013844 (2023).","journal-title":"JAMA"},{"key":"58763_CR18","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1038\/s41586-023-05881-4","volume":"616","author":"M Moor","year":"2023","unstructured":"Moor, M. et al. Foundation models for generalist medical artificial intelligence. Nature 616, 259\u2013265 (2023).","journal-title":"Nature"},{"key":"58763_CR19","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1055\/a-2369-8330","volume":"197","author":"MA Weber","year":"2025","unstructured":"Weber, M. A. Easily missed pathologies of the musculoskeletal system in the emergency radiology setting. ROFO Fortschr. Geb Rontgenstr Nuklearmed. 197, 277\u2013287 (2025).","journal-title":"ROFO Fortschr. Geb Rontgenstr Nuklearmed"},{"key":"58763_CR20","doi-asserted-by":"publisher","first-page":"524","DOI":"10.1016\/j.jemermed.2021.12.021","volume":"62","author":"W Li","year":"2022","unstructured":"Li, W. et al. Pediatric Musculoskeletal Radiographs: Anatomy and Fractures Prone to Diagnostic Error Among Emergency Physicians. J. Emerg. Med. 62, 524\u2013533 (2022).","journal-title":"J. Emerg. Med."},{"key":"58763_CR21","doi-asserted-by":"publisher","first-page":"6853","DOI":"10.1007\/s00330-025-11669-z","volume":"35","author":"T Till","year":"2025","unstructured":"Till, T. et al. Impact of test set composition on AI performance in pediatric wrist fracture detection in X-rays. Eur. Radiol. 35, 6853\u20136864 (2025).","journal-title":"Eur. Radiol."},{"key":"58763_CR22","doi-asserted-by":"publisher","first-page":"339","DOI":"10.1038\/s41746-024-01344-w","volume":"7","author":"L Riedemann","year":"2024","unstructured":"Riedemann, L., Labonne, M. & Gilbert, S. The path forward for large language models in medicine is open. Npj Digit. Med. 7, 339 (2024).","journal-title":"Npj Digit. Med."},{"key":"58763_CR23","doi-asserted-by":"publisher","first-page":"AIcs2500078","DOI":"10.1056\/AIcs2500078","volume":"2","author":"J Clusmann","year":"2025","unstructured":"Clusmann, J. et al. Incidental prompt injections on vision\u2013language models in real-life histopathology. NEJM AI. 2, AIcs2500078 (2025).","journal-title":"NEJM AI"},{"key":"58763_CR24","doi-asserted-by":"publisher","first-page":"e233270","DOI":"10.1148\/radiol.233270","volume":"311","author":"Y Zhou","year":"2024","unstructured":"Zhou, Y. et al. Evaluating GPT-4V (GPT-4 with Vision) on Detection of Radiologic Findings on Chest Radiographs. Radiology 311, e233270 (2024).","journal-title":"Radiology"},{"key":"58763_CR25","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","volume":"42","author":"G Litjens","year":"2017","unstructured":"Litjens, G. et al. A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60\u201388 (2017).","journal-title":"Med. Image Anal."},{"key":"58763_CR26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2201.09873","author":"F Shamshad","year":"2022","unstructured":"Shamshad, F. et al. Transformers in Medical Imaging: A Survey. Preprint at. https:\/\/doi.org\/10.48550\/arXiv.2201.09873 (2022).","journal-title":"Preprint at."},{"key":"58763_CR27","doi-asserted-by":"publisher","unstructured":"Dosovitskiy, A. et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. Preprint at (2021). https:\/\/doi.org\/10.48550\/arXiv.2010.11929","DOI":"10.48550\/arXiv.2010.11929"},{"key":"58763_CR28","first-page":"1","volume":"91 9","author":"E Nagy","year":"2022","unstructured":"Nagy, E., Janisch, M., Hr\u017ei\u0107, F., Sorantin, E. & Tschauner, S. A pediatric wrist trauma X-ray dataset (GRAZPEDWRI-DX) for machine learning. Sci. Data 2022. 91 9, 1\u201310 (2022).","journal-title":"Sci. Data 2022"},{"key":"58763_CR29","unstructured":"Brown, T. B. et al. Curran Associates Inc., Red Hook, NY, USA,. Language models are few-shot learners. in Proceedings of the 34th International Conference on Neural Information Processing Systems 1877\u20131901 (2020)."},{"key":"58763_CR30","doi-asserted-by":"publisher","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J. & Wojna, Z. Rethinking the Inception Architecture for Computer Vision. Preprint at (2015). https:\/\/doi.org\/10.48550\/arXiv.1512.00567","DOI":"10.48550\/arXiv.1512.00567"},{"key":"58763_CR31","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1002\/bimj.200410135","volume":"47","author":"R Fluss","year":"2005","unstructured":"Fluss, R., Faraggi, D. & Reiser, B. Estimation of the Youden Index and its associated cutoff point. Biom J. Biom Z. 47, 458\u2013472 (2005).","journal-title":"Biom J. Biom Z."},{"key":"58763_CR32","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1111\/j.2517-6161.1988.tb01731.x","volume":"50","author":"DV Hinkley","year":"1988","unstructured":"Hinkley, D. V. & Bootstrap Methods J. R Stat. Soc. Ser. B Stat. Methodol. 50, 321\u2013337 (1988).","journal-title":"J. R Stat. Soc. Ser. B Stat. Methodol."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-58763-w","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-58763-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-58763-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T20:55:18Z","timestamp":1781729718000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-58763-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,17]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["58763"],"URL":"https:\/\/doi.org\/10.1038\/s41598-026-58763-w","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,17]]},"assertion":[{"value":"16 December 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 June 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 June 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"This study is a secondary analysis of the publicly available, fully de-identified GRAZPEDWRI-DX dataset. As all data were anonymized prior to public release and no additional data were collected, no additional institutional review board approval was required. The analysis was conducted in accordance with the Declaration of Helsinki.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}],"article-number":"18873"}}