{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T22:33:17Z","timestamp":1782426797782,"version":"3.54.5"},"reference-count":38,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T00:00:00Z","timestamp":1770249600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: Bone age assessment is critical in pediatric endocrinology and forensic medicine. Although recently developed multimodal large language models (LLMs) show potential in medical imaging, their diagnostic performance in bone age determination has not been sufficiently evaluated. This study evaluates the performance of four multimodal LLMs (ChatGPT-5, Gemini 2.5 Pro, Grok-3, and Claude 4 Sonnet) in bone age determination using the Gilsanz\u2013Ratib (GR) atlas. Methods: This retrospective study included 245 pediatric patients (109 male, 136 female) under the age of 18 who underwent left wrist radiography. Each model estimated bone age using the patient\u2019s radiograph and GR atlas as reference (atlas-assisted prompting). Bone age assessments made by an experienced radiologist using the GR atlas were evaluated as the reference standard. Performance was assessed using mean absolute error (MAE), intraclass correlation coefficient (ICC), and Bland\u2013Altman analysis. Results: ChatGPT-5 demonstrated statistically superior performance, with an MAE of 1.46 years and ICC of 0.849, showing the highest alignment with the reference standard. Gemini 2.5 Pro showed moderate performance, with an MAE of 2.24 years; Grok-3 (MAE: 3.14 years) and Claude 4 Sonnet (MAE: 4.29 years) had error rates that were too high for clinical use. Conclusions: Significant performance differences exist among multimodal LLMs, despite atlas-supported prompting. Only ChatGPT-5 qualified as \u201cclinically useful,\u201d demonstrating potential as an auxiliary tool or educational support under expert supervision. Other models\u2019 reliability remains insufficient.<\/jats:p>","DOI":"10.3390\/diagnostics16030487","type":"journal-article","created":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T11:13:01Z","timestamp":1770289981000},"page":"487","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Atlas-Assisted Bone Age Estimation from Hand\u2013Wrist Radiographs Using Multimodal Large Language Models: A Comparative Study"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8120-7051","authenticated-orcid":false,"given":"Erdem","family":"Ozkan","sequence":"first","affiliation":[{"name":"Department of Radiology, Kastamonu Training and Research Hospital, Kastamonu 37150, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9811-4385","authenticated-orcid":false,"given":"Mustafa","family":"Koyun","sequence":"additional","affiliation":[{"name":"Department of Radiology, Kastamonu Training and Research Hospital, Kastamonu 37150, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1097\/00000441-195909000-00030","article-title":"Radiographic atlas of skeletal development of the hand and wrist","volume":"238","author":"Greulich","year":"1959","journal-title":"Am. J. Med. Sci."},{"key":"ref_2","unstructured":"Gilsanz, V., and Ratib, O. (2005). Hand Bone Age: A Digital Atlas of Skeletal Maturity, Springer."},{"key":"ref_3","unstructured":"Tanner, J.M. (1983). Assessment of Skeletal Maturity and Prediction of Adult Height (TW 2 Method), Academic Press."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1297\/cpe.24.143","article-title":"Bone age: Assessment methods and clinical applications","volume":"24","author":"Satoh","year":"2015","journal-title":"Clin. Pediatr. Endocrinol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"852","DOI":"10.1136\/bjsports-2015-094623","article-title":"Biological maturation of youth athletes: Assessment and implications","volume":"49","author":"Malina","year":"2015","journal-title":"Br. J. Sports Med."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"457","DOI":"10.1007\/s00414-008-0254-2","article-title":"Criteria for age estimation in living individuals","volume":"122","author":"Schmeling","year":"2008","journal-title":"Int. J. Leg. Med."},{"key":"ref_7","unstructured":"Tanner, J.M., Healy, M.J.R., Cameron, N., and Goldstein, H. (2001). Assessment of Skeletal Maturity and Prediction of Adult Height (TW3 Method), W.B. Saunders."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.forsciint.2007.02.023","article-title":"Is the assessment of bone age by the Greulich-Pyle method reliable at forensic age estimation for Turkish children?","volume":"173","author":"Safak","year":"2007","journal-title":"Forensic. Sci. Int."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1395","DOI":"10.2214\/ajr.167.6.8956565","article-title":"Bone age in children of diverse ethnicity","volume":"167","author":"Ontell","year":"1996","journal-title":"AJR Am. J. Roentgenol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1136\/adc.81.2.172","article-title":"Bone age assessment: A large scale comparison of the Greulich and Pyle, and Tanner and Whitehouse (TW2) methods","volume":"81","author":"Bull","year":"1999","journal-title":"Arch. Dis. Child"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"500","DOI":"10.1038\/s41568-018-0016-5","article-title":"Artificial intelligence in radiology","volume":"18","author":"Hosny","year":"2018","journal-title":"Nat. Rev. Cancer"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: The convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat. Med."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1186\/s41747-018-0061-6","article-title":"Artificial intelligence in medical imaging: Threat or opportunity? Radiologists again at the forefront of innovation in medicine","volume":"2","author":"Pesapane","year":"2018","journal-title":"Eur. Radiol. Exp."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1007\/s10278-017-9955-8","article-title":"Fully Automated Deep Learning System for Bone Age Assessment","volume":"30","author":"Lee","year":"2017","journal-title":"J. Digit. Imaging"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"498","DOI":"10.1148\/radiol.2018180736","article-title":"The RSNA Pediatric Bone Age Machine Learning Challenge","volume":"290","author":"Halabi","year":"2019","journal-title":"Radiology"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1148\/radiol.2017170236","article-title":"Performance of a Deep-Learning Neural Network Model in Assessing Skeletal Maturity on Pediatric Hand Radiographs","volume":"287","author":"Larson","year":"2018","journal-title":"Radiology"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.media.2016.10.010","article-title":"Deep learning for automated skeletal bone age assessment in X-ray images","volume":"36","author":"Spampinato","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Iglovikov, V.I., Rakhlin, A., Kalinin, A.A., and Shvets, A.A. (2018). Paediatric Bone Age Assessment Using Deep Convolutional Neural Networks. Proceedings of the Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, Springer.","DOI":"10.1101\/234120"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1038\/s41586-023-05881-4","article-title":"Foundation models for generalist medical artificial intelligence","volume":"616","author":"Moor","year":"2023","journal-title":"Nature"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1930","DOI":"10.1038\/s41591-023-02448-8","article-title":"Large language models in medicine","volume":"29","author":"Thirunavukarasu","year":"2023","journal-title":"Nat. Med."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","year":"2023","journal-title":"Nature"},{"key":"ref_22","first-page":"1877","article-title":"Language models are few-shot learners","volume":"33","author":"Brown","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_23","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18\u201324). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning, Virtual Event."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e233270","DOI":"10.1148\/radiol.233270","article-title":"Evaluating GPT-4V (GPT-4 with Vision) on Detection of Radiologic Findings on Chest Radiographs","volume":"311","author":"Zhou","year":"2024","journal-title":"Radiology"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"4374","DOI":"10.1007\/s00330-024-11339-6","article-title":"CXR-LLaVA: A multimodal large language model for interpreting chest X-ray images","volume":"35","author":"Lee","year":"2025","journal-title":"Eur. Radiol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1959","DOI":"10.1007\/s00330-024-11035-5","article-title":"Assessing GPT-4 multimodal performance in radiological image analysis","volume":"35","author":"Brin","year":"2025","journal-title":"Eur. Radiol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"900","DOI":"10.3348\/kjr.2025.0599","article-title":"Multimodal Large Language Models in Medical Imaging: Current State and Future Directions","volume":"26","author":"Nam","year":"2025","journal-title":"Korean J. Radiol."},{"key":"ref_28","unstructured":"xAI (2025, September 25). Grok 3 Beta\u2014The Age of Reasoning Agents. Available online: https:\/\/grok.com\/."},{"key":"ref_29","unstructured":"Google (2025, September 27). Gemini 2.5 Pro. Available online: https:\/\/gemini.google.com\/app."},{"key":"ref_30","unstructured":"Anthropic (2025, September 29). Introducing Claude 4. Available online: https:\/\/claude.ai\/new."},{"key":"ref_31","unstructured":"OpenAI (2025, October 01). Introducing GPT-5. Available online: https:\/\/chatgpt.com\/."},{"key":"ref_32","first-page":"34892","article-title":"Visual instruction tuning","volume":"36","author":"Liu","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_33","first-page":"49250","article-title":"Instructblip: Towards general-purpose vision-language models with instruction tuning","volume":"36","author":"Dai","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1002\/sim.1108","article-title":"Sample size requirements for estimating intraclass correlations with desired precision","volume":"21","author":"Bonett","year":"2002","journal-title":"Stat. Med."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1002\/(SICI)1097-0258(19980115)17:1<101::AID-SIM727>3.0.CO;2-E","article-title":"Sample size and optimal designs for reliability studies","volume":"17","author":"Walter","year":"1998","journal-title":"Stat. Med."},{"key":"ref_36","first-page":"24824","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"6905","DOI":"10.1016\/j.acra.2025.07.058","article-title":"Multimodal Large Language Models for Pediatric Bone-Age Assessment: A Comparative Accuracy Analysis","volume":"32","author":"Salbas","year":"2025","journal-title":"Acad. Radiol."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"624","DOI":"10.1203\/00006450-200111000-00015","article-title":"Skeletal age determinations in children of European and African descent: Applicability of the Greulich and Pyle standards","volume":"50","author":"Mora","year":"2001","journal-title":"Pediatr. Res."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/3\/487\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,5]],"date-time":"2026-02-05T11:23:41Z","timestamp":1770290621000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/3\/487"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,5]]},"references-count":38,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["diagnostics16030487"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16030487","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,5]]}}}