{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T11:42:30Z","timestamp":1776253350739,"version":"3.50.1"},"reference-count":34,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T00:00:00Z","timestamp":1776211200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background\/Objectives: Multimodal large language models (MLLMs) are increasingly evaluated for diagnostic tasks in medical imaging, including radiographic interpretation. However, most studies focus primarily on binary fracture detection and rarely assess clinically relevant fracture characteristics such as displacement or intra-articular extension, which influence treatment decisions. In addition, most evaluations rely on single-run inference designs that do not assess response reproducibility. This study evaluated the diagnostic performance and inter-run reliability of five MLLMs for radiographic assessment of distal radius fractures. Methods: Fifty fracture-positive distal radius radiographs were evaluated by five MLLMs (ChatGPT 5.3, Gemini 3.1 Pro, Claude Opus 4.6, Grok 4.1, and ERNIE 5.0) across five independent zero-shot inference runs (n = 1250 observations). Diagnostic tasks included fracture detection, intra-articular extension, and displacement. Sex and age were exploratory endpoints. Performance was summarized using sensitivity (fracture detection) and accuracy (other tasks), with inter-run reliability assessed via Fleiss\u2019 \u03ba. Results: Performance varied across tasks and models. Fracture detection sensitivity ranged from 39.6% to 99.6%, with two models exceeding 90%. Intra-articular extension accuracy ranged from 51.6% to 55.6%, consistent with chance-level performance. Displacement classification ranged from 34.8% to 70.4%. One model achieved substantial inter-run agreement across binary tasks (\u03ba &gt; 0.60), whereas two models showed slight agreement (\u03ba &lt; 0.20). Conclusions: Only two models exceeded 90% sensitivity for fracture detection, while intra-articular extension remained at chance level (\u226455.6%). Substantial inter-run reliability (\u03ba &gt; 0.60) was observed in only one model. These findings indicate that current MLLMs do not reliably support multidimensional fracture assessment and that single-run evaluations overestimate robustness.<\/jats:p>","DOI":"10.3390\/diagnostics16081171","type":"journal-article","created":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T10:30:23Z","timestamp":1776249023000},"page":"1171","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["AI in Hand and Wrist Radiography: Multimodal Large Language Models for Distal Radius Fracture Detection and Characterization"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6395-6670","authenticated-orcid":false,"given":"Ibrahim","family":"G\u00fcler","sequence":"first","affiliation":[{"name":"Department of Plastic, Aesthetic and Hand Surgery, Otto-von-Guericke University, 39120 Magdeburg, Germany"},{"name":"Department of Health Management, Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU), Lange Gasse 20, 90403 N\u00fcrnberg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6557-2163","authenticated-orcid":false,"given":"Armin","family":"Kraus","sequence":"additional","affiliation":[{"name":"Department of Plastic, Aesthetic and Hand Surgery, Otto-von-Guericke University, 39120 Magdeburg, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7302-210X","authenticated-orcid":false,"given":"Gerrit","family":"Grieb","sequence":"additional","affiliation":[{"name":"Department of Plastic Surgery and Hand Surgery, Gemeinschaftskrankenhaus Havelhoehe, Kladower Damm 221, 14089 Berlin, Germany"},{"name":"Department of Plastic Surgery and Hand Surgery, Medical Faculty, RWTH Aachen University, Pauwelsstrasse 30, 52074 Aachen, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Breidung","sequence":"additional","affiliation":[{"name":"Department of Health, University of Witten\/Herdecke, 58455 Witten, Germany"},{"name":"Department of Plastic Surgery and Hand Surgery, University Hospital Zurich, R\u00e4mistrasse 100, 8091 Zurich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Martin","family":"Lautenbach","sequence":"additional","affiliation":[{"name":"Department of Hand Surgery, Orthopedics, and Trauma Surgery, Waldfriede Hospital, Argentinische Allee 40, 14163 Berlin, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-9391-3918","authenticated-orcid":false,"given":"Henrik","family":"Stelling","sequence":"additional","affiliation":[{"name":"Department of Health Management, Friedrich-Alexander-Universit\u00e4t Erlangen-N\u00fcrnberg (FAU), Lange Gasse 20, 90403 N\u00fcrnberg, Germany"},{"name":"Practices for Nuclear Medicine, Rubensstra\u00dfe 125, 12157 Berlin, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,4,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"325","DOI":"10.4103\/2249-4863.148101","article-title":"Fractures of distal radius\u2014An overview","volume":"3","author":"Meena","year":"2014","journal-title":"J. Fam. Med. Prim. Care"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1177\/15589447231219286","article-title":"Upper extremity fractures in the emergency department: A database analysis of national trends in the United States","volume":"20","author":"Karzon","year":"2025","journal-title":"Hand"},{"key":"ref_3","first-page":"304","article-title":"Perioperative management of distal radius fractures","volume":"31","author":"Lim","year":"2021","journal-title":"J. Perioper. Pract."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1097\/01.blo.0000132884.51311.28","article-title":"Assessment of articular displacement of distal radius fractures","volume":"423","author":"Catalano","year":"2004","journal-title":"Clin. Orthop. Relat. Res."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Johnson, N., Leighton, P., Pailthorpe, C., and Dias, J. (2019). Distal Radius Fracture Delphi Study Group. Defining displacement thresholds for surgical intervention for distal radius fractures\u2014A Delphi study. PLoS ONE, 14.","DOI":"10.1371\/journal.pone.0210462"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1097\/00003086-199606000-00016","article-title":"Operative treatment of intraarticular distal radial fractures","volume":"327","author":"Lipton","year":"1996","journal-title":"Clin. Orthop. Relat. Res."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1457","DOI":"10.1302\/0301-620X.103B9.BJJ-2021-0313.R1","article-title":"Origins of the threshold for surgical intervention in intra-articular distal radius fractures","volume":"103","author":"Esworthy","year":"2021","journal-title":"Bone Jt. J."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"250617","DOI":"10.1148\/radiol.250617","article-title":"One year on: Assessing progress of multimodal large language model performance on RSNA 2024 Case of the Day questions","volume":"316","author":"Hou","year":"2025","journal-title":"Radiology"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3032","DOI":"10.1002\/ksa.12702","article-title":"Artificial intelligence-assisted analysis of musculoskeletal imaging\u2014A narrative review of the current state of machine learning models","volume":"33","author":"Oettl","year":"2025","journal-title":"Knee Surg. Sports Traumatol. Arthrosc."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"900","DOI":"10.3348\/kjr.2025.0599","article-title":"Multimodal large language models in medical imaging: Current state and future directions","volume":"26","author":"Nam","year":"2025","journal-title":"Korean J. Radiol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"7140","DOI":"10.1038\/s41598-025-88925-1","article-title":"ChatGPT performance in assessing musculoskeletal MRI scan appropriateness based on ACR appropriateness criteria","volume":"15","author":"Tan","year":"2025","journal-title":"Sci. Rep."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"111842","DOI":"10.1016\/j.ejrad.2024.111842","article-title":"Large language models in radiology: Fluctuating performance and decreasing discordance over time","volume":"182","author":"Gupta","year":"2025","journal-title":"Eur. J. Radiol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Holmstrom, J.A., Braithwaite, C.L., Alhankawi, A.R., Moore, M.L., Patel, K.A., and Miller, B.H. (2026). Comparing the efficacy between ChatGPT 5, Grok 3, and Claude 4.5 Sonnet in analyzing orthopedic trauma-related imaging. J. Orthop. Trauma, 10\u20131097.","DOI":"10.1097\/BOT.0000000000003166"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e43716","DOI":"10.1097\/MD.0000000000043716","article-title":"Diagnostic performance of ChatGPT-4.0 in elbow fracture detection: A comparative study of radial head, distal humerus, and olecranon fractures","volume":"104","author":"Uyar","year":"2025","journal-title":"Medicine"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Alic, T., Zehir, S., Yalcinkaya, M., Deniz, E., Kiran, H.E., and Afacan, O. (2025). Artificial intelligence-assisted accurate diagnosis of anterior cruciate ligament tears using customized CNN and YOLOv9. Front. Radiol., 5.","DOI":"10.3389\/fradi.2025.1691048"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1186\/s41747-019-0139-9","article-title":"Artificial intelligence in bone age assessment: Accuracy and efficiency of a novel fully automated algorithm compared to the Greulich-Pyle method","volume":"4","author":"Booz","year":"2020","journal-title":"Eur. Radiol. Exp."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"103138","DOI":"10.1016\/j.artmed.2025.103138","article-title":"An AI system for continuous knee osteoarthritis severity grading: An anomaly detection inspired approach with few labels","volume":"167","author":"Belton","year":"2025","journal-title":"Artif. Intell. Med."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Anggriani, D., Mustamin, S.B., Atnang, M., and Nuzry, K.A.P. (2025). High consistency, limited accuracy: Evaluating large language models for binary medical diagnosis. medRxiv.","DOI":"10.64898\/2025.12.08.25341823"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Shyr, C., Ren, B., Hsu, C.-Y., Tinker, R.J., Cassini, T.A., Hamid, R., Wright, A., Bastarache, L., Peterson, J.F., and Malin, B.A. (2025). A statistical framework for evaluating the repeatability and reproducibility of large language models. medRxiv.","DOI":"10.1101\/2025.08.06.25333170"},{"key":"ref_20","first-page":"e86654","article-title":"Diagnostic performance of ChatGPT-4o in detecting hip fractures on pelvic X-rays","volume":"17","author":"Erdem","year":"2025","journal-title":"Cureus"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e27","DOI":"10.1097\/BOT.0000000000003079","article-title":"Assessing interrater reliability of ChatGPT-4 and orthopaedic clinicians in radiographic fracture classification","volume":"40","author":"Walker","year":"2026","journal-title":"J. Orthop. Trauma"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"987","DOI":"10.14744\/tjtes.2025.98680","article-title":"Diagnostic capabilities of large language models in the detection of scaphoid fractures in the emergency department","volume":"31","author":"Bulut","year":"2025","journal-title":"Turk. J. Trauma Emerg. Surg."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1177\/15589447241298982","article-title":"Is artificial intelligence the future of radiology? Accuracy of ChatGPT in radiologic diagnosis of upper extremity bony pathology","volume":"21","author":"Hiredesai","year":"2024","journal-title":"Hand"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"e200029","DOI":"10.1148\/ryai.2020200029","article-title":"Checklist for Artificial Intelligence in Medical Imaging (CLAIM): A guide for authors and reviewers","volume":"2","author":"Mongan","year":"2020","journal-title":"Radiol. Artif. Intell."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1037\/h0031619","article-title":"Measuring nominal scale agreement among many raters","volume":"76","author":"Fleiss","year":"1971","journal-title":"Psychol. Bull."},{"key":"ref_26","unstructured":"(2026, March 02). Radiopaedia.org. License. Available online: https:\/\/radiopaedia.org\/licence."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1016\/0895-4356(90)90158-L","article-title":"High agreement but low kappa: I. The problems of two paradoxes","volume":"43","author":"Feinstein","year":"1990","journal-title":"J. Clin. Epidemiol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"287","DOI":"10.3758\/s13428-025-02746-8","article-title":"Measuring agreement among several raters classifying subjects into one or more (hierarchical) categories: A generalization of Fleiss\u2019 kappa","volume":"57","author":"Moons","year":"2025","journal-title":"Behav. Res. Methods"},{"key":"ref_30","unstructured":"Alvarado Gonzalez, M.A., Bruno Hernandez, M., Pe\u00f1aloza Perez, M.A., Lopez Orozco, B., Cruz Soto, J.T., and Malagon, S. (2025). Do Repetitions Matter? Strengthening Reliability in LLM Evaluations. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"e076954","DOI":"10.1136\/bmjopen-2023-076954","article-title":"AI-Based X-Ray Fracture Analysis of the Distal Radius: Accuracy between Representative Classification, Detection and Segmentation Deep Learning Models for Clinical Practice","volume":"14","author":"Russe","year":"2024","journal-title":"BMJ Open"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1007\/s10916-024-02105-8","article-title":"Comparison of Vision Transformers and Convolutional Neural Networks in Medical Image Analysis: A Systematic Review","volume":"48","author":"Takahashi","year":"2024","journal-title":"J. Med. Syst."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"108","DOI":"10.52312\/jdrs.2023.856","article-title":"Is tendinitis in volar plating related to the dorsally protruding screw length and its compartment?","volume":"34","author":"Alic","year":"2023","journal-title":"Jt. Dis. Relat. Surg."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Greulich, W.W., and Pyle, S.I. (1959). Radiographic Atlas of Skeletal Development of the Hand and Wrist, Stanford University Press. [2nd ed.].","DOI":"10.1097\/00000441-195909000-00030"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/8\/1171\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,15]],"date-time":"2026-04-15T10:44:12Z","timestamp":1776249852000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/8\/1171"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,15]]},"references-count":34,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2026,4]]}},"alternative-id":["diagnostics16081171"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16081171","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,15]]}}}