{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,20]],"date-time":"2026-10-20T12:27:35Z","timestamp":1792499255856,"version":"4.5.0"},"reference-count":30,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T00:00:00Z","timestamp":1770076800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background: Rapid and accurate fracture detection is critical in emergency departments (EDs), where high patient volume and time pressure increase the risk of diagnostic error, particularly in radiographic interpretation. Multimodal large language models (LLMs) with image-recognition capability have recently emerged as general-purpose tools for clinical decision support, but their diagnostic performance within routine emergency department imaging workflows in orthopedic trauma remains unclear. Methods: In this retrospective diagnostic accuracy study, we included 1136 consecutive patients referred from the ED to orthopedics between 1 January and 1 June 2025 at a single tertiary center. Given the single-center, retrospective design, the findings should be interpreted as hypothesis-generating and may not be fully generalizable to other institutions. Emergency radiographs and clinical data were processed by a multimodal LLM (2025 version) via an official API using a standardized, deterministic prompt. The model\u2019s outputs (\u201cFracture present\u201d, \u201cNo fracture\u201d, or \u201cUncertain\u201d) were compared with final diagnoses established by blinded orthopedic specialists, which served as the reference standard. Diagnostic agreement was analyzed using Cohen\u2019s kappa (\u03ba), sensitivity, specificity, accuracy, and 95% confidence intervals (CIs). False-negative (FN) cases were defined as instances where the LLM reported \u201cno acute fracture\u201d but the specialist identified a fracture. The evaluated system is a general-purpose multimodal LLM and was not trained specifically on orthopedic radiographs. Results: Overall, the LLM showed good diagnostic agreement with orthopedic specialists, with concordant results in 808 of 1136 patients (71.1%; \u03ba = 0.634; 95% CI: 68.4\u201373.7). The model achieved balanced performance with sensitivity of 76.9% and specificity of 66.8%. The highest agreement was observed in knee trauma (91.7%), followed by wrist (78.8%) and hand (69.6%). False-negative cases accounted for 184 patients (16.2% of the total cohort), representing 32.4% of all LLM-negative assessments. Most FN fractures were non-displaced (82.6%), and 17.4% of FN cases required surgical treatment. Ankle and foot regions showed the highest FN rates (30.4% and 17.4%, respectively), reflecting the anatomical and radiographic complexity of these areas. Positive predictive value (PPV) and negative predictive value (NPV) were 69.4% and 74.5%, respectively, with likelihood ratios indicating moderate shifts in post-test probability. Conclusions: In an emergency department-to-orthopedics consultation cohort reflecting routine clinical workflow, a multimodal LLM demonstrated moderate-to-good diagnostic agreement with orthopedic specialists, broadly within the range reported in prior fracture-detection AI studies; however, these comparisons are indirect because model architectures, training strategies, datasets, and endpoints differ across studies. However, its limited ability to detect non-displaced fractures\u2014especially in anatomically complex regions like the ankle and foot\u2014carries direct patient safety implications and confirms that specialist review remains indispensable. At present, such models may be explored as hypothesis-generating triage or decision-support tools, with mandatory specialist confirmation, rather than as standalone diagnostic systems. Prospective, multi-center studies using high-resolution imaging and anatomically optimized algorithms are needed before routine clinical adoption in emergency care.<\/jats:p>","DOI":"10.3390\/diagnostics16030476","type":"journal-article","created":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T13:58:49Z","timestamp":1770127129000},"page":"476","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Multimodal Large Language Model for Fracture Detection in Emergency Orthopedic Trauma: A Diagnostic Accuracy Study"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4146-8318","authenticated-orcid":false,"given":"Sad\u0131k Emre","family":"Ergino\u011flu","sequence":"first","affiliation":[{"name":"Department of Orthopedics and Traumatology, Ankara Sincan Training and Research Hospital, Sincan, Ankara 06935, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0301-3432","authenticated-orcid":false,"given":"Nuri Koray","family":"\u00dclgen","sequence":"additional","affiliation":[{"name":"Department of Orthopedics and Traumatology, Ankara Sincan Training and Research Hospital, Sincan, Ankara 06935, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-8670-5484","authenticated-orcid":false,"given":"Nihat","family":"Yi\u011fit","sequence":"additional","affiliation":[{"name":"Department of Orthopedics and Traumatology, Ankara Sincan Training and Research Hospital, Sincan, Ankara 06935, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3451-3691","authenticated-orcid":false,"given":"Ali Said","family":"Nazl\u0131g\u00fcl","sequence":"additional","affiliation":[{"name":"Department of Orthopedics and Traumatology, Ankara Sincan Training and Research Hospital, Sincan, Ankara 06935, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4935-0143","authenticated-orcid":false,"given":"Mehmet Or\u00e7un","family":"Akkurt","sequence":"additional","affiliation":[{"name":"Department of Orthopedics and Traumatology, Ankara Sincan Training and Research Hospital, Sincan, Ankara 06935, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"691","DOI":"10.1016\/j.injury.2006.04.130","article-title":"Epidemiology of adult fractures: A review","volume":"37","author":"Caesar","year":"2006","journal-title":"Injury"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"911","DOI":"10.1093\/ije\/22.5.911","article-title":"Epidemiology of acute wrist trauma","volume":"22","author":"Larsen","year":"1993","journal-title":"Int. J. Epidemiol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1007\/s13244-016-0534-1","article-title":"Error and discrepancy in radiology: Inevitable or avoidable?","volume":"8","author":"Brady","year":"2017","journal-title":"Insights Into Imaging"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.media.2017.07.005","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens","year":"2017","journal-title":"Med. Image Anal."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1146\/annurev-bioeng-071516-044442","article-title":"Deep Learning in Medical Image Analysis","volume":"19","author":"Shen","year":"2017","journal-title":"Annu. Rev. Biomed. Eng."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"11591","DOI":"10.1073\/pnas.1806905115","article-title":"Deep neural network improves fracture detection by clinicians","volume":"115","author":"Lindsey","year":"2018","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1080\/17453674.2017.1344459","article-title":"Artificial intelligence for analyzing orthopedic trauma radiographs: Deep learning algorithms\u2014Are they on par with humans for diagnosing fractures?","volume":"88","author":"Olczak","year":"2017","journal-title":"Acta Orthop."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1148\/radiol.211785","article-title":"Artificial Intelligence in Fracture Detection: A Systematic Review and Meta-Analysis","volume":"304","author":"Kuo","year":"2022","journal-title":"Radiology"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"105650","DOI":"10.1016\/j.jdent.2025.105650","article-title":"Artificial intelligence for osteoporosis detection on panoramic radiography: A systematic review and meta analysis","volume":"156","author":"Ghasemi","year":"2025","journal-title":"J. Dent."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1080\/17453674.2019.1600125","article-title":"Artificial intelligence detection of distal radius fractures: A comparison between 2D and 3D convolutional neural networks","volume":"90","author":"Gan","year":"2019","journal-title":"Acta Orthop."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1105","DOI":"10.1007\/s11596-021-2474-3","article-title":"Application of Artificial Intelligence in Medicine: An Overview","volume":"41","author":"Liu","year":"2021","journal-title":"Curr. Med. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: The convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat. Med."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1038\/s41591-018-0272-7","article-title":"Privacy in the age of medical big data","volume":"25","author":"Price","year":"2019","journal-title":"Nat. Med."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1727","DOI":"10.1038\/s41598-018-20132-7","article-title":"Automatic Knee Osteoarthritis Diagnosis from Plain Radiographs: A Deep Learning-Based Approach","volume":"8","author":"Tiulpin","year":"2018","journal-title":"Sci. Rep."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"e351","DOI":"10.1016\/S2589-7500(22)00004-8","article-title":"Validation and algorithmic audit of a deep learning system for the detection of proximal femoral fractures in patients in the emergency department: A diagnostic accuracy study","volume":"4","author":"Gale","year":"2022","journal-title":"Lancet Digit. Health"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"107350","DOI":"10.1016\/j.isci.2023.107350","article-title":"Application of a deep learning algorithm in the detection of hip fractures","volume":"26","author":"Gao","year":"2023","journal-title":"iScience"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"439","DOI":"10.1016\/j.crad.2017.11.015","article-title":"Artificial intelligence in fracture detection: Transfer learning from deep convolutional neural networks","volume":"73","author":"Kim","year":"2018","journal-title":"Clin. Radiol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1016\/j.jacr.2017.12.026","article-title":"Artificial intelligence and machine learning in radiology: Opportunities, challenges, pitfalls, and criteria for success","volume":"15","author":"Thrall","year":"2018","journal-title":"J. Am. Coll. Radiol."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kelly, C.J., Karthikesalingam, A., Suleyman, M., Corrado, G., and King, D. (2019). Key challenges for delivering clinical impact with artificial intelligence. BMC Med., 17.","DOI":"10.1186\/s12916-019-1426-2"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Rajpurkar, P., Irvin, J., Ball, R.L., Zhu, K., Yang, B., Mehta, H., Duan, T., Ding, D., Bagul, A., and Langlotz, C.P. (2018). Deep learning for chest radiograph diagnosis: A retrospective comparison of the CheXNeXt algorithm to practicing radiologists. PLoS Med., 15.","DOI":"10.1371\/journal.pmed.1002686"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1038\/s41591-018-0316-z","article-title":"A guide to deep learning in healthcare","volume":"25","author":"Esteva","year":"2019","journal-title":"Nat. Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"e3","DOI":"10.1136\/medethics-2019-105860","article-title":"Primer on an ethics of AI-based decision support systems in the clinic","volume":"47","author":"Braun","year":"2020","journal-title":"J. Med. Ethics"},{"key":"#cr-split#-ref_24.1","unstructured":"European Parliament and the Council of the European Union (2016). Regulation"},{"key":"#cr-split#-ref_24.2","unstructured":"(EU) 2016\/679 (General Data Protection Regulation). Off. J. Eur. Union, L119, 1-88."},{"key":"ref_25","first-page":"111","article-title":"Traumatic fractures in adults: Missed diagnosis on plain radiographs in the Emergency Department","volume":"89","author":"Pinto","year":"2018","journal-title":"Acta Biomed."},{"key":"ref_26","first-page":"e47732","article-title":"Application of Artificial Intelligence and Machine Learning in Diagnosing Scaphoid Fractures: A Systematic Review","volume":"15","author":"Orji","year":"2023","journal-title":"Cureus"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2613","DOI":"10.1038\/s41591-024-03097-1","article-title":"Evaluation and mitigation of the limitations of large language models in clinical decision-making","volume":"30","author":"Hager","year":"2024","journal-title":"Nat. Med."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1186\/s40634-023-00683-z","article-title":"A practical guide to the implementation of AI in orthopaedic research\u2014Part 1: Opportunities in clinical application and overcoming existing challenges","volume":"10","author":"Zsidai","year":"2023","journal-title":"J. Exp. Orthop."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1347","DOI":"10.1056\/NEJMra1814259","article-title":"Machine Learning in Medicine","volume":"380","author":"Rajkomar","year":"2019","journal-title":"N. Engl. J. Med."}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/3\/476\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T14:03:15Z","timestamp":1770127395000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/3\/476"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,3]]},"references-count":30,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2026,2]]}},"alternative-id":["diagnostics16030476"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16030476","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,3]]}}}