{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T19:14:58Z","timestamp":1792782898904,"version":"4.5.0"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","license":[{"start":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T00:00:00Z","timestamp":1789689600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T00:00:00Z","timestamp":1789689600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Technische Universit\u00e4t M\u00fcnchen","id":[{"id":"https:\/\/ror.org\/02kkvpp62","id-type":"ROR","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Digit Imaging. Inform. med."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Ankle fractures are among the most common injuries in trauma surgery and require accurate classification, consistent documentation, and individualized management. Large language models (LLMs) offer the potential to translate unstructured clinical text reports into structured, management-related information, yet their role in orthopedic trauma workflows remains insufficiently defined. In this retrospective study, the performance of Llama 3.1 (70B Instruct) was evaluated using routine radiology reports and clinical documentation from 54 patients with acute ankle fractures. Outputs were compared with reference standards derived from routine clinical documentation for Weber fracture classification, operative versus nonoperative management, and surgical procedure category. Four prompting strategies were systematically assessed. Accuracy for Weber classification ranged from 0.759 to 0.815, well above majority-class baseline (0.556), with strong performance for Weber B fractures and most errors occurring between adjacent categories. Macro-averaged\n                    <jats:italic>F<\/jats:italic>\n                    1 across the three Weber classes ranged from 0.744 to 0.822. Operative management prediction reached accuracies of 0.833 to 0.963 (sensitivity 0.872 to 1.000, specificity 0.571 to 0.714). Procedure category prediction demonstrated lower accuracy (0.426 to 0.610 depending on the endpoint), largely at or below a majority-class baseline (0.553), reflecting the limitations of text-only input for operative planning. These findings suggest that LLMs can extract structured information from routine clinical documentation, performing well for standardized classification but less reliably for procedure-level prediction. Prospective and multimodal validation is needed before clinical implementation.\n                  <\/jats:p>","DOI":"10.1007\/s10278-026-02311-5","type":"journal-article","created":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T15:57:24Z","timestamp":1789747044000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Large Language Models for Ankle Fracture Classification and Management Prediction from Routine Clinical Documentation: A Single-Center Exploratory Study"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-7753-2784","authenticated-orcid":false,"given":"Benjamin","family":"Schwarberg","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Conrad","family":"Ketzer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bosse","family":"Thiel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andreas","family":"Sauter","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marcus R.","family":"Makowski","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Spitzl","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus","family":"Mergen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Florian T.","family":"Gassert","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,9,18]]},"reference":[{"key":"2311_CR1","first-page":"377","volume":"111","author":"H Goost","year":"2014","unstructured":"1.Goost H, Wimmer MD, Barg A, Kabir K, Valderrabano V, Burger C: Fractures of the ankle joint: investigation and treatment options. Dtsch Arztebl Int 111:377-388, 2014","journal-title":"Dtsch Arztebl Int"},{"key":"2311_CR2","doi-asserted-by":"publisher","first-page":"441","DOI":"10.1186\/s12891-018-2326-x","volume":"19","author":"H Juto","year":"2018","unstructured":"2.Juto H, Nilsson H, Morberg P: Epidemiology of Adult Ankle Fractures: 1756 cases identified in Norrbotten County during 2009-2013 and classified according to AO\/OTA. BMC Musculoskelet Disord 19:441, 2018","journal-title":"BMC Musculoskelet Disord"},{"key":"2311_CR3","doi-asserted-by":"publisher","DOI":"10.3389\/fsurg.2022.853101","volume":"9","author":"J Zeng","year":"2022","unstructured":"3.Zeng J, Xu C, Xu G, Wang D, Zhang W, Li H, Gan X, Xiong Y, Li J, Zhang L, Tang P: The Global Status of Research in Ankle Fracture: A Bibliometric and Visualized Study. Front Surg 9:853101, 2022","journal-title":"Front Surg"},{"key":"2311_CR4","doi-asserted-by":"crossref","first-page":"372","DOI":"10.1055\/s-0040-1718508","volume":"56","author":"LS Ramos","year":"2021","unstructured":"4.Ramos LS, Gon\u00e7alves HM, Freitas A, Oliveira MP, Lima DMS, Carmargo WS: Evaluation of the Reproducibility of Lauge-Hansen, Danis-Weber, and AO Classifications for Ankle Fractures. Rev Bras Ortop (Sao Paulo) 56:372-378, 2021","journal-title":"Rev Bras Ortop (Sao Paulo)"},{"key":"2311_CR5","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1016\/S0020-1383(98)00116-8","volume":"29","author":"JG Kennedy","year":"1998","unstructured":"5.Kennedy JG, Johnson SM, Collins AL, DalloVedova P, McManus WF, Hynes DM, Walsh MG, Stephens MM: An evaluation of the Weber classification of ankle fractures. Injury 29:577-580, 1998","journal-title":"Injury"},{"issue":"Suppl 1","key":"2311_CR6","doi-asserted-by":"publisher","first-page":"S1","DOI":"10.1097\/BOT.0000000000001063","volume":"32","author":"EG Meinberg","year":"2018","unstructured":"6.Meinberg EG, Agel J, Roberts CS, Karam MD, Kellam JF: Fracture and Dislocation Classification Compendium-2018. J Orthop Trauma 32 Suppl 1:S1-S170, 2018","journal-title":"J Orthop Trauma"},{"key":"2311_CR7","first-page":"427","volume":"31","author":"H Kyriacou","year":"2021","unstructured":"7.Kyriacou H, Mostafa A, Davies BM, Khan WS: Principles and guidelines in the management of ankle fractures in adults. J Perioper Pract 31:427-434, 2021","journal-title":"J Perioper Pract"},{"key":"2311_CR8","volume":"17","author":"DI Koshy","year":"2025","unstructured":"8.Koshy DI, Koshy D, Ishaku Z: Assessment and Decision-Making in Complex Limb Fracture Management. Cureus 17:e83877, 2025","journal-title":"Cureus"},{"key":"2311_CR9","doi-asserted-by":"publisher","DOI":"10.1016\/j.injury.2023.111037","volume":"55","author":"TD Stringfellow","year":"2024","unstructured":"9.Stringfellow TD, Coffey D, Wek C, Bretherton C, Tan SP, Reichert I, Ahluwalia R: Epidemiology and management of complex ankle fractures in the United Kingdom: A multicentre cohort study. Injury 55:111037, 2024","journal-title":"Injury"},{"key":"2311_CR10","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1016\/j.artmed.2015.09.007","volume":"66","author":"S Hassanpour","year":"2016","unstructured":"10.Hassanpour S, Langlotz CP: Information extraction from multi-institutional radiology reports. Artif Intell Med 66:29-39, 2016","journal-title":"Artif Intell Med"},{"key":"2311_CR11","doi-asserted-by":"publisher","first-page":"59","DOI":"10.1007\/s10278-015-9823-3","volume":"29","author":"S Hassanpour","year":"2016","unstructured":"11.Hassanpour S, Langlotz CP: Unsupervised Topic Modeling in a Large Free Text Radiology Report Repository. J Digit Imaging 29:59-62, 2016","journal-title":"J Digit Imaging"},{"key":"2311_CR12","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2021.106304","volume":"208","author":"AW Olthof","year":"2021","unstructured":"12.Olthof AW, Shouche P, Fennema EM, FFA I, Koolstra RHC, Stirler VMA, van Ooijen PMA, Cornelissen LJ: Machine learning based natural language processing of radiology reports in orthopaedic trauma. Comput Methods Programs Biomed 208:106304, 2021","journal-title":"Comput Methods Programs Biomed"},{"key":"2311_CR13","volume":"17","author":"J Bartkowski","year":"2025","unstructured":"13.Bartkowski J, Zerdka J, Brasse P, Piszka M, Kwapien E, Staszkiewicz K, Kubicka M, Staszkiewicz KK, Czarnecki F: Artificial Intelligence in Medicine With Emphasis on Orthopedic Practice. Cureus 17:e98306, 2025","journal-title":"Cureus"},{"key":"2311_CR14","volume":"17","author":"A Mohamed","year":"2025","unstructured":"14.Mohamed A, Elasad A, Fuad U, Pengas I, Elsayed A, Bhamidipati P, Salib P: Artificial Intelligence in Trauma and Orthopaedic Surgery: A Comprehensive Review From Diagnosis to Rehabilitation. Cureus 17:e92280, 2025","journal-title":"Cureus"},{"key":"2311_CR15","doi-asserted-by":"publisher","DOI":"10.2196\/71916","volume":"27","author":"H Li","year":"2025","unstructured":"15.Li H, Fu JF, Python A: Implementing Large Language Models in Health Care: Clinician-Focused Review With Interactive Guideline. J Med Internet Res 27:e71916, 2025","journal-title":"J Med Internet Res"},{"key":"2311_CR16","doi-asserted-by":"crossref","unstructured":"Maity S, Saikia MJ: Large Language Models in Healthcare and Medical Applications: A Review. Bioengineering (Basel) 12, 2025","DOI":"10.3390\/bioengineering12060631"},{"key":"2311_CR17","doi-asserted-by":"publisher","DOI":"10.2196\/55318","volume":"12","author":"S Sivarajkumar","year":"2024","unstructured":"17.Sivarajkumar S, Kelley M, Samolyk-Mazzanti A, Visweswaran S, Wang Y: An Empirical Evaluation of Prompting Strategies for Large Language Models in Zero-Shot Clinical Natural Language Processing: Algorithm Development and Validation Study. JMIR Med Inform 12:e55318, 2024","journal-title":"JMIR Med Inform"},{"key":"2311_CR18","doi-asserted-by":"publisher","DOI":"10.2196\/60501","volume":"26","author":"J Zaghir","year":"2024","unstructured":"18.Zaghir J, Naguib M, Bjelogrlic M, N\u00e9v\u00e9ol A, Tannier X, Lovis C: Prompt Engineering Paradigms for Medical Applications: Scoping Review. J Med Internet Res 26:e60501, 2024","journal-title":"J Med Internet Res"},{"key":"2311_CR19","doi-asserted-by":"crossref","unstructured":"Ergino\u011flu SE, \u00dclgen NK, Yi\u011fit N, Nazl\u0131g\u00fcl AS, Akkurt MO: Multimodal Large Language Model for Fracture Detection in Emergency Orthopedic Trauma: A Diagnostic Accuracy Study. Diagnostics (Basel) 16, 2026","DOI":"10.3390\/diagnostics16030476"},{"key":"2311_CR20","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2024.103027","volume":"159","author":"J Liu","year":"2025","unstructured":"20.Liu J, Koopman B, Brown NJ, Chu K, Nguyen A: Generating synthetic clinical text with local large language models to identify misdiagnosed limb fractures in radiology reports. Artif Intell Med 159:103027, 2025","journal-title":"Artif Intell Med"},{"key":"2311_CR21","doi-asserted-by":"publisher","DOI":"10.2196\/72153","volume":"4","author":"V Kocaman","year":"2025","unstructured":"21.Kocaman V, Kaya MA, Feier AM, Talby D: Clinical Large Language Model Evaluation by Expert Review (CLEVER): Framework Development and Validation. JMIR AI 4:e72153, 2025","journal-title":"JMIR AI"},{"key":"2311_CR22","doi-asserted-by":"crossref","unstructured":"Mergen M, Spitzl D, Ketzer C, Strenzke M, Marka AW, Makowski MR, Bressem KK, Adams LC, Gassert FT: Leveraging Large Language Models for Accurate AO Fracture Classification from CT Text Reports. J Imaging Inform Med, 2025","DOI":"10.1007\/s10278-025-01603-6"},{"key":"2311_CR23","doi-asserted-by":"crossref","unstructured":"McKinney W: Data structures for statistical computing in Python. Proceedings of the 9th Python in Science Conference 445:51\u201361, 2010","DOI":"10.25080\/Majora-92bf1922-00a"},{"key":"2311_CR24","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"24.Pedregosa F, Varoquaux G, Gramfort A: Scikit-learn: Machine learning in Python. Journal of Machine Learning Research 12:2825\u20132830, 2011","journal-title":"Journal of Machine Learning Research"},{"key":"2311_CR25","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1109\/MCSE.2007.55","volume":"9","author":"J Hunter","year":"2007","unstructured":"25.Hunter J: Matplotlib: A 2D graphics environment. Computing in Science & Engineering 9:90-95, 2007","journal-title":"Computing in Science & Engineering"},{"key":"2311_CR26","doi-asserted-by":"crossref","unstructured":"Waskom M: seaborn: Statistical data visualization. Journal of Open Source Software 6, 2021","DOI":"10.21105\/joss.03021"},{"key":"2311_CR27","doi-asserted-by":"crossref","unstructured":"Seabold S, Perktold J: Statsmodels: Econometric and statistical modeling with Python. In Proceedings of the 9th Python in Science Conference, 2010","DOI":"10.25080\/Majora-92bf1922-011"},{"key":"2311_CR28","first-page":"2024","volume":"509\u2013514","author":"N Bhagat","year":"2024","unstructured":"28.Bhagat N, Mackey O, Wilcox A: Large Language Models for Efficient Medical Information Extraction. AMIA Jt Summits Transl Sci Proc 2024:509-514, 2024","journal-title":"AMIA Jt Summits Transl Sci Proc"},{"key":"2311_CR29","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2024.102900","volume":"154","author":"S Nerella","year":"2024","unstructured":"29.Nerella S, Bandyopadhyay S, Zhang J, Contreras M, Siegel S, Bumin A, Silva B, Sena J, Shickel B, Bihorac A, Khezeli K, Rashidi P: Transformers and large language models in healthcare: A review. Artif Intell Med 154:102900, 2024","journal-title":"Artif Intell Med"},{"key":"2311_CR30","doi-asserted-by":"crossref","unstructured":"Landman R, Healey SP, Loprinzo V, Kochendoerfer U, Winnier AR, Henstock PV, Lin W, Chen A, Rajendran A, Penshanwar S, Khan S, Madhavan S: Using large language models for safety-related table summarization in clinical study reports. JAMIA Open 7:ooae043, 2024","DOI":"10.1093\/jamiaopen\/ooae043"},{"key":"2311_CR31","doi-asserted-by":"publisher","first-page":"263","DOI":"10.1038\/s41746-025-01684-1","volume":"8","author":"F Gaber","year":"2025","unstructured":"31.Gaber F, Shaik M, Allega F, Bilecz AJ, Busch F, Goon K, Franke V, Akalin A: Evaluating large language model workflows in clinical decision support for triage and referral and diagnosis. NPJ Digit Med 8:263, 2025","journal-title":"NPJ Digit Med"},{"key":"2311_CR32","doi-asserted-by":"publisher","first-page":"2613","DOI":"10.1038\/s41591-024-03097-1","volume":"30","author":"P Hager","year":"2024","unstructured":"32.Hager P, Jungmann F, Holland R, Bhagat K, Hubrecht I, Knauer M, Vielhauer J, Makowski M, Braren R, Kaissis G, Rueckert D: Evaluation and mitigation of the limitations of large language models in clinical decision-making. Nat Med 30:2613-2622, 2024","journal-title":"Nat Med"},{"key":"2311_CR33","doi-asserted-by":"publisher","first-page":"230","DOI":"10.1186\/s13054-025-05468-7","volume":"29","author":"X Wu","year":"2025","unstructured":"33.Wu X, Huang Y, He Q: A large language model improves clinicians\u2019 diagnostic performance in complex critical illness cases. Crit Care 29:230, 2025","journal-title":"Crit Care"},{"key":"2311_CR34","volume":"17","author":"Elkohail A, Soffar A, Khalifa AM, Omar I, Mosaad M, Abdulaziz M, Elsaket A, Panhwer HS, Abdelglil M, Teama M, Swealem A","year":"2025","unstructured":"34.Elkohail A, Soffar A, Khalifa AM, Omar I, Mosaad M, Abdulaziz M, Elsaket A, Panhwer HS, Abdelglil M, Teama M, Swealem A: AI-Enhanced Surgical Decision-Making in Orthopedics: From Preoperative Planning to Intraoperative Guidance and Real-Time Adaptation. Cureus 17:e92762, 2025","journal-title":"Cureus"},{"key":"2311_CR35","doi-asserted-by":"crossref","unstructured":"Zheng Z, Ryu BY, Kim SE, Song DS, Kim SH, Park JW, Ro DH: Deep learning for automated hip fracture detection and classification: achieving superior accuracy. Bone Joint J 107-b:213\u2013220, 2025","DOI":"10.1302\/0301-620X.107B2.BJJ-2024-0791.R1"},{"key":"2311_CR36","doi-asserted-by":"publisher","first-page":"521","DOI":"10.2340\/17453674.2025.44248","volume":"96","author":"H Dijkstra","year":"2025","unstructured":"36.Dijkstra H, Parsons CS, Bremen HV, Willems HC, De Hond AAH, Van Munster BC, Doornberg JN, Oosterhoff JHF: Machine learning-based prediction of short- and long-term mortality for shared decision-making in older hip fracture patients: the Dutch Hip Fracture Audit algorithms in 74,396 cases. Acta Orthop 96:521-528, 2025","journal-title":"Acta Orthop"},{"key":"2311_CR37","doi-asserted-by":"publisher","first-page":"526","DOI":"10.1080\/17453674.2021.1932928","volume":"92","author":"PT Ogink","year":"2021","unstructured":"37.Ogink PT, Groot OQ, Karhade AV, Bongers MER, Oner FC, Verlaan JJ, Schwab JH: Wide range of applications for machine-learning prediction models in orthopedic surgical outcome: a systematic review. Acta Orthop 92:526-531, 2021","journal-title":"Acta Orthop"}],"container-title":["Journal of Imaging Informatics in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-026-02311-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10278-026-02311-5","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-026-02311-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,18]],"date-time":"2026-09-18T15:57:30Z","timestamp":1789747050000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10278-026-02311-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,18]]},"references-count":37,"alternative-id":["2311"],"URL":"https:\/\/doi.org\/10.1007\/s10278-026-02311-5","relation":{},"ISSN":["2948-2933"],"issn-type":[{"value":"2948-2933","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,18]]},"assertion":[{"value":"28 May 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 September 2026","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 September 2026","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 September 2026","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"This retrospective study was approved by the Ethics Committee of the Technical University of Munich (approval number 2024\u2013590-S-CB). All data were fully deidentified before analysis. The requirement for individual informed consent was waived by the ethics committee due to the retrospective design and the use of anonymized clinical data.","order":1,"name":"Ethics","label":"Ethics Approval and Consent to Participate","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","label":"Consent for Publication","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":3,"name":"Ethics","label":"Competing interests","group":{"name":"EthicsHeading","label":"Declarations"}}]}}