{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T20:39:47Z","timestamp":1772138387079,"version":"3.50.1"},"reference-count":47,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2024,4,8]],"date-time":"2024-04-08T00:00:00Z","timestamp":1712534400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000026","name":"National Institute on Drug Abuse","doi-asserted-by":"publisher","award":["R01DA051464"],"award-info":[{"award-number":["R01DA051464"]}],"id":[{"id":"10.13039\/100000026","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000057","name":"National Institute of General Medical Sciences","doi-asserted-by":"publisher","award":["R01HL157262"],"award-info":[{"award-number":["R01HL157262"]}],"id":[{"id":"10.13039\/100000057","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["K23HL157832"],"award-info":[{"award-number":["K23HL157832"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["R01LM010090"],"award-info":[{"award-number":["R01LM010090"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"National Library of Medicine","doi-asserted-by":"publisher","award":["R01LM012793"],"award-info":[{"award-number":["R01LM012793"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008028","name":"University of Wisconsin School of Medicine and Public Health","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100008028","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Wisconsin Partnership Program"},{"name":"Research Design Support: the Protocol Development, Informatics, and Biostatistics Module"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,5,20]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>The timely stratification of trauma injury severity can enhance the quality of trauma care but it requires intense manual annotation from certified trauma coders. The objective of this study is to develop machine learning models for the stratification of trauma injury severity across various body regions using clinical text and structured electronic health records (EHRs) data.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Materials and Methods<\/jats:title>\n                    <jats:p>Our study utilized clinical documents and structured EHR variables linked with the trauma registry data to create 2 machine learning models with different approaches to representing text. The first one fuses concept unique identifiers (CUIs) extracted from free text with structured EHR variables, while the second one integrates free text with structured EHR variables. Temporal validation was undertaken to ensure the models\u2019 temporal generalizability. Additionally, analyses to assess the variable importance were conducted.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Both models demonstrated impressive performance in categorizing leg injuries, achieving high accuracy with macro-F1 scores of over 0.8. Additionally, they showed considerable accuracy, with macro-F1 scores exceeding or near 0.7, in assessing injuries in the areas of the chest and head. We showed in our variable importance analysis that the most important features in the model have strong face validity in determining clinically relevant trauma injuries.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Discussion<\/jats:title>\n                    <jats:p>The CUI-based model achieves comparable performance, if not higher, compared to the free-text-based model, with reduced complexity. Furthermore, integrating structured EHR data improves performance, particularly when the text modalities are insufficiently indicative.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Our multi-modal, multiclass models can provide accurate stratification of trauma injury severity and clinically relevant interpretations.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/jamia\/ocae071","type":"journal-article","created":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T11:01:46Z","timestamp":1711018906000},"page":"1291-1302","source":"Crossref","is-referenced-by-count":4,"title":["Automated stratification of trauma injury severity across multiple body regions using multi-modal, multi-class machine learning models"],"prefix":"10.1093","volume":"31","author":[{"given":"Jifan","family":"Gao","sequence":"first","affiliation":[{"name":"Department of Biostatistics and Medical Informatics, University of Wisconsin\u2013Madison , Madison, WI 53726, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guanhua","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Medical Informatics, University of Wisconsin\u2013Madison , Madison, WI 53726, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ann P","family":"O\u2019Rourke","sequence":"additional","affiliation":[{"name":"Department of Surgery, University of Wisconsin\u2013Madison , Madison, WI 53792, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John","family":"Caskey","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Wisconsin\u2013Madison , Madison, WI 53705, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kyle A","family":"Carey","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Wisconsin\u2013Madison , Madison, WI 53705, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Madeline","family":"Oguss","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Wisconsin\u2013Madison , Madison, WI 53705, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anne","family":"Stey","sequence":"additional","affiliation":[{"name":"Department of Surgery, Northwestern University Feinberg School of Medicine , Chicago, IL 60611, United States"},{"name":"Center of Health Services and Outcomes Research, Institute for Public Health and Medicine , Chicago, IL 60611, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dmitriy","family":"Dligach","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Loyola University Chicago , Chicago, IL 60660, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4513-403X","authenticated-orcid":false,"given":"Timothy","family":"Miller","sequence":"additional","affiliation":[{"name":"Computational Health Informatics Program, Boston Children\u2019s Hospital , Boston, MA 02115, United States"},{"name":"Department of Pediatrics, Harvard Medical School , Boston, MA 02115, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anoop","family":"Mayampurath","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Medical Informatics, University of Wisconsin\u2013Madison , Madison, WI 53726, United States"},{"name":"Department of Medicine, University of Wisconsin\u2013Madison , Madison, WI 53705, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthew M","family":"Churpek","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Medical Informatics, University of Wisconsin\u2013Madison , Madison, WI 53726, United States"},{"name":"Department of Medicine, University of Wisconsin\u2013Madison , Madison, WI 53705, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Majid","family":"Afshar","sequence":"additional","affiliation":[{"name":"Department of Biostatistics and Medical Informatics, University of Wisconsin\u2013Madison , Madison, WI 53726, United States"},{"name":"Department of Medicine, University of Wisconsin\u2013Madison , Madison, WI 53705, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2024,4,8]]},"reference":[{"issue":"7","key":"2024052019461409900_ocae071-B1","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1056\/NEJMp2205162","article-title":"From survival to survivorship\u2014framing traumatic injury as a chronic condition","volume":"387","author":"Herrera-Escobar","year":"2022","journal-title":"N Engl J Med"},{"issue":"1","key":"2024052019461409900_ocae071-B2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s00068-011-0168-4","article-title":"Epidemiology of in-hospital trauma deaths","volume":"38","author":"Lefering","year":"2012","journal-title":"Eur J Trauma Emerg Surg"},{"issue":"1","key":"2024052019461409900_ocae071-B3","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1097\/MEJ.0000000000000397","article-title":"A criteria-directed protocol for in-hospital triage of trauma patients","volume":"25","author":"Granstr\u00f6m","year":"2018","journal-title":"Eur J Emerg Med"},{"key":"2024052019461409900_ocae071-B4","author":"Mock","year":"2009"},{"issue":"1","key":"2024052019461409900_ocae071-B5","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1016\/j.injury.2015.07.003","article-title":"Defining major trauma using the 2008 abbreviated injury scale","volume":"47","author":"Palmer","year":"2016","journal-title":"Injury"},{"issue":"1","key":"2024052019461409900_ocae071-B6","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1007\/BF01655906","article-title":"Trauma severity scoring to predict mortality","volume":"7","author":"Champion","year":"1983","journal-title":"World J Surg"},{"issue":"1","key":"2024052019461409900_ocae071-B7","doi-asserted-by":"crossref","first-page":"ooab015","DOI":"10.1093\/jamiaopen\/ooab015","article-title":"Comparison and interpretability of machine learning models to predict severity of chest injury","volume":"4","author":"Kulshrestha","year":"2021","journal-title":"JAMIA Open"},{"issue":"3","key":"2024052019461409900_ocae071-B8","doi-asserted-by":"crossref","first-page":"233","DOI":"10.1097\/00000433-199609000-00011","article-title":"The abbreviated injury scale: a valuable tool for forensic documentation of trauma","volume":"17","author":"Friedman","year":"1996","journal-title":"Am J Forensic Med Pathol"},{"issue":"12","key":"2024052019461409900_ocae071-B9","doi-asserted-by":"crossref","first-page":"1515","DOI":"10.1001\/jama.289.12.1515","article-title":"National inventory of hospital trauma centers","volume":"289","author":"MacKenzie","year":"2003","journal-title":"JAMA"},{"issue":"7","key":"2024052019461409900_ocae071-B10","doi-asserted-by":"crossref","first-page":"1292","DOI":"10.1093\/jamia\/ocac058","article-title":"Use of unstructured text in prognostic clinical prediction models: a systematic review","volume":"29","author":"Seinen","year":"2022","journal-title":"J Am Med Inform Assoc"},{"key":"2024052019461409900_ocae071-B11","author":"Devlin","year":"2018"},{"issue":"1","key":"2024052019461409900_ocae071-B12","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1038\/s41746-021-00455-y","article-title":"Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction","volume":"4","author":"Rasmy","year":"2021","journal-title":"NPJ Digit Med"},{"key":"2024052019461409900_ocae071-B13","author":"Alsentzer","year":"2019"},{"issue":"4","key":"2024052019461409900_ocae071-B14","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"2024052019461409900_ocae071-B15","first-page":"199","author":"Pawar","year":"2022"},{"key":"2024052019461409900_ocae071-B16","author":"He","year":"2020"},{"key":"2024052019461409900_ocae071-B17","doi-asserted-by":"crossref","first-page":"104363","DOI":"10.1016\/j.jbi.2023.104363","article-title":"A hybrid system to understand the relations between assessments and plans in progress notes","volume":"141","author":"Gao","year":"2023","journal-title":"J Biomed Inform"},{"key":"2024052019461409900_ocae071-B18","first-page":"5838","author":"Alambo","year":"2020"},{"key":"2024052019461409900_ocae071-B19","first-page":"138","article-title":"Natural language processing methods to identify oncology patients at high risk for acute care with clinical notes","volume":"2023","author":"Fanconi","year":"2023","journal-title":"AMIA Summits Transl Sci Proc"},{"key":"2024052019461409900_ocae071-B20","author":"Islam","year":"2023"},{"issue":"11","key":"2024052019461409900_ocae071-B21","doi-asserted-by":"crossref","first-page":"1364","DOI":"10.1093\/jamia\/ocz068","article-title":"Development and application of a high throughput natural language processing architecture to convert all clinical documents in a clinical data warehouse into standardized medical vocabularies","volume":"26","author":"Afshar","year":"2019","journal-title":"J Am Med Inform Assoc"},{"key":"2024052019461409900_ocae071-B22","doi-asserted-by":"crossref","first-page":"e44977","DOI":"10.2196\/44977","article-title":"Deployment of real-time natural language processing and deep learning clinical decision support in the electronic health record: pipeline implementation for an opioid misuse screener in hospitalized adults","volume":"11","author":"Afshar","year":"2023","journal-title":"JMIR Med Inform"},{"key":"2024052019461409900_ocae071-B23","first-page":"1497","author":"Gong","year":"2017"},{"issue":"Suppl 11","key":"2024052019461409900_ocae071-B24","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1186\/s12911-020-01318-4","article-title":"Predicting mortality in critically ill patients with diabetes using machine learning and clinical notes","volume":"20","author":"Ye","year":"2020","journal-title":"BMC Med Inform Decis Mak"},{"key":"2024052019461409900_ocae071-B25","first-page":"426","author":"Liang","year":"2022"},{"issue":"5","key":"2024052019461409900_ocae071-B26","doi-asserted-by":"crossref","first-page":"440","DOI":"10.1016\/j.annemergmed.2022.05.011","article-title":"Machine learning in the prediction of trauma outcomes: a systematic review","volume":"80","author":"Zhang","year":"2022","journal-title":"Ann Emerg Med"},{"issue":"5","key":"2024052019461409900_ocae071-B27","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical text analysis and knowledge extraction system (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","year":"2010","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2024052019461409900_ocae071-B28","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1186\/s40537-021-00444-8","article-title":"Review of deep learning: concepts, CNN architectures, challenges, applications, future directions","volume":"8","author":"Alzubaidi","year":"2021","journal-title":"J Big Data"},{"key":"2024052019461409900_ocae071-B29","author":"Peng","year":"2019"},{"issue":"1","key":"2024052019461409900_ocae071-B30","doi-asserted-by":"crossref","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"issue":"1","key":"2024052019461409900_ocae071-B31","first-page":"1","article-title":"Domain-specific language model pretraining for biomedical natural language processing","volume":"3","author":"Gu","year":"2021","journal-title":"ACM Trans Comput Healthc (HEALTH)"},{"issue":"4","key":"2024052019461409900_ocae071-B32","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","article-title":"BioBERT: a pre-trained biomedical language representation model for biomedical text mining","volume":"36","author":"Lee","year":"2020","journal-title":"Bioinformatics"},{"key":"2024052019461409900_ocae071-B33","author":"Beltagy","year":"2019"},{"issue":"1","key":"2024052019461409900_ocae071-B34","first-page":"1","article-title":"Domain-specific language model pretraining for biomedical natural language processing","volume":"3","author":"Gu","year":"2021","journal-title":"ACM Trans Comput Healthc (HEALTH)"},{"key":"2024052019461409900_ocae071-B35","first-page":"3","author":"Turner","year":"2021"},{"key":"2024052019461409900_ocae071-B36","first-page":"3319","author":"Sundararajan","year":"2017"},{"key":"2024052019461409900_ocae071-B37","author":"Bergquist","year":"2021"},{"issue":"430","key":"2024052019461409900_ocae071-B38","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1080\/01621459.1995.10476572","article-title":"Bayes factors","volume":"90","author":"Kass","year":"1995","journal-title":"J Am Stat Assoc"},{"key":"2024052019461409900_ocae071-B39","first-page":"4768","article-title":"A unified approach to interpreting model predictions","volume":"30","author":"Lundberg","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"2024052019461409900_ocae071-B40","first-page":"3145","author":"Shrikumar","year":"2017"},{"key":"2024052019461409900_ocae071-B41","first-page":"63","author":"Binder","year":"2016"},{"issue":"Database issue","key":"2024052019461409900_ocae071-B42","doi-asserted-by":"crossref","first-page":"D267","DOI":"10.1093\/nar\/gkh061","article-title":"The unified medical language system (UMLS): integrating biomedical terminology","volume":"32","author":"Bodenreider","year":"2004","journal-title":"Nucleic Acids Res"},{"key":"2024052019461409900_ocae071-B43","first-page":"26","author":"Aronson","year":"2006"},{"issue":"1","key":"2024052019461409900_ocae071-B44","first-page":"1","article-title":"Guidelines for field triage of injured patients: recommendations of the National Expert Panel on Field Triage, 2011","volume":"61","author":"Sasser","year":"2012","journal-title":"Morb Mortal Wkly Rep: Recomm Rep"},{"key":"2024052019461409900_ocae071-B45","author":"Asensio","year":"2008"},{"issue":"3","key":"2024052019461409900_ocae071-B46","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1097\/00005373-197403000-00001","article-title":"The injury severity score: a method for describing patients with multiple injuries and evaluating emergency care","volume":"14","author":"Baker","year":"1974","journal-title":"J Trauma Acute Care Surg"},{"issue":"11","key":"2024052019461409900_ocae071-B47","doi-asserted-by":"crossref","first-page":"2686","DOI":"10.1038\/s41591-023-02540-z","article-title":"External validation of AI models in health should be replaced with recurring local validation","volume":"29","author":"Youssef","year":"2023","journal-title":"Nat Med"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/advance-article-pdf\/doi\/10.1093\/jamia\/ocae071\/57768931\/ocae071.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/advance-article-pdf\/doi\/10.1093\/jamia\/ocae071\/57768931\/ocae071.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,20]],"date-time":"2024-05-20T15:46:56Z","timestamp":1716220016000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/31\/6\/1291\/7642220"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,8]]},"references-count":47,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2024,4,8]]},"published-print":{"date-parts":[[2024,5,20]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocae071","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2024.01.22.24301489","asserted-by":"object"}]},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"value":"1067-5027","type":"print"},{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,6,1]]},"published":{"date-parts":[[2024,4,8]]}}}