{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T05:54:42Z","timestamp":1791352482533,"version":"4.3.1"},"reference-count":21,"publisher":"American Medical Association (AMA)","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JAMA Netw Open"],"abstract":"Importance<\/jats:title>International Classification of Diseases<\/jats:italic>\u2013coded hospital discharge data do not accurately reflect whether firearm injuries were caused by assault, unintentional injury, self-harm, legal intervention, or were of undetermined intent. Applying natural language processing (NLP) and machine learning (ML) techniques to electronic health record (EHR) narrative text could be associated with improved accuracy of firearm injury intent data.<\/jats:p><\/jats:sec>Objective<\/jats:title>To assess the accuracy with which an ML model identified firearm injury intent.<\/jats:p><\/jats:sec>Design, Setting, and Participants<\/jats:title>A cross-sectional retrospective EHR review was conducted at 3 level I trauma centers, 2 from health care institutions in Boston, Massachusetts, and 1 from Seattle, Washington, between January 1, 2000, and December 31, 2019; data analysis was performed from January 18, 2021, to August 22, 2022. A total of 1915 incident cases of firearm injury in patients presenting to emergency departments at the model development institution and 769 from the external validation institution with a firearm injury code assigned according to International Classification of Diseases<\/jats:italic>, Ninth Revision, Clinical Modification<\/jats:italic> (ICD-9-CM<\/jats:italic>) or International Statistical Classification of Diseases and Related Health Problems, 10th Revision, Clinical Modification<\/jats:italic> (ICD-10-CM<\/jats:italic>), in discharge data were included.<\/jats:p><\/jats:sec>Exposures<\/jats:title>Classification of firearm injury intent.<\/jats:p><\/jats:sec>Main Outcomes and Measures<\/jats:title>Intent classification accuracy by the NLP model was compared with ICD<\/jats:italic> codes assigned by medical record coders in discharge data. The NLP model extracted intent-relevant features from narrative text that were then used by a gradient-boosting classifier to determine the intent of each firearm injury. Classification accuracy was evaluated against intent assigned by the research team. The model was further validated using an external data set.<\/jats:p><\/jats:sec>Results<\/jats:title>The NLP model was evaluated in 381 patients presenting with firearm injury at the model development site (mean [SD] age, 39.2 [13.0] years; 348 [91.3%] men) and 304 patients at the external development site (mean [SD] age, 31.8 [14.8] years; 263 [86.5%] men). The model proved more accurate than medical record coders in assigning intent to firearm injuries at the model development site (accident F-score, 0.78 vs 0.40; assault F-score, 0.90 vs 0.78). The model maintained this improvement on an external validation set from a second institution (accident F-score, 0.64 vs 0.58; assault F-score, 0.88 vs 0.81). While the model showed some degradation between institutions, retraining the model using data from the second institution further improved performance on that site\u2019s records (accident F-score, 0.75; assault F-score, 0.92).<\/jats:p><\/jats:sec>Conclusions and Relevance<\/jats:title>The findings of this study suggest that NLP ML can be used to improve the accuracy of firearm injury intent classification compared with ICD<\/jats:italic>-coded discharge data, particularly for cases of accident and assault intents (the most prevalent and commonly misclassified intent types). Future research could refine this model using larger and more diverse data sets.<\/jats:p><\/jats:sec>","DOI":"10.1001\/jamanetworkopen.2023.5870","type":"journal-article","created":{"date-parts":[[2023,4,6]],"date-time":"2023-04-06T15:04:20Z","timestamp":1680793460000},"page":"e235870","source":"Crossref","is-referenced-by-count":17,"title":["Classifying Firearm Injury Intent in Electronic Hospital Records Using Natural Language Processing"],"prefix":"10.1001","volume":"6","author":[{"given":"Erin","family":"MacPhaul","sequence":"first","affiliation":[{"name":"Department of Emergency Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Li","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stephen J.","family":"Mooney","sequence":"additional","affiliation":[{"name":"Firearm Injury & Policy Research Program, University of Washington, Seattle"},{"name":"Department of Epidemiology, School of Public Health, University of Washington, Seattle"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Deborah","family":"Azrael","sequence":"additional","affiliation":[{"name":"Harvard Injury Control Research Center, Harvard T. H. Chan School of Public Health, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Bowen","sequence":"additional","affiliation":[{"name":"Firearm Injury & Policy Research Program, University of Washington, Seattle"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ali","family":"Rowhani-Rahbar","sequence":"additional","affiliation":[{"name":"Firearm Injury & Policy Research Program, University of Washington, Seattle"},{"name":"Department of Epidemiology, School of Public Health, University of Washington, Seattle"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ravali","family":"Yenduri","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts"},{"name":"Center for Surgery and Public Health, Brigham and Women\u2019s Hospital, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Catherine","family":"Barber","sequence":"additional","affiliation":[{"name":"Harvard Injury Control Research Center, Harvard T. H. Chan School of Public Health, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric","family":"Goralnick","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Brigham and Women's Hospital, Harvard Medical School, Boston, Massachusetts"},{"name":"Center for Surgery and Public Health, Brigham and Women\u2019s Hospital, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew","family":"Miller","sequence":"additional","affiliation":[{"name":"Harvard Injury Control Research Center, Harvard T. H. Chan School of Public Health, Boston, Massachusetts"},{"name":"Department of Health Sciences, Bouve College of Health Sciences, Northeastern University, Boston, Massachusetts"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"10","published-online":{"date-parts":[[2023,4,6]]},"reference":[{"issue":"8","key":"zoi230200r3","doi-asserted-by":"publisher","first-page":"1132","DOI":"10.1001\/jamainternmed.2021.0382","article-title":"The problem with ICD-coded firearm injuries.","volume":"181","author":"Barber","year":"2021","journal-title":"JAMA Intern Med"},{"issue":"pt A","key":"zoi230200r4","article-title":"The emerging infrastructure of US firearms injury data.","volume":"165","author":"Barber","year":"2022","journal-title":"Prev Med"},{"issue":"12","key":"zoi230200r5","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2022.46429","article-title":"Assessment of the accuracy of firearm injury intent coding at 3 US hospitals.","volume":"5","author":"Miller","year":"2022","journal-title":"JAMA Netw Open"},{"issue":"12","key":"zoi230200r6","doi-asserted-by":"publisher","first-page":"1177","DOI":"10.1001\/jama.2022.16894","article-title":"The epidemiology of firearm injuries in the US: the need for comprehensive, real-time, actionable data.","volume":"328","author":"Kaufman","year":"2022","journal-title":"JAMA"},{"issue":"2","key":"zoi230200r7","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1001\/jamainternmed.2020.6696","article-title":"Epidemiologic trends in fatal and nonfatal firearm injuries in the US, 2009-2017.","volume":"181","author":"Kaufman","year":"2021","journal-title":"JAMA Intern Med"},{"issue":"4","key":"zoi230200r8","doi-asserted-by":"publisher","DOI":"10.1542\/peds.2020-011957","article-title":"Sociodemographic factors and outcomes by intent of firearm injury.","volume":"147","author":"Patel","year":"2021","journal-title":"Pediatrics"},{"issue":"1","key":"zoi230200r9","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1186\/s40621-017-0120-0","article-title":"Cost of hospitalization for firearm injuries by firearm type, intent, and payer in the United States.","volume":"4","author":"Peek-Asa","year":"2017","journal-title":"Inj Epidemiol"},{"issue":"3","key":"zoi230200r10","doi-asserted-by":"publisher","first-page":"462","DOI":"10.5811\/westjem.2021.5.53399","article-title":"Nonfatal firearm injuries by intent in the United States: 2016-2018 hospital discharge records from the Healthcare Cost and Utilization Project.","volume":"22","author":"Schnippel","year":"2021","journal-title":"West J Emerg Med"},{"issue":"1","key":"zoi230200r11","doi-asserted-by":"publisher","DOI":"10.1136\/tsaco-2017-000139","article-title":"A multi-decade joinpoint analysis of firearm injury severity.","volume":"3","author":"Kalesan","year":"2018","journal-title":"Trauma Surg Acute Care Open"},{"issue":"1","key":"zoi230200r12","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1186\/s40621-018-0168-5","article-title":"The epidemiology of firearm injuries managed in US emergency departments.","volume":"5","author":"Avraham","year":"2018","journal-title":"Inj Epidemiol"},{"issue":"4","key":"zoi230200r13","doi-asserted-by":"publisher","first-page":"360","DOI":"10.1001\/jamasurg.2018.4640","article-title":"US emergency department encounters for firearm injuries according to presentation at trauma vs nontrauma centers.","volume":"154","author":"Coupet","year":"2019","journal-title":"JAMA Surg"},{"issue":"5","key":"zoi230200r14","doi-asserted-by":"publisher","first-page":"1188","DOI":"10.1016\/j.surg.2020.11.009","article-title":"Prevalence and hospital charges from firearm injuries treated in US emergency departments from 2006 to 2016.","volume":"169","author":"Kalesan","year":"2021","journal-title":"Surgery"},{"issue":"3","key":"zoi230200r15","doi-asserted-by":"publisher","first-page":"621","DOI":"10.1016\/j.injury.2017.01.044","article-title":"Gunshot wounds resulting in hospitalization in the United States: 2004-2013.","volume":"48","author":"Cook","year":"2017","journal-title":"Injury"},{"issue":"10","key":"zoi230200r16","doi-asserted-by":"publisher","first-page":"1729","DOI":"10.1377\/hlthaff.2017.0625","article-title":"Emergency department visits for firearm-related injuries in the United States, 2006-14.","volume":"36","author":"Gani","year":"2017","journal-title":"Health Aff (Millwood)"},{"key":"zoi230200r17","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1016\/j.jbi.2017.07.012","article-title":"Natural language processing systems for capturing and standardizing unstructured clinical information: a systematic review.","volume":"73","author":"Kreimeyer","year":"2017","journal-title":"J Biomed Inform"},{"key":"zoi230200r18","first-page":"629","article-title":"Identifying patients with depression using free-text clinical documents.","volume":"216","author":"Zhou","year":"2015","journal-title":"Stud Health Technol Inform"},{"key":"zoi230200r19","first-page":"233","article-title":"Using natural language processing and machine learning to identify hospitalized patients with opioid use disorder.","volume":"2020","author":"Blackley","year":"2021","journal-title":"AMIA Annu Symp Proc"},{"key":"zoi230200r20","first-page":"2825","article-title":"Scikit-learn: machine learning in Python.","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"issue":"10","key":"zoi230200r21","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2020.20664","article-title":"Estimating nonfatal gunshot injury locations with natural language processing and machine learning models.","volume":"3","author":"Parker","year":"2020","journal-title":"JAMA Netw Open"},{"key":"zoi230200r2","volume-title":"Improving Data Infrastructure to Reduce Firearms Violence","author":"Barber","year":"2021"},{"key":"zoi230200r1","unstructured":"NORC at the University of Chicago. Expert panel on firearms data infrastructure. Accessed February 20, 2023. https:\/\/www.norc.org\/Research\/Projects\/Pages\/expert-panel-on-firearms-data-infrastructure.aspx"}],"container-title":["JAMA Network Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/articlepdf\/2803252\/macphaul_2023_oi_230200_1680032301.31853.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,6]],"date-time":"2023-04-06T15:04:28Z","timestamp":1680793468000},"score":1,"resource":{"primary":{"URL":"https:\/\/jamanetwork.com\/journals\/jamanetworkopen\/fullarticle\/2803252"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,6]]},"references-count":21,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,4,3]]}},"URL":"https:\/\/doi.org\/10.1001\/jamanetworkopen.2023.5870","relation":{},"ISSN":["2574-3805"],"issn-type":[{"value":"2574-3805","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,6]]}}}