{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,13]],"date-time":"2026-09-13T04:50:51Z","timestamp":1789275051267,"version":"build-2803163510"},"reference-count":41,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR AI"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n The recent availability of law enforcement and coroner or medical examiner reports for nearly every violent death in the United States expands the potential for natural language processing (NLP) research into violence.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n The objective of this work is to assess applications of supervised NLP to unstructured data in the National Violent Death Reporting System to predict circumstances and types of violent death.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n This analysis applied distilBERT, a compact large language model (LLM) with fewer parameters relative to full-scale LLMs, to unstructured narrative data to simulate the impacts of preprocessing, volume, and composition of training data on model performance, evaluated by\n F<\/jats:italic>\n 1<\/jats:sub>\n -scores, precision, recall, and the false negative rate. Model performance was evaluated for bias by race, ethnicity, and sex by comparing\n F<\/jats:italic>\n 1<\/jats:sub>\n -scores across subgroups.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n A minimum training set of 1500 cases was necessary to achieve an\n F<\/jats:italic>\n 1<\/jats:sub>\n -score of 0.6 and a false negative rate of 0.01-0.05 with a compact LLM. Replacement of domain-specific jargon improved model performance, while oversampling positive class cases to address class imbalance did not substantially improve\n F<\/jats:italic>\n 1<\/jats:sub>\n -scores. Between racial and ethnic groups,\n F<\/jats:italic>\n 1<\/jats:sub>\n -score disparities ranged from 0.2 to 0.25, and between male and female decedents, differences ranged from 0.12 to 0.2.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Compact LLMs with sufficient training data can be applied to supervised NLP tasks with a class imbalance in the National Violent Death Reporting System. Simulations of supervised text classification across the model-fitting process of preprocessing and training compact LLM-informed NLP applications to unstructured death narrative data.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/68212","type":"journal-article","created":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T15:42:37Z","timestamp":1750347757000},"page":"e68212-e68212","source":"Crossref","is-referenced-by-count":5,"title":["Supervised Natural Language Processing Classification of Violent Death Narratives: Development and Assessment of a Compact Large Language Model"],"prefix":"10.2196","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0459-4945","authenticated-orcid":false,"given":"Susan T","family":"Parker","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,6,19]]},"reference":[{"key":"R1","unstructured":"WISQARS leading causes of death visualization tool. Centers for Disease Control and Prevention. URL: https:\/\/wisqars.cdc.gov\/lcd [Accessed 08-05-2025]"},{"key":"R2","unstructured":"National Violent Death Reporting System (NVDRS). Centers for Disease Control and Prevention. 2024. URL: https:\/\/www.cdc.gov\/nvdrs\/about\/index.html [Accessed 08-05-2025]"},{"issue":"2","key":"R3","doi-asserted-by":"publisher","first-page":"115","DOI":"10.29333\/ajqr\/12129","article-title":"Mixed methods secondary analysis of older adult homicide-suicides from National Violent Death Reporting System (NVDRS) Data","volume":"6","author":"Chatfield","journal-title":"Am J Qual Res"},{"issue":"1","key":"R4","doi-asserted-by":"publisher","DOI":"10.1186\/s40621-021-00345-7","article-title":"Examining differences between mass, multiple, and single-victim homicides to inform prevention: findings from the National Violent Death Reporting System","volume":"8","author":"Fowler","journal-title":"Inj Epidemiol"},{"issue":"1","key":"R5","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1177\/10439862231189985","article-title":"The research utility of the National Violent Death Reporting System for understanding homicide trends","volume":"40","author":"Rogers","journal-title":"J Contemp Crim Justice"},{"issue":"1","key":"R6","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1016\/j.amepre.2018.08.028","article-title":"The role of intimate partner violence in homicides of children aged 2-14 years","volume":"56","author":"Adhia","journal-title":"Am J Prev Med"},{"issue":"2","key":"R7","doi-asserted-by":"publisher","first-page":"101","DOI":"10.7326\/M13-1301","article-title":"The accessibility of firearms and risk for suicide and homicide victimization among household members: a systematic review and meta-analysis","volume":"160","author":"Anglemyer","journal-title":"Ann Intern Med"},{"key":"R8","doi-asserted-by":"publisher","DOI":"10.1016\/j.amepre.2016.08.006","article-title":"Identifying and tracking gas suicides in the U.S. using the National Violent Death Reporting System, 2005-2012","volume":"51","author":"Azrael","journal-title":"Am J Prev Med"},{"key":"R9","doi-asserted-by":"publisher","DOI":"10.1016\/j.ypmed.2022.107066","article-title":"Who owned the gun in firearm suicides of men, women, and youth in five US states?","volume":"164","author":"Barber","journal-title":"Prev Med"},{"issue":"1","key":"R10","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1027\/0227-5910\/a000676","article-title":"Suicides at shooting ranges","volume":"42","author":"Barber","journal-title":"Crisis"},{"issue":"4","key":"R11","doi-asserted-by":"publisher","first-page":"578","DOI":"10.2105\/AJPH.2018.304904","article-title":"Validating the National Violent Death Reporting System as a source of data on fatal shootings of civilians by law enforcement officers","volume":"109","author":"Conner","journal-title":"Am J Public Health"},{"key":"R12","unstructured":"National violent death reporting system web coding manual, 6.0. Centers for Disease Control and Prevention. 2022. URL: https:\/\/stacks.cdc.gov\/view\/cdc\/44789 [Accessed 29-05-2025]"},{"issue":"1","key":"R13","doi-asserted-by":"publisher","DOI":"10.1186\/s40621-023-00433-w","article-title":"Research utility and limitations of textual data in the National Violent Death Reporting System: a scoping review and recommendations","volume":"10","author":"Dang","journal-title":"Inj Epidemiol"},{"issue":"9","key":"R14","doi-asserted-by":"publisher","DOI":"10.1002\/hsr2.1526","article-title":"Identifying suicide documentation in clinical notes through zero-shot learning","volume":"6","author":"Workman","journal-title":"Health Sci Rep"},{"issue":"1","key":"R15","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-25773-2","article-title":"Identifying suicide ideation and suicidal attempts in a psychiatric clinical research database using natural language processing","volume":"8","author":"Fernandes","journal-title":"Sci Rep"},{"issue":"7","key":"R16","doi-asserted-by":"publisher","DOI":"10.2196\/17784","article-title":"Identifying and predicting intentional self-harm in electronic health record clinical notes: deep learning approach","volume":"8","author":"Obeid","journal-title":"JMIR Med Inform"},{"issue":"2","key":"R17","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0211116","article-title":"Identification of suicidal behavior among psychiatrically hospitalized adolescents using natural language processing and machine learning of electronic health records","volume":"14","author":"Carson","journal-title":"PLOS ONE"},{"issue":"8","key":"R18","doi-asserted-by":"publisher","first-page":"1382","DOI":"10.1017\/S0033291720000173","article-title":"Natural language processing of clinical mental health notes may add predictive value to existing suicide risk models","volume":"51","author":"Levis","journal-title":"Psychol Med"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1038\/s44184-023-00046-7","article-title":"Natural language processing of multi-hospital electronic health records for public health surveillance of suicidality","volume":"3","author":"Bey","journal-title":"Npj Ment Health Res"},{"issue":"5","key":"R20","doi-asserted-by":"publisher","first-page":"781","DOI":"10.5811\/westjem.2022.7.56726","article-title":"A novel technique to identify intimate partner violence in a hospital setting","volume":"23","author":"Tabaie","journal-title":"West J Emerg Med"},{"key":"R21","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyt.2024.1181739","article-title":"Applying neural network algorithms to ascertain reported experiences of violence in routine mental healthcare records and distributions of reports by diagnosis","volume":"15","author":"Mason","journal-title":"Front Psychiatry"},{"issue":"2","key":"R22","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2021-052911","article-title":"Can natural language processing models extract and classify instances of interpersonal violence in mental healthcare electronic records: an applied evaluative study","volume":"12","author":"Botelle","journal-title":"BMJ Open"},{"issue":"10","key":"R23","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2020.20664","article-title":"Estimating nonfatal gunshot injury locations with natural language processing and machine learning models","volume":"3","author":"Parker","journal-title":"JAMA Netw Open"},{"key":"R24","doi-asserted-by":"publisher","DOI":"10.2196\/49359","article-title":"Identifying rare circumstances preceding female firearm suicides: validating a large language model approach","volume":"10","author":"Zhou","journal-title":"JMIR Ment Health"},{"issue":"8","key":"R25","doi-asserted-by":"publisher","first-page":"1408","DOI":"10.1093\/jamia\/ocad068","article-title":"An NLP approach to identify SDoH-related circumstance and suicide crisis from death investigation narratives","volume":"30","author":"Wang","journal-title":"J Am Med Inform Assoc"},{"key":"R26","volume":"29","author":"Ali","journal-title":"Int J Adv Soft Comput Appl"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2019.09.229","article-title":"Dealing with data imbalance in text classification","volume":"159","author":"Padurariu","journal-title":"Procedia Comput Sci"},{"issue":"S2","key":"R28","doi-asserted-by":"publisher","first-page":"S107","DOI":"10.2105\/AJPH.2021.306312","article-title":"Aggression, escalation, and other latent themes in legal intervention deaths of non-Hispanic Black and White men: results from the 2003\u20122017 National Violent Death Reporting System","volume":"111","author":"Arseniev-Koehler","journal-title":"Am J Public Health"},{"key":"R29","doi-asserted-by":"crossref","unstructured":"Subramanian S Rahimi A Baldwin T Cohn T Frermann L . Fairness-aware class imbalanced learning. arXiv. Preprint posted online on Sep 21, 2021. [doi: 10.48550\/arXiv.2109.10444]","DOI":"10.18653\/v1\/2021.emnlp-main.155"},{"issue":"3","key":"R30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3699955","article-title":"A comprehensive survey on rare event prediction","volume":"57","author":"Shyalika","journal-title":"ACM Comput Surv"},{"key":"R31","doi-asserted-by":"publisher","DOI":"10.1080\/10543406.2024.2364722","article-title":"A machine learning case study to predict rare clinical event of interest: imbalanced data, interpretability, and practical considerations","volume":"0","author":"Zhong","journal-title":"J Biopharm Stat"},{"issue":"4","key":"R32","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1002\/hcs2.61","article-title":"Large language models in health care: development, applications, and challenges","volume":"2","author":"Yang","journal-title":"Health Care Sci"},{"key":"R33","doi-asserted-by":"publisher","DOI":"10.2196\/48291","article-title":"Large language models in medical education: opportunities, challenges, and future directions","volume":"9","author":"Abd-Alrazaq","journal-title":"JMIR Med Educ"},{"key":"R34","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2024.108189","article-title":"A comprehensive evaluation of large language models on benchmark biomedical text processing tasks","volume":"171","author":"Jahan","journal-title":"Comput Biol Med"},{"issue":"8","key":"R35","doi-asserted-by":"publisher","first-page":"1478","DOI":"10.3390\/healthcare10081478","article-title":"Comparison of pretraining models and strategies for health-related social media text classification","volume":"10","author":"Guo","journal-title":"Healthcare (Basel)"},{"key":"R36","doi-asserted-by":"crossref","unstructured":"Shao Y Divita G Workman TE Redd D Garvin JH Zeng-Treitler Q . Clinical sublanguage trend and usage analysis from a large clinical corpus. Presented at: 2020 IEEE International Conference on Big Data (Big Data); Dec 10-13, 2020:3837-3845; Atlanta, GA, USA. [doi: 10.1109\/BigData50022.2020.9378203]","DOI":"10.1109\/BigData50022.2020.9378203"},{"key":"R37","doi-asserted-by":"crossref","unstructured":"Workman TE Divita G Zeng-Treitler Q . Discovering sublanguages in a large clinical corpus through unsupervised machine learning and information gain. Presented at: 2019 IEEE International Conference on Big Data (Big Data); Dec 9-12, 2019:4889-4898; Los Angeles, CA, USA. [doi: 10.1109\/BigData47090.2019.9006492]","DOI":"10.1109\/BigData47090.2019.9006492"},{"issue":"7","key":"R38","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0254417","article-title":"Not discussed: Inequalities in narrative text data for suicide deaths in the National Violent Death Reporting System","volume":"16","author":"Mezuk","journal-title":"PLoS One"},{"issue":"S3","key":"R39","doi-asserted-by":"publisher","first-page":"S268","DOI":"10.2105\/AJPH.2023.307427","article-title":"Gendered patterns in manifest and latent mental health indicators among suicide decedents: 2003-2020 National Violent Death Reporting System (NVDRS)","volume":"114","author":"Arseniev-Koehler","journal-title":"Am J Public Health"},{"issue":"4","key":"R40","doi-asserted-by":"publisher","first-page":"782","DOI":"10.1111\/sltb.12862","article-title":"Using natural language processing to improve suicide classification requires consideration of race","volume":"52","author":"Rahman","journal-title":"Suicide Life Threat Behav"},{"key":"R41","unstructured":"Sanh V Debut L Chaumond J Wolf T . DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv. Preprint posted online on Mar 1, 2020. [doi: 10.48550\/arXiv.1910.01108]"}],"container-title":["JMIR AI"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ai.jmir.org\/2025\/1\/e68212","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:27:15Z","timestamp":1785536835000},"score":1,"resource":{"primary":{"URL":"https:\/\/ai.jmir.org\/2025\/1\/e68212"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,19]]},"references-count":41,"URL":"https:\/\/doi.org\/10.2196\/68212","relation":{},"ISSN":["2817-1705"],"issn-type":[{"value":"2817-1705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,19]]},"article-number":"v4i2e68212"}}