{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T18:46:40Z","timestamp":1791917200634,"version":"4.3.4"},"reference-count":25,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"\n Background<\/jats:title>\n Trauma registries are crucial for evaluating the prognosis of injuries and the quality of trauma care. However, the calculation of Injury Severity Score (ISS) remains a predominantly manual process, which is labor-intensive, potentially error-prone, and difficult to scale. Automating this process is a critical step toward enhancing the efficiency and quality of trauma registry data.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study conducts a feasibility test on the use of large language models (LLMs) to automatically calculate ISS from free-text discharge diagnoses. We specifically investigate the efficacy of retrieval-augmented generation (RAG) to enhance LLM accuracy by incorporating knowledge from the Abbreviated Injury Scale. We compare the performance of baseline LLMs against models augmented with both a standard RAG and a novel, domain-specific RAG framework.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n In this retrospective validation study, we used deidentified, free-text discharge diagnoses from trauma patients at Chang Gung Memorial Hospital (Linkou, Taiwan) in 2022. Against a physician-verified gold standard, we compared the performance of three systems: (1) baseline LLMs, (2) LLMs with standard RAG, and (3) LLMs with domain-specific RAG, for which the retriever was fine-tuned on trauma-related ICD-10 (International Statistical Classification of Diseases, Tenth Revision) codes. Performance was evaluated on a physician-verified test set across exact ISS prediction, tolerance-based accuracy, and ISS severity categories.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Baseline state-of-the-art LLMs achieve modest exact-match (EM) accuracy, ranging from 47.3% to 64.7%. Standard RAG did not produce consistent gains (54.3% to 60.3%), whereas the domain-specific RAG substantially improved accuracy (62.7% to 75.3%). Performance gains were most pronounced in patients with moderate to severe injuries. GPT-4.1 with domain-specific RAG reached the highest EM accuracy (75.3%, 95% CI 66.1%-82.7%), compared with 70.0% for the deterministic International Classification of Diseases and R statistical software (ICDPIC-R) baseline. Tolerance-based and categorical analyses confirm substantial gains with the domain-specific RAG framework in clinically relevant ranges.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n This single-center feasibility study suggests that LLMs augmented with a domain-specific retriever can calculate the ISS from free-text discharge diagnoses with overall accuracy comparable to that of a deterministic coding tool, with the advantage concentrated in the most severe injuries. This approach may help alleviate the administrative burden of manual coding, enhance standardization, and improve the quality of trauma registry data. Future multi-institutional validation is needed to confirm these findings.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/93922","type":"journal-article","created":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T18:05:05Z","timestamp":1791914705000},"page":"e93922","source":"Crossref","is-referenced-by-count":0,"title":["Automating Injury Severity Score Calculation From Free-Text Discharge Diagnoses Using Domain-Specific Retrieval-Augmented Large Language Models: Retrospective Validation Study"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4347-0232","authenticated-orcid":false,"given":"Ying-Jia","family":"Lin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-8892-6925","authenticated-orcid":false,"given":"Kuan-Yueh","family":"Fang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2812-9773","authenticated-orcid":false,"given":"Chien-Hung","family":"Liao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2697-4642","authenticated-orcid":false,"given":"Chi-Tung","family":"Cheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,13]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1016\/j.injury.2017.02.015"},{"key":"ref2","doi-asserted-by":"publisher","DOI":"10.1097\/TA.0000000000000970"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1097\/00005373-197403000-00001"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.1016\/j.injury.2012.06.032"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0216206"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0257183"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-024-57538-5"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocae071"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.3390\/informatics11030057"},{"key":"ref10","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.8895"},{"key":"ref11","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.25981"},{"key":"ref12","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2025.11.064"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1038\/s44401-025-00018-3"},{"key":"ref14","unstructured":"LewisPPerezEPiktusAPetroniFKarpukhinVRetrieval-augmented generation for knowledge-intensive NLP tasks202012NIPS '20: Proceedings of the 34th International Conference on Neural Information Processing Systems2020 December 6-12Vancouver BC Canada94599474https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/6b493230 205f780e1bc26945df7481e5-Abstract.html\/>"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.5005\/jp-journals-10071-23120"},{"key":"ref16","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2010.04.012"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.52202\/068431-1800"},{"key":"ref18","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btad651"},{"key":"ref19","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.550"},{"key":"ref20","unstructured":"GPT-4.1 model cardOpenAI20252026-04-14https:\/\/openai.com\/index\/gpt-4-1\/"},{"key":"ref21","author":"Comanici, G","journal-title":"arXiv"},{"key":"ref22","unstructured":"System Card: Claude Sonnet 4.5Anthropic20252026-09-28https:\/\/www-cdn.anthropic.com\/963373e433e489a87a10c823c52 a0a013e9172dd\/Claude%20Sonnet%204.5%20System%20Card.pdf"},{"key":"ref23","unstructured":"The llama 4 herd: the beginning of a new era of natively multimodal AI innovationMeta AI20252026-09-19https:\/\/ai.meta.com\/blog\/llama-4-multimodal-intelligence\/"},{"key":"ref24","doi-asserted-by":"publisher","DOI":"10.1186\/s40621-018-0149-8"},{"key":"ref25","unstructured":"ila-lab\/cgmh-iss-testsetGitHub2026-09-30https:\/\/github.com\/ila-lab\/cgmh-iss-testset"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e93922","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T18:05:10Z","timestamp":1791914710000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e93922"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,13]]},"references-count":25,"URL":"https:\/\/doi.org\/10.2196\/93922","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.93922","asserted-by":"object"}]},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,13]]}}}