{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,2]],"date-time":"2025-08-02T18:30:38Z","timestamp":1754159438301,"version":"3.41.2"},"reference-count":16,"publisher":"BMJ","issue":"1","license":[{"start":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T00:00:00Z","timestamp":1753401600000},"content-version":"unspecified","delay-in-days":24,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":["bmj.com"],"crossmark-restriction":true},"short-container-title":["BMJ Health Care Inform"],"accepted":{"date-parts":[[2025,6,11]]},"published-print":{"date-parts":[[2025,7]]},"abstract":"<jats:sec>\n                  <jats:title>Objectives<\/jats:title>\n                  <jats:p>Identifying whether there is a traumatic intracranial bleed (ICB+) on head CT is critical for clinical care and research. Free text CT reports are unstructured and therefore must undergo time-consuming manual review. Existing artificial intelligence classification schemes are not optimised for the emergency department endpoint of classification of ICB+ or ICB\u2212. We sought to assess three methods for classifying CT reports: a text classification (TC) programme, a commercial natural language processing programme (Clinithink) and a generative pretrained transformer large language model (Digitalizing English-language CT Interpretation for Positive Haemorrhage Evaluation Reporting (DECIPHER)-LLM).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Methods<\/jats:title>\n                  <jats:p>Primary objective: determine the diagnostic classification performance of the dichotomous categorisation of each of the three approaches.<\/jats:p>\n                  <jats:p>Secondary objective: determine whether the LLM could achieve a substantial reduction in CT report review workload while maintaining 100% sensitivity.<\/jats:p>\n                  <jats:p>Anonymised radiology reports of head CT scans performed for trauma were manually labelled as ICB+\/\u2212. Training and validation sets were randomly created to train the TC and natural language processing models. Prompts were written to train the LLM.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>898 reports were manually labelled. Sensitivity and specificity (95% CI)) of TC, Clinithink and DECIPHER-LLM (with probability of ICB set at 10%) were respectively 87.9% (76.7% to 95.0%) and 98.2% (96.3% to 99.3%), 75.9% (62.8% to 86.1%) and 96.2% (93.8% to 97.8%) and 100% (93.8% to 100%) and 97.4% (95.3% to 98.8%).<\/jats:p>\n                  <jats:p>With DECIPHER-LLM probability of ICB+ threshold of 10% set to identify CT reports requiring manual evaluation, CT reports requiring manual classification reduced by an estimated 385\/449 cases (85.7% (95% CI 82.1% to 88.9%)) while maintaining 100% sensitivity.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion and conclusion<\/jats:title>\n                  <jats:p>DECIPHER-LLM outperformed other tested free-text classification methods.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1136\/bmjhci-2025-101433","type":"journal-article","created":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T10:20:15Z","timestamp":1753438815000},"page":"e101433","update-policy":"https:\/\/doi.org\/10.1136\/crossmarkpolicy","source":"Crossref","is-referenced-by-count":0,"title":["Digitalizing English-language CT Interpretation for Positive Haemorrhage Evaluation Reporting: the DECIPHER study"],"prefix":"10.1136","volume":"32","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3016-4925","authenticated-orcid":false,"given":"Ben","family":"Bloom","sequence":"first","affiliation":[{"name":"Emergency Department, Royal London Hospital, Barts Health NHS Trust, London, UK"},{"name":"Queen Mary University of London Blizard Institute, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adrian","family":"Haimovich","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Harvard Medical School, Boston, Massachusetts, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jason","family":"Pott","sequence":"additional","affiliation":[{"name":"Queen Mary University of London, London, UK"},{"name":"Emergency Department, Barts Health NHS Trust, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sophie L","family":"Williams","sequence":"additional","affiliation":[{"name":"Barts Life Sciences, Barts Health NHS Trust, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Cheetham","sequence":"additional","affiliation":[{"name":"Emergency Department, Barts Health NHS Trust, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sandra","family":"Langsted","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, Randers Regional Hospital, Randers, Denmark"},{"name":"Aarhus University Hospital, Aarhus, Denmark"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Imogen","family":"Skene","sequence":"additional","affiliation":[{"name":"Queen Mary University of London, London, UK"},{"name":"Barts Health NHS Trust, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Raine","family":"Astin-Chamberlain","sequence":"additional","affiliation":[{"name":"Queen Mary University of London, London, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Stephen H","family":"Thomas","sequence":"additional","affiliation":[{"name":"Queen Mary University of London, London, UK"},{"name":"Harvard Medical School, Boston, Massachusetts, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"239","published-online":{"date-parts":[[2025,7,25]]},"reference":[{"key":"2025072503200645000_32.1.e101433.1","first-page":"483","article-title":"CT overuse for mild traumatic brain injury","volume":"38","author":"Melnick","year":"2012","journal-title":"Jt Comm J Qual Patient Saf"},{"key":"2025072503200645000_32.1.e101433.2","doi-asserted-by":"crossref","DOI":"10.1542\/peds.2024-066955","article-title":"Interventions to Reduce Imaging in Children With Minor Traumatic Head Injury: A Systematic Review","volume":"154","author":"Lesyk","year":"2024","journal-title":"Pediatrics"},{"key":"2025072503200645000_32.1.e101433.3","doi-asserted-by":"publisher","DOI":"10.1111\/acem.12184"},{"key":"2025072503200645000_32.1.e101433.4","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1111\/acem.12136","article-title":"Agreement between routine emergency department care and clinical decision support recommended care in patients evaluated for mild traumatic brain injury","volume":"20","author":"Korley","year":"2013","journal-title":"Acad Emerg Med"},{"key":"2025072503200645000_32.1.e101433.5","doi-asserted-by":"publisher","DOI":"10.1056\/NEJM200007133430204"},{"key":"2025072503200645000_32.1.e101433.6","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(00)04561-X"},{"key":"2025072503200645000_32.1.e101433.7","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1089\/neu.2024.0332","article-title":"Outcomes after Traumatic Brain Injury with and Without Computed Tomography","volume":"42","author":"Mikoli\u0107","year":"2025","journal-title":"J Neurotrauma"},{"key":"2025072503200645000_32.1.e101433.8","article-title":"Accuracy of Canadian CT Head Rule and New Orleans Criteria for Minor Head Trauma; a Systematic Review and Meta-Analysis","volume":"8","author":"Alzuhairy","year":"2020","journal-title":"Arch Acad Emerg Med"},{"key":"2025072503200645000_32.1.e101433.9","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2022.27109","article-title":"Development and Validation of a Model to Identify Critical Brain Injuries Using Natural Language Processing of Text Computed Tomography Reports","volume":"5","author":"Torres-Lopez","year":"2022","journal-title":"JAMA Netw Open"},{"key":"2025072503200645000_32.1.e101433.10","doi-asserted-by":"crossref","first-page":"400","DOI":"10.2215\/CJN.0000000000000081","article-title":"Natural Language Processing Basics","volume":"18","author":"Arivazhagan","year":"2023","journal-title":"Clin J Am Soc Nephrol"},{"key":"2025072503200645000_32.1.e101433.11","doi-asserted-by":"crossref","DOI":"10.1136\/bmj.h5527","article-title":"STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies","volume":"351","author":"Bossuyt","year":"2015","journal-title":"BMJ"},{"key":"2025072503200645000_32.1.e101433.12","doi-asserted-by":"crossref","DOI":"10.1126\/scitranslmed.aat6177","article-title":"Diagnosis of genetic diseases in seriously ill children by rapid whole-genome sequencing and automated phenotyping and interpretation","volume":"11","author":"Clark","year":"2019","journal-title":"Sci Transl Med"},{"key":"2025072503200645000_32.1.e101433.13","doi-asserted-by":"publisher","DOI":"10.1002\/sim.4509"},{"key":"2025072503200645000_32.1.e101433.14","doi-asserted-by":"crossref","DOI":"10.1186\/s13040-023-00322-4","article-title":"The Matthews correlation coefficient (MCC) should replace the ROC AUC as the standard metric for assessing binary classification","volume":"16","author":"Chicco","year":"2023","journal-title":"BioData Min"},{"key":"2025072503200645000_32.1.e101433.15","doi-asserted-by":"crossref","DOI":"10.1038\/s41598-022-18751-2","article-title":"Experimental evidence of effective human-AI collaboration in medical decision-making","volume":"12","author":"Reverberi","year":"2022","journal-title":"Sci Rep"},{"key":"2025072503200645000_32.1.e101433.16","doi-asserted-by":"crossref","DOI":"10.1001\/jamanetworkopen.2024.0357","article-title":"Generative Artificial Intelligence to Transform Inpatient Discharge Summaries to Patient-Friendly Language and Format","volume":"7","author":"Zaretsky","year":"2024","journal-title":"JAMA Netw Open"}],"container-title":["BMJ Health &amp; Care Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/syndication.highwire.org\/content\/doi\/10.1136\/bmjhci-2025-101433","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T10:20:17Z","timestamp":1753438817000},"score":1,"resource":{"primary":{"URL":"https:\/\/informatics.bmj.com\/lookup\/doi\/10.1136\/bmjhci-2025-101433"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7]]},"references-count":16,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,7,25]]},"published-print":{"date-parts":[[2025,7]]}},"alternative-id":["10.1136\/bmjhci-2025-101433"],"URL":"https:\/\/doi.org\/10.1136\/bmjhci-2025-101433","relation":{},"ISSN":["2632-1009"],"issn-type":[{"value":"2632-1009","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7]]}}}