{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T17:46:41Z","timestamp":1755798401559,"version":"3.44.0"},"reference-count":42,"publisher":"Oxford University Press (OUP)","issue":"9","license":[{"start":{"date-parts":[[2025,7,23]],"date-time":"2025-07-23T00:00:00Z","timestamp":1753228800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/100000050","name":"NHLBI","doi-asserted-by":"publisher","award":["R01 HL 173037"],"award-info":[{"award-number":["R01 HL 173037"]}],"id":[{"id":"10.13039\/100000050","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000092","name":"NLM","doi-asserted-by":"publisher","award":["IH R01 LM012973"],"award-info":[{"award-number":["IH R01 LM012973"]}],"id":[{"id":"10.13039\/100000092","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000050","name":"NHLBI","doi-asserted-by":"publisher","award":["R01 HL157262"],"award-info":[{"award-number":["R01 HL157262"]}],"id":[{"id":"10.13039\/100000050","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Risk prediction models are used in hospitals to identify pediatric patients at risk of clinical deterioration, enabling timely interventions and rescue. The objective of this study was to develop a new explainer algorithm that uses a patient\u2019s clinical notes to generate text-based explanations for risk prediction alerts.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>We conducted a retrospective study of 39\u2009406 patient admissions to the American Family Children\u2019s Hospital at the University of Wisconsin-Madison (2009-2020). The pediatric Calculated Assessment of Risk and Triage (pCART) validated risk prediction model was used to identify children at risk for deterioration. A transformer model was trained to use clinical notes from the 12-hour period preceding each pCART score to predict whether a patient was flagged as at risk. Then, label-aware attention highlighted text phrases most important to an at-risk alert. The study cohort was randomly split into derivation (60%) and validation (20%) data, and a separate test (20%) was used to evaluate the explainer\u2019s performance.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Our pCART Explainer algorithm performed well in discriminating at-risk pCART alert vs no alert (c-statistic 0.805). Sample explanations from pCART Explainer revealed clinically important phrases such as \u201crapid breathing,\u201d \u201cfall risk,\u201d \u201cdistension,\u201d and \u201cgrunting,\u201d thereby demonstrating excellent face validity.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>The pCART Explainer could quickly orient clinicians to the patient\u2019s condition by drawing attention to key phrases in notes, potentially enhancing situational awareness and guiding decision-making.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>We developed pCART Explainer, a novel algorithm that highlights text within clinical notes to provide medically relevant context about deterioration alerts, thereby improving the explainability of the pCART model.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocaf121","type":"journal-article","created":{"date-parts":[[2025,7,14]],"date-time":"2025-07-14T14:11:10Z","timestamp":1752502270000},"page":"1445-1453","source":"Crossref","is-referenced-by-count":1,"title":["Explaining alerts from a pediatric risk prediction model using clinical text"],"prefix":"10.1093","volume":"32","author":[{"given":"Samuel","family":"Nycklemoe","sequence":"first","affiliation":[{"name":"Department of Biostatistics & Medical Informatics, University of Wisconsin-Madison , Madison, WI 53726,","place":["United States"]}]},{"given":"Sriharsha","family":"Devarapu","sequence":"additional","affiliation":[{"name":"Department of Biostatistics & Medical Informatics, University of Wisconsin-Madison , Madison, WI 53726,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9341-7360","authenticated-orcid":false,"given":"Yanjun","family":"Gao","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, University of Colorado Anschutz Medical Campus , Aurora, CO 80045,","place":["United States"]}]},{"given":"Kyle","family":"Carey","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Chicago , Chicago, IL 60637,","place":["United States"]}]},{"given":"Nicholas","family":"Kuehnel","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, University of Wisconsin-Madison , Madison, WI 53726,","place":["United States"]}]},{"given":"Neil","family":"Munjal","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, University of Wisconsin-Madison , Madison, WI 53726,","place":["United States"]}]},{"given":"Priti","family":"Jani","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, University of Chicago , Chicago, IL 60637,","place":["United States"]}]},{"given":"Matthew","family":"Churpek","sequence":"additional","affiliation":[{"name":"Department of Biostatistics & Medical Informatics, University of Wisconsin-Madison , Madison, WI 53726,","place":["United States"]},{"name":"Department of Medicine, University of Wisconsin-Madison , Madison, WI 53726,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2585-2707","authenticated-orcid":false,"given":"Dmitriy","family":"Dligach","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Loyola University , Chicago, IL 60626,","place":["United States"]}]},{"given":"Majid","family":"Afshar","sequence":"additional","affiliation":[{"name":"Department of Biostatistics & Medical Informatics, University of Wisconsin-Madison , Madison, WI 53726,","place":["United States"]},{"name":"Department of Medicine, University of Wisconsin-Madison , Madison, WI 53726,","place":["United States"]}]},{"given":"Anoop","family":"Mayampurath","sequence":"additional","affiliation":[{"name":"Department of Biostatistics & Medical Informatics, University of Wisconsin-Madison , Madison, WI 53726,","place":["United States"]},{"name":"Department of Medicine, University of Wisconsin-Madison , Madison, WI 53726,","place":["United States"]}]}],"member":"286","published-online":{"date-parts":[[2025,7,23]]},"reference":[{"key":"2025081903513105800_ocaf121-B1","doi-asserted-by":"publisher","first-page":"506","DOI":"10.1001\/jamapediatrics.2023.0184","article-title":"Epidemiology of intensive care admissions for children in the US from 2001 to 2019","volume":"177","author":"Killien","year":"2023","journal-title":"JAMA Pediatr"},{"key":"2025081903513105800_ocaf121-B2","doi-asserted-by":"publisher","first-page":"ofad218","DOI":"10.1093\/ofid\/ofad218","article-title":"Epidemiology of sepsis in US children and young adults","volume":"10","author":"Magill","year":"2023","journal-title":"Open Forum Infect Dis"},{"key":"2025081903513105800_ocaf121-B3","doi-asserted-by":"publisher","first-page":"798","DOI":"10.1097\/CCM.0000000000001484","article-title":"Incidence and outcomes of cardiopulmonary resuscitation in pediatric intensive care units","volume":"44","author":"Berg","year":"2016","journal-title":"Crit Care Med"},{"key":"2025081903513105800_ocaf121-B4","doi-asserted-by":"publisher","first-page":"804","DOI":"10.1164\/rccm.201905-0977ED","article-title":"Life after pediatric critical illness: risk factors for reduced health-related quality of life and functional decline","volume":"200","author":"Hopkins","year":"2019","journal-title":"Am J Respir Crit Care Med"},{"key":"2025081903513105800_ocaf121-B5","doi-asserted-by":"publisher","first-page":"900","DOI":"10.1164\/rccm.201810-1881OC","article-title":"Risk factors for functional decline and impaired quality of life after pediatric respiratory failure","volume":"200","author":"Watson","year":"2019","journal-title":"Am J Respir Crit Care Med"},{"key":"2025081903513105800_ocaf121-B6","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1001\/jamapediatrics.2013.3266","article-title":"Impact of rapid response system implementation on critical deterioration events in children","volume":"168","author":"Bonafide","year":"2014","journal-title":"JAMA Pediatr"},{"key":"2025081903513105800_ocaf121-B7","doi-asserted-by":"publisher","first-page":"e014497","DOI":"10.1136\/bmjopen-2016-014497","article-title":"Paediatric early warning systems for detecting and responding to clinical deterioration in children: a systematic review","volume":"7","author":"Lambert","year":"2017","journal-title":"BMJ Open"},{"key":"2025081903513105800_ocaf121-B8","doi-asserted-by":"publisher","first-page":"1002","DOI":"10.1001\/jama.2018.0948","article-title":"Effect of a pediatric early warning system on all-cause mortality in hospitalized pediatric patients: the epoch randomized clinical trial","volume":"319","author":"Parshuram","year":"2018","journal-title":"JAMA"},{"key":"2025081903513105800_ocaf121-B9","doi-asserted-by":"publisher","first-page":"514","DOI":"10.1097\/PCC.0000000000002965","article-title":"Development and external validation of a machine learning model for prediction of potential transfer to the ICU","volume":"23","author":"Mayampurath","year":"2022","journal-title":"Pediatr Crit Care Med"},{"key":"2025081903513105800_ocaf121-B10","doi-asserted-by":"publisher","first-page":"e146","DOI":"10.1097\/PCC.0000000000003656","article-title":"Machine learning-based pediatric early warning score: patient outcomes in a pre- versus post-implementation study, 2019-2023","volume":"26","author":"Mayampurath","year":"2025","journal-title":"Pediatr Crit Care Med"},{"key":"2025081903513105800_ocaf121-B11","doi-asserted-by":"publisher","first-page":"100540","DOI":"10.1016\/j.resplu.2023.100540","article-title":"Predicting transfers to intensive care in children using CEWT and other early warning systems","volume":"17","author":"McCaffery","year":"2024","journal-title":"Resusc Plus"},{"key":"2025081903513105800_ocaf121-B12","doi-asserted-by":"publisher","first-page":"199","DOI":"10.1002\/jhm.2774","article-title":"Clinical progress note: situation awareness for clinical deterioration in hospitalized children","volume":"17","author":"Sosa","year":"2022","journal-title":"J Hosp Med"},{"key":"2025081903513105800_ocaf121-B13","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical text analysis and knowledge extraction system (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","year":"2010","journal-title":"J Am Med Inform Assoc"},{"first-page":"4228","year":"2021","author":"Liu","key":"2025081903513105800_ocaf121-B14"},{"first-page":"4171","year":"2019","author":"Devlin","key":"2025081903513105800_ocaf121-B15"},{"first-page":"10","year":"2022","author":"Huang","key":"2025081903513105800_ocaf121-B16"},{"first-page":"6000","year":"2017","author":"Vaswani","key":"2025081903513105800_ocaf121-B17"},{"year":"2021","author":"Vu","key":"2025081903513105800_ocaf121-B18"},{"key":"2025081903513105800_ocaf121-B19","doi-asserted-by":"publisher","first-page":"59","DOI":"10.7326\/M19-2548","article-title":"Should health care demand interpretable artificial intelligence or accept \u201cBlack Box\u201d medicine?","volume":"172","author":"Wang","year":"2020","journal-title":"Ann Intern Med"},{"first-page":"359","year":"2019","author":"Tonekaboni","key":"2025081903513105800_ocaf121-B20"},{"key":"2025081903513105800_ocaf121-B21","doi-asserted-by":"publisher","first-page":"103655","DOI":"10.1016\/j.jbi.2020.103655","article-title":"The role of explainability in creating trustworthy artificial intelligence for health care: a comprehensive survey of the terminology, design choices, and evaluation strategies","volume":"113","author":"Markus","year":"2021","journal-title":"J Biomed Inform"},{"key":"2025081903513105800_ocaf121-B22","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1038\/s41746-020-0254-2","article-title":"Machine intelligence in healthcare\u2014perspectives on trustworthiness, explainability, usability, and transparency","volume":"3","author":"Cutillo","year":"2020","journal-title":"Npj Digit Med"},{"first-page":"1135","year":"2016","author":"Ribeiro","key":"2025081903513105800_ocaf121-B23"},{"key":"2025081903513105800_ocaf121-B24","article-title":"On the robustness of interpretability methods","author":"Alvarez-Melis","year":"2018","journal-title":"ICML Workshop on Human Interpretability in Machine Learning"},{"key":"2025081903513105800_ocaf121-B25","first-page":"91","article-title":"Statistical stability indices for LIME: obtaining reliable explanations for machine learning models","volume":"1","author":"Visani","year":"2021","journal-title":"J Oper Res Soc"},{"first-page":"9269","year":"2020","author":"Sundararajan","key":"2025081903513105800_ocaf121-B26"},{"year":"2021","author":"Jethani","key":"2025081903513105800_ocaf121-B27"},{"key":"2025081903513105800_ocaf121-B28","doi-asserted-by":"publisher","first-page":"103728","DOI":"10.1016\/j.jbi.2021.103728","article-title":"Explainable automated coding of clinical notes using hierarchical label-wise attention networks and label embedding initialisation","volume":"116","author":"Dong","year":"2021","journal-title":"J Biomed Inform"},{"author":"Liu","key":"2025081903513105800_ocaf121-B29","first-page":"440"},{"key":"2025081903513105800_ocaf121-B30","doi-asserted-by":"publisher","first-page":"104323","DOI":"10.1016\/j.jbi.2023.104323","article-title":"Explainable clinical coding with in-domain adapted transformers","volume":"139","author":"L\u00f3pez-Garc\u00eda","year":"2023","journal-title":"J Biomed Inform"},{"key":"2025081903513105800_ocaf121-B31","doi-asserted-by":"publisher","first-page":"7155","DOI":"10.1038\/s41598-020-62922-y","article-title":"BEHRT: transformer for electronic health records","volume":"10","author":"Li","year":"2020","journal-title":"Sci Rep"},{"key":"2025081903513105800_ocaf121-B32","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1038\/s41746-021-00455-y","article-title":"Med-BERT: pretrained contextualized embeddings on large-scale structured electronic health records for disease prediction","volume":"4","author":"Rasmy","year":"2021","journal-title":"Npj Digit Med"},{"key":"2025081903513105800_ocaf121-B33","doi-asserted-by":"publisher","first-page":"1333","DOI":"10.1109\/JBHI.2024.3483577","article-title":"A cross attention approach to diagnostic explainability using clinical practice guidelines for depression","volume":"29","author":"Dalal","year":"2025","journal-title":"IEEE J Biomed Health Inform"},{"key":"2025081903513105800_ocaf121-B34","doi-asserted-by":"publisher","first-page":"100185","DOI":"10.1016\/j.jpi.2022.100185","article-title":"Transformer-based personalized attention mechanism for medical images with clinical records","volume":"14","author":"Takagi","year":"2023","journal-title":"J Pathol Inform"},{"key":"2025081903513105800_ocaf121-B35","doi-asserted-by":"publisher","first-page":"1292","DOI":"10.1093\/jamia\/ocac058","article-title":"Use of unstructured text in prognostic clinical prediction models: a systematic review","volume":"29","author":"Seinen","year":"2022","journal-title":"J Am Med Inform Assoc"},{"key":"2025081903513105800_ocaf121-B36","doi-asserted-by":"publisher","first-page":"e745","DOI":"10.1016\/S2589-7500(21)00208-9","article-title":"The false hope of current approaches to explainable artificial intelligence in health care","volume":"3","author":"Ghassemi","year":"2021","journal-title":"Lancet Digit Health"},{"key":"2025081903513105800_ocaf121-B37","doi-asserted-by":"publisher","first-page":"e33960","DOI":"10.2196\/33960","article-title":"Factors influencing clinician trust in predictive clinical decision support systems for in-hospital deterioration: qualitative descriptive study","volume":"9","author":"Schwartz","year":"2022","journal-title":"JMIR Hum Factors"},{"key":"2025081903513105800_ocaf121-B38","doi-asserted-by":"publisher","first-page":"e2024066168","DOI":"10.1542\/peds.2024-066168","article-title":"Comparing AVPU and Glasgow coma scales among children seen by emergency medical services","volume":"154","author":"Ramgopal","year":"2024","journal-title":"Pediatrics"},{"key":"2025081903513105800_ocaf121-B39","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1016\/j.jpeds.2018.04.037","article-title":"Etiologies and yield of diagnostic testing in children presenting to the emergency department with altered mental status","volume":"200","author":"Button","year":"2018","journal-title":"J Pediatr"},{"key":"2025081903513105800_ocaf121-B40","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1186\/s40560-024-00763-x","article-title":"Early predictors of unfavorable outcomes in pediatric acute respiratory failure","volume":"12","author":"Miura","year":"2024","journal-title":"J Intensive Care"},{"key":"2025081903513105800_ocaf121-B41","doi-asserted-by":"publisher","first-page":"622","DOI":"10.1542\/hpeds.2023-007166","article-title":"Epidemiology of acute respiratory failure in US children: outcomes and resource use","volume":"14","author":"Odetola","year":"2024","journal-title":"Hosp Pediatr"},{"key":"2025081903513105800_ocaf121-B42","doi-asserted-by":"publisher","first-page":"836","DOI":"10.1001\/jama.2022.1480","article-title":"Association of acute respiratory failure in early childhood with long-term neurocognitive outcomes","volume":"327","author":"Watson","year":"2022","journal-title":"JAMA"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/9\/1445\/63835032\/ocaf121.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamia\/article-pdf\/32\/9\/1445\/63835032\/ocaf121.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,19]],"date-time":"2025-08-19T07:51:37Z","timestamp":1755589897000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/32\/9\/1445\/8211191"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,23]]},"references-count":42,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2025,7,23]]},"published-print":{"date-parts":[[2025,9,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaf121","relation":{},"ISSN":["1067-5027","1527-974X"],"issn-type":[{"type":"print","value":"1067-5027"},{"type":"electronic","value":"1527-974X"}],"subject":[],"published-other":{"date-parts":[[2025,9]]},"published":{"date-parts":[[2025,7,23]]}}}