{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,8]],"date-time":"2025-11-08T00:11:12Z","timestamp":1762560672844,"version":"build-2065373602"},"reference-count":44,"publisher":"Georg Thieme Verlag KG","issue":"05","funder":[{"name":"University of Missouri Health Care"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2025,10]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>End-of-life care (EoLC), such as advance care planning, advance directives, hospice, and palliative care consults, can improve patient quality of life and reduce costs, yet such interventions are underused. Machine learning-based prediction models show promise in identifying patients who may be candidates for EoLC based on increased risk of short-term (less than 1 year) mortality. Clinical decision support systems using these models can identify candidate patients at a time during their care when care teams can increase the provision of EoLC.<\/jats:p>\n                  <jats:p>Evaluate changes in the provision of EoLC with implementation of a machine learning-based mortality prediction model in an academic health center.<\/jats:p>\n                  <jats:p>A clinical decision support system based on a random forest machine learning mortality prediction model is described. The system was implemented in an academic health system, first in the medical intensive care unit, then house-wide. An interrupted time series analysis was performed over the 16 weeks prior to and 43 weeks after the implementations. Primary outcomes were the rates of documentation of advance directives, palliative care consultations, and do not attempt resuscitation (DNAR) orders among encounters with an alert for PRISM score over 50% (PRISM positive) compared with those without an alert (PRISM negative).<\/jats:p>\n                  <jats:p>Following a steep preintervention decline, the rate of advance directive documentation improved immediately after implementation. However, the implementations were not associated with improvements in any of the other primary outcomes. The model discrimination was substantially worse than that observed in model development, and after 16 months, it was withdrawn from production.<\/jats:p>\n                  <jats:p>A clinical decision support system based on a machine learning mortality prediction model failed to provide clinically meaningful improvements in EoLC measures. Possible causes for the failure include system-level factors, clinical decision support system design, and poor model performance.<\/jats:p>","DOI":"10.1055\/a-2630-3204","type":"journal-article","created":{"date-parts":[[2025,11,8]],"date-time":"2025-11-08T00:01:42Z","timestamp":1762560102000},"page":"1637-1645","source":"Crossref","is-referenced-by-count":0,"title":["A Machine Learning-Based Clinical Decision Support System to Improve End-of-Life Care"],"prefix":"10.1055","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6281-0848","authenticated-orcid":false,"given":"Robert P.","family":"Pierce","sequence":"additional","affiliation":[{"name":"Department of Family and Community Medicine, University of Missouri, Columbia, Missouri, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adam","family":"Kell","sequence":"additional","affiliation":[{"name":"Value Driven Outcomes and Analytics, University of Missouri Health Care, Columbia, Missouri, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bernie","family":"Eskridge","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, University of Missouri, Columbia, Missouri, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lea","family":"Brandt","sequence":"additional","affiliation":[{"name":"Center for Health Ethics, University of Missouri, Columbia, Missouri, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kevin W.","family":"Clary","sequence":"additional","affiliation":[{"name":"Department of Medicine, University of Missouri, Columbia, Missouri, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kevin","family":"Craig","sequence":"additional","affiliation":[{"name":"Department of Family and Community Medicine, University of Missouri, Columbia, Missouri, United States"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"194","published-online":{"date-parts":[[2025,11,7]]},"reference":[{"issue":"08","key":"ref1","doi-asserted-by":"crossref","first-page":"733","DOI":"10.1056\/NEJMoa1000678","article-title":"Early palliative care for patients with metastatic non-small-cell lung cancer","volume":"363","author":"J S Temel","year":"2010","journal-title":"N Engl J Med"},{"issue":"05","key":"ref2","doi-asserted-by":"crossref","first-page":"1102","DOI":"10.1097\/CCM.0000000000000852","article-title":"Estimating the effect of palliative care interventions and advance care planning on ICU utilization: a systematic review","volume":"43","author":"N Khandelwal","year":"2015","journal-title":"Crit Care Med"},{"issue":"01","key":"ref3","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1089\/jpm.2016.0330","article-title":"Palliative care interventions for patients with heart failure: A systematic review and meta-analysis","volume":"20","author":"M S Diop","year":"2017","journal-title":"J Palliat Med"},{"issue":"20","key":"ref4","doi-asserted-by":"crossref","first-page":"2104","DOI":"10.1001\/jama.2016.16840","article-title":"Association between palliative care and patient and caregiver outcomes: a systematic review and meta-analysis","volume":"316","author":"D Kavalieratos","year":"2016","journal-title":"JAMA"},{"issue":"04","key":"ref5","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1634\/theoncologist.2016-0277","article-title":"End-of-life care matters: Palliative cancer care results in better care and lower costs","volume":"22","author":"S Dalal","year":"2017","journal-title":"Oncologist"},{"issue":"06","key":"ref6","doi-asserted-by":"crossref","first-page":"820","DOI":"10.1001\/jamainternmed.2018.0750","article-title":"Economics of palliative care for hospitalized adults with serious illness: A meta-analysis","volume":"178","author":"P May","year":"2018","journal-title":"JAMA Intern Med"},{"issue":"04","key":"ref7","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1177\/1049909119867904","article-title":"Differential utilization of palliative care consultation between medical and surgical services","volume":"37","author":"M-T Nguyen","year":"2020","journal-title":"Am J Hosp Palliat Care"},{"issue":"03","key":"ref8","doi-asserted-by":"crossref","first-page":"594","DOI":"10.5500\/wjt.v6.i3.594","article-title":"Underutilization of palliative care services in the liver transplant population","volume":"6","author":"P Kathpalia","year":"2016","journal-title":"World J Transplant"},{"issue":"03","key":"ref9","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1001\/jama.2018.10062","article-title":"The status of end-of-life care in the United States: The glass is half full","volume":"320","author":"E J Emanuel","year":"2018","journal-title":"JAMA"},{"issue":"03","key":"ref10","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1177\/1049909117715215","article-title":"Awareness and misperceptions of hospice and palliative care: A population-based survey study","volume":"35","author":"A Shalev","year":"2018","journal-title":"Am J Hosp Palliat Care"},{"key":"ref11","volume-title":"Improving Medication Use and Outcome with Clinical Decision Support: A Step-by-Step Guide","author":"J A Osheroff","year":"2009"},{"issue":"01","key":"ref12","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1097\/00005650-200301000-00010","article-title":"Burden of illness score for elderly persons: risk adjustment incorporating the cumulative impact of diseases, physiologic abnormalities, and functional impairments","volume":"41","author":"S K Inouye","year":"2003","journal-title":"Med Care"},{"issue":"05","key":"ref13","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1016\/j.amjmed.2006.09.021","article-title":"A prognostic model for 1-year mortality in older adults after hospital discharge","volume":"120","author":"S K Levine","year":"2007","journal-title":"Am J Med"},{"issue":"23","key":"ref14","doi-asserted-by":"crossref","first-page":"2987","DOI":"10.1001\/jama.285.23.2987","article-title":"Development and validation of a prognostic index for 1-year mortality in older adults after hospitalization","volume":"285","author":"L C Walter","year":"2001","journal-title":"JAMA"},{"issue":"10","key":"ref15","doi-asserted-by":"crossref","first-page":"e1915997","DOI":"10.1001\/jamanetworkopen.2019.15997","article-title":"Machine learning approaches to predict 6-month mortality among patients with cancer","volume":"2","author":"R B Parikh","year":"2019","journal-title":"JAMA Netw Open"},{"issue":"03","key":"ref16","doi-asserted-by":"crossref","first-page":"192","DOI":"10.7763\/IJMLC.2015.V5.506","article-title":"Short-term mortality prediction for elderly patients using Medicare claims data","volume":"5","author":"M Makar","year":"2015","journal-title":"Int J Mach Learn Comput"},{"issue":"01","key":"ref17","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1089\/rej.2007.0569","article-title":"Development and validation of a multidimensional prognostic index for one-year mortality from comprehensive geriatric assessment in hospitalized older patients","volume":"11","author":"A Pilotto","year":"2008","journal-title":"Rejuvenation Res"},{"key":"ref18","first-page":"S16","article-title":"Prediction of survival for older hospitalized patients: the HELP survival model. Hospitalized Elderly Longitudinal Project","volume":"48","author":"J M Teno","year":"2000","journal-title":"J Am Geriatr Soc"},{"key":"ref19","first-page":"S429","article-title":"Derivation and validation of a mortality-risk index from a cohort of frail elderly patients hospitalised in medical wards via emergencies: The SAFES study","volume":"13","author":"M Dram\u00e9","year":"2009","journal-title":"J Nutr Health Aging"},{"issue":"08","key":"ref20","doi-asserted-by":"crossref","first-page":"991.e9","DOI":"10.1016\/j.amjmed.2017.03.008","article-title":"The HOMR-now! Model accurately predicts 1-year death risk for hospitalized patients on admission","volume":"130","author":"C van Walraven","year":"2017","journal-title":"Am J Med"},{"key":"ref21","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1016\/j.archger.2017.07.008","article-title":"Development and validation of a prognostic index for 6- and 12-month mortality in hospitalized older adults","volume":"73","author":"E Gallego Gonz\u00e1lez","year":"2017","journal-title":"Arch Gerontol Geriatr"},{"issue":"03","key":"ref22","doi-asserted-by":"crossref","first-page":"e0214365","DOI":"10.1371\/journal.pone.0214365","article-title":"Prediction of premature all-cause mortality: A prospective general population cohort study comparing machine-learning and standard epidemiological approaches","volume":"14","author":"S F Weng","year":"2019","journal-title":"PLoS ONE"},{"issue":"03","key":"ref23","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1111\/acem.12876","article-title":"Prediction of in-hospital mortality in emergency department patients with sepsis: A local big data-driven, machine learning approach","volume":"23","author":"R A Taylor","year":"2016","journal-title":"Acad Emerg Med"},{"issue":"03","key":"ref24","doi-asserted-by":"crossref","first-page":"e287","DOI":"10.1016\/j.jpainsymman.2021.11.006","article-title":"A comparison of models predicting one-year mortality at time of admission","volume":"63","author":"R P Pierce","year":"2022","journal-title":"J Pain Symptom Manage"},{"issue":"12","key":"ref25","doi-asserted-by":"crossref","first-page":"971","DOI":"10.1136\/bmjqs-2018-009285","article-title":"mHOMR: A feasibility study of an automated system for identifying inpatients having an elevated risk of 1-year mortality","volume":"28","author":"P Wegier","year":"2019","journal-title":"BMJ Qual Saf"},{"key":"ref26","doi-asserted-by":"crossref","first-page":"e969","DOI":"10.1136\/bmjspcare-2020-002870","article-title":"mHOMR: A prospective observational study of an automated mortality prediction model to identify patients with unmet palliative needs","volume":"14","author":"P Wegier","year":"2024","journal-title":"BMJ Support Palliat Care"},{"issue":"05","key":"ref27","doi-asserted-by":"crossref","first-page":"1040","DOI":"10.1055\/s-0042-1758222","article-title":"Improving the user experience with discount site-specific user testing","volume":"13","author":"R P Pierce","year":"2022","journal-title":"Appl Clin Inform"},{"issue":"01","key":"ref29","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1038\/s41746-020-0253-3","article-title":"Presenting machine learning model information to clinical end users with model facts labels","volume":"3","author":"M P Sendak","year":"2020","journal-title":"NPJ Digit Med"},{"issue":"11","key":"ref30","doi-asserted-by":"crossref","first-page":"2366","DOI":"10.1093\/jamia\/ocab140","article-title":"Expected clinical utility of automatable prediction models for improving palliative and end-of-life care outcomes: Toward routine decision analysis before implementation","volume":"28","author":"R Taseen","year":"2021","journal-title":"J Am Med Inform Assoc"},{"issue":"10","key":"ref31","doi-asserted-by":"crossref","first-page":"e003567","DOI":"10.1136\/bmjgh-2020-003567","article-title":"Use of interrupted time series methods in the evaluation of health system quality improvement interventions: a methodological systematic review","volume":"5","author":"C Hategeka","year":"2020","journal-title":"BMJ Glob Health"},{"issue":"06","key":"ref32","doi-asserted-by":"crossref","first-page":"2082","DOI":"10.1093\/ije\/dyy135","article-title":"The use of controls in interrupted time series studies of public health interventions","volume":"47","author":"J Lopez Bernal","year":"2018","journal-title":"Int J Epidemiol"},{"issue":"01","key":"ref33","doi-asserted-by":"crossref","first-page":"185","DOI":"10.2307\/2951684","article-title":"Testing the autocorrelation structure of disturbances in ordinary least squares and instrumental variables regressions","volume":"60","author":"R E Cumby","year":"1992","journal-title":"Econometrica"},{"issue":"02","key":"ref34","doi-asserted-by":"crossref","first-page":"480","DOI":"10.1177\/1536867X1501500208","article-title":"Conducting interrupted time-series analysis for single- and multiple-group comparisons","volume":"15","author":"A Linden","year":"2015","journal-title":"Stata J"},{"issue":"S 04","key":"ref36","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1186\/s12911-018-0677-8","article-title":"Improving palliative care with deep learning","volume":"18","author":"A Avati","year":"2018","journal-title":"BMC Med Inform Decis Mak"},{"issue":"06","key":"ref37","doi-asserted-by":"crossref","first-page":"1149","DOI":"10.1093\/jamia\/ocaa318","article-title":"A framework for making predictive models useful in practice","volume":"28","author":"K Jung","year":"2021","journal-title":"J Am Med Inform Assoc"},{"issue":"09","key":"ref38","doi-asserted-by":"crossref","first-page":"1841","DOI":"10.1007\/s11606-019-05169-2","article-title":"Electronic health record mortality prediction model for targeted palliative care among hospitalized medical patients: A pilot quasi-experimental study","volume":"34","author":"K R Courtright","year":"2019","journal-title":"J Gen Intern Med"},{"issue":"12","key":"ref39","doi-asserted-by":"crossref","first-page":"e204759","DOI":"10.1001\/jamaoncol.2020.4759","article-title":"Effect of Integrating Machine Learning Mortality Estimates With Behavioral Nudges to Clinicians on Serious Illness Conversations Among Patients With Cancer: A Stepped-Wedge Cluster Randomized Clinical Trial","volume":"6","author":"C R Manz","year":"2020","journal-title":"JAMA Oncol"},{"issue":"02","key":"ref40","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1136\/bmjinnov-2019-000359","article-title":"Bridging the implementation gap of machine learning in healthcare","volume":"6","author":"M G Seneviratne","year":"2020","journal-title":"BMJ Innov"},{"key":"ref41","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.eswa.2019.05.028","article-title":"A comparison of random forest variable selection methods for classification prediction modeling","volume":"134","author":"J L Speiser","year":"2019","journal-title":"Expert Syst Appl"},{"key":"ref42","volume-title":"Coronavirus Resource Center","author":"Johns Hopkins University and Medicine","year":"2024"},{"issue":"01","key":"ref43","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1186\/s12911-024-02428-z","article-title":"Susceptibility of AutoML mortality prediction algorithms to model drift caused by the COVID pandemic","volume":"24","author":"S M Kagerbauer","year":"2024","journal-title":"BMC Med Inform Decis Mak"},{"issue":"09","key":"ref44","doi-asserted-by":"crossref","first-page":"e27798","DOI":"10.2196\/27798","article-title":"Predicting the mortality and readmission of in-hospital cardiac arrest patients with electronic health records: A machine learning approach","volume":"23","author":"C-Y Chi","year":"2021","journal-title":"J Med Internet Res"},{"key":"ref45","doi-asserted-by":"crossref","first-page":"106077","DOI":"10.1016\/j.compbiomed.2022.106077","article-title":"Improving mortality prediction in acute pancreatitis by machine learning and data augmentation","volume":"150","author":"M AB Hameed","year":"2022","journal-title":"Comput Biol Med"},{"issue":"01","key":"ref46","doi-asserted-by":"crossref","first-page":"9307475","DOI":"10.1155\/2021\/9307475","article-title":"Handling censoring and censored data in survival analysis: A standalone systematic literature review","volume":"2021","author":"A J Turkson","year":"2021","journal-title":"Int J Math Math Sci"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/a-2630-3204.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,8]],"date-time":"2025-11-08T00:02:04Z","timestamp":1762560124000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/a-2630-3204"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10]]},"references-count":44,"journal-issue":{"issue":"05","published-online":{"date-parts":[[2025,10,17]]},"published-print":{"date-parts":[[2025,10]]}},"URL":"https:\/\/doi.org\/10.1055\/a-2630-3204","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10]]}}}