{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T02:56:57Z","timestamp":1780973817155,"version":"3.54.1"},"reference-count":23,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2021,2,21]],"date-time":"2021-02-21T00:00:00Z","timestamp":1613865600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"name":"Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,6,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Access to palliative care (PC) is important for many patients with uncontrolled symptom burden from serious or complex illness. However, many patients who could benefit from PC do not receive it early enough or at all. We sought to address this problem by building a predictive model into a comprehensive clinical framework with the aims to (i) identify in-hospital patients likely to benefit from a PC consult, and (ii) intervene on such patients by contacting their care team.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>Electronic health record data for 68\u00a0349 inpatient encounters in 2017 at a large hospital were used to train a model to predict the need for PC consult. This model was published as a web service, connected to institutional data pipelines, and consumed by a downstream display application monitored by the PC team. For those patients that the PC team deems appropriate, a team member then contacts the patient\u2019s corresponding care team.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>Training performance AUC based on a 20% holdout validation set was 0.90. The most influential variables were previous palliative care, hospital unit, Albumin, Troponin, and metastatic cancer. The model has been successfully integrated into the clinical workflow making real-time predictions on hundreds of patients per day. The model had an \u201cin-production\u201d AUC of 0.91. A clinical trial is currently underway to assess the effect on clinical outcomes.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusions<\/jats:title>\n                  <jats:p>A machine learning model can effectively predict the need for an inpatient PC consult and has been successfully integrated into practice to refer new patients to PC.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamia\/ocaa211","type":"journal-article","created":{"date-parts":[[2021,2,16]],"date-time":"2021-02-16T21:24:09Z","timestamp":1613510649000},"page":"1065-1073","source":"Crossref","is-referenced-by-count":43,"title":["Improving the delivery of palliative care through predictive modeling and healthcare informatics"],"prefix":"10.1093","volume":"28","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7800-455X","authenticated-orcid":false,"given":"Dennis H","family":"Murphree","sequence":"first","affiliation":[{"name":"Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA"},{"name":"Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Patrick M","family":"Wilson","sequence":"additional","affiliation":[{"name":"Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shusaku W","family":"Asai","sequence":"additional","affiliation":[{"name":"Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel J","family":"Quest","sequence":"additional","affiliation":[{"name":"Information Technology, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yaxiong","family":"Lin","sequence":"additional","affiliation":[{"name":"Information Technology, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Piyush","family":"Mukherjee","sequence":"additional","affiliation":[{"name":"Information Technology, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nirmal","family":"Chhugani","sequence":"additional","affiliation":[{"name":"Information Technology, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jacob J","family":"Strand","sequence":"additional","affiliation":[{"name":"Division of Palliative Care, Department of Internal Medicine, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gabriel","family":"Demuth","sequence":"additional","affiliation":[{"name":"Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Mead","sequence":"additional","affiliation":[{"name":"Information Technology, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brian","family":"Wright","sequence":"additional","affiliation":[{"name":"Information Technology, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrew","family":"Harrison","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jalal","family":"Soleimani","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vitaly","family":"Herasevich","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Brian W","family":"Pickering","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Curtis B","family":"Storlie","sequence":"additional","affiliation":[{"name":"Department of Health Sciences Research, Mayo Clinic, Rochester, Minnesota, USA"},{"name":"Mayo Clinic Robert D. and Patricia E. Kern Center for the Science of Health Care Delivery, Mayo Clinic, Rochester, Minnesota, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,2,21]]},"reference":[{"issue":"7","key":"2021061318595066600_ocaa211-B1","doi-asserted-by":"crossref","first-page":"587","DOI":"10.1177\/0269216316673547","article-title":"Dying in the hospital setting: a meta-synthesis identifying the elements of end-of-life care that patients and their families describe as being important","volume":"31","author":"Virdun","year":"2017","journal-title":"Palliat Med"},{"issue":"1","key":"2021061318595066600_ocaa211-B2","first-page":"d1773","article-title":"Transitions to palliative care in acute hospitals in England: qualitative study","volume":"342","author":"Gott","year":"2011","journal-title":"BMJ Support Palliat Care"},{"issue":"3","key":"2021061318595066600_ocaa211-B3","doi-asserted-by":"crossref","first-page":"727","DOI":"10.1016\/S0885-3924(01)00334-7","article-title":"Preparing for the end of life: preferences of patients, families, physicians, and other care providers","volume":"22","author":"Steinhauser","year":"2001","journal-title":"J Pain Symptom Manage"},{"issue":"1","key":"2021061318595066600_ocaa211-B4","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1089\/jpm.2015.0351","article-title":"The growth of palliative care in US hospitals: a status report","volume":"19","author":"Dumanovsky","year":"2016","journal-title":"J Palliat Med"},{"issue":"7233","key":"2021061318595066600_ocaa211-B5","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1136\/bmj.320.7233.469","article-title":"Extent and determinants of error in doctors\u2019 prognoses in terminally ill patients: prospective cohort study","volume":"320","author":"Christakis","year":"2000","journal-title":"BMJ"},{"issue":"4","key":"2021061318595066600_ocaa211-B6","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1016\/j.jpainsymman.2011.01.012","article-title":"Clinician accuracy when estimating survival duration: the role of the patient\u2019s performance status and time-based prognostic categories","volume":"42","author":"Selby","year":"2011","journal-title":"J Pain Symptom Manage"},{"issue":"1","key":"2021061318595066600_ocaa211-B7","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1002\/(SICI)1097-0142(19990701)86:1<170::AID-CNCR23>3.0.CO;2-S","article-title":"The relative accuracy of the clinical estimation of the duration of life for patients with end of life cancer","volume":"86","author":"Vigano","year":"1999","journal-title":"Cancer"},{"issue":"25","key":"2021061318595066600_ocaa211-B8","doi-asserted-by":"crossref","first-page":"2582","DOI":"10.1056\/NEJMcp035232","article-title":"Clinical practice. Palliative care","volume":"350","author":"Morrison","year":"2004","journal-title":"N Engl J Med"},{"issue":"S4","key":"2021061318595066600_ocaa211-B9","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1186\/s12911-018-0677-8","article-title":"Improving palliative care with deep learning","volume":"18","author":"Avati","year":"2018","journal-title":"BMC Med Inform Decis Mak"},{"key":"2021061318595066600_ocaa211-B10","doi-asserted-by":"crossref","first-page":"103115","DOI":"10.1016\/j.jbi.2019.103115","article-title":"Predicting need for advanced illness or palliative care in a primary care population using electronic health record data","volume":"92","author":"Jung","year":"2019","journal-title":"J Biomed Inform"},{"issue":"2","key":"2021061318595066600_ocaa211-B11","doi-asserted-by":"crossref","first-page":"280","DOI":"10.1016\/j.mayocp.2016.11.007","article-title":"Understanding palliative care and hospice: a review for primary care providers","volume":"92","author":"Buss","year":"2017","journal-title":"Mayo Clin Proc"},{"issue":"4","key":"2021061318595066600_ocaa211-B12","first-page":"348","article-title":"The role of palliative care at the end of life","volume":"11","author":"Rome","year":"2011","journal-title":"Ochsner J"},{"key":"2021061318595066600_ocaa211-B13","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-3294-8","volume-title":"Modeling Survival Data: Extending the Cox Model","author":"Therneau","year":"2000"},{"key":"2021061318595066600_ocaa211-B14","doi-asserted-by":"crossref","first-page":"21","DOI":"10.3389\/fnbot.2013.00021","article-title":"Gradient boosting machines, a tutorial","volume":"7","author":"Natekin","year":"2013","journal-title":"Front Neurorobot"},{"issue":"3","key":"2021061318595066600_ocaa211-B15","first-page":"55","article-title":"gbm: generalized boosted regression models","volume":"1","author":"Ridgeway","year":"2006","journal-title":"R package version"},{"key":"2021061318595066600_ocaa211-B16","volume-title":"R: A Language and Environment for Statistical Computing","year":"2014"},{"key":"2021061318595066600_ocaa211-B17","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4614-6849-3","volume-title":"Applied Predictive Modeling","author":"Kuhn","year":"2013"},{"key":"2021061318595066600_ocaa211-B18","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4614-7138-7","volume-title":"An Introduction to Statistical Learning: With Applications in R","author":"James","year":"2013"},{"issue":"4","key":"2021061318595066600_ocaa211-B19","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1007\/s13312-011-0055-4","article-title":"Receiver operating characteristic (ROC) curve for medical researchers","volume":"48","author":"Kumar","year":"2011","journal-title":"Indian Pediatr"},{"key":"2021061318595066600_ocaa211-B20","first-page":"6112","article-title":"Deploying predictive models in a healthcare environment\u2014an open source approach","volume":"2018","author":"Murphree","year":"2018","journal-title":"Conf Proc IEEE Eng Med Biol Soc"},{"issue":"6","key":"2021061318595066600_ocaa211-B21","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1002\/spe.2466","article-title":"Definition of REST web services with JSON schema","volume":"47","author":"Barbaglia","year":"2017","journal-title":"Softw Pract Exper"},{"key":"2021061318595066600_ocaa211-B22","author":"Smeets","year":"2017"},{"issue":"11","key":"2021061318595066600_ocaa211-B23","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1016\/j.ress.2009.05.007","article-title":"Implementation and evaluation of nonparametric regression procedures for sensitivity analysis of computationally demanding models","volume":"94","author":"Storlie","year":"2009","journal-title":"Reliab Eng Syst Safe"}],"container-title":["Journal of the American Medical Informatics Association"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/6\/1065\/38615528\/ocaa211.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/jamia\/article-pdf\/28\/6\/1065\/38615528\/ocaa211.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,13]],"date-time":"2021-06-13T19:01:55Z","timestamp":1623610915000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamia\/article\/28\/6\/1065\/6145883"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,21]]},"references-count":23,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,2,21]]},"published-print":{"date-parts":[[2021,6,12]]}},"URL":"https:\/\/doi.org\/10.1093\/jamia\/ocaa211","relation":{},"ISSN":["1527-974X"],"issn-type":[{"value":"1527-974X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,6,1]]},"published":{"date-parts":[[2021,2,21]]}}}