{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T07:44:33Z","timestamp":1774424673547,"version":"3.50.1"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,4,3]],"date-time":"2020-04-03T00:00:00Z","timestamp":1585872000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,4,3]],"date-time":"2020-04-03T00:00:00Z","timestamp":1585872000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Hospital systems, payers, and regulators have focused on reducing length of stay (LOS) and early readmission, with uncertain benefit. Interpretable machine learning (ML) may assist in transparently identifying the risk of important outcomes. We conducted a retrospective cohort study of hospitalizations at a tertiary academic medical center and its branches from January 2011 to May 2018. A consecutive sample of all hospitalizations in the study period were included. Algorithms were trained on medical, sociodemographic, and institutional variables to predict readmission, length of stay (LOS), and death within 48\u201372\u2009h. Prediction performance was measured by area under the receiver operator characteristic curve (AUC), Brier score loss (BSL), which measures how well predicted probability matches observed probability, and other metrics. Interpretations were generated using multiple feature extraction algorithms. The study cohort included 1,485,880 hospitalizations for 708,089 unique patients (median age of 59 years, first and third quartiles (QI) [39, 73]; 55.6% female; 71% white). There were 211,022 30-day readmissions for an overall readmission rate of 14% (for patients \u226565 years: 16%). Median LOS, including observation and labor and delivery patients, was 2.94 days (QI [1.67, 5.34]), or, if these patients are excluded, 3.71 days (QI [2.15, 6.51]). Predictive performance was as follows: 30-day readmission (AUC 0.76\/BSL 0.11); LOS\u2009&gt;\u20095 days (AUC 0.84\/BSL 0.15); death within 48\u201372\u2009h (AUC 0.91\/BSL 0.001). Explanatory diagrams showed factors that impacted each prediction.<\/jats:p>","DOI":"10.1038\/s41746-020-0249-z","type":"journal-article","created":{"date-parts":[[2020,4,3]],"date-time":"2020-04-03T10:07:13Z","timestamp":1585908433000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":67,"title":["Personalized predictions of patient outcomes during and after hospitalization using artificial intelligence"],"prefix":"10.1038","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1363-7452","authenticated-orcid":false,"given":"C. Beau","family":"Hilton","sequence":"first","affiliation":[]},{"given":"Alex","family":"Milinovich","sequence":"additional","affiliation":[]},{"given":"Christina","family":"Felix","sequence":"additional","affiliation":[]},{"given":"Nirav","family":"Vakharia","sequence":"additional","affiliation":[]},{"given":"Timothy","family":"Crone","sequence":"additional","affiliation":[]},{"given":"Chris","family":"Donovan","sequence":"additional","affiliation":[]},{"given":"Andrew","family":"Proctor","sequence":"additional","affiliation":[]},{"given":"Aziz","family":"Nazha","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,4,3]]},"reference":[{"key":"249_CR1","doi-asserted-by":"publisher","first-page":"733","DOI":"10.7326\/M17-3108","volume":"168","author":"AD Auerbach","year":"2018","unstructured":"Auerbach, A. D., Neinstein, A. & Khanna, R. Balancing innovation and safety when integrating digital tools into health care. Ann. Intern. Med. 168, 733\u2013734 (2018).","journal-title":"Ann. Intern. Med."},{"key":"249_CR2","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1001\/jama.2017.7797","volume":"318","author":"F Cabitza","year":"2017","unstructured":"Cabitza, F., Rasoini, R. & Gensini, G. F. Unintended consequences of machine learning in medicine. JAMA 318, 517 (2017).","journal-title":"JAMA"},{"key":"249_CR3","doi-asserted-by":"publisher","first-page":"25","DOI":"10.1001\/jama.2015.6177","volume":"314","author":"AD Sniderman","year":"2015","unstructured":"Sniderman, A. D., D\u2019Agostino, R. B. Sr & Pencina, M. J. The role of physicians in the era of predictive analytics. JAMA 314, 25\u201326 (2015).","journal-title":"JAMA"},{"key":"249_CR4","doi-asserted-by":"publisher","first-page":"2542","DOI":"10.1001\/jama.2018.19232","volume":"320","author":"RK Wadhera","year":"2018","unstructured":"Wadhera, R. K. et al. Association of the Hospital Readmissions Reduction Program with mortality among Medicare beneficiaries hospitalized for heart failure, acute myocardial infarction, and pneumonia. JAMA 320, 2542\u20132552 (2018).","journal-title":"JAMA"},{"key":"249_CR5","unstructured":"Bojarski, M. et al. End to end learning for self-driving cars. Preprint at https:\/\/arxiv.org\/abs\/1604.07316 (2016)."},{"key":"249_CR6","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1016\/j.knosys.2013.03.012","volume":"46","author":"J Bobadilla","year":"2013","unstructured":"Bobadilla, J., Ortega, F., Hernando, A. & Guti\u00e9rrez, A. Recommender systems survey. Knowledge-Based Syst. 46, 109\u2013132 (2013).","journal-title":"Knowledge-Based Syst."},{"key":"249_CR7","doi-asserted-by":"publisher","first-page":"1140","DOI":"10.1126\/science.aar6404","volume":"362","author":"D Silver","year":"2018","unstructured":"Silver, D. et al. A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Science 362, 1140\u20131144 (2018).","journal-title":"Science"},{"key":"249_CR8","doi-asserted-by":"publisher","first-page":"2402","DOI":"10.1001\/jama.2016.17216","volume":"316","author":"V Gulshan","year":"2016","unstructured":"Gulshan, V. et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316, 2402 (2016).","journal-title":"JAMA"},{"key":"249_CR9","doi-asserted-by":"publisher","first-page":"1559","DOI":"10.1038\/s41591-018-0177-5","volume":"24","author":"N Coudray","year":"2018","unstructured":"Coudray, N. et al. Classification and mutation prediction from non\u2013small cell lung cancer histopathology images using deep learning. Nat. Med. 24, 1559\u20131567 (2018).","journal-title":"Nat. Med."},{"key":"249_CR10","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nature21056","volume":"542","author":"A Esteva","year":"2017","unstructured":"Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115\u2013118 (2017).","journal-title":"Nature"},{"key":"249_CR11","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","volume":"1","author":"A Rajkomar","year":"2018","unstructured":"Rajkomar, A. et al. Scalable and accurate deep learning with electronic health records. NPJ Digital Med. 1, 18 (2018).","journal-title":"NPJ Digital Med."},{"key":"249_CR12","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1016\/j.cmpb.2018.06.006","volume":"164","author":"A Artetxe","year":"2018","unstructured":"Artetxe, A., Beristain, A. & Grana, M. Predictive models for hospital readmission risk: a systematic review of methods. Comput. Methods Prog. Biomed. 164, 49\u201364 (2018).","journal-title":"Comput. Methods Prog. Biomed."},{"key":"249_CR13","doi-asserted-by":"publisher","first-page":"128","DOI":"10.1097\/EDE.0b013e3181c30fb2","volume":"21","author":"EW Steyerberg","year":"2010","unstructured":"Steyerberg, E. W. et al. Assessing the performance of prediction models: a framework for some traditional and novel measures. Epidemiology 21, 128 (2010).","journal-title":"Epidemiology"},{"key":"249_CR14","doi-asserted-by":"publisher","first-page":"632","DOI":"10.1001\/jamainternmed.2013.3023","volume":"173","author":"J Donz\u00e9","year":"2013","unstructured":"Donz\u00e9, J., Aujesky, D., Williams, D. & Schnipper, J. L. Potentially avoidable 30-day hospital readmissions in medical patients: derivation and validation of a prediction model. JAMA Intern. Med. 173, 632\u2013638 (2013).","journal-title":"JAMA Intern. Med."},{"key":"249_CR15","doi-asserted-by":"publisher","first-page":"1095","DOI":"10.1001\/jamainternmed.2014.1608","volume":"174","author":"AL Leppin","year":"2014","unstructured":"Leppin, A. L. et al. Preventing 30-day hospital readmissions: a systematic review and meta-analysis of randomized trials. JAMA Intern. Med. 174, 1095\u20131107 (2014).","journal-title":"JAMA Intern. Med."},{"key":"249_CR16","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1097\/MLR.0000000000000665","volume":"55","author":"RE Burke","year":"2017","unstructured":"Burke, R. E. et al. The HOSPITAL score predicts potentially preventable 30-day readmissions in conditions targeted by the hospital readmissions reduction program. Med. Care 55, 285 (2017).","journal-title":"Med. Care"},{"key":"249_CR17","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1001\/jamainternmed.2015.7863","volume":"176","author":"AD Auerbach","year":"2016","unstructured":"Auerbach, A. D. et al. Preventability and causes of readmissions in a national cohort of general medicine patients. JAMA Intern. Med. 176, 484\u2013493 (2016).","journal-title":"JAMA Intern. Med."},{"key":"249_CR18","doi-asserted-by":"publisher","first-page":"e177","DOI":"10.1200\/JOP.2014.001546","volume":"11","author":"ND Saunders","year":"2015","unstructured":"Saunders, N. D. et al. Examination of unplanned 30-day readmissions to a comprehensive cancer hospital. J. Oncol. Pract. 11, e177\u2013e181 (2015).","journal-title":"J. Oncol. Pract."},{"key":"249_CR19","doi-asserted-by":"publisher","first-page":"k1479","DOI":"10.1136\/bmj.k1479","volume":"361","author":"D Agniel","year":"2018","unstructured":"Agniel, D., Kohane, I. S. & Weber, G. M. Biases in electronic health record data due to processes within the healthcare system: retrospective observational study. BMJ 361, k1479 (2018).","journal-title":"BMJ"},{"key":"249_CR20","doi-asserted-by":"publisher","first-page":"799","DOI":"10.1136\/bmjqs-2016-006239","volume":"26","author":"CE Aubert","year":"2017","unstructured":"Aubert, C. E. et al. Simplification of the HOSPITAL score for predicting 30-day readmissions. BMJ Qual. Saf. 26, 799\u2013805 (2017).","journal-title":"BMJ Qual. Saf."},{"key":"249_CR21","doi-asserted-by":"publisher","first-page":"524","DOI":"10.1111\/jep.12656","volume":"23","author":"GM Garrison","year":"2017","unstructured":"Garrison, G. M., Robelia, P. M., Pecina, J. L. & Dawson, N. L. Comparing performance of 30-day readmission risk classifiers among hospitalized primary care patients. J. Eval. Clin. Pract. 23, 524\u2013529 (2017).","journal-title":"J. Eval. Clin. Pract."},{"key":"249_CR22","doi-asserted-by":"crossref","unstructured":"Sun, C., Shrivastava, A., Singh, S. & Gupta, A. Revisiting unreasonable effectiveness of data in deep learning era. In Proceedings of the IEEE International Conference on Computer Vision 843\u2013852 (IEEE, 2017).","DOI":"10.1109\/ICCV.2017.97"},{"key":"249_CR23","unstructured":"US Census Bureau. American community survey 5-year estimates, https:\/\/data.census.gov\/cedsci\/table?q=United%20States&tid=ACSDP5Y2015.DP05 (2015)."},{"key":"249_CR24","doi-asserted-by":"publisher","first-page":"21","DOI":"10.3389\/fnbot.2013.00021","volume":"7","author":"A Natekin","year":"2013","unstructured":"Natekin, A. & Knoll, A. Gradient boosting machines, a tutorial. Front. Neurorobotics 7, 21 (2013).","journal-title":"Front. Neurorobotics"},{"key":"249_CR25","unstructured":"Ke, G. et al. Lightgbm: a highly efficient gradient boosting decision tree. in Advances in Neural Information Processing Systems 3146\u20133154 (Neural Information Processing Systems Foundation, Inc., 2017)."},{"key":"249_CR26","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R. & Friedman, J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (Springer Science & Business Media, 2009).","DOI":"10.1007\/978-0-387-84858-7"},{"key":"249_CR27","doi-asserted-by":"publisher","first-page":"1538","DOI":"10.1214\/009053605000000255","volume":"33","author":"T Zhang","year":"2005","unstructured":"Zhang, T. & Yu, B., others. Boosting with early stopping: convergence and consistency. Ann. Stat. 33, 1538\u20131579 (2005).","journal-title":"Ann. Stat."},{"key":"249_CR28","unstructured":"Lundberg, S. M. & Lee, S.-I. A unified approach to interpreting model predictions. in Advances in Neural Information Processing Systems 30 (eds Guyon, I. et al.) 4765\u20134774 (Curran Associates, Inc., 2017)."},{"key":"249_CR29","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa, F. et al. Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825\u20132830 (2011).","journal-title":"J. Mach. Learn. Res."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0249-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0249-z","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0249-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T02:08:03Z","timestamp":1670378883000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-020-0249-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,3]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["249"],"URL":"https:\/\/doi.org\/10.1038\/s41746-020-0249-z","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,4,3]]},"assertion":[{"value":"23 September 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"28 February 2020","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"3 April 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"51"}}