{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T22:19:22Z","timestamp":1775168362476,"version":"3.50.1"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,6,5]],"date-time":"2023-06-05T00:00:00Z","timestamp":1685923200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,6,5]],"date-time":"2023-06-05T00:00:00Z","timestamp":1685923200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1761009"],"award-info":[{"award-number":["1761009"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Advanced machine learning models have received wide attention in assisting medical decision making due to the greater accuracy they can achieve. However, their limited interpretability imposes barriers for practitioners to adopt them. Recent advancements in interpretable machine learning tools allow us to look inside the black box of advanced prediction methods to extract interpretable models while maintaining similar prediction accuracy, but few studies have investigated the specific hospital readmission prediction problem with this spirit.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>Our goal is to develop a machine-learning (ML) algorithm that can predict 30- and 90- day hospital readmissions as accurately as black box algorithms while providing medically interpretable insights into readmission risk factors. Leveraging a state-of-art interpretable ML model, we use a two-step Extracted Regression Tree approach to achieve this goal. In the first step, we train a black box prediction algorithm. In the second step, we extract a regression tree from the output of the black box algorithm that allows direct interpretation of medically relevant risk factors. We use data from a large teaching hospital in Asia to learn the ML model and verify our two-step approach.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>The two-step method can obtain similar prediction performance as the best black box model, such as Neural Networks, measured by three metrics: accuracy, the Area Under the Curve (AUC) and the Area Under the Precision-Recall Curve (AUPRC), while maintaining interpretability. Further, to examine whether the prediction results match the known medical insights (i.e., the model is truly interpretable and produces reasonable results), we show that key readmission risk factors extracted by the two-step approach are consistent with those found in the medical literature.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The proposed two-step approach yields meaningful prediction results that are both accurate and interpretable. This study suggests a viable means to improve the trust of machine learning based models in clinical practice for predicting readmissions through the two-step approach.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-023-02193-5","type":"journal-article","created":{"date-parts":[[2023,6,5]],"date-time":"2023-06-05T14:03:00Z","timestamp":1685973780000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Interpretable machine learning models for hospital readmission prediction: a two-step extracted regression tree approach"],"prefix":"10.1186","volume":"23","author":[{"given":"Xiaoquan","family":"Gao","sequence":"first","affiliation":[]},{"given":"Sabriya","family":"Alam","sequence":"additional","affiliation":[]},{"given":"Pengyi","family":"Shi","sequence":"additional","affiliation":[]},{"given":"Franklin","family":"Dexter","sequence":"additional","affiliation":[]},{"given":"Nan","family":"Kong","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,6,5]]},"reference":[{"key":"2193_CR1","unstructured":"Centers for Medicare and Medicaid Services, Readmissions reduction program., 2012. https:\/\/www.cms.gov\/Medicare\/Medicare-Fee-for-Service-Payment\/AcuteInpatientPPS\/Readmissions-Reduction-Program.html."},{"issue":"15","key":"2193_CR2","doi-asserted-by":"publisher","first-page":"1366","DOI":"10.1056\/NEJMp1201598","volume":"366","author":"KE Joynt","year":"2012","unstructured":"Joynt KE, Ashish K. Jha. \u201cThirty-day readmissions\u2014truth and consequences. N Engl j med. 2012;366(15):1366\u201313.","journal-title":"N Engl j med"},{"key":"2193_CR3","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1016\/j.knosys.2018.01.027","volume":"146","author":"S Jiang","year":"2018","unstructured":"Jiang S, Chin KS, Qu G, Tsui KL. An integrated machine learning framework for hospital readmission prediction. Knowl Based Syst. 2018;146:73\u201390.","journal-title":"Knowl Based Syst"},{"key":"2193_CR4","unstructured":"Bastani H, Bastani O, Kim C. \u201cInterpreting predictive models for human-in-the-loop analytics.\u201c arXiv preprint arXiv:1705.08504 (2018): 1\u201345."},{"issue":"3","key":"2193_CR5","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1007\/s10994-015-5528-6","volume":"102","author":"B Ustun","year":"2016","unstructured":"Ustun B, Rudin C. Supersparse linear integer models for optimized medical scoring systems. Mach Learn. 2016;102(3):349\u201391.","journal-title":"Mach Learn"},{"key":"2193_CR6","unstructured":"Wang F, Rudin C. \u201cFalling rule lists.\u201c Artificial Intelligence and Statistics. PMLR, 2015."},{"key":"2193_CR7","unstructured":"Letham B et al. \u201cInterpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model.\u201c."},{"issue":"3","key":"2193_CR8","first-page":"258","volume":"33","author":"JW Thomas","year":"1996","unstructured":"Thomas JW. Does risk-adjusted readmission rate provide valid information on hospital quality? Inquiry. 1996;33(3):258\u201370.","journal-title":"Inquiry"},{"issue":"5","key":"2193_CR9","doi-asserted-by":"publisher","first-page":"500","DOI":"10.1161\/CIRCOUTCOMES.108.832949","volume":"2","author":"MM Desai","year":"2009","unstructured":"Desai MM, Stauffer BD, Feringa H, Schreiner GC. Statistical models and patient predictors of readmission for acute myocardial infarction a systematic review. Circ Cardiovasc Qual Outcomes. 2009;2(5):500\u20137.","journal-title":"Circ Cardiovasc Qual Outcomes"},{"issue":"4","key":"2193_CR10","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1080\/08998280.2008.11928429","volume":"21","author":"MD Silverstein","year":"2008","unstructured":"Silverstein MD, Qin H, Mercer SQ, Fong J, Haydar Z. Risk factors for 30-day hospital readmission in patients _ 65 years of age. Bayl Univ Med Cent Proc. 2008;21(4):363\u201372.","journal-title":"Bayl Univ Med Cent Proc"},{"issue":"3","key":"2193_CR11","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1007\/BF02598964","volume":"6","author":"RL Reed","year":"1991","unstructured":"Reed RL, Pearlman RA, Buchner DM. Risk factors for early unplanned hospital readmission in the elderly. J Gen Intern Med. 1991;6(3):223\u20138.","journal-title":"J Gen Intern Med"},{"issue":"1","key":"2193_CR12","first-page":"81","volume":"27","author":"JM Corrigan","year":"1992","unstructured":"Corrigan JM, Martin JB. Identification of factors associated with hospital readmission and development of a predictive model. Health Serv Res. 1992;27(1):81\u2013101.","journal-title":"Health Serv Res"},{"issue":"1","key":"2193_CR13","doi-asserted-by":"publisher","first-page":"13","DOI":"10.1016\/S0002-9343(99)00159-X","volume":"107","author":"ER Marcantonio","year":"1990","unstructured":"Marcantonio ER, McKean S, Goldfinger M, Kleefield S, Yurkofsky M, Brennan TA. Factors associated with unplanned hospital readmission among patients 65 years of age and older in a Medicare managed care plan. Am J Med. 1990;107(1):13\u20137.","journal-title":"Am J Med"},{"issue":"4","key":"2193_CR14","doi-asserted-by":"publisher","first-page":"220","DOI":"10.1159\/000022091","volume":"45","author":"LW Chu","year":"1999","unstructured":"Chu LW, Pei CK. Risk factors for early emergency hospital readmission in elderly medical patients. Gerontology. 1999;45(4):220\u20136.","journal-title":"Gerontology"},{"issue":"4","key":"2193_CR15","doi-asserted-by":"publisher","first-page":"550","DOI":"10.1086\/526526","volume":"46","author":"H Jasti","year":"2008","unstructured":"Jasti H, Mortensen EM, Obrosky DS, Kapoor WN, Fine MJ. Causes and risk factors for rehospitalization of patients hospitalized with community acquired pneumonia. Clin Infect Dis. 2008;46(4):550\u20136.","journal-title":"Clin Infect Dis"},{"issue":"11","key":"2193_CR16","doi-asserted-by":"publisher","first-page":"1113","DOI":"10.1016\/S0895-4356(00)00236-5","volume":"53","author":"DM Smith","year":"2000","unstructured":"Smith DM, Giobbie-Hurder A, Weinberger M, Oddone EZ, Henderson WG, Asch DA, et al. Predicting non-elective hospital readmissions: a multi-site study. Department of Veterans Affairs Cooperative Study Group on Primary Care and Readmissions. J Clin Epidemiol. 2000;53(11):1113\u20138.","journal-title":"J Clin Epidemiol"},{"issue":"3","key":"2193_CR17","first-page":"289","volume":"32","author":"HJ Oh","year":"1999","unstructured":"Oh HJ, Yu SH. A case-control study of unexpected readmission in a university hospital. Korean J Prev Med. 1999;32(3):289. \u2013 296 (Korean).","journal-title":"Korean J Prev Med"},{"issue":"1289","key":"2193_CR18","first-page":"32","volume":"122","author":"J Runball-Smith","year":"2009","unstructured":"Runball-Smith J, Hider P, Graham P. The readmission rate as an indicator of the quality of elective surgical inpatient care for the elderly in New Zealand. N Z Med J. 2009;122(1289):32\u20139.","journal-title":"N Z Med J"},{"issue":"10","key":"2193_CR19","doi-asserted-by":"publisher","first-page":"1482","DOI":"10.1016\/j.amjcard.2012.01.362","volume":"109","author":"CV Thakar","year":"2012","unstructured":"Thakar CV, Parikh PJ, Liu Y. Acute kidney injury (AKI) and risk of readmissions in patients with heart failure. Am J Cardiol. 2012;109(10):1482\u20136.","journal-title":"Am J Cardiol"},{"key":"2193_CR20","unstructured":"Kansagra D. Risk prediction models for hospital readmission: a systematic review. Evidence-based Synthesis Program. Department of Veterans Affairs Health Services Research & Development Service; October 2011."},{"issue":"6","key":"2193_CR21","doi-asserted-by":"publisher","first-page":"551","DOI":"10.1503\/cmaj.091117","volume":"182","author":"C van Walraven","year":"2010","unstructured":"van Walraven C, Dhalla IA, Bell C, Etchells E, Stiell IG, Zarnke K, Austin PC, Forster AJ. Derivation and validation of an index to predict early death or unplanned readmission after discharge from hospital to the community. Can Med Assoc J. 2010;182(6):551\u20137.","journal-title":"Can Med Assoc J"},{"key":"2193_CR22","doi-asserted-by":"publisher","first-page":"632","DOI":"10.1001\/jamainternmed.2013.3023","volume":"173","author":"J Donz\u00e9","year":"2013","unstructured":"Donz\u00e9 J, Aujesky D, Williams D, Schnipper JL. Potentially avoidable 30-day hospital readmissions in medical patients: derivation and validation of a prediction model. JAMA Intern Med. 2013;173:632\u20138.","journal-title":"JAMA Intern Med"},{"key":"2193_CR23","doi-asserted-by":"publisher","first-page":"259","DOI":"10.3961\/jpmph.2012.45.4.259","volume":"45","author":"EW Lee","year":"2012","unstructured":"Lee EW. Selecting the best prediction model for readmission. J Prev Med Public Health. 2012;45:259\u201366.","journal-title":"J Prev Med Public Health"},{"key":"2193_CR24","doi-asserted-by":"crossref","unstructured":"Hosseinzadeh A, Izadi M, Verma A, Precup D, Buckeridge D. Assessing the predictability of hospital readmission using machine learning. In: Munoz-Avila H, Stracuzzi D, editors. Proceedings of the Twenty-Fifth Innovative Applications of Artificial Intelligence Conference, July 14 \u2013 18, 2013, Bellevue, Washington. Published by The AAAI Press, Palo Alto, California.","DOI":"10.1609\/aaai.v27i2.18995"},{"key":"2193_CR25","doi-asserted-by":"crossref","unstructured":"Caruana R et al. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 1721\u20131730 (ACM, 2015).","DOI":"10.1145\/2783258.2788613"},{"key":"2193_CR26","volume-title":"Predicting 30-day risk and cost of \u201call-cause\u201d hospital readmissions","author":"S Sushmita","year":"2016","unstructured":"Sushmita S, et al. Predicting 30-day risk and cost of \u201call-cause\u201d hospital readmissions. In AAAI Workshop: Expanding the Boundaries of Health Informatics Using AI; 2016."},{"key":"2193_CR27","doi-asserted-by":"crossref","unstructured":"Wang H et al. Predicting hospital readmission via cost-sensitive deep learning. IEEE\/ACM Trans Comput Biol Bioinforma (2018).","DOI":"10.1109\/TCBB.2018.2827029"},{"key":"2193_CR28","doi-asserted-by":"publisher","first-page":"e0195024","DOI":"10.1371\/journal.pone.0195024","volume":"13","author":"C Xiao","year":"2018","unstructured":"Xiao C, Ma T, Dieng AB, Blei DM, Wang F. Readmission prediction via deep contextual embedding of clinical concepts. PLoS ONE. 2018;13:e0195024.","journal-title":"PLoS ONE"},{"key":"2193_CR29","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","volume":"1","author":"A Rajkomar","year":"2018","unstructured":"Rajkomar A, et al. Scalable and accurate deep learning with electronic health records. NPJ Digit Medicine. 2018;1:18.","journal-title":"NPJ Digit Medicine"},{"key":"2193_CR30","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1016\/j.cmpb.2018.06.006","volume":"164","author":"A Artetxe","year":"2018","unstructured":"Artetxe A, Beristain A, Grana M. Predictive models for hospital readmission risk: a systematic review of methods. Comput Methods Programs Biomed. 2018;164:49\u201364.","journal-title":"Comput Methods Programs Biomed"},{"issue":"5","key":"2193_CR31","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1016\/0021-9681(87)90171-8","volume":"40","author":"ME Charlson","year":"1987","unstructured":"Charlson ME, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J chronic Dis. 1987;40(5):373\u201383.","journal-title":"J chronic Dis"},{"issue":"9","key":"2193_CR32","doi-asserted-by":"publisher","first-page":"1044","DOI":"10.1001\/jama.306.9.1044","volume":"306","author":"JM Torpy","year":"2011","unstructured":"Torpy JM, Golub RM, Medical Specialties. JAMA. 2011;306(9):1044. https:\/\/doi.org\/10.1001\/jama.306.9.1044.","journal-title":"JAMA"},{"key":"2193_CR33","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4","volume":"15","author":"E Frank","year":"1996","unstructured":"Frank E, et al. Tutorial in biostatistics multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15:361\u201387.","journal-title":"Stat Med"},{"issue":"3","key":"2193_CR34","doi-asserted-by":"publisher","first-page":"771","DOI":"10.1111\/rssa.12576","volume":"183","author":"J Jung","year":"2020","unstructured":"Jung J, et al. Simple rules to guide expert classifications. J Royal Stat Society: Ser (Statistics Society). 2020;183(3):771\u2013800.","journal-title":"J Royal Stat Society: Ser (Statistics Society)"},{"issue":"3","key":"2193_CR35","doi-asserted-by":"publisher","first-page":"689","DOI":"10.1111\/rssa.12227","volume":"180","author":"J Zeng","year":"2017","unstructured":"Zeng J, Ustun B, Rudin C. Interpretable classification models for recidivism prediction. J Royal Stat Society: Ser (Statistics Society). 2017;180(3):689\u2013722.","journal-title":"J Royal Stat Society: Ser (Statistics Society)"},{"key":"2193_CR36","doi-asserted-by":"crossref","unstructured":"Seo S et al. \u201cInterpretable convolutional neural networks with dual local and global attention for review rating prediction.\u201c Proceedings of the eleventh ACM conference on recommender systems. 2017.","DOI":"10.1145\/3109859.3109890"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-023-02193-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-023-02193-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-023-02193-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,5]],"date-time":"2023-06-05T14:03:29Z","timestamp":1685973809000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-023-02193-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,5]]},"references-count":36,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["2193"],"URL":"https:\/\/doi.org\/10.1186\/s12911-023-02193-5","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,5]]},"assertion":[{"value":"27 July 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 May 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 June 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The need for ethics approval is waived by approval IRB-2021-976 of Purdue University Human Research Protection Program\/Institutional Review Board (IRB). Purdue IRB determined that the study does not meet the usual requirement for human subject research. The authors confirm that all methods were performed in accordance with the relevant guidelines and regulations.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"104"}}