{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,19]],"date-time":"2026-04-19T22:24:19Z","timestamp":1776637459987,"version":"3.51.2"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,8,16]],"date-time":"2021-08-16T00:00:00Z","timestamp":1629072000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,8,16]],"date-time":"2021-08-16T00:00:00Z","timestamp":1629072000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BioData Mining"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Early prediction of hospital mortality is crucial for ICU patients with sepsis. This study aimed to develop a novel blending machine learning (ML) model for hospital mortality prediction in ICU patients with sepsis.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>Two ICU databases were employed: eICU Collaborative Research Database (eICU-CRD) and Medical Information Mart for Intensive Care III (MIMIC-III). All adult patients who fulfilled Sepsis-3 criteria were identified. Samples from eICU-CRD constituted training set and samples from MIMIC-III constituted test set. Stepwise logistic regression model was used for predictor selection. Blending ML model which integrated nine sorts of basic ML models was developed for hospital mortality prediction in ICU patients with sepsis. Model performance was evaluated by various measures related to discrimination or calibration.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Twelve thousand five hundred fifty-eight patients from eICU-CRD were included as the training set, and 12,095 patients from MIMIC-III were included as the test set. Both the training set and the test set showed a hospital mortality of 17.9%. Maximum and minimum lactate, maximum and minimum albumin, minimum PaO2\/FiO2 and age were important predictors identified by both random forest and extreme gradient boosting algorithm. Blending ML models based on corresponding set of predictors presented better discrimination than SAPS II (AUROC, 0.806 vs. 0.771; AUPRC 0.515 vs. 0.429) and SOFA (AUROC, 0.742 vs. 0.706; AUPRC 0.428 vs. 0.381) on the test set. In addition, calibration curves showed that blending ML models had better calibration than SAPS II.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>The blending ML model is capable of integrating different sorts of basic ML models efficiently, and outperforms conventional severity scores in predicting hospital mortality among septic patients in ICU.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s13040-021-00276-5","type":"journal-article","created":{"date-parts":[[2021,8,16]],"date-time":"2021-08-16T09:04:23Z","timestamp":1629104663000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Development and validation of a novel blending machine learning model for hospital mortality prediction in ICU patients with Sepsis"],"prefix":"10.1186","volume":"14","author":[{"given":"Zhixuan","family":"Zeng","sequence":"first","affiliation":[]},{"given":"Shuo","family":"Yao","sequence":"additional","affiliation":[]},{"given":"Jianfei","family":"Zheng","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3492-0190","authenticated-orcid":false,"given":"Xun","family":"Gong","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2021,8,16]]},"reference":[{"issue":"8","key":"276_CR1","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1001\/jama.2016.0287","volume":"315","author":"M Singer","year":"2016","unstructured":"Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for Sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):801\u201310. https:\/\/doi.org\/10.1001\/jama.2016.0287.","journal-title":"JAMA"},{"issue":"7","key":"276_CR2","doi-asserted-by":"publisher","first-page":"1303","DOI":"10.1097\/00003246-200107000-00002","volume":"29","author":"DC Angus","year":"2001","unstructured":"Angus DC, Linde-Zwirble WT, Lidicker J, Clermont G, Carcillo J, Pinsky MR. Epidemiology of severe sepsis in the United States: analysis of incidence, outcome, and associated costs of care. Crit Care Med. 2001;29(7):1303\u201310. https:\/\/doi.org\/10.1097\/00003246-200107000-00002.","journal-title":"Crit Care Med"},{"issue":"5","key":"276_CR3","doi-asserted-by":"publisher","first-page":"1167","DOI":"10.1097\/CCM.0b013e31827c09f8","volume":"41","author":"DF Gaieski","year":"2013","unstructured":"Gaieski DF, Edwards JM, Kallan MJ, Carr BG. Benchmarking the incidence and mortality of severe sepsis in the United States. Crit Care Med. 2013;41(5):1167\u201374. https:\/\/doi.org\/10.1097\/CCM.0b013e31827c09f8.","journal-title":"Crit Care Med"},{"issue":"3","key":"276_CR4","doi-asserted-by":"publisher","first-page":"227","DOI":"10.7326\/0003-4819-113-3-227","volume":"113","author":"JE Parrillo","year":"1990","unstructured":"Parrillo JE, Parker MM, Natanson C, Suffredini AF, Danner RL, Cunnion RE, et al. Septic shock in humans: advances in the understanding of pathogenesis, cardiovascular dysfunction, and therapy. Ann Intern Med. 1990;113(3):227\u201342. https:\/\/doi.org\/10.7326\/0003-4819-113-3-227.","journal-title":"Ann Intern Med"},{"issue":"12","key":"276_CR5","doi-asserted-by":"publisher","first-page":"1958","DOI":"10.1007\/s00134-016-4577-z","volume":"42","author":"A Perner","year":"2016","unstructured":"Perner A, Gordon AC, Backer DD, Dimopoulos G, Russell JA, Lipman J, et al. Sepsis: frontiers in diagnosis, resuscitation and antibiotic therapy. Intensive Care Med. 2016;42(12):1958\u201369. https:\/\/doi.org\/10.1007\/s00134-016-4577-z.","journal-title":"Intensive Care Med"},{"issue":"13","key":"276_CR6","doi-asserted-by":"publisher","first-page":"1308","DOI":"10.1001\/jama.2014.2637","volume":"311","author":"KM Kaukonen","year":"2014","unstructured":"Kaukonen KM, Bailey M, Suzuki S, Pilcher D, Bellomo R. Mortality related to severe sepsis and septic shock among critically ill patients in Australia and New Zealand, 2000-2012. JAMA. 2014;311(13):1308\u201316. https:\/\/doi.org\/10.1001\/jama.2014.2637.","journal-title":"JAMA"},{"issue":"4","key":"276_CR7","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1111\/j.1399-6576.2008.01586.x","volume":"52","author":"K Strand","year":"2008","unstructured":"Strand K, Flaatten H. Severity scoring in the ICU: a review. Acta Anaesthesiol Scand. 2008;52(4):467\u201378. https:\/\/doi.org\/10.1111\/j.1399-6576.2008.01586.x.","journal-title":"Acta Anaesthesiol Scand"},{"issue":"1","key":"276_CR8","doi-asserted-by":"publisher","first-page":"180178","DOI":"10.1038\/sdata.2018.178","volume":"5","author":"TJ Pollard","year":"2018","unstructured":"Pollard TJ, Johnson AEW, Raffa JD, Celi LA, Mark RG, Badawi O. The eICU collaborative research database, a freely available multi-center database for critical care research. Sci Data. 2018;5(1):180178. https:\/\/doi.org\/10.1038\/sdata.2018.178.","journal-title":"Sci Data"},{"issue":"1","key":"276_CR9","doi-asserted-by":"publisher","first-page":"160035","DOI":"10.1038\/sdata.2016.35","volume":"3","author":"AE Johnson","year":"2016","unstructured":"Johnson AE, Pollard TJ, Shen L, Lehman LW, Feng M, Ghassemi M, et al. MIMIC-III, a freely accessible critical care database. Sci Data. 2016;3(1):160035. https:\/\/doi.org\/10.1038\/sdata.2016.35.","journal-title":"Sci Data"},{"issue":"jan07 4","key":"276_CR10","doi-asserted-by":"publisher","first-page":"g7594","DOI":"10.1136\/bmj.g7594","volume":"350","author":"GS Collins","year":"2015","unstructured":"Collins GS, Reitsma JB, Altman DG, Moons KGM. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMJ. 2015;350(jan07 4):g7594. https:\/\/doi.org\/10.1136\/bmj.g7594.","journal-title":"BMJ"},{"issue":"8","key":"276_CR11","doi-asserted-by":"publisher","first-page":"762","DOI":"10.1001\/jama.2016.0288","volume":"315","author":"CW Seymour","year":"2016","unstructured":"Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, et al. Assessment of clinical criteria for Sepsis: for the third international consensus definitions for Sepsis and septic shock (Sepsis-3). JAMA. 2016;315(8):762\u201374. https:\/\/doi.org\/10.1001\/jama.2016.0288.","journal-title":"JAMA"},{"issue":"4","key":"276_CR12","doi-asserted-by":"publisher","first-page":"494","DOI":"10.1097\/CCM.0000000000002965","volume":"46","author":"AEW Johnson","year":"2018","unstructured":"Johnson AEW, Aboab J, Raffa JD, Pollard TJ, Deliberato RO, Celi LA, et al. A comparative analysis of Sepsis identification methods in an electronic database. Crit Care Med. 2018;46(4):494\u20139. https:\/\/doi.org\/10.1097\/CCM.0000000000002965.","journal-title":"Crit Care Med"},{"issue":"11","key":"276_CR13","doi-asserted-by":"publisher","first-page":"1716","DOI":"10.1038\/s41591-018-0213-5","volume":"24","author":"M Komorowski","year":"2018","unstructured":"Komorowski M, Celi LA, Badawi O, Gordon AC, Faisal AA. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care. Nat Med. 2018;24(11):1716\u201320. https:\/\/doi.org\/10.1038\/s41591-018-0213-5.","journal-title":"Nat Med"},{"issue":"3","key":"276_CR14","doi-asserted-by":"publisher","first-page":"290","DOI":"10.1001\/jama.2016.20328","volume":"317","author":"EP Raith","year":"2017","unstructured":"Raith EP, Udy AA, Bailey M, McGloughlin S, MacIsaac C, Bellomo R, et al. Prognostic accuracy of the SOFA score, SIRS criteria, and qSOFA score for in-hospital mortality among adults with suspected infection admitted to the intensive care unit. JAMA. 2017;317(3):290\u2013300. 28114553. https:\/\/doi.org\/10.1001\/jama.2016.20328.","journal-title":"JAMA"},{"issue":"24","key":"276_CR15","doi-asserted-by":"publisher","first-page":"2957","DOI":"10.1001\/jama.270.24.2957","volume":"270","author":"JR Le Gall","year":"1993","unstructured":"Le Gall JR, Lemeshow S, Saulnier F. A new simplified acute physiology score (SAPS II) based on a European\/north American multicenter study. JAMA. 1993;270(24):2957\u201363. https:\/\/doi.org\/10.1001\/jama.270.24.2957.","journal-title":"JAMA"},{"issue":"6","key":"276_CR16","doi-asserted-by":"publisher","first-page":"520","DOI":"10.1093\/bioinformatics\/17.6.520","volume":"17","author":"O Troyanskaya","year":"2001","unstructured":"Troyanskaya O, Cantor M, Sherlock G, Brown P, Hastie T, Tibshirani R, et al. Missing value estimation methods for DNA microarrays. Bioinformatics. 2001;17(6):520\u20135. https:\/\/doi.org\/10.1093\/bioinformatics\/17.6.520.","journal-title":"Bioinformatics"},{"issue":"3","key":"276_CR17","doi-asserted-by":"publisher","first-page":"837","DOI":"10.2307\/2531595","volume":"44","author":"ER DeLong","year":"1988","unstructured":"DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837\u201345. https:\/\/doi.org\/10.2307\/2531595.","journal-title":"Biometrics"},{"issue":"14","key":"276_CR18","doi-asserted-by":"publisher","first-page":"1377","DOI":"10.1001\/jama.2017.12126","volume":"318","author":"AC Alba","year":"2017","unstructured":"Alba AC, Agoritsas T, Walsh M, Hanna S, Iorio A, Devereaux PJ, et al. Discrimination and calibration of clinical prediction models: Users' guides to the medical literature. JAMA. 2017;318(14):1377\u201384. https:\/\/doi.org\/10.1001\/jama.2017.12126.","journal-title":"JAMA"},{"key":"276_CR19","doi-asserted-by":"publisher","first-page":"100","DOI":"10.1109\/IEMBS.2011.6089906","volume":"2011","author":"VJ Ribas","year":"2011","unstructured":"Ribas VJ, L\u00f3pez JC, Ruiz-Sanmartin A, Ruiz-Rodr\u00edguez JC, Rello J, Wojdel A, et al. Severe sepsis mortality prediction with relevance vector machines. Conf Proc IEEE Eng Med Biol Soc. 2011;2011:100\u20133. https:\/\/doi.org\/10.1109\/IEMBS.2011.6089906.","journal-title":"Conf Proc IEEE Eng Med Biol Soc"},{"issue":"3","key":"276_CR20","doi-asserted-by":"publisher","first-page":"269","DOI":"10.1111\/acem.12876","volume":"23","author":"RA Taylor","year":"2016","unstructured":"Taylor RA, Pare JR, Venkatesh AK, Mowafi H, Melnick ER, Fleischman W, et al. Prediction of in-hospital mortality in emergency department patients with Sepsis: a local big data-driven, Machine Learning Approach. Acad Emerg Med. 2016;23(3):269\u201378. https:\/\/doi.org\/10.1111\/acem.12876.","journal-title":"Acad Emerg Med"},{"issue":"11","key":"276_CR21","doi-asserted-by":"publisher","first-page":"1906","DOI":"10.3390\/jcm8111906","volume":"8","author":"JW Perng","year":"2019","unstructured":"Perng JW, Kao IH, Kung CT, Hung SC, Lai YH, Su CM. Mortality prediction of septic patients in the emergency department based on machine learning. J Clin Med. 2019;8(11):1906. https:\/\/doi.org\/10.3390\/jcm8111906.","journal-title":"J Clin Med"},{"key":"276_CR22","doi-asserted-by":"publisher","first-page":"445","DOI":"10.3389\/fmed.2020.00445","volume":"7","author":"RQ Yao","year":"2020","unstructured":"Yao RQ, Jin X, Wang GW, Yu Y, Wu GS, Zhu YB, et al. A machine learning-based prediction of hospital mortality in patients with postoperative Sepsis. Front Med. 2020;7:445. https:\/\/doi.org\/10.3389\/fmed.2020.00445.","journal-title":"Front Med"},{"issue":"1","key":"276_CR23","doi-asserted-by":"publisher","first-page":"251","DOI":"10.1186\/s12911-020-01271-2","volume":"20","author":"GL Kong","year":"2020","unstructured":"Kong GL, Lin K, Hu YH. Using machine learning methods to predict in-hospital mortality of sepsis patients in the ICU. BMC Med Inform Decis Mak. 2020;20(1):251. https:\/\/doi.org\/10.1186\/s12911-020-01271-2.","journal-title":"BMC Med Inform Decis Mak"},{"issue":"5","key":"276_CR24","doi-asserted-by":"publisher","first-page":"1670","DOI":"10.1097\/CCM.0b013e31819fcf68","volume":"37","author":"ME Mikkelsen","year":"2009","unstructured":"Mikkelsen ME, Miltiades AN, Gaieski DF, Goyal M, Fuchs BD, Shah CV, et al. Serum lactate is associated with mortality in severe sepsis independent of organ failure and shock. Crit Care Med. 2009;37(5):1670\u20137. https:\/\/doi.org\/10.1097\/CCM.0b013e31819fcf68.","journal-title":"Crit Care Med"},{"issue":"2","key":"276_CR25","doi-asserted-by":"publisher","first-page":"202","DOI":"10.1007\/s00134-015-4127-0","volume":"42","author":"SA Haas","year":"2016","unstructured":"Haas SA, Lange T, Saugel B, Petzoldt M, Fuhrmann V, Metschke M, et al. Severe hyperlactatemia, lactate clearance and mortality in unselected critically ill patients. Intensive Care Med. 2016;42(2):202\u201310. https:\/\/doi.org\/10.1007\/s00134-015-4127-0.","journal-title":"Intensive Care Med"},{"issue":"5","key":"276_CR26","doi-asserted-by":"publisher","first-page":"R197","DOI":"10.1186\/cc12891","volume":"17","author":"AH Rishu","year":"2013","unstructured":"Rishu AH, Khan R, Al-Dorzi HM, Tamim HM, Al-Qahtani S, Al-Ghamdi G, et al. Even mild hyperlactatemia is associated with increased mortality in critically ill patients. Crit Care. 2013;17(5):R197. https:\/\/doi.org\/10.1186\/cc12891.","journal-title":"Crit Care"},{"issue":"5","key":"276_CR27","doi-asserted-by":"publisher","first-page":"503","DOI":"10.1186\/s13054-014-0503-3","volume":"18","author":"M Garcia-Alvarez","year":"2014","unstructured":"Garcia-Alvarez M, Marik P, Bellomo R. Sepsis-associated hyperlactatemia. Crit Care. 2014;18(5):503. https:\/\/doi.org\/10.1186\/s13054-014-0503-3.","journal-title":"Crit Care"},{"issue":"12","key":"276_CR28","doi-asserted-by":"publisher","first-page":"687","DOI":"10.1177\/0885066616685300","volume":"33","author":"M Yin","year":"2018","unstructured":"Yin M, Si L, Qin WD, Li C, Zhang JN, Yang HN, et al. Predictive value of serum albumin level for the prognosis of severe Sepsis without exogenous human albumin administration: a prospective cohort study. J Intensive Care Med. 2018;33(12):687\u201394. https:\/\/doi.org\/10.1177\/0885066616685300.","journal-title":"J Intensive Care Med"},{"key":"276_CR29","doi-asserted-by":"publisher","first-page":"139","DOI":"10.1016\/j.jcrc.2019.02.004","volume":"51","author":"R Takegawa","year":"2019","unstructured":"Takegawa R, Kabata D, Shimizu K, Hisano S, Ogura H, Shintani A, et al. Serum albumin as a risk factor for death in patients with prolonged sepsis: an observational study. J Crit Care. 2019;51:139\u201344. https:\/\/doi.org\/10.1016\/j.jcrc.2019.02.004.","journal-title":"J Crit Care"},{"issue":"4","key":"276_CR30","doi-asserted-by":"publisher","first-page":"743","DOI":"10.1007\/s10096-019-03478-2","volume":"38","author":"I Arnau-Barr\u00e9s","year":"2019","unstructured":"Arnau-Barr\u00e9s I, G\u00fcerri-Fern\u00e1ndez R, Luque S, Sorli L, V\u00e1zquez O, Miralles R. Serum albumin is a strong predictor of sepsis outcome in elderly patients. Eur J Clin Microbiol Infect Dis. 2019;38(4):743\u20136. https:\/\/doi.org\/10.1007\/s10096-019-03478-2.","journal-title":"Eur J Clin Microbiol Infect Dis"},{"issue":"15","key":"276_CR31","doi-asserted-by":"publisher","first-page":"1412","DOI":"10.1056\/NEJMoa1305727","volume":"370","author":"P Caironi","year":"2014","unstructured":"Caironi P, Tognoni G, Masson S, Fumagalli R, Pesenti A, Romero M, et al. Albumin replacement in patients with severe sepsis or septic shock. N Engl J Med. 2014;370(15):1412\u201321. https:\/\/doi.org\/10.1056\/NEJMoa1305727.","journal-title":"N Engl J Med"},{"issue":"jul22 10","key":"276_CR32","doi-asserted-by":"publisher","first-page":"g4561","DOI":"10.1136\/bmj.g4561","volume":"349","author":"A Patel","year":"2014","unstructured":"Patel A, Laffan MA, Waheed U, Brett SJ. Randomised trials of human albumin for adults with sepsis: systematic review and meta-analysis with trial sequential analysis of all-cause mortality. BMJ. 2014;349(jul22 10):g4561. https:\/\/doi.org\/10.1136\/bmj.g4561.","journal-title":"BMJ"}],"container-title":["BioData Mining"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13040-021-00276-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13040-021-00276-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13040-021-00276-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,16]],"date-time":"2021-08-16T09:07:35Z","timestamp":1629104855000},"score":1,"resource":{"primary":{"URL":"https:\/\/biodatamining.biomedcentral.com\/articles\/10.1186\/s13040-021-00276-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,16]]},"references-count":32,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["276"],"URL":"https:\/\/doi.org\/10.1186\/s13040-021-00276-5","relation":{},"ISSN":["1756-0381"],"issn-type":[{"value":"1756-0381","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,16]]},"assertion":[{"value":"19 February 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 August 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 August 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study was an analysis of third-party deidentified publicly available databases with pre-existing ethical review board (ERB) approval.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"All authors declare that they have no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"40"}}