{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T16:42:01Z","timestamp":1774456921576,"version":"3.50.1"},"reference-count":28,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,7,21]],"date-time":"2025-07-21T00:00:00Z","timestamp":1753056000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,7,21]],"date-time":"2025-07-21T00:00:00Z","timestamp":1753056000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100013347","name":"European Society of Intensive Care Medicine","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100013347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:sec>\n            <jats:title>Background<\/jats:title>\n            <jats:p>Nosocomial infections are a major cause of morbidity and mortality in the ICU. Earlier identification of these complications may facilitate better clinical management and improve outcomes. We developed a dynamic prediction model that leveraged high-frequency longitudinal data to estimate infection risk 48\u00a0h ahead of clinically overt deterioration.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>We used electronic health record data from consecutive adults who had been treated for &gt;\u200948\u00a0h in a mixed tertiary ICU in the Netherlands enrolled in the Molecular Diagnosis and Risk Stratification of Sepsis (MARS) cohort from 2011 to 2018. All infectious episodes were prospectively adjudicated. ICU-acquired infection (ICU-AI) risk was estimated using a Cox landmark model with high-resolution vital sign data processed via a convolutional neural network (CNN).<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>We studied 32,178 observation days in 4444 patients and observed 1197 infections, yielding an overall infection risk of 3.5% per ICU day. Discrimination of the composite model was moderate with c-index values varying between 0.64 (95%CI: 0.58\u20130.69) and 0.72 (95%CI: 0.66\u20130.78) across timepoints, with some overestimation of ICU-AI risk overall (mean calibration slope 0.58). Compared to 38 common features of infection, a CNN risk score derived from five vital sign signals consistently ranked as a strong predictor of ICU-AI across all time points but did not substantially change risk prediction of ICU-AI.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion<\/jats:title>\n            <jats:p>A dynamic modelling approach that incorporates machine learning of high-frequency vital sign data shows promise as a continuous bedside index of infection risk. Further validation is needed to weigh added complexity and interpretability of the deep learning model against potential benefits for clinical decision support in the ICU.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1186\/s12911-025-03031-6","type":"journal-article","created":{"date-parts":[[2025,7,21]],"date-time":"2025-07-21T09:58:11Z","timestamp":1753091891000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Early detection of ICU-acquired infections using high-frequency electronic health record data"],"prefix":"10.1186","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2831-0292","authenticated-orcid":false,"given":"Meri R. J.","family":"Varkila","sequence":"first","affiliation":[]},{"given":"Giacomo","family":"Lancia","sequence":"additional","affiliation":[]},{"given":"Maarten","family":"van Smeden","sequence":"additional","affiliation":[]},{"given":"Marc J. M.","family":"Bonten","sequence":"additional","affiliation":[]},{"given":"Cristian","family":"Spitoni","sequence":"additional","affiliation":[]},{"given":"Olaf L.","family":"Cremer","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,7,21]]},"reference":[{"key":"3031_CR1","doi-asserted-by":"publisher","first-page":"607952","DOI":"10.3389\/fmed.2021.607952","volume":"8","author":"M Moor","year":"2021","unstructured":"Moor M, Rieck B, Horn M, Jutzeler CR, Borgwardt K. Early prediction of Sepsis in the ICU using machine learning: A systematic review. Front Med (Lausanne). 2021;8:607952.","journal-title":"Front Med (Lausanne)"},{"issue":"3","key":"3031_CR2","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1007\/s00134-019-05872-y","volume":"46","author":"LM Fleuren","year":"2020","unstructured":"Fleuren LM, Klausch TLT, Zwager CL, Schoonmade LJ, Guo T, Roggeveen LF, et al. Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy. Intensive Care Med. 2020;46(3):383\u2013400.","journal-title":"Intensive Care Med"},{"key":"3031_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.cmpb.2018.12.027","volume":"170","author":"MM Islam","year":"2019","unstructured":"Islam MM, Nasrin T, Walther BA, Wu CC, Yang HC, Li YC. Prediction of sepsis patients using machine learning approach: A meta-analysis. Comput Methods Programs Biomed. 2019;170:1\u20139.","journal-title":"Comput Methods Programs Biomed"},{"key":"3031_CR4","unstructured":"Perlin J. (2018). SPOT: How HCA healthcare is sniffing out sepsis early (2018). Available at: https:\/\/hcahealthcaretoday.com\/2018\/09\/10\/spot-how-hca-is-sniffing-out-sepsis-early\/. Accessed on Feb 7, 2024."},{"key":"3031_CR5","unstructured":"Wong A, Otles E, Donnelly JP, Krumm A, McCullough J, DeTroyer-Cooley O, et al. External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Intern Med. 2021;181(8):1065\u20131070. Erratum in: JAMA Intern Med. 2021;181(8):1144."},{"issue":"Suppl 1","key":"3031_CR6","doi-asserted-by":"publisher","first-page":"S53","DOI":"10.1513\/AnnalsATS.201706-449KV","volume":"15","author":"AH Morris","year":"2018","unstructured":"Morris AH. Human cognitive limitations. Broad, consistent, clinical application of physiological principles will require decision support. Ann Am Thorac Soc. 2018;15(Suppl 1):S53\u20136.","journal-title":"Ann Am Thorac Soc"},{"key":"3031_CR7","doi-asserted-by":"publisher","unstructured":"Collins GS, Moons KGM, Dhiman P, Riley RD, Beam AL, Van Calster B et al. TRIPOD\u2009+\u2009AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods. BMJ. 2024;385:e078378. https:\/\/doi.org\/10.1136\/bmj-2023-078378. Erratum in: BMJ. 2024;385:q902.","DOI":"10.1136\/bmj-2023-078378"},{"issue":"10","key":"3031_CR8","doi-asserted-by":"publisher","first-page":"2373","DOI":"10.1097\/CCM.0b013e3182923712","volume":"41","author":"PM Klein Klouwenberg","year":"2013","unstructured":"Klein Klouwenberg PM, Ong DS, Bos LD, de Beer FM, van Hooijdonk RT, Huson MA, et al. Interobserver agreement of centers for disease control and prevention criteria for classifying infections in critically ill patients. Crit Care Med. 2013;41(10):2373\u20138.","journal-title":"Crit Care Med"},{"key":"3031_CR9","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1111\/j.1467-9469.2006.00529.x","volume":"34","author":"HC van Houwelingen","year":"2007","unstructured":"van Houwelingen HC. Dynamic prediction by landmarking in event history analysis. Scand J Stat. 2007;34:70\u201385.","journal-title":"Scand J Stat"},{"issue":"12","key":"3031_CR10","doi-asserted-by":"publisher","first-page":"2031","DOI":"10.1002\/sim.5665","volume":"32","author":"MA Nicolaie","year":"2013","unstructured":"Nicolaie MA, van Houwelingen JC, de Witte TM, Putter H. Dynamic prediction by landmarking in competing risks. Stat Med. 2013;32(12):2031\u201347.","journal-title":"Stat Med"},{"issue":"11","key":"3031_CR11","doi-asserted-by":"publisher","first-page":"3166","DOI":"10.1177\/0962280220921415","volume":"29","author":"B Van Calster","year":"2020","unstructured":"Van Calster B, van Smeden M, De Cock B, Steyerberg EW. Regression shrinkage methods for clinical prediction models do not guarantee improved performance: simulation study. Stat Methods Med Res. 2020;29(11):3166\u201378.","journal-title":"Stat Methods Med Res"},{"key":"3031_CR12","doi-asserted-by":"publisher","first-page":"102862","DOI":"10.1016\/j.artmed.2024.102862","volume":"151","author":"G Lancia","year":"2024","unstructured":"Lancia G, Varkila MRJ, Cremer OL, Spitoni C. Two-step interpretable modeling of ICU-AIs. Artif Intell Med. 2024;151:102862.","journal-title":"Artif Intell Med"},{"issue":"4","key":"3031_CR13","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4","volume":"15","author":"FE Harrell Jr","year":"1996","unstructured":"Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15(4):361\u201387.","journal-title":"Stat Med"},{"issue":"2","key":"3031_CR14","doi-asserted-by":"publisher","first-page":"615","DOI":"10.1093\/ije\/dyab256","volume":"51","author":"CL Ramspek","year":"2022","unstructured":"Ramspek CL, Teece L, Snell KIE, Evans M, Riley RD, van Smeden M, et al. Lessons learnt when accounting for competing events in the external validation of time-to-event prognostic models. Int J Epidemiol. 2022;51(2):615\u201325.","journal-title":"Int J Epidemiol"},{"key":"3031_CR15","doi-asserted-by":"crossref","unstructured":"White N, Harries P, Harris AJ, Vickerstaff V, Lodge P, McGowan C et al. How do palliative care doctors recognise imminently dying patients? A judgement analysis. BMJ Open. 2018;8(11):e024996. Erratum in: BMJ Open. 2019;9(1):e024996corr1. Supplementary file 2. Understanding the Brier Score.","DOI":"10.1136\/bmjopen-2018-024996corr1"},{"key":"3031_CR16","doi-asserted-by":"publisher","first-page":"e069249","DOI":"10.1136\/bmj-2021-069249","volume":"377","author":"N van Geloven","year":"2022","unstructured":"van Geloven N, Giardiello D, Bonneville EF, Teece L, Ramspek CL, van Smeden M, et al. Validation of prediction models in the presence of competing risks: a guide through modern methods. BMJ. 2022;377:e069249.","journal-title":"BMJ"},{"issue":"1","key":"3031_CR17","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1186\/s12916-019-1466-7","volume":"17","author":"B Van Calster","year":"2019","unstructured":"Van Calster B, McLernon DJ, van Smeden M, Wynants L, Steyerberg EW. Topic group \u2018evaluating diagnostic tests and prediction models\u2019 of the STRATOS initiative. Calibration: the Achilles heel of predictive analytics. BMC Med. 2019;17(1):230.","journal-title":"BMC Med"},{"issue":"7","key":"3031_CR18","doi-asserted-by":"publisher","first-page":"1455","DOI":"10.1038\/s41591-022-01894-0","volume":"28","author":"R Adams","year":"2022","unstructured":"Adams R, Henry KE, Sridharan A, Soleimani H, Zhan A, Rawat N, et al. Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis. Nat Med. 2022;28(7):1455\u201360.","journal-title":"Nat Med"},{"issue":"4","key":"3031_CR19","doi-asserted-by":"publisher","first-page":"547","DOI":"10.1097\/CCM.0000000000002936","volume":"46","author":"S Nemati","year":"2018","unstructured":"Nemati S, Holder A, Razmi F, Stanley MD, Clifford GD, Buchman TG. An interpretable machine learning model for accurate prediction of Sepsis in the ICU. Crit Care Med. 2018;46(4):547\u201353.","journal-title":"Crit Care Med"},{"issue":"11","key":"3031_CR20","doi-asserted-by":"publisher","first-page":"1914","DOI":"10.1002\/sim.6077","volume":"33","author":"S Biswas","year":"2014","unstructured":"Biswas S, Arun B, Parmigiani G. Reclassification of predictions for Uncovering subgroup specific improvement. Stat Med. 2014;33(11):1914\u201327.","journal-title":"Stat Med"},{"key":"3031_CR21","doi-asserted-by":"crossref","unstructured":"Baker SG, Schuit E, Steyerberg EW, Pencina MJ, Vickers A, Moons KG et al. How to interpret a small increase in AUC with an additional risk prediction marker: decision analysis comes through. Stat Med. 2014;33(22):3946-59. Erratum in: Stat Med. 2014;34(5):900. Vickers, Andew [corrected to Vickers, Andrew]. Erratum in: Stat Med. 2014;33(22)3960.","DOI":"10.1002\/sim.6195"},{"issue":"9","key":"3031_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v035.i09","volume":"35","author":"D Rizopoulos","year":"2010","unstructured":"Rizopoulos D. JM: an R package for the joint modelling of longitudinal and Time-to-Event data. J Stat Softw. 2010;35(9):1\u201333.","journal-title":"J Stat Softw"},{"key":"3031_CR23","doi-asserted-by":"publisher","unstructured":"Klein Klouwenberg PMC, Spitoni C, van der Poll T, Bonten MJ, Cremer OL, MARS consortium. Predicting the clinical trajectory in critically ill patients with sepsis: a cohort study. Crit Care. 2019;23(1):408. https:\/\/doi.org\/10.1186\/s13054-019-2687-z. Erratum in: Crit Care. 2020;24(1):41.","DOI":"10.1186\/s13054-019-2687-z"},{"issue":"7","key":"3031_CR24","doi-asserted-by":"publisher","first-page":"1349","DOI":"10.1093\/jamia\/ocad075","volume":"30","author":"AH van der Vegt","year":"2023","unstructured":"van der Vegt AH, Scott IA, Dermawan K, Schnetler RJ, Kalke VR, Lane PJ. Deployment of machine learning algorithms to predict sepsis: systematic review and application of the SALIENT clinical AI implementation framework. J Am Med Inf Assoc. 2023;30(7):1349\u201361.","journal-title":"J Am Med Inf Assoc"},{"issue":"4","key":"3031_CR25","doi-asserted-by":"publisher","first-page":"541","DOI":"10.1093\/cid\/ciaa059","volume":"72","author":"C Rhee","year":"2021","unstructured":"Rhee C, Chiotos K, Cosgrove SE, Heil EL, Kadri SS, Kalil AC, et al. Infectious diseases society of America position paper: recommended revisions to the National severe Sepsis and septic shock early management bundle (SEP-1) Sepsis quality measure. Clin Infect Dis. 2021;72(4):541\u201352.","journal-title":"Clin Infect Dis"},{"issue":"1","key":"3031_CR26","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1186\/s13054-021-03862-5","volume":"26","author":"T Barichello","year":"2022","unstructured":"Barichello T, Generoso JS, Singer M, Dal-Pizzol F. Biomarkers for sepsis: more than just fever and leukocytosis-a narrative review. Crit Care. 2022;26(1):14.","journal-title":"Crit Care"},{"key":"3031_CR27","doi-asserted-by":"publisher","first-page":"104394","DOI":"10.1016\/j.ebiom.2022.104394","volume":"86","author":"M Komorowski","year":"2022","unstructured":"Komorowski M, Green A, Tatham KC, Seymour C, Antcliffe D. Sepsis biomarkers and diagnostic tools with a focus on machine learning. EBioMedicine. 2022;86:104394.","journal-title":"EBioMedicine"},{"issue":"1","key":"3031_CR28","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1186\/s40635-021-00383-x","volume":"9","author":"AR Schuurman","year":"2021","unstructured":"Schuurman AR, Reijnders TDY, Kullberg RFJ, Butler JM, van der Poll T, Wiersinga WJ. Sepsis: deriving biological meaning and clinical applications from high-dimensional data. Intensive Care Med Exp. 2021;9(1):27.","journal-title":"Intensive Care Med Exp"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-025-03031-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-025-03031-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-025-03031-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,7]],"date-time":"2025-09-07T16:22:40Z","timestamp":1757262160000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-025-03031-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,21]]},"references-count":28,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["3031"],"URL":"https:\/\/doi.org\/10.1186\/s12911-025-03031-6","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,21]]},"assertion":[{"value":"30 April 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 May 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"21 July 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study was performed using data from the Molecular Diagnosis and Risk Stratification of Sepsis (MARS) project (ClinicalTrials.gov identifier NCT01905033, date January 2011), a prospective ICU cohort study, for which the institutional review boards, Medical Research Ethics Committee Utrecht and Medical Research Ethics Committee AMC Amsterdam, approved an opt-out method of informed consent (protocol number 10\u2013056\u00a0C, date: June 16, 2010) whereby participants and family members were notified of the study by a brochure with an attached opt-out card that was provided at ICU admission. The current analysis was additionally reviewed by the Medical Research Ethics Committee Utrecht (Medisch-Ethische Toetsingscommissie UMC Utrecht) and deemed exempt from the need for consent to participate in accordance with national regulations (protocol number 19\u2013241\u00a0C, date April 3, 2019). For the current analysis, we obtained longitudinal data from consecutive subjects\u2009>\u200918 years who were admitted to the medical-surgical tertiary ICU of one of the participating centers (University Medical Center Utrecht in the Netherlands) between January 2011 and December 2018. All procedures were conducted in accordance with the Helsinki Declaration of 1975.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"Dr Varkila reported receiving grants from the ESICM Epimed Solutions Award on Quality Improvement and Patient Safety 2019 to support this work. Dr Bonten reported receiving grants through the Innovative Medicines Initiative Joint Undertaking through the COMBACTE-NET and COMBACTE-CARE projects, Johnson & Johnson, and Merck during the duration of this study, and personal fees from GSK and Shionogi for consulting outside the submitted work. Dr Cremer reported receiving grants from the ESICM Epimed Solutions Award on Quality Improvement and Patient Safety 2019 to support this work, and grants from ImmuneXpress Inc. (Seattle, WA), Abionic SA (Epalinges, Switzerland), Prolira BV (Utrecht, Netherlands) and Presymptom Health (Porton Down, UK) for biomarker studies related to sepsis diagnosis outside of this work. No other disclosures were reported.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"The peer review reports can be found at .","order":5,"name":"Ethics","group":{"name":"EthicsHeading","label":"Peer review"}}],"article-number":"273"}}