{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,12]],"date-time":"2026-10-12T21:09:02Z","timestamp":1791839342545,"version":"4.3.4"},"reference-count":12,"publisher":"Georg Thieme Verlag KG","issue":"04","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Appl Clin Inform"],"published-print":{"date-parts":[[2025,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Central line-associated bloodstream infections (CLABSIs) are associated with substantial pediatric morbidity and mortality. The capacity to predict which children with central lines are at greatest risk of CLABSI could inform surveillance and prevention efforts. Our team previously published in silico predictive models for CLABSI.<\/jats:p><jats:p>To prospectively implement a pediatric CLABSI predictive model and achieve adequate performance in offline validation for implementation in clinical practice.<\/jats:p><jats:p>Most performant predictive models were deep learning models requiring substantial pre-processing of many features into 8-hour windows including the current day and up to 56 days prior for the current admission. To replicate this pre-processing, we created a novel infrastructure to (1) organize current-day data for all the relevant features and (2) create a staged historical data store for those same features with application programming interfaces to connect the two. We compared predictive performance of these scores for CLABSI in the next 48\u2009hours with two labels, one based on manual review of positive blood cultures in children with central lines and another based on positive blood culture and receipt of at least 4 days of new IV antibiotics.<\/jats:p><jats:p>The area under the receiver-operating characteristic (AUROC) fell from 0.97 from retrospective data to &lt;0.60 despite multiple iterations of troubleshooting. Primary root causes included train\/serve skew, feature leakage, and overfitting. Hypothesized secondary drivers were complex model specification, poor data governance, inadequate testing, challenging feature translation between real-time and historical data models, limited monitoring and logging infrastructure for troubleshooting, and suboptimal handoff between the model development and deployment teams.<\/jats:p><jats:p>Bridging the gap from predictive model development to clinical deployment requires early and close coordination between data governance, data science, clinical informatics, and implementation engineers. Balancing predictive performance with implementation feasibility can accelerate the adoption of predictive clinical decision support systems.<\/jats:p>","DOI":"10.1055\/a-2605-1847","type":"journal-article","created":{"date-parts":[[2025,5,12]],"date-time":"2025-05-12T20:00:47Z","timestamp":1747080047000},"page":"1192-1199","source":"Crossref","is-referenced-by-count":3,"title":["Performance Degradation between Development and Deployment of a Predictive Model for Central Line-Associated Bloodstream Infections in Hospitalized Children"],"prefix":"10.1055","volume":"16","author":[{"given":"Jonathan M.","family":"Beus","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia, United States"},{"name":"Department of Information Systems and Technology, Children's Healthcare of Atlanta, Atlanta, Georgia, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mark","family":"Mai","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia, United States"},{"name":"Department of Information Systems and Technology, Children's Healthcare of Atlanta, Atlanta, Georgia, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nikolay P.","family":"Braykov","sequence":"additional","affiliation":[{"name":"Department of Information Systems and Technology, Children's Healthcare of Atlanta, Atlanta, Georgia, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Swaminathan","family":"Kandaswamy","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edwin","family":"Ray","sequence":"additional","affiliation":[{"name":"Department of Information Systems and Technology, Children's Healthcare of Atlanta, Atlanta, Georgia, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David B.","family":"Cundiff","sequence":"additional","affiliation":[{"name":"Department of Information Systems and Technology, Children's Healthcare of Atlanta, Atlanta, Georgia, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paulette","family":"Djachechi","sequence":"additional","affiliation":[{"name":"Department of Information Systems and Technology, Children's Healthcare of Atlanta, Atlanta, Georgia, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Thompson","sequence":"additional","affiliation":[{"name":"Department of Information Systems and Technology, Children's Healthcare of Atlanta, Atlanta, Georgia, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Azade","family":"Tabaie","sequence":"additional","affiliation":[{"name":"MedStar Health Research Institute, Center for Biostatistics, Informatics, and Data Science, Washington, DC, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ryan","family":"Birmingham","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, Georgia, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rishi","family":"Kamaleswaran","sequence":"additional","affiliation":[{"name":"Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, Georgia, United States"},{"name":"Departments of Biomedical Engineering and Electrical & Computer Engineering, Duke University, Durham, North Carolina, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Evan","family":"Orenstein","sequence":"additional","affiliation":[{"name":"Department of Pediatrics, Emory University School of Medicine, Atlanta, Georgia, United States"},{"name":"Department of Information Systems and Technology, Children's Healthcare of Atlanta, Atlanta, Georgia, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"194","published-online":{"date-parts":[[2025,9,26]]},"reference":[{"issue":"02","key":"ref3","first-page":"174","article-title":"Attributable mortality for pediatric and neonatal central line-associated bloodstream infections in Greece","volume":"13","author":"S Karagiannidou","year":"2021","journal-title":"J Pediatr Intensive Care"},{"issue":"06","key":"ref4","doi-asserted-by":"crossref","first-page":"e1525","DOI":"10.1542\/peds.2013-3795","article-title":"Attributable cost and length of stay for central line-associated bloodstream infections","volume":"133","author":"A Goudie","year":"2014","journal-title":"Pediatrics"},{"issue":"06","key":"ref5","doi-asserted-by":"crossref","first-page":"e1678","DOI":"10.1542\/peds.2014-0582","article-title":"Preventing CLABSIs among pediatric hematology\/oncology inpatients: national collaborative results","volume":"134","author":"D G Bundy","year":"2014","journal-title":"Pediatrics"},{"issue":"03","key":"ref6","doi-asserted-by":"crossref","first-page":"e660","DOI":"10.1097\/pq9.0000000000000660","article-title":"Eliminating central line associated bloodstream infections in pediatric oncology patients: a quality improvement effort","volume":"8","author":"D N Willis","year":"2023","journal-title":"Pediatr Qual Saf"},{"issue":"08","key":"ref7","doi-asserted-by":"crossref","first-page":"939","DOI":"10.1017\/ice.2016.83","article-title":"Risk factors for central venous catheter-associated bloodstream infection in pediatric patients: a cohort study","volume":"37","author":"J H Carter","year":"2016","journal-title":"Infect Control Hosp Epidemiol"},{"issue":"01","key":"ref8","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1016\/j.ajic.2019.06.027","article-title":"Risk factors for central line-associated bloodstream infections in children with tunneled central venous catheters","volume":"48","author":"P Paioni","year":"2020","journal-title":"Am J Infect Control"},{"key":"ref9","doi-asserted-by":"crossref","first-page":"104289","DOI":"10.1016\/j.compbiomed.2021.104289","article-title":"Predicting presumed serious infection among hospitalized children on central venous lines with machine learning","volume":"132","author":"A Tabaie","year":"2021","journal-title":"Comput Biol Med"},{"key":"ref10","doi-asserted-by":"crossref","first-page":"726870","DOI":"10.3389\/fped.2021.726870","article-title":"Deep learning model to predict serious infection among children with central venous lines","volume":"9","author":"A Tabaie","year":"2021","journal-title":"Front Pediatr"},{"issue":"04","key":"ref11","doi-asserted-by":"crossref","first-page":"969","DOI":"10.1038\/s41390-022-02116-6","article-title":"Integrating structured and unstructured data for timely prediction of bloodstream infection among children","volume":"93","author":"A Tabaie","year":"2023","journal-title":"Pediatr Res"},{"key":"ref13","doi-asserted-by":"crossref","first-page":"103208","DOI":"10.1016\/j.jbi.2019.103208","article-title":"The REDCap consortium: building an international community of software platform partners","volume":"95","author":"P A Harris","year":"2019","journal-title":"J Biomed Inform"},{"key":"ref14","doi-asserted-by":"crossref","first-page":"43","DOI":"10.6339\/JDS.201301_11(1).0003","article-title":"Visualizing and inspecting large datasets with tableplots","volume":"11","author":"M Tennekes","year":"2013","journal-title":"J Data Sci"},{"issue":"09","key":"ref15","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.1093\/jamia\/ocad114","article-title":"DEPLOYR: a technical framework for deploying custom real-time machine learning models into the electronic medical record","volume":"30","author":"C K Corbin","year":"2023","journal-title":"J Am Med Inform Assoc"}],"container-title":["Applied Clinical Informatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.thieme-connect.de\/products\/ejournals\/pdf\/10.1055\/a-2605-1847.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,26]],"date-time":"2025-09-26T23:33:07Z","timestamp":1758929587000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.thieme-connect.de\/DOI\/DOI?10.1055\/a-2605-1847"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,12]]},"references-count":12,"journal-issue":{"issue":"04","published-online":{"date-parts":[[2025,8,6]]},"published-print":{"date-parts":[[2025,8]]}},"URL":"https:\/\/doi.org\/10.1055\/a-2605-1847","archive":["Portico","CLOCKSS"],"relation":{},"ISSN":["1869-0327"],"issn-type":[{"value":"1869-0327","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,5,12]]},"published":{"date-parts":[[2025,5,12]]}}}