{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,18]],"date-time":"2026-04-18T02:59:51Z","timestamp":1776481191155,"version":"3.51.2"},"reference-count":47,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,1,20]],"date-time":"2025-01-20T00:00:00Z","timestamp":1737331200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>The adoption of machine learning (ML) has been slow within the healthcare setting. We launched Pediatric Real-world Evaluative Data sciences for Clinical Transformation (PREDICT) at a pediatric hospital. Its goal was to develop, deploy, evaluate and maintain clinical ML models to improve pediatric patient outcomes using electronic health records data.<\/jats:p><\/jats:sec><jats:sec><jats:title>Objective<\/jats:title><jats:p>To provide examples from the PREDICT experience illustrating how common challenges with clinical ML deployment were addressed.<\/jats:p><\/jats:sec><jats:sec><jats:title>Materials and methods<\/jats:title><jats:p>We present common challenges in developing and deploying models in healthcare related to the following: identify clinical scenarios, establish data infrastructure and utilization, create machine learning operations and integrate into clinical workflows.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We show examples of how these challenges were overcome and provide suggestions for pragmatic solutions while maintaining best practices.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>These approaches will require refinement over time as the number of deployments and experience increase.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fdgth.2025.1462751","type":"journal-article","created":{"date-parts":[[2025,1,20]],"date-time":"2025-01-20T05:24:33Z","timestamp":1737350673000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["A roadmap to implementing machine learning in healthcare: from concept to practice"],"prefix":"10.3389","volume":"7","author":[{"given":"Adam Paul","family":"Yan","sequence":"first","affiliation":[]},{"given":"Lin Lawrence","family":"Guo","sequence":"additional","affiliation":[]},{"given":"Jiro","family":"Inoue","sequence":"additional","affiliation":[]},{"given":"Santiago Eduardo","family":"Arciniegas","sequence":"additional","affiliation":[]},{"given":"Emily","family":"Vettese","sequence":"additional","affiliation":[]},{"given":"Agata","family":"Wolochacz","sequence":"additional","affiliation":[]},{"given":"Nicole","family":"Crellin-Parsons","sequence":"additional","affiliation":[]},{"given":"Brandon","family":"Purves","sequence":"additional","affiliation":[]},{"given":"Steven","family":"Wallace","sequence":"additional","affiliation":[]},{"given":"Azaz","family":"Patel","sequence":"additional","affiliation":[]},{"given":"Medhat","family":"Roshdi","sequence":"additional","affiliation":[]},{"given":"Karim","family":"Jessa","sequence":"additional","affiliation":[]},{"given":"Bren","family":"Cardiff","sequence":"additional","affiliation":[]},{"given":"Lillian","family":"Sung","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2025,1,20]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"121","DOI":"10.1136\/bmjqs-2022-015713","article-title":"Grand rounds in methodology: key considerations for implementing machine learning solutions in quality improvement initiatives","volume":"33","author":"Verma","year":"2024","journal-title":"BMJ Qual Saf"},{"key":"B2","doi-asserted-by":"publisher","first-page":"e40039","DOI":"10.2196\/40039","article-title":"Perspective toward machine learning implementation in pediatric medicine: mixed methods study","volume":"10","author":"Alexander","year":"2022","journal-title":"JMIR Med Inform"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1294","DOI":"10.1038\/bmt.2017.112","article-title":"The burden of chemotherapy-induced nausea and vomiting in children receiving hematopoietic stem cell transplantation conditioning: a prospective study","volume":"52","author":"Flank","year":"2017","journal-title":"Bone Marrow Transplant"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1365","DOI":"10.1007\/s00520-015-2924-1","article-title":"Poor chemotherapy-induced nausea and vomiting control in children receiving intermediate or high dose methotrexate","volume":"24","author":"Vol","year":"2016","journal-title":"Support Care Cancer"},{"key":"B5","doi-asserted-by":"publisher","first-page":"e30395","DOI":"10.1002\/pbc.30395","article-title":"Treatment of breakthrough and prevention of refractory chemotherapy-induced nausea and vomiting in pediatric cancer patients: clinical practice guideline update","volume":"70","author":"Patel","year":"2023","journal-title":"Pediatr Blood Cancer"},{"key":"B6","doi-asserted-by":"publisher","first-page":"e30001","DOI":"10.1002\/pbc.30001","article-title":"Prevention of acute and delayed chemotherapy-induced nausea and vomiting in pediatric cancer patients: a clinical practice guideline","volume":"69","author":"Patel","year":"2022","journal-title":"Pediatr Blood Cancer"},{"key":"B7","doi-asserted-by":"publisher","first-page":"e28947","DOI":"10.1002\/pbc.28947","article-title":"Prevention and treatment of anticipatory chemotherapy-induced nausea and vomiting in pediatric cancer patients and hematopoietic stem cell recipients: clinical practice guideline update","volume":"68","author":"Patel","year":"2021","journal-title":"Pediatr Blood Cancer"},{"key":"B8","doi-asserted-by":"publisher","first-page":"e31295","DOI":"10.1002\/pbc.31295","article-title":"Clinical practice guideline-inconsistent chemotherapy-induced vomiting prophylaxis in pediatric cancer patients in community settings: a children\u2019s oncology group study","volume":"71","author":"Sugalski","year":"2024","journal-title":"Pediatr Blood Cancer"},{"key":"B9","doi-asserted-by":"publisher","first-page":"939292","DOI":"10.3389\/fdgth.2022.939292","article-title":"Clinical deployment environments: five pillars of translational machine learning for health","volume":"4","author":"Harris","year":"2022","journal-title":"Front Digit Health"},{"key":"B10","doi-asserted-by":"publisher","first-page":"826","DOI":"10.4338\/ACI-2017-03-CR-0046","article-title":"Barriers to achieving economies of scale in analysis of EHR data. A cautionary tale","volume":"8","author":"Sendak","year":"2017","journal-title":"Appl Clin Inform"},{"key":"B11","doi-asserted-by":"publisher","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"key":"B12","doi-asserted-by":"publisher","first-page":"e21586","DOI":"10.1016\/j.heliyon.2023.e21586","article-title":"Development and validation of the SickKids enterprise-wide data in azure repository (SEDAR)","volume":"9","author":"Guo","year":"2023","journal-title":"Heliyon"},{"key":"B13","doi-asserted-by":"publisher","first-page":"274","DOI":"10.1093\/jamia\/ocad178","article-title":"A framework for understanding label leakage in machine learning for health care","volume":"31","author":"Davis","year":"2023","journal-title":"J Am Med Inform Assoc"},{"key":"B14","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach Learn"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1172","DOI":"10.1093\/jamia\/ocae060","article-title":"Unmasking bias in artificial intelligence: a systematic review of bias detection and mitigation strategies in electronic health record-based models","volume":"31","author":"Chen","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"B16","doi-asserted-by":"publisher","first-page":"104654","DOI":"10.1016\/j.jbi.2024.104654","article-title":"A roadmap to artificial intelligence (AI): methods for designing and building AI ready data to promote fairness","volume":"154","author":"Kidwai-Khan","year":"2024","journal-title":"J Biomed Inform"},{"key":"B17","doi-asserted-by":"publisher","first-page":"2613","DOI":"10.1093\/jamia\/ocae165","article-title":"Towards objective and systematic evaluation of bias in artificial intelligence for medical imaging","volume":"31","author":"Stanley","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"B18","first-page":"797","article-title":"Algorithmic decision making and the cost of fairness","author":"Corbett-Davies","year":"2017"},{"key":"B19","doi-asserted-by":"publisher","first-page":"3254","DOI":"10.1038\/s41598-022-07167-7","article-title":"A comparison of approaches to improve worst-case predictive model performance over patient subpopulations","volume":"12","author":"Pfohl","year":"2022","journal-title":"Sci Rep"},{"key":"B20","doi-asserted-by":"publisher","first-page":"104338","DOI":"10.1016\/j.jbi.2023.104338","article-title":"The need for multimodal health data modeling: a practical approach for a federated-learning healthcare platform","volume":"141","author":"Cremonesi","year":"2023","journal-title":"J Biomed Inform"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2205.02302","article-title":"Machine learning operations (MLOps): overview, definition, and architecture","author":"Kreuzberger","year":"2022"},{"key":"B22","doi-asserted-by":"crossref","DOI":"10.1145\/3399579.3399867","article-title":"Developments in MLflow: a system to accelerate the machine learning lifecycle","author":"Chen","year":"2020"},{"key":"B23","first-page":"506","article-title":"Mind the performance gap: examining dataset shift during prospective validation","volume-title":"Proceedings of the 6th Machine Learning for Healthcare Conference; Proceedings of Machine Learning Research: PMLR","author":"Otles","year":"2021"},{"key":"B24","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1177\/0272989X9801800115","article-title":"The effect of search procedures on utility elicitations","volume":"18","author":"Lenert","year":"1998","journal-title":"Med Decis Making"},{"key":"B25","doi-asserted-by":"publisher","first-page":"259","DOI":"10.1038\/s41586-023-05881-4","article-title":"Foundation models for generalist medical artificial intelligence","volume":"616","author":"Moor","year":"2023","journal-title":"Nature"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2108.07258","article-title":"On the opportunities and risks of foundation models","author":"Bommasani","year":"2021"},{"key":"B27","doi-asserted-by":"publisher","first-page":"103637","DOI":"10.1016\/j.jbi.2020.103637","article-title":"Language models are an effective representation learning technique for electronic health record data","volume":"113","author":"Steinberg","year":"2021","journal-title":"J Biomed Inform"},{"key":"B28","doi-asserted-by":"publisher","first-page":"2004","DOI":"10.1093\/jamia\/ocad175","article-title":"Self-supervised machine learning using adult inpatient data produces effective models for pediatric clinical prediction tasks","volume":"30","author":"Lemmon","year":"2023","journal-title":"J Am Med Inform Assoc"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01166-w","article-title":"A multi-center study on the adaptability of a shared foundation model for electronic health records","volume":"7","author":"Guo","year":"2024","journal-title":"NPJ Digit Med."},{"key":"B30","doi-asserted-by":"publisher","first-page":"3767","DOI":"10.1038\/s41598-023-30820-8","article-title":"EHR Foundation models improve robustness in the presence of temporal distribution shift","volume":"13","author":"Guo","year":"2023","journal-title":"Sci Rep"},{"key":"B31","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2307.02028","article-title":"EHRSHOT: an EHR benchmark for few-shot evaluation of foundation models","author":"Wornow","year":"2023"},{"key":"B32","doi-asserted-by":"publisher","first-page":"808","DOI":"10.1055\/s-0041-1735184","article-title":"Systematic review of approaches to preserve machine learning performance in the presence of temporal dataset shift in clinical medicine","volume":"12","author":"Guo","year":"2021","journal-title":"Appl Clin Inform"},{"key":"B33","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1093\/biostatistics\/kxz041","article-title":"From development to deployment: dataset shift, causality, and shift-stable models in health AI","volume":"21","author":"Subbaswamy","year":"2020","journal-title":"Biostatistics"},{"key":"B34","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1148\/radiol.2020200038","article-title":"Continuous learning AI in radiology: implementation principles and early applications","volume":"297","author":"Pianykh","year":"2020","journal-title":"Radiology"},{"key":"B35","doi-asserted-by":"publisher","first-page":"1195","DOI":"10.1093\/jamia\/ocae036","article-title":"Sustainable deployment of clinical prediction tools\u2014a 360\u00b0 approach to model maintenance","volume":"31","author":"Davis","year":"2024","journal-title":"J Am Med Inform Assoc"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1448","DOI":"10.1093\/jamia\/ocz127","article-title":"A nonparametric updating method to correct clinical prediction model drift","volume":"26","author":"Davis","year":"2019","journal-title":"J Am Med Inform Assoc"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2311.11463","article-title":"Designing monitoring strategies for deployed machine learning algorithms: navigating performativity through a causal lens","author":"Feng","year":"2023"},{"key":"B38","doi-asserted-by":"publisher","first-page":"32","DOI":"10.1186\/s40359-015-0089-9","article-title":"An introduction to implementation science for the non-specialist","volume":"3","author":"Bauer","year":"2015","journal-title":"BMC Psychol"},{"key":"B39","doi-asserted-by":"publisher","first-page":"e32215","DOI":"10.2196\/32215","article-title":"Implementation frameworks for artificial intelligence translation into health care practice: scoping review","volume":"24","author":"Gama","year":"2022","journal-title":"J Med Internet Res"},{"key":"B40","doi-asserted-by":"publisher","first-page":"e0305949","DOI":"10.1371\/journal.pone.0305949","article-title":"A comprehensive overview of barriers and strategies for AI implementation in healthcare: mixed-method design","volume":"19","author":"Nair","year":"2024","journal-title":"PLoS One"},{"key":"B41","doi-asserted-by":"publisher","first-page":"701","DOI":"10.1186\/s12913-024-11112-x","article-title":"Understanding the integration of artificial intelligence in healthcare organisations and systems through the NASSS framework: a qualitative study in a leading Canadian academic centre","volume":"24","author":"Alami","year":"2024","journal-title":"BMC Health Serv Res"},{"key":"B42","doi-asserted-by":"publisher","first-page":"e084398","DOI":"10.1136\/bmjopen-2024-084398","article-title":"Barriers and facilitators to implementing imaging-based diagnostic artificial intelligence-assisted decision-making software in hospitals in China: a qualitative study using the updated consolidated framework for implementation research","volume":"14","author":"Liao","year":"2024","journal-title":"BMJ Open"},{"key":"B43","first-page":"42","article-title":"The five \u201crights\u201d of clinical decision support","volume":"84","author":"Campbell","year":"2013","journal-title":"J Ahima"},{"key":"B44","doi-asserted-by":"publisher","first-page":"350","DOI":"10.1186\/s12885-023-10835-0","article-title":"Symptom management care pathway adaptation process and specific adaptation decisions","volume":"23","author":"Vettese","year":"2023","journal-title":"BMC Cancer"},{"key":"B45","doi-asserted-by":"publisher","first-page":"7923","DOI":"10.1007\/s00520-022-07216-x","article-title":"Creating and adapting an infection management care pathway in pediatric oncology","volume":"30","author":"Tomlinson","year":"2022","journal-title":"Support Care Cancer"},{"key":"B46","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1055\/s-0039-1677908","article-title":"AI In health: state of the art, challenges, and future directions","volume":"28","author":"Wang","year":"2019","journal-title":"Yearb Med Inform"},{"key":"B47","doi-asserted-by":"publisher","first-page":"826","DOI":"10.5858\/arpa.2021-0635-RA","article-title":"Building the model","volume":"147","author":"Yang","year":"2023","journal-title":"Arch Pathol Lab Med"}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2025.1462751\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,20]],"date-time":"2025-01-20T05:24:36Z","timestamp":1737350676000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2025.1462751\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,20]]},"references-count":47,"alternative-id":["10.3389\/fdgth.2025.1462751"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2025.1462751","relation":{},"ISSN":["2673-253X"],"issn-type":[{"value":"2673-253X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,20]]},"article-number":"1462751"}}