{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T16:01:09Z","timestamp":1753891269738,"version":"3.41.2"},"reference-count":18,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,2,8]],"date-time":"2022-02-08T00:00:00Z","timestamp":1644278400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:p>The increased use of electronic health records (EHRs) has improved the availability of routine care data for medical research. Combined with machine learning techniques this has spurred the development of early warning scores (EWSs) in hospitals worldwide. EWSs are commonly used in the hospital where they have been developed, yet few have been transported to external settings and\/or internationally. In this perspective, we describe our experiences in implementing the TREWScore, a septic shock EWS, and the transportability challenges regarding domain, predictors, and clinical outcome we faced. We used data of 53,330 ICU stays from Medical Information Mart for Intensive Care-III (MIMIC-III) and 18,013 ICU stays from the University Medical Center (UMC) Utrecht, including 17,023 (31.9%) and 2,557 (14.2%) cases of sepsis, respectively. The MIMIC-III and UMC populations differed significantly regarding the length of stay (6.9 vs. 9.0 days) and hospital mortality (11.6% vs. 13.6%). We mapped all 54 TREWScore predictors to the UMC database: 31 were readily available, seven required unit conversion, 14 had to be engineered, one predictor required text mining, and one predictor could not be mapped. Lastly, we classified sepsis cases for septic shock using the sepsis-2 criteria. Septic shock populations (UMC 31.3% and MIMIC-III 23.3%) and time to shock events showed significant differences between the two cohorts. In conclusion, we identified challenges to transportability and implementation regarding domain, predictors, and clinical outcome when transporting EWS between hospitals across two continents. These challenges need to be systematically addressed to improve model transportability between centers and unlock the potential clinical utility of EWS.<\/jats:p>","DOI":"10.3389\/fmed.2021.793815","type":"journal-article","created":{"date-parts":[[2022,2,8]],"date-time":"2022-02-08T04:38:19Z","timestamp":1644295099000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Transportability and Implementation Challenges of Early Warning Scores for Septic Shock in the ICU: A Perspective on the TREWScore"],"prefix":"10.3389","volume":"8","author":[{"given":"Michael S. A.","family":"Niemantsverdriet","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Meri R. J.","family":"Varkila","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jacqueline L. P.","family":"Vromen-Wijsman","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Imo E.","family":"Hoefer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Domenico","family":"Bellomo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Martin H.","family":"van Vliet","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wouter W.","family":"van Solinge","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Olaf L.","family":"Cremer","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Saskia","family":"Haitjema","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2022,2,8]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1793","DOI":"10.1097\/00003246-199811000-00016","article-title":"Use of the SOFA score to assess the incidence of organ dysfunction\/failure in intensive care units: results of a multicenter, prospective study","volume":"26","author":"Vincent","year":"1998","journal-title":"Crit Care Med."},{"key":"B2","doi-asserted-by":"publisher","first-page":"2957","DOI":"10.1001\/jama.1993.03510240069035","article-title":"A new simplified acute physiology score (SAPS II) based on a European\/North American multicenter study","volume":"270","author":"Le Gall","year":"1993","journal-title":"JAMA."},{"key":"B3","doi-asserted-by":"publisher","first-page":"818","DOI":"10.1097\/00003246-198510000-00009","article-title":"APACHE II a severity of disease classification system","volume":"13","author":"Knaus","year":"1985","journal-title":"Crit Care Med."},{"key":"B4","doi-asserted-by":"publisher","first-page":"383","DOI":"10.1007\/s00134-019-05872-y","article-title":"Machine learning for the prediction of sepsis: a systematic review and meta-analysis of diagnostic test accuracy","volume":"46","author":"Fleuren","year":"2020","journal-title":"Intensive Care Med."},{"key":"B5","doi-asserted-by":"publisher","first-page":"284","DOI":"10.1186\/s13054-019-2564-9","article-title":"Use of machine learning to analyse routinely collected intensive care unit data: a systematic review","volume":"23","author":"Shillan","year":"2019","journal-title":"Crit Care."},{"key":"B6","doi-asserted-by":"publisher","first-page":"5668","DOI":"10.1038\/s41467-020-19551-w","article-title":"Cross-site transportability of an explainable artificial intelligence model for acute kidney injury prediction","volume":"11","author":"Song","year":"2020","journal-title":"Nat Commun."},{"key":"B7","unstructured":"Review of challenges and opportunities in machine learning for health191\n            GhassemiM\n            NaumannT\n            SchulamP\n            BeamAL\n            ChenIY\n            RanganathR\n          32477638AMIA Summits Transl Sci Proc.20202020"},{"key":"B8","doi-asserted-by":"publisher","first-page":"299ra122","DOI":"10.1126\/scitranslmed.aab3719","article-title":"A targeted real-time early warning score (TREWScore) for septic shock","volume":"7","author":"Henry","year":"2015","journal-title":"Sci Transl Med."},{"key":"B9","doi-asserted-by":"publisher","first-page":"E215","DOI":"10.1161\/01.cir.101.23.e215","article-title":"Physiobank, physiotoolkit, and physionet: Components of a new research resource for complex physiologic signals","volume":"101","author":"Goldberger","year":"2000","journal-title":"Circulation."},{"key":"B10","doi-asserted-by":"publisher","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data."},{"key":"B11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.18637\/jss.v033.i01","article-title":"Regularization paths for generalized linear models via coordinate descent","volume":"33","author":"Friedman","year":"2010","journal-title":"J Stat Softw."},{"key":"B12","doi-asserted-by":"publisher","first-page":"2787","DOI":"10.1097\/CCM.0b013e318186aec8","article-title":"Variation in critical care services across North America and Western Europe","volume":"36","author":"Wunsch","year":"2008","journal-title":"Crit Care Med"},{"key":"B13","doi-asserted-by":"publisher","first-page":"1065","DOI":"10.1001\/jamainternmed.2021.2626","article-title":"External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients","volume":"181","author":"Wong","year":"2021","journal-title":"JAMA Intern Med"},{"key":"B14","doi-asserted-by":"publisher","first-page":"m3164","DOI":"10.1136\/bmj.m3164","article-title":"Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension","volume":"370","author":"Liu","year":"2020","journal-title":"BMJ"},{"key":"B15","doi-asserted-by":"publisher","first-page":"m3210","DOI":"10.1136\/bmj.m3210","article-title":"Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension","volume":"370","author":"Rivera","year":"2020","journal-title":"BMJ."},{"key":"B16","doi-asserted-by":"publisher","first-page":"899","DOI":"10.1093\/jamia\/ocv189","article-title":"SMART on FHIR: a standards-based, interoperable apps platform for electronic health records","volume":"23","author":"Mandel","year":"2016","journal-title":"J Am Med Informatics Assoc."},{"key":"B17","unstructured":"SNOMED-CT: the advanced terminology and coding system for eHealth279\n            DonnellyK\n          17095826Stud Health Technol Inform.1212006"},{"key":"B18","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1161\/CIRCULATIONAHA.114.014508","article-title":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) the TRIPOD statement","volume":"131","author":"Collins","year":"2015","journal-title":"Circulation."}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2021.793815\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,2,8]],"date-time":"2022-02-08T04:38:21Z","timestamp":1644295101000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2021.793815\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,8]]},"references-count":18,"alternative-id":["10.3389\/fmed.2021.793815"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2021.793815","relation":{},"ISSN":["2296-858X"],"issn-type":[{"type":"electronic","value":"2296-858X"}],"subject":[],"published":{"date-parts":[[2022,2,8]]},"article-number":"793815"}}