{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T13:29:16Z","timestamp":1789046956462,"version":"build-2803163510"},"reference-count":36,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T00:00:00Z","timestamp":1788998400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T00:00:00Z","timestamp":1788998400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003945","name":"Link\u00f6ping University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003945","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Sepsis remains a major public health concern and is associated with high mortality. Early detection and timely intervention are critical for improving outcomes, yet no standardised approach has been universally adopted. The LiSep LSTM model, a Long Short-Term Memory neural network developed using the MIMIC-III database, has demonstrated promising predictive performance but generalisability across regions has not yet been explored. This study evaluated the LiSep LSTM model in an independent European cohort from the AmsterdamUMC (AUMC) database. Model performance was assessed using internal, external, and pooled validation to examine the effects of distributional shift and data heterogeneity across cohorts. Permutation feature importance (PFI) was further applied to quantify each feature\u2019s contribution to model performance across evaluation settings. When tested on the AUMC cohort, the model achieved an Area Under the Receiver Operating Characteristic Curve (AUROC) of 0.8851 (95% confidence interval [CI] 0.8612\u20130.909) with a median Hours Before Onset (HBO) of 47.0\u00a0hours (interquartile range [IQR]: 16.0\u2013255.0), slightly outperforming its performance on MIMIC-III. External validation showed reduced calibration and recall, while pooled training improved AUROC and Brier score relative to external-only evaluation. Accuracy was moderate (0.60\u20130.77) and precision remained low (0.01\u20130.17), indicating a substantial false-positive burden. These findings suggest that pooled training enhances robustness and supports earlier identification of septic shock, but local calibration and strategies to manage false alarms remain necessary for reliable deployment.<\/jats:p>","DOI":"10.1038\/s41598-026-63350-0","type":"journal-article","created":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T12:46:29Z","timestamp":1789044389000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Evaluating the generalisability of LiSep LSTM for early prediction of septic shock across US and european cohorts"],"prefix":"10.1038","volume":"16","author":[{"given":"Hong K.","family":"Tan","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Logan","family":"Froese","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Wilhelms","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Josef","family":"Fagerstr\u00f6m","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michelle. S.","family":"Chew","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,9,10]]},"reference":[{"key":"63350_CR1","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1001\/jama.2016.0287","volume":"315","author":"M Singer","year":"2016","unstructured":"Singer, M. et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 315, 801 (2016).","journal-title":"JAMA"},{"key":"63350_CR2","doi-asserted-by":"publisher","first-page":"701","DOI":"10.1586\/eri.12.50","volume":"10","author":"GS Martin","year":"2012","unstructured":"Martin, G. S. Sepsis, severe sepsis and septic shock: Changes in incidence, pathogens and outcomes. Expert Rev. Anti Infect. Ther. 10, 701\u2013706 (2012).","journal-title":"Expert Rev. Anti Infect. Ther."},{"key":"63350_CR3","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1186\/s13054-020-02950-2","volume":"24","author":"M Bauer","year":"2020","unstructured":"Bauer, M. et al. Mortality in sepsis and septic shock in Europe, North America and Australia between 2009 and 2019\u2014 results from a systematic review and meta-analysis. Crit Care 24, 239 (2020).","journal-title":"Crit Care"},{"key":"63350_CR4","doi-asserted-by":"publisher","first-page":"117959721985656","DOI":"10.1177\/1179597219856564","volume":"10","author":"CV Cosgriff","year":"2019","unstructured":"Cosgriff, C. V., Celi, L. A. & Stone, D. J. Critical care, critical data. Biomed. Eng. Comput. Biol. 10, 117959721985656 (2019).","journal-title":"Biomed. Eng. Comput. Biol."},{"key":"63350_CR5","doi-asserted-by":"publisher","first-page":"117822261988514","DOI":"10.1177\/1178222619885147","volume":"11","author":"CR Yee","year":"2019","unstructured":"Yee, C. R., Narain, N. R., Akmaev, V. R. & Vemulapalli, V. A data-driven approach to predicting septic shock in the intensive care unit. Biomed. Inform. Insights 11, 1178222619885147 (2019).","journal-title":"Biomed. Inform. Insights"},{"key":"63350_CR6","doi-asserted-by":"publisher","first-page":"1589","DOI":"10.1097\/01.CCM.0000217961.75225.E9","volume":"34","author":"A Kumar","year":"2006","unstructured":"Kumar, A. et al. Duration of hypotension before initiation of effective antimicrobial therapy is the critical determinant of survival in human septic shock*. Crit. Care Med. 34, 1589\u20131596 (2006).","journal-title":"Crit. Care Med."},{"key":"63350_CR7","doi-asserted-by":"publisher","first-page":"113890","DOI":"10.1126\/scitranslmed.aab3719","volume":"7","author":"KE Henry","year":"2015","unstructured":"Henry, K. E., Hager, D. N., Pronovost, P. J. & Saria, S. A targeted real-time early warning score (TREWScore) for septic shock. Sci. Transl. Med. 7, 113890 (2015).","journal-title":"Sci. Transl. Med."},{"key":"63350_CR8","doi-asserted-by":"publisher","first-page":"711","DOI":"10.1038\/s41467-021-20910-4","volume":"12","author":"KH Goh","year":"2021","unstructured":"Goh, K. H. et al. Artificial intelligence in sepsis early prediction and diagnosis using unstructured data in healthcare. Nat. Commun. 12, 711 (2021).","journal-title":"Nat. Commun."},{"key":"63350_CR9","doi-asserted-by":"publisher","first-page":"104110","DOI":"10.1016\/j.compbiomed.2020.104110","volume":"128","author":"A Rafiei","year":"2021","unstructured":"Rafiei, A., Rezaee, A., Hajati, F., Gheisari, S. & Golzan, M. SSP: Early prediction of sepsis using fully connected LSTM-CNN model. Comput. Biol. Med. 128, 104110 (2021).","journal-title":"Comput. Biol. Med."},{"key":"63350_CR10","doi-asserted-by":"publisher","first-page":"101820","DOI":"10.1016\/j.artmed.2020.101820","volume":"104","author":"SM Lauritsen","year":"2020","unstructured":"Lauritsen, S. M. et al. Early detection of sepsis utilizing deep learning on electronic health record event sequences. Artif. Intell. Med. 104, 101820 (2020).","journal-title":"Artif. Intell. Med."},{"key":"63350_CR11","doi-asserted-by":"publisher","first-page":"1447","DOI":"10.1038\/s41591-022-01895-z","volume":"28","author":"KE Henry","year":"2022","unstructured":"Henry, K. E. et al. Factors driving provider adoption of the TREWS machine learning-based early warning system and its effects on sepsis treatment timing. Nat. Med. 28, 1447\u20131454 (2022).","journal-title":"Nat. Med."},{"key":"63350_CR12","doi-asserted-by":"publisher","first-page":"1455","DOI":"10.1038\/s41591-022-01894-0","volume":"28","author":"R Adams","year":"2022","unstructured":"Adams, R. et al. Prospective, multi-site study of patient outcomes after implementation of the TREWS machine learning-based early warning system for sepsis. Nat. Med. 28, 1455\u20131460 (2022).","journal-title":"Nat. Med."},{"key":"63350_CR13","doi-asserted-by":"publisher","first-page":"322","DOI":"10.1186\/s12967-021-03005-y","volume":"19","author":"J Weng","year":"2021","unstructured":"Weng, J. et al. Development and validation of a score to predict mortality in ICU patients with sepsis: A multicenter retrospective study. J. Transl. Med. 19, 322 (2021).","journal-title":"J. Transl. Med."},{"key":"63350_CR14","doi-asserted-by":"publisher","first-page":"102124","DOI":"10.1016\/j.eclinm.2023.102124","volume":"62","author":"M Moor","year":"2023","unstructured":"Moor, M. et al. Predicting sepsis using deep learning across international sites: A retrospective development and validation study. eClinicalMedicine 62, 102124 (2023).","journal-title":"eClinicalMedicine"},{"key":"63350_CR15","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1186\/s40001-024-01756-0","volume":"29","author":"G Zhang","year":"2024","unstructured":"Zhang, G. et al. Predicting sepsis in-hospital mortality with machine learning: A multi-center study using clinical and inflammatory biomarkers. Eur. J. Med. Res. 29, 156 (2024).","journal-title":"Eur. J. Med. Res."},{"key":"63350_CR16","doi-asserted-by":"publisher","first-page":"160035","DOI":"10.1038\/sdata.2016.35","volume":"3","author":"AEW Johnson","year":"2016","unstructured":"Johnson, A. E. W. et al. MIMIC-III, a freely accessible critical care database. Sci. Data 3, 160035 (2016).","journal-title":"Sci. Data"},{"key":"63350_CR17","doi-asserted-by":"publisher","first-page":"15132","DOI":"10.1038\/s41598-019-51219-4","volume":"9","author":"J Fagerstr\u00f6m","year":"2019","unstructured":"Fagerstr\u00f6m, J., B\u00e5ng, M., Wilhelms, D. & Chew, M. S. LiSep LSTM: A machine learning algorithm for early detection of septic shock. Sci. Rep. 9, 15132 (2019).","journal-title":"Sci. Rep."},{"key":"63350_CR18","doi-asserted-by":"publisher","first-page":"e563","DOI":"10.1097\/CCM.0000000000004916","volume":"49","author":"PJ Thoral","year":"2021","unstructured":"Thoral, P. J. et al. Sharing ICU patient data responsibly under the Society of Critical Care Medicine\/European Society of Intensive Care Medicine joint data science collaboration: The Amsterdam University Medical Centers Database (AmsterdamUMCdb) example*. Crit. Care Med. 49, e563\u2013e577 (2021).","journal-title":"Crit. Care Med."},{"key":"63350_CR19","unstructured":"PostgreSQL, Inc. PostgreSQL Global Development Group. PostgreSQL, Inc."},{"key":"63350_CR20","unstructured":"Chollet, F. & others. Keras: The Python Deep Learning library. Astrophysics Source Code Library ascl:1806.022 (2018)."},{"key":"63350_CR21","unstructured":"Abadi, M. et al. TensorFlow: Large-scale machine learning on heterogeneous systems. (2015)."},{"key":"63350_CR22","doi-asserted-by":"publisher","first-page":"1644","DOI":"10.1378\/chest.101.6.1644","volume":"101","author":"RC Bone","year":"1992","unstructured":"Bone, R. C. et al. Definitions for sepsis and organ failure and guidelines for the use of innovative therapies in sepsis. Chest 101, 1644\u20131655 (1992).","journal-title":"Chest"},{"key":"63350_CR23","doi-asserted-by":"publisher","first-page":"1303","DOI":"10.1097\/00003246-200107000-00002","volume":"29","author":"DC Angus","year":"2001","unstructured":"Angus, D. C. et al. Epidemiology of severe sepsis in the United States: Analysis of incidence, outcome, and associated costs of care. Crit. Care Med. 29, 1303\u20131310 (2001).","journal-title":"Crit. Care Med."},{"key":"63350_CR24","doi-asserted-by":"publisher","first-page":"580","DOI":"10.1097\/CCM.0b013e31827e83af","volume":"41","author":"RP Dellinger","year":"2013","unstructured":"Dellinger, R. P. et al. Surviving sepsis campaign: International guidelines for management of severe sepsis and septic shock. Crit. Care Med. 41, 580\u2013637 (2013).","journal-title":"Crit. Care Med."},{"key":"63350_CR25","doi-asserted-by":"publisher","unstructured":"Pereira, R. D. M. A. et al. Predicting septic shock outcomes in a database with missing data using fuzzy modeling: Influence of pre-processing techniques on real-world data-based classification. In 2011 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE 2011) 2507\u20132512 (IEEE, Taipei, Taiwan, 2011). https:\/\/doi.org\/10.1109\/FUZZY.2011.6007606.","DOI":"10.1109\/FUZZY.2011.6007606"},{"key":"63350_CR26","doi-asserted-by":"publisher","unstructured":"Sak, H., Senior, A. & Beaufays, F. Long Short-Term Memory Based Recurrent Neural Network Architectures for Large Vocabulary Speech Recognition. Preprint at https:\/\/doi.org\/10.48550\/ARXIV.1402.1128 (2014).","DOI":"10.48550\/ARXIV.1402.1128"},{"key":"63350_CR27","first-page":"177","volume":"20","author":"A Fisher","year":"2019","unstructured":"Fisher, A., Rudin, C. & Dominici, F. All models are wrong, but many are useful: Learning a variable\u2019s importance by studying an entire class of prediction models simultaneously. J Mach Learn Res 20, 177 (2019).","journal-title":"J Mach Learn Res"},{"key":"63350_CR28","volume-title":"Interpretable Machine Learning: A Guide for Making Black Box Models Explainable","author":"C Molnar","year":"2025","unstructured":"Molnar, C. Interpretable Machine Learning: A Guide for Making Black Box Models Explainable (Christoph Molnar, 2025)."},{"key":"63350_CR29","doi-asserted-by":"publisher","first-page":"104269","DOI":"10.1016\/j.jbi.2022.104269","volume":"139","author":"E Getzen","year":"2023","unstructured":"Getzen, E., Ungar, L., Mowery, D., Jiang, X. & Long, Q. Mining for equitable health: Assessing the impact of missing data in electronic health records. J. Biomed. Inform. 139, 104269 (2023).","journal-title":"J. Biomed. Inform."},{"key":"63350_CR30","doi-asserted-by":"publisher","first-page":"780","DOI":"10.3390\/biomedinformatics4010043","volume":"4","author":"Z Jabbar","year":"2024","unstructured":"Jabbar, Z. & Washington, P. The effect of data missingness on machine learning predictions of uncontrolled diabetes using All of Us data. BioMedInformatics 4, 780\u2013795 (2024).","journal-title":"BioMedInformatics"},{"key":"63350_CR31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41597-022-01899-x","volume":"10","author":"AEW Johnson","year":"2023","unstructured":"Johnson, A. E. W. et al. MIMIC-IV, a freely accessible electronic health record dataset. Sci. Data 10, 1 (2023).","journal-title":"Sci. Data"},{"key":"63350_CR32","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1038\/s41591-020-0789-4","volume":"26","author":"SL Hyland","year":"2020","unstructured":"Hyland, S. L. et al. Early prediction of circulatory failure in the intensive care unit using machine learning. Nat. Med. 26, 364\u2013373 (2020).","journal-title":"Nat. Med."},{"key":"63350_CR33","doi-asserted-by":"publisher","first-page":"11760","DOI":"10.1038\/s41598-023-38858-4","volume":"13","author":"JK Valik","year":"2023","unstructured":"Valik, J. K. et al. Predicting sepsis onset using a machine learned causal probabilistic network algorithm based on electronic health records data. Sci. Rep. 13, 11760 (2023).","journal-title":"Sci. Rep."},{"key":"63350_CR34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3501813","volume":"13","author":"RS Antunes","year":"2022","unstructured":"Antunes, R. S., Andr\u00e9 Da Costa, C., K\u00fcderle, A., Yari, I. A. & Eskofier, B. Federated learning for healthcare: Systematic review and architecture proposal. ACM Trans. Intell. Syst. Technol. 13, 1\u201323 (2022).","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"63350_CR35","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1038\/s41746-020-00323-1","volume":"3","author":"N Rieke","year":"2020","unstructured":"Rieke, N. et al. The future of digital health with federated learning. npj Digit. Med. 3, 119 (2020).","journal-title":"npj Digit. Med."},{"key":"63350_CR36","doi-asserted-by":"publisher","first-page":"e215","DOI":"10.1161\/01.CIR.101.23.e215","volume":"101","author":"AL Goldberger","year":"2000","unstructured":"Goldberger, A. L. et al. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation 101, e215 (2000).","journal-title":"Circulation"}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-63350-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-63350-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-63350-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,10]],"date-time":"2026-09-10T13:00:39Z","timestamp":1789045239000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-026-63350-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,10]]},"references-count":36,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,12]]}},"alternative-id":["63350"],"URL":"https:\/\/doi.org\/10.1038\/s41598-026-63350-0","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,10]]},"assertion":[{"value":"7 January 2026","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 July 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 September 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","label":"Competing interests","group":{"name":"EthicsHeading","label":"Declarations"}}],"article-number":"28358"}}