{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T13:00:00Z","timestamp":1790686800375,"version":"4.1.0"},"reference-count":55,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T00:00:00Z","timestamp":1790294400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Background: Longitudinal severity assessment may provide more informative risk stratification than reliance on a single admission score in intensive care. This study developed and internally validated an explainable machine learning framework for predicting a subsequent APACHE-II-defined high-severity state using repeated APACHE-II assessments in intensive care unit (ICU) patients receiving total parenteral nutrition (TPN). The endpoint was defined as an APACHE-II score of \u2265 20 at the final assessment (T10), while predictor information was restricted to measurements obtained at T0\u2013T9. Methods: A retrospective institutional cohort of 844 adult ICU patients was analyzed, of whom 214 (25.4%) met the predefined high-severity endpoint. Four principal feature representations were evaluated: admission APACHE-II, longitudinal APACHE-II information combining the raw T0\u2013T9 sequence with derived trajectory descriptors, longitudinal laboratory information, and multimodal combinations including baseline characteristics. Six machine learning algorithms were compared using stratified five-fold cross-validation and independent hold-out testing. Additional analyses included simpler APACHE-II comparators, sequential observation truncation, nested cross-validation, calibration assessment, threshold sensitivity analysis, and SHAP-based model interpretation. Because equivalent longitudinal APACHE-II measurements and an equivalent endpoint were unavailable in eICU, a separate harmonized XGBoost model was used only for exploratory cross-cohort portability assessment. Results: The longitudinal APACHE-II CatBoost model achieved the highest performance, with a cross-validated ROC-AUC of 0.924 and an independent hold-out ROC-AUC of 0.921 (95% CI: 0.878\u20130.960). Hold-out calibration was good (Brier score = 0.095, calibration intercept = \u2212 0.03, slope = 0.98), and nested cross-validation yielded a pooled out-of-fold ROC-AUC of 0.908 and a mean outer-fold ROC-AUC of 0.920 \u00b1 0.038. The latest APACHE-II assessment alone (T9; ROC-AUC = 0.553), change from baseline (0.576), ordinal slope (0.539), and logistic regression using engineered APACHE-II descriptors (0.633) performed substantially below the full longitudinal CatBoost model. Sequential truncation showed that discrimination was retained after removing later observations, although performance varied non-monotonically across truncated sequences. SHAP analysis showed that multiple APACHE-II observations contributed prominently to model predictions, with trajectory-derived descriptors providing complementary contributions. Adding laboratory trajectories or baseline characteristics did not improve discrimination over the longitudinal APACHE-II representation. Conclusions: Nonlinear modeling of repeated APACHE-II assessments provided strong internal discrimination of a subsequent APACHE-II-defined high-severity state in this TPN-treated ICU cohort and substantially outperformed single-score and simpler longitudinal comparators. However, the predictors and endpoint share the APACHE-II construct, observation indices were sequential rather than standardized clock-time intervals, and the primary model could not undergo conventional external validation in eICU. These findings therefore support the internal predictive value of longitudinal APACHE-II modeling for this specific severity-state task and warrant prospective multicenter evaluation using standardized timing and independent clinical outcomes.<\/jats:p>","DOI":"10.3390\/diagnostics16193118","type":"journal-article","created":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T14:36:34Z","timestamp":1790346994000},"page":"3118","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Explainable Machine Learning for Prediction of Future High-Severity States Using Longitudinal APACHE-II Trajectories: Internal Validation and Cross-Cohort Portability Assessment"],"prefix":"10.3390","volume":"16","author":[{"given":"\u0130rem","family":"Akpolat","sequence":"first","affiliation":[{"name":"Department of Nutrition and Dietetics, \u0130stanbul Okan University, Istanbul 34959, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"software","vocabulary":"credit"},{"role":"data-curation","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"formal-analysis","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"}]},{"given":"Fatma","family":"\u00c7elik","sequence":"additional","affiliation":[{"name":"Department of Nutrition and Dietetics, Faculty of Health Sciences, Biruni University, Istanbul 34015, T\u00fcrkiye"}],"role":[{"vocabulary":"crossref","role":"author"},{"role":"supervision","vocabulary":"credit"},{"role":"conceptualization","vocabulary":"credit"},{"role":"methodology","vocabulary":"credit"},{"role":"writing-original-draft","vocabulary":"credit"},{"role":"writing-review-editing","vocabulary":"credit"},{"role":"investigation","vocabulary":"credit"},{"role":"visualization","vocabulary":"credit"},{"role":"validation","vocabulary":"credit"}]}],"member":"1968","published-online":{"date-parts":[[2026,9,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1016\/S2213-2600(14)70061-X","article-title":"Assessment of the worldwide burden of critical illness: The Intensive Care Over Nations (ICON) audit","volume":"2","author":"Vincent","year":"2014","journal-title":"Lancet Respir. Med."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1001\/jama.2017.17687","article-title":"Enhancing recovery from sepsis: A review","volume":"319","author":"Prescott","year":"2018","journal-title":"JAMA"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"22129","DOI":"10.1038\/s41598-020-79142-z","article-title":"Dynamic survival prediction in intensive care units from heterogeneous time series without the need for variable selection or curation","volume":"10","author":"Deasy","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1671","DOI":"10.1016\/j.clnu.2023.07.011","article-title":"ESPEN practical and partially revised guideline: Clinical nutrition in the intensive care unit","volume":"42","author":"Singer","year":"2023","journal-title":"Clin. Nutr."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1951","DOI":"10.1056\/NEJMsa2001090","article-title":"Automated identification of adults at risk for in-hospital clinical deterioration","volume":"383","author":"Escobar","year":"2020","journal-title":"N. Engl. J. Med."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1093\/qjmed\/94.10.521","article-title":"Validation of a modified early warning score in medical admissions","volume":"94","author":"Subbe","year":"2001","journal-title":"QJM"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"818","DOI":"10.1097\/00003246-198510000-00009","article-title":"APACHE II: A severity of disease classification system","volume":"13","author":"Knaus","year":"1985","journal-title":"Crit. Care Med."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"707","DOI":"10.1007\/BF01709751","article-title":"The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction\/failure","volume":"22","author":"Vincent","year":"1996","journal-title":"Intensive Care Med."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2957","DOI":"10.1001\/jama.1993.03510240069035","article-title":"A new Simplified Acute Physiology Score (SAPS II) based on a European\/North American multicenter study","volume":"270","author":"Lemeshow","year":"1993","journal-title":"JAMA"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1097\/CCM.0b013e3181b090d0","article-title":"Critical care medicine in the United States 2000\u20132005: An analysis of bed numbers, occupancy rates, payer mix, and costs","volume":"38","author":"Halpern","year":"2010","journal-title":"Crit. Care Med."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2712","DOI":"10.1097\/CCM.0b013e318298a139","article-title":"ICU occupancy and mechanical ventilator use in the United States","volume":"41","author":"Wunsch","year":"2013","journal-title":"Crit. Care Med."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci. Data"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1038\/s41746-018-0029-1","article-title":"Scalable and accurate deep learning with electronic health records","volume":"1","author":"Rajkomar","year":"2018","journal-title":"npj Digit. Med."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1297","DOI":"10.1097\/01.CCM.0000215112.84523.F0","article-title":"Acute Physiology and Chronic Health Evaluation (APACHE) IV: Hospital mortality assessment for today\u2019s critically ill patients","volume":"34","author":"Zimmerman","year":"2006","journal-title":"Crit. Care Med."},{"key":"ref_15","first-page":"4","article-title":"APACHE-II score correlation with mortality and length of stay in an intensive care unit","volume":"21","author":"Naved","year":"2011","journal-title":"J. Coll. Physicians Surg. Pak."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1378\/chest.11-0330","article-title":"Severity scoring in the critically ill: Part 1\u2014Interpretation and accuracy of outcome prediction scoring systems","volume":"141","author":"Breslow","year":"2012","journal-title":"Chest"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1378\/chest.11-0331","article-title":"Severity scoring in the critically ill: Part 2\u2014Maximizing value from outcome prediction scoring systems","volume":"141","author":"Breslow","year":"2012","journal-title":"Chest"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"R4","DOI":"10.1186\/cc3913","article-title":"A comparison of admission and worst 24-hour Acute Physiology and Chronic Health Evaluation II scores in predicting hospital mortality: A retrospective cohort study","volume":"10","author":"Ho","year":"2006","journal-title":"Crit. Care"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1186\/cc8204","article-title":"Clinical review: Scoring systems in the critically ill","volume":"14","author":"Vincent","year":"2010","journal-title":"Crit. Care"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e179","DOI":"10.1016\/S2589-7500(20)30018-2","article-title":"Dynamic and explainable machine learning prediction of mortality in patients in the intensive care unit","volume":"2","author":"Nielsen","year":"2020","journal-title":"Lancet Digit. Health"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Tian, Y., Yao, Y., Zhou, J., Diao, X., Chen, H., Cai, K., Ma, X., and Wang, S. (2022). Dynamic APACHE II score to predict the outcome of intensive care unit patients. Front. Med., 8.","DOI":"10.3389\/fmed.2021.744907"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1109\/JPROC.2015.2501978","article-title":"Machine learning and decision support in critical care","volume":"104","author":"Johnson","year":"2016","journal-title":"Proc. IEEE"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1347","DOI":"10.1056\/NEJMra1814259","article-title":"Machine learning in medicine","volume":"380","author":"Rajkomar","year":"2019","journal-title":"N. Engl. J. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A scalable tree boosting system. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_26","first-page":"3146","article-title":"LightGBM: A highly efficient gradient boosting decision tree","volume":"30","author":"Ke","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_27","first-page":"6638","article-title":"CatBoost: Unbiased boosting with categorical features","volume":"31","author":"Prokhorenkova","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Luo, Y., Wang, Z., and Wang, C. (2021). Improvement of APACHE II score system for disease severity based on XGBoost algorithm. BMC Med. Inform. Decis. Mak., 21.","DOI":"10.1186\/s12911-021-01591-x"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Pang, K., Li, L., Ouyang, W., Liu, X., and Tang, Y. (2022). Establishment of ICU Mortality Risk Prediction Models with Machine Learning Algorithm Using MIMIC-IV Database. Diagnostics, 12.","DOI":"10.3390\/diagnostics12051068"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Mu\u0219at, F., P\u0103duraru, D.N., Bolocan, A., Palc\u0103u, C.A., Cop\u0103ceanu, A.-M., Ion, D., Jinga, V., and Andronic, O. (2024). Machine Learning Models in Sepsis Outcome Prediction for ICU Patients: Integrating Routine Laboratory Tests\u2014A Systematic Review. Biomedicines, 12.","DOI":"10.3390\/biomedicines12122892"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1038\/s42256-019-0138-9","article-title":"From local explanations to global understanding with explainable AI for trees","volume":"2","author":"Lundberg","year":"2020","journal-title":"Nat. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Cho, K., van Merri\u00ebnboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., and Bengio, Y. (2014, January 25\u201329). Learning phrase representations using RNN encoder-decoder for statistical machine translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, Qatar.","DOI":"10.3115\/v1\/D14-1179"},{"key":"ref_34","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems 30, Long Beach, CA, USA."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"112","DOI":"10.1016\/j.jbi.2018.04.007","article-title":"Benchmarking deep learning models on large healthcare datasets","volume":"83","author":"Purushotham","year":"2018","journal-title":"J. Biomed. Inform."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"7155","DOI":"10.1038\/s41598-020-62922-y","article-title":"BEHRT: Transformer for electronic health records","volume":"10","author":"Li","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1589","DOI":"10.1109\/JBHI.2017.2767063","article-title":"Deep EHR: A survey of recent advances in deep learning techniques for electronic health record analysis","volume":"22","author":"Shickel","year":"2018","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"7180","DOI":"10.1038\/s41598-022-11226-4","article-title":"Mortality prediction of patients in intensive care units using machine learning algorithms based on electronic health records","volume":"12","author":"Choi","year":"2022","journal-title":"Sci. Rep."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"154106","DOI":"10.1016\/j.jcrc.2022.154106","article-title":"Development of a machine learning model for the prediction of the short-term mortality in patients in the intensive care unit","volume":"71","author":"Yang","year":"2022","journal-title":"J. Crit. Care"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Kelly, C.J., Karthikesalingam, A., Suleyman, M., Corrado, G., and King, D. (2019). Key challenges for delivering clinical impact with artificial intelligence. BMC Med., 17.","DOI":"10.1186\/s12916-019-1426-2"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41597-022-01899-x","article-title":"MIMIC-IV, a freely accessible electronic health record dataset","volume":"10","author":"Johnson","year":"2023","journal-title":"Sci. Data"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"e1312","DOI":"10.1002\/widm.1312","article-title":"Causability and explainability of artificial intelligence in medicine","volume":"9","author":"Holzinger","year":"2019","journal-title":"WIREs Data Min. Knowl. Discov."},{"key":"ref_44","first-page":"359","article-title":"What clinicians want: Contextualizing explainable machine learning for clinical end use","volume":"106","author":"Tonekaboni","year":"2019","journal-title":"Proc. Mach. Learn. Res."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1186\/s12931-024-02874-3","article-title":"An explainable machine learning-based model to predict intensive care unit admission among patients with community-acquired pneumonia and connective tissue disease","volume":"25","author":"Huang","year":"2024","journal-title":"Respir. Res."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1186\/s40635-025-00765-5","article-title":"Explainable machine learning models for mortality prediction in patients with sepsis in tertiary care hospital ICU in low- to middle-income countries","volume":"13","author":"Diwan","year":"2025","journal-title":"Intensive Care Med. Exp."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1577","DOI":"10.1016\/S0140-6736(19)30037-6","article-title":"Reporting of artificial intelligence prediction models","volume":"393","author":"Collins","year":"2019","journal-title":"Lancet"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1038\/s42256-021-00307-0","article-title":"Common pitfalls and recommendations for using machine learning to detect and prognosticate for COVID-19 using chest radiographs and CT scans","volume":"3","author":"Roberts","year":"2021","journal-title":"Nat. Mach. Intell."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"m689","DOI":"10.1136\/bmj.m689","article-title":"Artificial intelligence versus clinicians: Systematic review of design, reporting standards, and claims of deep learning studies","volume":"368","author":"Nagendran","year":"2020","journal-title":"BMJ"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1186\/s13054-024-04866-7","article-title":"Real-time machine learning model to predict short-term mortality in critically ill patients: Development and international validation","volume":"28","author":"Lim","year":"2024","journal-title":"Crit. Care"},{"key":"ref_51","first-page":"994","article-title":"Real-time mortality prediction in the Intensive Care Unit","volume":"2017","author":"Johnson","year":"2017","journal-title":"AMIA Annu. Symp. Proc."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"180178","DOI":"10.1038\/sdata.2018.178","article-title":"The eICU Collaborative Research Database, a freely available multi-center database for critical care research","volume":"5","author":"Pollard","year":"2018","journal-title":"Sci. Data"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Hosmer, D.W., Lemeshow, S., and Sturdivant, R.X. (2013). Applied Logistic Regression, John Wiley & Sons, Inc.. [3rd ed.].","DOI":"10.1002\/9781118548387"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s10994-006-6226-1","article-title":"Extremely randomized trees","volume":"63","author":"Geurts","year":"2006","journal-title":"Mach. Learn."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Ribeiro, M.T., Singh, S., and Guestrin, C. (2016, January 13\u201317). \u201cWhy Should I Trust You?\u201d: Explaining the predictions of any classifier. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939778"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/19\/3118\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T12:32:38Z","timestamp":1790685158000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/16\/19\/3118"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,25]]},"references-count":55,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2026,10]]}},"alternative-id":["diagnostics16193118"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics16193118","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,25]]}}}