{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T01:28:26Z","timestamp":1790558906451,"version":"4.1.0"},"reference-count":53,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,5,17]],"date-time":"2024-05-17T00:00:00Z","timestamp":1715904000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Med."],"abstract":"<jats:sec>\n                    <jats:title>Background and objective<\/jats:title>\n                    <jats:p>Delirium is the most common neuropsychological complication among older adults admitted to the intensive care unit (ICU) and is often associated with a poor prognosis. This study aimed to construct and validate an interpretable machine learning (ML) for early delirium prediction in older ICU patients.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>This was a retrospective observational cohort study and patient data were extracted from the Medical Information Mart for Intensive Care-IV database. Feature variables associated with delirium, including predisposing factors, disease-related factors, and iatrogenic and environmental factors, were selected using least absolute shrinkage and selection operator regression, and prediction models were built using logistic regression, decision trees, support vector machines, extreme gradient boosting (XGBoost), k-nearest neighbors and naive Bayes methods. Multiple metrics were used for evaluation of performance of the models, including the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, recall, F1 score, calibration plot, and decision curve analysis. SHapley Additive exPlanations (SHAP) were used to improve the interpretability of the final model.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Nine thousand seven hundred forty-eight adults aged 65\u2009years or older were included for analysis. Twenty-six features were selected to construct ML prediction models. Among the models compared, the XGBoost model demonstrated the best performance including the highest AUC (0.836), accuracy (0.765), sensitivity (0.713), recall (0.713), and F1 score (0.725) in the training set. It also exhibited excellent discrimination with AUC of 0.810, good calibration, and had the highest net benefit in the validation cohort. The SHAP summary analysis showed that Glasgow Coma Scale, mechanical ventilation, and sedation were the top three risk features for outcome prediction. The SHAP dependency plot and SHAP force analysis interpreted the model at both the factor level and individual level, respectively.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>ML is a reliable tool for predicting the risk of critical delirium in elderly patients. By combining XGBoost and SHAP, it can provide clear explanations for personalized risk prediction and more intuitive understanding of the effect of key features in the model. The establishment of such a model would facilitate the early risk assessment and prompt intervention for delirium.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.3389\/fmed.2024.1399848","type":"journal-article","created":{"date-parts":[[2024,5,17]],"date-time":"2024-05-17T00:34:10Z","timestamp":1715906050000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":26,"title":["Interpretable machine learning model for early prediction of delirium in elderly patients following intensive care unit admission: a derivation and validation study"],"prefix":"10.3389","volume":"11","author":[{"given":"Dayu","family":"Tang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chengyong","family":"Ma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Xu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,5,17]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","first-page":"ITC49","DOI":"10.7326\/AITC202010060","article-title":"Delirium","volume":"173","author":"Mattison","year":"2020","journal-title":"Ann Intern Med"},{"key":"ref2","doi-asserted-by":"publisher","first-page":"785","DOI":"10.1176\/appi.ajp.2018.18070893","article-title":"Delirium","volume":"176","author":"Thom","year":"2019","journal-title":"Am J Psychiatry"},{"key":"ref3","doi-asserted-by":"publisher","first-page":"1089","DOI":"10.1007\/s00134-021-06503-1","article-title":"Delirium in critical illness: clinical manifestations, outcomes, and management","volume":"47","author":"Stollings","year":"2021","journal-title":"Intensive Care Med"},{"key":"ref4","doi-asserted-by":"publisher","first-page":"210","DOI":"10.1038\/nrneurol.2009.24","article-title":"Delirium in elderly adults: diagnosis, prevention and treatment","volume":"5","author":"Fong","year":"2009","journal-title":"Nat Rev Neurol"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"911","DOI":"10.1016\/S0140-6736(13)60688-1","article-title":"Delirium in elderly people","volume":"383","author":"Inouye","year":"2014","journal-title":"Lancet"},{"key":"ref6","doi-asserted-by":"publisher","first-page":"1456","DOI":"10.1056\/NEJMcp1605501","article-title":"Delirium in hospitalized older adults","volume":"377","author":"Marcantonio","year":"2017","journal-title":"N Engl J Med"},{"key":"ref7","doi-asserted-by":"publisher","first-page":"322","DOI":"10.1097\/JNR.0000000000000082","article-title":"Factors associated with the development of delirium in elderly patients in intensive care units","volume":"23","author":"Lin","year":"2015","journal-title":"J Nurs Res"},{"key":"ref8","doi-asserted-by":"publisher","first-page":"n1007","DOI":"10.1136\/bmj.n1007","article-title":"Delirium and long term cognition in critically ill patients","volume":"373","author":"Wilcox","year":"2021","journal-title":"BMJ"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1186\/s13054-015-0886-9","article-title":"A systematic review of implementation strategies for assessment, prevention, and management of ICU delirium and their effect on clinical outcomes","volume":"19","author":"Trogrli\u0107","year":"2015","journal-title":"Crit Care"},{"key":"ref10","doi-asserted-by":"publisher","first-page":"1326","DOI":"10.1016\/j.chest.2021.06.002","article-title":"How we prevent and treat delirium in the ICU","volume":"160","author":"Palakshappa","year":"2021","journal-title":"Chest"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"e825","DOI":"10.1097\/CCM.0000000000003299","article-title":"Clinical practice guidelines for the prevention and management of pain, agitation\/sedation, delirium, immobility, and sleep disruption in adult patients in the ICU","volume":"46","author":"Devlin","year":"2018","journal-title":"Crit Care Med"},{"key":"ref12","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1007\/s00134-013-3202-7","article-title":"Recalibration of the delirium prediction model for ICU patients (PRE-DELIRIC): a multinational observational study","volume":"40","author":"van den Boogaard","year":"2014","journal-title":"Intensive Care Med"},{"key":"ref13","doi-asserted-by":"publisher","first-page":"e7543","DOI":"10.1097\/MD.0000000000007543","article-title":"Development and validation of risk-stratification delirium prediction model for critically ill patients: a prospective, observational, single-center study","volume":"96","author":"Chen","year":"2017","journal-title":"Medicine"},{"key":"ref14","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1016\/j.chest.2018.03.013","article-title":"Acute Brain Dysfunction: Development and Validation of a Daily Prediction Model","volume":"154","author":"Marra","year":"2018","journal-title":"Chest"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"e420","DOI":"10.1136\/bmj.e420","article-title":"Development and validation of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction model for intensive care patients: observational multicentre study","volume":"344","author":"van den Boogaard","year":"2012","journal-title":"BMJ"},{"key":"ref16","doi-asserted-by":"publisher","first-page":"801","DOI":"10.1016\/S2213-2600(18)30425-9","article-title":"Opening the black box of machine learning","volume":"6","year":"2018","journal-title":"Lancet Respir Med"},{"key":"ref17","doi-asserted-by":"publisher","first-page":"56","DOI":"10.1038\/s42256-019-0138-9","article-title":"From local explanations to global understanding with explainable AI for trees","volume":"2","author":"Lundberg","year":"2020","journal-title":"Nat Mach Intell"},{"key":"ref18","doi-asserted-by":"publisher","first-page":"265","DOI":"10.1007\/s11606-020-06238-7","article-title":"Machine Learning to Develop and Internally Validate a Predictive Model for Post-operative Delirium in a Prospective, Observational Clinical Cohort Study of Older Surgical Patients","volume":"36","author":"Racine","year":"2021","journal-title":"J Gen Intern Med."},{"key":"ref19","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1038\/s41398-024-02762-w","article-title":"Prediction models for postoperative delirium in elderly patients with machine-learning algorithms and SHapley additive exPlanations","volume":"14","author":"Song","year":"2024","journal-title":"Transl Psychiatry"},{"key":"ref20","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1093\/gerona\/glaa138","article-title":"Machine learning in aging: an example of developing prediction models for serious fall injury in older adults","volume":"76","author":"Speiser","year":"2021","journal-title":"J Gerontol A"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1016\/j.ccc.2023.02.001","article-title":"Predictive modeling using artificial intelligence and machine learning algorithms on electronic health record data: advantages and challenges","volume":"39","author":"Patton","year":"2023","journal-title":"Crit Care Clin"},{"key":"ref22","doi-asserted-by":"publisher","first-page":"749","DOI":"10.1038\/s41551-018-0304-0","article-title":"Explainable machine-learning predictions for the prevention of hypoxaemia during surgery","volume":"2","author":"Lundberg","year":"2018","journal-title":"Nat Biomed Eng"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"2861","DOI":"10.1016\/j.csbj.2022.06.003","article-title":"Application of interpretable machine learning for early prediction of prognosis in acute kidney injury","volume":"20","author":"Hu","year":"2022","journal-title":"Comput Struct Biotechnol J"},{"key":"ref24","doi-asserted-by":"publisher","first-page":"107772","DOI":"10.1016\/j.cmpb.2023.107772","article-title":"Interpretable machine-learning model for real-time, clustered risk factor analysis of sepsis and septic death in critical care","volume":"241","author":"Jiang","year":"2023","journal-title":"Comput Methods Prog Biomed"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"160035","DOI":"10.1038\/sdata.2016.35","article-title":"MIMIC-III, a freely accessible critical care database","volume":"3","author":"Johnson","year":"2016","journal-title":"Sci Data"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41597-022-01899-x","article-title":"MIMIC-IV, a freely accessible electronic health record dataset","volume":"10","author":"Johnson","year":"2023","journal-title":"Sci Data"},{"key":"ref27","doi-asserted-by":"publisher","first-page":"g7594","DOI":"10.1136\/bmj.g7594","article-title":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement","volume":"350","author":"Collins","year":"2015","journal-title":"BMJ"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"1370","DOI":"10.1097\/00003246-200107000-00012","article-title":"Evaluation of delirium in critically ill patients: validation of the confusion assessment method for the intensive care unit (CAM-ICU)","volume":"29","author":"Ely","year":"2001","journal-title":"Crit Care Med"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1016\/j.ahj.2020.11.011","article-title":"Diagnosis, prevention, and management of delirium in the intensive cardiac care unit","volume":"232","author":"Cort\u00e9s-Beringola","year":"2021","journal-title":"Am Heart J"},{"key":"ref30","doi-asserted-by":"publisher","first-page":"104582","DOI":"10.1016\/j.ijnurstu.2023.104582","article-title":"Inclusion of frailty improved performance of delirium prediction for elderly patients in the cardiac intensive care unit (D-FRAIL): a prospective derivation and external validation study","volume":"147","author":"Guo","year":"2023","journal-title":"Int J Nurs Stud"},{"key":"ref31","doi-asserted-by":"publisher","first-page":"683","DOI":"10.1016\/j.ejso.2016.09.018","article-title":"Development of a nomogram for predicting the probability of postoperative delirium in patients undergoing free flap reconstruction for head and neck cancer","volume":"43","author":"Choi","year":"2017","journal-title":"Eur J Surg Oncol"},{"key":"ref32","doi-asserted-by":"publisher","first-page":"106063","DOI":"10.1016\/j.ijsu.2021.106063","article-title":"Risk factors for postoperative delirium: an umbrella review of systematic reviews","volume":"93","author":"Bramley","year":"2021","journal-title":"Int J Surg"},{"key":"ref33","doi-asserted-by":"publisher","first-page":"40","DOI":"10.1097\/CCM.0000000000000625","article-title":"A systematic review of risk factors for delirium in the ICU","volume":"43","author":"Zaal","year":"2015","journal-title":"Crit Care Med"},{"key":"ref34","doi-asserted-by":"publisher","first-page":"312","DOI":"10.1016\/j.jshs.2021.12.002","article-title":"Heart rate response and recovery during exercise predict future delirium risk-a prospective cohort study in middle- to older-aged adults","volume":"12","author":"Gao","year":"2023","journal-title":"J Sport Health Sci"},{"key":"ref35","doi-asserted-by":"publisher","first-page":"690","DOI":"10.1016\/j.ejim.2009.07.008","article-title":"Risk factors for delirium tremens in patients with alcohol withdrawal syndrome in a hospital setting","volume":"20","author":"Monte","year":"2009","journal-title":"Eur J Intern Med"},{"key":"ref36","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1016\/S2213-2600(20)30552-X","article-title":"Prevalence and risk factors for delirium in critically ill patients with COVID-19 (COVID-D): a multicentre cohort study","volume":"9","author":"Pun","year":"2021","journal-title":"Lancet Respir Med"},{"key":"ref37","doi-asserted-by":"publisher","first-page":"m441","DOI":"10.1136\/bmj.m441","article-title":"Calculating the sample size required for developing a clinical prediction model","volume":"368","author":"Riley","year":"2020","journal-title":"BMJ"},{"key":"ref38","doi-asserted-by":"publisher","first-page":"1373","DOI":"10.1016\/S0895-4356(96)00236-3","article-title":"A simulation study of the number of events per variable in logistic regression analysis","volume":"49","author":"Peduzzi","year":"1996","journal-title":"J Clin Epidemiol"},{"key":"ref39","doi-asserted-by":"publisher","first-page":"7175","DOI":"10.1007\/s00432-023-04661-x","article-title":"A clinically applicable prediction model for the risk of in-hospital mortality in solid cancer patients admitted to intensive care units with sepsis","volume":"149","author":"Ma","year":"2023","journal-title":"J Cancer Res Clin Oncol"},{"key":"ref40","doi-asserted-by":"publisher","first-page":"486","DOI":"10.3390\/metabo10120486","article-title":"A workflow for missing values imputation of untargeted metabolomics data","volume":"10","author":"Faquih","year":"2020","journal-title":"Metabolites"},{"key":"ref41","doi-asserted-by":"publisher","first-page":"1386","DOI":"10.1109\/TNNLS.2016.2542866","article-title":"Design and application of a variable selection method for multilayer perceptron neural network with LASSO","volume":"28","author":"Sun","year":"2017","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"ref42","doi-asserted-by":"publisher","first-page":"406","DOI":"10.1186\/s13054-023-04683-4","article-title":"Interpretable machine learning models for predicting venous thromboembolism in the intensive care unit: an analysis based on data from 207 centers","volume":"27","author":"Guan","year":"2023","journal-title":"Crit Care"},{"key":"ref43","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1186\/s40001-023-01593-7","article-title":"Interpretable machine learning model for early prediction of 28-day mortality in ICU patients with sepsis-induced coagulopathy: development and validation","volume":"29","author":"Zhou","year":"2024","journal-title":"Eur J Med Res"},{"key":"ref44","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1093\/postmj\/qgad144","article-title":"Explainable artificial intelligence model for mortality risk prediction in the intensive care unit: a derivation and validation study","volume":"100","author":"Hu","year":"2024","journal-title":"Postgrad Med J"},{"key":"ref45","doi-asserted-by":"publisher","first-page":"428","DOI":"10.1097\/CCM.0000000000003577","article-title":"Prediction of ICU delirium: validation of current delirium predictive models in routine clinical practice","volume":"47","author":"Green","year":"2019","journal-title":"Crit Care Med"},{"key":"ref46","doi-asserted-by":"publisher","first-page":"158","DOI":"10.1111\/cns.13991","article-title":"Comparison of logistic regression and machine learning methods for predicting postoperative delirium in elderly patients: a retrospective study","volume":"29","author":"Song","year":"2023","journal-title":"CNS Neurosci Ther"},{"key":"ref47","doi-asserted-by":"publisher","first-page":"114","DOI":"10.1186\/s13054-018-2037-6","article-title":"Delirium prediction in the intensive care unit: comparison of two delirium prediction models","volume":"22","author":"Wassenaar","year":"2018","journal-title":"Crit Care"},{"key":"ref48","doi-asserted-by":"publisher","first-page":"e000262","DOI":"10.1136\/fmch-2019-000262","article-title":"Variable selection strategies and its importance in clinical prediction modelling","volume":"8","author":"Chowdhury","year":"2020","journal-title":"Fam Med Community Health"},{"key":"ref49","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1002\/(SICI)1097-0258(19970228)16:4<385::AID-SIM380>3.0.CO;2-3","article-title":"The lasso method for variable selection in the Cox model","volume":"16","author":"Tibshirani","year":"1997","journal-title":"Stat Med"},{"key":"ref50","doi-asserted-by":"publisher","first-page":"1048","DOI":"10.1007\/s00134-015-3777-2","article-title":"Multinational development and validation of an early prediction model for delirium in ICU patients","volume":"41","author":"Wassenaar","year":"2015","journal-title":"Intensive Care Med"},{"key":"ref51","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1017\/S1041610221002556","article-title":"Delirium, depression, and long-term cognition","volume":"35","author":"Andrews","year":"2023","journal-title":"Int Psychogeriatr"},{"key":"ref52","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1016\/j.jad.2017.06.007","article-title":"The depression and marital status relationship is modified by both age and gender","volume":"223","author":"Bulloch","year":"2017","journal-title":"J Affect Disord"},{"key":"ref53","doi-asserted-by":"publisher","first-page":"12697","DOI":"10.1038\/s41598-023-38650-4","article-title":"Development of a machine learning-based prediction model for sepsis-associated delirium in the intensive care unit","volume":"13","author":"Zhang","year":"2023","journal-title":"Sci Rep"}],"container-title":["Frontiers in Medicine"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2024.1399848\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,17]],"date-time":"2024-05-17T00:34:12Z","timestamp":1715906052000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fmed.2024.1399848\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,17]]},"references-count":53,"alternative-id":["10.3389\/fmed.2024.1399848"],"URL":"https:\/\/doi.org\/10.3389\/fmed.2024.1399848","relation":{},"ISSN":["2296-858X"],"issn-type":[{"value":"2296-858X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,17]]},"article-number":"1399848"}}