{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T22:04:48Z","timestamp":1789596288525,"version":"build-2803163510"},"reference-count":45,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Med Inform"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Delirium is a common and clinically important form of acute in-hospital mental status deterioration. Electronic health record (EHR)\u2013based prediction models may support early identification and targeted prevention, but their methodological quality, validation rigor, and clinical readiness remain uncertain.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This systematic review aimed to synthesize and critically evaluate prediction models for in-hospital delirium developed using routinely collected EHR data, focusing on model characteristics, validation strategies, performance, risk of bias, and clinical applicability.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n We searched PubMed, MEDLINE, Embase, PsycINFO, and Web of Science from inception to November 11, 2025. Eligible studies developed, validated, or evaluated multivariable prediction models using routinely collected EHR or administrative data to predict acute mental status deterioration during adult hospital admissions. Although eligibility criteria were broad, all included studies operationalized deterioration as delirium. Data extraction was informed by CHARMS (Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modeling Studies) and TRIPOD (Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis) or TRIPOD\u2013artificial intelligence guidance. Model performance, validation, calibration, and implementation features were synthesized narratively. Risk of bias and applicability were assessed using PROBAST (Prediction Model Risk of Bias Assessment Tool).<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n Twenty-nine studies met the inclusion criteria. The evidence clustered into 4 overlapping prediction tasks: admission or early-stay risk stratification, perioperative or postoperative prediction, dynamic intensive care unit prediction, and external validation or workflow evaluation of existing tools. Most studies were retrospective cohorts (20\/29, 69%) and were conducted in general ward, mixed ward\u2013intensive care unit, intensive care unit, or emergency department settings. Machine learning or hybrid approaches were common (18\/29, 62%), but more complex models did not consistently outperform statistical or rule-based approaches. Of 29 studies, internal discrimination was reported in 24 (83%; area under the receiver operating characteristic curve range 0.77-0.97) studies, whereas external discrimination was reported in 12 studies and calibration in 15 studies. Decision curve analysis was reported in 3 studies, and prospective evaluation or workflow integration remained limited. Overall risk of bias was low in 8 studies, unclear in 10 studies, and high in 11 studies, mainly because of analysis-domain limitations.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Routinely collected EHR data can support delirium risk prediction across hospital settings, and many models show moderate to high discrimination. However, no single algorithm is ready for routine adoption. The field remains limited by heterogeneous prediction tasks, inconsistent outcome ascertainment, weak calibration and decision-analytic reporting, and insufficient external or prospective evaluation. Future studies should define the intended clinical use case before model development, evaluate calibration and clinical usefulness alongside discrimination, and test models across institutions, time periods, and workflows before deployment.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/91618","type":"journal-article","created":{"date-parts":[[2026,8,14]],"date-time":"2026-08-14T13:00:05Z","timestamp":1786712405000},"page":"e91618-e91618","source":"Crossref","is-referenced-by-count":0,"title":["Prediction Models for In-Hospital Delirium Using Routinely Collected Electronic Health Record Data: Systematic Review"],"prefix":"10.2196","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-4386-5020","authenticated-orcid":false,"given":"Hung-Min","family":"Huang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-4010-6638","authenticated-orcid":false,"given":"Chun-Shun","family":"Lu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-2382-6030","authenticated-orcid":false,"given":"Geng-Wei","family":"Chang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-5565-4181","authenticated-orcid":false,"given":"Ming-Hsu","family":"Tien","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-6162-0187","authenticated-orcid":false,"given":"Yu-Kai","family":"Hsu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,9,16]]},"reference":[{"issue":"1","key":"R1","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1016\/j.ncl.2021.08.004","article-title":"Approach to altered mental status and inpatient delirium","volume":"40","author":"LaHue","journal-title":"Neurol Clin"},{"issue":"7","key":"R2","doi-asserted-by":"publisher","first-page":"101","DOI":"10.3238\/arztebl.2019.0101","article-title":"Acute confusional states in hospital","volume":"116","author":"Zoremba","journal-title":"Dtsch Arztebl Int"},{"key":"R3","unstructured":"Waszynski C . The confusion assessment method (CAM). Semantic Scholar. 2007. URL: https:\/\/www.semanticscholar.org\/paper\/The-Confusion-Assessment-Method-(CAM)-Waszynski\/ad2f4dd6f43a70ddc4be594207b061cce10f6892 [Accessed 13-01-2026]"},{"issue":"10","key":"R4","doi-asserted-by":"publisher","first-page":"2145","DOI":"10.1016\/j.mayocp.2019.08.001","article-title":"Predicting when a patient would be \u201cout of the furrow\u201d\u2014a perspective on delirium prediction","volume":"94","author":"Pagali","journal-title":"Mayo Clin Proc"},{"issue":"3","key":"R5","doi-asserted-by":"publisher","first-page":"524","DOI":"10.1093\/gerona\/glaa275","article-title":"Ascertainment of delirium status using natural language processing from electronic health records","volume":"77","author":"Fu","journal-title":"J Gerontol A Biol Sci Med Sci"},{"issue":"4","key":"R6","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2017-019223","article-title":"Systematic review of prediction models for delirium in the older adult inpatient","volume":"8","author":"Lindroth","journal-title":"BMJ Open"},{"issue":"12","key":"R7","doi-asserted-by":"publisher","DOI":"10.1097\/CCE.0000000000000296","article-title":"ICU delirium-prediction models: a systematic review","volume":"2","author":"Ruppert","journal-title":"Crit Care Explor"},{"issue":"10","key":"R8","doi-asserted-by":"publisher","first-page":"1655","DOI":"10.1016\/j.jamda.2022.06.020","article-title":"Machine learning-based prediction models for delirium: a systematic review and meta-analysis","volume":"23","author":"Xie","journal-title":"J Am Med Dir Assoc"},{"key":"R9","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.n71","article-title":"The PRISMA 2020 statement: an updated guideline for reporting systematic reviews","volume":"372","author":"Page","journal-title":"BMJ"},{"key":"R10","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.g7594","article-title":"Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement","volume":"350","author":"Collins","journal-title":"BMJ"},{"key":"R11","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.q902","article-title":"TRIPOD+AI statement: updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","journal-title":"BMJ"},{"issue":"10","key":"R12","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.1001744","article-title":"Critical appraisal and data extraction for systematic reviews of prediction modelling studies: the CHARMS checklist","volume":"11","author":"Moons","journal-title":"PLoS Med"},{"issue":"1","key":"R13","doi-asserted-by":"publisher","first-page":"51","DOI":"10.7326\/M18-1376","article-title":"PROBAST: a tool to assess the risk of bias and applicability of prediction model studies","volume":"170","author":"Wolff","journal-title":"Ann Intern Med"},{"issue":"5","key":"R14","doi-asserted-by":"publisher","first-page":"1128","DOI":"10.1007\/s11096-023-01641-6","article-title":"External validity of an automated delirium prediction model (DEMO) and comparison to the manual VMS-questions: a retrospective cohort study","volume":"45","author":"Ali","journal-title":"Int J Clin Pharm"},{"issue":"11","key":"R15","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2025.40920","article-title":"Validation of 4 risk stratification tools for delirium in the emergency department","volume":"8","author":"Bartolacci","journal-title":"JAMA Netw Open"},{"issue":"1","key":"R16","doi-asserted-by":"publisher","DOI":"10.1186\/s12871-021-01543-y","article-title":"Postoperative delirium prediction using machine learning models and preoperative electronic health record data","volume":"22","author":"Bishara","journal-title":"BMC Anesthesiol"},{"issue":"3","key":"R17","doi-asserted-by":"publisher","first-page":"298","DOI":"10.1016\/j.jaclp.2020.12.005","article-title":"Development and external validation of a delirium prediction model for hospitalized patients with coronavirus disease 2019","volume":"62","author":"Castro","journal-title":"J Acad Consult Liaison Psychiatry"},{"issue":"12","key":"R18","doi-asserted-by":"publisher","first-page":"1931","DOI":"10.1016\/j.jamda.2023.07.003","article-title":"Assessing the risk of developing delirium on admission to inpatient rehabilitation: a clinical prediction model","volume":"24","author":"Ceppi","journal-title":"J Am Med Dir Assoc"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-025-22634-7","article-title":"A large language model for delirium prediction in the intensive care unit using structured electronic health records","volume":"15","author":"Contreras","journal-title":"Sci Rep"},{"key":"R20","doi-asserted-by":"publisher","DOI":"10.1109\/bhi58575.2023.10313445","article-title":"Dynamic delirium prediction in the intensive care unit using machine learning on electronic health records","volume":"2023","author":"Contreras","journal-title":"IEEE EMBS Int Conf Biomed Health Inform"},{"issue":"12","key":"R21","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-018-1109-0","article-title":"Prediction of incident delirium using a Random Forest classifier","volume":"42","author":"Corradi","journal-title":"J Med Syst"},{"key":"R22","doi-asserted-by":"crossref","unstructured":"Davoudi A Ozrazgat-Baslanti T Ebadi A Bursian AC Bihorac A Rashidi P . Delirium prediction using machine learning models on predictive electronic health records data. Presented at: 2017 IEEE 17th International Conference on Bioinformatics and Bioengineering (BIBE); Oct 23-25, 2017:568-573; Washington, DC, USA. 2017.[doi: 10.1109\/BIBE.2017.00014]","DOI":"10.1109\/BIBE.2017.00014"},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.2147\/NDT.S479756","article-title":"Using machine learning and electronic health records to identify neuropsychiatric risk scores for delirium in ICU and general hospital settings","volume":"20","author":"Heikal","journal-title":"Neuropsychiatr Dis Treat"},{"key":"R24","doi-asserted-by":"publisher","DOI":"10.2196\/59422","article-title":"Development and Validation of a routine electronic health record-based delirium prediction model for surgical patients without dementia: retrospective case-control study","volume":"8","author":"Holler","journal-title":"JMIR Perioper Med"},{"issue":"7","key":"R25","doi-asserted-by":"publisher","DOI":"10.2196\/23401","article-title":"A machine learning-based algorithm for the Prediction of Intensive Care Unit Delirium (PRIDE): retrospective study","volume":"9","author":"Hur","journal-title":"JMIR Med Inform"},{"issue":"9","key":"R26","doi-asserted-by":"publisher","first-page":"1383","DOI":"10.1093\/jamia\/ocaa113","article-title":"Risk prediction of delirium in hospitalized patients using machine learning: an implementation and prospective evaluation study","volume":"27","author":"Jauk","journal-title":"J Am Med Inform Assoc"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.3233\/SHTI220353","article-title":"External validation of a machine learning based delirium prediction software in clinical routine","volume":"293","author":"Jauk","journal-title":"Stud Health Technol Inform"},{"issue":"3","key":"R28","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooae091","article-title":"Machine learning-based delirium prediction in surgical in-patients: a prospective validation study","volume":"7","author":"Jauk","journal-title":"JAMIA Open"},{"issue":"1","key":"R29","doi-asserted-by":"publisher","DOI":"10.1186\/s12888-022-04067-y","article-title":"A machine-learning model to predict postoperative delirium following knee arthroplasty using electronic health records","volume":"22","author":"Jung","journal-title":"BMC Psychiatry"},{"issue":"1","key":"R30","doi-asserted-by":"publisher","DOI":"10.1186\/s12872-024-03723-3","article-title":"A machine learning-based prediction model for postoperative delirium in cardiac valve surgery using electronic health records","volume":"24","author":"Li","journal-title":"BMC Cardiovasc Disord"},{"issue":"1","key":"R31","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1093\/jamia\/ocac210","article-title":"New onset delirium prediction using machine learning and long short-term memory (LSTM) in electronic health record","volume":"30","author":"Liu","journal-title":"J Am Med Inform Assoc"},{"issue":"4","key":"R32","doi-asserted-by":"publisher","first-page":"492","DOI":"10.1097\/CCM.0000000000005789","article-title":"Deep learning-based recurrent delirium prediction in critically ill patients","volume":"51","author":"Lucini","journal-title":"Crit CARE Med"},{"key":"R33","doi-asserted-by":"publisher","DOI":"10.2196\/50895","article-title":"Temporal generalizability of machine learning models for predicting postoperative delirium using electronic health record data: model development and validation study","volume":"6","author":"Matsumoto","journal-title":"JMIR Perioper Med"},{"key":"R34","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijnurstu.2017.09.014","article-title":"Development and validation of an automated delirium risk assessment system (Auto-DelRAS) implemented in the electronic health record system","volume":"77","author":"Moon","journal-title":"Int J Nurs Stud"},{"issue":"5","key":"R35","doi-asserted-by":"publisher","first-page":"493","DOI":"10.1111\/acps.13551","article-title":"Evaluating the performance of machine learning methods for risk estimation of delirium in patients hospitalized from the emergency department","volume":"147","author":"Mueller","journal-title":"Acta Psychiatr Scand"},{"issue":"6","key":"R36","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1016\/j.jaclp.2022.05.006","article-title":"Validation and recalibration of modified Mayo delirium prediction tool in a hospitalized cohort","volume":"63","author":"Pagali","journal-title":"J Acad Consult Liaison Psychiatry"},{"issue":"1","key":"R37","doi-asserted-by":"publisher","DOI":"10.1136\/bmjhci-2024-101291","article-title":"Prospective external validation of the automated PIPRA multivariable prediction model for postoperative delirium on real-world data from a consecutive cohort of non-cardiac surgery inpatients","volume":"32","author":"Reeve","journal-title":"BMJ Health Care Inform"},{"issue":"3","key":"R38","doi-asserted-by":"publisher","first-page":"244","DOI":"10.1016\/j.jamda.2015.10.020","article-title":"Validation of a delirium risk assessment using electronic medical record information","volume":"17","author":"Rudolph","journal-title":"J Am Med Dir Assoc"},{"key":"R39","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2023.102659","article-title":"An interpretable deep learning model for time-series electronic health records: case study of delirium prediction in critical care","volume":"144","author":"Sheikhalishahi","journal-title":"Artif Intell Med"},{"key":"R40","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2021.103783","article-title":"A scalable approach for developing clinical risk prediction applications in different hospitals","volume":"118","author":"Sun","journal-title":"J Biomed Inform"},{"issue":"6","key":"R41","doi-asserted-by":"publisher","DOI":"10.2196\/34295","article-title":"Machine learning-based prediction models for different clinical risks in different hospitals: evaluation of live performance","volume":"24","author":"Sun","journal-title":"J Med Internet Res"},{"issue":"4","key":"R42","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2018.1018","article-title":"Development and validation of an electronic health record-based machine learning model to estimate delirium risk in newly hospitalized patients without known cognitive impairment","volume":"1","author":"Wong","journal-title":"JAMA Netw Open"},{"issue":"1","key":"R43","doi-asserted-by":"publisher","DOI":"10.1186\/s13054-019-2564-9","article-title":"Use of machine learning to analyse routinely collected intensive care unit data: a systematic review","volume":"23","author":"Shillan","journal-title":"Crit Care"},{"key":"R44","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.n2281","article-title":"Risk of bias in studies on prediction models developed using supervised machine learning techniques: systematic review","volume":"375","author":"Andaur Navarro","journal-title":"BMJ"},{"key":"R45","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2021.06.017","article-title":"Large-scale validation of the prediction model risk of bias assessment Tool (PROBAST) using a short form: high risk of bias models show poorer discrimination","volume":"138","author":"Venema","journal-title":"J Clin Epidemiol"}],"container-title":["JMIR Medical Informatics"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,9,16]],"date-time":"2026-09-16T21:43:09Z","timestamp":1789594989000},"score":1,"resource":{"primary":{"URL":"https:\/\/medinform.jmir.org\/2026\/1\/e91618"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,9,16]]},"references-count":45,"URL":"https:\/\/doi.org\/10.2196\/91618","relation":{},"ISSN":["2291-9694"],"issn-type":[{"value":"2291-9694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,16]]},"article-number":"v14i1e91618"}}