{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,22]],"date-time":"2026-10-22T09:09:12Z","timestamp":1792660152291,"version":"4.5.0"},"reference-count":47,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T00:00:00Z","timestamp":1726617600000},"content-version":"vor","delay-in-days":79,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,7,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:sec>\n                  <jats:title>Objective<\/jats:title>\n                  <jats:p>Delirium is a syndrome that leads to severe complications in hospitalized patients, but is considered preventable in many cases. One of the biggest challenges is to identify patients at risk in a hectic clinical routine, as most screening tools cause additional workload. The aim of this study was to validate a machine learning (ML)-based delirium prediction tool on surgical in-patients undergoing a systematic assessment of delirium.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Materials and Methods<\/jats:title>\n                  <jats:p>738 in-patients of a vascular surgery, a trauma surgery and an orthopedic surgery department were screened for delirium using the DOS scale twice a day over their hospital stay. Concurrently, delirium risk was predicted by the ML algorithm in real-time for all patients at admission and evening of admission. The prediction was performed automatically based on existing EHR data and without any additional documentation needed.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Results<\/jats:title>\n                  <jats:p>103 patients (14.0%) were screened positive for delirium using the DOS scale. Out of them, 85 (82.5%) were correctly identified by the ML algorithm. Specificity was slightly lower, detecting 463 (72.9%) out of 635 patients without delirium. The AUROC of the algorithm was 0.883 (95% CI, 0.8523-0.9147).<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Discussion<\/jats:title>\n                  <jats:p>In this prospective validation study, the implemented machine-learning algorithm was able to detect patients with delirium in surgical departments with high discriminative performance.<\/jats:p>\n               <\/jats:sec>\n               <jats:sec>\n                  <jats:title>Conclusion<\/jats:title>\n                  <jats:p>In future, this tool or similar decision support systems may help to replace time-intensive screening tools and enable efficient prevention of delirium.<\/jats:p>\n               <\/jats:sec>","DOI":"10.1093\/jamiaopen\/ooae091","type":"journal-article","created":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T06:22:18Z","timestamp":1726640538000},"source":"Crossref","is-referenced-by-count":13,"title":["Machine learning-based delirium prediction in surgical in-patients: a prospective validation study"],"prefix":"10.1093","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1287-594X","authenticated-orcid":false,"given":"Stefanie","family":"Jauk","sequence":"first","affiliation":[{"name":"Division of Technology and IT, Steierm\u00e4rkische Krankenanstaltengesellschaft m.b.H. (KAGes) , 8010 Graz, Austria"},{"name":"PH Predicting Health GmbH , 8010 Graz, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Diether","family":"Kramer","sequence":"additional","affiliation":[{"name":"Division of Technology and IT, Steierm\u00e4rkische Krankenanstaltengesellschaft m.b.H. (KAGes) , 8010 Graz, Austria"},{"name":"PH Predicting Health GmbH , 8010 Graz, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Stefan","family":"Sumerauer","sequence":"additional","affiliation":[{"name":"Department of Neurology, Steierm\u00e4rkische Krankenanstaltengesellschaft m.b.H. (KAGes) , 8036 Graz, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sai Pavan Kumar","family":"Veeranki","sequence":"additional","affiliation":[{"name":"Division of Technology and IT, Steierm\u00e4rkische Krankenanstaltengesellschaft m.b.H. (KAGes) , 8010 Graz, Austria"},{"name":"PH Predicting Health GmbH , 8010 Graz, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Schrempf","sequence":"additional","affiliation":[{"name":"PH Predicting Health GmbH , 8010 Graz, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul","family":"Puchwein","sequence":"additional","affiliation":[{"name":"Department of Orthopaedics and Trauma, Medical University of Graz , 8036 Graz, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,9,17]]},"reference":[{"key":"2024091806220690900_ooae091-B1","first-page":"e2021569","article-title":"Delirium risk factors analysis post proximal femur fracture surgery in elderly","volume":"92","author":"Leigheb","year":"2022","journal-title":"Acta Biomed Atenei Parm"},{"key":"2024091806220690900_ooae091-B2","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1017\/S104161020600425X","article-title":"The incidence of delirium associated with orthopedic surgery: a meta-analytic review","volume":"19","author":"Bruce","year":"2007","journal-title":"Int Psychogeriatr"},{"key":"2024091806220690900_ooae091-B3","doi-asserted-by":"crossref","first-page":"312","DOI":"10.1093\/ageing\/afr014","article-title":"Mortality associated with delirium after hip-surgery: a 2-year follow-up study","volume":"40","author":"Kat","year":"2011","journal-title":"Age Ageing"},{"key":"2024091806220690900_ooae091-B4","doi-asserted-by":"crossref","first-page":"911","DOI":"10.1016\/S0140-6736(13)60688-1","article-title":"Delirium in elderly people","volume":"383","author":"Inouye","year":"2014","journal-title":"Lancet"},{"key":"2024091806220690900_ooae091-B5","doi-asserted-by":"crossref","first-page":"e420","DOI":"10.1136\/bmj.e420","article-title":"Development and validation of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction model for intensive care patients: observational multicentre study","volume":"344","author":"Boogaard","year":"2012","journal-title":"BMJ"},{"key":"2024091806220690900_ooae091-B6","doi-asserted-by":"crossref","first-page":"e12321","DOI":"10.1111\/opn.12321","article-title":"Family caregiver\u2019s experience of caring for an older adult with delirium: a systematic review","volume":"15","author":"Shrestha","year":"2020","journal-title":"Int J Older People Nurs"},{"key":"2024091806220690900_ooae091-B7","doi-asserted-by":"crossref","first-page":"1496","DOI":"10.1111\/imj.14287","article-title":"Differential clinical characteristics, management and outcome of delirium among ward compared with intensive care unit patients","volume":"49","author":"Canet","year":"2019","journal-title":"Intern Med J"},{"key":"2024091806220690900_ooae091-B8","doi-asserted-by":"crossref","first-page":"1151","DOI":"10.1016\/j.jtcvs.2021.05.031","article-title":"Diagnostic accuracy of the \u201c4 A\u2019s test\u201d delirium screening tool for the postoperative cardiac surgery ward","volume":"165","author":"Chang","year":"2023","journal-title":"J Thorac Cardiovasc Surg"},{"key":"2024091806220690900_ooae091-B9","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1007\/s00268-009-0376-9","article-title":"A comparison of three scores to screen for delirium on the surgical ward","volume":"34","author":"Radtke","year":"2010","journal-title":"World J Surg"},{"key":"2024091806220690900_ooae091-B10","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1186\/s12916-019-1458-7","article-title":"Delirium is prevalent in older hospital inpatients and associated with adverse outcomes: results of a prospective multi-Centre study on world delirium awareness day","volume":"17","author":"Collaborative GMR","year":"2019","journal-title":"BMC Med"},{"key":"2024091806220690900_ooae091-B11","doi-asserted-by":"crossref","first-page":"512","DOI":"10.1001\/jamainternmed.2014.7779","article-title":"Effectiveness of multicomponent nonpharmacological delirium interventions: a meta-analysis","volume":"175","author":"Hshieh","year":"2015","journal-title":"JAMA Intern Med"},{"key":"2024091806220690900_ooae091-B12","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1891\/rtnp.17.1.31.53169","article-title":"The Delirium Observation Screening scale: a screening instrument for delirium","volume":"17","author":"Schuurmans","year":"2003","journal-title":"Res Theory Nurs Pract"},{"key":"2024091806220690900_ooae091-B13","doi-asserted-by":"crossref","first-page":"941","DOI":"10.7326\/0003-4819-113-12-941","article-title":"Clarifying confusion: the confusion assessment method. A new method for detection of delirium","volume":"113","author":"Inouye","year":"1990","journal-title":"Ann Intern Med"},{"key":"2024091806220690900_ooae091-B14","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1136\/svn-2017-000101","article-title":"Artificial intelligence in healthcare: past, present and future","volume":"2","author":"Jiang","year":"2017","journal-title":"Stroke Vasc Neurol"},{"key":"2024091806220690900_ooae091-B15","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1093\/jamia\/ocw042","article-title":"Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review","volume":"24","author":"Goldstein","year":"2017","journal-title":"J Am Med Inform Assoc"},{"key":"2024091806220690900_ooae091-B16","doi-asserted-by":"crossref","first-page":"30","DOI":"10.1038\/s41591-018-0307-0","article-title":"The practical implementation of artificial intelligence technologies in medicine","volume":"25","author":"He","year":"2019","journal-title":"Nat Med"},{"key":"2024091806220690900_ooae091-B17","doi-asserted-by":"crossref","first-page":"l6927","DOI":"10.1136\/bmj.l6927","article-title":"Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness","volume":"368","author":"Vollmer","year":"2020","journal-title":"BMJ"},{"key":"2024091806220690900_ooae091-B18","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41591-018-0300-7","article-title":"High-performance medicine: the convergence of human and artificial intelligence","volume":"25","author":"Topol","year":"2019","journal-title":"Nat Med"},{"key":"2024091806220690900_ooae091-B19","doi-asserted-by":"crossref","first-page":"2383","DOI":"10.1111\/jgs.13138","article-title":"Risk prediction models for postoperative delirium: a systematic review and Meta-Analysis","volume":"62","author":"van Meenen","year":"2014","journal-title":"J Am Geriatr Soc"},{"key":"2024091806220690900_ooae091-B20","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1093\/bja\/aew476","article-title":"Risk prediction models for delirium in the intensive care unit after cardiac surgery: a systematic review and independent external validation","volume":"118","author":"Lee","year":"2017","journal-title":"Br J Anaesth"},{"key":"2024091806220690900_ooae091-B21","doi-asserted-by":"crossref","first-page":"e019223","DOI":"10.1136\/bmjopen-2017-019223","article-title":"Systematic review of prediction models for delirium in the older adult inpatient","volume":"8","author":"Lindroth","year":"2018","journal-title":"BMJ Open"},{"key":"2024091806220690900_ooae091-B22","doi-asserted-by":"crossref","first-page":"e181018","DOI":"10.1001\/jamanetworkopen.2018.1018","article-title":"Development and validation of an electronic health record-based machine learning model to estimate delirium risk in newly hospitalized patients without known cognitive impairment","volume":"1","author":"Wong","year":"2018","journal-title":"JAMA Netw Open"},{"key":"2024091806220690900_ooae091-B23","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1007\/s10916-018-1109-0","article-title":"Prediction of incident delirium using a random Forest classifier","volume":"42","author":"Corradi","year":"2018","journal-title":"J Med Syst"},{"key":"2024091806220690900_ooae091-B24","doi-asserted-by":"crossref","first-page":"e100767","DOI":"10.1136\/bmjhci-2023-100767","article-title":"Navigating the machine learning pipeline: a scoping review of inpatient delirium prediction models","volume":"30","author":"Strating","year":"2023","journal-title":"BMJ Health Care Inform"},{"key":"2024091806220690900_ooae091-B25","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1016\/j.jaclp.2022.05.006","article-title":"Validation and recalibration of modified Mayo delirium prediction tool in a hospitalized cohort","volume":"63","author":"Pagali","year":"2022","journal-title":"J Acad Consult Liaison Psychiatry"},{"key":"2024091806220690900_ooae091-B26","doi-asserted-by":"crossref","first-page":"e34295","DOI":"10.2196\/34295","article-title":"Machine learning-based prediction models for different clinical risks in different hospitals: evaluation of live performance","volume":"24","author":"Sun","year":"2022","journal-title":"J Med Internet Res"},{"key":"2024091806220690900_ooae091-B27","doi-asserted-by":"crossref","first-page":"e0010","DOI":"10.1097\/EA9.0000000000000010","article-title":"Leveraging artificial intelligence for the management of postoperative delirium following cardiac surgery","volume":"2","author":"Fliegenschmidt","year":"2023","journal-title":"Eur J Anaesthesiol Intensive Care"},{"key":"2024091806220690900_ooae091-B28","doi-asserted-by":"crossref","first-page":"1383","DOI":"10.1093\/jamia\/ocaa113","article-title":"Risk prediction of delirium in hospitalized patients using machine learning: an implementation and prospective evaluation study","volume":"27","author":"Jauk","year":"2020","journal-title":"J Am Med Inform Assoc JAMIA"},{"key":"2024091806220690900_ooae091-B29","doi-asserted-by":"crossref","first-page":"2537","DOI":"10.1111\/jocn.14833","article-title":"Undiagnosed delirium is frequent and difficult to predict: results from a prevalence survey of a tertiary hospital","volume":"28","author":"Lange","year":"2019","journal-title":"J Clin Nurs"},{"key":"2024091806220690900_ooae091-B30","doi-asserted-by":"crossref","first-page":"494","DOI":"10.1002\/jhm.2580","article-title":"Validation of the Delirium Observation Screening scale in a hospitalized older population","volume":"11","author":"Gavinski","year":"2016","journal-title":"J Hosp Med"},{"key":"2024091806220690900_ooae091-B31","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1891\/1061-3749.25.3.504","article-title":"Inter-rater reliability and user-friendliness of the delirium Observation Screening scale","volume":"25","author":"Mueller","year":"2017","journal-title":"J Nurs Meas"},{"key":"2024091806220690900_ooae091-B32","author":"Kuhn","year":"2017"},{"key":"2024091806220690900_ooae091-B33","doi-asserted-by":"crossref","first-page":"837","DOI":"10.2307\/2531595","article-title":"Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach","volume":"44","author":"DeLong","year":"1988","journal-title":"Biometrics"},{"key":"2024091806220690900_ooae091-B34","doi-asserted-by":"crossref","first-page":"230","DOI":"10.1186\/s12916-019-1466-7","article-title":"Calibration: the achilles heel of predictive analytics","volume":"17","author":"Van Calster","year":"2019","journal-title":"BMC Med"},{"key":"2024091806220690900_ooae091-B35","doi-asserted-by":"crossref","first-page":"938","DOI":"10.1016\/j.jclinepi.2009.11.009","article-title":"Use of brier score to assess binary predictions","volume":"63","author":"Rufibach","year":"2010","journal-title":"J Clin Epidemiol"},{"key":"2024091806220690900_ooae091-B36","author":"Heymans","year":"2021"},{"key":"2024091806220690900_ooae091-B37","doi-asserted-by":"crossref","first-page":"e0000022","DOI":"10.1371\/journal.pdig.0000022","article-title":"Sources of bias in artificial intelligence that perpetuate healthcare disparities\u2014a global review","volume":"1","author":"Celi","year":"2022","journal-title":"PLOS Digit Health"},{"key":"2024091806220690900_ooae091-B38","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1186\/s12873-018-0170-9","article-title":"Assessment of the feasibility of automated, real-time clinical decision support in the emergency department using electronic health record data","volume":"18","author":"Perry","year":"2018","journal-title":"BMC Emerg Med"},{"key":"2024091806220690900_ooae091-B39","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1111\/wvn.12649","article-title":"Incidence of postoperative delirium in older adults undergoing surgical procedures: a systematic literature review and meta-analysis","volume":"20","author":"Igwe","year":"2023","journal-title":"Worldviews Evid Based Nurs"},{"key":"2024091806220690900_ooae091-B40","doi-asserted-by":"crossref","first-page":"1327","DOI":"10.1046\/j.1532-5415.2001.49260.x","article-title":"Environmental risk factors for delirium in hospitalized older people","volume":"49","author":"McCusker","year":"2001","journal-title":"J Am Geriatr Soc"},{"key":"2024091806220690900_ooae091-B41","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1186\/s12912-020-00464-4","article-title":"Validation of two nurse-based screening tools for delirium in elderly patients in general medical wards","volume":"19","author":"Bergjan","year":"2020","journal-title":"BMC Nurs"},{"key":"2024091806220690900_ooae091-B42","first-page":"93","article-title":"External validation of a machine learning based delirium prediction software in clinical routine","volume":"293","author":"Jauk","year":"2022","journal-title":"Stud Health Technol Inform"},{"key":"2024091806220690900_ooae091-B43","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1038\/s41746-020-00323-1","article-title":"The future of digital health with federated learning","volume":"3","author":"Rieke","year":"2020","journal-title":"NPJ Digit Med"},{"key":"2024091806220690900_ooae091-B44","doi-asserted-by":"crossref","first-page":"2041","DOI":"10.1093\/jamia\/ocad170","article-title":"Federated and distributed learning applications for electronic health records and structured medical data: a scoping review","volume":"30","author":"Li","year":"2023","journal-title":"J Am Med Inform Assoc"},{"key":"2024091806220690900_ooae091-B45","first-page":"4768","article-title":"A unified approach to interpreting model predictions","volume":"30","author":"Lundberg","year":"2017","journal-title":"Adv Neural Inf Process Syst"},{"key":"2024091806220690900_ooae091-B46","first-page":"1135","author":"Ribeiro","year":"2016"},{"key":"2024091806220690900_ooae091-B47","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1176\/appi.psy.50.3.248","article-title":"Phenomenological subtypes of delirium in older persons: patterns, prevalence, and prognosis","volume":"50","author":"Yang","year":"2009","journal-title":"Psychosomatics"}],"container-title":["JAMIA Open"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/7\/3\/ooae091\/59172462\/ooae091.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article-pdf\/7\/3\/ooae091\/59172462\/ooae091.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T06:22:28Z","timestamp":1726640548000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jamiaopen\/article\/doi\/10.1093\/jamiaopen\/ooae091\/7759860"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,1]]},"references-count":47,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,7,1]]}},"URL":"https:\/\/doi.org\/10.1093\/jamiaopen\/ooae091","relation":{},"ISSN":["2574-2531"],"issn-type":[{"value":"2574-2531","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,10]]},"published":{"date-parts":[[2024,7,1]]},"article-number":"ooae091"}}