{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,24]],"date-time":"2026-01-24T17:42:28Z","timestamp":1769276548017,"version":"3.49.0"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2019,9,9]],"date-time":"2019-09-09T00:00:00Z","timestamp":1567987200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,9,9]],"date-time":"2019-09-09T00:00:00Z","timestamp":1567987200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000025","name":"U.S. Department of Health & Human Services | NIH | National Institute of Mental Health","doi-asserted-by":"publisher","award":["1K08MH11613501"],"award-info":[{"award-number":["1K08MH11613501"]}],"id":[{"id":"10.13039\/100000025","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Over- and under-sedation are common in the ICU, and contribute to poor ICU outcomes including delirium. Behavioral assessments, such as Richmond Agitation-Sedation Scale (RASS) for monitoring levels of sedation and Confusion Assessment Method for the ICU (CAM-ICU) for detecting signs of delirium, are often used. As an alternative, brain monitoring with electroencephalography (EEG) has been proposed in the operating room, but is challenging to implement in ICU due to the differences between critical illness and elective surgery, as well as the duration of sedation. Here we present a deep learning model based on a combination of convolutional and recurrent neural networks that automatically tracks both the level of consciousness and delirium using frontal EEG signals in the ICU. For level of consciousness, the system achieves a median accuracy of 70% when allowing prediction to be within one RASS level difference across all patients, which is comparable or higher than the median technician\u2013nurse agreement at 59%. For delirium, the system achieves an AUC of 0.80 with 69% sensitivity and 83% specificity at the optimal operating point. The results show it is feasible to continuously track level of consciousness and delirium in the ICU.<\/jats:p>","DOI":"10.1038\/s41746-019-0167-0","type":"journal-article","created":{"date-parts":[[2019,9,9]],"date-time":"2019-09-09T10:03:08Z","timestamp":1568023388000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":36,"title":["Automated tracking of level of consciousness and delirium in critical illness using deep learning"],"prefix":"10.1038","volume":"2","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5041-8312","authenticated-orcid":false,"given":"Haoqi","family":"Sun","sequence":"first","affiliation":[]},{"given":"Eyal","family":"Kimchi","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6740-1250","authenticated-orcid":false,"given":"Oluwaseun","family":"Akeju","sequence":"additional","affiliation":[]},{"given":"Sunil B.","family":"Nagaraj","sequence":"additional","affiliation":[]},{"given":"Lauren M.","family":"McClain","sequence":"additional","affiliation":[]},{"given":"David W.","family":"Zhou","sequence":"additional","affiliation":[]},{"given":"Emily","family":"Boyle","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9474-6369","authenticated-orcid":false,"given":"Wei-Long","family":"Zheng","sequence":"additional","affiliation":[]},{"given":"Wendong","family":"Ge","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4803-312X","authenticated-orcid":false,"given":"M. Brandon","family":"Westover","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,9,9]]},"reference":[{"key":"167_CR1","doi-asserted-by":"publisher","first-page":"444","DOI":"10.1056\/NEJMra1208705","volume":"370","author":"MC Reade","year":"2014","unstructured":"Reade, M. C. & Finfer, S. Sedation and delirium in the intensive care unit. N. Engl. J. Med. 370, 444\u2013454 (2014).","journal-title":"N. Engl. J. Med."},{"key":"167_CR2","doi-asserted-by":"crossref","unstructured":"Kaplan, L. & Bailey, H. Bispectral index (BIS) monitoring of ICU patients on continuous infusion of sedatives and paralytics reduces sedative drug utilization and cost. Crit. Care 4, P190 (BioMed Central 2000).","DOI":"10.1186\/cc910"},{"key":"167_CR3","doi-asserted-by":"publisher","DOI":"10.1186\/cc13929","volume":"18","author":"AE Wolters","year":"2014","unstructured":"Wolters, A. E. et al. Long-term outcome of delirium during intensive care unit stay in survivors of critical illness: a prospective cohort study. Crit. Care 18, R125 (2014).","journal-title":"Crit. Care"},{"key":"167_CR4","first-page":"53","volume":"4","author":"CG Hughes","year":"2012","unstructured":"Hughes, C. G., McGrane, S. & Pandharipande, P. P. Sedation in the intensive care setting. Clin. Pharmacol. Adv. Appl. 4, 53 (2012).","journal-title":"Clin. Pharmacol. Adv. Appl."},{"key":"167_CR5","doi-asserted-by":"publisher","first-page":"656","DOI":"10.1136\/bmj.2.5920.656","volume":"2","author":"M Ramsay","year":"1974","unstructured":"Ramsay, M., Savege, T., Simpson, B. & Goodwin, R. Controlled sedation with alphaxalone-alphadolone. Br. Med. J. 2, 656 (1974).","journal-title":"Br. Med. J."},{"key":"167_CR6","doi-asserted-by":"publisher","first-page":"1325","DOI":"10.1097\/00003246-199907000-00022","volume":"27","author":"RR Riker","year":"1999","unstructured":"Riker, R. R., Picard, J. T. & Fraser, G. L. Prospective evaluation of the Sedation-Agitation Scale for adult critically ill patients. Crit. Care Med. 27, 1325\u20131329 (1999).","journal-title":"Crit. Care Med."},{"key":"167_CR7","doi-asserted-by":"publisher","first-page":"1338","DOI":"10.1164\/rccm.2107138","volume":"166","author":"CN Sessler","year":"2002","unstructured":"Sessler, C. N. et al. The Richmond Agitation\u2013Sedation Scale: validity and reliability in adult intensive care unit patients. Am. J. Respir. Crit. Care Med. 166, 1338\u20131344 (2002).","journal-title":"Am. J. Respir. Crit. Care Med."},{"key":"167_CR8","doi-asserted-by":"publisher","first-page":"1370","DOI":"10.1097\/00003246-200107000-00012","volume":"29","author":"EW Ely","year":"2001","unstructured":"Ely, E. W. et al. Evaluation of delirium in critically ill patients: validation of the Confusion Assessment Method for the Intensive Care Unit (CAM-ICU). Crit. Care Med. 29, 1370\u20131379 (2001).","journal-title":"Crit. Care Med."},{"key":"167_CR9","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1007\/s001340100909","volume":"27","author":"N Bergeron","year":"2001","unstructured":"Bergeron, N., Dubois, M.-J., Dumont, M., Dial, S. & Skrobik, Y. Intensive Care Delirium Screening Checklist: evaluation of a new screening tool. Intensive Care Med. 27, 859\u2013864 (2001).","journal-title":"Intensive Care Med."},{"key":"167_CR10","doi-asserted-by":"publisher","first-page":"2983","DOI":"10.1001\/jama.289.22.2983","volume":"289","author":"EW Ely","year":"2003","unstructured":"Ely, E. W. et al. Monitoring sedation status over time in ICU patients: reliability and validity of the Richmond Agitation-Sedation Scale (RASS). JAMA 289, 2983\u20132991 (2003).","journal-title":"JAMA"},{"key":"167_CR11","doi-asserted-by":"publisher","first-page":"367","DOI":"10.1513\/AnnalsATS.201306-210OC","volume":"11","author":"CR Dale","year":"2014","unstructured":"Dale, C. R. et al. Improved analgesia, sedation, and delirium protocol associated with decreased duration of delirium and mechanical ventilation. Ann. Am. Thorac. Soc. 11, 367\u2013374 (2014).","journal-title":"Ann. Am. Thorac. Soc."},{"key":"167_CR12","doi-asserted-by":"publisher","DOI":"10.1186\/cc7029","volume":"12","author":"PV Sackey","year":"2008","unstructured":"Sackey, P. V. Frontal EEG for intensive care unit sedation: treating numbers or patients? Crit. Care 12, 186 (2008).","journal-title":"Crit. Care"},{"key":"167_CR13","doi-asserted-by":"publisher","first-page":"506","DOI":"10.1213\/ane.0b013e3181a9d8b5","volume":"109","author":"D Friedman","year":"2009","unstructured":"Friedman, D., Claassen, J. & Hirsch, L. J. Continuous electroencephalogram monitoring in the intensive care unit. Anesth. Analg. 109, 506\u2013523 (2009).","journal-title":"Anesth. Analg."},{"key":"167_CR14","doi-asserted-by":"publisher","DOI":"10.1186\/s40560-018-0310-z","volume":"6","author":"Y Kubota","year":"2018","unstructured":"Kubota, Y., Nakamoto, H., Egawa, S. & Kawamata, T. Continuous EEG monitoring in ICU. J. Intensive Care 6, 39 (2018).","journal-title":"J. Intensive Care"},{"key":"167_CR15","first-page":"96","volume":"32","author":"ST Herman","year":"2015","unstructured":"Herman, S. T. et al. Consensus statement on continuous EEG in critically ill adults and children, Part II: personnel, technical specifications and clinical practice. J. Clin. Neurophysiol. Publ. Am. Electroencephalogr. Soc. 32, 96 (2015).","journal-title":"J. Clin. Neurophysiol. Publ. Am. Electroencephalogr. Soc."},{"key":"167_CR16","doi-asserted-by":"crossref","first-page":"288","DOI":"10.23736\/S0375-9393.16.10886-7","volume":"83","author":"B Bilgili","year":"2017","unstructured":"Bilgili, B. et al. Utilizing bi-spectral index (BIS) for the monitoring of sedated adult ICU patients: a systematic review. Minerva Anestesiol. 83, 288\u2013301 (2017).","journal-title":"Minerva Anestesiol."},{"key":"167_CR17","doi-asserted-by":"publisher","first-page":"719","DOI":"10.1038\/s41551-018-0305-z","volume":"2","author":"K-H Yu","year":"2018","unstructured":"Yu, K.-H., Beam, A. L. & Kohane, I. S. Artificial intelligence in healthcare. Nat. Biomed. Eng. 2, 719 (2018).","journal-title":"Nat. Biomed. Eng."},{"key":"167_CR18","doi-asserted-by":"publisher","first-page":"2676","DOI":"10.1002\/sim.4509","volume":"31","author":"X Liu","year":"2012","unstructured":"Liu, X. Classification accuracy and cut point selection. Stat. Med. 31, 2676\u20132686 (2012).","journal-title":"Stat. Med."},{"key":"167_CR19","doi-asserted-by":"publisher","first-page":"3179","DOI":"10.1093\/brain\/awy251","volume":"141","author":"DA Engemann","year":"2018","unstructured":"Engemann, D. A. et al. Robust EEG-based cross-site and cross-protocol classification of states of consciousness. Brain 141, 3179\u20133192 (2018).","journal-title":"Brain"},{"key":"167_CR20","doi-asserted-by":"publisher","first-page":"2684","DOI":"10.1109\/TBME.2018.2813265","volume":"65","author":"SB Nagaraj","year":"2018","unstructured":"Nagaraj, S. B. et al. Electroencephalogram based detection of deep sedation in ICU patients using atomic decomposition. IEEE Trans. Biomed. Eng. 65, 2684\u20132691 (2018).","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"167_CR21","doi-asserted-by":"publisher","first-page":"2137","DOI":"10.1016\/j.clinph.2014.02.010","volume":"125","author":"AW van der Kooi","year":"2014","unstructured":"van der Kooi, A. W., Slooter, A., van Het, K. M. & Leijten, F. EEG in delirium: increased spectral variability and decreased complexity. Clin. Neurophysiol. 125, 2137 (2014).","journal-title":"Clin. Neurophysiol."},{"key":"167_CR22","first-page":"328","volume":"121","author":"E van Dellen","year":"2014","unstructured":"van Dellen, E. et al. Decreased functional connectivity and disturbed directionality of information flow in the electroencephalography of intensive care unit patients with delirium after cardiac surgery. Anesthesiol. J. Am. Soc. Anesthesiol. 121, 328\u2013335 (2014).","journal-title":"Anesthesiol. J. Am. Soc. Anesthesiol."},{"key":"167_CR23","doi-asserted-by":"publisher","first-page":"472","DOI":"10.1176\/appi.neuropsych.11110347","volume":"24","author":"AW van der Kooi","year":"2012","unstructured":"van der Kooi, A. W., Leijten, F. S., van der Wekken, R. J. & Slooter, A. J. What are the opportunities for EEG-based monitoring of delirium in the ICU? J. Neuropsychiatry Clin. Neurosci. 24, 472\u2013477 (2012).","journal-title":"J. Neuropsychiatry Clin. Neurosci."},{"key":"167_CR24","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.e420","volume":"344","author":"M Van den Boogaard","year":"2012","unstructured":"Van den Boogaard, M. et al. Development and validation of PRE-DELIRIC (PREdiction of DELIRium in ICu patients) delirium prediction model for intensive care patients: observational multicentre study. BMJ 344, e420 (2012).","journal-title":"BMJ"},{"key":"167_CR25","doi-asserted-by":"publisher","first-page":"1048","DOI":"10.1007\/s00134-015-3777-2","volume":"41","author":"A Wassenaar","year":"2015","unstructured":"Wassenaar, A. et al. Multinational development and validation of an early prediction model for delirium in ICU patients. Intensive Care Med. 41, 1048\u20131056 (2015).","journal-title":"Intensive Care Med."},{"key":"167_CR26","doi-asserted-by":"publisher","first-page":"94","DOI":"10.1378\/chest.13-3050","volume":"147","author":"AW Van Der Kooi","year":"2015","unstructured":"Van Der Kooi, A. W. et al. Delirium detection using EEG. Chest 147, 94\u2013101 (2015).","journal-title":"Chest"},{"key":"167_CR27","doi-asserted-by":"publisher","first-page":"394","DOI":"10.1093\/bja\/aep198","volume":"103","author":"R Zanner","year":"2009","unstructured":"Zanner, R., Pilge, S., Kochs, E., Kreuzer, M. & Schneider, G. Time delay of electroencephalogram index calculation: analysis of cerebral state, bispectral, and narcotrend indices using perioperatively recorded electroencephalographic signals. Br. J. Anaesth. 103, 394\u2013399 (2009).","journal-title":"Br. J. Anaesth."},{"key":"167_CR28","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1038\/s41591-018-0268-3","volume":"25","author":"AY Hannun","year":"2019","unstructured":"Hannun, A. Y. et al. Cardiologist-level arrhythmia detection and classification in ambulatory electrocardiograms using a deep neural network. Nat. Med. 25, 65\u201369 (2019).","journal-title":"Nat. Med."},{"key":"167_CR29","first-page":"937","volume":"123","author":"PL Purdon","year":"2015","unstructured":"Purdon, P. L., Sampson, A., Pavone, K. J. & Brown, E. N. Clinical electroencephalography for anesthesiologistspart I: background and basic signatures. Anesthesiol. J. Amer. Soc. Anesthesiol. 123, 937\u2013960 (2015).","journal-title":"Anesthesiol. J. Amer. Soc. Anesthesiol."}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0167-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0167-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0167-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,17]],"date-time":"2022-12-17T18:32:57Z","timestamp":1671301977000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-019-0167-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,9]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2019,12]]}},"alternative-id":["167"],"URL":"https:\/\/doi.org\/10.1038\/s41746-019-0167-0","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,9,9]]},"assertion":[{"value":"22 April 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 August 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 September 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"89"}}