{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T15:35:33Z","timestamp":1787240133557,"version":"build-2736575974"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,8,20]],"date-time":"2025-08-20T00:00:00Z","timestamp":1755648000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,20]],"date-time":"2025-08-20T00:00:00Z","timestamp":1755648000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100007259","name":"SUNY | Stony Brook University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007259","id-type":"DOI","asserted-by":"publisher"}]},{"name":"This work was supported by OVPR SEED Funding, SUNY SEED, Targeted Research Opportunity Program (Stony Brook TRO, Fusion)."}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Commun Med"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Many brain injury patients who appear unresponsive retain subtle, purposeful motor behaviors, signaling capacity for recovery. We hypothesized that low-amplitude movements precede larger-amplitude voluntary movements detectable by clinicians after acute brain injury. To test this hypothesis, we developed a novel, as far as we are aware, computer vision-based tool (SeeMe) that detects and quantifies low-amplitude facial movements in response to auditory commands.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We enrolled 16 healthy volunteers and 37 comatose acute brain injury patients (Glasgow Coma Scale \u22648) aged 18\u201385 with no prior neurological diagnoses. We measured facial movements to command assessed using SeeMe and compared them to clinicians\u2019 exams. The primary outcome was the detection of facial movement in response to auditory commands. To assess comprehension, we tested whether movements were specific to command type (i.e., eye-opening to open your eyes and not stick out your tongue) with a machine learning-based classifier.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Here we show that SeeMe detects eye-opening in comatose patients 4.1 days earlier than clinicians. SeeMe also detects eye-opening in more comatose patients (30\/36, 85.7%) than clinical examination (25\/36, 71.4%). In patients without an obscuring endotracheal tube, SeeMe detects mouth movements in 16\/17 (94.1%) patients. The amplitude and number of SeeMe-detected responses correlate with clinical outcome at discharge. Using our classifier, eye-opening is specific (81%) to the command open your eyes.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion<\/jats:title>\n                    <jats:p>Acute brain injury patients have low-amplitude movements before overt movements. Thus, many covertly conscious patients may have motor behavior currently undetected by clinicians.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1038\/s43856-025-01042-y","type":"journal-article","created":{"date-parts":[[2025,8,20]],"date-time":"2025-08-20T11:16:30Z","timestamp":1755688590000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Computer vision detects covert voluntary facial movements in unresponsive brain injury patients"],"prefix":"10.1038","volume":"5","author":[{"given":"Xi","family":"Cheng","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sujith","family":"Swarna","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jermaine","family":"Robertson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nathaniel A.","family":"Cleri","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jordan R.","family":"Saadon","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chiemeka","family":"Uwakwe","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-0938-7071","authenticated-orcid":false,"given":"Yindong","family":"Hua","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7372-0143","authenticated-orcid":false,"given":"Seyed Morsal","family":"Mosallami Aghili","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cassie","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert S.","family":"Kleyner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xuwen","family":"Zheng","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ariana","family":"Forohar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John","family":"Servider","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kurt","family":"Butler","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chao","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jordane","family":"Dimidschstein","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Petar M.","family":"Djuri\u0107","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0701-2325","authenticated-orcid":false,"given":"Charles B.","family":"Mikell","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4509-6080","authenticated-orcid":false,"given":"Sima","family":"Mofakham","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,8,20]]},"reference":[{"key":"1042_CR1","doi-asserted-by":"publisher","first-page":"2497","DOI":"10.1056\/NEJMoa1812757","volume":"380","author":"J Claassen","year":"2019","unstructured":"Claassen, J. et al. Detection of brain activation in unresponsive patients with acute brain injury. N. Engl. J. Med. 380, 2497\u20132505 (2019).","journal-title":"N. Engl. J. Med."},{"key":"1042_CR2","doi-asserted-by":"publisher","first-page":"598","DOI":"10.1056\/NEJMoa2400645","volume":"391","author":"YG Bodien","year":"2024","unstructured":"Bodien, Y. G. et al. Cognitive motor dissociation in disorders of consciousness. N. Engl. J. Med. 391, 598\u2013608 (2024).","journal-title":"N. Engl. J. Med."},{"key":"1042_CR3","doi-asserted-by":"publisher","first-page":"646","DOI":"10.1002\/ana.25995","volume":"89","author":"R Sokoliuk","year":"2021","unstructured":"Sokoliuk, R. et al. Covert speech comprehension predicts recovery from acute unresponsive states. Ann. Neurol. 89, 646\u2013656 (2021).","journal-title":"Ann. Neurol."},{"key":"1042_CR4","doi-asserted-by":"publisher","first-page":"fcad094","DOI":"10.1093\/braincomms\/fcad094","volume":"5","author":"P Jain","year":"2023","unstructured":"Jain, P., Conte, M. M., Voss, H. U., Victor, J. D. & Schiff, N. D. Low-level language processing in brain-injured patients. Brain Commun. 5, fcad094 (2023).","journal-title":"Brain Commun."},{"key":"1042_CR5","doi-asserted-by":"publisher","first-page":"579","DOI":"10.1056\/NEJMoa0905370","volume":"362","author":"MM Monti","year":"2010","unstructured":"Monti, M. M. et al. Willful modulation of brain activity in disorders of consciousness. N. Engl. J. Med. 362, 579\u2013589 (2010).","journal-title":"N. Engl. J. Med."},{"key":"1042_CR6","doi-asserted-by":"publisher","first-page":"2399","DOI":"10.1093\/brain\/awx176","volume":"140","author":"BL Edlow","year":"2017","unstructured":"Edlow, B. L. et al. Early detection of consciousness in patients with acute severe traumatic brain injury. Brain 140, 2399\u20132414 (2017).","journal-title":"Brain"},{"key":"1042_CR7","doi-asserted-by":"publisher","first-page":"1144","DOI":"10.1212\/WNL.0000000000207067","volume":"100","author":"K Diserens","year":"2023","unstructured":"Diserens, K. et al. A focus on subtle signs and motor behavior to unveil awareness in unresponsive brain-impaired patients: the importance of being clinical. Neurology 100, 1144\u20131150 (2023).","journal-title":"Neurology"},{"key":"1042_CR8","doi-asserted-by":"publisher","first-page":"e0156882","DOI":"10.1371\/journal.pone.0156882","volume":"11","author":"J-M Pignat","year":"2016","unstructured":"Pignat, J.-M. et al. Outcome prediction of consciousness disorders in the acute stage based on a complementary motor behavioural tool. PLoS ONE 11, e0156882 (2016).","journal-title":"PLoS ONE"},{"key":"1042_CR9","doi-asserted-by":"publisher","first-page":"443","DOI":"10.1002\/ana.25417","volume":"85","author":"A Pincherle","year":"2019","unstructured":"Pincherle, A. et al. Motor behavior unmasks residual cognition in disorders of consciousness. Ann. Neurol. 85, 443\u2013447 (2019).","journal-title":"Ann. Neurol."},{"key":"1042_CR10","doi-asserted-by":"publisher","first-page":"454","DOI":"10.1080\/09602010443000425","volume":"15","author":"K Kalmar","year":"2005","unstructured":"Kalmar, K. & Giacino, J. T. The JFK coma recovery scale\u2014revised. Neuropsychol. Rehabil. 15, 454\u2013460 (2005).","journal-title":"Neuropsychol. Rehabil."},{"key":"1042_CR11","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1113\/jphysiol.2005.088542","volume":"567","author":"GO Paradiso","year":"2005","unstructured":"Paradiso, G. O., Cunic, D. I., Gunraj, C. A. & Chen, R. Representation of facial muscles in human motor cortex. J. Physiol. 567, 323\u2013336 (2005).","journal-title":"J. Physiol."},{"key":"1042_CR12","doi-asserted-by":"publisher","first-page":"135","DOI":"10.1016\/j.neubiorev.2013.11.002","volume":"38","author":"L Cattaneo","year":"2014","unstructured":"Cattaneo, L. & Pavesi, G. The facial motor system. Neurosci. Biobehav. Rev. 38, 135\u2013159 (2014).","journal-title":"Neurosci. Biobehav. Rev."},{"key":"1042_CR13","doi-asserted-by":"publisher","first-page":"e0282730","DOI":"10.1371\/journal.pone.0282730","volume":"18","author":"JR Saadon","year":"2023","unstructured":"Saadon, J. R. et al. Real-time emotion detection by quantitative facial motion analysis. PLoS ONE 18, e0282730 (2023).","journal-title":"PLoS ONE"},{"key":"1042_CR14","doi-asserted-by":"publisher","first-page":"8","DOI":"10.1016\/j.jneumeth.2006.11.017","volume":"162","author":"JW Peirce","year":"2007","unstructured":"Peirce, J. W. PsychoPy\u2014psychophysics software in Python. J. Neurosci. Methods 162, 8\u201313 (2007).","journal-title":"J. Neurosci. Methods"},{"key":"1042_CR15","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1016\/j.neunet.2005.06.042","volume":"18","author":"A Graves","year":"2005","unstructured":"Graves, A. & Schmidhuber, J. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18, 602\u2013610 (2005).","journal-title":"Neural Netw."},{"key":"1042_CR16","doi-asserted-by":"publisher","first-page":"28","DOI":"10.2344\/0003-3006-59.1.28","volume":"59","author":"DE Becker","year":"2012","unstructured":"Becker, D. E. Pharmacodynamic considerations for moderate and deep sedation. Anesth. Prog. 59, 28\u201342 (2012).","journal-title":"Anesth. Prog."},{"key":"1042_CR17","unstructured":"Miller, R. D. et al. Miller\u2019s Anesthesia (Elsevier Health Sciences, 2014)."},{"key":"1042_CR18","unstructured":"Barash, P. G. Clinical Anesthesia (Lippincott Williams and Wilkins, 2009)."},{"key":"1042_CR19","unstructured":"Reel, B. & Maani, C. V. Dexmedetomidine. in StatPearls. https:\/\/www.ncbi.nlm.nih.gov\/books\/NBK513303\/ (StatPearls Publishing, 2024)."},{"key":"1042_CR20","doi-asserted-by":"crossref","unstructured":"Barr, J. et al. Clinical practice guidelines for the management of pain, agitation, and delirium in adult patients in the intensive care unit. Crit. Care Med. 41, 263\u2013306 (2013).","DOI":"10.1097\/CCM.0b013e3182a167d7"},{"key":"1042_CR21","unstructured":"Ramos-Matos, C. F., Bistas, K. G. & Lopez-Ojeda, W. Fentanyl. in StatPearls. https:\/\/www.ncbi.nlm.nih.gov\/books\/NBK459275\/ (StatPearls Publishing, 2024)."},{"key":"1042_CR22","doi-asserted-by":"publisher","first-page":"358","DOI":"10.4103\/0019-5049.118558","volume":"57","author":"S Sen","year":"2013","unstructured":"Sen, S., Chakraborty, J., Santra, S., Mukherjee, P. & Das, B. The effect of dexmedetomidine infusion on propofol requirement for maintenance of optimum depth of anaesthesia during elective spine surgery. Indian J. Anaesth. 57, 358\u2013363 (2013).","journal-title":"Indian J. Anaesth."},{"key":"1042_CR23","doi-asserted-by":"crossref","unstructured":"Omar, M. Complications following hospital admission for traumatic brain injury: a multicenter cohort study. J. Crit. Care 41, 1\u20138 (2017).","DOI":"10.1016\/j.jcrc.2017.04.031"},{"key":"1042_CR24","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1016\/S0140-6736(74)91639-0","volume":"2","author":"G Teasdale","year":"1974","unstructured":"Teasdale, G. & Jennett, B. Assessment of coma and impaired consciousness. A practical scale. Lancet 2, 81\u201384 (1974).","journal-title":"Lancet"},{"key":"1042_CR25","doi-asserted-by":"publisher","first-page":"1614","DOI":"10.1212\/01.wnl.0000334754.15330.69","volume":"71","author":"C Schnakers","year":"2008","unstructured":"Schnakers, C. et al. Voluntary brain processing in disorders of consciousness. Neurology 71, 1614\u20131620 (2008).","journal-title":"Neurology"},{"key":"1042_CR26","doi-asserted-by":"publisher","first-page":"718","DOI":"10.1002\/ana.24779","volume":"80","author":"S Casarotto","year":"2016","unstructured":"Casarotto, S. et al. Stratification of unresponsive patients by an independently validated index of brain complexity: complexity Index. Ann. Neurol. 80, 718\u2013729 (2016).","journal-title":"Ann. Neurol."},{"key":"1042_CR27","doi-asserted-by":"publisher","first-page":"4645","DOI":"10.1093\/brain\/awad197","volume":"146","author":"E Franzova","year":"2023","unstructured":"Franzova, E. et al. Injury patterns associated with cognitive motor dissociation. Brain 146, 4645\u20134658 (2023).","journal-title":"Brain"},{"key":"1042_CR28","doi-asserted-by":"publisher","first-page":"2258","DOI":"10.1093\/brain\/awu141","volume":"137","author":"JD Sitt","year":"2014","unstructured":"Sitt, J. D. et al. Large scale screening of neural signatures of consciousness in patients in a vegetative or minimally conscious state. Brain 137, 2258\u20132270 (2014).","journal-title":"Brain"},{"key":"1042_CR29","doi-asserted-by":"publisher","first-page":"1855","DOI":"10.1016\/j.apmr.2012.10.041","volume":"94","author":"J Whyte","year":"2013","unstructured":"Whyte, J. et al. Functional outcomes in traumatic disorders of consciousness: 5-year outcomes from the National Institute on Disability and Rehabilitation Research Traumatic Brain Injury Model Systems. Arch. Phys. Med. Rehabil. 94, 1855\u20131860 (2013).","journal-title":"Arch. Phys. Med. Rehabil."},{"key":"1042_CR30","doi-asserted-by":"publisher","first-page":"548","DOI":"10.1001\/jamaneurol.2021.0084","volume":"78","author":"RG Kowalski","year":"2021","unstructured":"Kowalski, R. G. et al. Recovery of consciousness and functional outcome in moderate and severe traumatic brain injury. JAMA Neurol. 78, 548\u2013557 (2021).","journal-title":"JAMA Neurol."},{"key":"1042_CR31","doi-asserted-by":"publisher","first-page":"1404","DOI":"10.1093\/brain\/awy070","volume":"141","author":"WH Curley","year":"2018","unstructured":"Curley, W. H., Forgacs, P. B., Voss, H. U., Conte, M. M. & Schiff, N. D. Characterization of EEG signals revealing covert cognition in the injured brain. Brain 141, 1404\u20131421 (2018).","journal-title":"Brain"},{"key":"1042_CR32","doi-asserted-by":"publisher","first-page":"97","DOI":"10.4103\/1817-1737.62473","volume":"5","author":"M Bahloul","year":"2010","unstructured":"Bahloul, M. et al. Pulmonary embolism in intensive care unit: predictive factors, clinical manifestations and outcome. Ann. Thorac. Med. 5, 97\u2013103 (2010).","journal-title":"Ann. Thorac. Med."},{"key":"1042_CR33","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2023.1123793","volume":"10","author":"OR P\u00e9rez-Nieto","year":"2023","unstructured":"P\u00e9rez-Nieto, O. R. et al. Hemodynamic and respiratory support in pulmonary embolism: a narrative review. Front. Med.10, 1123793 (2023).","journal-title":"Front. Med."},{"key":"1042_CR34","doi-asserted-by":"publisher","DOI":"10.1186\/cc8907","volume":"14","author":"S Nseir","year":"2010","unstructured":"Nseir, S., Makris, D., Mathieu, D., Durocher, A. & Marquette, C.-H. Intensive care unit-acquired infection as a side effect of sedation. Crit. Care 14, R30 (2010).","journal-title":"Crit. Care"},{"key":"1042_CR35","doi-asserted-by":"publisher","first-page":"541","DOI":"10.4103\/1735-9066.193420","volume":"21","author":"M Shahabi","year":"2016","unstructured":"Shahabi, M., Yousefi, H., Yazdannik, A. R. & Alikiaii, B. The effect of daily sedation interruption protocol on early incidence of ventilator-associated pneumonia among patients hospitalized in critical care units receiving mechanical ventilation. Iran. J. Nurs. Midwifery Res. 21, 541\u2013546 (2016).","journal-title":"Iran. J. Nurs. Midwifery Res."},{"key":"1042_CR36","doi-asserted-by":"publisher","first-page":"627008","DOI":"10.3389\/fsurg.2021.627008","volume":"8","author":"Z Wang","year":"2021","unstructured":"Wang, Z. et al. Agitation following severe traumatic brain injury is a clinical sign of recovery of consciousness. Front. Surg. 8, 627008 (2021).","journal-title":"Front. Surg."},{"key":"1042_CR37","doi-asserted-by":"publisher","DOI":"10.1007\/s12028-023-01924-w","author":"N Schiff","year":"2024","unstructured":"Schiff, N. et al. Brain-computer interfaces for communication in patients with disorders of consciousness: a gap analysis and scientific roadmap. Neurocrit. Care https:\/\/doi.org\/10.1007\/s12028-023-01924-w (2024).","journal-title":"Neurocrit. Care"},{"key":"1042_CR38","doi-asserted-by":"publisher","first-page":"143","DOI":"10.1161\/STROKEAHA.115.010477","volume":"47","author":"J Dawson","year":"2016","unstructured":"Dawson, J. et al. Safety, feasibility, and efficacy of vagus nerve stimulation paired with upper-limb rehabilitation after ischemic stroke. Stroke 47, 143\u2013150 (2016).","journal-title":"Stroke"},{"key":"1042_CR39","doi-asserted-by":"publisher","first-page":"1545","DOI":"10.1016\/S0140-6736(21)00475-X","volume":"397","author":"J Dawson","year":"2021","unstructured":"Dawson, J. et al. Vagus nerve stimulation paired with rehabilitation for upper limb motor function after ischaemic stroke (VNS-REHAB): a randomised, blinded, pivotal, device trial. Lancet 397, 1545\u20131553 (2021).","journal-title":"Lancet"},{"key":"1042_CR40","doi-asserted-by":"publisher","first-page":"100071","DOI":"10.1016\/j.crneur.2022.100071","volume":"4","author":"YB Saalmann","year":"2023","unstructured":"Saalmann, Y. B., Mofakham, S., Mikell, C. B. & Djuric, P. M. Microscale multicircuit brain stimulation: achieving real-time brain state control for novel applications. Curr. Res. Neurobiol. 4, 100071 (2023).","journal-title":"Curr. Res. Neurobiol."}],"container-title":["Communications Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s43856-025-01042-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s43856-025-01042-y","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s43856-025-01042-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,9]],"date-time":"2025-09-09T09:31:38Z","timestamp":1757410298000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s43856-025-01042-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,20]]},"references-count":40,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["1042"],"URL":"https:\/\/doi.org\/10.1038\/s43856-025-01042-y","relation":{},"ISSN":["2730-664X"],"issn-type":[{"value":"2730-664X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,20]]},"assertion":[{"value":"19 March 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"17 July 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 August 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare the following competing interests: S.M., C.B.M., and J.D. disclose interests in SeeMe Technologies, a startup based on the methodology described in this report.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"361"}}