{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T13:40:36Z","timestamp":1791898836617,"version":"4.3.4"},"reference-count":91,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2022,8,1]],"date-time":"2022-08-01T00:00:00Z","timestamp":1659312000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["DC016353"],"award-info":[{"award-number":["DC016353"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurosci."],"abstract":"<jats:p>A key goal of cognitive neuroscience is to better understand how dynamic brain activity relates to behavior. Such dynamics, in terms of spatial and temporal patterns of brain activity, are directly measured with neurophysiological methods such as EEG, but can also be indirectly expressed by the body. Autonomic nervous system activity is the best-known example, but, muscles in the eyes and face can also index brain activity. Mostly parallel lines of artificial intelligence research show that EEG and facial muscles both encode information about emotion, pain, attention, and social interactions, among other topics. In this study, we examined adults who stutter (AWS) to understand the relations between dynamic brain and facial muscle activity and predictions about future behavior (fluent or stuttered speech). AWS can provide insight into brain-behavior dynamics because they naturally fluctuate between episodes of fluent and stuttered speech behavior. We focused on the period when speech preparation occurs, and used EEG and facial muscle activity measured from video to predict whether the upcoming speech would be fluent or stuttered. An explainable self-supervised multimodal architecture learned the temporal dynamics of both EEG and facial muscle movements during speech preparation in AWS, and predicted fluent or stuttered speech at 80.8% accuracy (chance=50%). Specific EEG and facial muscle signals distinguished fluent and stuttered trials, and systematically varied from early to late speech preparation time periods. The self-supervised architecture successfully identified multimodal activity that predicted upcoming behavior on a trial-by-trial basis. This approach could be applied to understanding the neural mechanisms driving variable behavior and symptoms in a wide range of neurological and psychiatric disorders. The combination of direct measures of neural activity and simple video data may be applied to developing technologies that estimate brain state from subtle bodily signals.<\/jats:p>","DOI":"10.3389\/fnins.2022.912798","type":"journal-article","created":{"date-parts":[[2022,8,1]],"date-time":"2022-08-01T06:27:15Z","timestamp":1659335235000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Multimodal explainable AI predicts upcoming speech behavior in adults who stutter"],"prefix":"10.3389","volume":"16","author":[{"given":"Arun","family":"Das","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeffrey","family":"Mock","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Farzan","family":"Irani","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yufei","family":"Huang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peyman","family":"Najafirad","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edward","family":"Golob","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2022,8,1]]},"reference":[{"key":"B1","first-page":"265","article-title":"\u201cTensorflow: a system for large-scale machine learning,\u201d","volume-title":"12th","author":"Abadi","year":"2016"},{"key":"B2","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR42600.2020.01469","article-title":"\u201cMultimodal categorization of crisis events in social media,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Abavisani","year":"2020"},{"key":"B3","first-page":"4979","article-title":"\u201cCross-modal deep face normals with deactivable skip connections,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Abrevaya","year":"2020"},{"key":"B4","doi-asserted-by":"crossref","first-page":"4409","DOI":"10.1109\/EMBC.2019.8856500","article-title":"\u201cMultimodal data fusion of deep learning and dynamic functional connectivity features to predict Alzheimer's disease progression,\u201d","volume-title":"2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)","author":"Abrol","year":"2019"},{"key":"B5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/MLSP.2019.8918771","article-title":"\u201cA spatiotemporal deep learning solution for automatic micro-expressions recognition from local facial regions,\u201d","volume-title":"2019 IEEE 29th International Workshop on Machine Learning for Signal Processing (MLSP)","author":"Aouayeb","year":"2019"},{"key":"B6","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1177\/1529100619832930","article-title":"Emotional expressions reconsidered: challenges to inferring emotion from human facial movements","volume":"20","author":"Barrett","year":"2019","journal-title":"Psychol. Sci. Public Interest"},{"key":"B7","article-title":"The trajectory of gray matter development in Broca's area is abnormal in people who stutter","volume-title":"Front. Hum. Neurosci","author":"Beal","year":"2015"},{"key":"B8","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1109\/MeMeA.2011.5966735","article-title":"\u201cA wearable device for recording of biopotentials and body movements,\u201d","volume-title":"2011 IEEE International Symposium on Medical Measurements and Applications","author":"Bifulco","year":"2011"},{"key":"B9","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR42600.2020.01170","article-title":"\u201cSeeing through fog without seeing fog: deep multimodal sensor fusion in unseen adverse weather,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Bijelic","year":"2020"},{"key":"B10","volume-title":"A Handbook on Stuttering","author":"Bloodstein","year":"2021"},{"key":"B11","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1111\/j.1469-8986.2008.00654.x","article-title":"The pupil as a measure of emotional arousal and autonomic activation","volume":"45","author":"Bradley","year":"2008","journal-title":"Psychophysiology"},{"key":"B12","first-page":"330","article-title":"Remarks on the seat of the faculty of articulated language, following an observation of aphemia (loss of speech)","volume":"6","author":"Broca","year":"1861","journal-title":"Bulletin de la Soci\u00e9t\u00e9 Anatomique"},{"key":"B13","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1002\/hbm.20140","article-title":"Stuttered and fluent speech production: an ale meta-analysis of functional neuroimaging studies","volume":"25","author":"Brown","year":"2005","journal-title":"Hum. Brain Mapp"},{"key":"B14","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1016\/j.bandl.2014.10.002","article-title":"Stuttering, induced fluency, and natural fluency: a hierarchical series of activation likelihood estimation meta-analyses","volume":"139","author":"Budde","year":"2014","journal-title":"Brain Lang"},{"key":"B15","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1038\/nrn3241","article-title":"The origin of extracellular fields and currents-EEG, ECOG, LFP and spikes","volume":"13","author":"Buzs\u00e1ki","year":"2012","journal-title":"Nat. Rev. Neurosci"},{"key":"B16","doi-asserted-by":"crossref","DOI":"10.1017\/9781107415782","volume-title":"Handbook of Psychophysiology, 4th Edn","author":"Cacioppo","year":"2016"},{"key":"B17","doi-asserted-by":"crossref","DOI":"10.3389\/fnhum.2014.00054","article-title":"Diffusion imaging of cerebral white matter in persons who stutter: evidence for network-level anomalies","volume-title":"Front. Hum. Neurosci","author":"Cai","year":"2014"},{"key":"B18","doi-asserted-by":"publisher","first-page":"566","DOI":"10.1177\/1073858418803594","article-title":"Functional and neuroanatomical bases of developmental stuttering: current insights","volume":"25","author":"Chang","year":"2019","journal-title":"Neuroscientist"},{"key":"B19","doi-asserted-by":"crossref","DOI":"10.1109\/TCYB.2021.3122969","article-title":"Neurograsp: real-time EEG classification of high-level motor imagery tasks using a dual-stage deep learning framework","volume-title":"IEEE Trans. Cybernet","author":"Cho","year":"2021"},{"key":"B20","doi-asserted-by":"publisher","first-page":"3345","DOI":"10.1002\/hbm.23590","article-title":"White matter developmental trajectories associated with persistence and recovery of childhood stuttering","volume":"38","author":"Chow","year":"2017","journal-title":"Hum. Brain Mapp"},{"key":"B21","doi-asserted-by":"publisher","first-page":"14090","DOI":"10.1109\/JSEN.2021.3069113","article-title":"Design of UHF RFID sensor-tags for the biomechanical analysis of human body movements","volume":"21","author":"Colella","year":"2021","journal-title":"IEEE Sensors J"},{"key":"B22","doi-asserted-by":"publisher","first-page":"456","DOI":"10.2307\/1127968","article-title":"Synchrony demonstrated between movements of the neonate and adult speech","volume":"45","author":"Condon","year":"1974","journal-title":"Child Dev"},{"key":"B23","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1016\/j.jfludis.2009.05.002","article-title":"The impact of stuttering on the quality of life in adults who stutter","volume":"34","author":"Craig","year":"2009","journal-title":"J. Fluency Disord"},{"key":"B24","doi-asserted-by":"publisher","first-page":"37","DOI":"10.1016\/j.bandl.2015.08.008","article-title":"Electrophysiological evidence for a general auditory prediction deficit in adults who stutter","volume":"150","author":"Daliri","year":"2015","journal-title":"Brain Lang"},{"key":"B25","doi-asserted-by":"crossref","DOI":"10.1037\/10001-000","volume-title":"The Expression of the Emotions in Man and Animals","author":"Darwin","year":"1872"},{"key":"B26","article-title":"Stuttering speech disfluency prediction using explainable attribution vectors of facial muscle movements","volume-title":"arXiv [Preprint] arXiv:","author":"Das","year":"2020"},{"key":"B27","first-page":"818","article-title":"\u201cInterpretable self-supervised facial micro-expression learning to predict cognitive state and neurological disorders,\u201d","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Das","year":"2021"},{"key":"B28","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1016\/j.jfludis.2003.07.002","article-title":"A positron emission tomography study of short-and long-term treatment effects on functional brain activation in adults who stutter","volume":"28","author":"De Nil","year":"2003","journal-title":"J. Fluency Disord"},{"key":"B29","doi-asserted-by":"publisher","first-page":"9377","DOI":"10.1109\/JSEN.2021.3057076","article-title":"Deep learning for patient-independent epileptic seizure prediction using scalp EEG signals","volume":"21","author":"Dissanayake","year":"2021","journal-title":"IEEE Sensors J"},{"key":"B30","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1098\/rstb.1992.0008","article-title":"Facial expressions of emotion: an old controversy and new findings","volume":"335","author":"Ekman","year":"1992","journal-title":"Philos. Trans. R. Soc. Lond. Ser. B Biol. Sci"},{"key":"B31","volume-title":"Telling Lies: Clues to Deceit in the Marketplace, Politics, and Marriage","author":"Ekman","year":"2009"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1208","DOI":"10.1126\/science.6612338","article-title":"Autonomic nervous system activity distinguishes among emotions","volume":"221","author":"Ekman","year":"1983","journal-title":"Science"},{"key":"B33","doi-asserted-by":"crossref","DOI":"10.1093\/acprof:oso\/9780195179644.001.0001","volume-title":"What the Face Reveals Basic and Applied Studies of Spontaneous Expression Using the Facial Action Coding System (FACS)","author":"Ekman","year":"2005"},{"key":"B34","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1111\/j.2044-835X.1998.tb00766.x","article-title":"Facial expression and EEG responses to happy and sad faces\/voices by 3-month-old infants of depressed mothers","volume":"16","author":"Field","year":"1998","journal-title":"Brit. J. Dev. Psychol"},{"key":"B35","volume-title":"Facial Action Coding System: A Technique for the Measurement of Facial Movement","author":"Friesen","year":"1978"},{"key":"B36","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1177\/1745691616654457","article-title":"The eyes as windows into other minds: an integrative perspective","volume":"12","author":"Grossmann","year":"2017","journal-title":"Perspect. Psychol. Sci"},{"key":"B37","volume-title":"Stuttering: An Integrated Approach to Its Nature and Treatment, 4th Edn","author":"Guitar","year":"2014"},{"key":"B38","doi-asserted-by":"crossref","DOI":"10.1109\/CVPRW.2019.00016","article-title":"\u201cBag-of-lies: a multimodal dataset for deception detection,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops","author":"Gupta","year":"2019"},{"key":"B39","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1016\/j.jneumeth.2011.06.023","article-title":"Automated facial action coding system for dynamic analysis of facial expressions in neuropsychiatric disorders","volume":"200","author":"Hamm","year":"2011","journal-title":"J. Neurosci. Methods"},{"key":"B40","doi-asserted-by":"publisher","first-page":"1242","DOI":"10.1044\/jslhr.4106.1242","article-title":"Two-to six-year controlled-trial stuttering outcomes for children and adolescents","volume":"41","author":"Hancock","year":"1998","journal-title":"J. Speech Lang. Hear. Res"},{"key":"B41","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1016\/j.brainresrev.2004.07.013","article-title":"Investigating the cortical origins of motor overflow","volume":"46","author":"Hoy","year":"2004","journal-title":"Brain Res. Rev"},{"key":"B42","doi-asserted-by":"crossref","DOI":"10.1016\/j.cmpb.2021.106121","article-title":"An integrated deep learning model for motor intention recognition of multi-class EEG signals in upper limb amputees","volume-title":"Comput. Methods Prog. Biomed","author":"Idowu","year":"2021"},{"key":"B43","doi-asserted-by":"crossref","DOI":"10.1155\/2020\/2398542","article-title":"Automatic facial paralysis assessment via computational image analysis","volume-title":"J. Healthcare Eng","author":"Jiang","year":"2020"},{"key":"B44","doi-asserted-by":"crossref","DOI":"10.1155\/2014\/627892","article-title":"EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation","volume-title":"Sci. World J","author":"Jirayucharoensak","year":"2014"},{"key":"B45","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1016\/j.neubiorev.2016.08.020","article-title":"Spontaneous eye blink rate as predictor of dopamine-related cognitive function\u2013a review","volume":"71","author":"Jongkees","year":"2016","journal-title":"Neurosci. Biobehav. Rev"},{"key":"B46","doi-asserted-by":"publisher","first-page":"2747","DOI":"10.1093\/brain\/awp185","article-title":"How the brain repairs stuttering","volume":"132","author":"Kell","year":"2009","journal-title":"Brain"},{"key":"B47","doi-asserted-by":"publisher","first-page":"1025","DOI":"10.1044\/jshr.3805.1025","article-title":"Orofacial muscle activity of children who stutter: a preliminary study","volume":"38","author":"Kelly","year":"1995","journal-title":"J. Speech Lang. Hear. Res"},{"key":"B48","volume-title":"Dynamic Patterns: The Self-Organization of Brain and Behavior","author":"Kelso","year":"1995"},{"key":"B49","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1002\/mds.870110311","article-title":"Stuttering may be a type of action dystonia","volume":"11","author":"Kiziltan","year":"1996","journal-title":"Movement Disord"},{"key":"B50","first-page":"1106","article-title":"\u201cImageNet classification with deep convolutional neural networks,\u201d","volume-title":"Advances in Neural Information Processing Systems, Vol. 25","author":"Krizhevsky","year":"2012"},{"key":"B51","doi-asserted-by":"publisher","first-page":"535","DOI":"10.1097\/j.pain.0000000000001424","article-title":"Facial muscle movements encoding pain\u2013a systematic review","volume":"160","author":"Kunz","year":"2019","journal-title":"Pain"},{"key":"B52","doi-asserted-by":"publisher","first-page":"4022","DOI":"10.1162\/jocn_a_00077","article-title":"Behavioral interpretations of intrinsic connectivity networks","volume":"23","author":"Laird","year":"2011","journal-title":"J. Cogn. Neurosci"},{"key":"B53","doi-asserted-by":"crossref","DOI":"10.3389\/frai.2021.610197","article-title":"An explainable multimodal neural network architecture for predicting epilepsy comorbidities based on administrative claims data","volume-title":"Front. Artif. Intell","author":"Linden","year":"2021"},{"key":"B54","doi-asserted-by":"publisher","first-page":"678","DOI":"10.1111\/j.1526-4637.2007.00358.x","article-title":"A psychophysical investigation of the facial action coding system as an index of pain variability among older adults with and without Alzheimer's disease","volume":"8","author":"Lints-Martindale","year":"2007","journal-title":"Pain Med"},{"key":"B55","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.bandl.2017.01.001","article-title":"Reorganization of brain function after a short-term behavioral intervention for stuttering","volume":"168","author":"Lu","year":"2017","journal-title":"Brain Lang"},{"key":"B56","first-page":"4765","article-title":"\u201cA unified approach to interpreting model predictions,\u201d","volume-title":"Advances in Neural Information Processing Systems","author":"Lundberg","year":"2017"},{"key":"B57","doi-asserted-by":"publisher","first-page":"204","DOI":"10.1016\/j.tics.2004.03.008","article-title":"Mining event-related brain dynamics","volume":"8","author":"Makeig","year":"2004","journal-title":"Trends Cogn. Sci"},{"key":"B58","doi-asserted-by":"publisher","first-page":"1131","DOI":"10.1016\/j.clinph.2011.10.003","article-title":"Exploring semantic and phonological picture-word priming in adults who stutter using event-related potentials","volume":"123","author":"Maxfield","year":"2012","journal-title":"Clin. Neurophysiol"},{"key":"B59","doi-asserted-by":"publisher","first-page":"458","DOI":"10.1111\/1460-6984.12021","article-title":"The impact of adolescent stuttering and other speech problems on psychological well-being in adulthood: evidence from a birth cohort study","volume":"48","author":"McAllister","year":"2013","journal-title":"Int. J. Lang. Commun. Disord"},{"key":"B60","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1109\/TAFFC.2017.2749299","article-title":"Listen to your face: inferring facial action units from audio channel","volume":"10","author":"Meng","year":"2017","journal-title":"IEEE Trans. Affect. Comput"},{"key":"B61","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1016\/j.bandl.2015.05.009","article-title":"Speech preparation in adults with persistent developmental stuttering","volume":"149","author":"Mock","year":"2015","journal-title":"Brain Lang"},{"key":"B62","doi-asserted-by":"publisher","first-page":"3093","DOI":"10.1016\/j.clinph.2016.06.005","article-title":"Cortical activity during cued picture naming predicts individual differences in stuttering frequency","volume":"127","author":"Mock","year":"2016","journal-title":"Clin. Neurophysiol"},{"key":"B63","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1080\/87565641.2012.661816","article-title":"Atypical brain torque in boys with developmental stuttering","volume":"37","author":"Mock","year":"2012","journal-title":"Dev. Neuropsychol"},{"key":"B64","doi-asserted-by":"publisher","first-page":"511","DOI":"10.1146\/annurev-psych-120709-145401","article-title":"Human intracranial recordings and cognitive neuroscience","volume":"63","author":"Mukamel","year":"2012","journal-title":"Annu. Rev. Psychol"},{"key":"B65","first-page":"57","article-title":"\u201cSingle-trial classification of disfluent brain states in adults who stutter,\u201d","volume-title":"Proc","author":"Myers","year":"2019"},{"key":"B66","first-page":"689","article-title":"\u201cMultimodal deep learning,\u201d","volume-title":"Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML'11","author":"Ngiam","year":"2011"},{"key":"B67","article-title":"A deep learning approach for Parkinson's disease diagnosis from EEG signals","volume-title":"Neural Comput. Appl","author":"Oh","year":"2018"},{"key":"B68","first-page":"8779","article-title":"\u201cMultimodal explanations: justifying decisions and pointing to the evidence,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Park","year":"2018"},{"key":"B69","doi-asserted-by":"crossref","DOI":"10.1109\/CVPRW50498.2020.00149","article-title":"\u201cPredicting brainwaves from face videos,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops","author":"Pilz","year":"2020"},{"key":"B70","unstructured":"Stuttering: an overview12711276\n            PrasseJ. E.\n            KikanoG. E.\n          Am. Fam. Phys772008"},{"key":"B71","doi-asserted-by":"publisher","first-page":"433","DOI":"10.1177\/1073858407305618","article-title":"Human facial expressions are organized functionally across the upper-lower facial axis","volume":"13","author":"Ross","year":"2007","journal-title":"Neuroscientist"},{"key":"B72","first-page":"169","article-title":"\u201cCar-driver drowsiness monitoring by multi-layers deep learning framework and motion analysis,\u201d","volume-title":"AISEM Annual Conference on Sensors and Microsystems","author":"Rundo","year":"2019"},{"key":"B73","doi-asserted-by":"publisher","first-page":"11","DOI":"10.1136\/jnnp.20.1.11","article-title":"Loss of recent memory after bilateral hippocampal lesions","volume":"20","author":"Scoville","year":"1957","journal-title":"J. Neurol. Neurosurg. Psychiatry"},{"key":"B74","doi-asserted-by":"publisher","first-page":"1299","DOI":"10.1126\/science.3045969","article-title":"Computational neuroscience","volume":"241","author":"Sejnowski","year":"1988","journal-title":"Science"},{"key":"B75","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1016\/0021-9924(74)90031-8","article-title":"Stuttering behavior: a phonetic analysis","volume":"7","author":"Sheehan","year":"1974","journal-title":"J. Commun. Disord"},{"key":"B76","first-page":"5709","article-title":"\u201cRevisiting image aesthetic assessment via self-supervised feature learning,\u201d","volume-title":"AAAI","author":"Sheng","year":"2020"},{"key":"B77","article-title":"Learning important features through propagating activation differences","volume-title":"arXiv [Preprint] arXiv:","author":"Shrikumar","year":"2017"},{"key":"B78","doi-asserted-by":"publisher","first-page":"2483","DOI":"10.1044\/2017_JSLHR-S-16-0343","article-title":"How stuttering develops: the multifactorial dynamic pathways theory","volume":"60","author":"Smith","year":"2017","journal-title":"J. Speech Lang. Hear. Res"},{"key":"B79","doi-asserted-by":"publisher","first-page":"13040","DOI":"10.1073\/pnas.0905267106","article-title":"Correspondence of the brain's functional architecture during activation and rest","volume":"106","author":"Smith","year":"2009","journal-title":"Proc. Natl. Acad. Sci. U.S.A"},{"key":"B80","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1016\/j.jfludis.2005.11.004","article-title":"The transition to increased automaticity during finger sequence learning in adult males who stutter","volume":"31","author":"Smits-Bandstra","year":"2006","journal-title":"J. Fluency Disord"},{"key":"B81","first-page":"6809","article-title":"\u201cDeep learning human mind for automated visual classification,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Spampinato","year":"2017"},{"key":"B82","doi-asserted-by":"publisher","first-page":"2266","DOI":"10.1016\/j.clinph.2005.06.011","article-title":"Nonlinear dynamical analysis of EEG and MEG: review of an emerging field","volume":"116","author":"Stam","year":"2005","journal-title":"Clin. Neurophysiol"},{"key":"B83","doi-asserted-by":"publisher","first-page":"22","DOI":"10.1111\/j.1469-8986.1984.tb02312.x","article-title":"The endogenous eyeblink","volume":"21","author":"Stern","year":"1984","journal-title":"Psychophysiology"},{"key":"B84","doi-asserted-by":"crossref","DOI":"10.3390\/brainsci10020085","article-title":"Multimodal affective state assessment using fNIRS+ EEG and spontaneous facial expression","volume-title":"Brain Sci","author":"Sun","year":"2020"},{"key":"B85","doi-asserted-by":"crossref","DOI":"10.1109\/CVPR42600.2020.01382","article-title":"\u201cSelf-supervised learning of video-induced visual invariances,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Tschannen","year":"2020"},{"key":"B86","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-018-21495-7","article-title":"Predicting sex from brain rhythms with deep learning","volume":"8","author":"Van Putten","year":"2018","journal-title":"Sci. Rep"},{"key":"B87","doi-asserted-by":"publisher","first-page":"1618","DOI":"10.1109\/TIP.2019.2912358","article-title":"Learnet: dynamic imaging network for micro expression recognition","volume":"29","author":"Verma","year":"2019","journal-title":"IEEE Trans. Image Process"},{"key":"B88","article-title":"Multi-task self-supervised learning for disfluency detection","volume-title":"arXiv [Preprint] arXiv:","author":"Wang","year":"2019"},{"key":"B89","doi-asserted-by":"crossref","first-page":"3422","DOI":"10.1109\/CVPR.2013.439","article-title":"\u201cCapturing complex spatio-temporal relations among facial muscles for facial expression recognition,\u201d","volume-title":"2013 IEEE Conference on Computer Vision and Pattern Recognition","author":"Wang","year":"2013"},{"key":"B90","doi-asserted-by":"crossref","DOI":"10.1145\/3474085.3479209","article-title":"\u201cFacial action unit-based deep learning framework for spotting macro-and micro-expressions in long video sequences,\u201d","volume-title":"Proceedings of the 29th ACM International Conference on Multimedia","author":"Yang","year":"2021"},{"key":"B91","doi-asserted-by":"publisher","first-page":"1571","DOI":"10.1109\/TMI.2019.2953584","article-title":"Imaging of nonlinear and dynamic functional brain connectivity based on EEG recordings with the application on the diagnosis of Alzheimer's disease","volume":"39","author":"Zhao","year":"2019","journal-title":"IEEE Trans. Med. Imaging"}],"container-title":["Frontiers in Neuroscience"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2022.912798\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,1]],"date-time":"2022-08-01T06:27:51Z","timestamp":1659335271000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnins.2022.912798\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,1]]},"references-count":91,"alternative-id":["10.3389\/fnins.2022.912798"],"URL":"https:\/\/doi.org\/10.3389\/fnins.2022.912798","relation":{},"ISSN":["1662-453X"],"issn-type":[{"value":"1662-453X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,1]]},"article-number":"912798"}}