{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T01:27:38Z","timestamp":1773192458154,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2022,9,6]],"date-time":"2022-09-06T00:00:00Z","timestamp":1662422400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003696","name":"Electronics and Telecommunications Research Institute (ETRI)","doi-asserted-by":"publisher","award":["22ZS1100"],"award-info":[{"award-number":["22ZS1100"]}],"id":[{"id":"10.13039\/501100003696","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Simultaneous activation of brain regions (i.e., brain connection features) is an essential mechanism of brain activity in emotion recognition of visual content. The occipital cortex of the brain is involved in visual processing, but the frontal lobe processes cranial nerve signals to control higher emotions. However, recognition of emotion in visual content merits the analysis of eye movement features, because the pupils, iris, and other eye structures are connected to the nerves of the brain. We hypothesized that when viewing video content, the activation features of brain connections are significantly related to eye movement characteristics. We investigated the relationship between brain connectivity (strength and directionality) and eye movement features (left and right pupils, saccades, and fixations) when 47 participants viewed an emotion-eliciting video on a two-dimensional emotion model (valence and arousal). We found that the connectivity eigenvalues of the long-distance prefrontal lobe, temporal lobe, parietal lobe, and center are related to cognitive activity involving high valance. In addition, saccade movement was correlated with long-distance occipital-frontal connectivity. Finally, short-distance connectivity results showed emotional fluctuations caused by unconscious stimulation.<\/jats:p>","DOI":"10.3390\/s22186736","type":"journal-article","created":{"date-parts":[[2022,9,8]],"date-time":"2022-09-08T04:18:32Z","timestamp":1662610712000},"page":"6736","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Recognition of Emotion by Brain Connectivity and Eye Movement"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5230-6285","authenticated-orcid":false,"given":"Jing","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Emotion Engineering, Sangmyung University, Seoul 03016, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1242-9264","authenticated-orcid":false,"given":"Sung","family":"Park","sequence":"additional","affiliation":[{"name":"Department of Emotion Engineering, Sangmyung University, Seoul 03016, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0035-3853","authenticated-orcid":false,"given":"Ayoung","family":"Cho","sequence":"additional","affiliation":[{"name":"Department of Emotion Engineering, Sangmyung University, Seoul 03016, Korea"}]},{"given":"Mincheol","family":"Whang","sequence":"additional","affiliation":[{"name":"Department of Human-Centered Artificial Intelligence, Sangmyung University, Seoul 03016, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1016\/j.ijhcs.2009.03.005","article-title":"Short-term emotion assessment in a recall paradigm","volume":"67","author":"Chanel","year":"2009","journal-title":"Int. Hum. J. Comput. Stud."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"13","DOI":"10.1089\/brain.2011.0008","article-title":"Functional and effective connectivity: A review","volume":"1","author":"Friston","year":"2011","journal-title":"Brain Connect."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1080\/02699930802204677","article-title":"Measures of emotion: A review","volume":"23","author":"Mauss","year":"2009","journal-title":"Cogn. Emot."},{"key":"ref_4","unstructured":"Kim, J.-H., Kim, S.-L., Cha, Y.-S., Park, S.-I., and Hwang, M.-C. (2012, January 11). Analysis of CNS functional connectivity by relationship duration in positive emotion sharing relation. Proceedings of the Korean Society for Emotion and Sensibility Conference, Seoul, Korea."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Moon, S.-E., Jang, S., and Lee, J.-S. (2018, January 15\u201320). Convolutional neural network approach for EEG-based emotion recognition using brain connectivity and its spatial information. Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Calgary, AB, Canada.","DOI":"10.1109\/ICASSP.2018.8461315"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"598","DOI":"10.1002\/(SICI)1097-4571(1999)50:7<598::AID-ASI5>3.0.CO;2-6","article-title":"Searching scientific information on the Internet: A Dutch academic user survey","volume":"50","author":"Voorbij","year":"1999","journal-title":"Am. J. Soc. Inf. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"9479","DOI":"10.1523\/JNEUROSCI.0741-05.2005","article-title":"Neuronal basis of covert spatial attention in the frontal eye field","volume":"25","author":"Thompson","year":"2005","journal-title":"Neurosci. J."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"741","DOI":"10.1109\/TPAMI.2010.86","article-title":"Eye movement analysis for activity recognition using electrooculography","volume":"33","author":"Bulling","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"130","DOI":"10.1111\/j.1469-8986.2007.00606.x","article-title":"The coupling of emotion and cognition in the eye: Introducing the pupil old\/new effect","volume":"45","author":"Jacobs","year":"2008","journal-title":"Psychophysiology"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1037\/rev0000054","article-title":"LATEST: A model of saccadic decisions in space and time","volume":"124","author":"Tatler","year":"2017","journal-title":"Psychol. Rev."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"R291","DOI":"10.1016\/S0960-9822(00)00430-9","article-title":"The neural control of looking","volume":"10","author":"Carpenter","year":"2000","journal-title":"Curr. Biol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1146\/annurev.neuro.26.010302.081134","article-title":"The neurobiology of visual-saccadic decision making","volume":"26","author":"Glimcher","year":"2003","journal-title":"Annu. Rev. Neurosci."},{"key":"ref_13","unstructured":"Argyle, M., and Cook, M. (1976). Gaze and Mutual Gaze, Cambridge University Press."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1111\/j.2044-835X.1995.tb00687.x","article-title":"Are children with autism blind to the mentalistic significance of the eyes?","volume":"13","author":"Campbell","year":"1995","journal-title":"Br. Dev. J. Psychol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1016\/S0149-7634(00)00025-7","article-title":"The eyes have it: The neuroethology, function and evolution of social gaze","volume":"24","author":"Emery","year":"2000","journal-title":"Neurosci. Biobehav. Rev."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1037\/0033-2909.100.1.78","article-title":"Gaze and eye contact: A research review","volume":"100","author":"Kleinke","year":"1986","journal-title":"Psychol. Bull."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1111\/1467-9280.00024","article-title":"Adult\u2019s eyes trigger shifts of visual attention in human infants","volume":"9","author":"Hood","year":"1998","journal-title":"Psychol. Sci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1023\/A:1016374617369","article-title":"Visual scanning of faces in autism","volume":"32","author":"Pelphrey","year":"2002","journal-title":"J. Autism Dev. Disord."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1037\/a0020692","article-title":"Eye movement monitoring as a process tracing methodology in decision making research","volume":"4","author":"Glaholt","year":"2011","journal-title":"Neurosci. J. Psychol. Econ."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Peitek, N., Siegmund, J., Parnin, C., Apel, S., Hofmeister, J.C., and Brechmann, A. (2018, January 11\u201312). Simultaneous measurement of program comprehension with fmri and eye tracking: A case study. Proceedings of the 12th ACM\/IEEE International Symposium on Empirical Software Engineering and Measurement, Oulu, Finland.","DOI":"10.1145\/3239235.3240495"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, Z., Li, Z., Yang, F., Wang, L., Li, J., Zhou, C., and Pan, J. (2020). Advances in multimodal emotion recognition based on brain\u2013computer interfaces. Brain Sci., 10.","DOI":"10.3390\/brainsci10100687"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"16012","DOI":"10.1088\/1741-2552\/ac49a7","article-title":"Investigating EEG-based functional connectivity patterns for multimodal emotion recognition","volume":"19","author":"Wu","year":"2022","journal-title":"J. Neural Eng."},{"key":"ref_23","unstructured":"Lu, Y., Zheng, W.-L., Li, B., and Lu, B.-L. (2015, January 25\u201331). Combining eye movements and EEG to enhance emotion recognition. Proceedings of the 24th International Conference on Artificial Intelligence, Buenos Aires, Argentina."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1110","DOI":"10.1109\/TCYB.2018.2797176","article-title":"Emotionmeter: A multimodal framework for recognizing human emotions","volume":"49","author":"Zheng","year":"2018","journal-title":"IEEE Trans. Cybern."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1109\/T-AFFC.2011.37","article-title":"Multimodal emotion recognition in response to videos","volume":"3","author":"Soleymani","year":"2011","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1161","DOI":"10.1037\/h0077714","article-title":"A circumplex model of affect","volume":"39","author":"Russell","year":"1980","journal-title":"Pers. J. Soc. Psychol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1016\/j.physa.2015.03.087","article-title":"Brain connectivity analysis from EEG signals using stable phase-synchronized states during face perception tasks","volume":"434","author":"Jamal","year":"2015","journal-title":"Phys. A Stat. Mech. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Savine, A.C., and Braver, T.S. (2010). Motivated cognitive control: Reward incentives modulate preparatory neural activity during task-switching. Soc Neurosci., 10294\u201310305.","DOI":"10.1523\/JNEUROSCI.2052-10.2010"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ashby, F.G., and Isen, A.M. (1999). A neuropsychological theory of positive affect and its influence on cognition. Psychol. Rev., 106.","DOI":"10.1037\/\/0033-295X.106.3.529"},{"key":"ref_30","unstructured":"Moore, S.C., and Oaksford, M. (2002). The effects of positive affect and arousal and working memory and executive attention: Neurobiology and computational models. Emotional Cognition: From Brain to Behaviour, John Benjamins Publishing Company."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"781","DOI":"10.1007\/s11571-020-09625-2","article-title":"Brain connectivity analysis in fathers of children with autism","volume":"14","author":"Mehdizadehfar","year":"2020","journal-title":"Cogn. Neurodyn."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1971","DOI":"10.1038\/s41598-021-81454-7","article-title":"The role of dorsolateral and ventromedial prefrontal cortex in the processing of emotional dimensions","volume":"11","author":"Nejati","year":"2021","journal-title":"Sci. Rep."},{"key":"ref_33","unstructured":"Miller, B.L., and Cummings, J.L. (2007). New approaches to prefrontal lobe testing. The Human Frontal Lobes: Functions and Disorders, The Guilford Press."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"704","DOI":"10.1212\/CON.0000000000000609","article-title":"Clinical assessment of prefrontal lobe functions","volume":"24","author":"Stuss","year":"2018","journal-title":"Contin. Lifelong Learn. Neurol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1195","DOI":"10.1016\/j.cortex.2012.05.022","article-title":"Dorsolateral prefrontal contributions to human working memory","volume":"49","author":"Barbey","year":"2013","journal-title":"Cortex"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"6059","DOI":"10.1073\/pnas.1522551113","article-title":"Causal evidence for frontal cortex organization for perceptual decision making","volume":"113","author":"Rahnev","year":"2016","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"3700","DOI":"10.1038\/s41598-019-40273-7","article-title":"Differential role of prefrontal, temporal and parietal cortices in verbal and figural fluency: Implications for the supramodal contribution of executive functions","volume":"9","author":"Ghanavati","year":"2019","journal-title":"Sci. Rep."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1545","DOI":"10.1111\/j.1469-8986.2012.01484.x","article-title":"Cross-frequency dynamics of neuromagnetic oscillatory activity: Two mechanisms of emotion regulation","volume":"49","author":"Popov","year":"2012","journal-title":"Psychophysiology"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"466","DOI":"10.1016\/j.tics.2020.03.005","article-title":"Pupil size as a window on neural substrates of cognition","volume":"24","author":"Joshi","year":"2020","journal-title":"Trends Cogn. Sci."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1001\/archneurpsyc.1950.02310270002001","article-title":"Role of sympathetic and parasympathetic systems in reflex dilatation of the pupil: Pupillographic studies","volume":"64","author":"Lowenstein","year":"1950","journal-title":"Arch. Neurol. Psychiatry"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"828","DOI":"10.1152\/physrev.1972.52.4.828","article-title":"Accommodation function of the human eye","volume":"52","author":"Toates","year":"1972","journal-title":"Physiol. Rev."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6736\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:24:18Z","timestamp":1760142258000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/18\/6736"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,6]]},"references-count":41,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["s22186736"],"URL":"https:\/\/doi.org\/10.3390\/s22186736","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,6]]}}}