{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T12:34:42Z","timestamp":1783427682141,"version":"3.54.6"},"reference-count":57,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,10,10]],"date-time":"2023-10-10T00:00:00Z","timestamp":1696896000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>Significant efforts have been made in the past decade to humanize both the form and function of social robots to increase their acceptance among humans. To this end, social robots have recently been combined with brain-computer interface (BCI) systems in an attempt to give them an understanding of human mental states, particularly emotions. However, emotion recognition using BCIs poses several challenges, such as subjectivity of emotions, contextual dependency, and a lack of reliable neuro-metrics for real-time processing of emotions. Furthermore, the use of BCI systems introduces its own set of limitations, such as the bias-variance trade-off, dimensionality, and noise in the input data space. In this study, we sought to address some of these challenges by detecting human emotional states from EEG brain activity during human-robot interaction (HRI). EEG signals were collected from 10 participants who interacted with a Pepper robot that demonstrated either a positive or negative personality. Using emotion valence and arousal measures derived from frontal brain asymmetry (FBA), several machine learning models were trained to classify human's mental states in response to the robot personality. To improve classification accuracy, all proposed classifiers were subjected to a Global Optimization Model (GOM) based on feature selection and hyperparameter optimization techniques. The results showed that it is possible to classify a user's emotional responses to the robot's behavior from the EEG signals with an accuracy of up to 92%. The outcome of the current study contributes to the first level of the Theory of Mind (ToM) in Human-Robot Interaction, enabling robots to comprehend users' emotional responses and attribute mental states to them. Our work advances the field of social and assistive robotics by paving the way for the development of more empathetic and responsive HRI in the future.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1191127","type":"journal-article","created":{"date-parts":[[2023,10,10]],"date-time":"2023-10-10T04:56:17Z","timestamp":1696913777000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":19,"title":["Classifying human emotions in HRI: applying global optimization model to EEG brain signals"],"prefix":"10.3389","volume":"17","author":[{"given":"Mariacarla","family":"Staffa","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lorenzo","family":"D'Errico","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Simone","family":"Sansalone","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Maryam","family":"Alimardani","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,10,10]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1016\/j.iswa.2022.200171","article-title":"A systematic survey on multimodal emotion recognition using learning algorithms","author":"Ahmed","year":"2023","journal-title":"Int. J. Intell. Syst. Appl."},{"key":"B2","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1007\/s11517-017-1733-8","article-title":"Towards multilevel mental stress assessment using SVM with ECOC: an EEG approach","volume":"56","author":"Al shargie","year":"2018","journal-title":"Med. Biol. Eng. Comput."},{"key":"B3","doi-asserted-by":"publisher","first-page":"125","DOI":"10.3389\/frobt.2020.00125","article-title":"Passive brain-computer interfaces for enhanced human-robot interaction","volume":"7","author":"Alimardani","year":"2020","journal-title":"Front. Robot. AI"},{"key":"B4","first-page":"683","article-title":"\u201cRobot-assisted mindfulness practice: analysis of neurophysiological responses and affective state change,\u201d","author":"Alimardani","year":"2020","journal-title":"2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)"},{"key":"B5","doi-asserted-by":"publisher","first-page":"2396","DOI":"10.1038\/srep02396","article-title":"Humanlike robot hands controlled by brain activity arouse illusion of ownership in operators","volume":"3","author":"Alimardani","year":"2013","journal-title":"Sci. Rep."},{"key":"B6","doi-asserted-by":"crossref","first-page":"671","DOI":"10.1007\/978-3-030-90525-5_59","article-title":"\u201cAssessment of engagement and learning during child-robot interaction using EEG signals,\u201d","volume-title":"Social Robotics: 13th International Conference, ICSR 2021","author":"Alimardani","year":"2021"},{"key":"B7","doi-asserted-by":"publisher","first-page":"419","DOI":"10.14569\/IJACSA.2017.080955","article-title":"Classification of human emotions from electroencephalogram (EEG) signal using deep neural network","volume":"8","author":"Al-Nafjan","year":"2017","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"B8","doi-asserted-by":"publisher","first-page":"e38","DOI":"10.7717\/peerj.38","article-title":"Validation of the Emotiv EPOC EEG gaming system for measuring research quality auditory ERPs","volume":"1","author":"Badcock","year":"2013","journal-title":"PeerJ"},{"key":"B9","doi-asserted-by":"publisher","first-page":"403","DOI":"10.1007\/s12369-019-00588-x","article-title":"Theory of mind in social robots: replication of five established human tests","volume":"12","author":"Banks","year":"2020","journal-title":"Int. J. Soc. Robot."},{"key":"B10","doi-asserted-by":"publisher","first-page":"293","DOI":"10.1007\/s11257-022-09347-6","article-title":"A dichotomic approach to adaptive interaction for socially assistive robots","volume":"33","author":"Benedictis","year":"2023","journal-title":"User Model. User Adapt. Interact."},{"key":"B11","first-page":"2546","article-title":"\u201cAlgorithms for hyper-parameter optimization,\u201d","volume-title":"Proceedings of the 24th International Conference on Neural Information Processing Systems, NIPS'11","author":"Bergstra","year":"2011"},{"key":"B12","first-page":"281","article-title":"Random search for hyper-parameter optimization","volume":"13","author":"Bergstra","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"B13","doi-asserted-by":"publisher","first-page":"3676","DOI":"10.1016\/j.patcog.2012.04.015","article-title":"Application of global optimization methods to model and feature selection","volume":"45","author":"Boubezoul","year":"2012","journal-title":"Pattern Recogn."},{"key":"B14","article-title":"Hyperparameter search in machine learning","author":"Claesen","year":"2015","journal-title":"arXiv preprint arXiv:1502.02127"},{"key":"B15","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1023\/B:JONB.0000023655.25550.be","article-title":"Attributing emotion to static body postures: recognition accuracy, confusions, and viewpoint dependence","volume":"28","author":"Coulson","year":"2004","journal-title":"J. Nonverbal Behav."},{"key":"B16","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1037\/0022-3514.58.2.330","article-title":"Approach-withdrawal and cerebral asymmetry: emotional expression and brain physiology. I","volume":"58","author":"Davidson","year":"1990","journal-title":"J. Pers. Soc. Psychol."},{"key":"B17","author":"Feurer","year":"2019","journal-title":"Hyperparameter Optimization"},{"key":"B18","doi-asserted-by":"publisher","first-page":"857","DOI":"10.1016\/j.procs.2020.04.093","article-title":"Emotion recognition in valence-arousal space from multi-channel EEG data and wavelet based deep learning framework","volume":"171","author":"Garg","year":"2020","journal-title":"Proc. Comput. Sci."},{"key":"B19","doi-asserted-by":"publisher","first-page":"255","DOI":"10.26599\/BSA.2020.9050026","article-title":"Video-triggered EEG-emotion public databases and current methods: a survey","volume":"6","author":"Hu","year":"2020","journal-title":"Brain Sci. Adv."},{"key":"B20","first-page":"1135","article-title":"\u201cAttentional and emotional regulation in human-robot interaction,\u201d","volume-title":"2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication","author":"Iengo","year":"2012"},{"key":"B21","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1109\/IJCNN.2011.6033297","article-title":"\u201cA brain-computer interface for classifying EEG correlates of chronic mental stress,\u201d","author":"Khosrowabadi","year":"2011","journal-title":"The 2011 International Joint Conference on Neural Networks"},{"key":"B22","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1109\/T-AFFC.2012.16","article-title":"Affective body expression perception and recognition: a survey","volume":"4","author":"Kleinsmith","year":"2013","journal-title":"IEEE Trans. Affect. Comput."},{"key":"B23","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1109\/T-AFFC.2011.15","article-title":"DEAP: a database for emotion analysis using physiological signals","volume":"3","author":"Koelstra","year":"2011","journal-title":"IEEE Trans. Affect. Comput."},{"key":"B24","first-page":"1","article-title":"\u201cPrediction of human empathy based on EEG cortical asymmetry,\u201d","author":"Kuijt","year":"2020","journal-title":"2020 IEEE International Conference on Human-Machine Systems (ICHMS)"},{"key":"B25","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1016\/j.biopsycho.2007.03.004","article-title":"The effect of a naturalistic stressor on frontal EEG asymmetry, stress, and health","volume":"75","author":"Lewis","year":"2007","journal-title":"Biol. Psychol."},{"key":"B26","doi-asserted-by":"publisher","first-page":"550","DOI":"10.1109\/TAFFC.2017.2660485","article-title":"Real-time movie-induced discrete emotion recognition from EEG signals","volume":"9","author":"Liu","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"B27","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/978-981-10-8354-9_19","article-title":"\u201cEvaluating the induced emotions on physiological response,\u201d","volume-title":"Computational Signal Processing and Analysis","author":"Menon","year":"2018"},{"key":"B28","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1080\/2326263X.2014.912881","article-title":"A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges","volume":"1","author":"M\u00fchl","year":"2014","journal-title":"Brain Comput. Interfaces"},{"key":"B29","doi-asserted-by":"publisher","first-page":"2553","DOI":"10.1109\/TBME.2015.2481482","article-title":"Real-time neuroimaging and cognitive monitoring using wearable dry EEG","volume":"62","author":"Mullen","year":"2015","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"B30","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1109\/T-AFFC.2011.9","article-title":"Continuous prediction of spontaneous affect from multiple cues and modalities in valence-arousal space","volume":"2","author":"Nicolaou","year":"2011","journal-title":"IEEE Trans. Affect. Comput."},{"key":"B31","doi-asserted-by":"publisher","first-page":"171","DOI":"10.1007\/s12369-012-0171-x","article-title":"Making social robots more attractive: the effects of voice pitch, humor and empathy","volume":"5","author":"Niculescu","year":"2013","journal-title":"Int. J. Soc. Robot."},{"key":"B32","doi-asserted-by":"publisher","first-page":"7607592","DOI":"10.1155\/2022\/7607592","article-title":"Stress classification using brain signals based on LSTM network","volume":"2022","author":"Nishtha","year":"2022","journal-title":"Comput. Intell. Neurosci."},{"key":"B33","doi-asserted-by":"crossref","first-page":"322","DOI":"10.1109\/AMC.2019.8371111","article-title":"\u201cAn EEG-based robot arm control to express human emotions,\u201d","volume-title":"2018 IEEE 15th International Workshop on Advanced Motion Control (AMC)","author":"Ogino","year":"2018"},{"key":"B34","doi-asserted-by":"publisher","first-page":"e14320","DOI":"10.1111\/psyp.14320","article-title":"Validation of the g.tec unicorn hybrid black wireless EEG system","volume":"60","author":"Pontifex","year":"2023","journal-title":"Psychophysiology"},{"key":"B35","doi-asserted-by":"publisher","first-page":"515","DOI":"10.1017\/S0140525X00076512","article-title":"Does the chimpanzee have a theory of mind","volume":"1","author":"Premack","year":"1978","journal-title":"Behav. Brain Sci."},{"key":"B36","doi-asserted-by":"crossref","first-page":"1967","DOI":"10.1109\/SMC53654.2022.9945182","article-title":"\u201cA passive brain-computer interface for monitoring engagement during robot-assisted language learning,\u201d","volume-title":"2022 IEEE International Conference on Systems, Man, and Cybernetics (SMC)","author":"Prinsen","year":"2022"},{"key":"B37","doi-asserted-by":"publisher","first-page":"104696","DOI":"10.1016\/j.compbiomed.2021.104696","article-title":"Recognition of human emotions using EEG signals: a review","volume":"136","author":"Rahman","year":"2021","journal-title":"Comput. Biol. Med."},{"key":"B38","doi-asserted-by":"publisher","first-page":"1857","DOI":"10.1149\/10701.1857ecst","article-title":"Analysis and classification of stress among students using EEG as biomarker","volume":"107","author":"Rajendran","year":"2022","journal-title":"ECS Trans."},{"key":"B39","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1145\/2157689.2157764","article-title":"\u201cHow to use non-linguistic utterances to convey emotion in child-robot interaction,\u201d","volume-title":"HRI","author":"Read","year":"2012"},{"key":"B40","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1109\/ICICIP.2016.7885895","article-title":"\u201cEfficient feature extraction framework for EEG signals classification,\u201d","volume-title":"2016 Seventh International Conference on Intelligent Control and Information Processing (ICICIP)","author":"Ren","year":"2016"},{"key":"B41","doi-asserted-by":"crossref","first-page":"1161","DOI":"10.1037\/h0077714","article-title":"A circumplex model of affect","volume":"39","author":"Russell","year":"1980","journal-title":"J. Pers. Soc. Psychol."},{"key":"B42","doi-asserted-by":"publisher","first-page":"532279","DOI":"10.3389\/frobt.2020.532279","article-title":"Emotion recognition for human-robot interaction: recent advances and future perspectives","volume":"7","author":"Spezialetti","year":"2020","journal-title":"Front. Robot. AI"},{"key":"B43","first-page":"285","article-title":"\u201cEEG-based machine learning models for emotion recognition in HRI,\u201d","volume-title":"International Conference on Human-Computer Interaction","author":"Staffa","year":"2023"},{"key":"B44","doi-asserted-by":"publisher","first-page":"749","DOI":"10.1007\/s12369-019-00576-1","article-title":"A wisard network approach for a BCI-based robotic prosthetic control","volume":"12","author":"Staffa","year":"2019","journal-title":"Int. J. Soc. Robot"},{"key":"B45","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-47437-3_20","article-title":"\u201cRecommender interfaces: the more human-like, the more humans like,\u201d","volume-title":"Social Robotics. ICSR 2016","author":"Staffa","year":"2016"},{"key":"B46","first-page":"884","article-title":"\u201cEnhancing affective robotics via human internal state monitoring,\u201d","author":"Staffa","year":"2022","journal-title":"2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN)"},{"key":"B47","doi-asserted-by":"publisher","first-page":"8875426","DOI":"10.1155\/2020\/8875426","article-title":"EEG-based emotion recognition: a state-of-the-art review of current trends and opportunities","volume":"2020","author":"Suhaimi","year":"2020","journal-title":"Intell. Neurosci."},{"key":"B48","doi-asserted-by":"publisher","DOI":"10.3390\/s21144626","article-title":"Emotion-driven analysis and control of human-robot interactions in collaborative applications","author":"Toichoa Eyam","year":"2021","journal-title":"Sensors"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.3390\/s20185083","article-title":"EEG-based BCI emotion recognition: a survey","author":"Torres","year":"2020","journal-title":"Sensors"},{"key":"B50","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3457188","article-title":"Explainable embodied agents through social cues: a review","volume":"10","author":"Wallk\u00f6tter","year":"2021","journal-title":"J. Hum. Robot Interact."},{"key":"B51","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1016\/j.cogr.2021.04.001","article-title":"Review of the emotional feature extraction and classification using EEG signals","volume":"1","author":"Wang","year":"2021","journal-title":"Cogn. Robot."},{"key":"B52","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1111\/j.1469-8986.1993.tb03207.x","article-title":"Frontal brain asymmetry and emotional reactivity: a biological substrate of affective style","volume":"30","author":"Wheeler","year":"1993","journal-title":"Psychophysiology"},{"key":"B53","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-31413-6_18","volume-title":"Reflections on the Design Challenges Prompted by Affect-Aware Socially Assistive Robots","author":"Wilson","year":"2016"},{"key":"B54","first-page":"1127","article-title":"\u201cEstimation of valence of emotion using two frontal EEG channels,\u201d","volume-title":"BIBM","author":"Wu","year":"2017"},{"key":"B55","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1016\/j.inffus.2020.01.011","article-title":"Emotion recognition using multi-modal data and machine learning techniques: a tutorial and review","volume":"59","author":"Zhang","year":"2020","journal-title":"Inform. Fusion"},{"key":"B56","doi-asserted-by":"publisher","first-page":"225","DOI":"10.3389\/fnbeh.2018.00225","article-title":"Frontal EEG asymmetry and middle line power difference in discrete emotions","volume":"12","author":"Zhao","year":"2018","journal-title":"Front. Behav. Neurosci."},{"key":"B57","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abe2987","article-title":"Understanding robots","author":"Ziemke","year":"2020","journal-title":"Sci. Robot."}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1191127\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,10]],"date-time":"2023-10-10T04:56:30Z","timestamp":1696913790000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1191127\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,10]]},"references-count":57,"alternative-id":["10.3389\/fnbot.2023.1191127"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1191127","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,10]]},"article-number":"1191127"}}