{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,29]],"date-time":"2026-03-29T21:07:09Z","timestamp":1774818429911,"version":"3.50.1"},"reference-count":69,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,1,6]],"date-time":"2020-01-06T00:00:00Z","timestamp":1578268800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In order to develop more precise and functional affective applications, it is necessary to achieve a balance between the psychology and the engineering applied to emotions. Signals from the central and peripheral nervous systems have been used for emotion recognition purposes, however, their operation and the relationship between them remains unknown. In this context, in the present work, we have tried to approach the study of the psychobiology of both systems in order to generate a computational model for the recognition of emotions in the dimension of valence. To this end, the electroencephalography (EEG) signal, electrocardiography (ECG) signal and skin temperature of 24 subjects have been studied. Each methodology has been evaluated individually, finding characteristic patterns of positive and negative emotions in each of them. After feature selection of each methodology, the results of the classification showed that, although the classification of emotions is possible at both central and peripheral levels, the multimodal approach did not improve the results obtained through the EEG alone. In addition, differences have been observed between cerebral and peripheral responses in the processing of emotions by separating the sample by sex; though, the differences between men and women were only notable at the peripheral nervous system level.<\/jats:p>","DOI":"10.3390\/s20010313","type":"journal-article","created":{"date-parts":[[2020,1,6]],"date-time":"2020-01-06T10:34:46Z","timestamp":1578306886000},"page":"313","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Brain and Body Emotional Responses: Multimodal Approximation for Valence Classification"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8726-4847","authenticated-orcid":false,"given":"Jennifer","family":"Sorinas","sequence":"first","affiliation":[{"name":"The Institute of Bioengineering, University Miguel Hernandez, 03202 Elche, Spain"},{"name":"Department of Electronics and Computer Technology, University of Cartagena, 30202 Cartagena, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4613-6101","authenticated-orcid":false,"given":"Jose Manuel","family":"Ferr\u00e1ndez","sequence":"additional","affiliation":[{"name":"Department of Electronics and Computer Technology, University of Cartagena, 30202 Cartagena, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7052-6011","authenticated-orcid":false,"given":"Eduardo","family":"Fernandez","sequence":"additional","affiliation":[{"name":"The Institute of Bioengineering, University Miguel Hernandez, 03202 Elche, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1007\/BF00993889","article-title":"A categorized list of motivation definitions, with a suggestion for a consensual definition","volume":"5","author":"Kleinginna","year":"1981","journal-title":"Motiv. Emot."},{"key":"ref_2","unstructured":"Darwin, C., and Prodger, P. (1988). The Expression of the Emotions in Man and Animals, Oxford University Press."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1161","DOI":"10.1037\/h0077714","article-title":"A circumplex model of affect","volume":"39","author":"Russell","year":"1980","journal-title":"J. Personal. Soc. Psychol."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wundt, W. (1894). Lectures on Human and Animal Psychology, Swan Sonnenschein & Co.","DOI":"10.1037\/12937-000"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"330","DOI":"10.1037\/0022-3514.58.2.330","article-title":"Approach-withdrawal and cerebral asymmetry: Emotional expression and brain physiology: I","volume":"58","author":"Davidson","year":"1990","journal-title":"J. Personal. Soc. Psychol."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Picard, R.W. (1997). Affective Computing, The MIT Press.","DOI":"10.1037\/e526112012-054"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/T-AFFC.2010.1","article-title":"Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications","volume":"1","author":"Calvo","year":"2010","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/j.biopsycho.2013.04.010","article-title":"Real-time fMRI brain computer interfaces: Self-regulation of single brain regions to networks","volume":"95","author":"Ruiz","year":"2014","journal-title":"Biol. Psychol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"209","DOI":"10.1080\/02699930802204677","article-title":"Measures of emotion: A review","volume":"23","author":"Mauss","year":"2009","journal-title":"Cogn. Emot."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"676","DOI":"10.1037\/0022-3514.62.4.676","article-title":"Individual Differences in Anterior Brain Asymmetry and Fundamental Dimensions of Emotion","volume":"62","author":"Tomarken","year":"1992","journal-title":"J. Personal. Soc. Psychol."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"951","DOI":"10.1111\/psyp.12243","article-title":"Thermal infrared imaging in psychophysiology: Potentialities and limits","volume":"51","author":"Ioannou","year":"2014","journal-title":"Psychophysiology"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/S0278-2626(03)00011-3","article-title":"Central and autonomic nervous system integration in emotion","volume":"52","author":"Hagemann","year":"2003","journal-title":"Brain Cogn."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1111\/j.1469-8986.1993.tb03352.x","article-title":"Looking at picture: Affective, facial, visceral, and behavioral reactions","volume":"30","author":"Lang","year":"1993","journal-title":"Psychophysiology"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"394","DOI":"10.1016\/j.biopsycho.2010.03.010","article-title":"Autonomic nervous system activity in emotion: A review","volume":"84","author":"Kreibig","year":"2010","journal-title":"Biol. Psychol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"348","DOI":"10.1196\/annals.1280.016","article-title":"Blood, Sweat, and Fears the Autonomic Architecture of Emotion","volume":"1000","author":"Levenson","year":"2003","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1093\/mind\/os-IX.34.188","article-title":"What is an Emotion?","volume":"9","author":"James","year":"1884","journal-title":"Mind"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1037\/h0046234","article-title":"Cognitive, social, and physiological determinants of emotional state","volume":"69","author":"Schachter","year":"1962","journal-title":"Psychol. Rev."},{"key":"ref_18","first-page":"207","article-title":"Universals and cultural differences in facial expressions of emotion","volume":"19","author":"Eckman","year":"1972","journal-title":"Neb. Symp. Motiv."},{"key":"ref_19","unstructured":"Wagner, H.L. (1988). Emotion and the autonomic nervous system: A prospectus for research on autonomic specificity. Social Psychophysiology and Emotion: Theory and Clinical Applications, John Wiley & Sons Ltd."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1016\/j.bj.2017.11.001","article-title":"An accurate emotion recognition system using ECG and GSR signals and matching pursuit method","volume":"40","author":"Goshvarpour","year":"2017","journal-title":"Biomed. J."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Valderas, T., Bolea, J., Laguna, P., and Ieee, S.M. (2015, January 25\u201329). Human Emotion Recognition Using Heart Rate Variability Analysis with Spectral Bands Based on Respiration. Proceedings of the 2015 37th IEEE Engineering in Medicine and Biology Society, Milan, Italy.","DOI":"10.1109\/EMBC.2015.7319792"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1007\/BF00999346","article-title":"Relationship of Skin Temperature Changes to the Emotions Accompanying Music","volume":"10","author":"McFarland","year":"1985","journal-title":"Biofeedback Self-Regul."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1007\/BF02251007","article-title":"The Psychological Significance of Changes in Skin Temperature","volume":"20","author":"Kagan","year":"1996","journal-title":"Motiv. Emot."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wu, G., Liu, G., and Hao, M. (2010, January 28\u201329). The analysis of emotion recognition from GSR based on PSO. Proceedings of the 2010 International Symposium on Intelligence Information Processing and Trusted Computing, Huanggang, China.","DOI":"10.1109\/IPTC.2010.60"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Torres, C.A., Orozco, A., and Alvarez, M.A. (2013, January 3\u20137). Feature Selection for Multimodal Emotion Recognition in the Arousal-Valence Space. Proceedings of the 35th Annual International Conference of the IEEE EMBS, Osaka, Japan.","DOI":"10.1109\/EMBC.2013.6610504"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1016\/j.imavis.2012.10.002","article-title":"Fusion of facial expressions and EEG for implicit affective tagging","volume":"31","author":"Koelstra","year":"2013","journal-title":"Image Vis. Comput."},{"key":"ref_27","unstructured":"Li, L., and Chen, J.H. (December, January 29). Emotion recognition using physiological signals. Proceedings of the International Conference on Artificial Reality and Telexistence, Hangzhou, China."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Sorinas, J., Grima, M.D., and Ferrandez, J.M. (2018). Identifying Suitable Brain Regions and Trial Size Segmentation for Positive\/Negative Emotion Recognition. Int. J. Neural Syst., 29.","DOI":"10.1142\/S0129065718500442"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Sorinas, J., Fernandez-Troyano, J.C., Calvo, M.V., Ferrandez, J.M., and Fernandez, E. (2019). A new model for the implementation of positive and negative emotion recognition. arXiv.","DOI":"10.1101\/638205"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1080\/00029238.1985.11080163","article-title":"Ten percent electrode system for topographic studies of spontaneous and evoked EEG activities","volume":"25","author":"Chatrian","year":"1985","journal-title":"Am. J. EEG Technol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"536","DOI":"10.1016\/S1388-2457(00)00533-2","article-title":"Scalp electrode impedance, infection risk, and EEG data quality","volume":"112","author":"Ferree","year":"2001","journal-title":"Clin. Neurophysiol."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.jneumeth.2003.10.009","article-title":"EEGLAB: An Open Source Toolbox for Analysis of Sin-gle-trail EEG Dynamics Including Independent Compo-nent Anlaysis","volume":"134","author":"Delorme","year":"2004","journal-title":"J. Neurosci. Methods"},{"key":"ref_33","first-page":"894","article-title":"Extended ICA removes artifacts from electroencephalographic recordings","volume":"10","author":"Jung","year":"1998","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"71","DOI":"10.3109\/00207458808985694","article-title":"EEG Measures of Cerebral Asymmetry: Conceptual and Methodological Issues","volume":"39","author":"Davidson","year":"1988","journal-title":"Int. J. Neurosci."},{"key":"ref_35","unstructured":"Hoboken, N.J. (1999). Nonparametric Statistical Methods, John Wiley & Sons Inc."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1161","DOI":"10.3758\/s13428-011-0107-7","article-title":"ARTiiFACT: A tool for heart rate artifact processing and heart rate variability analysis","volume":"43","author":"Kaufmann","year":"2011","journal-title":"Behav. Res. Methods"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Khandoker, A.H., Karmakar, C., Brennan, M., Voss, A., and Palaniswami, M. (2013). Quantitative Poincar\u00e9 Plot. Poincar\u00e9 Plot Methods for Heart Rate Variability Analysis, Springer.","DOI":"10.1007\/978-1-4614-7375-6"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1016\/j.autneu.2013.05.004","article-title":"Poincar\u00e9 plot indexes of heart rate variability: Relationships with other nonlinear variables","volume":"177","author":"Hoshi","year":"2013","journal-title":"Auton. Neurosci. Basic Clin."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"704","DOI":"10.1016\/0002-8703(92)90510-3","article-title":"Patterns of beat-to-beat heart rate variability in advanced heart failure","volume":"123","author":"Woo","year":"1992","journal-title":"Am. Heart J."},{"key":"ref_40","first-page":"244","article-title":"Quantitative beat-to-beat analysis of heart rate dynamics during exercise","volume":"271","author":"Tulppo","year":"1996","journal-title":"Am. J. Physiol."},{"key":"ref_41","unstructured":"Task Force of the European Society of Cardiology and the North American Society of Pacing and Electrophysiology (1996). Heart rate variability: Standards of measurement, physiological interpretation, and clinical use. Eur. Heart J., 17, 354\u2013381."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"482","DOI":"10.1161\/01.CIR.84.2.482","article-title":"Cardiovascular neural regulation explored in the frequency domain","volume":"84","author":"Malliani","year":"1991","journal-title":"Circulation"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1111\/psyp.12027","article-title":"The utility of low frequency heart rate variability as an index of sympathetic cardiac tone: A review with emphasis on a reanalysis of previous studies","volume":"50","author":"Langewitz","year":"2013","journal-title":"Psychophysiology"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"3224","DOI":"10.1161\/01.CIR.96.9.3224","article-title":"Sympathovagal balance: A critical appraisal","volume":"96","author":"Eckberg","year":"1997","journal-title":"Circulation"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1007\/s00421-003-0917-0","article-title":"Quantitative Poincare plot analysis of heart rate variability: Effect of endurance training","volume":"91","author":"Mourot","year":"2004","journal-title":"Eur. J. Appl. Physiol."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1212","DOI":"10.1080\/02699930701438020","article-title":"On the interdependence of cognition and emotion","volume":"21","author":"Storbeck","year":"2007","journal-title":"Cogn. Emot."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1016\/j.neuroimage.2008.03.059","article-title":"Functional grouping and cortical\u2014Subcortical interactions in emotion: A meta-analysis of neuroimaging studies","volume":"42","author":"Kober","year":"2008","journal-title":"Neuroimage"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1080\/02699930126048","article-title":"Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions valence and intensity of musical emotions","volume":"15","author":"Schmidt","year":"2015","journal-title":"Cogn. Emot."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"207","DOI":"10.3758\/CABN.3.3.207","article-title":"Functional neuroanatomy of emotions: A meta-analysis","volume":"3","author":"Murphy","year":"2003","journal-title":"Cogn. Affect. Behav. Neurosci."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1016\/j.biopsycho.2004.03.002","article-title":"Frontal EEG asymmetry as a moderator and mediator of emotion","volume":"67","author":"Coan","year":"2004","journal-title":"Biol. Psychol."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1007\/s11517-011-0747-x","article-title":"Spectral EEG frontal asymmetries correlate with the experienced pleasantness of TV commercial advertisements","volume":"49","author":"Vecchiato","year":"2011","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_52","unstructured":"Nie, D., Wang, X., Shi, L., and Member, B.L.S. (May, January 27). EEG-based emotion recognition during watching movies. Proceedings of the 5th International IEEE EMBS Conference on Neural Engineering, Cancun, Mexico."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1913","DOI":"10.1016\/S1388-2457(99)00151-0","article-title":"Processing of affective pictures modulates right-hemispheric gamma band EEG activity","volume":"110","author":"Muller","year":"1999","journal-title":"Clin. Neurophysiol."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"86","DOI":"10.3949\/ccjm.76.s2.17","article-title":"The polyvagal theory: New insights into adaptive reactions of the autonomic nervous system","volume":"76","author":"Porges","year":"2009","journal-title":"Clevel. Clin. J. Med."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Guo, H., and Huang, Y. (November, January 31). Heart Rate Variability Signal Features for Emotion Recognition by using Principal Component Analysis and Support Vectors Machine. Proceedings of the 2016 IEEE 16th International Conference on Bioinformatics and Bioengineering, Taichung, Taiwan.","DOI":"10.1109\/BIBE.2016.40"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1089","DOI":"10.1016\/S0002-9149(99)80309-9","article-title":"The effects of emotions on short-term power spectrum analysis of heart rate variability","volume":"76","author":"McCraty","year":"1995","journal-title":"Am. J. Cardiol."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"753","DOI":"10.1589\/jpts.25.753","article-title":"Frequency Band Analysis of Electrocardiogram (ECG) Signals for Human Emotional State Classification Using Discrete Wavelet Transform(DWT)","volume":"25","author":"Murugappan","year":"2013","journal-title":"J. Phys. Ther. Sci."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1016\/S0735-1097(97)00554-8","article-title":"Twenty-Four Hour Time Domain Heart Rate Variability and Heart Rate: Relations to Age and Gender Over Nine Decades","volume":"31","author":"Umetani","year":"1998","journal-title":"J. Am. Coll. Cardiol."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Moodithaya, S., and Avadhany, S.T. (2012). Gender Differences in Age-Related Changes in Cardiac Autonomic Nervous Function. J. Aging Res.","DOI":"10.1155\/2012\/679345"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1016\/j.ijpsycho.2006.05.011","article-title":"Sex differences in emotional and psychophysiological responses to musical stimuli","volume":"62","author":"Nater","year":"2006","journal-title":"Int. J. Psychophysiol."},{"key":"ref_61","first-page":"389","article-title":"Differences between men and women in their response to heat and cold","volume":"26","author":"Hardy","year":"1940","journal-title":"Physiology"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1139\/h01-026","article-title":"Gender differences in temperature and vascular characteristics during exercise recovery","volume":"26","author":"Marchand","year":"2001","journal-title":"Can. J. Appl. Physiol."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/j.neuroimage.2013.11.007","article-title":"Multimodal fusion framework: A multiresolution approach for emotion classification and recognition from physiological signals","volume":"102","author":"Verma","year":"2014","journal-title":"NeuroImage"},{"key":"ref_64","unstructured":"Chen, J., Hu, B., Xu, L., Moore, P., and Su, Y. (2015, January 9\u201312). Feature-level fusion of multimodal physiological signals for emotion recognition. Proceedings of the 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Washington, DC, USA."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"1020","DOI":"10.1166\/jmihi.2015.1490","article-title":"Subject-dependent and subject-independent emotion classification using unimodal and multimodal physiological signals","volume":"5","author":"Jatupaiboon","year":"2015","journal-title":"J. Med. Imaging Health Inform."},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/s12193-016-0222-y","article-title":"SVM-based feature selection methods for emotion recognition from multimodal data","volume":"11","year":"2017","journal-title":"J. Multimodal User Interfaces"},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1016\/j.ijhcs.2007.10.011","article-title":"Real-time classification of evoked emotions using facial feature tracking and physiological responses","volume":"66","author":"Bailenson","year":"2008","journal-title":"Int. J. Hum. Comput. Stud."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"1509","DOI":"10.1093\/bioinformatics\/bti171","article-title":"Optimal number of features as a function of sample size for various classification rules","volume":"21","author":"Hua","year":"2005","journal-title":"Bioinformatics"},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"838","DOI":"10.1111\/1469-8986.00121","article-title":"Clarifying the emotive functions of asymmetrical frontal cortical activity","volume":"40","year":"2003","journal-title":"Psychophysiology"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/313\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:18:57Z","timestamp":1760361537000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/1\/313"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,6]]},"references-count":69,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["s20010313"],"URL":"https:\/\/doi.org\/10.3390\/s20010313","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,1,6]]}}}