{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T19:06:51Z","timestamp":1778872011117,"version":"3.51.4"},"reference-count":31,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2022,4,20]],"date-time":"2022-04-20T00:00:00Z","timestamp":1650412800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Science &amp; Technology Department of Sichuan Province","award":["2020JDRC0007"],"award-info":[{"award-number":["2020JDRC0007"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In recent decades, emotion recognition has received considerable attention. As more enthusiasm has shifted to the physiological pattern, a wide range of elaborate physiological emotion data features come up and are combined with various classifying models to detect one\u2019s emotional states. To circumvent the labor of artificially designing features, we propose to acquire affective and robust representations automatically through the Stacked Denoising Autoencoder (SDA) architecture with unsupervised pre-training, followed by supervised fine-tuning. In this paper, we compare the performances of different features and models through three binary classification tasks based on the Valence-Arousal-Dominance (VAD) affection model. Decision fusion and feature fusion of electroencephalogram (EEG) and peripheral signals are performed on hand-engineered features; data-level fusion is performed on deep-learning methods. It turns out that the fusion data perform better than the two modalities. To take advantage of deep-learning algorithms, we augment the original data and feed it directly into our training model. We use two deep architectures and another generative stacked semi-supervised architecture as references for comparison to test the method\u2019s practical effects. The results reveal that our scheme slightly outperforms the other three deep feature extractors and surpasses the state-of-the-art of hand-engineered features.<\/jats:p>","DOI":"10.3390\/e24050577","type":"journal-article","created":{"date-parts":[[2022,4,21]],"date-time":"2022-04-21T01:55:51Z","timestamp":1650506151000},"page":"577","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Semi-Supervised Cross-Subject Emotion Recognition Based on Stacked Denoising Autoencoder Architecture Using a Fusion of Multi-Modal Physiological Signals"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8435-007X","authenticated-orcid":false,"given":"Junhai","family":"Luo","sequence":"first","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 610056, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuxin","family":"Tian","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 610056, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hang","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 610056, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 610056, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Man","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 610056, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,4,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Nogueiras, A., Moreno, A., Bonafonte, A., and Mari\u00f1o, J.B. (2001, January 3\u20137). Speech emotion recognition using hidden Markov models. Proceedings of the EUROSPEECH 2001-SCANDINAVIA-7th European Conference on Speech Communication and Technology, Aalborg, Denmark.","DOI":"10.21437\/Eurospeech.2001-627"},{"key":"ref_2","unstructured":"Kumar, N., Khaund, K., and Hazarika, S.M. (2022, March 18). Bispectral Analysis of EEG for Emotion Recognition. Available online: https:\/\/www.sciencedirect.com\/science\/article\/pii\/S187705091630076X?via%3Dihub."},{"key":"ref_3","unstructured":"Mayer, J.D. (2022, March 18). What is Emotional Intelligence?. Available online: https:\/\/scholars.unh.edu\/personality_lab\/8\/."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Picard, R.W. (2000). Affective Computing, MIT Press.","DOI":"10.1007\/978-3-540-45012-2_2"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1037\/0022-3514.53.4.712","article-title":"Universals and Cultural Differences in the Judgments of Facial Expressions of Emotion","volume":"53","author":"Ekman","year":"1987","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1175","DOI":"10.1109\/34.954607","article-title":"Toward machine emotional intelligence: Analysis of affective physiological state","volume":"23","author":"Picard","year":"2001","journal-title":"IEEE Trans. Pattern Anal."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1145\/2001269.2001295","article-title":"Unsupervised learning of hierarchical representations with convolutional deep belief networks","volume":"54","author":"Lee","year":"2011","journal-title":"Commun. ACM"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1109\/T-AFFC.2011.25","article-title":"A multimodal database for affect recognition and implicit tagging","volume":"3","author":"Soleymani","year":"2012","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/T-AFFC.2011.15","article-title":"DEAP: A database for emotion analysis; Using physiological signals","volume":"3","author":"Koelstra","year":"2012","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Garc\u00eda, H.F., \u00c1lvarez, M.A., and Orozco, \u00c1.A. (2016, January 16\u201320). Gaussian process dynamical models for multimodal affect recognition. Proceedings of the 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Orlando, FL, USA.","DOI":"10.1109\/EMBC.2016.7590834"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Tripathi, S., Acharya, S., Sharma, R.D., Mittal, S., and Bhattacharya, S. (2017, January 4\u20139). Using deep and convolutional neural networks for accurate emotion classification on DEAP dataset. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i2.19105"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"77905","DOI":"10.1109\/ACCESS.2019.2922047","article-title":"An Efficient Mixture Model Approach in Brain-Machine Interface Systems for Extracting the Psychological Status of Mentally Impaired Persons Using EEG Signals","volume":"7","author":"Murali","year":"2019","journal-title":"IEEE Access."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1109\/TAFFC.2017.2768030","article-title":"Emotion Recognition Based on High-Resolution EEG Recordings and Reconstructed Brain Sources","volume":"11","author":"Becker","year":"2020","journal-title":"IEEE T Affect Comput."},{"key":"ref_14","unstructured":"Izard, C.E. (2004). The Psychology of Emotions, Springer."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1016\/0013-4694(70)90143-4","article-title":"EEG analysis based on time domain properties","volume":"29","author":"Hjorth","year":"1970","journal-title":"Electroencephal. Clin. Neurophysiol."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kroupi, E., Yazdani, A., and Ebrahimi, T. (2011, January 9\u201312). Eeg correlates of different emotional states elicited during watching music videos. Proceedings of the International Conference on Affective Computing and Intelligent Interaction, Memphis, TN, USA.","DOI":"10.1007\/978-3-642-24571-8_58"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1016\/0167-2789(88)90081-4","article-title":"Approach to an irregular time series on the basis of the fractal theory","volume":"31","author":"Higuchi","year":"1988","journal-title":"Phys. D"},{"key":"ref_18","first-page":"327","article-title":"Feature extraction and selection for emotion recognition from EEG","volume":"5","author":"Jenke","year":"2014","journal-title":"IEEE Trans. Nucl. Sci."},{"key":"ref_19","unstructured":"Solomon, O. (1991). Psd Computations Using Welch\u2019s Method, NASA STI\/Recon Technical Report N."},{"key":"ref_20","unstructured":"Swami, A. (2021, June 05). Hosa-Higher Order Spectral Analysis Tool-Box. Available online: https:\/\/www.mathworks.com\/matlabcentral\/fileexchange\/3013-hosa-higher-order-spectral-analysis-toolbox."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"869","DOI":"10.1109\/PROC.1987.13824","article-title":"Bispectrum Estimation: A Digital Signal Processing Framework","volume":"75","author":"Nikias","year":"1987","journal-title":"Proc. IEEE"},{"key":"ref_22","first-page":"299","article-title":"EEG feature extraction for classifying emotions using FCM and FKM","volume":"1","author":"Murugappan","year":"2008","journal-title":"Int. J. Comput. Commun."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1080\/02699930126048","article-title":"Frontal brain electrical activity (eeg) distinguishes valence and intensity of musical emotions","volume":"15","author":"Schmidt","year":"2001","journal-title":"Cogn. Emot."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"903","DOI":"10.1098\/rspa.1998.0193","article-title":"The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis","volume":"454","author":"Huang","year":"1998","journal-title":"Proc. R. Soc. Lond. A"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1038\/nbt0308-303","article-title":"What is principal component analysis?","volume":"26","year":"2008","journal-title":"Nat Biotechnol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","article-title":"Representation learning: A review and new perspectives","volume":"35","author":"Bengio","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the dimensionality of data with neural networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Bengio, Y., Lamblin, P., Popovici, D., and Larochelle, H. (2007). Greedy layer-wise training of deep networks. Advances in Neural Information Processing Systems, MIT Press.","DOI":"10.7551\/mitpress\/7503.003.0024"},{"key":"ref_29","first-page":"3371","article-title":"Stacked denoising autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion","volume":"11","author":"Vincent","year":"2010","journal-title":"J. Mach. Learn Res."},{"key":"ref_30","first-page":"2825","article-title":"Scikit-learn: Machine learning in python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1527","DOI":"10.1162\/neco.2006.18.7.1527","article-title":"A fast learning algorithm for deep belief nets","volume":"18","author":"Hinton","year":"2006","journal-title":"Neural Comput."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/5\/577\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:57:25Z","timestamp":1760137045000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/5\/577"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,20]]},"references-count":31,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2022,5]]}},"alternative-id":["e24050577"],"URL":"https:\/\/doi.org\/10.3390\/e24050577","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,20]]}}}