{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T05:05:48Z","timestamp":1787029548670,"version":"3.56.0"},"reference-count":45,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T00:00:00Z","timestamp":1557705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100017664","name":"Foundation for University Key Teacher by Henan Province","doi-asserted-by":"publisher","award":["2015GGJS068"],"award-info":[{"award-number":["2015GGJS068"]}],"id":[{"id":"10.13039\/501100017664","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Fundamental Research Funds for the Universities of Henan Province","award":["NSFRF1616"],"award-info":[{"award-number":["NSFRF1616"]}]},{"name":"Key Scientific Research Projects of Universities in Henan","award":["19A520004"],"award-info":[{"award-number":["19A520004"]}]},{"name":"Foundation for Scientific and Technological Project of Henan Province","award":["172102210279"],"award-info":[{"award-number":["172102210279"]}]},{"name":"National Nature Science Foundation of China","award":["61502150 and 61403128"],"award-info":[{"award-number":["61502150 and 61403128"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Emotion recognition based on multi-channel electroencephalograph (EEG) signals is becoming increasingly attractive. However, the conventional methods ignore the spatial characteristics of EEG signals, which also contain salient information related to emotion states. In this paper, a deep learning framework based on a multiband feature matrix (MFM) and a capsule network (CapsNet) is proposed. In the framework, the frequency domain, spatial characteristics, and frequency band characteristics of the multi-channel EEG signals are combined to construct the MFM. Then, the CapsNet model is introduced to recognize emotion states according to the input MFM. Experiments conducted on the dataset for emotion analysis using EEG, physiological, and video signals (DEAP) indicate that the proposed method outperforms most of the common models. The experimental results demonstrate that the three characteristics contained in the MFM were complementary and the capsule network was more suitable for mining and utilizing the three correlation characteristics.<\/jats:p>","DOI":"10.3390\/s19092212","type":"journal-article","created":{"date-parts":[[2019,5,13]],"date-time":"2019-05-13T11:00:57Z","timestamp":1557745257000},"page":"2212","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":265,"title":["Emotion Recognition from Multiband EEG Signals Using CapsNet"],"prefix":"10.3390","volume":"19","author":[{"given":"Hao","family":"Chao","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo 454000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9708-5391","authenticated-orcid":false,"given":"Liang","family":"Dong","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo 454000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yongli","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo 454000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Baoyun","family":"Lu","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Henan Polytechnic University, Jiaozuo 454000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,5,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1038\/372669a0","article-title":"Impaired recognition of emotion in facial expressions following bilateral damage to the human amygdala","volume":"372","author":"Adolphs","year":"1994","journal-title":"Nature"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.heares.2014.10.003","article-title":"Voice emotion recognition by cochlear-implanted children and their normally-hearing peers","volume":"322","author":"Chatterjee","year":"2015","journal-title":"Hearing Res."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ross, P.D., Polson, L., and Grosbras, M.H. (2012). Developmental Changes in Emotion Recognition from Full-Light and Point-Light Displays of Body Movement. Plos One.","DOI":"10.1371\/journal.pone.0044815"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1016\/j.cmpb.2016.12.005","article-title":"Recognition of emotions using multimodal physiological signals and an ensemble deep learning model","volume":"140","author":"Zhong","year":"2017","journal-title":"Comput. Meth. Prog. Bio."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Abadi, M.K., Kia, M., Subramanian, R., Avesani, P., and Sebe, N. (2013, January 22\u201326). Decoding affect in videos employing the MEG brain signal. Proceedings of the 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), Shanghai, China.","DOI":"10.1109\/FG.2013.6553809"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"9750904","DOI":"10.1155\/2018\/9750904","article-title":"Recognition of Emotions Using Multichannel EEG Data and DBN-GC-Based Ensemble Deep Learning Framework","volume":"2018","author":"Chao","year":"2018","journal-title":"Comput. Intel. Neurosc."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Li, Y., Huang, J., Zhou, H., Zhou, H.Y., and Zhong, N. (2017). Human Emotion Recognition with Electroencephalographic Multidimensional Features by Hybrid Deep Neural Networks. Appl. Sci., 7.","DOI":"10.3390\/app7101060"},{"key":"ref_8","unstructured":"Zheng, W.L., Dong, B.N., and Lu, B.L. (2014, January 26\u201330). Multimodal emotion recognition using EEG and eye tracking data. Proceedings of the 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Chicago, IL, USA."},{"key":"ref_9","unstructured":"Li, X., Zhang, P., Song, D.W., Yu, G.L., Hou, Y.X., and Hu, B. (2015, January 13). EEG Based Emotion Identification Using Unsupervised Deep Feature Learning. Proceedings of the SIGIR2015 Workshop on Neuro-Physiological Methods in IR Research, Santiago, Chile."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Bilalpur, M., Kia, S.M., Chawla, M., Chua, T.-S., and Subramanian, R. (2017, January 13\u201317). Gender and emotion recognition with implicit user signals. Proceedings of the 19th ACM International Conference on Multimodal Interaction, Glasgow, Scotland.","DOI":"10.1145\/3136755.3136790"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zheng, W.L., Zhu, J.Y., Yong, P., and Lu, B.L. (2014, January 14\u201318). EEG-based emotion classification using deep belief networks. Proceedings of the 2014 IEEE International Conference on Multimedia and Expo (ICME), Chengdu, China.","DOI":"10.1109\/ICME.2014.6890166"},{"key":"ref_12","unstructured":"Liu, W., Zheng, W.L., and Lu, B.L. (2016, September 30). Multimodal Emotion Recognition Using Multimodal Deep Learning. Available online: https:\/\/arxiv.org\/abs\/1602.08225."},{"key":"ref_13","first-page":"101","article-title":"Real-Time Fractal-Based Valence Level Recognition from EEG","volume":"7848","author":"Liu","year":"2013","journal-title":"IEEE Trans. Serv. Comput"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"390","DOI":"10.4236\/jbise.2010.34054","article-title":"Classification of human emotion from EEG using discrete wavelet transform","volume":"3","author":"Murugappan","year":"2010","journal-title":"JBiSE."},{"key":"ref_15","first-page":"1","article-title":"Classification of human emotions from EEG signals using statistical features and neural network","volume":"1","author":"Chai","year":"2009","journal-title":"Int. J. Eng. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1109\/TITB.2010.2041553","article-title":"Toward emotion aware computing: An integrated approach using multichannel neurophysiological recordings and affective visual stimuli","volume":"4","author":"Frantzidis","year":"2010","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"306","DOI":"10.1016\/0013-4694(70)90143-4","article-title":"EEG analysis based on time domain properties","volume":"29","author":"Hjorth","year":"1970","journal-title":"Electroencephalogr. Clin. Neurophysiol."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kroupi, E., Yazdani, A., and Ebrahimi, T. (2011, January 9\u201312). EEG correlates of different emotional states elicited during watching music videos. Proceedings of the 4th International Conference on Affective Computing and Intelligent Interaction, Memphis, TN, USA.","DOI":"10.1007\/978-3-642-24571-8_58"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1109\/TITB.2009.2034649","article-title":"Emotion recognition from EEG using higher order crossings","volume":"14","author":"Petrantonakis","year":"2010","journal-title":"IEEE Trans. Inf. Technol. Biomed."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1109\/T-AFFC.2010.7","article-title":"Emotion recognition from brain signals using hybrid adaptive filtering and higher order crossings analysis","volume":"1","author":"Petrantonakis","year":"2010","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1109\/TAFFC.2014.2339834","article-title":"Feature Extraction and Selection for Emotion Recognition from EEG","volume":"5","author":"Jenke","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"19","DOI":"10.3389\/fnbot.2017.00019","article-title":"Cross-Subject EEG Feature Selection for Emotion Recognition Using Transfer Recursive Feature Elimination","volume":"11","author":"Yin","year":"2017","journal-title":"Front Neurorobot."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Kwon, Y.H., Shin, S.B., and Kim, S.D. (2018). Electroencephalography Based Fusion Two-Dimensional (2D)-Convolution Neural Networks (CNN) Model for Emotion Recognition System. Sensors, 18.","DOI":"10.3390\/s18051383"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.1109\/TBME.2010.2048568","article-title":"EEG-based emotion recognition in music listening","volume":"57","author":"Lin","year":"2010","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1109\/TAFFC.2017.2660485","article-title":"Real-time movie-induced discrete emotion recognition from EEG signals","volume":"9","author":"Liu","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1234","DOI":"10.1587\/transinf.2015EDP7251","article-title":"Continuous music-emotion recognition based on electroencephalogram","volume":"99","author":"Thammasan","year":"2016","journal-title":"IEICE Trans. Inf. Syst."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Rozgic, V., Vitaladevuni, S.N., and Prasad, R. (2013, January 26\u201331). Robust EEG emotion classification using segment level decision fusion. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6637858"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wichakam, I., and Vateekul, P. (2014, January 14\u201316). An evaluation of feature extraction in EEG-based emotion prediction with support vector machines. Proceedings of the 2014 11th international Joint Conference on Computer Science and Software Engineering, Chon Buri, Thailand.","DOI":"10.1109\/JCSSE.2014.6841851"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1185","DOI":"10.3233\/BME-130919","article-title":"Emotion recognition based on the sample entropy of EEG","volume":"24","author":"Jie","year":"2014","journal-title":"Bio-med. Mater. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"509","DOI":"10.3233\/THC-174836","article-title":"Emotion recognition from multichannel EEG signals using K-nearest neighbor classification","volume":"26","author":"Li","year":"2018","journal-title":"Technol. Health Care"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Sohaib, A.T., Qureshi, S., Hagelb\u00e4ck, J., Hilborn, O., and Jerica, P. (2013, January 21\u201326). Evaluating classifiers for emotion recognition using EEG. Proceedings of the International conference on augmented cognition, Berlin, Heidelberg.","DOI":"10.1007\/978-3-642-39454-6_53"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"627892","DOI":"10.1155\/2014\/627892","article-title":"EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation","volume":"2014","author":"Jirayucharoensak","year":"2014","journal-title":"The Scientific World J."},{"key":"ref_33","unstructured":"Liu, W., Zheng, W.L., and Lu, B.L. (2016, January 16-21). Emotion Recognition Using Multimodal Deep Learning. Proceedings of the 23rd International Conference on Neural Information Processing, Kyoto, Japan."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Yang, B., Han, X., and Tang, J. (2017, January 14\u201316). Three class emotions recognition based on deep learning using staked autoencoder. Proceedings of the 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Shanghai, China.","DOI":"10.1109\/CISP-BMEI.2017.8302098"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/T-AFFC.2011.15","article-title":"DEAP: A database for emotion analysis; using physiological signals","volume":"3","author":"Koelstra","year":"2012","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"2159","DOI":"10.1007\/s11042-015-3119-y","article-title":"Affect representation and recognition in 3D continuous valence\u2013arousal\u2013dominance space","volume":"76","author":"Verma","year":"2017","journal-title":"Multimed. Tools Appl."},{"key":"ref_37","first-page":"63","article-title":"SAM: The Self-Assessment Manikin an Efficient Cross-Cultural Measurement of Emotional Response","volume":"35","author":"Morris","year":"1995","journal-title":"Advertising Research."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"6057","DOI":"10.1016\/j.eswa.2014.03.050","article-title":"Classification of emotions induced by music videos and correlation with participants\u2019 rating","volume":"41","author":"Naser","year":"2014","journal-title":"Expert Syst. Appli."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Naser, D.S., and Saha, G. (2013, January 28\u201330). Recognition of emotions induced by music videos using DT-CWPT. Proceedings of the IEEE Indian Conference on Medical Informatics and Telemedicine (ICMIT), Kharagpur, India.","DOI":"10.1109\/IndianCMIT.2013.6529408"},{"key":"ref_40","unstructured":"Chung, S.Y., and Yoon, H.J. (2012, January 17\u201321). Affective classification using Bayesian classifier and supervised learning. Proceedings of the 2012 12th International Conference on Control, Automation and Systems, Je Ju Island, Korea."},{"key":"ref_41","first-page":"505","article-title":"Modeling Physiological Data with Deep Belief Networks","volume":"3","author":"Wang","year":"2013","journal-title":"Int. J. Inf. Educ. Technol."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhuang, N., Zeng, Y., Tong, L., Zhang, C., Zhang, H., and Yan, B. (2017). Emotion Recognition from EEG Signals Using Multidimensional Information in EMD Domain. BioMed Res. Int., 2017.","DOI":"10.1155\/2017\/8317357"},{"key":"ref_43","unstructured":"Sabour, S., Frosst, N., and Hinton, G.E. (2017, January 4\u20139). Dynamic routing between capsules. Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Long Beach, CA, USA."},{"key":"ref_44","unstructured":"Mobiny, A., and Van, N.H. (2018, June 19). Fast CapsNet for Lung Cancer Screening. Available online: https:\/\/arxiv.org\/abs\/1806.07416."},{"key":"ref_45","first-page":"2825","article-title":"Scikit-learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2012","journal-title":"J. Mach. Learn. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/9\/2212\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:51:35Z","timestamp":1760187095000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/9\/2212"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,13]]},"references-count":45,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2019,5]]}},"alternative-id":["s19092212"],"URL":"https:\/\/doi.org\/10.3390\/s19092212","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5,13]]}}}