{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T16:22:51Z","timestamp":1775665371932,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2021,7,27]],"date-time":"2021-07-27T00:00:00Z","timestamp":1627344000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT)","award":["NRF-2020R1A4A1019191"],"award-info":[{"award-number":["NRF-2020R1A4A1019191"]}]},{"name":"the Bio &amp; Medical Technology Development Program of the National Research Foundation (NRF)&amp; funded by the Korean government (MSIT)","award":["NRF-2019M3E5D1A02067961"],"award-info":[{"award-number":["NRF-2019M3E5D1A02067961"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Besides facial or gesture-based emotion recognition, Electroencephalogram (EEG) data have been drawing attention thanks to their capability in countering the effect of deceptive external expressions of humans, like faces or speeches. Emotion recognition based on EEG signals heavily relies on the features and their delineation, which requires the selection of feature categories converted from the raw signals and types of expressions that could display the intrinsic properties of an individual signal or a group of them. Moreover, the correlation or interaction among channels and frequency bands also contain crucial information for emotional state prediction, and it is commonly disregarded in conventional approaches. Therefore, in our method, the correlation between 32 channels and frequency bands were put into use to enhance the emotion prediction performance. The extracted features chosen from the time domain were arranged into feature-homogeneous matrices, with their positions following the corresponding electrodes placed on the scalp. Based on this 3D representation of EEG signals, the model must have the ability to learn the local and global patterns that describe the short and long-range relations of EEG channels, along with the embedded features. To deal with this problem, we proposed the 2D CNN with different kernel-size of convolutional layers assembled into a convolution block, combining features that were distributed in small and large regions. Ten-fold cross validation was conducted on the DEAP dataset to prove the effectiveness of our approach. We achieved the average accuracies of 98.27% and 98.36% for arousal and valence binary classification, respectively.<\/jats:p>","DOI":"10.3390\/s21155092","type":"journal-article","created":{"date-parts":[[2021,7,27]],"date-time":"2021-07-27T22:35:02Z","timestamp":1627425302000},"page":"5092","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":36,"title":["EEG-Based Emotion Recognition by Convolutional Neural Network with Multi-Scale Kernels"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2113-9904","authenticated-orcid":false,"given":"Tran-Dac-Thinh","family":"Phan","sequence":"first","affiliation":[{"name":"Department of Artificial Intelligence Convergence, Chonnam National University, 77 Yongbong-ro, Gwangju 500-757, Korea"}]},{"given":"Soo-Hyung","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence Convergence, Chonnam National University, 77 Yongbong-ro, Gwangju 500-757, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3024-5060","authenticated-orcid":false,"given":"Hyung-Jeong","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence Convergence, Chonnam National University, 77 Yongbong-ro, Gwangju 500-757, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8756-1382","authenticated-orcid":false,"given":"Guee-Sang","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Artificial Intelligence Convergence, Chonnam National University, 77 Yongbong-ro, Gwangju 500-757, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,27]]},"reference":[{"key":"ref_1","unstructured":"Li, S., and Deng, W. (2020). Deep facial expression recognition: A survey. IEEE Trans. Affect. Comput."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"102951","DOI":"10.1016\/j.dsp.2020.102951","article-title":"A survey of speech emotion recognition in natural environment","volume":"110","author":"Fahad","year":"2020","journal-title":"Digit. Signal Process."},{"key":"ref_3","unstructured":"Noroozi, F., Kaminska, D., Corneanu, C., Sapinski, T., Escalera, S., and Anbarjafari, G. (2018). Survey on emotional body gesture recognition. IEEE Trans. Affect. Comput."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1109\/TAFFC.2014.2327617","article-title":"Emotion recognition based on multi-variant correlation of physiological signals","volume":"5","author":"Wen","year":"2014","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Torres, E.P., Torres, E.A., Hern\u00e1ndez-\u00c1lvarez, M., and Yoo, S.G. (2020). EEG-based BCI emotion recognition: A survey. Sensors, 20.","DOI":"10.3390\/s20185083"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Liu, Y., and Sourina, O. (2013). Real-time fractal-based valence level recognition from EEG. Transactions on Computational Science XVIII, Springer.","DOI":"10.1007\/978-3-642-38803-3_6"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shaw, L., and Routray, A. (2016, January 29\u201331). Statistical features extraction for multivariate pattern analysis in meditation EEG using PCA. Proceedings of the 2016 IEEE EMBS International Student Conference (ISC), Ottawa, ON, Canada.","DOI":"10.1109\/EMBSISC.2016.7508624"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"37","DOI":"10.3389\/fnbot.2019.00037","article-title":"SAE+ LSTM: A New framework for emotion recognition from multi-channel EEG","volume":"13","author":"Xing","year":"2019","journal-title":"Front. Neurorobot."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1531","DOI":"10.1109\/JPROC.2015.2455028","article-title":"Tensor analysis and fusion of multimodal brain images","volume":"103","author":"Karahan","year":"2015","journal-title":"Proc. IEEE"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.eswa.2015.10.049","article-title":"Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers","volume":"47","author":"Atkinson","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"ref_11","first-page":"1","article-title":"Methods of EEG signal features extraction using linear analysis in frequency and time-frequency domains","volume":"2014","year":"2014","journal-title":"Int. Sch. Res. Not."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1109\/TCDS.2019.2949306","article-title":"Domain adaptation for EEG emotion recognition based on latent representation similarity","volume":"12","author":"Li","year":"2019","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2869","DOI":"10.1109\/TBME.2019.2897651","article-title":"EEG based emotion recognition by combining functional connectivity network and local activations","volume":"66","author":"Li","year":"2019","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"390","DOI":"10.4236\/jbise.2010.34054","article-title":"Classification of human emotion from EEG using discrete wavelet transform","volume":"3","author":"Murugappan","year":"2010","journal-title":"J. Biomed. Sci. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"274","DOI":"10.20965\/jaciii.2019.p0274","article-title":"A common spatial pattern and wavelet packet decomposition combined method for EEG-based emotion recognition","volume":"23","author":"Chen","year":"2019","journal-title":"J. Adv. Comput. Intell. Intell. Inform."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1016\/j.chb.2016.01.005","article-title":"Toward automatic detection of brain responses to emotional music through analysis of EEG effective connectivity","volume":"58","author":"Shahabi","year":"2016","journal-title":"Comput. Hum. Behav."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ackermann, P., Kohlschein, C., Bitsch, J.\u00c1., Wehrle, K., and Jeschke, S. (2016, January 14\u201317). EEG-based Automatic Emotion Recognition: Feature Extraction, Selection and Classification Methods. Proceedings of the 2016 IEEE 18th International Conference on E-Health Networking, Applications and Services (Healthcom), Munich, Germany.","DOI":"10.1109\/HealthCom.2016.7749447"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"1985","DOI":"10.1007\/s00521-015-2149-8","article-title":"Wavelet-based emotion recognition system using EEG signal","volume":"28","author":"Mohammadi","year":"2017","journal-title":"Neural Comput. Appl."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long short-term memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kwon, Y.H., Shin, S.B., and Kim, S.D. (2018). Electroencephalography based fusion two-dimensional (2D)-convolution neural networks (CNN) model for emotion recognition system. Sensors, 18.","DOI":"10.3390\/s18051383"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1186\/s40708-019-0100-y","article-title":"A EEG-based emotion recognition model with rhythm and time characteristics","volume":"6","author":"Yan","year":"2019","journal-title":"Brain Inform."},{"key":"ref_23","first-page":"329","article-title":"EEG-based emotion recognition using 3D convolutional neural networks","volume":"9","author":"Salama","year":"2018","journal-title":"Int. J. Adv. Comput. Sci. Appl"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Lee, Y.Y., and Hsieh, S. (2014). Classifying different emotional states by means of EEG-based functional connectivity patterns. PLoS ONE, 9.","DOI":"10.1371\/journal.pone.0095415"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"368","DOI":"10.1007\/s12559-017-9533-x","article-title":"Hierarchical convolutional neural networks for EEG-based emotion recognition","volume":"10","author":"Li","year":"2018","journal-title":"Cogn. Comput."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1109\/TAMD.2015.2431497","article-title":"Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks","volume":"7","author":"Zheng","year":"2015","journal-title":"IEEE Trans. Auton. Ment. Dev."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1109\/JBHI.2020.2995767","article-title":"Emotion recognition from multi-channel eeg via deep forest","volume":"25","author":"Cheng","year":"2020","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"106954","DOI":"10.1016\/j.asoc.2020.106954","article-title":"EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM","volume":"100","author":"Yin","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"532","DOI":"10.1109\/TAFFC.2018.2817622","article-title":"EEG emotion recognition using dynamical graph convolutional neural networks","volume":"11","author":"Song","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/T-AFFC.2011.15","article-title":"Deap: A database for emotion analysis; using physiological signals","volume":"3","author":"Koelstra","year":"2011","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1109\/TCDS.2016.2587290","article-title":"Multichannel EEG-based emotion recognition via group sparse canonical correlation analysis","volume":"9","author":"Zheng","year":"2016","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Rozgi\u2019c, V., Vitaladevuni, S.N., and Prasad, R. (2013, January 26\u201331). Robust EEG emotion classification using segment level decision fusion. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6637858"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhuang, X., Rozgi\u2019c, V., and Crystal, M. (2014, January 1\u20134). Compact unsupervised EEG response representation for emotion recognition. Proceedings of the IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Valencia, Spain.","DOI":"10.1109\/BHI.2014.6864469"},{"key":"ref_34","unstructured":"Pandey, P., and Seeja, K. (2019). Subject independent emotion recognition from EEG using VMD and deep learning. J. King Saud Univ. Comput. Inf. Sci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1109\/TCYB.2017.2788081","article-title":"Spatial\u2013temporal recurrent neural network for emotion recognition","volume":"49","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Cybern."},{"key":"ref_36","unstructured":"Chen, J., Hu, B., Xu, L., Moore, P., and Su, Y. (2015, January 9\u201312). Feature-level fusion of multimodal physiological signals for emotion recognition. Proceedings of the 2015 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), Washington, DC, USA."},{"key":"ref_37","unstructured":"Zhang, J., Chen, M., Hu, S., Cao, Y., and Kozma, R. (2016, January 9\u201312). PNN for EEG-based Emotion Recognition. Proceedings of the 2016 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Budapest, Hungary."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Kim, Y., and Choi, A. (2020). EEG-Based Emotion Classification Using Long Short-Term Memory Network with Attention Mechanism. Sensors, 20.","DOI":"10.3390\/s20236727"},{"key":"ref_39","unstructured":"Tao, W., Li, C., Song, R., Cheng, J., Liu, Y., Wan, F., and Chen, X. (2020). EEG-based Emotion Recognition Via Channel-Wise Attention and Self Attention. IEEE Trans. Affect. Comput."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"107626","DOI":"10.1016\/j.patcog.2020.107626","article-title":"A prototype-based SPD matrix network for domain adaptation EEG emotion recognition","volume":"110","author":"Wang","year":"2021","journal-title":"Pattern Recognit."},{"key":"ref_41","unstructured":"Du, X., Ma, C., Zhang, G., Li, J., Lai, Y.K., Zhao, G., Deng, X., Liu, Y.J., and Wang, H. (2020). An Efficient LSTM Network for Emotion Recognition from Multichannel EEG Signals. IEEE Trans. Affect. Comput."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/15\/5092\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:35:47Z","timestamp":1760164547000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/15\/5092"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,27]]},"references-count":41,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["s21155092"],"URL":"https:\/\/doi.org\/10.3390\/s21155092","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,27]]}}}