{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,17]],"date-time":"2026-02-17T12:41:29Z","timestamp":1771332089308,"version":"3.50.1"},"reference-count":33,"publisher":"Wiley","license":[{"start":{"date-parts":[[2021,5,25]],"date-time":"2021-05-25T00:00:00Z","timestamp":1621900800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Natural Science Research Project of Anhui Province","award":["KJ2019A1031"],"award-info":[{"award-number":["KJ2019A1031"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational and Mathematical Methods in Medicine"],"published-print":{"date-parts":[[2021,5,25]]},"abstract":"<jats:p>As one of the key issues in the field of emotional computing, emotion recognition has rich application scenarios and important research value. However, the single biometric recognition in the actual scene has the problem of low accuracy of emotion recognition classification due to its own limitations. In response to this problem, this paper combines deep neural networks to propose a deep learning-based expression-EEG bimodal fusion emotion recognition method. This method is based on the improved VGG-FACE network model to realize the rapid extraction of facial expression features and shorten the training time of the network model. The wavelet soft threshold algorithm is used to remove artifacts from EEG signals to extract high-quality EEG signal features. Then, based on the long- and short-term memory network models and the decision fusion method, the model is built and trained using the signal feature data extracted under the expression-EEG bimodality to realize the final bimodal fusion emotion classification and identification research. Finally, the proposed method is verified based on the MAHNOB-HCI data set. Experimental results show that the proposed model can achieve a high recognition accuracy of 0.89, which can increase the accuracy of 8.51% compared with the traditional LSTM model. In terms of the running time of the identification method, the proposed method can effectively be shortened by about 20\u2009s compared with the traditional method.<\/jats:p>","DOI":"10.1155\/2021\/9940148","type":"journal-article","created":{"date-parts":[[2021,5,26]],"date-time":"2021-05-26T18:13:14Z","timestamp":1622052794000},"page":"1-10","source":"Crossref","is-referenced-by-count":13,"title":["Expression-EEG Bimodal Fusion Emotion Recognition Method Based on Deep Learning"],"prefix":"10.1155","volume":"2021","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5512-6925","authenticated-orcid":true,"given":"Yu","family":"Lu","sequence":"first","affiliation":[{"name":"Fuyang Vocational and Technical College, Fuyang, Anhui 236031, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0012-7752","authenticated-orcid":true,"given":"Hua","family":"Zhang","sequence":"additional","affiliation":[{"name":"Fuyang Vocational and Technical College, Fuyang, Anhui 236031, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6813-8071","authenticated-orcid":true,"given":"Lei","family":"Shi","sequence":"additional","affiliation":[{"name":"Fuyang Vocational and Technical College, Fuyang, Anhui 236031, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4233-7181","authenticated-orcid":true,"given":"Fei","family":"Yang","sequence":"additional","affiliation":[{"name":"Fuyang Vocational and Technical College, Fuyang, Anhui 236031, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6168-6171","authenticated-orcid":true,"given":"Jing","family":"Li","sequence":"additional","affiliation":[{"name":"Department of Electrical & Information Engineering, Sichuan Engineering Technical College, Deyang, Sichuan 618000, China"}]}],"member":"311","reference":[{"key":"1","doi-asserted-by":"publisher","DOI":"10.1007\/s10660-017-9265-8"},{"key":"2","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2015.1064638"},{"key":"3","doi-asserted-by":"publisher","DOI":"10.1016\/j.intcom.2004.06.002"},{"key":"4","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2015.02.008"},{"key":"5","doi-asserted-by":"publisher","DOI":"10.4028\/www.scientific.net\/AMR.143-144.677"},{"key":"6","doi-asserted-by":"publisher","DOI":"10.1145\/3078833"},{"issue":"3","key":"7","first-page":"1","article-title":"Real-time facial affective computing on mobile devices","volume":"20","author":"Y. Guo","year":"2020","journal-title":"Sensors (Basel, Switzerland)"},{"key":"8","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2015.07.314"},{"key":"9","doi-asserted-by":"publisher","DOI":"10.1093\/iwc\/iwy018"},{"key":"10","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2016.2635124"},{"issue":"10","key":"11","first-page":"61","article-title":"Speech emotion recognition based on separable convolution and LSTM","volume":"44","author":"W. J. Li","year":"2020","journal-title":"Information technology"},{"issue":"12","key":"12","first-page":"2142","article-title":"A review of multimodal dimension emotion prediction","volume":"44","author":"X. Li","year":"2018","journal-title":"Acta Automatica Sinica"},{"key":"13","doi-asserted-by":"publisher","DOI":"10.1109\/JSEN.2019.2928781"},{"key":"14","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2016.07.026"},{"issue":"2","key":"15","first-page":"1","article-title":"Expression recognition model based on deep learning and evidence theory","volume":"29","author":"Q. Xu","year":"2020","journal-title":"Computer engineering and science"},{"key":"16","doi-asserted-by":"publisher","DOI":"10.1037\/pspa0000107"},{"key":"17","doi-asserted-by":"publisher","DOI":"10.1007\/s12193-013-0145-9"},{"key":"18","doi-asserted-by":"publisher","DOI":"10.1016\/j.ergon.2019.102899"},{"issue":"2","key":"19","first-page":"1","article-title":"EmoNets: multimodal deep learning approaches for emotion recognition in video","volume":"10","author":"S. E. Kahou","year":"2015","journal-title":"Journal on Multimodal User Interfaces"},{"key":"20","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2017.2714671"},{"key":"21","doi-asserted-by":"publisher","DOI":"10.1109\/BIBM.2016.7822545"},{"key":"22","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2981760"},{"key":"23","doi-asserted-by":"crossref","first-page":"521","DOI":"10.1007\/978-3-319-46672-9_58","article-title":"Emotion recognition using multimodal deep learning","volume-title":"international conference on neural information processing","author":"W. Liu","year":"2016"},{"key":"24","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2017.140"},{"key":"25","doi-asserted-by":"publisher","DOI":"10.1109\/MMUL.2019.2960219"},{"key":"26","doi-asserted-by":"publisher","DOI":"10.1049\/iet-spr.2017.0320"},{"key":"27","first-page":"1675","article-title":"Speech emotion classification using attention-based LSTM","volume-title":"IEEE\/ACM transactions on audio","author":"Y. Xie","year":"2019"},{"issue":"12","key":"28","first-page":"109","article-title":"Study of emotion recognition based on fusion multi-modal bio-signal with SAE and LSTM recurrent neural network","volume":"38","author":"L. I. YJ","year":"2017","journal-title":"Journal on Communications"},{"key":"29","doi-asserted-by":"publisher","DOI":"10.1007\/s11571-020-09634-1"},{"key":"30","doi-asserted-by":"publisher","DOI":"10.1007\/s11042-020-09412-5"},{"key":"31","doi-asserted-by":"publisher","DOI":"10.1049\/el.2018.6932"},{"key":"32","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2017.01.096"},{"key":"33","doi-asserted-by":"publisher","DOI":"10.1007\/s12193-015-0203-6"}],"container-title":["Computational and Mathematical Methods in Medicine"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/cmmm\/2021\/9940148.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cmmm\/2021\/9940148.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cmmm\/2021\/9940148.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,5,26]],"date-time":"2021-05-26T18:13:31Z","timestamp":1622052811000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.hindawi.com\/journals\/cmmm\/2021\/9940148\/"}},"subtitle":[],"editor":[{"given":"Kaijian","family":"Xia","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2021,5,25]]},"references-count":33,"alternative-id":["9940148","9940148"],"URL":"https:\/\/doi.org\/10.1155\/2021\/9940148","relation":{},"ISSN":["1748-6718","1748-670X"],"issn-type":[{"value":"1748-6718","type":"electronic"},{"value":"1748-670X","type":"print"}],"subject":[],"published":{"date-parts":[[2021,5,25]]}}}