{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T03:03:41Z","timestamp":1792206221933,"version":"4.4.0"},"reference-count":22,"publisher":"Wiley","license":[{"start":{"date-parts":[[2017,1,1]],"date-time":"2017-01-01T00:00:00Z","timestamp":1483228800000},"content-version":"unspecified","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61503143"],"award-info":[{"award-number":["61503143"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2014A030310244"],"award-info":[{"award-number":["2014A030310244"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["201710010038"],"award-info":[{"award-number":["201710010038"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003453","name":"Natural Science Foundation of Guangdong Province","doi-asserted-by":"publisher","award":["61503143"],"award-info":[{"award-number":["61503143"]}],"id":[{"id":"10.13039\/501100003453","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003453","name":"Natural Science Foundation of Guangdong Province","doi-asserted-by":"publisher","award":["2014A030310244"],"award-info":[{"award-number":["2014A030310244"]}],"id":[{"id":"10.13039\/501100003453","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003453","name":"Natural Science Foundation of Guangdong Province","doi-asserted-by":"publisher","award":["201710010038"],"award-info":[{"award-number":["201710010038"]}],"id":[{"id":"10.13039\/501100003453","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100009334","name":"Pearl River S&T Nova Program of Guangzhou","doi-asserted-by":"crossref","award":["61503143"],"award-info":[{"award-number":["61503143"]}],"id":[{"id":"10.13039\/501100009334","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100009334","name":"Pearl River S&T Nova Program of Guangzhou","doi-asserted-by":"crossref","award":["2014A030310244"],"award-info":[{"award-number":["2014A030310244"]}],"id":[{"id":"10.13039\/501100009334","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100009334","name":"Pearl River S&T Nova Program of Guangzhou","doi-asserted-by":"crossref","award":["201710010038"],"award-info":[{"award-number":["201710010038"]}],"id":[{"id":"10.13039\/501100009334","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computational Intelligence and Neuroscience"],"published-print":{"date-parts":[[2017]]},"abstract":"<jats:p>This paper proposes two multimodal fusion methods between brain and peripheral signals for emotion recognition. The input signals are electroencephalogram and facial expression. The stimuli are based on a subset of movie clips that correspond to four specific areas of valance-arousal emotional space (happiness, neutral, sadness, and fear). For facial expression detection, four basic emotion states (happiness, neutral, sadness, and fear) are detected by a neural network classifier. For EEG detection, four basic emotion states and three emotion intensity levels (strong, ordinary, and weak) are detected by two support vector machines (SVM) classifiers, respectively. Emotion recognition is based on two decision-level fusion methods of both EEG and facial expression detections by using a sum rule or a production rule. Twenty healthy subjects attended two experiments. The results show that the accuracies of two multimodal fusion detections are 81.25% and 82.75%, respectively, which are both higher than that of facial expression (74.38%) or EEG detection (66.88%). The combination of facial expressions and EEG information for emotion recognition compensates for their defects as single information sources.<\/jats:p>","DOI":"10.1155\/2017\/2107451","type":"journal-article","created":{"date-parts":[[2017,9,19]],"date-time":"2017-09-19T21:01:20Z","timestamp":1505854880000},"page":"1-8","source":"Crossref","is-referenced-by-count":101,"title":["Fusion of Facial Expressions and EEG for Multimodal Emotion Recognition"],"prefix":"10.1155","volume":"2017","author":[{"given":"Yongrui","family":"Huang","sequence":"first","affiliation":[{"name":"School of Software, South China Normal University, Guangzhou 510641, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianhao","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Software, South China Normal University, Guangzhou 510641, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pengkai","family":"Liao","sequence":"additional","affiliation":[{"name":"School of Software, South China Normal University, Guangzhou 510641, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7576-6743","authenticated-orcid":true,"given":"Jiahui","family":"Pan","sequence":"additional","affiliation":[{"name":"School of Software, South China Normal University, Guangzhou 510641, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","reference":[{"key":"1","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-005-1081-1"},{"issue":"2","key":"2","first-page":"126","volume":"47","year":"1978","journal-title":"Rivista di Psichiatria"},{"issue":"10","key":"3","first-page":"3474","volume":"74","year":"1991","journal-title":"IEICE Transactions"},{"key":"5","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2009.03.005"},{"issue":"8","key":"6","first-page":"87","volume":"3","year":"2012","journal-title":"WSEAS Transactions on Signal Processing"},{"key":"10","doi-asserted-by":"publisher","DOI":"10.1109\/T-AFFC.2011.12"},{"key":"12","doi-asserted-by":"publisher","DOI":"10.1109\/tbme.2013.2270283"},{"issue":"6","key":"13","first-page":"856","volume":"21","year":"2015","journal-title":"Journal of Universal Computer Science"},{"key":"15","doi-asserted-by":"publisher","DOI":"10.1016\/0005-7916(94)90063-9"},{"key":"16","doi-asserted-by":"publisher","DOI":"10.3389\/fnins.2014.00322"},{"key":"17","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2011.07.091"},{"key":"19","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2560\/10\/2\/026005"},{"key":"21","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2560\/8\/2\/025011"},{"key":"24","doi-asserted-by":"publisher","DOI":"10.1016\/s0167-8655(03)00079-5"},{"key":"25","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2004.825930"},{"key":"26","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2560\/4\/2\/r01"},{"key":"27","doi-asserted-by":"publisher","DOI":"10.1016\/j.clinph.2010.06.034"},{"key":"28","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2015.2469106"},{"key":"29","doi-asserted-by":"publisher","DOI":"10.1007\/s11571-013-9253-1"},{"key":"31","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2014.2342533"},{"issue":"5","key":"32","first-page":"1241","volume":"28","year":"2016","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"33","doi-asserted-by":"publisher","DOI":"10.1088\/1741-2552\/aa5a98"}],"container-title":["Computational Intelligence and Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2017\/2107451.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2017\/2107451.xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/downloads.hindawi.com\/journals\/cin\/2017\/2107451.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,5,18]],"date-time":"2020-05-18T07:55:31Z","timestamp":1589788531000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.hindawi.com\/journals\/cin\/2017\/2107451\/"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017]]},"references-count":22,"alternative-id":["2107451","2107451"],"URL":"https:\/\/doi.org\/10.1155\/2017\/2107451","relation":{},"ISSN":["1687-5265","1687-5273"],"issn-type":[{"value":"1687-5265","type":"print"},{"value":"1687-5273","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017]]}}}