{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,16]],"date-time":"2026-02-16T08:26:33Z","timestamp":1771230393081,"version":"3.50.1"},"reference-count":37,"publisher":"SAGE Publications","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["ICA"],"published-print":{"date-parts":[[2020,12,21]]},"abstract":"<jats:p>Facial emotion recognition (FER) has been extensively researched over the past two decades due to its direct impact in the computer vision and affective robotics fields. However, the available datasets to train these models include often miss-labelled data due to the labellers bias that drives the model to learn incorrect features. In this paper, a facial emotion recognition system is proposed, addressing automatic face detection and facial expression recognition separately, the latter is performed by a set of only four deep convolutional neural network respect to an ensembling approach, while a label smoothing technique is applied to deal with the miss-labelled training data. The proposed system takes only 13.48 ms using a dedicated graphics processing unit (GPU) and 141.97 ms using a CPU to recognize facial emotions and reaches the current state-of-the-art performances regarding the challenging databases, FER2013, SFEW 2.0, and ExpW, giving recognition accuracies of 72.72%, 51.97%, and 71.82% respectively.<\/jats:p>","DOI":"10.3233\/ica-200643","type":"journal-article","created":{"date-parts":[[2020,9,1]],"date-time":"2020-09-01T18:09:02Z","timestamp":1598983742000},"page":"97-111","source":"Crossref","is-referenced-by-count":43,"title":["Real-time facial expression recognition using smoothed deep neural network ensemble"],"prefix":"10.1177","volume":"28","author":[{"given":"Nadir Kamel","family":"Benamara","sequence":"first","affiliation":[{"name":"Laboratoire Signaux et Images, Universit\u00e9 des Sciences et de la Technologie d\u2019Oran Mohamed Boudiaf, USTO-MB, BP1505, El M\u2019naouer, Oran, Algeria"}]},{"given":"Mikel","family":"Val-Calvo","sequence":"additional","affiliation":[{"name":"Dpto. de Inteligencia Artificial, Universidad Nacional de Educaci\u00f3n a Distancia, Madrid, Spain"},{"name":"Dpto. Electr\u00f3nica, Tecnolog\u00eda de Computadoras y Proyectos, University Polit\u00e9cnica de Cartagena, Cartagena, Spain"}]},{"given":"Jose Ram\u00f3n","family":"\u00c1lvarez-S\u00e1nchez","sequence":"additional","affiliation":[{"name":"Dpto. de Inteligencia Artificial, Universidad Nacional de Educaci\u00f3n a Distancia, Madrid, Spain"}]},{"given":"Alejandro","family":"D\u00edaz-Morcillo","sequence":"additional","affiliation":[{"name":"Dpto. Tecnolog\u00edas de la Informaci\u00f3n y las Comunicaciones, University Polit\u00e9cnica de Cartagena, Cartagena, Spain"}]},{"given":"Jose Manuel","family":"Ferr\u00e1ndez-Vicente","sequence":"additional","affiliation":[{"name":"Dpto. Electr\u00f3nica, Tecnolog\u00eda de Computadoras y Proyectos, University Polit\u00e9cnica de Cartagena, Cartagena, Spain"}]},{"given":"Eduardo","family":"Fern\u00e1ndez-Jover","sequence":"additional","affiliation":[{"name":"Instituto de Bioingenier\u00eda, University Miguel Hern\u00e1ndez, Elche, Spain"}]},{"given":"Tarik Boudghene","family":"Stambouli","sequence":"additional","affiliation":[{"name":"Laboratoire Signaux et Images, Universit\u00e9 des Sciences et de la Technologie d\u2019Oran Mohamed Boudiaf, USTO-MB, BP1505, El M\u2019naouer, Oran, Algeria"}]}],"member":"179","reference":[{"key":"10.3233\/ICA-200643_ref2","doi-asserted-by":"crossref","unstructured":"Cunningham AG, Galceran E, Mehta D, Ferrer G, Eustice RM, Olson E. MPDM: Multi-policy Decision-Making from Autonomous Driving to Social Robot Navigation. In: Control Strategies for Advanced Driver Assistance Systems and Autonomous Driving Functions. Springer; 2019. pp. 201\u2013223.","DOI":"10.1007\/978-3-319-91569-2_10"},{"key":"10.3233\/ICA-200643_ref3","doi-asserted-by":"crossref","unstructured":"Chiu CC, Sainath TN, Wu Y, Prabhavalkar R, Nguyen P, Chen Z, et al. State-of-the-art speech recognition with sequence-to-sequence models. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE; 2018. pp. 4774\u20134778.","DOI":"10.1109\/ICASSP.2018.8462105"},{"issue":"3","key":"10.3233\/ICA-200643_ref4","doi-asserted-by":"crossref","first-page":"301","DOI":"10.3233\/ICA-200624","article-title":"Semantic visual recognition in a cognitive architecture for social robots","volume":"27","author":"Martin-Rico","year":"2020","journal-title":"Integrated Computer-Aided Engineering"},{"issue":"1","key":"10.3233\/ICA-200643_ref5","doi-asserted-by":"crossref","first-page":"97","DOI":"10.3233\/ICA-180585","article-title":"Speech gestural interpretation by applying word representations in robotics","volume":"26","author":"Almagro","year":"2018","journal-title":"Integrated Computer-Aided Engineering"},{"key":"10.3233\/ICA-200643_ref6","doi-asserted-by":"crossref","unstructured":"\u0160abanovi\u0107 S, Bennett CC, Chang WL, Huber L. PARO robot affects diverse interaction modalities in group sensory therapy for older adults with dementia. In: 2013 IEEE 13th International Conference on Rehabilitation Robotics (ICORR). IEEE; 2013. pp. 1\u20136.","DOI":"10.1109\/ICORR.2013.6650427"},{"key":"10.3233\/ICA-200643_ref8","doi-asserted-by":"crossref","unstructured":"Jeong S, Zisook M, Plummer L, Breazeal C, Weinstock P, Logan DE, et al. A Social Robot to Mitigate Stress, Anxiety, and Pain in Hospital Pediatric Care. In: Proceedings of the Tenth Annual ACM\/IEEE International Conference on Human-Robot Interaction Extended Abstracts \u2013 HRI\u201915 Extended Abstracts. Portland, Oregon, USA: ACM Press; 2015. pp. 103\u2013104.","DOI":"10.1145\/2701973.2702028"},{"key":"10.3233\/ICA-200643_ref9","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.cogsys.2018.12.010","article-title":"Object affordance based multimodal fusion for natural Human-Robot interaction","volume":"54","author":"Mi","year":"2019","journal-title":"Cognitive Systems Research"},{"issue":"19","key":"10.3233\/ICA-200643_ref10","doi-asserted-by":"crossref","first-page":"7241","DOI":"10.1073\/pnas.1200155109","article-title":"Facial expressions of emotion are not culturally universal","volume":"109","author":"Jack","year":"2012","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"1","key":"10.3233\/ICA-200643_ref12","doi-asserted-by":"crossref","first-page":"61","DOI":"10.3233\/ICA-2009-0304","article-title":"Integrating a mixed-feature model and multiclass support vector machine for facial expression recognition","volume":"16","author":"Lin","year":"2009","journal-title":"Integrated Computer-Aided Engineering"},{"issue":"2","key":"10.3233\/ICA-200643_ref13","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1504\/IJBM.2018.091633","article-title":"New face expression recognition using polar angular radial transform and principal component analysis","volume":"10","author":"Taleb","year":"2018","journal-title":"International Journal of Biometrics"},{"key":"10.3233\/ICA-200643_ref16","doi-asserted-by":"crossref","first-page":"56","DOI":"10.1016\/j.specom.2019.12.001","article-title":"Speech emotion recognition: Emotional models, databases, features, preprocessing methods, supporting modalities, and classifiers","volume":"116","author":"Ak\u00e7ay","year":"2020","journal-title":"Speech Communication"},{"key":"10.3233\/ICA-200643_ref17","unstructured":"Emotion Recognition From EEG Using Higher Order Crossings; 14."},{"issue":"2","key":"10.3233\/ICA-200643_ref18","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1007\/s11063-018-9829-1","article-title":"EEG emotion recognition based on graph regularized sparse linear regression","volume":"49","author":"Li","year":"2019","journal-title":"Neural Processing Letters"},{"issue":"12","key":"10.3233\/ICA-200643_ref19","doi-asserted-by":"crossref","first-page":"2333","DOI":"10.1016\/j.sigpro.2009.02.016","article-title":"Learned local gabor patterns for face representation and recognition","volume":"89","author":"Xie","year":"2009","journal-title":"Signal Processing"},{"issue":"1","key":"10.3233\/ICA-200643_ref20","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/0031-3203(95)00067-4","article-title":"A comparative study of texture measures with classification based on featured distributions","volume":"29","author":"Ojala","year":"1996","journal-title":"Pattern Recognition"},{"issue":"3","key":"10.3233\/ICA-200643_ref21","doi-asserted-by":"crossref","first-page":"247","DOI":"10.3233\/ICA-180560","article-title":"Regional parallel structure based CNN for thermal infrared face identification","volume":"25","author":"Wang","year":"2018","journal-title":"Integrated Computer-Aided Engineering"},{"issue":"1","key":"10.3233\/ICA-200643_ref22","doi-asserted-by":"crossref","first-page":"85","DOI":"10.3233\/ICA-180584","article-title":"DeepEye: Deep convolutional network for pupil detection in real environments","volume":"26","author":"Vera-Olmos","year":"2018","journal-title":"Integrated Computer-Aided Engineering"},{"key":"10.3233\/ICA-200643_ref25","doi-asserted-by":"crossref","unstructured":"Dhall A, Goecke R, Lucey S, Gedeon T. Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops). Barcelona, Spain: IEEE; 2011. pp. 2106\u20132112.","DOI":"10.1109\/ICCVW.2011.6130508"},{"key":"10.3233\/ICA-200643_ref27","doi-asserted-by":"crossref","unstructured":"Mollahosseini A, Chan D, Mahoor MH. Going deeper in facial expression recognition using deep neural networks. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV); 2016. pp. 1\u201310.","DOI":"10.1109\/WACV.2016.7477450"},{"key":"10.3233\/ICA-200643_ref28","unstructured":"Guo Y, Tao D, Yu J, Xiong H, Li Y, Tao D. Deep Neural Networks with Relativity Learning for facial expression recognition. In: 2016 IEEE International Conference on Multimedia Expo Workshops (ICMEW); 2016. pp. 1\u20136."},{"key":"10.3233\/ICA-200643_ref29","doi-asserted-by":"crossref","unstructured":"Devries T, Biswaranjan K, Taylor GW. Multi-task Learning of Facial Landmarks and Expression. In: 2014 Canadian Conference on Computer and Robot Vision; 2014. pp. 98\u2013103.","DOI":"10.1109\/CRV.2014.21"},{"issue":"1","key":"10.3233\/ICA-200643_ref31","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1007\/s11633-019-1176-9","article-title":"Expression analysis based on face regions in real-world conditions","volume":"17","author":"Lian","year":"2020","journal-title":"International Journal of Automation and Computing"},{"key":"10.3233\/ICA-200643_ref32","doi-asserted-by":"crossref","unstructured":"Levi G, Hassner T. Emotion Recognition in the Wild via Convolutional Neural Networks and Mapped Binary Patterns. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction \u2013 ICMI \u201915. Seattle, Washington, USA: ACM Press; 2015. pp. 503\u2013510.","DOI":"10.1145\/2818346.2830587"},{"key":"10.3233\/ICA-200643_ref33","doi-asserted-by":"crossref","unstructured":"Kim BK, Dong SY, Roh J, Kim G, Lee SY. Fusing Aligned and Non-aligned Face Information for Automatic Affect Recognition in the Wild: A Deep Learning Approach. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Las Vegas, NV, USA: IEEE; 2016. pp.\u00a01499\u20131508.","DOI":"10.1109\/CVPRW.2016.187"},{"key":"10.3233\/ICA-200643_ref36","doi-asserted-by":"crossref","unstructured":"Viola P, Jones M. Rapid object detection using a boosted cascade of simple features. In: Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on. Vol. 1. IEEE; 2001. pp. 1\u20131.","DOI":"10.1109\/CVPR.2001.990517"},{"key":"10.3233\/ICA-200643_ref41","doi-asserted-by":"crossref","unstructured":"Redmon J, Divvala S, Girshick R, Farhadi A. You Only Look Once: Unified, Real-Time Object Detection. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE; 2016. pp. 779\u2013788.","DOI":"10.1109\/CVPR.2016.91"},{"issue":"2","key":"10.3233\/ICA-200643_ref42","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1007\/s11263-019-01247-4","article-title":"Deep learning for generic object detection: A survey","volume":"128","author":"Liu","year":"2020","journal-title":"International Journal of Computer Vision"},{"key":"10.3233\/ICA-200643_ref43","doi-asserted-by":"crossref","unstructured":"Redmon J, Farhadi A. YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; 2017. pp. 7263\u20137271.","DOI":"10.1109\/CVPR.2017.690"},{"key":"10.3233\/ICA-200643_ref44","doi-asserted-by":"crossref","unstructured":"Yang S, Luo P, Loy CC, Tang X. WIDER FACE: A Face Detection Benchmark. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE; 2016. pp. 5525\u20135533.","DOI":"10.1109\/CVPR.2016.596"},{"issue":"3","key":"10.3233\/ICA-200643_ref47","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/MMUL.2012.26","article-title":"Collecting large, richly annotated facial-expression databases from movies","volume":"19","author":"Dhall","year":"2012","journal-title":"IEEE Multi Media"},{"key":"10.3233\/ICA-200643_ref48","doi-asserted-by":"crossref","unstructured":"Ng HW, Nguyen VD, Vonikakis V, Winkler S. Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction \u2013 ICMI \u201915. Seattle, Washington, USA: ACM Press; 2015. pp. 443\u2013449.","DOI":"10.1145\/2818346.2830593"},{"key":"10.3233\/ICA-200643_ref49","doi-asserted-by":"crossref","unstructured":"Meng Z, Liu P, Cai J, Han S, Tong Y. Identity-Aware Convolutional Neural Network for Facial Expression Recognition. In: 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017). Washington, DC, DC, USA: IEEE; 2017. pp. 558\u2013565.","DOI":"10.1109\/FG.2017.140"},{"key":"10.3233\/ICA-200643_ref50","doi-asserted-by":"crossref","unstructured":"Li S, Deng W, Du J. Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI: IEEE; 2017. pp.\u00a02584\u20132593.","DOI":"10.1109\/CVPR.2017.277"},{"key":"10.3233\/ICA-200643_ref51","doi-asserted-by":"crossref","unstructured":"Cai J, Meng Z, Khan AS, Li Z, OReilly J, Tong Y. Island Loss for Learning Discriminative Features in Facial Expression Recognition. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018). Xi\u2019an: IEEE; 2018. pp. 302\u2013309.","DOI":"10.1109\/FG.2018.00051"},{"key":"10.3233\/ICA-200643_ref53","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"Journal of Machine Learning Research"},{"key":"10.3233\/ICA-200643_ref54","doi-asserted-by":"crossref","unstructured":"Manderson T, Dudek G. GPU-Assisted Learning on an Autonomous Marine Robot for Vision-Based Navigation and Image Understanding. In: OCEANS 2018 MTS\/IEEE Charleston. IEEE; 2018, pp. 1\u20136.","DOI":"10.1109\/OCEANS.2018.8604645"},{"key":"10.3233\/ICA-200643_ref56","doi-asserted-by":"crossref","unstructured":"Buettner R, Baumgartl H. A highly effective deep learning based escape route recognition module for autonomous robots in crisis and emergency situations. In: Proceedings of the 52nd Hawaii International Conference on System Sciences; 2019.","DOI":"10.24251\/HICSS.2019.081"}],"container-title":["Integrated Computer-Aided Engineering"],"original-title":[],"link":[{"URL":"https:\/\/content.iospress.com\/download?id=10.3233\/ICA-200643","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,11]],"date-time":"2025-03-11T11:08:19Z","timestamp":1741691299000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/full\/10.3233\/ICA-200643"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,12,21]]},"references-count":37,"journal-issue":{"issue":"1"},"URL":"https:\/\/doi.org\/10.3233\/ica-200643","relation":{},"ISSN":["1069-2509","1875-8835"],"issn-type":[{"value":"1069-2509","type":"print"},{"value":"1875-8835","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,12,21]]}}}