{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T23:24:45Z","timestamp":1772148285183,"version":"3.50.1"},"reference-count":50,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2020,2,17]],"date-time":"2020-02-17T00:00:00Z","timestamp":1581897600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Applied Technology of Inner Mangolia Autonomous region, China","award":["201802005"],"award-info":[{"award-number":["201802005"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Facial expression recognition has been well studied for its great importance in the areas of human\u2013computer interaction and social sciences. With the evolution of deep learning, there have been significant advances in this area that also surpass human-level accuracy. Although these methods have achieved good accuracy, they are still suffering from two constraints (high computational power and memory), which are incredibly critical for small hardware-constrained devices. To alleviate this issue, we propose a new Convolutional Neural Network (CNN) architecture eXnet (Expression Net) based on parallel feature extraction which surpasses current methods in accuracy and contains a much smaller number of parameters (eXnet: 4.57 million, VGG19: 14.72 million), making it more efficient and lightweight for real-time systems. Several modern data augmentation techniques are applied for generalization of eXnet; these techniques improve the accuracy of the network by overcoming the problem of overfitting while containing the same size. We provide an extensive evaluation of our network against key methods on Facial Expression Recognition 2013 (FER-2013), Extended Cohn-Kanade Dataset (CK+), and Real-world Affective Faces Database (RAF-DB) benchmark datasets. We also perform ablation evaluation to show the importance of different components of our architecture. To evaluate the efficiency of eXnet on embedded systems, we deploy it on Raspberry Pi 4B. All these evaluations show the superiority of eXnet for emotion recognition in the wild in terms of accuracy, the number of parameters, and size on disk.<\/jats:p>","DOI":"10.3390\/s20041087","type":"journal-article","created":{"date-parts":[[2020,2,20]],"date-time":"2020-02-20T03:20:03Z","timestamp":1582168803000},"page":"1087","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":66,"title":["eXnet: An Efficient Approach for Emotion Recognition in the Wild"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0250-5913","authenticated-orcid":false,"given":"Muhammad Naveed","family":"Riaz","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yao","family":"Shen","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Muhammad","family":"Sohail","sequence":"additional","affiliation":[{"name":"Department of Automation, Shanghai Jiao Tong University, Shanghai 200240, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Minyi","family":"Guo","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai 200240, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,2,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Fabiano, D., and Canavan, S.J. (2019, January 14\u201318). Deformable Synthesis Model for Emotion Recognition. Proceedings of the 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), Lille, France.","DOI":"10.1109\/FG.2019.8756614"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Fujii, K., Sugimura, D., and Hamamoto, T. (2019, January 14\u201318). Hierarchical Group-level Emotion Recognition in the Wild. Proceedings of the 2019 14th IEEE International Conference on Automatic Face Gesture Recognition (FG 2019), Lille, France.","DOI":"10.1109\/FG.2019.8756573"},{"key":"ref_3","unstructured":"Ionescu, R.T., and Grozea, C. (2019, December 23). Local Learning to Improve Bag of Visual Words Model for Facial Expression Recognition. Available online: http:\/\/deeplearning.net\/wp-content\/uploads\/2013\/03\/VV-NN-LL-WREPL.pdf."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zafer, A., Nawaz, R., and Iqbal, J. (2013, January 9\u201310). Face recognition with expression variation via robust NCC. Proceedings of the 2013 IEEE 9th International Conference on Emerging Technologies (ICET), Islamabad, Pakistan.","DOI":"10.1109\/ICET.2013.6743520"},{"key":"ref_5","unstructured":"Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., and Metaxas, D.N. (2012, January 16\u201321). Learning active facial patches for expression analysis. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA."},{"key":"ref_6","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20138). ImageNet classification with deep convolutional neural networks. Proceedings of the Neural Information Processing Systems 2012, Lake Tahoe, CA, USA."},{"key":"ref_7","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_8","unstructured":"Khan, F. (2014). Facial Expression Recognition using Facial Landmark Detection and Feature Extraction via Neural Networks. arXiv."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Sang, D.V., Van Dat, N., and Thuan, D.P. (2017, January 19\u201321). Facial expression recognition using deep convolutional neural networks. Proceedings of the 2017 9th International Conference on Knowledge and Systems Engineering (KSE), Hue, Vietnam.","DOI":"10.1109\/KSE.2017.8119447"},{"key":"ref_10","unstructured":"Tautkute, I., and Trzcinski, T. (2018). Classifying and Visualizing Emotions with Emotional DAN. arXiv."},{"key":"ref_11","unstructured":"Tang, Y. (2013). Deep Learning using Support Vector Machines. arXiv."},{"key":"ref_12","unstructured":"Shah, J.H., Sharif, M., Yasmin, M., and Fernandes, S.L. (2017). Facial expressions classification and false label reduction using LDA and threefold SVM. Pattern Recognit. Lett."},{"key":"ref_13","unstructured":"Burkert, P., Trier, F., Afzal, M.Z., Dengel, A., and Liwicki, M. (2015). DeXpression: Deep Convolutional Neural Network for Expression Recognition. arXiv."},{"key":"ref_14","first-page":"405","article-title":"Using CNN for facial expression recognition: A study of the effects of kernel size and number of filters on accuracy","volume":"2","author":"Agrawal","year":"2019","journal-title":"Visual Comput."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Goodfellow, I.J., Erhan, D., Carrier, P.L., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., and Lee, D.-H. (2013, January 3\u20137). Challenges in Representation Learning: A report on three machine learning contests. Proceedings of the ICONIP: International Conference on Neural Information Processing, Daegu, Korea.","DOI":"10.1007\/978-3-642-42051-1_16"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1016\/j.patcog.2016.07.026","article-title":"Facial expression recognition with Convolutional Neural Networks: Coping with few data and the training sample order","volume":"61","author":"Lopes","year":"2017","journal-title":"Pattern Recognit."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.patrec.2019.01.008","article-title":"Extended deep neural network for facial emotion recognition","volume":"120","author":"Jain","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.neucom.2019.05.005","article-title":"Three convolutional neural network models for facial expression recognition in the wild","volume":"355","author":"Shao","year":"2019","journal-title":"Neurocomputing"},{"key":"ref_19","unstructured":"Breuer, R., and Kimmel, R. (2017). A Deep Learning Perspective on the Origin of Facial Expressions. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015, January 7\u201312). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), Boston, MA, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S.E., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going Deeper with Convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2015), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_22","unstructured":"DeVries, T., and Taylor, G.W. (2017). Improved Regularization of Convolutional Neural Networks with Cutout. arXiv."},{"key":"ref_23","unstructured":"Zhang, H., Cisse, M., Dauphin, Y.N., and Lopez-Paz, D. (2018). mixup: Beyond Empirical Risk Minimization. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Yun, S., Han, D., Oh, S.J., Chun, S., Choe, J., and Yoo, Y. (November, January 27). CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features. Proceedings of the 2019 International Conference on Computer Vision, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00612"},{"key":"ref_25","unstructured":"Verma, V., Lamb, A., Beckham, C., Najafi, A., Courville, A., Mitliagkas, I., and Bengio, Y. (2019, December 02). Manifold Mixup: Learning Better Representations by Interpolating Hidden States. Available online: https:\/\/arxiv.org\/pdf\/1806.05236.pdf."},{"key":"ref_26","unstructured":"Gastaldi, X. (2017). Shake-Shake regularization. arXiv."},{"key":"ref_27","unstructured":"Yamada, Y., Iwamura, M., and Kise, K. (2018). ShakeDrop regularization. arXiv."},{"key":"ref_28","unstructured":"Ghiasi, G., Lin, T.Y., and Le, Q.V. (2018, January 2\u20138). DropBlock: A regularization method for convolutional networks. Proceedings of the Advances in Neural Information Processing Systems 31, Montreal, QC, Canada."},{"key":"ref_29","unstructured":"Bettadapura, V. (2012). Face Expression Recognition and Analysis: The State of the Art. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Roychowdhury, S., and Emmons, M. (2015). A Survey of the Trends in Facial and Expression Recognition Databases and Methods. Int. J. Comput. Sci. Eng. Surv., 6.","DOI":"10.5121\/ijcses.2015.6501"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Canedo, D., and Neves, A.J.R. (2019). Facial Expression Recognition Using Computer Vision: A Systematic Review. Appl. Sci., 9.","DOI":"10.3390\/app9214678"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Huang, Y., Chen, F., Lv, S., and Wang, X. (2019). Facial Expression Recognition: A Survey. Symmetry, 11.","DOI":"10.3390\/sym11101189"},{"key":"ref_33","unstructured":"Glorot, X., Bordes, A., and Bengio, Y. (2011, January 11\u201313). Deep Sparse Rectifier Neural Networks. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, Lauderdale, FL, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Liu, C., Tang, T., Lv, K., and Wang, M. (2018, January 16\u201320). Multi-Feature Based Emotion Recognition for Video Clips. Proceedings of the International Conference on Multimodal Interaction (ICMI \u201918), Boulder, CO, USA.","DOI":"10.1145\/3242969.3264989"},{"key":"ref_35","unstructured":"Arriaga, O., Valdenegro-Toro, M., and Pl\u00f6ger, P. (2017). Real-time Convolutional Neural Networks for Emotion and Gender Classification. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Liu, K., Zhang, M., and Pan, Z. (2016, January 28\u201330). Facial Expression Recognition with CNN Ensemble. Proceedings of the 2016 International Conference on Cyberworlds (CW), Chongqing, China.","DOI":"10.1109\/CW.2016.34"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Mehta, D., Siddiqui, M.F.H., and Javaid, A.Y. (2019). Recognition of Emotion Intensities Using Machine Learning Algorithms: A Comparative Study. Sensors, 19.","DOI":"10.3390\/s19081897"},{"key":"ref_38","unstructured":"Li, S., and Deng, W. (2018). Deep Facial Expression Recognition: A Survey. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Yang, H., Han, J., and Min, K. (2019). A Multi-Column CNN Model for Emotion Recognition from EEG Signals. Sensors, 19.","DOI":"10.3390\/s19214736"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1109\/ACCESS.2018.2883213","article-title":"Using Deep Convolutional Neural Network for Emotion Detection on a Physiological Signals Dataset (AMIGOS)","volume":"7","author":"Abdulhay","year":"2019","journal-title":"IEEE Access"},{"key":"ref_41","unstructured":"Amodei, D., Ananthanarayanan, S., Anubhai, R., Bai, J., Battenberg, E., Case, C., Casper, J., Catanzaro, B., Cheng, Q., and Chrzanowski, M. (2016, January 19\u201324). Deep Speech 2: End-to-end Speech Recognition in English and Mandarin. Proceedings of the 33rd International Conference on International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Zhu, X., Liu, Y., Qin, Z., and Li, J. (2017). Data Augmentation in Emotion Classification Using Generative Adversarial Networks. arXiv.","DOI":"10.1007\/978-3-319-93040-4_28"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zhu, J., Park, T., Isola, P., and Efros, A.A. (2017). Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. arXiv.","DOI":"10.1109\/ICCV.2017.244"},{"key":"ref_44","unstructured":"(2018). EEG-Based Emotion Recognition using 3D Convolutional Neural Networks. Int. J. Adv. Comput. Sci. Appl., 9, 329\u2013337."},{"key":"ref_45","unstructured":"Ioffe, S., and Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv."},{"key":"ref_46","unstructured":"Lin, M., Chen, Q., and Yan, S. (2013). Network In Network. arXiv."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., and Weinberger, K.Q. (2016). Densely Connected Convolutional Networks. arXiv.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_48","unstructured":"Paszke, A., Gross, S., Chintala, S., Chanan, G., Yang, E., DeVito, Z., Lin, Z., Desmaison, A., Antiga, L., and Lerer, A. (2019, December 12). Automatic Differentiation in PyTorch. NIPS Autodiff Workshop. Available online: https:\/\/openreview.net\/pdf?id=BJJsrmfCZ."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1109\/TIP.2018.2868382","article-title":"Reliable Crowdsourcing and Deep Locality-Preserving Learning for Unconstrained Facial Expression Recognition","volume":"28","author":"Li","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Fan, Y., Lam, J.C.K., and Li, V.O.K. (2018, January 4\u20137). Multi-region Ensemble Convolutional Neural Network for Facial Expression Recognition. Proceedings of the 27th International Conference on Artificial Neural Networks, Rhodes, Greece.","DOI":"10.1007\/978-3-030-01418-6_9"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/4\/1087\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T08:58:31Z","timestamp":1760173111000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/4\/1087"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,2,17]]},"references-count":50,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2020,2]]}},"alternative-id":["s20041087"],"URL":"https:\/\/doi.org\/10.3390\/s20041087","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,2,17]]}}}