{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T17:13:22Z","timestamp":1772298802208,"version":"3.50.1"},"reference-count":36,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2019,10,31]],"date-time":"2019-10-31T00:00:00Z","timestamp":1572480000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"undefined  &lt;span style=&quot;color:gray;font-size:10px;&quot;&gt;undefined&lt;\/span&gt;","award":["61673245"],"award-info":[{"award-number":["61673245"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>In recent years, with the development of artificial intelligence and human\u2013computer interaction, more attention has been paid to the recognition and analysis of facial expressions. Despite much great success, there are a lot of unsatisfying problems, because facial expressions are subtle and complex. Hence, facial expression recognition is still a challenging problem. In most papers, the entire face image is often chosen as the input information. In our daily life, people can perceive other\u2019s current emotions only by several facial components (such as eye, mouth and nose), and other areas of the face (such as hair, skin tone, ears, etc.) play a smaller role in determining one\u2019s emotion. If the entire face image is used as the only input information, the system will produce some unnecessary information and miss some important information in the process of feature extraction. To solve the above problem, this paper proposes a method that combines multiple sub-regions and the entire face image by weighting, which can capture more important feature information that is conducive to improving the recognition accuracy. Our proposed method was evaluated based on four well-known publicly available facial expression databases: JAFFE, CK+, FER2013 and SFEW. The new method showed better performance than most state-of-the-art methods.<\/jats:p>","DOI":"10.3390\/a12110227","type":"journal-article","created":{"date-parts":[[2019,10,31]],"date-time":"2019-10-31T06:33:29Z","timestamp":1572503609000},"page":"227","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Facial Expression Recognition Based on Auxiliary Models"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8805-6724","authenticated-orcid":false,"given":"Yingying","family":"Wang","sequence":"first","affiliation":[{"name":"School of Control Science and Engineering, Shandong University, Jinan 250061, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yibin","family":"Li","sequence":"additional","affiliation":[{"name":"School of Control Science and Engineering, Shandong University, Jinan 250061, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yong","family":"Song","sequence":"additional","affiliation":[{"name":"School of Mechanical, Electrical and Information Engineering, Shandong University, Weihai 264209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xuewen","family":"Rong","sequence":"additional","affiliation":[{"name":"School of Control Science and Engineering, Shandong University, Jinan 250061, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,10,31]]},"reference":[{"key":"ref_1","first-page":"193","article-title":"Communication without words","volume":"2","author":"Mehrabian","year":"1968","journal-title":"Psychol. Today"},{"key":"ref_2","first-page":"146","article-title":"Expression of the emotions in man and animals","volume":"123","author":"Darwin","year":"2003","journal-title":"Portable Darwin"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1037\/h0030377","article-title":"Constants across cultures in the face and emotion","volume":"17","author":"Ekman","year":"1971","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Alshamsi, H., Meng, H., and Li, M. (2016, January 13\u201315). Real time facial expression recognition app development on mobile phones. Proceedings of the International Conference on Natural Computation, Changsha, China.","DOI":"10.1109\/FSKD.2016.7603442"},{"key":"ref_5","first-page":"17","article-title":"Heart beat classification using wavelet feature based on neural network","volume":"10","author":"Jatmiko","year":"2011","journal-title":"WSEAS Trans. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1007\/3-540-45869-7_4","article-title":"Machine Recognition of Printed Kannada Text","volume":"2423","author":"Kumar","year":"2002","journal-title":"Lect. Notes Comput. Sci."},{"key":"ref_7","unstructured":"Ying, Z., and Fang, X. (2008, January 26\u201329). Combining LBP and Adaboost for facial expression recognition. Proceedings of the International Conference on Signal Processing, Beijing, China."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"80","DOI":"10.1016\/j.patcog.2011.05.006","article-title":"Facial expression recognition using radial encoding of local Gabor features and classifier synthesis","volume":"45","author":"Gu","year":"2012","journal-title":"Pattern Recognit."},{"key":"ref_9","first-page":"1787","article-title":"Facial expression recognition based on LGBP features and sparse representation","volume":"34","author":"Ming","year":"2013","journal-title":"Comput. Eng. Des."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1021","DOI":"10.1007\/s00371-011-0611-x","article-title":"3D facial expression recognition using SIFT descriptors of automatically detected keypoints","volume":"27","author":"Berretti","year":"2011","journal-title":"Vis. Comput."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wang, X., Chao, J., Wei, L., Min, H., Xu, L., and Ren, F. (2013, January 15\u201317). Feature fusion of HOG and WLD for facial expression recognition. Proceedings of the IEEE\/SICE International Symposium on System Integration, Kobe, Japan.","DOI":"10.1109\/SII.2013.6776664"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2767","DOI":"10.1016\/j.ijleo.2012.08.040","article-title":"Facial expression recognition based on fusion feature of PCA and LBP with SVM","volume":"124","author":"Yuan","year":"2013","journal-title":"Opt. Int. J. Light Electron Opt."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1173","DOI":"10.1016\/j.neucom.2015.05.005","article-title":"Facial expression recognition from image sequences using twofold random forest classifier","volume":"168","author":"Pu","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1109\/MCG.2012.41","article-title":"Sparse Coding for Flexible, Robust 3D Facial-Expression Synthesis","volume":"32","author":"Lin","year":"2012","journal-title":"IEEE Comput. Graph. Appl."},{"key":"ref_15","unstructured":"Meng, Z., Ping, L., Jie, C., Han, S., and Yan, T. (June, January 30). Identity-Aware Convolutional Neural Network for Facial Expression Recognition. Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition, Washington, DC, USA."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Liu, Y., Li, Y., Ma, X., and Song, R. (2017). Facial Expression Recognition with Fusion Features Extracted from Salient Facial Areas. Sensors, 17.","DOI":"10.20944\/preprints201701.0102.v1"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Lv, Y., Feng, Z., and Chao, X. (2014, January 3\u20135). Facial expression recognition via deep learning. Proceedings of the International Conference on Smart Computing, Hong Kong, China.","DOI":"10.1109\/SMARTCOMP.2014.7043872"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"246","DOI":"10.1016\/j.neucom.2018.04.063","article-title":"A Complementary Facial Representation Extracting Method based on Deep Learning","volume":"306","author":"Sun","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2816","DOI":"10.1109\/TMM.2017.2713408","article-title":"Multimodal 2D+3D Facial Expression Recognition With Deep Fusion Convolutional Neural Network","volume":"19","author":"Li","year":"2017","journal-title":"IEEE Trans. Multimed."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"803","DOI":"10.1016\/j.imavis.2008.08.005","article-title":"Facial expression recognition based on Local Binary Patterns: A comprehensive study","volume":"27","author":"Shan","year":"2009","journal-title":"Image Vis. Comput."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1016\/S1077-3142(03)00081-X","article-title":"Facial expression recognition from video sequences: Temporal and static modeling","volume":"91","author":"Cohen","year":"2003","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Happy, S.L., George, A., and Routray, A. (2012, January 27\u201329). A real time facial expression classification system using Local Binary Patterns. Proceedings of the International Conference on Intelligent Human Computer Interaction, Kharagpur, India.","DOI":"10.1109\/IHCI.2012.6481802"},{"key":"ref_23","unstructured":"Zhang, B., Liu, G., and Xie, G. (2016, January 16\u201317). Facial expression recognition using LBP and LPQ based on Gabor wavelet transform. Proceedings of the IEEE International Conference on Computer & Communications, Chengdu, China."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Li, L., Ying, Z., and Yang, T. (2014, January 19\u201323). Facial expression recognition by fusion of Gabor texture features and local phase quantization. Proceedings of the International Conference on Signal Processing, Hangzhou, China.","DOI":"10.1109\/ICOSP.2014.7015300"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Shin, M., Kim, M., and Kwon, D.S. (2016, January 26\u201331). Baseline CNN structure analysis for facial expression recognition. Proceedings of the IEEE International Symposium on Robot & Human Interactive Communication, New York, NY, USA.","DOI":"10.1109\/ROMAN.2016.7745199"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yu, Z., and Zhang, C. (2015, January 9\u201313). Image based Static Facial Expression Recognition with Multiple Deep Network Learning. Proceedings of the Acm on International Conference on Multimodal Interaction, Seattle, WA, USA.","DOI":"10.1145\/2818346.2830595"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Jung, H., Lee, S., Yim, J., Park, S., and Kim, J. (2015, January 13\u201316). Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.341"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Abdulrahman, M., and Eleyan, A. (2015, January 16\u201319). Facial expression recognition using Support Vector Machines. Proceedings of the Signal Processing & Communications Applications Conference, Malatya, Turkey.","DOI":"10.1109\/SIU.2015.7129813"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lucey, P., Cohn, J.F., Kanade, T., and Saragih, J. (2010, January 13\u201318). The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression. Proceedings of the Computer Vision & Pattern Recognition Workshops, San Francisco, CA, USA.","DOI":"10.1109\/CVPRW.2010.5543262"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Mollahosseini, A., Chan, D., and Mahoor, M.H. (2015, January 7\u201310). Going deeper in facial expression recognition using deep neural networks. Proceedings of the IEEE Winter Conference on Applications of Computer Vision, Lake Placid, NY, USA.","DOI":"10.1109\/WACV.2016.7477450"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Dhall, A., Goecke, R., Lucey, S., and Gedeon, T. (2011, January 6\u201313). Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark. Proceedings of the IEEE International Conference on Computer Vision Workshops, Barcelona, Spain.","DOI":"10.1109\/ICCVW.2011.6130508"},{"key":"ref_32","unstructured":"Liu, M., Li, S., Shan, S., Wang, R., and Chen, X. (2014, January 1\u20135). Deeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis. Proceedings of the Asian Conference on Computer Vision, Singapore."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/TAFFC.2016.2593719","article-title":"Facial Expression Recognition in Video with Multiple Feature Fusion","volume":"9","author":"Chen","year":"2018","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s12559-017-9472-6","article-title":"Ensemble of Deep Neural Networks with Probability-Based Fusion for Facial Expression Recognition","volume":"9","author":"Wen","year":"2017","journal-title":"Cognit. Comput."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2439","DOI":"10.1109\/TIP.2018.2886767","article-title":"Occlusion aware facial expression recognition using CNN with attention mechanism","volume":"28","author":"Li","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Liu, X., Kumar, B.V.K.V., You, J., and Ping, J. (2017, January 21\u201326). Adaptive Deep Metric Learning for Identity-Aware Facial Expression Recognition. Proceedings of the IEEE Conference on Computer Vision & Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.79"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/11\/227\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:30:46Z","timestamp":1760189446000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/12\/11\/227"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,31]]},"references-count":36,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2019,11]]}},"alternative-id":["a12110227"],"URL":"https:\/\/doi.org\/10.3390\/a12110227","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,10,31]]}}}