{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T11:09:50Z","timestamp":1762340990889,"version":"build-2065373602"},"reference-count":33,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2017,3,29]],"date-time":"2017-03-29T00:00:00Z","timestamp":1490745600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Shandong Province Science and Technology Major Projects","award":["China (2015ZDXX0801A02)"],"award-info":[{"award-number":["China (2015ZDXX0801A02)"]}]},{"name":"National High Technology Research and Development Program 863","award":["China (2015AA042307)"],"award-info":[{"award-number":["China (2015AA042307)"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In the pattern recognition domain, deep architectures are currently widely used and they have achieved fine results. However, these deep architectures make particular demands, especially in terms of their requirement for big datasets and GPU. Aiming to gain better results without deep networks, we propose a simplified algorithm framework using fusion features extracted from the salient areas of faces. Furthermore, the proposed algorithm has achieved a better result than some deep architectures. For extracting more effective features, this paper firstly defines the salient areas on the faces. This paper normalizes the salient areas of the same location in the faces to the same size; therefore, it can extracts more similar features from different subjects. LBP and HOG features are extracted from the salient areas, fusion features\u2019 dimensions are reduced by Principal Component Analysis (PCA) and we apply several classifiers to classify the six basic expressions at once. This paper proposes a salient areas definitude method which uses peak expressions frames compared with neutral faces. This paper also proposes and applies the idea of normalizing the salient areas to align the specific areas which express the different expressions. As a result, the salient areas found from different subjects are the same size. In addition, the gamma correction method is firstly applied on LBP features in our algorithm framework which improves our recognition rates significantly. By applying this algorithm framework, our research has gained state-of-the-art performances on CK+ database and JAFFE database.<\/jats:p>","DOI":"10.3390\/s17040712","type":"journal-article","created":{"date-parts":[[2017,3,29]],"date-time":"2017-03-29T11:26:44Z","timestamp":1490786804000},"page":"712","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":59,"title":["Facial Expression Recognition with Fusion Features Extracted from Salient Facial Areas"],"prefix":"10.3390","volume":"17","author":[{"given":"Yanpeng","family":"Liu","sequence":"first","affiliation":[{"name":"School of Control Science and Engineering, Shandong University, Jinan 250061, China"}]},{"given":"Yibin","family":"Li","sequence":"additional","affiliation":[{"name":"School of Control Science and Engineering, Shandong University, Jinan 250061, China"}]},{"given":"Xin","family":"Ma","sequence":"additional","affiliation":[{"name":"School of Control Science and Engineering, Shandong University, Jinan 250061, China"}]},{"given":"Rui","family":"Song","sequence":"additional","affiliation":[{"name":"School of Control Science and Engineering, Shandong University, Jinan 250061, China"}]}],"member":"1968","published-online":{"date-parts":[[2017,3,29]]},"reference":[{"key":"ref_1","unstructured":"Izard, C.E. (1971). The Face of Emotion, Appleton-Century-Crofts."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/j.neucom.2015.02.011","article-title":"Au-inspired deep networks for facial expression feature learning","volume":"159","author":"Liu","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_3","unstructured":"Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., and Metaxas, D.N. (2012, January 16\u201321). Learning active facial patches for expression analysis. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"803","DOI":"10.1016\/j.imavis.2008.08.005","article-title":"Facial expression recognition based on local binary patterns: A comprehensive study","volume":"27","author":"Shan","year":"2009","journal-title":"Image Vis. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., and Matthews, I. (2010, January 13\u201318). The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), San Francisco, CA, USA.","DOI":"10.1109\/CVPRW.2010.5543262"},{"key":"ref_6","unstructured":"Wang, J., Yin, L., Wei, X., and Sun, Y. (2006, January 17\u201322). 3D facial expression recognition based on primitive surface feature distribution. Proceedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, NY, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1016\/j.patcog.2013.07.018","article-title":"An efficient 3D face recognition approach using local geometrical signatures","volume":"47","author":"Lei","year":"2014","journal-title":"Pattern Recognit."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1768","DOI":"10.1016\/j.robot.2014.07.009","article-title":"3D face recognition: An automatic strategy based on geometrical descriptors and landmarks","volume":"62","author":"Vezzetti","year":"2014","journal-title":"Robot. Auton. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1007\/s00266-014-0334-2","article-title":"3D Landmarking in multiexpression face analysis: A preliminary study on eyebrows and mouth","volume":"38","author":"Vezzetti","year":"2014","journal-title":"Aesthet. Plast. Surg."},{"key":"ref_10","unstructured":"Jung, H., Lee, S., Park, S., Lee, I., Ahn, C., and Kim, J. (arXiv, 2015). Deep Temporal Appearance-Geometry Network for Facial Expression Recognition, arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"646","DOI":"10.1016\/j.eswa.2012.07.074","article-title":"Fusion of feature sets and classifiers for facial expression recognition","volume":"40","author":"Zavaschi","year":"2013","journal-title":"Expert Syst. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hu, Y., Zeng, Z., Yin, L., Wei, X., Zhou, X., and Huang, T.S. (2008, January 17\u201319). Multi-view facial expression recognition. Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, Amsterdam, The Netherlands.","DOI":"10.1109\/AFGR.2008.4813445"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Happy, S., and Routray, A. (2015, January 4\u20137). Robust facial expression classification using shape and appearance features. Proceedings of the Eighth International Conference on Advances in Pattern Recognition (ICAPR), Kolkata, India.","DOI":"10.1109\/ICAPR.2015.7050661"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Liu, P., Han, S., Meng, Z., and Tong, Y. (2014, January 23\u201328). Facial expression recognition via a boosted deep belief network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.233"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Khorrami, P., Paine, T., and Huang, T. (2015, January 7\u201313). Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition?. Proceedings of the IEEE International Conference on Computer Vision Workshops, Santiago, Chile.","DOI":"10.1109\/ICCVW.2015.12"},{"key":"ref_16","unstructured":"Tian, Y. (July, January 27). Evaluation of face resolution for expression analysis. Proceedings of the Conference on Computer Vision and Pattern Recognition Workshop, Washington, DC, USA."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Tzimiropoulos, G., and Pantic, M. (2013, January 1\u20138). Optimization problems for fast aam fitting in-the-wild. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.79"},{"key":"ref_18","unstructured":"Zhu, X., and Ramanan, D. (2012, January 16\u201321). Face detection, pose estimation, and landmark localization in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, USA."},{"key":"ref_19","first-page":"59","article-title":"Thirteen ways to look at the correlation coefficient","volume":"42","author":"Nicewander","year":"1988","journal-title":"Am. Stat."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Tariq, U., Lin, K.H., Li, Z., Zhou, X., Wang, Z., Le, V., Huang, T.S., Lv, X., and Han, T.X. (2011, January 21\u201325). Emotion recognition from an ensemble of features. Proceedings of the IEEE International Conference on Automatic Face & Gesture Recognition and Workshops (FG 2011), Santa Barbara, CA, USA.","DOI":"10.1109\/FG.2011.5771365"},{"key":"ref_21","first-page":"41","article-title":"Multi-task feature learning","volume":"19","author":"Evgeniou","year":"2007","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_22","unstructured":"Ojala, T., Pietikainen, M., and Harwood, D. (1994, January 9\u201313). Performance evaluation of texture measures with classification based on Kullback discrimination of distributions. Proceedings of the 12th IAPR International Conference Pattern Recognit. Conference A: Computer Vision & Image Processing, Jerusalem, Israel."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/0031-3203(95)00067-4","article-title":"A comparative study of texture measures with classification based on featured distributions","volume":"29","author":"Ojala","year":"1996","journal-title":"Pattern Recognit."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1109\/TPAMI.2006.68","article-title":"A texture-based method for modeling the background and detecting moving objects","volume":"28","author":"Heikkila","year":"2006","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_25","unstructured":"Wikipedia (2017, January 23). Histogram of Oriented Gradients. Available online: https:\/\/en.wikipedia.org\/wiki\/Histogram_of_oriented_gradients."},{"key":"ref_26","unstructured":"Dalal, N., and Triggs, B. (2005, January 20\u201325). Histograms of oriented gradients for human detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3191","DOI":"10.1016\/j.patcog.2015.04.012","article-title":"Multimodal learning for facial expression recognition","volume":"48","author":"Zhang","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1080\/14786440109462720","article-title":"On Lines and Planes of Closest Fit to System of Points in Space","volume":"2","author":"Person","year":"1901","journal-title":"Philiosoph. Mag."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1037\/h0071325","article-title":"Analysis of a complex of statistical variables into principal components","volume":"24","author":"Hotelling","year":"1933","journal-title":"J. Educ. Psychol."},{"key":"ref_30","unstructured":"Kanade, T., Cohn, J.F., and Tian, Y. (2000, January 26\u201330). Comprehensive database for facial expression analysis. Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition, Grenoble, France."},{"key":"ref_31","unstructured":"Lyons, M., Akamatsu, S., Kamachi, M., and Gyoba, J. (1998, January 14\u201316). Coding facial expressions with gabor wavelets. Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1357","DOI":"10.1109\/34.817413","article-title":"Automatic classification of single facial images","volume":"21","author":"Lyons","year":"1999","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1145\/1961189.1961199","article-title":"LIBSVM: A library for support vector machines","volume":"2","author":"Chang","year":"2011","journal-title":"ACM Trans. Intell. Syst. Technol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/4\/712\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:31:34Z","timestamp":1760207494000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/17\/4\/712"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,3,29]]},"references-count":33,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2017,4]]}},"alternative-id":["s17040712"],"URL":"https:\/\/doi.org\/10.3390\/s17040712","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2017,3,29]]}}}