{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T09:00:34Z","timestamp":1777539634317,"version":"3.51.4"},"reference-count":63,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2022,10,22]],"date-time":"2022-10-22T00:00:00Z","timestamp":1666396800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2020YFC2003500"],"award-info":[{"award-number":["2020YFC2003500"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In recent years, the focus of facial expression recognition (FER) has gradually shifted from laboratory settings to challenging natural scenes. This requires a great deal of real-world facial expression data. However, most existing real-world databases are based on European-American cultures, and only one is for Asian cultures. This is mainly because the data on European-American expressions are more readily accessed and publicly available online. Owing to the diversity of huge data, FER in European-American cultures has recently developed rapidly. In contrast, the development of FER in Asian cultures is limited by the data. To narrow this gap, we construct a challenging real-world East Asian facial expression (EAFE) database, which contains 10,000 images collected from 113 Chinese, Japanese, and Korean movies and five search engines. We apply three neural network baselines including VGG-16, ResNet-50, and Inception-V3 to classify the images in EAFE. Then, we conduct two sets of experiments to find the optimal learning rate schedule and loss function. Finally, by training with the cosine learning rate schedule and island loss, ResNet-50 can achieve the best accuracy of 80.53% on the testing set, proving that the database is challenging. In addition, we used the Microsoft Cognitive Face API to extract facial attributes in EAFE, so that the database can also be used for facial recognition and attribute analysis. The release of the EAFE can encourage more research on Asian FER in natural scenes and can also promote the development of FER in cross-cultural domains.<\/jats:p>","DOI":"10.3390\/s22218089","type":"journal-article","created":{"date-parts":[[2022,10,24]],"date-time":"2022-10-24T10:09:23Z","timestamp":1666606163000},"page":"8089","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Towards East Asian Facial Expression Recognition in the Real World: A New Database and Deep Recognition Baseline"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8880-6724","authenticated-orcid":false,"given":"Shanshan","family":"Li","sequence":"first","affiliation":[{"name":"School of Mathematics and Statistics, Shandong University, Weihai 264209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Liang","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Mathematics and Statistics, Shandong University, Weihai 264209, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianya","family":"Liu","sequence":"additional","affiliation":[{"name":"Data Science Institute, Shandong University, Jinan 250100, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Minaee, S., Minaei, M., and Abdolrashidi, A. (2021). Deep-emotion: Facial expression recognition using attentional convolutional network. Sensors, 21.","DOI":"10.3390\/s21093046"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1016\/j.ins.2021.10.005","article-title":"A survey on facial emotion recognition techniques: A state-of-the-art literature review","volume":"582","author":"Canal","year":"2022","journal-title":"Inf. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/TAFFC.2017.2740923","article-title":"Affectnet: A database for facial expression, valence, and arousal computing in the wild","volume":"10","author":"Mollahosseini","year":"2017","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Fabian Benitez-Quiroz, C., Srinivasan, R., and Martinez, A.M. (2016, January 27\u201330). Emotionet: An accurate, real-time algorithm for the automatic annotation of a million facial expressions in the wild. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.600"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Li, S., Deng, W., and Du, J. (2017, January 21\u201326). Reliable crowdsourcing and deep locality-preserving learning for expression recognition in the wild. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.277"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, T., Pu, T., Wu, H., Xie, Y., Liu, L., and Lin, L. (2021). Cross-domain facial expression recognition: A unified evaluation benchmark and adversarial graph learning. IEEE Trans. Pattern Anal. Mach. Intell.","DOI":"10.1109\/TPAMI.2021.3131222"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1126\/science.164.3875.86","article-title":"Pan-cultural elements in facial displays of emotion","volume":"164","author":"Ekman","year":"1969","journal-title":"Science"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1007\/BF00990324","article-title":"The effects of language on judgments of universal facial expressions of emotion","volume":"16","author":"Matsumoto","year":"1992","journal-title":"J. Nonverbal Behav."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1543","DOI":"10.1016\/j.cub.2009.07.051","article-title":"Cultural confusions show that facial expressions are not universal","volume":"19","author":"Jack","year":"2009","journal-title":"Curr. Biol."},{"key":"ref_10","unstructured":"Lyons, M., Akamatsu, S., Kamachi, M., and Gyoba, J. (1998, January 14\u201316). Coding facial expressions with gabor wavelets. Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1037\/h0030377","article-title":"Constants across cultures in the face and emotion","volume":"17","author":"Ekman","year":"1971","journal-title":"J. Personal. Soc. Psychol."},{"key":"ref_12","unstructured":"Chen, L., and Yen, Y. (2007). Taiwanese Facial Expression Image Database, Brain Mapping Laboratory, Institute of Brain Science, National Yang-Ming University."},{"key":"ref_13","unstructured":"Lee, H.S., Park, S., Kang, B.N., Shin, J., Lee, J.Y., Je, H., Jun, B., and Kim, D. (2008, January 17\u201319). The POSTECH face database (PF07) and performance evaluation. Proceedings of the 2008 8th IEEE International Conference on Automatic Face & Gesture Recognition, Amsterdam, The Netherlands."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"682","DOI":"10.1109\/TMM.2010.2060716","article-title":"A natural visible and infrared facial expression database for expression recognition and emotion inference","volume":"12","author":"Wang","year":"2010","journal-title":"IEEE Trans. Multimed."},{"key":"ref_15","first-page":"40","article-title":"Revision of the Chinese facial affective picture system","volume":"25","author":"Gong","year":"2011","journal-title":"Chin. Ment. Health J."},{"key":"ref_16","unstructured":"Nguyen, H., Kotani, K., Chen, F., and Le, B. (November, January 28). A thermal facial emotion database and its analysis. Proceedings of the Pacific-Rim Symposium on Image and Video Technology, Guanajuato, Mexico."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"769","DOI":"10.3389\/fpsyg.2017.00769","article-title":"Development of the Korean facial emotion stimuli: Korea university facial expression collection 2nd edition","volume":"8","author":"Kim","year":"2017","journal-title":"Front. Psychol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2358","DOI":"10.3389\/fpsyg.2018.02358","article-title":"East Asian young and older adult perceptions of emotional faces from an age-and sex-fair East Asian facial expression database","volume":"9","author":"Tu","year":"2018","journal-title":"Front. Psychol."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"221","DOI":"10.2117\/psysoc.2019-A009","article-title":"Development and validation of the Kokoro Research Center (KRC) facial expression database","volume":"61","author":"Ueda","year":"2019","journal-title":"Psychologia"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yang, T., Yang, Z., Xu, G., Gao, D., Zhang, Z., Wang, H., Liu, S., Han, L., Zhu, Z., and Tian, Y. (2020). Tsinghua facial expression database\u2014A database of facial expressions in Chinese young and older women and men: Development and validation. PLoS ONE, 15.","DOI":"10.1371\/journal.pone.0231304"},{"key":"ref_21","first-page":"5826","article-title":"Video-based facial micro-expression analysis: A survey of datasets, features and algorithms","volume":"44","author":"Ben","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1424","DOI":"10.1109\/34.895976","article-title":"Automatic analysis of facial expressions: The state of the art","volume":"22","author":"Pantic","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1016\/j.neucom.2005.07.005","article-title":"A new nonlinear feature extraction method for face recognition","volume":"69","author":"Pang","year":"2006","journal-title":"Neurocomputing"},{"key":"ref_24","unstructured":"Viola, P., and Jones, M. (2001, January 8\u201314). Rapid object detection using a boosted cascade of simple features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint face detection and alignment using multitask cascaded convolutional networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Canedo, D., and Neves, A.J. (2019). Facial expression recognition using computer vision: A systematic review. Appl. Sci., 9.","DOI":"10.3390\/app9214678"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1261","DOI":"10.1016\/j.eswa.2014.08.049","article-title":"Facial emotion recognition using empirical mode decomposition","volume":"42","author":"Ali","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TAFFC.2014.2386334","article-title":"Automatic facial expression recognition using features of salient facial patches","volume":"6","author":"Happy","year":"2014","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Puthanidam, R.V., and Moh, T.S. (2018, January 5\u20137). A hybrid approach for facial expression recognition. Proceedings of the 12th International Conference on Ubiquitous Information Management and Communication, Langkawi, Malaysia.","DOI":"10.1145\/3164541.3164593"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.eswa.2017.08.013","article-title":"Enhanced Gabor (E-Gabor), Hypersphere-based normalization and Pearson General Kernel-based discriminant analysis for dimension reduction and classification of facial emotions","volume":"90","author":"Alphonse","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cossetin, M.J., Nievola, J.C., and Koerich, A.L. (2016, January 24\u201329). Facial expression recognition using a pairwise feature selection and classification approach. Proceedings of the 2016 International Joint Conference on Neural Networks (IJCNN), Vancouver, BC, Canada.","DOI":"10.1109\/IJCNN.2016.7727879"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"784","DOI":"10.4218\/etrij.10.1510.0132","article-title":"Robust facial expression recognition based on local directional pattern","volume":"32","author":"Jabid","year":"2010","journal-title":"ETRI J."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1049\/iet-cvi.2015.0273","article-title":"Extraction of informative regions of a face for facial expression recognition","volume":"10","author":"Kumar","year":"2016","journal-title":"IET Comput. Vis."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1574","DOI":"10.1109\/TMM.2014.2321113","article-title":"Prototype-based modeling for facial expression analysis","volume":"16","author":"Dahmane","year":"2014","journal-title":"IEEE Trans. Multimed."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Mehta, N., and Jadhav, S. (2016, January 12\u201313). Facial emotion recognition using log Gabor filter and PCA. Proceedings of the 2016 International Conference on Computing Communication Control and automation (ICCUBEA), Pune, India.","DOI":"10.1109\/ICCUBEA.2016.7860054"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1386","DOI":"10.1109\/TIP.2015.2405346","article-title":"Human facial expression recognition using stepwise linear discriminant analysis and hidden conditional random fields","volume":"24","author":"Siddiqi","year":"2015","journal-title":"IEEE Trans. Image Process."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1007\/s00521-014-1569-1","article-title":"A new facial expression recognition based on curvelet transform and online sequential extreme learning machine initialized with spherical clustering","volume":"27","author":"Demir","year":"2016","journal-title":"Neural Comput. Appl."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1109\/T-AFFC.2011.13","article-title":"Facial expression recognition using facial movement features","volume":"2","author":"Zhang","year":"2011","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Lv, Y., Feng, Z., and Xu, C. (2014, January 3\u20135). Facial expression recognition via deep learning. Proceedings of the 2014 International Conference on Smart Computing, Hong Kong, China.","DOI":"10.1109\/SMARTCOMP.2014.7043872"},{"key":"ref_40","unstructured":"Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Huang, R., Zhang, S., Li, T., and He, R. (2017, January 22\u201329). Beyond face rotation: Global and local perception gan for photorealistic and identity preserving frontal view synthesis. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.267"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"4389","DOI":"10.1007\/s00500-017-2634-3","article-title":"Facial expression recognition using a combination of multiple facial features and support vector machine","volume":"22","author":"Tsai","year":"2018","journal-title":"Soft Comput."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Slimani, K., Kas, M., El Merabet, Y., Messoussi, R., and Ruichek, Y. (2018, January 27\u201328). Facial emotion recognition: A comparative analysis using 22 LBP variants. Proceedings of the 2nd Mediterranean Conference on Pattern Recognition and Artificial Intelligence, Rabat, Morocco.","DOI":"10.1145\/3177148.3180092"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"104","DOI":"10.2197\/ipsjtcva.7.104","article-title":"Facial expression recognition and analysis: A comparison study of feature descriptors","volume":"7","author":"Liew","year":"2015","journal-title":"IPSJ Trans. Comput. Vis. Appl."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1007\/s11760-010-0177-5","article-title":"Automatic facial expression recognition: Feature extraction and selection","volume":"6","author":"Lajevardi","year":"2012","journal-title":"Signal Image Video Process."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Wang, Y., Li, Y., Song, Y., and Rong, X. (2019). Facial expression recognition based on random forest and convolutional neural network. Information, 10.","DOI":"10.3390\/info10120375"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Hamester, D., Barros, P., and Wermter, S. (2015, January 12\u201317). Face expression recognition with a 2-channel convolutional neural network. Proceedings of the 2015 International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland.","DOI":"10.1109\/IJCNN.2015.7280539"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/MMUL.2012.26","article-title":"Collecting large, richly annotated facial-expression databases from movies","volume":"19","author":"Dhall","year":"2012","journal-title":"IEEE Multimed."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Feng, Y., Zhang, H., Chen, H., Yuan, D., and Wang, H. (2015, January 3\u20136). Visual tracking via multi-experts combined with average hash model. Proceedings of the 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ACPR.2015.7486520"},{"key":"ref_50","unstructured":"Zauner, C. (2010). Implementation and Benchmarking of Perceptual Image Hash Functions. [Master\u2019s Thesis, Upper Austria University of Applied Sciences]."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Wang, D.z., and Liang, J.y. (2019, January 19\u201321). Research and design of theme image crawler based on difference hash algorithm. Proceedings of the IOP Conference Series: Materials Science and Engineering, Changsha, China.","DOI":"10.1088\/1757-899X\/563\/4\/042080"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"486","DOI":"10.1016\/j.procs.2015.08.011","article-title":"Facial expression recognition: A survey","volume":"58","author":"Kumari","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_53","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the inception architecture for computer vision. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"ImageNet Large Scale Visual Recognition Challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis. (IJCV)"},{"key":"ref_57","unstructured":"Brownlee, J. (2018). Better Deep Learning: Train Faster, Reduce Overfitting, and Make Better Predictions, Machine Learning Mastery."},{"key":"ref_58","unstructured":"Li, Z., and Arora, S. (2019). An exponential learning rate schedule for deep learning. arXiv."},{"key":"ref_59","first-page":"14977","article-title":"The step decay schedule: A near optimal, geometrically decaying learning rate procedure for least squares","volume":"32","author":"Ge","year":"2019","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_60","unstructured":"Loshchilov, I., and Hutter, F. (2016). Sgdr: Stochastic gradient descent with warm restarts. arXiv."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Wen, Y., Zhang, K., Li, Z., and Qiao, Y. (2016, January 8\u201316). A discriminative feature learning approach for deep face recognition. Proceedings of the EEuropean Conference on Computer Vision, Amsterdam, The Netherlands.","DOI":"10.1007\/978-3-319-46478-7_31"},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Cai, J., Meng, Z., Khan, A.S., Li, Z., O\u2019Reilly, J., and Tong, Y. (2018, January 15\u201319). Island loss for learning discriminative features in facial expression recognition. Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00051"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.patrec.2008.08.010","article-title":"An experimental comparison of performance measures for classification","volume":"30","author":"Ferri","year":"2009","journal-title":"Pattern Recognit. Lett."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8089\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:00:45Z","timestamp":1760144445000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/21\/8089"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,22]]},"references-count":63,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2022,11]]}},"alternative-id":["s22218089"],"URL":"https:\/\/doi.org\/10.3390\/s22218089","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,10,22]]}}}