{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,30]],"date-time":"2026-03-30T12:41:26Z","timestamp":1774874486083,"version":"3.50.1"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"20","license":[{"start":{"date-parts":[[2023,4,1]],"date-time":"2023-04-01T00:00:00Z","timestamp":1680307200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,4,1]],"date-time":"2023-04-01T00:00:00Z","timestamp":1680307200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100009827","name":"Alexandria University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100009827","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2023,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Automatic facial expression recognition (AFER), sometimes referred to as emotional recognition, is important for socializing. Automatic methods in the past two years faced challenges due to Covid-19 and the vital wearing of a mask. Machine learning techniques tremendously increase the amount of data processed and achieved good results in such AFER to detect emotions; however, those techniques are not designed for masked faces and thus achieved poor recognition. This paper introduces a hybrid convolutional neural network aided by a local binary pattern to extract features in an accurate way, especially for masked faces. The basic seven emotions classified into anger, happiness, sadness, surprise, contempt, disgust, and fear are to be recognized. The proposed method is applied on two datasets: the first represents CK and CK +, while the second represents M-LFW-FER. Obtained results show that emotion recognition with a face mask achieved an accuracy of 70.76% on three emotions. Results are compared to existing techniques and show significant improvement.<\/jats:p>","DOI":"10.1007\/s00521-023-08498-w","type":"journal-article","created":{"date-parts":[[2023,4,3]],"date-time":"2023-04-03T05:57:57Z","timestamp":1680501477000},"page":"14963-14972","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["An automatic improved facial expression recognition for masked faces"],"prefix":"10.1007","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7332-858X","authenticated-orcid":false,"given":"Yasmeen","family":"ELsayed","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ashraf","family":"ELSayed","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohamed A.","family":"Abdou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,4,1]]},"reference":[{"key":"8498_CR1","unstructured":"Yang B, Wu J, Hattori G (2021) Occlusion aware facial landmark detection based facial Ex- pression recognition with face mask"},{"key":"8498_CR2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0226211","author":"J Song","year":"2019","unstructured":"Song J, Wei Y, Ke H (2019) The effect of emotional information from eyes on empathy for pain: A subliminal ERP study. PLoS ONE. https:\/\/doi.org\/10.1371\/journal.pone.0226211","journal-title":"PLoS ONE"},{"key":"8498_CR3","doi-asserted-by":"publisher","unstructured":"Manfred S (2020) Masked education? The benefits and burdens of wearing face masks in schools during the current Corona pandemic. Trends in Neuroscience and Education. https:\/\/doi.org\/10.1016\/j.tine.2020.100138","DOI":"10.1016\/j.tine.2020.100138"},{"key":"8498_CR4","doi-asserted-by":"publisher","unstructured":"Fuzail K (2019) Facial expression recognition using facial landmark detection and feature extraction via neural networks. arXiv preprint arXiv. https:\/\/doi.org\/10.48550\/arXiv.1812.04510","DOI":"10.48550\/arXiv.1812.04510"},{"issue":"10","key":"8498_CR5","doi-asserted-by":"publisher","first-page":"e0186027","DOI":"10.1371\/journal.pone.0186027","volume":"12","author":"S Iacozza","year":"2017","unstructured":"Iacozza S, Costa A, Dun\u02dcabeitia JA (2017) What do your eyes reveal about your foreign language? Reading emotional sentences in a native and foreign language. IPloS one. 12(10):e0186027. https:\/\/doi.org\/10.1371\/journal.pone.0186027","journal-title":"IPloS one."},{"key":"8498_CR6","doi-asserted-by":"publisher","DOI":"10.1037\/1528-3542.3.2.150","author":"G Horstmann","year":"2003","unstructured":"Horstmann G (2003) What do facial expressions convey: Feeling states, behavioral intentions, or action requests? Emotion. https:\/\/doi.org\/10.1037\/1528-3542.3.2.150","journal-title":"Emotion"},{"key":"8498_CR7","doi-asserted-by":"crossref","unstructured":"Islam Chowdhury A, and Munem Shahriar M, Islam A, Ahmed Karim EA, and Rezwanul Islam M (2020) An automated system in atm booth using face encoding and emotion recognition process","DOI":"10.1145\/3421558.3421567"},{"key":"8498_CR8","volume-title":"Introduction to convolutional neural networks","author":"J Wu","year":"2017","unstructured":"Wu J (2017) Introduction to convolutional neural networks. Nan- jing Univ China, National Key Lab for Novel Software Technology"},{"issue":"11","key":"8498_CR9","doi-asserted-by":"publisher","first-page":"3212","DOI":"10.1109\/TNNLS.2018.2876865","volume":"30","author":"ZQ Zhao","year":"2019","unstructured":"Zhao ZQ, Zheng P, Xu ST, Wu X (2019) Object detection with deep learning: a review. IEEE Trans Neural Netw Learn Syst. 30(11):3212\u20133232","journal-title":"IEEE Trans Neural Netw Learn Syst."},{"key":"8498_CR10","doi-asserted-by":"publisher","unstructured":"Ahmed TU, Hossain S, Hossain MS, Ul Islam R, Andersson K (2019) Facial expression recognition using convolutional neural network with data augmentation. IEEE, 336\u2013341. https:\/\/doi.org\/10.1109\/ICIEV.2019.8858529","DOI":"10.1109\/ICIEV.2019.8858529"},{"key":"8498_CR11","doi-asserted-by":"publisher","DOI":"10.1016\/j.measurement.2019.107357","author":"T He","year":"2020","unstructured":"He T, Liu Y, Yu Y, Zhao Q, Hu Z (2020) Application of deep convolutional neural network on feature extraction and detection of wood defects. Measurement. https:\/\/doi.org\/10.1016\/j.measurement.2019.107357","journal-title":"Measurement"},{"key":"8498_CR12","doi-asserted-by":"crossref","unstructured":"Kumar G, Bhatia PK (2014) A detailed review of feature extraction in image processing systems. IEEE, 5\u201312","DOI":"10.1109\/ACCT.2014.74"},{"key":"8498_CR13","unstructured":"Chatterjee S (2020) What is feature extraction? feature extraction in image processing"},{"key":"8498_CR14","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-01604-7_21","volume-title":"Feature selection for enhanced 3D facial expression recognition based on varying feature point distance","author":"K Yurtkan","year":"2013","unstructured":"Yurtkan K, Soyel H, Demirel H (2013) Feature selection for enhanced 3D facial expression recognition based on varying feature point distance. Springer"},{"key":"8498_CR15","doi-asserted-by":"crossref","unstructured":"Ekweariri AN, and Yurtkan K (2017) Facial expression recognition using enhanced local binary patterns. IEEE","DOI":"10.1109\/CICN.2017.8319353"},{"key":"8498_CR16","doi-asserted-by":"publisher","DOI":"10.1109\/TAFFC.2014.2386334","author":"SL Happy","year":"2014","unstructured":"Happy SL, Routray A (2014) Automatic facial expression recognition using features of salient facial patches. IEEE Trans Affect Comput. https:\/\/doi.org\/10.1109\/TAFFC.2014.2386334","journal-title":"IEEE Trans Affect Comput"},{"issue":"6","key":"8498_CR17","first-page":"2273","volume":"5","author":"S Sawardekar","year":"2018","unstructured":"Sawardekar S, Naik SR (2018) Facial expression recognition using efficient LBP and CNN. Int Res J Eng Technol (IRJET). 5(6):2273\u20132277","journal-title":"Int Res J Eng Technol (IRJET)."},{"key":"8498_CR18","doi-asserted-by":"publisher","DOI":"10.1155\/2021\/8828245","author":"B Niu","year":"2021","unstructured":"Niu B, Gao Z, Guo B (2021) Facial expression recognition with LBP and ORB features. Comput Intell Neurosci. https:\/\/doi.org\/10.1155\/2021\/8828245","journal-title":"Comput Intell Neurosci"},{"key":"8498_CR19","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2020.2972114","author":"F Zhang","year":"2020","unstructured":"Zhang F, Zhang T, Mao Q, Xu C (2020) Geometry guided poseinvariant facial expression recognition. IEEE Trans Image Process. https:\/\/doi.org\/10.1109\/TIP.2020.2972114","journal-title":"IEEE Trans Image Process"},{"key":"8498_CR20","doi-asserted-by":"publisher","unstructured":"Al-agha LSA, Saleh PHH, and Ghani PRF (2017) Geometric-based feature extraction and classification for emotion expressions of 3D video film. J Adv Inform Technol, https:\/\/doi.org\/10.12720\/jait.8.2.74-79","DOI":"10.12720\/jait.8.2.74-79"},{"key":"8498_CR21","doi-asserted-by":"publisher","first-page":"42532","DOI":"10.1109\/ACCESS.2018.2858278","volume":"6","author":"Y Tang","year":"2018","unstructured":"Tang Y, Zhang XM, Wang H (2018) Geometric-convolutional feature fusion based on learning propagation for facial expression recognition. IEEE Access. 6:42532\u201342540","journal-title":"IEEE Access."},{"key":"8498_CR22","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-019-04242-5","author":"W Wan","year":"2019","unstructured":"Wan W, Gao Y, Lee HJ (2019) Transfer deep feature learning for face sketch recognition. Neural Comput Appl. https:\/\/doi.org\/10.1007\/s00521-019-04242-5","journal-title":"Neural Comput Appl"},{"key":"8498_CR23","doi-asserted-by":"publisher","unstructured":"Yang B, Wu J, and Hat-tori G (2020) Facial expression recognition with the advent of face masks. https:\/\/doi.org\/10.1109\/ICIEV.2019.8858529","DOI":"10.1109\/ICIEV.2019.8858529"},{"key":"8498_CR24","unstructured":"Huang GB, Mattar M, Berg, T and Learned-Miller E (2008) Labeled faces in the wild: a database for studying face recognition in unconstrained environments"},{"key":"8498_CR25","doi-asserted-by":"publisher","unstructured":"Wang Z, Wang G, Huang B, Xiong Z, Hong Q, Wu H, Yi P, Jiang K, Wang N, and Pei Y others (2020) Masked face recognition dataset and application, arXiv preprint. https:\/\/doi.org\/10.48550\/arXiv.2003.09093","DOI":"10.48550\/arXiv.2003.09093"},{"key":"8498_CR26","unstructured":"Zhang, Ligang and Verma, Brijesh and"},{"key":"8498_CR27","doi-asserted-by":"publisher","DOI":"10.1145\/3158369","author":"D Tjondronegoro","year":"2018","unstructured":"Tjondronegoro D, Chandran V (2018) Facial expression analysis under partial occlusion: a survey. ACM Comput Surv (CSUR). https:\/\/doi.org\/10.1145\/3158369","journal-title":"ACM Comput Surv (CSUR)"},{"key":"8498_CR28","doi-asserted-by":"crossref","unstructured":"Yang B, Jianming W, and Hat-tori G (2021) Face mask aware robust facial expression recognition during the COVID-19 pandemic, IEEE 240\u2013244","DOI":"10.1109\/ICIP42928.2021.9506047"},{"key":"8498_CR29","doi-asserted-by":"publisher","unstructured":"Local binary patterns: a comprehensive study. Image and vision Computing.https:\/\/doi.org\/10.1016\/j.imavis.2008.08.005","DOI":"10.1016\/j.imavis.2008.08.005"},{"key":"8498_CR30","doi-asserted-by":"publisher","unstructured":"Pramerdorfer, Christopher and Kampel, Martin (2016)Facial expression recognition using convolutional neural networks: state of the art. arXiv preprint arXiv. https:\/\/doi.org\/10.48550\/arXiv.1612.02903","DOI":"10.48550\/arXiv.1612.02903"},{"issue":"11","key":"8498_CR31","doi-asserted-by":"publisher","first-page":"1892","DOI":"10.3390\/electronics9111892","volume":"9","author":"S Porcu","year":"2020","unstructured":"Porcu S, Floris A, Atzori L (2020) Evaluation of data augmentation techniques for facial expression recognition systems. Electronics 9(11):1892","journal-title":"Electronics"},{"issue":"10","key":"8498_CR32","doi-asserted-by":"publisher","first-page":"2409","DOI":"10.1109\/TIFS.2018.2800901","volume":"13","author":"H Zhang","year":"2018","unstructured":"Zhang H, Li Q, Sun Z, Liu Y (2018) Combining data-driven and model-driven methods for robust facial landmark detection. IEEE Trans Inform Forens Secur. 13(10):2409\u20132422. https:\/\/doi.org\/10.1109\/TIFS.2018.2800901","journal-title":"IEEE Trans Inform Forens Secur."},{"key":"8498_CR33","first-page":"1755","volume":"10","author":"DE King","year":"2009","unstructured":"King DE (2009) Dlib-ml: A machine learning toolkit. J Mach Learn Res 10:1755\u20131758","journal-title":"J Mach Learn Res"},{"issue":"6","key":"8498_CR34","doi-asserted-by":"publisher","first-page":"803","DOI":"10.1016\/j.imavis.2008.08.005","volume":"27","author":"C Shan","year":"2009","unstructured":"Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27(6):803\u2013816","journal-title":"Image Vis Comput"},{"key":"8498_CR35","doi-asserted-by":"publisher","unstructured":"Image and vision computing. https:\/\/doi.org\/10.1016\/j.imavis.2008.08.005","DOI":"10.1016\/j.imavis.2008.08.005"},{"key":"8498_CR36","doi-asserted-by":"crossref","unstructured":"Boyko N, Basystiuk O, Shakhovska N (2018) Performance evaluation and comparison of software for face recognition, based on Dlib and OpenCV library. IEEE, 478\u2013482","DOI":"10.1109\/DSMP.2018.8478556"},{"issue":"2","key":"8498_CR37","doi-asserted-by":"publisher","first-page":"45","DOI":"10.14203\/j.inkom.420","volume":"9","author":"E Prakasa","year":"2016","unstructured":"Prakasa E (2016) Texture feature extraction by using local binary pattern. INKOM Journal. 9(2):45\u201348. https:\/\/doi.org\/10.14203\/j.inkom.420","journal-title":"INKOM Journal."},{"issue":"6","key":"8498_CR38","doi-asserted-by":"publisher","first-page":"765","DOI":"10.1109\/TSMCC.2011.2118750","volume":"41","author":"D Huang","year":"2011","unstructured":"Huang D, Shan C, Ardabilian M, Wang Y, Chen L (2011) Local binary patterns and its application to facial image analysis: a survey. IEEE Trans Syst Man Cyber Part C Appl Rev. 41(6):765\u2013781","journal-title":"IEEE Trans Syst Man Cyber Part C Appl Rev."},{"key":"8498_CR39","doi-asserted-by":"publisher","DOI":"10.1109\/TGRS.2014.2381602","author":"W Li","year":"2015","unstructured":"Li W, Chen C, Su H, Du Q (2015) Local binary patterns and extreme learning machine for hyperspectral imagery classification. IEEE Trans Geosci Remote Sens. https:\/\/doi.org\/10.1109\/TGRS.2014.2381602","journal-title":"IEEE Trans Geosci Remote Sens"},{"key":"8498_CR40","doi-asserted-by":"publisher","unstructured":"Saravanan A, Perichetla G, Gayathri DKS (2019) Facial Emotion Recognition using Convolutional Neural Networks. arXiv preprint arXiv:1910.05602. https:\/\/doi.org\/10.48550\/arXiv.1910.05602","DOI":"10.48550\/arXiv.1910.05602"},{"key":"8498_CR41","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2021.627561","author":"W Georg","year":"2021","unstructured":"Georg W et al (2021) Automatic facial expression recognition in stan2ardized and non-standardized emotional expressions. Front Psychol. https:\/\/doi.org\/10.3389\/fpsyg.2021.627561","journal-title":"Front Psychol"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-023-08498-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-023-08498-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-023-08498-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,2]],"date-time":"2023-06-02T10:03:21Z","timestamp":1685700201000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-023-08498-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,1]]},"references-count":41,"journal-issue":{"issue":"20","published-print":{"date-parts":[[2023,7]]}},"alternative-id":["8498"],"URL":"https:\/\/doi.org\/10.1007\/s00521-023-08498-w","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,1]]},"assertion":[{"value":"12 August 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 March 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 April 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Authors declare that there is neither funding nor conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}