{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,12]],"date-time":"2026-01-12T06:59:19Z","timestamp":1768201159482,"version":"3.49.0"},"reference-count":49,"publisher":"MDPI AG","issue":"20","license":[{"start":{"date-parts":[[2023,10,18]],"date-time":"2023-10-18T00:00:00Z","timestamp":1697587200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61602540"],"award-info":[{"award-number":["61602540"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>With the accomplishment of deep neural networks, face recognition methods have achieved great success in research and are now being applied at a human level. However, existing face recognition models fail to achieve state-of-the-art performance in recognizing occluded face images, which are common scenarios captured in the real world. One of the potential reasons for this is the lack of large-scale training datasets, requiring labour-intensive and costly labelling of the occlusions. To resolve these issues, we propose an Adversarially Learning Occlusions by Backpropagation (ALOB) model, a simple yet powerful double-network framework used to mitigate manual labelling by contrastively learning the corrupted features against personal identity labels, thereby maximizing the loss. To investigate the performance of the proposed method, we compared our model to the existing state-of-the-art methods, which function under the supervision of occlusion learning, in various experiments. Extensive experimentation on LFW, AR, MFR2, and other synthetic masked or occluded datasets confirmed the effectiveness of the proposed model in occluded face recognition by sustaining better results in terms of masked face recognition and general face recognition. For the AR datasets, the ALOB model outperformed other advanced methods by obtaining a 100% recognition rate for images with sunglasses (protocols 1 and 2). We also achieved the highest accuracies of 94.87%, 92.05%, 78.93%, and 71.57% TAR@FAR = 1 \u00d7 10\u22123 in LFW-OCC-2.0 and LFW-OCC-3.0, respectively. Furthermore, the proposed method generalizes well in terms of FR and MFR, yielding superior results in three datasets, LFW, LFW-Masked, and MFR2, and producing accuracies of 98.77%, 97.62%, and 93.76%, respectively.<\/jats:p>","DOI":"10.3390\/s23208559","type":"journal-article","created":{"date-parts":[[2023,10,18]],"date-time":"2023-10-18T10:36:56Z","timestamp":1697625416000},"page":"8559","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Adversarially Learning Occlusions by Backpropagation for Face Recognition"],"prefix":"10.3390","volume":"23","author":[{"given":"Caijie","family":"Zhao","sequence":"first","affiliation":[{"name":"PAMI Research Group, Department of Computer and Information Science, University of Macau, Taipa 999078, Macau SAR, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-7335-4962","authenticated-orcid":false,"given":"Ying","family":"Qin","sequence":"additional","affiliation":[{"name":"PAMI Research Group, Department of Computer and Information Science, University of Macau, Taipa 999078, Macau SAR, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2497-9519","authenticated-orcid":false,"given":"Bob","family":"Zhang","sequence":"additional","affiliation":[{"name":"PAMI Research Group, Department of Computer and Information Science, University of Macau, Taipa 999078, Macau SAR, China"},{"name":"Centre for Artificial Intelligence and Robotics, Institute of Collaborative Innovation, University of Macau, Taipa 999078, Macau SAR, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,10,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Deng, J.K., Guo, J., Xue, N.N., and Zafeiriou, S. (2019, January 16\u201320). ArcFace: Additive Angular Margin Loss for Deep Face Recognition. Proceedings of the 2019 IEEE\/Cvf Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00482"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wang, H., Wang, Y.T., Zhou, Z., Ji, X., Gong, D.H., Zhou, J.C., Li, Z., and Liu, W. (2018, January 18\u201323). CosFace: Large Margin Cosine Loss for Deep Face Recognition. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00552"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Liu, W.Y., Wen, Y.D., Yu, Z.D., Li, M., Raj, B., and Song, L. (2016, January 21\u201326). SphereFace: Deep Hypersphere Embedding for Face Recognition. Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2017), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.713"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Guo, Y., Zhang, L., Hu, Y., He, X., and Gao, J. (2016, January 11\u201314). MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition. Proceedings of the European Conference on Computer Vision, Amstedrdam, The Netherlands.","DOI":"10.1007\/978-3-319-46487-9_6"},{"key":"ref_7","unstructured":"Huang, G.B., Mattar, M., Berg, T., and Learned-Miller, E. (2008, January 25). Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. Proceedings of the Workshop on faces in \u2018Real-Life\u2019 Images: Detection, Alignment, and Recognition, Marseille, France."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Kim, J.H., Poulose, A., and Han, D.S. (2021). The Extensive Usage of the Facial Image Threshing Machine for Facial Emotion Recognition Performance. Sensors, 21.","DOI":"10.3390\/s21062026"},{"key":"ref_9","first-page":"1180","article-title":"Unsupervised Domain Adaptation by Backpropagation","volume":"37","author":"Ganin","year":"2015","journal-title":"Proc. Mach. Learn. Res."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"6939","DOI":"10.1109\/TPAMI.2021.3098962","article-title":"End2End Occluded Face Recognition by Masking Corrupted Features","volume":"44","author":"Qiu","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","unstructured":"Song, L., Gong, D., Li, Z., Liu, C., and Liu, W. (November, January 27). Occlusion Robust Face Recognition Based on Mask Learning With Pairwise Differential Siamese Network. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yin, B., Tran, L., Li, H., Shen, X., and Liu, X. (2019, January 16\u201320). Towards Interpretable Face Recognition. Proceedings of the 2019 IEEE\/Cvf Conference on Computer Vision and Pattern Recognition (CVPR 2019), Long Beach, CA, USA.","DOI":"10.1109\/ICCV.2019.00944"},{"key":"ref_13","first-page":"3197","article-title":"MSML: Enhancing Occlusion-Robustness by Multi-Scale Segmentation-Based Mask Learning for Face Recognition","volume":"36","author":"Yuan","year":"2022","journal-title":"Proc. Conf. AAAI Artif. Intell."},{"key":"ref_14","unstructured":"Anwar, A., and Raychowdhury, A. (2020). Masked face recognition for secure authentication. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ding, F.F., Peng, P.X., Huang, Y.R., Geng, M.Y., and Tian, Y.H. (2020, January 12\u201316). Masked Face Recognition with Latent Part Detection. Proceedings of the Mm\u201920: 28th ACM International Conference on Multimedia, Seattle, WA, USA.","DOI":"10.1145\/3394171.3413731"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"108522","DOI":"10.1016\/j.patcog.2022.108522","article-title":"Learning upper patch attention using dual-branch training strategy for masked face recognition","volume":"126","author":"Zhang","year":"2022","journal-title":"Pattern Recognit."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3012","DOI":"10.1007\/s10489-020-02100-9","article-title":"Cropping and attention based approach for masked face recognition","volume":"51","author":"Li","year":"2021","journal-title":"Appl. Intell."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Wan, W., and Chen, J. (2017, January 17\u201320). Occlusion robust face recognition based on mask learning. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China.","DOI":"10.1109\/ICIP.2017.8296992"},{"key":"ref_19","first-page":"1","article-title":"Understanding Deep Learning Techniques for Recognition of Human Emotions Using Facial Expressions: A Comprehensive Survey","volume":"72","author":"Karnati","year":"2023","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Qin, Q., Hu, W., and Liu, B. (2020, January 5\u201310). Feature projection for improved text classification. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.726"},{"key":"ref_21","unstructured":"Martinez, A., and Benavente, R. (1998). The AR Face Database: CVC Technical Report, Purdue University."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1109\/TPAMI.2008.79","article-title":"Robust Face Recognition via Sparse Representation","volume":"31","author":"Wright","year":"2008","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yang, M., Zhang, L., Yang, J., and Zhang, D. (2011, January 20\u201325). Robust Sparse Coding for Face Recognition. Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Colorado Springs, CO, USA.","DOI":"10.1109\/CVPR.2011.5995393"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1889","DOI":"10.1109\/TIP.2013.2237920","article-title":"Structured Sparse Error Coding for Face Recognition with Occlusion","volume":"22","author":"Li","year":"2013","journal-title":"IEEE Trans. Image Process."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1109\/TCYB.2016.2529300","article-title":"Largest Matching Areas for Illumination and Occlusion Robust Face Recognition","volume":"47","author":"McLaughlin","year":"2016","journal-title":"IEEE Trans. Cybern."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1163","DOI":"10.1109\/TIP.2016.2515987","article-title":"Robust Point Set Matching for Partial Face Recognition","volume":"25","author":"Weng","year":"2016","journal-title":"IEEE Trans. Image Process."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Chen, W., and Gao, Y. (2010, January 5\u201311). Recognizing Partially Occluded Faces from a Single Sample Per Class Using String-Based Matching. Proceedings of the Computer Vision\u2013ECCV 2010: 11th European Conference on Computer Vision, Heraklion, Crete, Greece.","DOI":"10.1007\/978-3-642-15558-1_36"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1016\/j.engappai.2017.06.018","article-title":"A membrane-inspired bat algorithm to recognize faces in unconstrained scenarios","volume":"64","author":"Alsalibi","year":"2017","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3620","DOI":"10.1109\/TNNLS.2019.2945372","article-title":"Laplacian-Uniform Mixture-Driven Iterative Robust Coding with Applications to Face Recognition Against Dense Errors","volume":"31","author":"Zheng","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1109\/TIP.2017.2771408","article-title":"Robust LSTM-Autoencoders for Face De-Occlusion in the Wild","volume":"27","author":"Zhao","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"104052","DOI":"10.1016\/j.engappai.2020.104052","article-title":"Deep softmax collaborative representation for robust degraded face recognition","volume":"97","author":"Vo","year":"2020","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"105814","DOI":"10.1016\/j.engappai.2022.105814","article-title":"RFA-Net: Residual feature attention network for fine-grained image inpainting","volume":"119","author":"Chen","year":"2023","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"3387","DOI":"10.1109\/TCSVT.2020.2967754","article-title":"Occluded Face Recognition in the Wild by Identity-Diversity Inpainting","volume":"30","author":"Ge","year":"2020","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"5497","DOI":"10.1007\/s10489-021-02728-1","article-title":"Masked face recognition with convolutional neural networks and local binary patterns","volume":"52","author":"Vu","year":"2021","journal-title":"Appl. Intell."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"35162","DOI":"10.1109\/ACCESS.2022.3163565","article-title":"Learning Discriminant Spatial Features with Deep Graph-Based Convolutions for Occluded Face Detection","volume":"10","author":"Albalas","year":"2022","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Hoffer, E., and Ailon, N. (2015, January 12\u201314). Deep Metric Learning Using Triplet Network. Proceedings of the Similarity-Based Pattern Recognition: Third International Workshop, SIMBAD 2015, Copenhagen, Denmark.","DOI":"10.1007\/978-3-319-24261-3_7"},{"key":"ref_37","unstructured":"Chopra, S., Hadsell, R., and LeCun, Y. (2005, January 20\u201325). Learning a similarity metric discriminatively, with application to face verification. Proceedings of the 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Diego, CA, USA."},{"key":"ref_38","unstructured":"Yi, D., Lei, Z., Liao, S., and Li, S.Z. (2014). Learning face representation from scratch. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Ghazi, M.M., and Ekenel, H.K. (July, January 26). A Comprehensive Analysis of Deep Learning Based Representation for Face Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Las Vegas, NV, USA.","DOI":"10.1109\/CVPRW.2016.20"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Li, C., Ge, S., Zhang, D., and Li, J. (2020, January 12\u201316). Look Through Masks: Towards Masked Face Recognition with De-Occlusion Distillation. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.","DOI":"10.1145\/3394171.3413960"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1016\/j.ins.2021.10.005","article-title":"A survey on facial emotion recognition techniques: A state-of-the-art literature review","volume":"582","author":"Canal","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2884","DOI":"10.1109\/TIFS.2018.2833032","article-title":"A Light CNN for Deep Face Representation With Noisy Labels","volume":"13","author":"Wu","year":"2018","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Dosi, M., Chiranjeev, M., Agarwal, S., Chaudhary, J., Manchanda, S., Balutia, K., Bhagwatkar, K., Vatsa, M., and Singh, R. (2023). Seg-DGDNet: Segmentation based Disguise Guided Dropout Network for Low Resolution Face Recognition. IEEE J. Sel. Top. Signal Process., 1\u201313.","DOI":"10.1109\/JSTSP.2023.3288398"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1109\/TPAMI.2016.2535218","article-title":"Nuclear Norm Based Matrix Regression with Applications to Face Recognition with Occlusion and Illumination Changes","volume":"39","author":"Yang","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Weng, R.L., Lu, J.W., Hu, J.L., Yang, G., and Tan, Y.P. (2013, January 1\u20138). Robust Feature Set Matching for Partial Face Recognition. Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV), Sydney, Australia.","DOI":"10.1109\/ICCV.2013.80"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"900","DOI":"10.1109\/TNNLS.2013.2245340","article-title":"Robust Kernel Representation With Statistical Local Features for Face Recognition","volume":"24","author":"Yang","year":"2013","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"He, L.X., Li, H.Q., Zhang, Q., and Sun, Z.N. (2018, January 18\u201323). Dynamic Feature Learning for Partial Face Recognition. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00737"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. (2017, January 22\u201329). Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.74"},{"key":"ref_49","first-page":"2579","article-title":"Visualizing Data using t-SNE","volume":"9","author":"Hinton","year":"2008","journal-title":"J. Mach. Learn. Res."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/20\/8559\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:09:13Z","timestamp":1760130553000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/20\/8559"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,18]]},"references-count":49,"journal-issue":{"issue":"20","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["s23208559"],"URL":"https:\/\/doi.org\/10.3390\/s23208559","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,18]]}}}