{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T18:16:36Z","timestamp":1772302596850,"version":"3.50.1"},"reference-count":41,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,2,23]],"date-time":"2023-02-23T00:00:00Z","timestamp":1677110400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"German Science Foundation (DFG)","award":["TRR 169"],"award-info":[{"award-number":["TRR 169"]}]},{"name":"German Science Foundation (DFG)","award":["UIDB\/EEA\/50008\/2020"],"award-info":[{"award-number":["UIDB\/EEA\/50008\/2020"]}]},{"name":"FCT\/MCTES","award":["TRR 169"],"award-info":[{"award-number":["TRR 169"]}]},{"name":"FCT\/MCTES","award":["UIDB\/EEA\/50008\/2020"],"award-info":[{"award-number":["UIDB\/EEA\/50008\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Periocular recognition has emerged as a particularly valuable biometric identification method in challenging scenarios, such as partially occluded faces due to COVID-19 protective masks masks, in which face recognition might not be applicable. This work presents a periocular recognition framework based on deep learning, which automatically localises and analyses the most important areas in the periocular region. The main idea is to derive several parallel local branches from a neural network architecture, which in a semi-supervised manner learn the most discriminative areas in the feature map and solve the identification problem solely upon the corresponding cues. Here, each local branch learns a transformation matrix that allows for basic geometrical transformations (cropping and scaling), which is used to select a region of interest in the feature map, further analysed by a set of shared convolutional layers. Finally, the information extracted by the local branches and the main global branch are fused together for recognition. The experiments carried out on the challenging UBIRIS-v2 benchmark show that by integrating the proposed framework with various ResNet architectures, we consistently obtain an improvement in mAP of more than 4% over the \u201cvanilla\u201d architecture. In addition, extensive ablation studies were performed to better understand the behavior of the network and how the spatial transformation and the local branches influence the overall performance of the model. The proposed method can be easily adapted to other computer vision problems, which is also regarded as one of its strengths.<\/jats:p>","DOI":"10.3390\/s23052456","type":"journal-article","created":{"date-parts":[[2023,2,23]],"date-time":"2023-02-23T03:56:58Z","timestamp":1677124618000},"page":"2456","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Adaptive Spatial Transformation Networks for Periocular Recognition"],"prefix":"10.3390","volume":"23","author":[{"given":"Diana Laura","family":"Borza","sequence":"first","affiliation":[{"name":"Informatics Department, Faculty of Mathematics and Informatics, Babes Bolyai University, 1st Mihail Kogalniceanu Street, 400084 Cluj-Napoca, Romania"}]},{"given":"Ehsan","family":"Yaghoubi","sequence":"additional","affiliation":[{"name":"Department of Informatics, Hamburg University, 177 Mittelweg, 20148 Hamburg, Germany"}]},{"given":"Simone","family":"Frintrop","sequence":"additional","affiliation":[{"name":"Department of Informatics, Hamburg University, 177 Mittelweg, 20148 Hamburg, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2551-8570","authenticated-orcid":false,"given":"Hugo","family":"Proen\u00e7a","sequence":"additional","affiliation":[{"name":"IT: Instituto de Telecomunica\u00e7\u00f5es, University of Beira Interior, Marqu\u00eas de \u00c1vila e Bolama, 6201-001 Covilh\u00e3, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,23]]},"reference":[{"key":"ref_1","unstructured":"Damer, N., Grebe, J.H., Chen, C., Boutros, F., Kirchbuchner, F., and Kuijper, A. (2020, January 16\u201318). The effect of wearing a mask on face recognition performance: An exploratory study. Proceedings of the 2020 International Conference of the Biometrics Special Interest Group (BIOSIG), Darmstadt, Germany."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Woodard, D.L., Pundlik, S.J., Lyle, J.R., and Miller, P.E. (2010, January 13\u201318). Periocular region appearance cues for biometric identification. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, San Francisco, CA, USA.","DOI":"10.1109\/CVPRW.2010.5544621"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"103583","DOI":"10.1016\/j.cviu.2022.103583","article-title":"Periocular biometrics and its relevance to partially masked faces: A survey","volume":"226","author":"Sharma","year":"2023","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1109\/TIFS.2011.2173932","article-title":"Human and machine performance on periocular biometrics under near-infrared light and visible light","volume":"7","author":"Hollingsworth","year":"2011","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Park, U., Ross, A., and Jain, A.K. (2009, January 28\u201330). Periocular biometrics in the visible spectrum: A feasibility study. Proceedings of the 2009 IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems, Washington, DC, USA.","DOI":"10.1109\/BTAS.2009.5339068"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Adams, J., Woodard, D.L., Dozier, G., Miller, P., Bryant, K., and Glenn, G. (2010, January 23\u201326). Genetic-based type II feature extraction for periocular biometric recognition: Less is more. Proceedings of the 2010 20th International Conference on Pattern Recognition, Istanbul, Turkey.","DOI":"10.1109\/ICPR.2010.59"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Xu, J., Cha, M., Heyman, J.L., Venugopalan, S., Abiantun, R., and Savvides, M. (2010, January 10\u201313). Robust local binary pattern feature sets for periocular biometric identification. Proceedings of the 2010 Fourth IEEE International Conference on Biometrics: Theory, Applications and Systems (BTAS), Tampa, FL, USA.","DOI":"10.1109\/BTAS.2010.5634504"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Cao, Z.X., and Schmid, N.A. (2014, January 27\u201330). Matching heterogeneous periocular regions: Short and long standoff distances. Proceedings of the 2014 IEEE International Conference on Image Processing (ICIP), Paris, France.","DOI":"10.1109\/ICIP.2014.7026006"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Ahuja, K., Bose, A., Nagar, S., Dey, K., and Barbhuiya, F. (2016, January 25\u201328). ISURE: User authentication in mobile devices using ocular biometrics in visible spectrum. Proceedings of the 2016 IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA.","DOI":"10.1109\/ICIP.2016.7532374"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Raja, K.B., Raghavendra, R., Stokkenes, M., and Busch, C. (2015, January 19\u201322). Multi-modal authentication system for smartphones using face, iris and periocular. Proceedings of the 2015 International Conference on Biometrics (ICB), Phuket, Thailand.","DOI":"10.1109\/ICB.2015.7139044"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Mikaelyan, A., Alonso-Fernandez, F., and Bigun, J. (2014, January 23\u201327). Periocular recognition by detection of local symmetry patterns. Proceedings of the 2014 Tenth International Conference on Signal-Image Technology and Internet-Based Systems, Marrakech, Morocco.","DOI":"10.1109\/SITIS.2014.105"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1875","DOI":"10.1109\/TIFS.2015.2434271","article-title":"Probabilistic deformation models for challenging periocular image verification","volume":"10","author":"Smereka","year":"2015","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Smereka, J.M., Kumar, B.V., and Rodriguez, A. (March, January 29). Selecting discriminative regions for periocular verification. Proceedings of the 2016 IEEE International Conference on Identity, Security and Behavior Analysis (ISBA), Sendai, Japan.","DOI":"10.1109\/ISBA.2016.7477247"},{"key":"ref_14","first-page":"7961","article-title":"Region specific and subimage based neighbour gradient feature extraction for robust periocular recognition","volume":"34","author":"Ramachandra","year":"2022","journal-title":"J. King Saud Univ.-Comput. Inf. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.patcog.2016.02.014","article-title":"Stable, fast computation of high-order Zernike moments using a recursive method","volume":"56","author":"Deng","year":"2016","journal-title":"Pattern Recognit."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Abdulhussain, S.H., Mahmmod, B.M., Flusser, J., AL-Utaibi, K.A., and Sait, S.M. (2022). Fast overlapping block processing algorithm for feature extraction. Symmetry, 14.","DOI":"10.3390\/sym14040715"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.inffus.2015.03.005","article-title":"Ocular biometrics: A survey of modalities and fusion approaches","volume":"26","author":"Nigam","year":"2015","journal-title":"Inf. Fusion"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1016\/j.patrec.2015.08.026","article-title":"A survey on periocular biometrics research","volume":"82","author":"Bigun","year":"2016","journal-title":"Pattern Recognit. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1016\/j.procs.2020.03.234","article-title":"Periocular biometrics for non-ideal images: With off-the-shelf deep cnn & transfer learning approach","volume":"167","author":"Kumari","year":"2020","journal-title":"Procedia Comput. Sci."},{"key":"ref_20","first-page":"888","article-title":"Deep-prwis: Periocular recognition without the iris and sclera using deep learning frameworks","volume":"13","author":"Neves","year":"2017","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Talreja, V., Nasrabadi, N.M., and Valenti, M.C. (2022, January 4\u20138). Attribute-Based Deep Periocular Recognition: Leveraging Soft Biometrics to Improve Periocular Recognition. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Waikoloa, HI, USA.","DOI":"10.1109\/WACV51458.2022.00121"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1017","DOI":"10.1109\/TIFS.2016.2636093","article-title":"Accurate periocular recognition under less constrained environment using semantics-assisted convolutional neural network","volume":"12","author":"Zhao","year":"2016","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"866","DOI":"10.1109\/TIFS.2020.3023289","article-title":"Periocular-assisted multi-feature collaboration for dynamic iris recognition","volume":"16","author":"Wang","year":"2020","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Ng, T.S., Low, C.Y., Chai, J.C.L., and Teoh, A.B.J. (2022, January 21\u201325). Conditional Multimodal Biometrics Embedding Learning For Periocular and Face in the Wild. Proceedings of the 2022 26th International Conference on Pattern Recognition (ICPR), Montreal, QC, Canada.","DOI":"10.1109\/ICPR56361.2022.9956636"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Zou, Q., Wang, C., Yang, S., and Chen, B. (2022). A compact periocular recognition system based on deep learning framework AttenMidNet with the attention mechanism. Multimedia Tools and Applications, Springer.","DOI":"10.1007\/s11042-022-14017-1"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201323). Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00745"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Abate, A., Cimmino, L., Nappi, M., and Narducci, F. (2022, January 23\u201327). Fusion of Periocular Deep Features in a Dual-Input CNN for Biometric Recognition. Proceedings of the Image Analysis and Processing\u2014ICIAP 2022: 21st International Conference, Lecce, Italy.","DOI":"10.1007\/978-3-031-06427-2_31"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yang, K., Xu, Z., and Fei, J. (2021, January 19\u201325). Dualsanet: Dual spatial attention network for iris recognition. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Waikola, HI, USA.","DOI":"10.1109\/WACV48630.2021.00093"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Brito, J., and Proen\u00e7a, H. (2021, January 20\u201325). A Deep Adversarial Framework for Visually Explainable Periocular Recognition. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPRW53098.2021.00161"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Brito, J., and Proen\u00e7a, H. (2021). A Short Survey on Machine Learning Explainability: An Application to Periocular Recognition. Electronics, 10.","DOI":"10.3390\/electronics10151861"},{"key":"ref_31","unstructured":"Jaderberg, M., Simonyan, K., Zisserman, A., and Kavukcuoglu, K. (2015). Spatial transformer networks. Adv. Neural Inf. Process. Syst., 28."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Li, W., Zhu, X., and Gong, S. (2018, January 18\u201323). Harmonious attention network for person re-identification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00243"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Woo, S., Park, J., Lee, J.Y., and Kweon, I.S. (2018, January 8\u201314). Cbam: Convolutional block attention module. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_35","unstructured":"Hermans, A., Beyer, L., and Leibe, B. (2017). In defense of the triplet loss for person re-identification. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 7\u201312). Facenet: A unified embedding for face recognition and clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., and Fei-Fei, L. (2009, January 20\u201325). Imagenet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Proen\u00e7a, H., and Alexandre, L.A. (2005, January 6\u20138). UBIRIS: A noisy iris image database. Proceedings of the International Conference on Image Analysis and Processing, Cagliari, Italy.","DOI":"10.1007\/11553595_119"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"2872","DOI":"10.1109\/TPAMI.2021.3054775","article-title":"Deep learning for person re-identification: A survey and outlook","volume":"44","author":"Ye","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zhou, K., and Xiang, T. (2019). Torchreid: A library for deep learning person re-identification in pytorch. arXiv.","DOI":"10.1109\/ICCV.2019.00380"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2456\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:40:15Z","timestamp":1760121615000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2456"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,23]]},"references-count":41,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23052456"],"URL":"https:\/\/doi.org\/10.3390\/s23052456","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,23]]}}}