{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T17:28:17Z","timestamp":1768411697804,"version":"3.49.0"},"reference-count":46,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2021,1,19]],"date-time":"2021-01-19T00:00:00Z","timestamp":1611014400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010198","name":"Ministerio de Econom\u00eda, Industria y Competitividad, Gobierno de Espa\u00f1a","doi-asserted-by":"publisher","award":["DPI2014-55826-R"],"award-info":[{"award-number":["DPI2014-55826-R"]}],"id":[{"id":"10.13039\/501100010198","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100009473","name":"Universidad de M\u00e1laga","doi-asserted-by":"publisher","award":["postdoc contract from the I-PPIT-UMA program"],"award-info":[{"award-number":["postdoc contract from the I-PPIT-UMA program"]}],"id":[{"id":"10.13039\/100009473","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Face recognition is a technology with great potential in the field of robotics, due to its prominent role in human-robot interaction (HRI). This interaction is a keystone for the successful deployment of robots in areas requiring a customized assistance like education and healthcare, or assisting humans in everyday tasks. These unconstrained environments present additional difficulties for face recognition, extreme head pose variability being one of the most challenging. In this paper, we address this issue and make a fourfold contribution. First, it has been designed a tool for gathering an uniform distribution of head pose images from a person, which has been used to collect a new dataset of faces, both presented in this work. Then, the dataset has served as a testbed for analyzing the detrimental effects this problem has on a number of state-of-the-art methods, showing their decreased effectiveness outside a limited range of poses. Finally, we propose an optimization method to mitigate said negative effects by considering key pose samples in the recognition system\u2019s set of known faces. The conducted experiments demonstrate that this optimized set of poses significantly improves the performance of a state-of-the-art, cutting-edge system based on Multitask Cascaded Convolutional Neural Networks (MTCNNs) and ArcFace.<\/jats:p>","DOI":"10.3390\/s21020659","type":"journal-article","created":{"date-parts":[[2021,1,20]],"date-time":"2021-01-20T03:34:25Z","timestamp":1611113665000},"page":"659","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Improving the Head Pose Variation Problem in Face Recognition for Mobile Robots"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9847-8763","authenticated-orcid":false,"given":"Samuel-Felipe","family":"Baltanas","sequence":"first","affiliation":[{"name":"Machine Perception and Intelligent Robotics Group (MAPIR), Department of System Engineering and Automation, Biomedical Research Institute of Malaga (IBIMA), University of Malaga, 29071 M\u00e1laga, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9929-5309","authenticated-orcid":false,"given":"Jose-Raul","family":"Ruiz-Sarmiento","sequence":"additional","affiliation":[{"name":"Machine Perception and Intelligent Robotics Group (MAPIR), Department of System Engineering and Automation, Biomedical Research Institute of Malaga (IBIMA), University of Malaga, 29071 M\u00e1laga, Spain"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3845-3497","authenticated-orcid":false,"given":"Javier","family":"Gonzalez-Jimenez","sequence":"additional","affiliation":[{"name":"Machine Perception and Intelligent Robotics Group (MAPIR), Department of System Engineering and Automation, Biomedical Research Institute of Malaga (IBIMA), University of Malaga, 29071 M\u00e1laga, Spain"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"204","DOI":"10.1162\/PRES_a_00262","article-title":"ExCITE Project: A review of forty-two months of robotic telepresence technology evolution","volume":"25","author":"Orlandini","year":"2016","journal-title":"Presence Teleoperators Virtual Environ."},{"key":"ref_2","unstructured":"Casiddu, N., Porfirione, C., Monteri\u00f9, A., and Cavallo, F. (2019). Heterogeneous Non Obtrusive Platform to Monitor, Assist and Provide Recommendations to Elders at Home: The MoveCare Platform. Ambient Assisted Living, Springer International Publishing. Lecture Notes in Electrical Engineering."},{"key":"ref_3","first-page":"2276","article-title":"A Comprehensive Survey on Various Biometric Systems","volume":"13","author":"Sabhanayagam","year":"2018","journal-title":"Int. J. Appl. Eng. Res."},{"key":"ref_4","unstructured":"Pereira, F., Burges, C.J.C., Bottou, L., and Weinberger, K.Q. (2012). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems 25, Curran Associates, Inc."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Taigman, Y., Yang, M., Ranzato, M., and Wolf, L. (2014, January 23\u201328). DeepFace: Closing the Gap to Human-Level Performance in Face Verification. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA.","DOI":"10.1109\/CVPR.2014.220"},{"key":"ref_6","unstructured":"Huang, G.B., Mattar, M., Berg, T., and Learned-Miller, E. (2007). Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments, University of Massachusetts. Technical report."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Klare, B.F., Klein, B., Taborsky, E., Blanton, A., Cheney, J., Allen, K., Grother, P., Mah, A., Burge, M., and Jain, A.K. (2015, January 7\u201312). Pushing the frontiers of unconstrained face detection and recognition: IARPA Janus Benchmark A. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA. ISSN: 1063-6919.","DOI":"10.1109\/CVPR.2015.7298803"},{"key":"ref_8","unstructured":"Zheng, T., and Deng, W. (2018). Cross-Pose LFW: A Database for Studying Cross-Pose Face Recognition in Unconstrained Environments, Beijing University of Posts and Telecommunications. Technical Report 18-01."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zou, X., Kittler, J., and Messer, K. (2007, January 27\u201329). Illumination Invariant Face Recognition: A Survey. Proceedings of the 2007 First IEEE International Conference on Biometrics: Theory, Applications, and Systems, Crystal City, VA, USA.","DOI":"10.1109\/BTAS.2007.4401921"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Hong, S., Im, W., Ryu, J., and Yang, H.S. (2017, January 17\u201320). SSPP-DAN: Deep domain adaptation network for face recognition with single sample per person. Proceedings of the 2017 IEEE International Conference on Image Processing (ICIP), Beijing, China. ISSN: 2381-8549.","DOI":"10.1109\/ICIP.2017.8296396"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Schroff, F., Kalenichenko, D., and Philbin, J. (2015, January 7\u201312). FaceNet: A Unified Embedding for Face Recognition and Clustering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Deng, J., Guo, J., Xue, N., and Zafeiriou, S. (2019, January 15\u201320). ArcFace: Additive Angular Margin Loss for Deep Face Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00482"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wang, H., Wang, Y., Zhou, Z., Ji, X., Gong, D., Zhou, J., Li, Z., and Liu, W. (2018, January 18\u201323). CosFace: Large Margin Cosine Loss for Deep Face Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00552"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Gonz\u00e1lez-Jim\u00e9nez, J., Galindo, C., and Ruiz-Sarmiento, J.R. (2012, January 9\u201313). Technical improvements of the Giraff telepresence robot based on users\u2019 evaluation. Proceedings of the 2012 IEEE RO-MAN: The 21st IEEE International Symposium on Robot and Human Interactive Communication, Paris, France.","DOI":"10.1109\/ROMAN.2012.6343854"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Prince, S.J. (2012). Computer Vision: Models, Learning, and Inference, Cambridge University Press.","DOI":"10.1017\/CBO9780511996504"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Adjabi, I., Ouahabi, A., Benzaoui, A., and Taleb-Ahmed, A. (2020). Past, Present, and Future of Face Recognition: A Review. Electronics, 9.","DOI":"10.20944\/preprints202007.0479.v1"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1162\/jocn.1991.3.1.71","article-title":"Eigenfaces for Recognition","volume":"3","author":"Turk","year":"1991","journal-title":"J. Cogn. Neurosci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1109\/34.598228","article-title":"Eigenfaces vs. Fisherfaces: Recognition using class specific linear projection","volume":"19","author":"Belhumeur","year":"1997","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","unstructured":"Bledsoe, W.W. (1964). The Model Method in Facial Recognition, Panoramic Research Inc."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Cao, Q., Shen, L., Xie, W., Parkhi, O.M., and Zisserman, A. (2018, January 15\u201319). VGGFace2: A Dataset for Recognising Faces across Pose and Age. Proceedings of the 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00020"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Leibe, B., Matas, J., Sebe, N., and Welling, M. (2016). MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition. Computer Vision\u2014ECCV 2016, Springer International Publishing. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-46478-7"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Kemelmacher-Shlizerman, I., Seitz, S.M., Miller, D., and Brossard, E. (2016, January 27\u201330). The MegaFace Benchmark: 1 Million Faces for Recognition at Scale. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.527"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Nech, A., and Kemelmacher-Shlizerman, I. (2017, January 21\u201326). Level Playing Field for Million Scale Face Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.363"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_25","unstructured":"Simonyan, K., and Zisserman, A. (2015). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA. ISSN: 1063-6919.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Huang, R., Zhang, S., Li, T., and He, R. (2017). Beyond Face Rotation: Global and Local Perception GAN for Photorealistic and Identity Preserving Frontal View Synthesis. arXiv.","DOI":"10.1109\/ICCV.2017.267"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Cao, K., Rong, Y., Li, C., Tang, X., and Change Loy, C. (2018, January 18\u201323). Pose-Robust Face Recognition via Deep Residual Equivariant Mapping. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00544"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1007\/s11263-010-0380-4","article-title":"Deformable Model Fitting by Regularized Landmark Mean-Shift","volume":"91","author":"Saragih","year":"2011","journal-title":"Int. J. Comput. Vis."},{"key":"ref_30","unstructured":"Zhu, X., and Ramanan, D. (2012, January 16\u201321). Face detection, pose estimation, and landmark localization in the wild. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1109\/TPAMI.2018.2792452","article-title":"Learning Pose-Aware Models for Pose-Invariant Face Recognition in the Wild","volume":"41","author":"Masi","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"681","DOI":"10.1109\/34.927467","article-title":"Active appearance models","volume":"23","author":"Cootes","year":"2001","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1023\/B:VISI.0000029666.37597.d3","article-title":"Active Appearance Models Revisited","volume":"60","author":"Matthews","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhu, X., Lei, Z., Liu, X., Shi, H., and Li, S.Z. (2016, January 27\u201330). Face Alignment Across Large Poses: A 3D Solution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.23"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Ruiz, N., Chong, E., and Rehg, J.M. (2018). Fine-Grained Head Pose Estimation Without Keypoints. arXiv.","DOI":"10.1109\/CVPRW.2018.00281"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Yang, T.Y., Chen, Y.T., Lin, Y.Y., and Chuang, Y.Y. (2019, January 15\u201320). FSA-Net: Learning Fine-Grained Structure Aggregation for Head Pose Estimation from a Single Image. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00118"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1007\/s11263-012-0549-0","article-title":"Random Forests for Real Time 3D Face Analysis","volume":"101","author":"Fanelli","year":"2013","journal-title":"Int. J. Comput. Vis."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1007\/s00138-012-0410-z","article-title":"Multi-camera head pose estimation","volume":"23","author":"Saffiotti","year":"2012","journal-title":"Mach. Vis. Appl."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Bansal, A., Nanduri, A., Castillo, C., Ranjan, R., and Chellappa, R. (2017). UMDFaces: An Annotated Face Dataset for Training Deep Networks. arXiv.","DOI":"10.1109\/BTAS.2017.8272731"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., and Pantic, M. (2013, January 2\u20138). 300 Faces in-the-Wild Challenge: The First Facial Landmark Localization Challenge. Proceedings of the IEEE International Conference on Computer Vision Workshops, Sydney, Australia.","DOI":"10.1109\/ICCVW.2013.59"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Schouten, B., Juul, N.C., Drygajlo, A., and Tistarelli, M. (2008). Bosphorus Database for 3D Face Analysis. Biometrics and Identity Management, Springer. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-540-89991-4"},{"key":"ref_42","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press. Available online: http:\/\/www.deeplearningbook.org."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Liu, W., Wen, Y., Yu, Z., Li, M., Raj, B., and Song, L. (2017, January 21\u201326). Sphereface: Deep hypersphere embedding for face recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.713"},{"key":"ref_45","unstructured":"Shi, Y., and Jain, A.K. (November, January 27). Probabilistic Face Embeddings. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea."},{"key":"ref_46","unstructured":"Goldberg, D.E. (1989). Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley Longman Publishing Co., Inc.. [1st ed.]."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/2\/659\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:12:45Z","timestamp":1760159565000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/2\/659"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,19]]},"references-count":46,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["s21020659"],"URL":"https:\/\/doi.org\/10.3390\/s21020659","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,19]]}}}