{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T05:01:08Z","timestamp":1775278868780,"version":"3.50.1"},"reference-count":30,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2021,7,1]],"date-time":"2021-07-01T00:00:00Z","timestamp":1625097600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Military Academy Research Center (CINAMIL)","award":["project Multi-Spectral Facial Recognition"],"award-info":[{"award-number":["project Multi-Spectral Facial Recognition"]}]},{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["FCT Project UIDB\/50009\/2020"],"award-info":[{"award-number":["FCT Project UIDB\/50009\/2020"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Facial recognition is a method of identifying or authenticating the identity of people through their faces. Nowadays, facial recognition systems that use multispectral images achieve better results than those that use only visible spectral band images. In this work, a novel architecture for facial recognition that uses multiple deep convolutional neural networks and multispectral images is proposed. A domain-specific transfer-learning methodology applied to a deep neural network pre-trained in RGB images is shown to generalize well to the multispectral domain. We also propose a skin detector module for forgery detection. Several experiments were planned to assess the performance of our methods. First, we evaluate the performance of the forgery detection module using face masks and coverings of different materials. A second study was carried out with the objective of tuning the parameters of our domain-specific transfer-learning methodology, in particular which layers of the pre-trained network should be retrained to obtain good adaptation to multispectral images. A third study was conducted to evaluate the performance of support vector machines (SVM) and k-nearest neighbor classifiers using the embeddings obtained from the trained neural network. Finally, we compare the proposed method with other state-of-the-art approaches. The experimental results show performance improvements in the Tufts and CASIA NIR-VIS 2.0 multispectral databases, with a rank-1 score of 99.7% and 99.8%, respectively.<\/jats:p>","DOI":"10.3390\/s21134520","type":"journal-article","created":{"date-parts":[[2021,7,1]],"date-time":"2021-07-01T12:03:27Z","timestamp":1625141007000},"page":"4520","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Multispectral Face Recognition Using Transfer Learning with Adaptation of Domain Specific Units"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0314-1207","authenticated-orcid":false,"given":"Luis Lopes","family":"Chambino","sequence":"first","affiliation":[{"name":"Instituto Superior T\u00e9cnico, Universidade de Lisboa, 1049-001 Lisbon, Portugal"},{"name":"Portuguese Military Academy, Rua Gomes Freire, 1169-203 Lisbon, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7529-6422","authenticated-orcid":false,"given":"Jos\u00e9 Silvestre","family":"Silva","sequence":"additional","affiliation":[{"name":"Portuguese Military Academy, Rua Gomes Freire, 1169-203 Lisbon, Portugal"},{"name":"Military Academy Research Center (CINAMIL), Rua Gomes Freire, 1169-203 Lisbon, Portugal"},{"name":"Laboratory for Instrumentation, Biomedical Engineering and Radiation Physics (LIBPhys-UC), Rua Larga, 3004-516 Coimbra, Portugal"}]},{"given":"Alexandre","family":"Bernardino","sequence":"additional","affiliation":[{"name":"Instituto Superior T\u00e9cnico, Universidade de Lisboa, 1049-001 Lisbon, Portugal"},{"name":"Institute for Systems and Robotics (ISR), 1049-001 Lisbon, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2021,7,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Jain, A.K., Ross, A.A., and Nandakumar, K. (2011). Introduction to Biometrics, Springer Science & Business Media.","DOI":"10.1007\/978-0-387-77326-1"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"102660","DOI":"10.1016\/j.jvcir.2019.102660","article-title":"An extensive review on spectral imaging in biometric systems: Challenges & advancements","volume":"65","author":"Munir","year":"2019","journal-title":"J. Vis. Commun. Image Represent."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1109\/TPAMI.2018.2803179","article-title":"Improving Shadow Suppression for Illumination Robust Face Recognition","volume":"41","author":"Zhang","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"75305","DOI":"10.1109\/ACCESS.2019.2920855","article-title":"Heterogeneous Face Recognition Based on Multiple Deep Networks with Scatter Loss and Diversity Combination","volume":"7","author":"Hu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_5","first-page":"143","article-title":"Detection of Camouflaged People","volume":"5","author":"Bento","year":"2016","journal-title":"Int. J. Sens. Netw. Data Commun."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"9341","DOI":"10.1109\/JSEN.2019.2925203","article-title":"Landmine Detection Using Multispectral Images","volume":"19","author":"Silva","year":"2019","journal-title":"IEEE Sens. J."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"207871","DOI":"10.1109\/ACCESS.2020.3037451","article-title":"Multispectral Facial Recognition: A Review","volume":"8","author":"Chambino","year":"2020","journal-title":"IEEE Access"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, S., Yi, D., Lei, Z., and Liao, S. (2013, January 23\u201328). The CASIA NIR-VIS 2.0 Face Database. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Portland, OR, USA.","DOI":"10.1109\/CVPRW.2013.59"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1109\/TIFS.2019.2916652","article-title":"Biometric Face Presentation Attack Detection With Multi-Channel Convolutional Neural Network","volume":"15","author":"George","year":"2020","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1803","DOI":"10.1109\/TIFS.2018.2885284","article-title":"Heterogeneous Face Recognition Using Domain Specific Units","volume":"14","author":"Pereira","year":"2019","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2884","DOI":"10.1109\/TIFS.2018.2833032","article-title":"A Light CNN for Deep Face Representation with Noisy Labels","volume":"13","author":"Wu","year":"2018","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Fu, X., Lu, J., Zhang, X., Yang, X., and Unwala, I. (2019, January 1\u20133). Intelligent In-Vehicle Safety and Security Monitoring System with Face Recognition. Proceedings of the IEEE International Conference on Computational Science and IEEE International Conference on Embedded and Ubiquitous Computing Engineering, New York, NY, USA.","DOI":"10.1109\/CSE\/EUC.2019.00050"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1109\/TPAMI.2018.2884458","article-title":"A Comprehensive Database for Benchmarking Imaging Systems","volume":"42","author":"Panetta","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.cviu.2017.05.007","article-title":"Systematic Evaluation of Convolution Neural Network Advances on the Imagenet","volume":"161","author":"Mishkin","year":"2017","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_15","unstructured":"Goodfellow, I., Bengio, Y., and Courville, A. (2016). Deep Learning, MIT Press."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1145\/1882471.1882479","article-title":"Apples-to-Apples in Cross-Validation Studies: Pitfalls in Classifier Performance Measurement","volume":"12","author":"Forman","year":"2010","journal-title":"ACM Sigkdd Explor. Newsl."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1540023","DOI":"10.1142\/S0218213015400230","article-title":"Performance-Estimation Properties of Cross-Validation-Based Protocols with Simultaneous Hyper-Parameter Optimization","volume":"24","author":"Tsamardinos","year":"2015","journal-title":"Int. J. Artif. Intell. Tools"},{"key":"ref_18","unstructured":"Kezebou, L., Oludare, V., Panetta, K., and Agaian, S. (May, January 27). TR-GAN: Thermal to RGB face synthesis with generative adversarial network for cross-modal face recognition. Proceedings of the SPIE Mobile Multimedia\/Image Processing, Security, and Applications, Available online: https:\/\/www.spiedigitallibrary.org\/conference-proceedings-of-spie\/11399\/113990P\/TR-GAN--thermal-to-RGB-face-synthesis-with-generative\/10.1117\/12.2558166.short?SSO=1."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Rajeev, S., Shreyas, K., Wan, Q., Panetta, K., and Agaian, S. (2019). Illumination Invariant NIR Face Recognition Using Directional Visibility. Electron. Imagingimage Process. Algorithms Syst., 273-1\u2013273-7.","DOI":"10.2352\/ISSN.2470-1173.2019.11.IPAS-273"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1109\/TIFS.2015.2390414","article-title":"Coupled Discriminative Feature Learning for Heterogeneous Face Recognition","volume":"10","author":"Jin","year":"2015","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2856058","article-title":"Mutual Component Analysis for Heterogeneous Face Recognition","volume":"7","author":"Li","year":"2016","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"905","DOI":"10.1007\/s11045-016-0401-8","article-title":"Multi-task clustering ELM for VIS-NIR cross-modal feature learning","volume":"28","author":"Jin","year":"2017","journal-title":"Multidimens. Syst. Signal Process."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2079","DOI":"10.1109\/TIP.2017.2651380","article-title":"Heterogeneous Face Recognition: A Common Encoding Feature Discriminant Approach","volume":"26","author":"Gong","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1016\/j.neucom.2015.11.137","article-title":"A Gabor-based network for heterogeneous face recognition","volume":"261","author":"Oh","year":"2017","journal-title":"Neurocomputing"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1016\/j.patcog.2019.01.041","article-title":"DLFace: Deep local descriptor for cross-modality face recognition","volume":"90","author":"Peng","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1761","DOI":"10.1109\/TPAMI.2018.2842770","article-title":"Wasserstein CNN: Learning Invariant Features for NIR-VIS Face Recognition","volume":"41","author":"He","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1016\/j.cviu.2019.04.003","article-title":"Discriminant Deep Feature Learning based on joint supervision Loss and Multi-layer Feature Fusion for heterogeneous face recognition","volume":"184","author":"Hu","year":"2019","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"4553","DOI":"10.1109\/TIP.2019.2912360","article-title":"Re-Ranking High-Dimensional Deep Local Representation for NIR-VIS Face Recognition","volume":"28","author":"Peng","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1025","DOI":"10.1109\/TPAMI.2019.2961900","article-title":"Adversarial cross-spectral face completion for NIR-VIS face recognition","volume":"42","author":"He","year":"2019","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"50452","DOI":"10.1109\/ACCESS.2020.2980047","article-title":"Non-Visual to Visual Translation for Cross-Domain Face Recognition","volume":"8","author":"Bae","year":"2020","journal-title":"IEEE Access"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/13\/4520\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:24:37Z","timestamp":1760163877000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/13\/4520"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,1]]},"references-count":30,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2021,7]]}},"alternative-id":["s21134520"],"URL":"https:\/\/doi.org\/10.3390\/s21134520","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,1]]}}}