{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T12:05:23Z","timestamp":1780315523569,"version":"3.54.1"},"reference-count":31,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2022,8,12]],"date-time":"2022-08-12T00:00:00Z","timestamp":1660262400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Tianjin Municipal Education Commission Scientific Research Plan","award":["2018KJ102"],"award-info":[{"award-number":["2018KJ102"]}]},{"name":"Tianjin University","award":["2018KJ102"],"award-info":[{"award-number":["2018KJ102"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In today\u2019s information age, how to accurately identify a person\u2019s identity and protect information security has become a hot topic of people from all walks of life. At present, a more convenient and secure solution to identity identification is undoubtedly biometric identification, but a single biometric identification cannot support increasingly complex and diversified authentication scenarios. Using multimodal biometric technology can improve the accuracy and safety of identification. This paper proposes a biometric method based on finger vein and face bimodal feature layer fusion, which uses a convolutional neural network (CNN), and the fusion occurs in the feature layer. The self-attention mechanism is used to obtain the weights of the two biometrics, and combined with the RESNET residual structure, the self-attention weight feature is cascaded with the bimodal fusion feature channel Concat. To prove the high efficiency of bimodal feature layer fusion, AlexNet and VGG-19 network models were selected in the experimental part for extracting finger vein and face image features as inputs to the feature fusion module. The extensive experiments show that the recognition accuracy of both models exceeds 98.4%, demonstrating the high efficiency of the bimodal feature fusion.<\/jats:p>","DOI":"10.3390\/s22166039","type":"journal-article","created":{"date-parts":[[2022,8,15]],"date-time":"2022-08-15T23:44:03Z","timestamp":1660607043000},"page":"6039","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":61,"title":["Convolutional Neural Network Approach Based on Multimodal Biometric System with Fusion of Face and Finger Vein Features"],"prefix":"10.3390","volume":"22","author":[{"given":"Yang","family":"Wang","sequence":"first","affiliation":[{"name":"School of Electronic Information and Automation, Tianjin University of Science and Technology, Tianjin 300453, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2201-5134","authenticated-orcid":false,"given":"Dekai","family":"Shi","sequence":"additional","affiliation":[{"name":"School of Electronic Information and Automation, Tianjin University of Science and Technology, Tianjin 300453, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9217-7679","authenticated-orcid":false,"given":"Weibin","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Electronic Information and Automation, Tianjin University of Science and Technology, Tianjin 300453, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,12]]},"reference":[{"key":"ref_1","unstructured":"Amine, N.-A. (2019). Hidden Biometrics: When Biometric Security Meets Biomedical Engineering, Springer."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.cose.2014.03.005","article-title":"User identification and authentication using multi-modal behavioral biometrics","volume":"43","author":"Bailey","year":"2014","journal-title":"Comput. Secur."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"379","DOI":"10.2307\/1542358","article-title":"Growth, size rank, and maturation of the freshwater prawn, Macrobrachium rosenbergii: Analysis of marked prawns in an experimental population","volume":"181","author":"Sagi","year":"1991","journal-title":"Biol. Bull."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Ram\u00edrez-Mendoza, R.A., Lozoya-Santos, J.D.J., Zavala-Yo\u00e9, R., Alonso-Valerdi, L.M., Morales-Menendez, R., Carri\u00f3n, B., Cruz, P.P., and Gonzalez-Hernandez, H.G. (2022). Biometry: Technology, Trends and Applications, CRC Press.","DOI":"10.1201\/9781003145240"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Jomaa, R.M., Mathkour, H., Bazi, Y., and Islam, M.S. (2020). End-to-end deep learning fusion of fingerprint and electrocardiogram signals for presentation attack detection. Sensors, 20.","DOI":"10.3390\/s20072085"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Mitra, S., and Gofman, M. (2016). Biometrics in a Data Driven World: Trends, Technologies, and Challenges, CRC Press.","DOI":"10.1201\/9781315317083"},{"key":"ref_7","unstructured":"Lowe, J. (2020). Ocular Motion Classification for Mobile Device Presentation Attack Detection, University of Missouri-Kansas City."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ammour, B., Boubchir, L., Bouden, T., and Ramdani, M. (2020). Face\u2013iris multimodal biometric identification system. Electronics, 9.","DOI":"10.3390\/electronics9010085"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Zheng, Y., Blasch, E., and Liu, Z. (2018). Multispectral Image Fusion and Colorization, SPIE Press.","DOI":"10.1117\/3.2316455"},{"key":"ref_10","first-page":"106","article-title":"An exploration of the impacts of three factors in multimodal biometric score fusion: Score modality, recognition method, and fusion process","volume":"9","author":"Zheng","year":"2015","journal-title":"J. Adv. Inf. Fusion"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1984","DOI":"10.1109\/TIFS.2016.2569061","article-title":"Discriminant correlation analysis: Real-time feature level fusion for multimodal biometric recognition","volume":"11","author":"Haghighat","year":"2016","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"116288","DOI":"10.1016\/j.eswa.2021.116288","article-title":"DS-CNN: A pre-trained Xception model based on depth-wise separable convolutional neural network for finger vein recognition","volume":"191","author":"Shaheed","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Xu, H., Qi, M., and Lu, Y. (2019, January 19\u201321). Multimodal Biometrics Based on Convolutional Neural Network by Two-Layer Fusion. Proceedings of the 2019 12th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Suzhou, China.","DOI":"10.1109\/CISP-BMEI48845.2019.8966036"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"e248","DOI":"10.7717\/peerj-cs.248","article-title":"Convolutional neural networks approach for multimodal biometric identification system using the fusion of fingerprint, finger-vein and face images","volume":"6","author":"Cherrat","year":"2020","journal-title":"PeerJ Comput. Sci."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Abinaya, R., Indira, D., and Swarup Kumar, J. (2022). Multimodal Biometric Person Identification System Based on Speech and Keystroke Dynamics. EAI\/Springer Innovations in Communication and Computing Book Series (EAISICC), Proceedings of the International Conference on Computing, Communication, Electrical and Biomedical Systems, Online, 28 February 2022, Springer.","DOI":"10.1007\/978-3-030-86165-0_24"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2049","DOI":"10.1109\/TMM.2015.2477042","article-title":"Robust face recognition via multimodal deep face representation","volume":"17","author":"Ding","year":"2015","journal-title":"IEEE Trans. Multimed."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Chawla, B., Tyagi, S., Jain, R., Talegaonkar, A., and Srivastava, S. (2021). Finger Vein Recognition Using Deep Learning. Formal Ontology in Information Systems, Proceedings of the International Conference on Artificial Intelligence and Applications, Trento, Italy, 6\u20138 June 2021, Springer.","DOI":"10.1007\/978-981-15-4992-2_7"},{"key":"ref_18","first-page":"5523","article-title":"Deep learning approach for multimodal biometric recognition system based on fusion of iris, face, and finger vein traits","volume":"20","author":"Nada","year":"2022","journal-title":"Sensors"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wan, K., Song, J.M., and Park, K.R. (2018). Multimodal biometric recognition based on convolutional neural network by the fusion of finger-vein and finger shape using near-infrared (NIR) camera sensor. Sensors, 18.","DOI":"10.3390\/s18072296"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1016\/j.patcog.2018.01.026","article-title":"A fingerprint and finger-vein based cancelable multi-biometric system","volume":"78","author":"Yang","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Soleymani, S., Dabouei, A., Kazemi, H., Dawson, J., and Nasrabadi, N.M. (2018, January 20\u201324). Multi-Level Feature Abstraction from Convolutional Neural Networks for Multimodal Biometric Identification. Proceedings of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China.","DOI":"10.1109\/ICPR.2018.8545061"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"783","DOI":"10.1007\/s10044-017-0656-1","article-title":"A multi-biometric iris recognition system based on a deep learning approach","volume":"21","author":"Qahwaji","year":"2018","journal-title":"Pattern Anal. Appl."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"107159","DOI":"10.1016\/j.knosys.2021.107159","article-title":"Finger vein recognition system with template protection based on convolutional neural network","volume":"227","author":"Ren","year":"2021","journal-title":"Knowl.-Based Syst."},{"key":"ref_24","unstructured":"Chollet, F. (2021). Deep Learning with Python, Simon and Schuster."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"320","DOI":"10.1007\/s40484-016-0081-2","article-title":"Performance measures in evaluating machine learning based bioinformatics predictors for classifications","volume":"4","author":"Jiao","year":"2016","journal-title":"Quant. Biol."},{"key":"ref_26","unstructured":"Simonyan, K., and Andrew, Z. (2015). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1016\/j.compag.2018.03.032","article-title":"A comparative study of fine-tuning deep learning models for plant disease identification","volume":"161","author":"Too","year":"2019","journal-title":"Comput. Electron. Agric."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Tan, C., Sun, F., Kong, T., Zhang, W., Yang, C., and Liu, C. (2018, January 17\u201319). A Survey on Deep Transfer Learning. Proceedings of the International Conference on Artificial Neural Networks, Munich, Germany.","DOI":"10.1007\/978-3-030-01424-7_27"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1049\/ccs.2019.0010","article-title":"Improved softmax loss for deep learning-based face and expression recognition","volume":"1","author":"Zhou","year":"2019","journal-title":"Cogn. Comput. Syst."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Yin, Y., Liu, L., and Sun, X. (2011). SDUMLA-HMT: A multimodal biometric database. Chinese Conference on Biometric Recognition, Springer.","DOI":"10.1007\/978-3-642-25449-9_33"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1016\/j.ajo.2020.03.035","article-title":"Efficacy for Differentiating Nonglaucomatous versus Glaucomatous Optic Neuropathy Using Deep Learning Systems","volume":"216","author":"Yang","year":"2022","journal-title":"Am. J. Ophthalmol."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/16\/6039\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:08:00Z","timestamp":1760141280000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/16\/6039"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,12]]},"references-count":31,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["s22166039"],"URL":"https:\/\/doi.org\/10.3390\/s22166039","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,12]]}}}