{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:41:00Z","timestamp":1760236860120,"version":"build-2065373602"},"reference-count":36,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,12,31]],"date-time":"2021-12-31T00:00:00Z","timestamp":1640908800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The presence of fake pictures affects the reliability of visible face images under specific circumstances. This paper presents a novel adversarial neural network designed named as the FTSGAN for infrared and visible image fusion and we utilize FTSGAN model to fuse the face image features of infrared and visible image to improve the effect of face recognition. In FTSGAN model design, the Frobenius norm (F), total variation norm (TV), and structural similarity index measure (SSIM) are employed. The F and TV are used to limit the gray level and the gradient of the image, while the SSIM is used to limit the image structure. The FTSGAN fuses infrared and visible face images that contains bio-information for heterogeneous face recognition tasks. Experiments based on the FTSGAN using hundreds of face images demonstrate its excellent performance. The principal component analysis (PCA) and linear discrimination analysis (LDA) are involved in face recognition. The face recognition performance after fusion improved by 1.9% compared to that before fusion, and the final face recognition rate was 94.4%. This proposed method has better quality, faster rate, and is more robust than the methods that only use visible images for face recognition.<\/jats:p>","DOI":"10.3390\/s22010304","type":"journal-article","created":{"date-parts":[[2022,1,9]],"date-time":"2022-01-09T23:08:26Z","timestamp":1641769706000},"page":"304","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["A Novel Infrared and Visible Image Fusion Approach Based on Adversarial Neural Network"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3165-1434","authenticated-orcid":false,"given":"Xianglong","family":"Chen","sequence":"first","affiliation":[{"name":"School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8970-3508","authenticated-orcid":false,"given":"Haipeng","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yaohui","family":"Liang","sequence":"additional","affiliation":[{"name":"School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0872-7023","authenticated-orcid":false,"given":"Ying","family":"Meng","sequence":"additional","affiliation":[{"name":"School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China"},{"name":"Key Laboratory of Optoelectronic Measurement, Optical Information Transmission Technology of Ministry of Education, School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shifeng","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Optoelectronic Engineering, Changchun University of Science and Technology, Changchun 130022, China"},{"name":"Zhongshan Institute, Changchun University of Science and Technology, Changchun 130022, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,12,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"4825","DOI":"10.1007\/s11042-020-09850-1","article-title":"Classical and modern face recognition approaches: A complete review","volume":"80","author":"Ali","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_2","first-page":"LB180","article-title":"Abstract LB180: Clinical biomarker studies with two fusion-enhanced versions of oncolytic HSV (RP1 and RP2) alone and in combination with nivolumab in cancer patients indicate potent immune activation","volume":"81","author":"Harrington","year":"2021","journal-title":"Immunology"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Yi, W., Zeng, Y., Wang, Y., Deng, J., Su, W., and Yuan, Z. (2021, January 16\u201318). An improved IHS fusion method of GF-2 remote sensing images. Proceedings of the International Conference on Signal Image Processing and Communication (ICSIPC 2021), Chengdu, China.","DOI":"10.1117\/12.2600387"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.inffus.2021.04.005","article-title":"Attribute filter based infrared and visible image fusion","volume":"75","author":"Mo","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_5","first-page":"139","article-title":"Generative adversarial nets","volume":"63","author":"Goodfellow","year":"2020","journal-title":"Assoc. Comput. Mach."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Omar, Z., and Stathaki, T. (2014, January 15\u201316). Image Fusion: An Overview. Proceedings of the 2014 5th International Conference on Intelligent Systems, Hunan, China.","DOI":"10.1109\/ISMS.2014.58"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Shahdoosti, H.R., and Ghassemian, H. (2012, January 2\u20133). Spatial PCA as a new method for image fusion. Proceedings of the 16th CSI International Symposium on Artificial Intelligence and Signal Processing (AISP 2012), Shiraz, Iran.","DOI":"10.1109\/AISP.2012.6313724"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1117\/12.467800","article-title":"Remote sensing image fusion based on IHS transform, wavelet transform, and HPF","volume":"4898","author":"Li","year":"2003","journal-title":"Image Process. Pattern Recognit. Remote Sens."},{"key":"ref_9","first-page":"481","article-title":"Data image fusion using combinatorial maps","volume":"5909","author":"Kamel","year":"2005","journal-title":"Appl. Digit. Image Process. XXVIII"},{"key":"ref_10","unstructured":"Luo, R.C., and Kay, M.G. (1990, January 27\u201330). A tutorial on multisensor integration and fusion. Proceedings of the 16th Annual Conference of IEEE Industrial Electronics Society, Pacific Grove, CA, USA."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., Fergus, R., David, F., Tomas, P., Bernt, S., and Tinne, T. (2014, January 6\u201312). Visualizing and Understanding Convolutional Networks. Proceedings of the Computer Vision\u2014ECCV 2014, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10590-1_53"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4980","DOI":"10.1109\/TIP.2020.2977573","article-title":"DDcGAN: A Dual-Discriminator Conditional Generative Adversarial Network for Multi-Resolution Image Fusion","volume":"29","author":"Ma","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.inffus.2018.09.004","article-title":"FusionGAN: A generative adversarial network for infrared and visible image fusion","volume":"48","author":"Ma","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"35673","DOI":"10.1109\/ACCESS.2019.2905511","article-title":"Convolutional autoencoder-based multispectral image fusion","volume":"7","author":"Azarang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"507","DOI":"10.4064\/aa-24-5-507-528","article-title":"On the theorem of Gauss-Kusmin-L\u00e9vy and a Frobenius-type theorem for function spaces","volume":"24","author":"Wirsing","year":"1974","journal-title":"Acta Arith."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2419","DOI":"10.1109\/TIP.2009.2028250","article-title":"Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems","volume":"18","author":"Beck","year":"2009","journal-title":"IEEE Trans. Image Process."},{"key":"ref_18","unstructured":"Mirza, M., and Osindero, S. (2014). Conditional generative adversarial nets. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zeiler, M.D., Krishnan, D., Taylor, W., and Fergus, R. (2010, January 13\u201318). Deconvolutional Networks. Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Francisco, CA, USA.","DOI":"10.1109\/CVPR.2010.5539957"},{"key":"ref_20","first-page":"4700","article-title":"Densely connected convolutional networks","volume":"5","author":"Huang","year":"2017","journal-title":"Comput. Vis. Pattern Recognit."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"569","DOI":"10.1109\/TPAMI.2009.187","article-title":"Sensitivity Analysis of k-Fold Cross Validation in Prediction Error Estimation","volume":"32","author":"Rodriguez","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2839","DOI":"10.1016\/j.patcog.2015.03.009","article-title":"Performance evaluation of classification algorithms by k-fold and leave-one-out cross- validation","volume":"48","author":"Wong","year":"2015","journal-title":"Pattern Recognit."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"228","DOI":"10.1109\/34.908974","article-title":"PCA versus LDA","volume":"23","author":"Martinez","year":"2001","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Bansal, A., Mehta, K., and Arora, S. (2012, January 7\u20138). Face Recognition Using PCA and LDA Algorithm. Proceedings of the 2012 Second International Conference on Advanced Computing & Communication Technologies, Rohtak, India.","DOI":"10.1109\/ACCT.2012.52"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1007\/978-3-642-72201-1_4","article-title":"Discriminant Analysis of Principal Components for Face Recognition","volume":"163","author":"Zhao","year":"1998","journal-title":"Face Recognit. Theory Appl."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Borade, S.N., Deshmukh, R.R., and Ramu, S. (2016, January 21\u201324). Face recognition using fusion of PCA and LDA: Borda count approach. Proceedings of the 2016 24th Mediterranean Conference on Control and Automation (MED), Athens, Greece.","DOI":"10.1109\/MED.2016.7536065"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Marcialis, G.L., and Roli, F. (2002, January 1). Fusion of LDA and PCA for Face Verification. Proceedings of the International Workshop on Biometric Authentication, Copenhagen, Denmark.","DOI":"10.1007\/3-540-47917-1_4"},{"key":"ref_28","first-page":"946","article-title":"BDPCA plus LDA: A novel fast feature extraction technique for face recognition","volume":"36","author":"Zuo","year":"2006","journal-title":"IEEE Trans. Syst."},{"key":"ref_29","first-page":"509","article-title":"A comprehensive database for benchmarking imaging systems","volume":"42","author":"Karen","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1850018","DOI":"10.1142\/S0219691318500182","article-title":"Infrared and visible image fusion with convolutional neural networks","volume":"16","author":"Liu","year":"2018","journal-title":"Int. J. Wavelets Multiresolut. Inf. Process."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.inffus.2016.02.001","article-title":"Infrared and visible image fusion via gradient transfer and total variation minimization","volume":"31","author":"Ma","year":"2016","journal-title":"Inf. Fusion"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"7203","DOI":"10.1109\/TIP.2020.2999855","article-title":"MEF-GAN: Multi-Exposure Image Fusion via Generative Adversarial Networks","volume":"29","author":"Xu","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Ram Prabhakar, K., Sai Srikar, V., and Venkatesh Babu, R. (2017, January 22\u201329). Deepfuse: A deep unsupervised approach for exposure fusion with extreme exposure image pairs. Proceedings of the IEEE international conference on computer vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.505"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"023522","DOI":"10.1117\/1.2945910","article-title":"Assessment of image fusion procedures using entropy, image quality, and multispectral classifification","volume":"2","author":"Roberts","year":"2008","journal-title":"J. Appl. Remote Sens."},{"key":"ref_35","first-page":"2959","article-title":"Image quality measures and their performance","volume":"43","author":"Eskicioglu","year":"1995","journal-title":"IEEE Trans. Image Process."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1049\/el:20000267","article-title":"Objective image fusion performance measure","volume":"36","author":"Xydeas","year":"2000","journal-title":"Electron. Lett."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/1\/304\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:56:58Z","timestamp":1760169418000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/1\/304"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12,31]]},"references-count":36,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["s22010304"],"URL":"https:\/\/doi.org\/10.3390\/s22010304","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2021,12,31]]}}}