{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T22:25:44Z","timestamp":1779920744186,"version":"3.53.1"},"reference-count":33,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2019,6,5]],"date-time":"2019-06-05T00:00:00Z","timestamp":1559692800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003052","name":"Ministry of Trade, Industry and Energy","doi-asserted-by":"publisher","award":["10080619"],"award-info":[{"award-number":["10080619"]}],"id":[{"id":"10.13039\/501100003052","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this paper, we present a new effective infrared (IR) and visible (VIS) image fusion method by using a deep neural network. In our method, a Siamese convolutional neural network (CNN) is applied to automatically generate a weight map which represents the saliency of each pixel for a pair of source images. A CNN plays a role in automatic encoding an image into a feature domain for classification. By applying the proposed method, the key problems in image fusion, which are the activity level measurement and fusion rule design, can be figured out in one shot. The fusion is carried out through the multi-scale image decomposition based on wavelet transform, and the reconstruction result is more perceptual to a human visual system. In addition, the visual qualitative effectiveness of the proposed fusion method is evaluated by comparing pedestrian detection results with other methods, by using the YOLOv3 object detector using a public benchmark dataset. The experimental results show that our proposed method showed competitive results in terms of both quantitative assessment and visual quality.<\/jats:p>","DOI":"10.3390\/e21060570","type":"journal-article","created":{"date-parts":[[2019,6,6]],"date-time":"2019-06-06T03:38:01Z","timestamp":1559792281000},"page":"570","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":54,"title":["A New Deep Learning Based Multi-Spectral Image Fusion Method"],"prefix":"10.3390","volume":"21","author":[{"given":"Jingchun","family":"Piao","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Hanyang University, Ansan 15588, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yunfan","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Hanyang University, Ansan 15588, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hyunchul","family":"Shin","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Hanyang University, Ansan 15588, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,6,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1016\/j.infrared.2015.10.004","article-title":"A fusion algorithm for infrared and visible images based on saliency analysis and non-subsampled Shearlet transform","volume":"73","author":"Zhang","year":"2015","journal-title":"Infrared Phys. Technol."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"134","DOI":"10.1016\/j.infrared.2014.02.013","article-title":"A fusion method for visible and infrared images based on contrast pyramid with teaching learning based optimization","volume":"64","author":"Jin","year":"2014","journal-title":"Infrared Phys. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/j.optcom.2014.12.032","article-title":"Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition","volume":"341","author":"Cui","year":"2015","journal-title":"Opt. Commun."},{"key":"ref_4","unstructured":"Fan, X., Shi, P., Ni, J., and Li, M. (2015). A thermal infrared and visible images fusion based approach for multi target detection under complex environment. Math. Probl. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1016\/j.neucom.2016.02.047","article-title":"Union Laplacian pyramid with multiple features for medical image fusion","volume":"194","author":"Du","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"255","DOI":"10.1016\/0167-8655(89)90004-4","article-title":"A morphological pyramidal image decomposition","volume":"9","author":"Toet","year":"1989","journal-title":"Pattern Recognit. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.inffus.2012.09.005","article-title":"Fusion of multimodal medical images using Daubechies complex wavelet transform c a multiresolution approach","volume":"19","author":"Singh","year":"2014","journal-title":"Inf. Fusion"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1006\/gmip.1995.1022","article-title":"Multi sensor image fusion using the wavelet transform","volume":"57","author":"Li","year":"1995","journal-title":"Graph. Models Image Process."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.inffus.2005.09.006","article-title":"Pixel- and region-based image fusion with complex wavelets","volume":"8","author":"Lewis","year":"2007","journal-title":"Inf. Fusion"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1016\/j.neucom.2008.02.025","article-title":"Multimodality medical image fusion based on multiscale geometric analysis of contourlet transform","volume":"72","author":"Yang","year":"2008","journal-title":"Neurocomputing"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1014","DOI":"10.1109\/TMM.2013.2244870","article-title":"Directive contrast based multimodal medical image fusion in nsct domain","volume":"15","author":"Zheng","year":"2013","journal-title":"IEEE Trans. Multimedia"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1109\/TBME.2013.2279301","article-title":"Multimodal medical volumetric data fusion using 3-d discrete shearlet transform and global-to-local rule","volume":"61","author":"Wang","year":"2014","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1016\/j.inffus.2010.04.001","article-title":"Pixel-level image fusion with simultaneous orthogonal matching pursuit","volume":"13","author":"Yang","year":"2012","journal-title":"Inf. Fusion"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3450","DOI":"10.1109\/TBME.2012.2217493","article-title":"Group-sparse representation with dictionary learning for medical image denoising and fusion","volume":"59","author":"Li","year":"2012","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1049\/iet-ipr.2014.0311","article-title":"Simultaneous image fusion and denosing with adaptive sparse representation","volume":"9","author":"Liu","year":"2015","journal-title":"IET Image Process."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.inffus.2018.09.004","article-title":"FusionGAN: A generative adversarial network for infrared and visible image fusion","volume":"48","author":"Ma","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1069","DOI":"10.1109\/TBME.2012.2211017","article-title":"Cross-scale coefficient selection for volumetric medical image fusion","volume":"60","author":"Shen","year":"2013","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.inffus.2014.09.004","article-title":"A general framework for image fusion based on multi-scale transform and sparse representation","volume":"24","author":"Liu","year":"2015","journal-title":"Inf. Fusion"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based leaning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"478","DOI":"10.1016\/j.infrared.2017.07.010","article-title":"A survey of infrared and visible image fusion methods","volume":"85","author":"Jin","year":"2017","journal-title":"Infrared Phys. Technol."},{"key":"ref_21","unstructured":"(2019, March 01). DLP: Deep Learning Platform. Available online: http:\/\/www.ai-dlp.com\/."},{"key":"ref_22","unstructured":"Glorot, X., and Bengio, Y. (2010, January 13\u201315). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the thirteenth International Conference on Artificial Intelligence and Statistics, Sardinia, Italy."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Liu, J., Yin, Q., and Guo, P. (2006, January 13\u201316). A New Strategy to Improve Image Fusion Effect. Proceedings of the 2016 International Conference on Machine Learning and Cybernetics, Dalian, China.","DOI":"10.1109\/ICMLC.2006.258681"},{"key":"ref_24","unstructured":"(2019, March 01). FigShare. Available online: https:\/\/figshare.com\/articles\/TNO_Image_Fusion_Dataset\/1008029."},{"key":"ref_25","unstructured":"(2019, March 01). OTCBVS Benchmark Dataset Collection. Available online: http:\/\/vcipl-okstate.org\/pbvs\/bench\/."},{"key":"ref_26","unstructured":"(2019, March 01). Multispectral Object Detection Dataset. Available online: https:\/\/www.mi.t.u-tokyo.ac.jp\/projects\/mil_multispectral\/."},{"key":"ref_27","unstructured":"(2019, March 01). YOLO: Real-Time Object Detection. Available online: https:\/\/pjreddie.com\/darknet\/yolo\/."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.inffus.2011.08.002","article-title":"A new image fusion performance metric based on visual information fidelity","volume":"14","author":"Han","year":"2013","journal-title":"Inf. Fusion"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1421","DOI":"10.1016\/j.imavis.2007.12.002","article-title":"A new automated quality assessment algorithm for image fusion","volume":"27","author":"Chen","year":"2009","journal-title":"Image Vis. Comput."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"023522","DOI":"10.1117\/1.2945910","article-title":"Assessment of image fusion procedures using entropy, image quality, and multispectral classification","volume":"2","author":"Roberts","year":"2008","journal-title":"J. Appl. Remote Sens."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1049\/el:20020212","article-title":"Information measure for performance of image fusion","volume":"38","author":"Qu","year":"2002","journal-title":"Electron. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1109\/97.995823","article-title":"A universal image quality index","volume":"9","author":"Wang","year":"2002","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.inffus.2018.02.004","article-title":"Infrared and visible image fusion methods and applications: A survey","volume":"14","author":"Ma","year":"2019","journal-title":"Inf. Fusion"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/6\/570\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:56:27Z","timestamp":1760187387000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/6\/570"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,6,5]]},"references-count":33,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2019,6]]}},"alternative-id":["e21060570"],"URL":"https:\/\/doi.org\/10.3390\/e21060570","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,6,5]]}}}