{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T17:34:09Z","timestamp":1774287249658,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2023,9,25]],"date-time":"2023-09-25T00:00:00Z","timestamp":1695600000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Beijing Municipal Natural Science Foundation","award":["BJXZ2021-012-00046"],"award-info":[{"award-number":["BJXZ2021-012-00046"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Infrared and visible image fusion (IVIF) aims to render fused images that maintain the merits of both modalities. To tackle the challenge in fusing cross-modality information and avoiding texture loss in IVIF, we propose a novel edge-consistent and correlation-driven fusion framework (ECFuse). This framework leverages our proposed edge-consistency fusion module to maintain rich and coherent edges and textures, simultaneously introducing a correlation-driven deep learning network to fuse the cross-modality global features and modality-specific local features. Firstly, the framework employs a multi-scale transformation (MST) to decompose the source images into base and detail layers. Then, the edge-consistent fusion module fuses detail layers while maintaining the coherence of edges through consistency verification. A correlation-driven fusion network is proposed to fuse the base layers containing both modalities\u2019 main features in the transformation domain. Finally, the final fused spatial image is reconstructed by inverse MST. We conducted experiments to compare our ECFuse with both conventional and deep leaning approaches on TNO, LLVIP and M3FD datasets. The qualitative and quantitative evaluation results demonstrate the effectiveness of our framework. We also show that ECFuse can boost the performance in downstream infrared\u2013visible object detection in a unified benchmark.<\/jats:p>","DOI":"10.3390\/s23198071","type":"journal-article","created":{"date-parts":[[2023,9,26]],"date-time":"2023-09-26T03:45:23Z","timestamp":1695699923000},"page":"8071","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["ECFuse: Edge-Consistent and Correlation-Driven Fusion Framework for Infrared and Visible Image Fusion"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-4072-0153","authenticated-orcid":false,"given":"Hanrui","family":"Chen","sequence":"first","affiliation":[{"name":"Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science & Technology University, Beijing 100192, China"},{"name":"Beijing Laboratory of Optical Fiber Sensing and System, Beijing Information Science & Technology University, Beijing 100016, China"},{"name":"Guangzhou Nansha Intelligent Photonic Sensing Research Institute, Guangzhou 511462, China"}]},{"given":"Lei","family":"Deng","sequence":"additional","affiliation":[{"name":"Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science & Technology University, Beijing 100192, China"},{"name":"Beijing Laboratory of Optical Fiber Sensing and System, Beijing Information Science & Technology University, Beijing 100016, China"},{"name":"Guangzhou Nansha Intelligent Photonic Sensing Research Institute, Guangzhou 511462, China"}]},{"given":"Lianqing","family":"Zhu","sequence":"additional","affiliation":[{"name":"Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science & Technology University, Beijing 100192, China"},{"name":"Beijing Laboratory of Optical Fiber Sensing and System, Beijing Information Science & Technology University, Beijing 100016, China"},{"name":"Guangzhou Nansha Intelligent Photonic Sensing Research Institute, Guangzhou 511462, China"}]},{"given":"Mingli","family":"Dong","sequence":"additional","affiliation":[{"name":"Key Laboratory of the Ministry of Education for Optoelectronic Measurement Technology and Instrument, Beijing Information Science & Technology University, Beijing 100192, China"},{"name":"Beijing Laboratory of Optical Fiber Sensing and System, Beijing Information Science & Technology University, Beijing 100016, China"},{"name":"Guangzhou Nansha Intelligent Photonic Sensing Research Institute, Guangzhou 511462, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,9,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Yin, R., Yang, B., Huang, Z., and Zhang, X. (2023). DSA-Net: Infrared and Visible Image Fusion via Dual-Stream Asymmetric Network. Sensors, 23.","DOI":"10.20944\/preprints202306.2220.v1"},{"key":"ref_2","first-page":"1","article-title":"STDFusionNet: An infrared and visible image fusion network based on salient target detection","volume":"70","author":"Ma","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1016\/j.bspc.2017.10.001","article-title":"Multi-modality medical image fusion based on image decomposition framework and nonsubsampled shearlet transform","volume":"40","author":"Liu","year":"2018","journal-title":"Biomed. Signal Process. Control"},{"key":"ref_4","first-page":"27","article-title":"Image Fusion by Modified Spatial Frequency and Nonsubsampled Shearlet Transform","volume":"10","author":"Liu","year":"2017","journal-title":"Int. J. Signal Process. Image Process. Pattern Recognit."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Liu, J., Fan, X., Huang, Z., Wu, G., Liu, R., Zhong, W., and Luo, Z. (2022, January 18\u201324). Target-aware Dual Adversarial Learning and a Multi-scenario Multi-Modality Benchmark to Fuse Infrared and Visible for Object Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00571"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.inffus.2018.09.004","article-title":"FusionGAN: A generative adversarial network for infrared and visible image fusion","volume":"48","author":"Ma","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.inffus.2021.12.004","article-title":"Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network","volume":"82","author":"Tang","year":"2022","journal-title":"Inf. Fusion"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2614","DOI":"10.1109\/TIP.2018.2887342","article-title":"DenseFuse: A Fusion Approach to Infrared and Visible Images","volume":"28","author":"Li","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2847","DOI":"10.1007\/s11042-020-09647-2","article-title":"Multimodal image fusion based on global-regional-local rule in NSST domain","volume":"80","author":"Zhang","year":"2021","journal-title":"Multimed. Tools Appl."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhao, Z., Xu, S., Zhang, C., Liu, J., Li, P., and Zhang, J. (2020, January 11\u201317). DIDFuse: Deep image decomposition for infrared and visible image fusion. Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence (IJCAI-20), Yokohama, Japan.","DOI":"10.24963\/ijcai.2020\/135"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yang, D., Wang, X., Zhu, N., Li, S., and Hou, N. (2023). MJ-GAN: Generative Adversarial Network with Multi-Grained Feature Extraction and Joint Attention Fusion for Infrared and Visible Image Fusion. Sensors, 23.","DOI":"10.3390\/s23146322"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhu, H., Wu, H., Wang, X., He, D., Liu, Z., and Pan, X. (2023). DPACFuse: Dual-Branch Progressive Learning for Infrared and Visible Image Fusion with Complementary Self-Attention and Convolution. Sensors, 23.","DOI":"10.3390\/s23167205"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.inffus.2018.02.004","article-title":"Infrared and visible image fusion methods and applications: A survey","volume":"45","author":"Ma","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.inffus.2014.09.004","article-title":"A general framework for image fusion based on multi-scale transform and sparse representation","volume":"24","author":"Liu","year":"2015","journal-title":"Inf. Fusion"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"371","DOI":"10.1080\/net.12.3.371.393","article-title":"Can recent innovations in harmonic analysisexplain\u2019key findings in natural image statistics?","volume":"12","author":"Donoho","year":"2001","journal-title":"Netw. Comput. Neural Syst."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Zhao, Y., Cheng, J., Zhou, W., Zhang, C., and Pan, X. (2019, January 18\u201321). Infrared pedestrian detection with converted temperature map. Proceedings of the 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Lanzhou, China.","DOI":"10.1109\/APSIPAASC47483.2019.9023228"},{"key":"ref_17","unstructured":"Zhou, S., Yang, P., and Xie, W. (2011, January 26\u201328). Infrared image segmentation based on Otsu and genetic algorithm. Proceedings of the 2011 International Conference on Multimedia Technology, Hangzhou, China."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"101870","DOI":"10.1016\/j.inffus.2023.101870","article-title":"Rethinking the necessity of image fusion in high-level vision tasks: A practical infrared and visible image fusion network based on progressive semantic injection and scene fidelity","volume":"99","author":"Tang","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1007\/s10278-013-9664-x","article-title":"CT and MR image fusion scheme in nonsubsampled contourlet transform domain","volume":"27","author":"Ganasala","year":"2014","journal-title":"J. Digit. Imaging"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3089","DOI":"10.1109\/TIP.2006.877507","article-title":"The Nonsubsampled Contourlet Transform: Theory, Design, and Applications","volume":"15","author":"Zhou","year":"2006","journal-title":"IEEE Trans. Image Process."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2091","DOI":"10.1109\/TIP.2005.859376","article-title":"The Contourlet Transform: An Efficient Directional Multiresolution Image Representation","volume":"14","author":"Do","year":"2005","journal-title":"IEEE Trans. Image Process."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.acha.2007.09.003","article-title":"Sparse directional image representations using the discrete shearlet transform","volume":"25","author":"Easley","year":"2008","journal-title":"Appl. Comput. Harmon. Anal."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1540","DOI":"10.1016\/j.optcom.2010.11.048","article-title":"A novel algorithm of image fusion using shearlets","volume":"284","author":"Miao","year":"2011","journal-title":"Opt. Commun."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2015). Deep Residual Learning for Image Recognition. arXiv.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., and Yang, M.H. (2022, January 18\u201324). Restormer: Efficient Transformer for High-Resolution Image Restoration. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00564"},{"key":"ref_27","unstructured":"Wu, Z., Liu, Z., Lin, J., Lin, Y., and Han, S. (2020, January 26\u201330). Lite Transformer with Long-Short Range Attention. Proceedings of the International Conference on Learning Representations (ICLR), Addis Ababa, Ethiopia."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1016\/j.inffus.2022.10.034","article-title":"DIVFusion: Darkness-free infrared and visible image fusion","volume":"91","author":"Tang","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1855","DOI":"10.1016\/j.patcog.2004.03.010","article-title":"A wavelet-based image fusion tutorial","volume":"37","author":"Pajares","year":"2004","journal-title":"Pattern Recognit."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1109\/JSEN.2015.2478655","article-title":"Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-Loeve transform","volume":"16","author":"Bavirisetti","year":"2015","journal-title":"IEEE Sens. J."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.infrared.2016.01.009","article-title":"Two-scale image fusion of visible and infrared images using saliency detection","volume":"76","author":"Bavirisetti","year":"2016","journal-title":"Infrared Phys. Technol."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Li, H., and Wu, X.J. (2018). Infrared and visible image fusion using latent low-rank representation. arXiv.","DOI":"10.1109\/ICPR.2018.8546006"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Bavirisetti, D.P., Xiao, G., and Liu, G. (2017, January 10\u201313). Multi-sensor image fusion based on fourth order partial differential equations. Proceedings of the 2017 20th International Conference on Information Fusion (Fusion), Xi\u2019an, China.","DOI":"10.23919\/ICIF.2017.8009719"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.infrared.2017.05.007","article-title":"Infrared and visual image fusion through infrared feature extraction and visual information preservation","volume":"83","author":"Zhang","year":"2017","journal-title":"Infrared Phys. Technol."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"010901","DOI":"10.1117\/1.OE.51.1.010901","article-title":"Progress in color night vision","volume":"51","author":"Toet","year":"2012","journal-title":"Opt. Eng."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Jia, X., Zhu, C., Li, M., Tang, W., and Zhou, W. (2021, January 11\u201317). LLVIP: A visible-infrared paired dataset for low-light vision. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCVW54120.2021.00389"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2959","DOI":"10.1109\/26.477498","article-title":"Image quality measures and their performance","volume":"43","author":"Eskicioglu","year":"1995","journal-title":"IEEE Trans. Commun."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"023522","DOI":"10.1117\/1.2945910","article-title":"Assessment of image fusion procedures using entropy, image quality, and multispectral classification","volume":"2","author":"Roberts","year":"2008","journal-title":"J. Appl. Remote Sens."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1049\/el:20000267","article-title":"Objective image fusion performance measure","volume":"36","author":"Xydeas","year":"2000","journal-title":"Electron. Lett."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1088\/0957-0233\/8\/4\/002","article-title":"In-fibre Bragg grating sensors","volume":"8","author":"Rao","year":"1997","journal-title":"Meas. Sci. Technol."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1016\/j.optcom.2014.12.032","article-title":"Detail preserved fusion of visible and infrared images using regional saliency extraction and multi-scale image decomposition","volume":"341","author":"Cui","year":"2015","journal-title":"Opt. Commun."},{"key":"ref_42","unstructured":"Jocher, G. (2023, March 14). ultralytics\/yolov5. Available online: https:\/\/github.com\/ultralytics\/yolov5."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/19\/8071\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:57:32Z","timestamp":1760129852000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/19\/8071"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,25]]},"references-count":42,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["s23198071"],"URL":"https:\/\/doi.org\/10.3390\/s23198071","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,25]]}}}