{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,17]],"date-time":"2026-06-17T22:12:30Z","timestamp":1781734350335,"version":"3.54.5"},"reference-count":55,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T00:00:00Z","timestamp":1702944000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004739","name":"Youth Innovation Promotion Association CAS","doi-asserted-by":"publisher","award":["2014216"],"award-info":[{"award-number":["2014216"]}],"id":[{"id":"10.13039\/501100004739","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>In the field of image fusion, the integration of infrared and visible images aims to combine complementary features into a unified representation. However, not all regions within an image bear equal importance. Target objects, often pivotal in subsequent decision-making processes, warrant particular attention. Conventional deep-learning approaches for image fusion primarily focus on optimizing textural detail across the entire image at a pixel level, neglecting the pivotal role of target objects and their relevance to downstream visual tasks. In response to these limitations, TDDFusion, a Target-Driven Dual-Branch Fusion Network, has been introduced. It is explicitly designed to enhance the prominence of target objects within the fused image, thereby bridging the existing performance disparity between pixel-level fusion and downstream object detection tasks. The architecture consists of a parallel, dual-branch feature extraction network, incorporating a Global Semantic Transformer (GST) and a Local Texture Encoder (LTE). During the training phase, a dedicated object detection submodule is integrated to backpropagate semantic loss into the fusion network, enabling task-oriented optimization of the fusion process. A novel loss function is devised, leveraging target positional information to amplify visual contrast and detail specific to target objects. Extensive experimental evaluation on three public datasets demonstrates the model\u2019s superiority in preserving global environmental information and local detail, outperforming state-of-the-art alternatives in balancing pixel intensity and maintaining the texture of target objects. Most importantly, it exhibits significant advantages in downstream object detection tasks.<\/jats:p>","DOI":"10.3390\/s24010020","type":"journal-article","created":{"date-parts":[[2023,12,19]],"date-time":"2023-12-19T11:17:24Z","timestamp":1702984644000},"page":"20","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["TDDFusion: A Target-Driven Dual Branch Network for Infrared and Visible Image Fusion"],"prefix":"10.3390","volume":"24","author":[{"given":"Siyu","family":"Lu","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, ShanghaiTech University, Shanghai 201210, China"},{"name":"Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China"},{"name":"Key Laboratory of Infrared System Detection and Imaging Technology, Chinese Academy of Sciences, Shanghai 200083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiangzhou","family":"Ye","sequence":"additional","affiliation":[{"name":"Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China"},{"name":"Key Laboratory of Infrared System Detection and Imaging Technology, Chinese Academy of Sciences, Shanghai 200083, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3008-1705","authenticated-orcid":false,"given":"Junmin","family":"Rao","sequence":"additional","affiliation":[{"name":"Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China"},{"name":"Key Laboratory of Infrared System Detection and Imaging Technology, Chinese Academy of Sciences, Shanghai 200083, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fanming","family":"Li","sequence":"additional","affiliation":[{"name":"Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China"},{"name":"Key Laboratory of Infrared System Detection and Imaging Technology, Chinese Academy of Sciences, Shanghai 200083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shijian","family":"Liu","sequence":"additional","affiliation":[{"name":"Shanghai Institute of Technical Physics, Chinese Academy of Sciences, Shanghai 200083, China"},{"name":"Key Laboratory of Infrared System Detection and Imaging Technology, Chinese Academy of Sciences, Shanghai 200083, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Zeng, Y., Zhang, D., Wang, C., Miao, Z., Liu, T., Zhan, X., Hao, D., and Ma, C. (2022, January 18\u201324). Lift: Learning 4D lidar image fusion transformer for 3D object detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01666"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"206","DOI":"10.1016\/j.inffus.2018.06.005","article-title":"Pedestrian detection with unsupervised multispectral feature learning using deep neural networks","volume":"46","author":"Cao","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_3","first-page":"1","article-title":"Illumination-Guided RGBT Object Detection With Inter-and Intra-Modality Fusion","volume":"72","author":"Zhang","year":"2023","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"7831","DOI":"10.1109\/TITS.2021.3073046","article-title":"Adaptive fusion CNN features for RGBT object tracking","volume":"23","author":"Wang","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.inffus.2021.12.004","article-title":"Image fusion in the loop of high-level vision tasks: A semantic-aware real-time infrared and visible image fusion network","volume":"82","author":"Tang","year":"2022","journal-title":"Inf. Fusion"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, X., Li, X., and Liu, W. (2023). CBFM: Contrast Balance Infrared and Visible Image Fusion Based on Contrast-Preserving Guided Filter. Remote Sens., 15.","DOI":"10.3390\/rs15122969"},{"key":"ref_7","first-page":"12","article-title":"Image fusion technology for security and surveillance applications","volume":"Volume 6402","author":"Riley","year":"2006","journal-title":"Optics and Photonics for Counterterrorism and Crime Fighting II"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4224","DOI":"10.1109\/TII.2018.2822828","article-title":"Object classification using CNN-based fusion of vision and LIDAR in autonomous vehicle environment","volume":"14","author":"Gao","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_9","unstructured":"Han, X., Ma, J., Le, Z., Jiang, J., and Guo, X. (2020, January 7\u201312). Fusiondn: A unified densely connected network for image fusion. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhu, H., Wu, H., Wang, X., He, D., Liu, Z., and Pan, X. (2023). DPACFuse: Dual-Branch Progressive Learning for Infrared and Visible Image Fusion with Complementary Self-Attention and Convolution. Sensors, 23.","DOI":"10.3390\/s23167205"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"5413","DOI":"10.1109\/TMM.2022.3192661","article-title":"YDTR: Infrared and visible image fusion via Y-shape dynamic transformer","volume":"25","author":"Tang","year":"2022","journal-title":"IEEE Trans. Multimed."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yin, R., Yang, B., Huang, Z., and Zhang, X. (2023). DSA-Net: Infrared and Visible Image Fusion via Dual-Stream Asymmetric Network. Sensors, 23.","DOI":"10.20944\/preprints202306.2220.v1"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1186","DOI":"10.1109\/TCSVT.2021.3075745","article-title":"Efficient and model-based infrared and visible image fusion via algorithm unrolling","volume":"32","author":"Zhao","year":"2021","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhao, Z., Xu, S., Zhang, C., Liu, J., Li, P., and Zhang, J. (2020). DIDFuse: Deep image decomposition for infrared and visible image fusion. arXiv.","DOI":"10.24963\/ijcai.2020\/135"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1818","DOI":"10.1109\/LSP.2021.3109818","article-title":"Smoa: Searching a modality-oriented architecture for infrared and visible image fusion","volume":"28","author":"Liu","year":"2021","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"103016","DOI":"10.1016\/j.cviu.2020.103016","article-title":"Infrared and visible image fusion via gradientlet filter","volume":"197","author":"Ma","year":"2020","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4980","DOI":"10.1109\/TIP.2020.2977573","article-title":"DDcGAN: A dual-discriminator conditional generative adversarial network for multi-resolution image fusion","volume":"29","author":"Ma","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"336","DOI":"10.1016\/j.inffus.2022.12.007","article-title":"AT-GAN: A generative adversarial network with attention and transition for infrared and visible image fusion","volume":"92","author":"Rao","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.inffus.2018.09.004","article-title":"FusionGAN: A generative adversarial network for infrared and visible image fusion","volume":"48","author":"Ma","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Hou, J., Zhang, D., Wu, W., Ma, J., and Zhou, H. (2021). A generative adversarial network for infrared and visible image fusion based on semantic segmentation. Entropy, 23.","DOI":"10.3390\/e23030376"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zheng, B., Xiang, T., Lin, M., Cheng, S., and Zhang, P. (2023). Real-Time Semantics-Driven Infrared and Visible Image Fusion Network. Sensors, 23.","DOI":"10.3390\/s23136113"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Liu, Y., Wu, Z., Han, X., Sun, Q., Zhao, J., and Liu, J. (2022). Infrared and Visible Image Fusion Based on Visual Saliency Map and Image Contrast Enhancement. Sensors, 22.","DOI":"10.3390\/s22176390"},{"key":"ref_23","first-page":"1","article-title":"STDFusionNet: An infrared and visible image fusion network based on salient target detection","volume":"70","author":"Ma","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, Z., Wang, F., Wu, D., and Gao, G. (2022). Infrared and Visible Image Fusion Method Using Salience Detection and Convolutional Neural Network. Sensors, 22.","DOI":"10.3390\/s22145430"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Jia, X., Zhu, C., Li, M., Tang, W., and Zhou, W. (2021, January 11\u201317). LLVIP: A visible-infrared paired dataset for low-light vision. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada. Virtual Conference.","DOI":"10.1109\/ICCVW54120.2021.00389"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1016\/j.dib.2017.09.038","article-title":"The TNO multiband image data collection","volume":"15","author":"Toet","year":"2017","journal-title":"Data Brief"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1109\/TPAMI.2020.3012548","article-title":"U2Fusion: A unified unsupervised image fusion network","volume":"44","author":"Xu","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"103039","DOI":"10.1016\/j.infrared.2019.103039","article-title":"Infrared and visible image fusion with ResNet and zero-phase component analysis","volume":"102","author":"Li","year":"2019","journal-title":"Infrared Phys. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"9645","DOI":"10.1109\/TIM.2020.3005230","article-title":"NestFuse: An infrared and visible image fusion architecture based on nest connection and spatial\/channel attention models","volume":"69","author":"Li","year":"2020","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.inffus.2021.02.023","article-title":"RFN-Nest: An end-to-end residual fusion network for infrared and visible images","volume":"73","author":"Li","year":"2021","journal-title":"Inf. Fusion"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"126248","DOI":"10.1016\/j.neucom.2023.126248","article-title":"Infrared and visible image fusion based on a two-stage class conditioned auto-encoder network","volume":"544","author":"Cao","year":"2023","journal-title":"Neurocomputing"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/j.inffus.2019.07.011","article-title":"IFCNN: A general image fusion framework based on convolutional neural network","volume":"54","author":"Zhang","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"107528","DOI":"10.1016\/j.optlaseng.2023.107528","article-title":"Infrared and visible image fusion based on infrared background suppression","volume":"164","author":"Yang","year":"2023","journal-title":"Opt. Lasers Eng."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1016\/j.inffus.2022.03.007","article-title":"PIAFusion: A progressive infrared and visible image fusion network based on illumination aware","volume":"83","author":"Tang","year":"2022","journal-title":"Inf. Fusion"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"104570","DOI":"10.1016\/j.infrared.2023.104570","article-title":"An attention-guided and wavelet-constrained generative adversarial network for infrared and visible image fusion","volume":"129","author":"Liu","year":"2023","journal-title":"Infrared Phys. Technol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.neucom.2023.01.033","article-title":"THFuse: An infrared and visible image fusion network using transformer and hybrid feature extractor","volume":"527","author":"Chen","year":"2023","journal-title":"Neurocomputing"},{"key":"ref_37","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). \u201cAttention is all you need\u201d. Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_38","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An image is worth 16 \u00d7 16 words: Transformers for image recognition at scale. arXiv."},{"key":"ref_39","unstructured":"Hugo, T., Cord, M., Matthijs, D., Francisco, M., Alexandre, S., and Herve, J. (2021, January 18). Training data-efficient image transformers & distillation through attention. Proceedings of the International Conference on Machine Learning, Virtual."},{"key":"ref_40","unstructured":"Zhu, X., Su, W., Lu, L., Li, B., Wang, X., and Dai, J. (2020). Deformable detr: Deformable transformers for end-to-end object detection. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zheng, S., Lu, J., Zhao, H., Zhu, X., Luo, Z., Wang, Y., Fu, Y., Feng, J., Xiang, T., and Torr, P.H. (2021, January 20\u201325). Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00681"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2022.3216413","article-title":"SwinFuse: A residual swin transformer fusion network for infrared and visible images","volume":"71","author":"Wang","year":"2022","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"109295","DOI":"10.1016\/j.patcog.2022.109295","article-title":"TCCFusion: An infrared and visible image fusion method based on transformer and cross correlation","volume":"137","author":"Tang","year":"2023","journal-title":"Pattern Recognit."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Zhao, Z., Bai, H., Zhang, J., Zhang, Y., Xu, S., Lin, Z., Timofte, R., and Van Gool, L. (2023, January 18\u201322). Cddfuse: Correlation-driven dual-branch feature decomposition for multi-modality image fusion. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00572"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Li, B., Peng, X., Wang, Z., Xu, J., and Feng, D. (2017, January 22\u201329). Aod-net: All-in-one dehazing network. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.511"},{"key":"ref_46","unstructured":"Guo, M., Chen, M., Ma, C., Li, Y., Li, X., and Xie, X. (2020, January 23\u201327). High-level task-driven single image deraining: Segmentation in rainy days. Proceedings of the Neural Information Processing: 27th International Conference, ICONIP 2020, Bangkok, Thailand. Part I 27 2020."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"101828","DOI":"10.1016\/j.inffus.2023.101828","article-title":"An interactively reinforced paradigm for joint infrared-visible image fusion and saliency object detection","volume":"98","author":"Wang","year":"2023","journal-title":"Inf. Fusion"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Liu, J., Fan, X., Huang, Z., Wu, G., Liu, R., Zhong, W., and Luo, Z. (2022, January 18\u201324). Target-aware dual adversarial learning and a multi-scenario multi-modality benchmark to fuse infrared and visible for object detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00571"},{"key":"ref_49","unstructured":"Jocher, G., Stoken, A., Borovec, J., Chaurasia, A., Changyu, L., Hogan, A., Hajek, J., Diaconu, L., Kwon, Y., and Defretin, Y. (2023, October 17). ultralytics\/yolov5: v5. 0-YOLOv5-P6 1280 models, AWS, Supervise. ly and YouTube integrations. Zenodo. Available online: https:\/\/github.com\/ultralytics\/yolov5."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.inffus.2016.02.001","article-title":"Infrared and visible image fusion via gradient transfer and total variation minimization","volume":"31","author":"Ma","year":"2016","journal-title":"Inf. Fusion"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1109\/JSEN.2015.2478655","article-title":"Fusion of infrared and visible sensor images based on anisotropic diffusion and Karhunen-Loeve transform","volume":"16","author":"Bavirisetti","year":"2015","journal-title":"IEEE Sens. J."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"2614","DOI":"10.1109\/TIP.2018.2887342","article-title":"DenseFuse: A fusion approach to infrared and visible images","volume":"28","author":"Li","year":"2018","journal-title":"IEEE Trans. Image Process."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"023522","DOI":"10.1117\/1.2945910","article-title":"Assessment of image fusion procedures using entropy, image quality, and multispectral classification","volume":"2","author":"Roberts","year":"2008","journal-title":"J. Appl. Remote Sens."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1049\/el:20020212","article-title":"Information measure for performance of image fusion","volume":"38","author":"Qu","year":"2002","journal-title":"Electron. Lett."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1016\/j.inffus.2011.08.002","article-title":"A new image fusion performance metric based on visual information fidelity","volume":"14","author":"Han","year":"2013","journal-title":"Inf. Fusion"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/1\/20\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:41:43Z","timestamp":1760132503000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/1\/20"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,19]]},"references-count":55,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["s24010020"],"URL":"https:\/\/doi.org\/10.3390\/s24010020","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,19]]}}}