{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T16:19:43Z","timestamp":1791303583504,"version":"4.1.0"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"10","license":[{"start":{"date-parts":[[2024,10,29]],"date-time":"2024-10-29T00:00:00Z","timestamp":1730160000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2023YFF1105102, 2023YFF1105105"],"award-info":[{"award-number":["2023YFF1105102, 2023YFF1105105"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62020106012, 62332008, 62106089, U1836218, 62336004"],"award-info":[{"award-number":["62020106012, 62332008, 62106089, U1836218, 62336004"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"111 Project of Ministry of Education of China","award":["B12018"],"award-info":[{"award-number":["B12018"]}]},{"name":"UK EPSRC","award":["EP\/N007743\/1, MURI\/EPSRC\/DSTL, EP\/R018456\/1"],"award-info":[{"award-number":["EP\/N007743\/1, MURI\/EPSRC\/DSTL, EP\/R018456\/1"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,10,31]]},"abstract":"<jats:p>\n            Due to the limitations of visible (RGB) sensors in challenging scenarios, such as nighttime and foggy environments, the thermal infrared (TIR) modality draws increasing attention as an auxiliary source for robust tracking systems. Currently, the existing methods extract both the RGB and TIR (RGBT) clues in a similar approach, i.e., utilising RGB-pretrained models with or without finetuning, and then aggregate the multi-modal information through a fusion block embedded in a single level. However, the different imaging principles of RGB and TIR data raise questions about the suitability of RGB-pretrained models for thermal data. In this article, it is argued that the modality gap is overlooked, and an alternative training paradigm is proposed for TIR data to ensure consistency between the training and test data, which is achieved by optimising the TIR feature extractor with only TIR data involved. Furthermore, with the goal of making better use of the enhanced thermal representations, a multi-level fusion strategy is inspired by the observation that various fusion strategies at different levels can contribute to a better performance. Specifically, fusion modules at both the feature and decision levels are derived for a comprehensive fusion procedure while the pixel-level fusion strategy is not considered due to the misalignment of multi-modal image pairs. The effectiveness of our method is demonstrated by extensive qualitative and quantitative experiments conducted on several challenging benchmarks. Code will be released at\n            <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"url\" xlink:href=\"https:\/\/github.com\/Zhangyong-Tang\/MELT\">https:\/\/github.com\/Zhangyong-Tang\/MELT<\/jats:ext-link>\n            .\n          <\/jats:p>","DOI":"10.1145\/3678176","type":"journal-article","created":{"date-parts":[[2024,7,15]],"date-time":"2024-07-15T13:33:20Z","timestamp":1721050400000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Multi-Level Fusion for Robust RGBT Tracking via Enhanced Thermal Representation"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8187-9384","authenticated-orcid":false,"given":"Zhangyong","family":"Tang","sequence":"first","affiliation":[{"name":"School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9015-3128","authenticated-orcid":false,"given":"Tianyang","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0199-5001","authenticated-orcid":false,"given":"Xiao-Jun","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8110-9205","authenticated-orcid":false,"given":"Josef","family":"Kittler","sequence":"additional","affiliation":[{"name":"Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,10,29]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-48881-3_56"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i2.27852"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00803"},{"issue":"6","key":"e_1_3_2_5_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3596496","article-title":"Complementary Coarse-to-Fine Matching for Video Object Segmentation","volume":"19","author":"Chen Zhen","year":"2023","unstructured":"Zhen Chen, Ming Yang, and Shiliang Zhang. 2023. Complementary Coarse-to-Fine Matching for Video Object Segmentation. ACM Transactions on Multimedia Computing, Communications and Applications 19, 6 (2023), 1\u201321.","journal-title":"ACM Transactions on Multimedia Computing, Communications and Applications"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00670"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2007.383433"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jvcir.2020.102881"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-20047-2_9"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2019.00017"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3131342"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01310"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01225-0_6"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2019.00276"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2016.2614135"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2019.106977"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58542-6_14"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2021.3130533"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/3123266.3123289"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2019.00279"},{"issue":"6","key":"e_1_3_2_22_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3595379","article-title":"PANet: An End-To-End Network Based on Relative Motion for Online Multi-Object Tracking","volume":"19","author":"Li Rui","year":"2023","unstructured":"Rui Li, Baopeng Zhang, Wei Liu, Zhu Teng, and Jianping Fan. 2023. PANet: An End-To-End Network Based on Relative Motion for Online Multi-Object Tracking. ACM Transactions on Multimedia Computing, Communications and Applications 19, 6 (2023), 1\u201321.","journal-title":"ACM Transactions on Multimedia Computing, Communications and Applications"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i07.6828"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2023.3236895"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2021.3087341"},{"key":"e_1_3_2_26_2","first-page":"1","article-title":"Duality-Gated Mutual Condition Network for RGBT Tracking","author":"Lu Andong","year":"2022","unstructured":"Andong Lu, Cun Qian, Chenglong Li, Jin Tang, and Liang Wang. 2022. Duality-Gated Mutual Condition Network for RGBT Tracking. IEEE Transactions on Neural Networks and Learning Systems (2022), 1\u201314.","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.515"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infrared.2019.04.017"},{"key":"e_1_3_2_29_2","first-page":"354","volume-title":"Proceedings of the Informatik 2006 - Informatik f\u00fcr Menschen, Band 1, Beitr\u00e4ge der 36. Jahrestagung der Gesellschaft f\u00fcr Informatik e.V. (GI)","author":"Mihaylova L.","year":"2006","unstructured":"L. Mihaylova, A. Loza, S. G. Nikolov, J. J. Lewis, and D. R. Bull. 2006. The Influence of Multi-Sensor Video Fusion on Object Tracking Using a Particle Filter. In Proceedings of the Informatik 2006 - Informatik f\u00fcr Menschen, Band 1, Beitr\u00e4ge der 36. Jahrestagung der Gesellschaft f\u00fcr Informatik e.V. (GI) 2006, 354\u2013358."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01246-5_19"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.465"},{"key":"e_1_3_2_32_2","first-page":"8886","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Pengyu Zhang","year":"2022","unstructured":"Zhang Pengyu, Jie Zhao, Dong Wang, Huchuan Lu, and Xiang Ruan. 2022. Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 8886\u20138895."},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01495-3"},{"key":"e_1_3_2_34_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","volume":"31","author":"Pu Shi","year":"2018","unstructured":"Shi Pu, Yibing Song, Chao Ma, Honggang Zhang, and Ming-Hsuan Yang. 2018. Deep Attentive Tracking via Reciprocative Learning. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 31."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2023.3273451"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.rcim.2022.102476"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2023.101881"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i6.28325"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","unstructured":"Zhangyong Tang Tianyang Xu and Xiao-Jun Wu. 2022. Temporal Aggregation for Adaptive RGBT Tracking. arXiv:2201.08949. Retrieved from https:\/\/doi.org\/10.48550\/arxiv.2201.08949","DOI":"10.48550\/arxiv.2201.08949"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00709"},{"issue":"6","key":"e_1_3_2_41_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3588574","article-title":"Deep Convolutional Pooling Transformer for Deepfake Detection","volume":"19","author":"Wang Tianyi","year":"2023","unstructured":"Tianyi Wang, Harry Cheng, Kam Pui Chow, and Liqiang Nie. 2023. Deep Convolutional Pooling Transformer for Deepfake Detection. ACM Transactions on Multimedia Computing, Communications and Applications 19, 6 (2023), 1\u201320.","journal-title":"ACM Transactions on Multimedia Computing, Communications and Applications"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i3.20187"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/TMM.2021.3055362"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00804"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-20047-2_20"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3486678"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.3390\/s20020393"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2019.00278"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2018.2879249"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/s41095-023-0345-5"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2021.3060862"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2021.3072207"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2936914"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.image.2019.115756"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00472"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2022.3212987"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2022.3212987"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00918"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-024-01999-8"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i3.25500"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3343031.3350928"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIV.2020.2980735"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2021.3067997"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678176","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3678176","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:54:08Z","timestamp":1750287248000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3678176"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,29]]},"references-count":62,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2024,10,31]]}},"alternative-id":["10.1145\/3678176"],"URL":"https:\/\/doi.org\/10.1145\/3678176","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,29]]},"assertion":[{"value":"2023-11-29","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-08","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}