{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,19]],"date-time":"2026-09-19T17:26:34Z","timestamp":1789838794126,"version":"4.0.1"},"reference-count":55,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T00:00:00Z","timestamp":1679616000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the Science and Technology Department of Sichuan Province","award":["2023NSFSC1987"],"award-info":[{"award-number":["2023NSFSC1987"]}]},{"name":"the Science and Technology Department of Sichuan Province","award":["Y2022147"],"award-info":[{"award-number":["Y2022147"]}]},{"name":"the Science and Technology Department of Sichuan Province","award":["2019RZJ04"],"award-info":[{"award-number":["2019RZJ04"]}]},{"name":"the Postgraduate Innovation Fund Project of Sichuan University of Science and Engineering","award":["2023NSFSC1987"],"award-info":[{"award-number":["2023NSFSC1987"]}]},{"name":"the Postgraduate Innovation Fund Project of Sichuan University of Science and Engineering","award":["Y2022147"],"award-info":[{"award-number":["Y2022147"]}]},{"name":"the Postgraduate Innovation Fund Project of Sichuan University of Science and Engineering","award":["2019RZJ04"],"award-info":[{"award-number":["2019RZJ04"]}]},{"name":"the Artificial Intelligence Key Laboratory of Sichuan Province","award":["2023NSFSC1987"],"award-info":[{"award-number":["2023NSFSC1987"]}]},{"name":"the Artificial Intelligence Key Laboratory of Sichuan Province","award":["Y2022147"],"award-info":[{"award-number":["Y2022147"]}]},{"name":"the Artificial Intelligence Key Laboratory of Sichuan Province","award":["2019RZJ04"],"award-info":[{"award-number":["2019RZJ04"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The fusion tracking of RGB and thermal infrared image (RGBT) is paid wide attention to due to their complementary advantages. Currently, most algorithms obtain modality weights through attention mechanisms to integrate multi-modalities information. They do not fully exploit the multi-scale information and ignore the rich contextual information among features, which limits the tracking performance to some extent. To solve this problem, this work proposes a new multi-scale feature interactive fusion network (MSIFNet) for RGBT tracking. Specifically, we use different convolution branches for multi-scale feature extraction and aggregate them through the feature selection module adaptively. At the same time, a Transformer interactive fusion module is proposed to build long-distance dependencies and enhance semantic representation further. Finally, a global feature fusion module is designed to adjust the global information adaptively. Numerous experiments on publicly available GTOT, RGBT234, and LasHeR datasets show that our algorithm outperforms the current mainstream tracking algorithms.<\/jats:p>","DOI":"10.3390\/s23073410","type":"journal-article","created":{"date-parts":[[2023,3,24]],"date-time":"2023-03-24T03:16:46Z","timestamp":1679627806000},"page":"3410","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Multi-Scale Feature Interactive Fusion Network for RGBT Tracking"],"prefix":"10.3390","volume":"23","author":[{"given":"Xianbing","family":"Xiao","sequence":"first","affiliation":[{"name":"School of Automation and Information Engineering, Sichuan University of Science and Engineering, Yibin 644000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xingzhong","family":"Xiong","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Key Laboratory of Sichuan Province, Sichuan University of Science and Engineering, Yibin 644000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fanqin","family":"Meng","sequence":"additional","affiliation":[{"name":"Artificial Intelligence Key Laboratory of Sichuan Province, Sichuan University of Science and Engineering, Yibin 644000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhen","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Automation and Information Engineering, Sichuan University of Science and Engineering, Yibin 644000, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"4051","DOI":"10.1109\/TIM.2018.2884039","article-title":"PAC interaction inspection using real-time contact point tracking","volume":"68","author":"Shen","year":"2018","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Mehmood, K., Jalil, A., Ali, A., Khan, B., Murad, M., Khan, W.U., and He, Y. (2020). Context-aware and occlusion handling mechanism for online visual object tracking. Electronics, 10.","DOI":"10.3390\/electronics10010043"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1007\/s00138-013-0570-5","article-title":"Thermal cameras and applications: A survey","volume":"25","author":"Gade","year":"2014","journal-title":"Mach. Vis. Appl."},{"key":"ref_4","unstructured":"Schnelle, S.R., and Chan, A.L. (2011, January 5\u20138). Enhanced target tracking through infrared-visible image fusion. Proceedings of the 14th International Conference on Information Fusion, Chicago, IL, USA."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chan, A.L., and Schnelle, S.R.J.O.E. (2013). Fusing concurrent visible and infrared videos for improved tracking performance. Opt. Eng., 52.","DOI":"10.1117\/1.OE.52.1.017004"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"115756","DOI":"10.1016\/j.image.2019.115756","article-title":"DSiamMFT: An RGB-T fusion tracking method via dynamic Siamese networks using multi-layer feature fusion","volume":"84","author":"Zhang","year":"2020","journal-title":"Signal Process. Image Commun."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1016\/j.neucom.2022.04.017","article-title":"CIRNet: An improved RGBT tracking via cross-modality interaction and re-identification","volume":"493","author":"Xia","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_8","unstructured":"Lu, A., Qian, C., Li, C., Tang, J., and Wang, L. (2022). IEEE Transactions on Neural Networks and Learning Systems, IEEE."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"He, F., Chen, M., Chen, X., Han, J., and Bai, L. (SSRN, 2022). SiamDL: Siamese Dual-Level Fusion Attention Network for RGBT Tracking, SSRN, submitted.","DOI":"10.2139\/ssrn.4209345"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"5757","DOI":"10.1007\/s00521-021-06704-1","article-title":"Response map evaluation for RGBT tracking","volume":"34","author":"Wang","year":"2022","journal-title":"Neural Comput. Appl."},{"key":"ref_11","unstructured":"Long Li, C., Lu, A., Hua Zheng, A., Tu, Z., and Tang, J. (2019, January 27\u201328). Multi-adapter RGBT trackin. Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea."},{"key":"ref_12","unstructured":"Li, C., Liu, L., Lu, A., Ji, Q., and Tang, J. (2020). Part XXII 16, Proceedings of the Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK, 23\u201328 August 2020, Springer."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Li, C., Luo, B., Tang, J., and Wang, X. (2019, January 21\u201325). Dense feature aggregation and pruning for RGBT tracking. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.","DOI":"10.1145\/3343031.3350928"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1109\/TMM.2021.3055362","article-title":"Multimodal cross-layer bilinear pooling for RGBT tracking","volume":"24","author":"Xu","year":"2021","journal-title":"IEEE Trans. Multimedia"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_16","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (July, January 26). Rethinking the inception architecture for computer vision. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Lasvegas, NV, USA."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Ioffe, S., Vanhoucke, V., and Alemi, A. (2017, January 4\u20139). Inception-v4, inception-resnet and the impact of residual connections on learning. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.11231"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1109\/TPAMI.2014.2345390","article-title":"High-speed tracking with kernelized correlation filters","volume":"37","author":"Henriques","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Danelljan, M., Hager, G., Shahbaz Khan, F., and Felsberg, M. (2015, January 7\u201313). Learning spatially regularized correlation filters for visual tracking. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.490"},{"key":"ref_20","unstructured":"Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., and Torr, P.H. (2016). Part II 14, Proceedings of the Computer Vision\u2013ECCV 2016 Workshops, Amsterdam, The Netherlands, 8\u201310, 15\u201316 October 2016, Springer."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, B., Yan, J., Wu, W., Zhu, Z., and Hu, X. (2018, January 18\u201323). High performance visual tracking with siamese region proposal network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00935"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., and Yan, J. (2019, January 16\u201317). Siamrpn++: Evolution of siamese visual tracking with very deep networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00441"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Xu, Y., Wang, Z., Li, Z., Yuan, Y., and Yu, G. (2020, January 7\u201312). Siamfc++: Towards robust and accurate visual tracking with target estimation guidelines. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i07.6944"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, Z., Zhong, B., Li, G., Zhang, S., and Ji, R. (2020, January 13\u201319). Siamese box adaptive network for visual tracking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00670"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Guo, D., Wang, J., Cui, Y., Wang, Z., and Chen, S. (2020, January 13\u201319). SiamCAR: Siamese fully convolutional classification and regression for visual tracking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00630"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Danelljan, M., Bhat, G., Khan, F.S., and Felsberg, M. (2019, January 15\u201320). Atom: Accurate tracking by overlap maximization. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00479"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wang, N., Zhou, W., Wang, J., and Li, H. (2021, January 19\u201325). Transformer meets tracker: Exploiting temporal context for robust visual tracking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00162"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Mayer, C., Danelljan, M., Bhat, G., Paul, M., Paudel, D.P., Yu, F., and Van Gool, L. (2022, January 18\u201324). Transforming model prediction for tracking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00853"},{"key":"ref_29","unstructured":"Wu, Y., Blasch, E., Chen, G., Bai, L., and Ling, H. (2011, January 5\u20138). Multiple source data fusion via sparse representation for robust visual tracking. Proceedings of the 14th International Conference on Information Fusion, Chicago, IL, USA."},{"key":"ref_30","unstructured":"Li, L., Li, C., Tu, Z., and Tang, J. (2018). Proceedings of the Image and Graphics Technologies and Applications: 13th Conference on Image and Graphics Technologies and Applications, IGTA 2018, Beijing, China, 8\u201310 April 2018, Springer. Revised Selected Papers 13."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1016\/j.patrec.2018.10.002","article-title":"Modality-correlation-aware sparse representation for RGB-infrared object tracking","volume":"130","author":"Lan","year":"2020","journal-title":"Pattern Recognit. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Li, C., Zhu, C., Huang, Y., Tang, J., and Wang, L. (2018, January 8\u201314). Cross-modal ranking with soft consistency and noisy labels for robust RGB-T tracking. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01261-8_49"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"370","DOI":"10.1016\/j.neucom.2022.04.032","article-title":"RGBT tracking based on cooperative low-rank graph model","volume":"492","author":"Shen","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Xu, N., Xiao, G., Zhang, X., and Bavirisetti, D.P. (2018, January 24\u201326). Relative object tracking algorithm based on convolutional neural network for visible and infrared video sequences. Proceedings of the 4th International Conference on Virtual Reality, Hong Kong, China.","DOI":"10.1145\/3198910.3198918"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Gao, Y., Li, C., Zhu, Y., Tang, J., He, T., and Wang, F. (2019, January 27\u201328). Deep adaptive fusion network for high performance RGBT tracking. Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea.","DOI":"10.1109\/ICCVW.2019.00017"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Wang, C., Xu, C., Cui, Z., Zhou, L., Zhang, T., Zhang, X., and Yang, J. (2020, January 13\u201319). Cross-modal pattern-propagation for RGB-T tracking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00709"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"2714","DOI":"10.1007\/s11263-021-01495-3","article-title":"Learning adaptive attribute-driven representation for real-time RGB-T tracking","volume":"129","author":"Zhang","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Xiao, Y., Yang, M., Li, C., Liu, L., and Tang, J. (2022, January 7\u201314). Attribute-based progressive fusion network for rgbt tracking. Proceedings of the AAAI Conference on Artificial Intelligence, Washington, DC, USA.","DOI":"10.1609\/aaai.v36i3.20187"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Li, X., Wang, W., Hu, X., and Yang, J. (2019, January 16\u201320). Selective kernel networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00060"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Chatfield, K., Simonyan, K., Vedaldi, A., and Zisserman, A. (2014, January 1\u20135). Return of the devil in the details: Delving deep into convolutional nets. Proceedings of the British Machine Vision Conference 2014, Nottinghamshire, UK.","DOI":"10.5244\/C.28.6"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Chen, X., Yan, B., Zhu, J., Wang, D., Yang, X., and Lu, H. (2021, January 20\u201325). Transformer tracking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00803"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"Imagenet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"5743","DOI":"10.1109\/TIP.2016.2614135","article-title":"Learning collaborative sparse representation for grayscale-thermal tracking","volume":"25","author":"Li","year":"2016","journal-title":"IEEE Trans. Image Process."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"106977","DOI":"10.1016\/j.patcog.2019.106977","article-title":"RGB-T object tracking: Benchmark and baseline","volume":"96","author":"Li","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1109\/TIP.2021.3130533","article-title":"LasHeR: A large-scale high-diversity benchmark for RGBT tracking","volume":"31","author":"Li","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_46","unstructured":"Nam, H., and Han, B. (July, January 26). Learning multi-domain convolutional neural networks for visual tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Li, C., Zhao, N., Lu, Y., Zhu, C., and Tang, J. (2017, January 23\u201327). Weighted sparse representation regularized graph learning for RGB-T object tracking. Proceedings of the 25th ACM International Conference on Multimedia, New York, NY, USA.","DOI":"10.1145\/3123266.3123289"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Muller, M., Bibi, A., Giancola, S., Alsubaihi, S., and Ghanem, B. (2018, January 8\u201314). Trackingnet: A large-scale dataset and benchmark for object tracking in the wild. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01246-5_19"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1109\/TIP.2021.3125504","article-title":"M 5 l: Multi-modal multi-margin metric learning for RGBT tracking","volume":"31","author":"Tu","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"16915","DOI":"10.1109\/JSEN.2021.3078455","article-title":"Hdinet: Hierarchical dual-sensor interaction network for rgbt tracking","volume":"21","author":"Mei","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Zhang, H., Zhang, L., Zhuo, L., and Zhang, J. (2020). Object tracking in RGB-T videos using modal-aware attention network and competitive learning. Sensors, 20.","DOI":"10.3390\/s20020393"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Danelljan, M., Bhat, G., Shahbaz Khan, F., and Felsberg, M. (2017, January 21\u201326). Eco: Efficient convolution operators for tracking. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.733"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Jung, I., Son, J., Baek, M., and Han, B. (2018, January 8\u201314). Real-time mdnet. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01225-0_6"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"3335","DOI":"10.1109\/TIP.2021.3060862","article-title":"Jointly modeling motion and appearance cues for robust RGB-T tracking","volume":"30","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"5613","DOI":"10.1109\/TIP.2021.3087341","article-title":"RGBT tracking via multi-adapter network with hierarchical divergence loss","volume":"30","author":"Lu","year":"2021","journal-title":"IEEE Trans. Image Process."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3410\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:02:03Z","timestamp":1760122923000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3410"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,24]]},"references-count":55,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["s23073410"],"URL":"https:\/\/doi.org\/10.3390\/s23073410","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,24]]}}}