{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T05:02:59Z","timestamp":1780462979385,"version":"3.54.1"},"reference-count":45,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2023,7,22]],"date-time":"2023-07-22T00:00:00Z","timestamp":1689984000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Key Research and Development Program of China","award":["2022YFB3901800"],"award-info":[{"award-number":["2022YFB3901800"]}]},{"name":"National Key Research and Development Program of China","award":["2022YFB3901805"],"award-info":[{"award-number":["2022YFB3901805"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>RGB-T tracking involves the use of images from both visible and thermal modalities. The primary objective is to adaptively leverage the relatively dominant modality in varying conditions to achieve more robust tracking compared to single-modality tracking. An RGB-T tracker based on a mixed-attention mechanism to achieve a complementary fusion of modalities (referred to as MACFT) is proposed in this paper. In the feature extraction stage, we utilize different transformer backbone branches to extract specific and shared information from different modalities. By performing mixed-attention operations in the backbone to enable information interaction and self-enhancement between the template and search images, a robust feature representation is constructed that better understands the high-level semantic features of the target. Then, in the feature fusion stage, a modality shared-specific feature interaction structure was designed based on a mixed-attention mechanism, effectively suppressing low-quality modality noise while enhancing the information from the dominant modality. Evaluation on multiple RGB-T public datasets demonstrates that our proposed tracker outperforms other RGB-T trackers on general evaluation metrics while also being able to adapt to long-term tracking scenarios.<\/jats:p>","DOI":"10.3390\/s23146609","type":"journal-article","created":{"date-parts":[[2023,7,24]],"date-time":"2023-07-24T03:03:25Z","timestamp":1690167805000},"page":"6609","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":29,"title":["Learning Modality Complementary Features with Mixed Attention Mechanism for RGB-T Tracking"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-1510-2903","authenticated-orcid":false,"given":"Yang","family":"Luo","sequence":"first","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100040, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiqing","family":"Guo","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100040, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mingtao","family":"Dong","sequence":"additional","affiliation":[{"name":"Institute of Image Recognition and Machine Intelligence, Northeastern University, Shenyang 110167, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jin","family":"Yu","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China"},{"name":"University of Chinese Academy of Sciences, Beijing 100040, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,7,22]]},"reference":[{"key":"ref_1","first-page":"2831","article-title":"Attribute-Based Progressive Fusion Network for RGBT Tracking","volume":"36","author":"Xiao","year":"2022","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_2","unstructured":"Tang, Z., Xu, T., and Wu, X.-J. (2022). A Survey for Deep RGBT Tracking. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, H., Zhang, L., Zhuo, L., and Zhang, J. (2020). Object Tracking in RGB-T Videos Using Modal-Aware Attention Network and Competitive Learning. Sensors, 20.","DOI":"10.3390\/s20020393"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2714","DOI":"10.1007\/s11263-021-01495-3","article-title":"Learning Adaptive Attribute-Driven Representation for Real-Time RGB-T Tracking","volume":"129","author":"Zhang","year":"2021","journal-title":"Int. J. Comput. Vis."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"122122","DOI":"10.1109\/ACCESS.2019.2936914","article-title":"SiamFT: An RGB-Infrared Fusion Tracking Method via Fully Convolutional Siamese Networks","volume":"7","author":"Zhang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Bhat, G., Danelljan, M., Van Gool, L., and Timofte, R. (November, January 27). Learning Discriminative Model Prediction for Tracking. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea.","DOI":"10.1109\/ICCV.2019.00628"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1109\/TCSVT.2021.3067997","article-title":"RGBT Tracking by Trident Fusion Network","volume":"32","author":"Zhu","year":"2022","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Nam, H., and Han, B. (2016, January 27\u201330). Learning Multi-domain Convolutional Neural Networks for Visual Tracking. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.465"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"5613","DOI":"10.1109\/TIP.2021.3087341","article-title":"RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss","volume":"30","author":"Lu","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_10","unstructured":"Long Li, C., Lu, A., Hua Zheng, A., Tu, Z., and Tang, J. (November, January 27). Multi-adapter RGBT tracking. Proceedings of the IEEE\/CVF International Conference on Computer Vision Workshops, Seoul, Republic of Korea."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1109\/TIV.2020.2980735","article-title":"Quality-Aware Feature Aggregation Network for Robust RGBT Tracking","volume":"6","author":"Zhu","year":"2021","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Lu, A., Qian, C., Li, C., Tang, J., and Wang, L. (2022). Duality-Gated Mutual Condition Network for RGBT Tracking. IEEE Trans. Neural Netw. Learn. Syst., 1\u201314.","DOI":"10.1109\/TNNLS.2022.3157594"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Gao, Y., Li, C., Zhu, Y., Tang, J., He, T., and Wang, F. (2019, January 27\u201328). Deep Adaptive Fusion Network for High Performance RGBT Tracking. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea.","DOI":"10.1109\/ICCVW.2019.00017"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Li, C., Luo, B., Tang, J., and Wang, X. (2019, January 21\u201325). Dense Feature Aggregation and Pruning for RGBT Tracking. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.","DOI":"10.1145\/3343031.3350928"},{"key":"ref_15","unstructured":"Li, C., Liu, L., Lu, A., Ji, Q., and Tang, J. (2020). Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK, August 23\u201328 2020, Proceedings, Part XXII 16, Springer."},{"key":"ref_16","first-page":"4905","article-title":"Understanding the Effective Receptive Field in Deep Convolutional Neural Networks","volume":"29","author":"Luo","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_17","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2010). An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Danelljan, M., Bhat, G., Khan, F.S., and Felsberg, M. (2019, January 15\u201320). Atom: Accurate tracking by overlap maximization. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00479"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Jung, I., Son, J., Baek, M., and Han, B. (2018, January 8\u201314). Real-Time MDNet. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01225-0_6"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Li, B., Yan, J., Wu, W., Zhu, Z., and Hu, X. (2018, January 18\u201322). High Performance Visual Tracking with Siamese Region Proposal Network. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00935"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"850","DOI":"10.1007\/978-3-319-48881-3_56","article-title":"Fully-Convolutional Siamese Networks for Object Tracking","volume":"Volume 9914","author":"Hua","year":"2016","journal-title":"Computer Vision\u2014ECCV 2016 Workshops"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., and Yan, J. (2019, January 15\u201320). SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00441"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, X., Yan, B., Zhu, J., Wang, D., Yang, X., and Lu, H. (2021, January 19\u201325). Transformer Tracking. Proceedings of the 2021 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00803"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Yan, B., Peng, H., Fu, J., Wang, D., and Lu, H. (2021, January 11\u201317). Learning Spatio-Temporal Transformer for Visual Tracking. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.01028"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Cui, Y., Jiang, C., Wang, L., and Wu, G. (2022, January 18\u201324). MixFormer: End-to-End Tracking with Iterative Mixed Attention. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.01324"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"111997","DOI":"10.1016\/j.measurement.2022.111997","article-title":"A novel entropy-based sparsity measure for prognosis of bearing defects and development of a sparsogram to select sensitive filtering band of an axial piston pump","volume":"203","author":"Zhou","year":"2022","journal-title":"Measurement"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1504\/IJHM.2022.125090","article-title":"TCM in milling processes based on attention mechanism-combined long short-term memory using a sound sensor under different working conditions","volume":"5","author":"Zheng","year":"2022","journal-title":"Int. J. Hydromechatron."},{"key":"ref_29","unstructured":"Chen, B., Li, P., Bai, L., Qiao, L., Shen, Q., Li, B., Gan, W., Wu, W., and Ouyang, W. (2022). Computer Vision\u2013ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23\u201327 2022, Proceedings, Part XXII, Springer."},{"key":"ref_30","unstructured":"Wang, Y., Li, C., and Tang, J. (2018). Chinese Conference on Pattern Recognition and Computer Vision (PRCV), Springer."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3335","DOI":"10.1109\/TIP.2021.3060862","article-title":"Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking","volume":"30","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Image Process."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Danelljan, M., Bhat, G., Khan, F.S., and Felsberg, M. (2017, January 21\u201326). ECO: Efficient Convolution Operators for Tracking. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.733"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1403","DOI":"10.1109\/TCSVT.2021.3072207","article-title":"SiamCDA: Complementarity-and distractor-aware RGB-T tracking based on Siamese network","volume":"32","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Li, X., Wang, W., Hu, X., and Yang, J. (2019, January 15\u201320). Selective kernel networks. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00060"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Zhang, P., Zhao, J., Wang, D., Lu, H., and Ruan, X. (2022, January 18\u201324). Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00868"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhang, L., Danelljan, M., Gonzalez-Garcia, A., van de Weijer, J., and Shahbaz Khan, F. (2019, January 27\u201328). Multi-Modal Fusion for End-to-End RGB-T Tracking. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea.","DOI":"10.1109\/ICCVW.2019.00278"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Zhu, J., Lai, S., Chen, X., Wang, D., and Lu, H. (2023, January 18\u201322). Visual prompt multi-modal tracking. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Vancouver, CAN.","DOI":"10.1109\/CVPR52729.2023.00918"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., and Lim, S.-N. (2022, January 23\u201327). Visual prompt tuning. Proceedings of the European Conference on Computer Vision, Tel Aviv, Israel.","DOI":"10.1007\/978-3-031-19827-4_41"},{"key":"ref_39","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18\u201324). Learning transferable visual models from natural language supervision. Proceedings of the International Conference on Machine Learning, Virtual Event."},{"key":"ref_40","unstructured":"Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. (2020). Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK, August 23\u201328, 2020, Proceedings, Part I 16, Springer."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"106977","DOI":"10.1016\/j.patcog.2019.106977","article-title":"RGB-T Object Tracking: Benchmark and Baseline","volume":"96","author":"Li","year":"2019","journal-title":"Pattern Recognit."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1109\/TIP.2021.3130533","article-title":"LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking","volume":"31","author":"Li","year":"2022","journal-title":"IEEE Trans. Image Process."},{"key":"ref_43","unstructured":"Loshchilov, I., and Hutter, F. (2017). Decoupled weight decay regularization. arXiv."},{"key":"ref_44","first-page":"11037","article-title":"GlobalTrack: A Simple and Strong Baseline for Long-Term Tracking","volume":"34","author":"Huang","year":"2020","journal-title":"Proc. AAAI Conf. Artif. Intell."},{"key":"ref_45","unstructured":"Kristan, M., Matas, J., Leonardis, A., Felsberg, M., Pflugfelder, R., Kamarainen, J.-K., Cehovin Zajc, L., Drbohlav, O., Lukezic, A., and Berg, A. (2019, January 27\u201328). The Seventh Visual Object Tracking VOT2019 Challenge Results. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision Workshop (ICCVW), Seoul, Republic of Korea."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/14\/6609\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:17:09Z","timestamp":1760127429000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/14\/6609"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,22]]},"references-count":45,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2023,7]]}},"alternative-id":["s23146609"],"URL":"https:\/\/doi.org\/10.3390\/s23146609","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,22]]}}}