{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T16:08:57Z","timestamp":1781194137759,"version":"3.54.1"},"reference-count":38,"publisher":"MDPI AG","issue":"15","license":[{"start":{"date-parts":[[2022,7,22]],"date-time":"2022-07-22T00:00:00Z","timestamp":1658448000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["62071360"],"award-info":[{"award-number":["62071360"]}]},{"name":"National Natural Science Foundation of China","award":["61571345"],"award-info":[{"award-number":["61571345"]}]},{"name":"National Natural Science Foundation of China","award":["91538101"],"award-info":[{"award-number":["91538101"]}]},{"name":"National Natural Science Foundation of China","award":["61501346"],"award-info":[{"award-number":["61501346"]}]},{"name":"National Natural Science Foundation of China","award":["61502367"],"award-info":[{"award-number":["61502367"]}]},{"name":"National Natural Science Foundation of China","award":["61701360"],"award-info":[{"award-number":["61701360"]}]},{"name":"National Natural Science Foundation of China","award":["JB210103"],"award-info":[{"award-number":["JB210103"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["62071360"],"award-info":[{"award-number":["62071360"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["61571345"],"award-info":[{"award-number":["61571345"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["91538101"],"award-info":[{"award-number":["91538101"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["61501346"],"award-info":[{"award-number":["61501346"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["61502367"],"award-info":[{"award-number":["61502367"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["61701360"],"award-info":[{"award-number":["61701360"]}]},{"name":"Fundamental Research Funds for the Central Universities","award":["JB210103"],"award-info":[{"award-number":["JB210103"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Hyperspectral (HS) videos can describe objects at the material level due to their rich spectral bands, which are more conducive to object tracking compared with color videos. However, the existing HS object trackers cannot make good use of deep-learning models to mine their semantic information due to limited annotation data samples. Moreover, the high-dimensional characteristics of HS videos makes the training of a deep-learning model challenging. To address the above problems, this paper proposes a spatial\u2013spectral cross-correlation embedded dual-transfer network (SSDT-Net). Specifically, first, we propose to use transfer learning to transfer the knowledge of traditional color videos to the HS tracking task and develop a dual-transfer strategy to gauge the similarity between the source and target domain. In addition, a spectral weighted fusion method is introduced to obtain the inputs of the Siamese network, and we propose a spatial\u2013spectral cross-correlation module to better embed the spatial and material information between the two branches of the Siamese network for classification and regression. The experimental results demonstrate that, compared to the state of the art, the proposed SSDT-Net tracker offers more satisfactory performance based on a similar speed to the traditional color trackers.<\/jats:p>","DOI":"10.3390\/rs14153512","type":"journal-article","created":{"date-parts":[[2022,7,25]],"date-time":"2022-07-25T01:42:13Z","timestamp":1658713333000},"page":"3512","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Spatial\u2013Spectral Cross-Correlation Embedded Dual-Transfer Network for Object Tracking Using Hyperspectral Videos"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0851-6565","authenticated-orcid":false,"given":"Jie","family":"Lei","sequence":"first","affiliation":[{"name":"State Key Laboratory of Integrated Service Networks, School of Telecommunications Engineering, Xidian University, Xi\u2019an 710071, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pan","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Integrated Service Networks, School of Telecommunications Engineering, Xidian University, Xi\u2019an 710071, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8310-024X","authenticated-orcid":false,"given":"Weiying","family":"Xie","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Integrated Service Networks, School of Telecommunications Engineering, Xidian University, Xi\u2019an 710071, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Long","family":"Gao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Integrated Service Networks, School of Telecommunications Engineering, Xidian University, Xi\u2019an 710071, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yunsong","family":"Li","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Integrated Service Networks, School of Telecommunications Engineering, Xidian University, Xi\u2019an 710071, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qian","family":"Du","sequence":"additional","affiliation":[{"name":"The Department of Electrical and Computer Engineering, Mississippi State University, Starkville, MS 39762, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,7,22]]},"reference":[{"key":"ref_1","first-page":"1","article-title":"Deep Learning for Visual Tracking: A Comprehensive Survey","volume":"23","author":"Cheng","year":"2021","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1845","DOI":"10.1109\/TIP.2017.2656628","article-title":"Deep Relative Tracking","volume":"26","author":"Gao","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"476","DOI":"10.1016\/j.patcog.2017.11.024","article-title":"Material based salient object detection from hyperspectral images","volume":"76","author":"Liang","year":"2018","journal-title":"Pattern Recognit."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"107298","DOI":"10.1016\/j.patcog.2020.107298","article-title":"Deep support vector machine for hyperspectral image classification","volume":"103","author":"Okwuashi","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Qian, K., Zhou, J., Xiong, F., Zhou, H., and Du, J. (2018). Object tracking in hyperspectral videos with convolutional features and kernelized correlation filter. International Conference on Smart Multimedia, Springer.","DOI":"10.1007\/978-3-030-04375-9_26"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3719","DOI":"10.1109\/TIP.2020.2965302","article-title":"Material based object tracking in hyperspectral videos","volume":"29","author":"Xiong","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1109\/TGRS.2018.2856370","article-title":"Tracking in aerial hyperspectral videos using deep kernelized correlation filters","volume":"57","author":"Uzkent","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Li, Z., Xiong, F., Zhou, J., Wang, J., Lu, J., and Qian, Y. (2020, January 25\u201328). BAE-Net: A Band Attention Aware Ensemble Network for Hyperspectral Object Tracking. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/ICIP40778.2020.9191105"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Song, Y., Ma, C., Wu, X., Gong, L., Bao, L., Zuo, W., Shen, C., Lau, R.W., and Yang, M.H. (2018, January 18\u201323). Vital: Visual tracking via adversarial learning. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00937"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Li, Z., Ye, X., Xiong, F., Lu, J., Zhou, J., and Qian, Y. (2021, March 26). Spectral-Spatial-Temporal Attention Network for Hyperspectral Tracking. Available online: http:\/\/www.ieee-whispers.com\/wp-content\/uploads\/2021\/03\/WHISPERS_2021_paper_55.pdf.","DOI":"10.1109\/WHISPERS52202.2021.9484032"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Liu, Z., Wang, X., Shu, M., Li, G., Sun, C., Liu, Z., and Zhong, Y. (2021, March 26). An Anchor-Free Siamese Target Tracking Network for Hyperspectral Video. Available online: http:\/\/www.ieee-whispers.com\/wp-content\/uploads\/2021\/03\/WHISPERS_2021_paper_52.pdf.","DOI":"10.1109\/WHISPERS52202.2021.9483958"},{"key":"ref_13","first-page":"1345","article-title":"A survey on transfer learning","volume":"22","author":"Pan","year":"2009","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A., and Torr, P.H. (2016). Fully-convolutional siamese networks for object tracking. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-48881-3_56"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Held, D., Thrun, S., and Savarese, S. (2016). Learning to track at 100 fps with deep regression networks. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46448-0_45"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Li, B., Yan, J., Wu, W., Zhu, Z., and Hu, X. (2018, January 18\u201323). High performance visual tracking with siamese region proposal network. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00935"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Valmadre, J., Bertinetto, L., Henriques, J., Vedaldi, A., and Torr, P.H. (2017, January 21\u201326). End-to-end representation learning for correlation filter based tracking. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.531"},{"key":"ref_18","unstructured":"Wang, Q., Gao, J., Xing, J., Zhang, M., and Hu, W. (2017). Dcfnet: Discriminant correlation filters network for visual tracking. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhu, Z., Wang, Q., Li, B., Wu, W., Yan, J., and Hu, W. (2018). Distractor-aware siamese networks for visual object tracking. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-030-01240-3_7"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Pitie, F., and Kokaram, A. (2007, January 27\u201328). The linear monge-kantorovitch linear colour mapping for example-based colour transfer. Proceedings of the 4th European Conference on Visual Media Production, London, UK.","DOI":"10.1049\/cp:20070055"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"He, A., Luo, C., Tian, X., and Zeng, W. (2018, January 18\u201323). A twofold siamese network for real-time object tracking. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00508"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Li, B., Wu, W., Wang, Q., Zhang, F., Xing, J., and Yan, J. (2019, January 15\u201320). Siamrpn++: Evolution of siamese visual tracking with very deep networks. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00441"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Guo, Q., Feng, W., Zhou, C., Huang, R., Wan, L., and Wang, S. (2017, January 22\u201329). Learning dynamic siamese network for visual object tracking. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.196"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Han, Y., Huang, G., Song, S., Yang, L., Wang, H., and Wang, Y. (2021). Dynamic neural networks: A survey. IEEE Trans. Pattern Anal. Mach. Intell.","DOI":"10.1109\/TPAMI.2021.3117837"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4218","DOI":"10.1109\/TGRS.2018.2890212","article-title":"Structure tensor and guided filtering-based algorithm for hyperspectral anomaly detection","volume":"57","author":"Xie","year":"2019","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_28","first-page":"670","article-title":"Robust Visual Tracking via Hierarchical Convolutional Features","volume":"25","author":"Ma","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Doll\u00e1r, P., and Zitnick, C.L. (2014). Microsoft coco: Common objects in context. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Real, E., Shlens, J., Mazzocchi, S., Pan, X., and Vanhoucke, V. (2017, January 21\u201326). Youtube-boundingboxes: A large high-precision human-annotated data set for object detection in video. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.789"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/s11263-015-0816-y","article-title":"Imagenet large scale visual recognition challenge","volume":"115","author":"Russakovsky","year":"2015","journal-title":"Int. J. Comput. Vis."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"He, K., Gkioxari, G., Doll\u00e1r, P., and Girshick, R. (2017, January 22\u201329). Mask r-cnn. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.322"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Long, J., Shelhamer, E., and Darrell, T. (2015, January 7\u201312). Fully convolutional networks for semantic segmentation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Kiani Galoogahi, H., Fagg, A., and Lucey, S. (2017, January 22\u201329). Learning background-aware correlation filters for visual tracking. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.129"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1561","DOI":"10.1109\/TPAMI.2016.2609928","article-title":"Discriminative scale space tracking","volume":"39","author":"Danelljan","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Wang, N., Zhou, W., Tian, Q., Hong, R., Wang, M., and Li, H. (2018, January 18\u201323). Multi-cue correlation filters for robust visual tracking. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00509"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Danelljan, M., Hager, G., Shahbaz Khan, F., and Felsberg, M. (2015, January 7\u201313). Learning spatially regularized correlation filters for visual tracking. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.490"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Guo, D., Wang, J., Cui, Y., Wang, Z., and Chen, S. (2020, January 13\u201319). SiamCAR: Siamese fully convolutional classification and regression for visual tracking. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00630"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/15\/3512\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T23:54:46Z","timestamp":1760140486000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/14\/15\/3512"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,22]]},"references-count":38,"journal-issue":{"issue":"15","published-online":{"date-parts":[[2022,8]]}},"alternative-id":["rs14153512"],"URL":"https:\/\/doi.org\/10.3390\/rs14153512","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7,22]]}}}