{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T22:09:03Z","timestamp":1740175743697,"version":"3.37.3"},"reference-count":60,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2023,4,10]],"date-time":"2023-04-10T00:00:00Z","timestamp":1681084800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,4,10]],"date-time":"2023-04-10T00:00:00Z","timestamp":1681084800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Complex Intell. Syst."],"published-print":{"date-parts":[[2023,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Visual tracking is an important field of computer vision research. Although transformer-based trackers have achieved remarkable performance, the transformer structure is globally computationally inefficient, it does not screen important patches, and it cannot focus on key target regions. At the same time, temporal motion features are easily overlooked. To solve these problems, this paper proposes a new method, SKRT, that removes the CNN structure and directly uses a transformer as the backbone network to extract multiframe video features. Then, these feature maps are mixed and superimposed to obtain spatiotemporal information. To focus on important parts efficiently, we use key region extraction to obtain a small set of template and search feature map patches and reinput them into the transformer as a cross-correlation computation. Finally, we predict the position of a tracking object through center-corner prediction. To demonstrate the effectiveness of our method, we conduct experiments on challenging benchmark datasets (GOT-10K, TrackingNet, VOT2018, OTB100, LaSOT), and the results show that SKRT is competitive with other state-of-the-art methods.<\/jats:p>","DOI":"10.1007\/s40747-023-01040-4","type":"journal-article","created":{"date-parts":[[2023,4,10]],"date-time":"2023-04-10T03:13:32Z","timestamp":1681096412000},"page":"5865-5879","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Spatiotemporal key region transformer for visual tracking"],"prefix":"10.1007","volume":"9","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6136-3388","authenticated-orcid":false,"given":"Ruixu","family":"Wu","sequence":"first","affiliation":[]},{"given":"Xianbin","family":"Wen","sequence":"additional","affiliation":[]},{"given":"Liming","family":"Yuan","sequence":"additional","affiliation":[]},{"given":"Haixia","family":"Xu","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2023,4,10]]},"reference":[{"key":"1040_CR1","doi-asserted-by":"crossref","unstructured":"Galoogahi HK, Fagg A, Lucey S (2017) Learning background-aware correlation filters for visual tracking. In International Conference on Computer Vision (ICCV)","DOI":"10.1109\/ICCV.2017.129"},{"issue":"7","key":"1040_CR2","first-page":"1442","volume":"36","author":"AW Smeulders","year":"2013","unstructured":"Smeulders AW, Chu MD, Cucchiara R, Calderara S, Dehghan A (2013) Visual tracking: an experimental survey. IEEE Trans Pattern Anal Mach Intell 36(7):1442\u20131468","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"5","key":"1040_CR3","doi-asserted-by":"publisher","first-page":"1158","DOI":"10.1109\/TPAMI.2018.2829180","volume":"41","author":"W Zuo","year":"2018","unstructured":"Zuo W, Wu X, Lin L, Zhang L, Yang MH (2018) Learning support correlation filters for visual tracking. IEEE Trans Pattern Anal Mach Intell 41(5):1158\u20131172","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"1040_CR4","doi-asserted-by":"crossref","unstructured":"Alismail H, Browning B, Lucey S (2016) Robust tracking in low light and sudden illumination changes. In Fourth International Conference on 3d Vision(3DV), pages 389\u2013398","DOI":"10.1109\/3DV.2016.48"},{"key":"1040_CR5","doi-asserted-by":"crossref","unstructured":"Bolme DS, Beveridge JR, Draper BA, Lui YM (2010) Visual object tracking using adaptive correlation filters. In International Conference on Computer Vision and Pattern Recogintion (CVPR)","DOI":"10.1109\/CVPR.2010.5539960"},{"issue":"2","key":"1040_CR6","doi-asserted-by":"publisher","first-page":"1201","DOI":"10.1007\/s11042-014-2364-9","volume":"75","author":"I Bouchrika","year":"2016","unstructured":"Bouchrika I, Carter JN, Nixon MS (2016) Towards automated visual surveillance using gait for identity recognition and tracking across multiple non-intersecting cameras. Multimed Tools Appl 75(2):1201\u20131221","journal-title":"Multimed Tools Appl"},{"issue":"12","key":"1040_CR7","doi-asserted-by":"publisher","first-page":"1915","DOI":"10.1007\/s11548-015-1243-9","volume":"10","author":"X Du","year":"2015","unstructured":"Du X, Clancy N, Arya S, Hanna GB, Kelly J, Elson DS, Stoyanov D (2015) Robust surface tracking combining features, intensity and illumination compensation. Int J Comput Assist Radiol Surg (IJCARS) 10(12):1915\u20131926","journal-title":"Int J Comput Assist Radiol Surg (IJCARS)"},{"issue":"3","key":"1040_CR8","doi-asserted-by":"publisher","first-page":"583","DOI":"10.1109\/TPAMI.2014.2345390","volume":"37","author":"JF Henriques","year":"2015","unstructured":"Henriques JF, Caseiro R, Martins P, Batista J (2015) High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell 37(3):583\u2013596","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"1","key":"1040_CR9","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1007\/s11390-017-1764-5","volume":"33","author":"K Li","year":"2018","unstructured":"Li K, He FZ, Yu HP (2018) Robust visual tracking based on convolutional features with illumination and occlusion handing. J Comput Sci Technol 33(1):223\u2013236","journal-title":"J Comput Sci Technol"},{"key":"1040_CR10","doi-asserted-by":"crossref","unstructured":"Tokekar P, Isler V, Franchi A (2014) Multi-target visual tracking with aerial robots. In IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","DOI":"10.1109\/IROS.2014.6942986"},{"key":"1040_CR11","doi-asserted-by":"crossref","unstructured":"Bertinetto L, Valmadre J, Henriques Joo\u00a0F, Vedaldi A, Torr Phs (2016) Fully-convolutional siamese networks for object tracking. In European Conference on Computer Vision (ECCV)","DOI":"10.1007\/978-3-319-48881-3_56"},{"key":"1040_CR12","unstructured":"Bo L, Yan J, Wei W, Zheng Z, Hu X (2018) High performance visual tracking with siamese region proposal network. In International Conference on Computer Vision and Pattern Recogintion (CVPR)"},{"key":"1040_CR13","unstructured":"Xin C, Bin Y, Jiawen Z, Dong W, Xiaoyun Y, Huchuan L (2021) Transformer tracking. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 8126\u20138135"},{"key":"1040_CR14","doi-asserted-by":"crossref","unstructured":"Bin Yan, Houwen Peng, Jianlong Fu, Dong Wang, and Huchuan Lu (2021) Learning spatio-temporal transformer for visual tracking. In International Conference on Computer Vision (ICCV), pages 10448\u201310457,","DOI":"10.1109\/ICCV48922.2021.01028"},{"key":"1040_CR15","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan\u00a0N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin (2017). Attention is all you need. In Advances in neural information processing systems, pages 5998\u20136008,"},{"key":"1040_CR16","unstructured":"Salman K, Muzammal N, Munawar H, Syed\u00a0Waqas Z, Fahad\u00a0Shahbaz K, Mubarak S (2021) Transformers in vision: a survey. ACM Computing Surveys (CSUR)"},{"key":"1040_CR17","doi-asserted-by":"crossref","unstructured":"Li B, Wu W, Wang Q, Zhang F, Xing J, Yan J (2020) Siamrpn++: evolution of siamese visual tracking with very deep networks. In International Conference on Computer Vision and Pattern Recogintion (CVPR)","DOI":"10.1109\/CVPR.2019.00441"},{"key":"1040_CR18","doi-asserted-by":"crossref","unstructured":"Krizhevsky A, Sutskever I, Hinton GE (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84\u201390","DOI":"10.1145\/3065386"},{"key":"1040_CR19","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In International Conference on Computer Vision and Pattern Recogintion (CVPR)","DOI":"10.1109\/CVPR.2016.90"},{"key":"1040_CR20","doi-asserted-by":"crossref","unstructured":"Zhang Z, Peng H (2020) Deeper and wider Siamese networks for real-time visual tracking. In International Conference on Computer Vision and Pattern Recogintion (CVPR)","DOI":"10.1109\/CVPR.2019.00472"},{"key":"1040_CR21","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2019.105448","volume":"193","author":"Peng Gao","year":"2020","unstructured":"Gao Peng, Yuan Ruyue, Wang Fei, Xiao Liyi, Fujita Hamido, Zhang Yan (2020) Siamese attentional keypoint network for high performance visual tracking. Knowl Based Syst 193:105448","journal-title":"Knowl Based Syst"},{"key":"1040_CR22","unstructured":"Martin D, Luc\u00a0Van G, Radu T (2020) Probabilistic regression for visual tracking. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 7183\u20137192"},{"key":"1040_CR23","unstructured":"Paul V, Jonathon L, Philip HS T, Bastian L (2020) Siam r-cnn: visual tracking by re-detection. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 6578\u20136588"},{"key":"1040_CR24","doi-asserted-by":"crossref","unstructured":"Yinda X, Zeyu W, Zuoxin L, Ye Y, Gang Yu (2020) Siamfc++: towards robust and accurate visual tracking with target estimation guidelines. In AAAI Conference on Artificial Intelligence (AAAI), pages 12549\u201312556","DOI":"10.1609\/aaai.v34i07.6944"},{"key":"1040_CR25","unstructured":"Dongyan G, Jun W, Ying C, Zhenhua W, Shengyong C (2020) Siamcar: Siamese fully convolutional classification and regression for visual tracking. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 6269\u20136277"},{"key":"1040_CR26","doi-asserted-by":"crossref","unstructured":"Zhipeng Z, Houwen P, Jianlong F, Bing L, Weiming H (2020) Ocean: object-aware anchor-free tracking. In European Conference on Computer Vision (ECCV), pages 771\u2013787. Springer","DOI":"10.1007\/978-3-030-58589-1_46"},{"key":"1040_CR27","unstructured":"Fei D, Peng L, Wei Z, Xianglong T (2020) Correlation-guided attention for corner detection based visual tracking. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 6836\u20136845"},{"key":"1040_CR28","doi-asserted-by":"publisher","first-page":"260","DOI":"10.1016\/j.neucom.2020.11.046","volume":"436","author":"Dawei Zhang","year":"2021","unstructured":"Zhang Dawei, Zheng Zhonglong, Li Minglu, Liu Rixian (2021) Csart: channel and spatial attention-guided residual learning for real-time object tracking. Neurocomputing 436:260\u2013272","journal-title":"Neurocomputing"},{"key":"1040_CR29","doi-asserted-by":"crossref","unstructured":"Nicolas C, Francisco M, Gabriel S, Nicolas U, Alexander K, Sergey Z (2020) End-to-end object detection with transformers. In European Conference on Computer Vision (ECCV), pages 213\u2013229. Springer","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"1040_CR30","unstructured":"Alexey D, Lucas B, Alexander K, Dirk W, Xiaohua Z, Thomas U, Mostafa D, Matthias M, Georg H, Sylvain G, et\u00a0al (2020) An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929"},{"key":"1040_CR31","unstructured":"Ze L, Yutong L, Yue C, Han H, Yixuan W, Zheng Z, Stephen L, Baining G (2021) Swin transformer: Hierarchical vision transformer using shifted windows. In International Conference on Computer Vision (ICCV), pages 10012\u201310022"},{"key":"1040_CR32","unstructured":"Gedas B, Heng W, Lorenzo T (2021) Is space-time attention all you need for video understanding? arXiv preprint arXiv:2102.05095"},{"key":"1040_CR33","doi-asserted-by":"crossref","unstructured":"Ning Wang, Wengang Zhou, Jie Wang, and Houqiang Li (2021) Transformer meets tracker: Exploiting temporal context for robust visual tracking. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 1571\u20131580,","DOI":"10.1109\/CVPR46437.2021.00162"},{"key":"1040_CR34","unstructured":"Moju Z, Kei O, Masayuki I (2021) Trtr: visual tracking with transformer. arXiv preprint arXiv:2105.03817"},{"key":"1040_CR35","doi-asserted-by":"crossref","unstructured":"Kaihua Z, Lei Z, Qingshan L, David Z, Ming-Hsuan Y (2014) Fast visual tracking via dense spatio-temporal context learning. In European Conference on Computer Vision (ECCV), pages 127\u2013141. Springer","DOI":"10.1007\/978-3-319-10602-1_9"},{"key":"1040_CR36","unstructured":"Zhu T, Xing J, Qiang W, Lang C, Yi J (2017) Robust object tracking based on temporal and spatial deep networks. In International Conference on Computer Vision (ICCV)"},{"key":"1040_CR37","unstructured":"Yuan Liu, Ruoteng L, Robby TT, Yu C, Xiubao S (2020) Object tracking using spatio-temporal future prediction. arXiv preprint arXiv:2010.07605"},{"key":"1040_CR38","doi-asserted-by":"crossref","unstructured":"Gao J, Zhang T, Xu C (2019) Graph convolutional tracking. In International Conference on Computer Vision and Pattern Recogintion (CVPR)","DOI":"10.1109\/CVPR.2019.00478"},{"key":"1040_CR39","doi-asserted-by":"publisher","first-page":"2162","DOI":"10.1109\/TMM.2020.3008035","volume":"23","author":"Bo Jiang","year":"2020","unstructured":"Jiang Bo, Zhang Yuan, Luo Bin, Cao Xiaochun, Tang Jin (2020) Stgl: spatial-temporal graph representation and learning for visual tracking. IEEE Trans Multimed 23:2162\u20132171","journal-title":"IEEE Trans Multimed"},{"key":"1040_CR40","unstructured":"Hasan S, Hakan C, Okan K, Bedirhan U (2020) Trat: tracking by attention using spatio-temporal features. arXiv preprint arXiv:2011.09524"},{"key":"1040_CR41","unstructured":"Martin D, Goutam B, Fahad\u00a0Shahbaz K, Michael F (2019) Atom: accurate tracking by overlap maximization. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 4660\u20134669"},{"key":"1040_CR42","unstructured":"Jinpu Z, Yuehuan W (2021) Spatio-temporal matching for Siamese visual tracking. arXiv preprint arXiv:2105.02408"},{"key":"1040_CR43","unstructured":"Hamid R, Nathan T, JunYoung G, Amir S, Ian R, Silvio S (2019) Generalized intersection over union: a metric and a loss for bounding box regression. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 658\u2013666"},{"issue":"6","key":"1040_CR44","doi-asserted-by":"publisher","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","volume":"39","author":"S Ren","year":"2017","unstructured":"Ren S, He K, Girshick R, Sun J (2017) Faster r-cnn: towards real-time object detection with region proposal networks. IEEE Trans Pattern Anal Mach Intell 39(6):1137\u20131149","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"1040_CR45","unstructured":"Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2014) Imagenet large scale visual recognition challenge. International Journal of Computer Vision(IJCV), pages 1\u201342"},{"key":"1040_CR46","unstructured":"Heng F, Liting L, Fan Y, Peng C, Ge D, Sijia Y, Hexin B, Yong X, Chunyuan L, Haibin L (2019) Lasot: a high-quality benchmark for large-scale single object tracking. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 5374\u20135383"},{"issue":"5","key":"1040_CR47","doi-asserted-by":"publisher","first-page":"1562","DOI":"10.1109\/TPAMI.2019.2957464","volume":"43","author":"L Huang","year":"2019","unstructured":"Huang L, Zhao X, Huang K (2019) Got-10k: a large high-diversity benchmark for generic object tracking in the wild. IEEE Trans Pattern Anal Mach Intell 43(5):1562\u201377","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"1040_CR48","unstructured":"Matthias M, Adel B, Silvio G, Salman A, Bernard G (2018) Trackingnet: a large-scale dataset and benchmark for object tracking in the wild. In European Conference on Computer Vision (ECCV), pages 300\u2013317"},{"key":"1040_CR49","doi-asserted-by":"crossref","unstructured":"Tsung-Yi L, Michael M, Serge B, James H, Pietro P, Deva R, Piotr D, Lawrence Zitnick C (2014) Microsoft coco: Common objects in context. In European Conference on Computer Vision (ECCV), pages 740\u2013755. Springer","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"1040_CR50","unstructured":"Ilya L, Frank H (2018) Fixing weight decay regularization in adam. arXiv preprint arXiv:1711.05101"},{"key":"1040_CR51","unstructured":"Yutao C, Cheng J, Limin W, Gangshan W (2020) Fully convolutional online tracking. arXiv preprint arXiv:2004.07109"},{"key":"1040_CR52","doi-asserted-by":"crossref","unstructured":"Zhu Z, Wang Q, Li B, Wu W, Yan J, Hu W (2018) Distractor-aware Siamese networks for visual object tracking. In European Conference on Computer Vision (ECCV)","DOI":"10.1007\/978-3-030-01240-3_7"},{"key":"1040_CR53","unstructured":"Hyeonseob N, Bohyung H (2016) Learning multi-domain convolutional neural networks for visual tracking. In International Conference on Computer Vision and Pattern Recogintion (CVPR)"},{"key":"1040_CR54","doi-asserted-by":"crossref","unstructured":"Yuechen Y, Yilei X, Weilin H, Matthew\u00a0RS (2020) Deformable siamese attention networks for visual object tracking. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 6728\u20136737","DOI":"10.1109\/CVPR42600.2020.00676"},{"key":"1040_CR55","unstructured":"Alan L, Jiri M, Matej K (2020) D3s-a discriminative single shot segmentation tracker. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 7133\u20137142"},{"key":"1040_CR56","unstructured":"Matej K, Ales L, Jiri M, Michael F, Roman P, LukaCehovin Z, Tomas V, Goutam B, Alan L, Abdelrahman E, et\u00a0al (2018) The sixth visual object tracking vot2018 challenge results. In European Conference on Computer Vision (ECCV)"},{"issue":"9","key":"1040_CR57","doi-asserted-by":"publisher","first-page":"1834","DOI":"10.1109\/TPAMI.2014.2388226","volume":"37","author":"Y Wu","year":"2015","unstructured":"Wu Y, Lim J, Yang Ming Hsuan (2015) Object tracking benchmark. IEEE Trans Pattern Anal Mach Intell 37(9):1834\u20131848","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"1040_CR58","doi-asserted-by":"crossref","unstructured":"Lianghua H, Xin Z, Kaiqi H (2020) Globaltrack: a simple and strong baseline for long-term tracking. In AAAI Conference on Artificial Intelligence (AAAI), pages 11037\u201311044","DOI":"10.1609\/aaai.v34i07.6758"},{"key":"1040_CR59","unstructured":"Goutam B, Martin D, Luc\u00a0Van G, Radu T (2019) Learning discriminative model prediction for tracking. In International Conference on Computer Vision (ICCV), pages 6182\u20136191"},{"key":"1040_CR60","unstructured":"Heng F ,Haibin L (2019) Siamese cascaded region proposal networks for real-time visual tracking. In International Conference on Computer Vision and Pattern Recogintion (CVPR), pages 7952\u20137961"}],"container-title":["Complex &amp; Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01040-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40747-023-01040-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40747-023-01040-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,22]],"date-time":"2023-09-22T17:27:52Z","timestamp":1695403672000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40747-023-01040-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,10]]},"references-count":60,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2023,10]]}},"alternative-id":["1040"],"URL":"https:\/\/doi.org\/10.1007\/s40747-023-01040-4","relation":{},"ISSN":["2199-4536","2198-6053"],"issn-type":[{"type":"print","value":"2199-4536"},{"type":"electronic","value":"2198-6053"}],"subject":[],"published":{"date-parts":[[2023,4,10]]},"assertion":[{"value":"29 August 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 March 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 April 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}