{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,3]],"date-time":"2026-11-03T17:51:13Z","timestamp":1793728273963,"version":"4.6.1"},"reference-count":46,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,10,30]],"date-time":"2023-10-30T00:00:00Z","timestamp":1698624000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>This paper presents an innovative Intelligent Robot Sports Competition Tactical Analysis Model that leverages multimodal perception to tackle the pressing challenge of analyzing opponent tactics in sports competitions. The current landscape of sports competition analysis necessitates a comprehensive understanding of opponent strategies. However, traditional methods are often constrained to a single data source or modality, limiting their ability to capture the intricate details of opponent tactics.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>Our system integrates the Swin Transformer and CLIP models, harnessing cross-modal transfer learning to enable a holistic observation and analysis of opponent tactics. The Swin Transformer is employed to acquire knowledge about opponent action postures and behavioral patterns in basketball or football games, while the CLIP model enhances the system's comprehension of opponent tactical information by establishing semantic associations between images and text. To address potential imbalances and biases between these models, we introduce a cross-modal transfer learning technique that mitigates modal bias issues, thereby enhancing the model's generalization performance on multimodal data.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Through cross-modal transfer learning, tactical information learned from images by the Swin Transformer is effectively transferred to the CLIP model, providing coaches and athletes with comprehensive tactical insights. Our method is rigorously tested and validated using Sport UV, Sports-1M, HMDB51, and NPU RGB+D datasets. Experimental results demonstrate the system's impressive performance in terms of prediction accuracy, stability, training time, inference time, number of parameters, and computational complexity. Notably, the system outperforms other models, with a remarkable 8.47% lower prediction error (MAE) on the Kinetics dataset, accompanied by a 72.86-second reduction in training time.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>The presented system proves to be highly suitable for real-time sports competition assistance and analysis, offering a novel and effective approach for an Intelligent Robot Sports Competition Tactical Analysis Model that maximizes the potential of multimodal perception technology. By harnessing the synergies between the Swin Transformer and CLIP models, we address the limitations of traditional methods and significantly advance the field of sports competition analysis. This innovative model opens up new avenues for comprehensive tactical analysis in sports, benefiting coaches, athletes, and sports enthusiasts alike.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fnbot.2023.1275645","type":"journal-article","created":{"date-parts":[[2023,10,30]],"date-time":"2023-10-30T09:33:18Z","timestamp":1698658398000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Sports competition tactical analysis model of cross-modal transfer learning intelligent robot based on Swin Transformer and CLIP"],"prefix":"10.3389","volume":"17","author":[{"given":"Li","family":"Jiang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wang","family":"Lu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,10,30]]},"reference":[{"key":"B1","first-page":"6299","article-title":"\u201cQuo vadis, action recognition? A new model and the kinetics dataset,\u201d","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Carreira","year":"2017"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3447744","article-title":"Deep learning for sensor-based human activity recognition: overview, challenges, and opportunities","volume":"54","author":"Chen","year":"2021","journal-title":"ACM Comput. Surveys"},{"key":"B3","first-page":"1910","article-title":"\u201cMM-VIT: Multi-modal video transformer for compressed video action recognition,\u201d","volume-title":"Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision","author":"Chen","year":"2022"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11063-023-11367-1","article-title":"Swin-fusion: swin-transformer with feature fusion for human action recognition","author":"Chen","year":"2023","journal-title":"Neural Process. Lett"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2110.01442","article-title":"A review of generative adversarial networks (GANs) and its applications in a wide variety of disciplines-from medical to remote sensing","author":"Dash","year":"2021","journal-title":"arXiv preprint arXiv:2110.01442"},{"key":"B6","doi-asserted-by":"publisher","first-page":"5966","DOI":"10.1109\/TGRS.2020.3015157","article-title":"Graph convolutional networks for hyperspectral image classification","volume":"59","author":"Hong","year":"2021","journal-title":"IEEE Trans. Geosci. Remote Sens"},{"key":"B7","doi-asserted-by":"publisher","first-page":"2114","DOI":"10.1109\/TCSVT.2019.2912988","article-title":"A survey of human action analysis in HRI applications","volume":"30","author":"Ji","year":"2019","journal-title":"IEEE Trans. Circuits Syst. Video Technol"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3505244","article-title":"Transformers in vision: a survey","volume":"54","author":"Khan","year":"2022","journal-title":"ACM Comput. Surveys"},{"key":"B9","doi-asserted-by":"publisher","first-page":"6086","DOI":"10.1109\/TCSVT.2022.3156634","article-title":"Spatio-temporal player relation modeling for tactic recognition in sports videos","volume":"32","author":"Kong","year":"2022","journal-title":"IEEE Trans. Circuits Syst. Video Technol"},{"key":"B10","first-page":"6232","article-title":"\u201cScSampler: sampling salient clips from video for efficient action recognition,\u201d","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Korbar","year":"2019"},{"key":"B11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TCSVT.2023.3288565","article-title":"Energy-motion features aggregation network for players' fine-grained action analysis in soccer videos","author":"Li","year":"2023","journal-title":"IEEE Trans. Circuits Syst. Video Technol"},{"key":"B12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TGRS.2023.3279834","article-title":"LRR-Net: an interpretable deep unfolding network for hyperspectral anomaly detection","volume":"61","author":"Li","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens"},{"key":"B13","doi-asserted-by":"publisher","first-page":"107037","DOI":"10.1016\/j.patcog.2019.107037","article-title":"Spatio-temporal deformable 3D convnets with attention for action recognition","volume":"98","author":"Li","year":"2020","journal-title":"Pattern Recogn"},{"key":"B14","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1016\/j.ins.2020.01.025","article-title":"Dual-stream generative adversarial networks for distributionally robust zero-shot learning","volume":"519","author":"Liu","year":"2020","journal-title":"Inform. Sci"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1002","DOI":"10.1631\/FITEE.2200055","article-title":"Prospects for multi-agent collaboration and gaming: challenge, technology, and application","volume":"23","author":"Liu","year":"2022","journal-title":"Front. Inform. Technol. Electron. Eng"},{"key":"B16","doi-asserted-by":"publisher","first-page":"109741","DOI":"10.1016\/j.knosys.2022.109741","article-title":"Dual-stream cross-modality fusion transformer for rgb-d action recognition","volume":"255","author":"Liu","year":"2022","journal-title":"Knowl. Based Syst"},{"key":"B17","doi-asserted-by":"publisher","first-page":"10012","DOI":"10.1109\/ICCV48922.2021.00986","article-title":"\u201cSwin transformer: hierarchical vision transformer using shifted windows,\u201d","author":"Liu","year":"2021","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision"},{"key":"B18","doi-asserted-by":"publisher","first-page":"973","DOI":"10.26599\/TST.2021.9010068","article-title":"A survey of human action recognition and posture prediction","volume":"27","author":"Ma","year":"2022","journal-title":"Tsinghua Sci. Technol"},{"key":"B19","first-page":"3461","article-title":"\u201cEfficient tracking of team sport players with few game-specific annotations,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Maglo","year":"2022"},{"key":"B20","doi-asserted-by":"publisher","first-page":"109216","DOI":"10.1016\/j.patcog.2022.109216","article-title":"Hyper-sausage coverage function neuron model and learning algorithm for image classification","volume":"136","author":"Ning","year":"2023","journal-title":"Pattern Recogn"},{"key":"B21","doi-asserted-by":"publisher","first-page":"233","DOI":"10.1016\/j.eswa.2018.03.056","article-title":"Deep learning algorithms for human activity recognition using mobile and wearable sensor networks: state of the art and research challenges","volume":"105","author":"Nweke","year":"2018","journal-title":"Expert Syst. Appl"},{"key":"B22","doi-asserted-by":"publisher","first-page":"605","DOI":"10.1016\/j.jbusres.2022.03.008","article-title":"Artificial intelligence and knowledge sharing: contributing factors to organizational performance","volume":"145","author":"Olan","year":"2022","journal-title":"J. Bus. Res"},{"key":"B23","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1007\/978-981-16-6554-7_27","article-title":"\u201cResearch on assistant application of artificial intelligence robot coach in university sports courses,\u201d","volume-title":"Proceedings of the 11th International Conference on Computer Engineering and Networks","author":"Pan","year":"2022"},{"key":"B24","doi-asserted-by":"publisher","first-page":"2259","DOI":"10.1007\/s10462-020-09904-8","article-title":"A survey on video-based human action recognition: recent updates, datasets, challenges, and applications","volume":"54","author":"Pareek","year":"2021","journal-title":"Artif. Intell. Rev"},{"key":"B25","unstructured":"\u201cZero-shot active visual search (ZAVIS): intelligent object search for robotic assistants,\u201d20042010\n            ParkJ.\n            YoonT.\n            HongJ.\n            YuY.\n            PanM.\n            ChoiS.\n          10.1109\/ICRA48891.2023.10161345London2023 IEEE International Conference on Robotics and Automation (ICRA)2023"},{"key":"B26","first-page":"898","article-title":"\u201cGroup activity detection from trajectory and video data in soccer,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops","author":"Sanford","year":"2020"},{"key":"B27","first-page":"8430","article-title":"\u201cObjects365: a large-scale, high-quality dataset for object detection,\u201d","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Shao","year":"2019"},{"key":"B28","doi-asserted-by":"publisher","first-page":"103874","DOI":"10.1016\/j.robot.2021.103874","article-title":"Multi-modal feature fusion for better understanding of human personality traits in social human-robot interaction","volume":"146","author":"Shen","year":"2021","journal-title":"Robot. Auton. Syst"},{"key":"B29","article-title":"How much can clip benefit vision-and-language tasks?","author":"Shen","year":"2021","journal-title":"arXiv preprint arXiv:2107.06383"},{"key":"B30","doi-asserted-by":"publisher","first-page":"13552","DOI":"10.1109\/JSEN.2020.3005443","article-title":"Comparative study of table tennis forehand strokes classification using deep learning and SVM","volume":"20","author":"Tabrizi","year":"2020","journal-title":"IEEE Sensors J"},{"key":"B31","doi-asserted-by":"publisher","first-page":"10781","DOI":"10.1109\/CVPR42600.2020.01079","article-title":"\u201cEfficientdet: scalable and efficient object detection,\u201d","author":"Tan","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B32","doi-asserted-by":"publisher","first-page":"109165","DOI":"10.1016\/j.sigpro.2023.109165","article-title":"A comparative review on multi-modal sensors fusion based on deep learning","volume":"2023","author":"Tang","year":"2023","journal-title":"Signal Process"},{"key":"B33","doi-asserted-by":"publisher","first-page":"103868","DOI":"10.1016\/j.engappai.2020.103868","article-title":"Multi-modal recognition of worker activity for human-centered intelligent manufacturing","volume":"95","author":"Tao","year":"2020","journal-title":"Eng. Appl. Artif. Intell"},{"key":"B34","first-page":"358","article-title":"\u201cMotionclip: exposing human motion generation to clip space,\u201d","volume-title":"European Conference on Computer Vision","author":"Tevet","year":"2022"},{"key":"B35","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1016\/j.cogsys.2020.07.005","article-title":"A multi-modal approach to cognitive training and assistance in minimally invasive surgery","volume":"64","author":"Vajsbaher","year":"2020","journal-title":"Cogn. Syst. Res"},{"key":"B36","doi-asserted-by":"publisher","first-page":"11951","DOI":"10.1007\/s00521-021-05888-w","article-title":"Enhancing trajectory prediction using sparse outputs: application to team sports","volume":"33","author":"Victor","year":"2021","journal-title":"Neural Comput. Appl"},{"key":"B37","doi-asserted-by":"publisher","first-page":"108498","DOI":"10.1016\/j.patcog.2021.108498","article-title":"Uncertainty estimation for stereo matching based on evidential deep learning","volume":"124","author":"Wang","year":"2022","journal-title":"Pattern Recogn"},{"key":"B38","doi-asserted-by":"publisher","first-page":"3048","DOI":"10.1109\/TPAMI.2021.3055564","article-title":"Knowledge distillation and student-teacher learning for visual intelligence: a review and new outlooks","volume":"44","author":"Wang","year":"2021","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B39","doi-asserted-by":"publisher","first-page":"119668","DOI":"10.1109\/ACCESS.2019.2936709","article-title":"Improving human pose estimation with self-attention generative adversarial networks","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"B40","doi-asserted-by":"publisher","first-page":"24","DOI":"10.2478\/ijcss-2020-0002","article-title":"Performance of machine learning models in application to beach volleyball data","volume":"19","author":"Wenninger","year":"2020","journal-title":"Int. J. Comput. Sci. Sport"},{"key":"B41","doi-asserted-by":"publisher","first-page":"2126","DOI":"10.1109\/TPAMI.2020.3026709","article-title":"Privacy-preserving deep action recognition: an adversarial learning framework and a new dataset","volume":"44","author":"Wu","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell"},{"key":"B42","first-page":"532","article-title":"\u201cThe Korean sign language dataset for action recognition,\u201d","volume-title":"International Conference on Multimedia Modeling","author":"Yang","year":"2019"},{"key":"B43","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TGRS.2023.3284671","article-title":"Extended vision transformer (exvit) for land use and land cover classification: a multimodal deep learning framework","volume":"61","author":"Yao","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens"},{"key":"B44","first-page":"558","article-title":"\u201cTokens-to-token VIT: training vision transformers from scratch on imageNet,\u201d","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Yuan","year":"2021"},{"key":"B45","unstructured":"\u201cGraph transformer networks,\u201d\n            YunS.\n            JeongM.\n            KimR.\n            KangJ.\n            KimH. J.\n          Advances in Neural Information Processing Systems, Vol. 322019"},{"key":"B46","doi-asserted-by":"publisher","first-page":"798","DOI":"10.1109\/TNNLS.2020.3029181","article-title":"Deep multimodal transfer learning for cross-modal retrieval","volume":"33","author":"Zhen","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1275645\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,30]],"date-time":"2023-10-30T09:33:30Z","timestamp":1698658410000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1275645\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,30]]},"references-count":46,"alternative-id":["10.3389\/fnbot.2023.1275645"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1275645","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,30]]},"article-number":"1275645"}}