{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T11:47:25Z","timestamp":1770464845462,"version":"3.49.0"},"reference-count":30,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,11,10]],"date-time":"2023-11-10T00:00:00Z","timestamp":1699574400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>The emergence of cross-modal perception and deep learning technologies has had a profound impact on modern robotics. This study focuses on the application of these technologies in the field of robot control, specifically in the context of volleyball tasks. The primary objective is to achieve precise control of robots in volleyball tasks by effectively integrating information from different sensors using a cross-modal self-attention mechanism.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>Our approach involves the utilization of a cross-modal self-attention mechanism to integrate information from various sensors, providing robots with a more comprehensive scene perception in volleyball scenarios. To enhance the diversity and practicality of robot training, we employ Generative Adversarial Networks (GANs) to synthesize realistic volleyball scenarios. Furthermore, we leverage transfer learning to incorporate knowledge from other sports datasets, enriching the process of skill acquisition for robots.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>To validate the feasibility of our approach, we conducted experiments where we simulated robot volleyball scenarios using multiple volleyball-related datasets. We measured various quantitative metrics, including accuracy, recall, precision, and F1 score. The experimental results indicate a significant enhancement in the performance of our approach in robot volleyball tasks.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>The outcomes of this study offer valuable insights into the application of multi-modal perception and deep learning in the field of sports robotics. By effectively integrating information from different sensors and incorporating synthetic data through GANs and transfer learning, our approach demonstrates improved robot performance in volleyball tasks. These findings not only advance the field of robotics but also open up new possibilities for human-robot collaboration in sports and athletic performance improvement. This research paves the way for further exploration of advanced technologies in sports robotics, benefiting both the scientific community and athletes seeking performance enhancement through robotic assistance.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fnbot.2023.1288463","type":"journal-article","created":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T10:13:14Z","timestamp":1699870394000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Cross-modal self-attention mechanism for controlling robot volleyball motion"],"prefix":"10.3389","volume":"17","author":[{"given":"Meifang","family":"Wang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhange","family":"Liang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,11,10]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1412.4446","article-title":"Domain-adversarial neural networks","author":"Ajakan","year":"2014","journal-title":"arXiv preprint arXiv:1412.4446"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1900837","DOI":"10.1002\/admt.201900837","article-title":"Soft crawling robots: design, actuation, and locomotion","volume":"5","author":"Chen","year":"2020","journal-title":"Adv. Mater. Technol."},{"key":"B3","doi-asserted-by":"publisher","first-page":"330","DOI":"10.1109\/TG.2021.3065410","article-title":"AI world cup: robot-soccer-based competitions","volume":"13","author":"Hong","year":"2021","journal-title":"IEEE Trans. Games"},{"key":"B4","doi-asserted-by":"publisher","first-page":"4429","DOI":"10.1109\/TSMC.2023.3249123","article-title":"A knee-guided evolutionary computation design for motor performance limitations of a class of robot with strong nonlinear dynamic coupling","volume":"53","author":"Hu","year":"2023","journal-title":"IEEE Trans. Syst. Man Cybernet. Syst"},{"key":"B5","unstructured":"\u201cA hierarchical deep temporal model for group activity recognition,\u201d19711980\n            IbrahimM. S.\n            MuralidharanS.\n            DengZ.\n            VahdatA.\n            MoriG.\n          Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition2016"},{"key":"B6","unstructured":"\u201cHierarchical reinforcement learning for precise soccer shooting skills using a quadrupedal robot,\u201d14791486\n            JiY.\n            LiZ.\n            SunY.\n            PengX. B.\n            LevineS.\n            BersethG.\n          KyotoIEEE2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)2022"},{"key":"B7","doi-asserted-by":"publisher","first-page":"1678","DOI":"10.1007\/s10618-017-0495-0","article-title":"Activity recognition in beach volleyball using a deep convolutional neural network: leveraging the potential of deep learning in sports","volume":"31","author":"Kautz","year":"2017","journal-title":"Data Mining Knowledge Discov."},{"key":"B8","doi-asserted-by":"publisher","first-page":"6116144","DOI":"10.1155\/2023\/6116144","article-title":"Volleyball movement standardization recognition model based on convolutional neural network","volume":"2023","author":"Li","year":"2023","journal-title":"Comput. Intell. Neurosci."},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.34133\/2022\/9797562","article-title":"Implementing monocular visual-tactile sensors for robust manipulation","author":"Li","year":"2022","journal-title":"Cyborg Bionic Syst"},{"key":"B10","unstructured":"\u201cMultisports: a multi-person video dataset of spatio-temporally localized sports actions,\u201d1353613545\n            LiY.\n            ChenL.\n            HeR.\n            WangZ.\n            WuG.\n            WangL.\n          Montreal, QCProceedings of the IEEE\/CVF International Conference on Computer Vision2021"},{"key":"B11","doi-asserted-by":"publisher","first-page":"125908","DOI":"10.1109\/ACCESS.2022.3221530","article-title":"The application of deep convolution neural network in volleyball video behavior recognition","volume":"10","author":"Liang","year":"2022","journal-title":"IEEE Access"},{"key":"B12","doi-asserted-by":"publisher","first-page":"969","DOI":"10.1109\/JSTSP.2020.2994523","article-title":"Gan-generated image detection with self-attention mechanism against gan generator defect","volume":"14","author":"Mi","year":"2020","journal-title":"IEEE J. Select. Top. Signal Process."},{"key":"B13","doi-asserted-by":"publisher","first-page":"48","DOI":"10.1016\/j.neucom.2021.03.091","article-title":"A review on the attention mechanism of deep learning","volume":"452","author":"Niu","year":"2021","journal-title":"Neurocomputing"},{"key":"B14","unstructured":"\u201cComputational intelligence in iot healthcare,\u201d297310\n            OlaniyanO. T.\n            AdetunjiC. O.\n            AdeniyiM. J.\n            HefftD. I.\n          Deep Learning, Machine Learning and IoT in Biomedical and Health Informatics2022"},{"key":"B15","doi-asserted-by":"publisher","first-page":"326","DOI":"10.1016\/j.jmsy.2020.06.018","article-title":"Reinforcement learning for facilitating human-robot-interaction in manufacturing","volume":"56","author":"Oliff","year":"2020","journal-title":"J. Manufact. Syst."},{"key":"B16","doi-asserted-by":"publisher","first-page":"239","DOI":"10.1177\/1747954119899881","article-title":"A systematic review of volleyball spike kinematics: implications for practice and research","volume":"15","author":"Oliveira","year":"2020","journal-title":"Int. J. Sports Sci. Coach."},{"key":"B17","first-page":"50","article-title":"\u201cVolleyball action modelling for behavior analysis and interactive multi-modal feedback,\u201d","volume-title":"Proceedings of the 15th International Summer Workshop on Multimodal Interfaces","author":"Salim","year":"2019"},{"key":"B18","author":"Siedentop","year":"2022","journal-title":"Introduction to Physical Education, Fitness, and Sport"},{"key":"B19","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1007\/978-3-030-50801-2_4","article-title":"\u201cRobotics, automation, and the future of sports,\u201d","author":"Siegel","year":"2020","journal-title":"21st Century Sports: How Technologies Will Change Sports in the Digital Age"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.34133\/2021\/9843894","article-title":"Shape estimation of soft manipulator using stretchable sensor","author":"So","year":"2021","journal-title":"Cyborg Bionic Syst"},{"key":"B21","article-title":"UCF101: a dataset of 101 human actions classes from videos in the wild","author":"Soomro","year":"2012","journal-title":"arXiv preprint arXiv:1212.0402"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2021\/3934443","article-title":"An action recognition method for volleyball players using deep learning","volume":"2021","author":"Tang","year":"2021","journal-title":"Sci. Prog."},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aav1488","article-title":"Soft robot perception using embedded soft sensors and recurrent neural networks","author":"Thuruthel","year":"2019","journal-title":"Sci. Robot."},{"key":"B24","doi-asserted-by":"publisher","first-page":"2515","DOI":"10.1109\/TMM.2021.3083109","article-title":"Cross-modal food retrieval: learning a joint embedding of food images and recipes with semantic consistency and attention mechanism","volume":"24","author":"Wang","year":"2021","journal-title":"IEEE Trans. Multimedia"},{"key":"B25","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1109\/THMS.2021.3092684","article-title":"Cobots in industry 4.0: a roadmap for future practice studies on human\u2013robot collaboration","volume":"51","author":"Weiss","year":"2021","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"B26","doi-asserted-by":"publisher","first-page":"24","DOI":"10.2478\/ijcss-2020-0002","article-title":"Performance of machine learning models in application to beach volleyball data","volume":"19","author":"Wenninger","year":"2020","journal-title":"Int. J. Comput. Sci. Sport"},{"key":"B27","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1109\/ICKG55886.2022.00050","article-title":"\u201cVREN: volleyball rally dataset with expression notation language,\u201d","volume-title":"2022 IEEE International Conference on Knowledge Graph (ICKG)","author":"Xia","year":"2022"},{"key":"B28","unstructured":"\u201cModeling tabular data using conditional gan,\u201d\n            XuL.\n            SkoularidouM.\n            Cuesta-InfanteA.\n            VeeramachaneniK.\n          Advances in Neural Information Processing Systems 322019"},{"key":"B29","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1903.03614","article-title":"Gradient descent based optimization algorithms for deep learning models training","author":"Zhang","year":"2019","journal-title":"arXiv preprint arXiv:1903.03614"},{"key":"B30","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1109\/JPROC.2020.3004555","article-title":"A comprehensive survey on transfer learning","volume":"109","author":"Zhuang","year":"2020","journal-title":"Proc. IEEE"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1288463\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,13]],"date-time":"2023-11-13T10:13:22Z","timestamp":1699870402000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1288463\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,10]]},"references-count":30,"alternative-id":["10.3389\/fnbot.2023.1288463"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1288463","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,10]]},"article-number":"1288463"}}