{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,8]],"date-time":"2026-09-08T15:04:27Z","timestamp":1788879867925,"version":"build-2803163510"},"reference-count":39,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,10,13]],"date-time":"2023-10-13T00:00:00Z","timestamp":1697155200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>In the realm of basketball, refining shooting skills and decision-making levels using intelligent agents has garnered significant interest. This study addresses the challenge by introducing an innovative framework that combines multi-modal perception and deep reinforcement learning. The goal is to create basketball robots capable of executing precise shots and informed choices by effectively integrating sensory inputs and learned strategies.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>The proposed approach consists of three main components: multi-modal perception, deep reinforcement learning, and end-to-end architecture. Multi-modal perception leverages the multi-head attention mechanism (MATT) to merge visual, motion, and distance cues for a holistic perception of the basketball scenario. The deep reinforcement learning framework utilizes the Deep Q-Network (DQN) algorithm, enabling the robots to learn optimal shooting strategies over iterative interactions with the environment. The end-to-end architecture connects these components, allowing seamless integration of perception and decision-making processes.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The experiments conducted demonstrate the effectiveness of the proposed approach. Basketball robots equipped with multi-modal perception and deep reinforcement learning exhibit improved shooting accuracy and enhanced decision-making abilities. The multi-head attention mechanism enhances the robots' perception of complex scenes, leading to more accurate shooting decisions. The application of the DQN algorithm results in gradual skill improvement and strategic optimization through interaction with the environment.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>The integration of multi-modal perception and deep reinforcement learning within an end-to-end architecture presents a promising avenue for advancing basketball robot training and performance. The ability to fuse diverse sensory inputs and learned strategies empowers robots to make informed decisions and execute accurate shots. The research not only contributes to the field of robotics but also has potential implications for human basketball training and coaching methodologies.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fnbot.2023.1274543","type":"journal-article","created":{"date-parts":[[2023,10,13]],"date-time":"2023-10-13T13:22:42Z","timestamp":1697203362000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Research on deep reinforcement learning basketball robot shooting skills improvement based on end to end architecture and multi-modal perception"],"prefix":"10.3389","volume":"17","author":[{"given":"Jun","family":"Zhang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dayong","family":"Tao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,10,13]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1292","DOI":"10.3390\/s21041292","article-title":"Reinforcement learning approaches in social robotics","volume":"21","author":"Akalin","year":"2021","journal-title":"Sensors"},{"key":"B2","article-title":"A doubly self-exciting poisson model for describing scoring levels in nba basketball","author":"Briz-Red\u00f3n","year":"2023","journal-title":"arXiv preprint arXiv"},{"key":"B3","doi-asserted-by":"publisher","first-page":"252","DOI":"10.1145\/3511808.3557429","article-title":"\u201cRelax: Reinforcement learning agent explainer for arbitrary predictive models,\u201d","author":"Chen","year":"2022","journal-title":"Proceedings of the 31st ACM International Conference on Information &Knowledge Management"},{"key":"B4","first-page":"486","article-title":"\u201cA theoretical analysis of deep q-learning,\u201d","volume-title":"Learning for Dynamics and Control","author":"Fan","year":"2020"},{"key":"B5","doi-asserted-by":"publisher","first-page":"e9","DOI":"10.4108\/eetsis.v9i6.2591","article-title":"Multi-target trajectory tracking in multi-frame video images of basketball sports based on deep learning","volume":"10","author":"Gong","year":"2023","journal-title":"EAI Endorsed Trans. Scal. Inf. Syst"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989385","author":"Gu","year":"2016","journal-title":"Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates"},{"key":"B7","article-title":"\u201cRobust multi-agent reinforcement learning with state uncertainty,\u201d","author":"He","year":"2023","journal-title":"Transactions on Machine Learning Research"},{"key":"B8","doi-asserted-by":"publisher","first-page":"4340","DOI":"10.1109\/TGRS.2020.3016820","article-title":"More diverse means better: Multimodal deep learning meets remote-sensing imagery classification","volume":"59","author":"Hong","year":"2021","journal-title":"IEEE Trans. Geosci. Rem. Sens"},{"key":"B9","doi-asserted-by":"publisher","first-page":"12","DOI":"10.1016\/j.isprsjprs.2020.06.014","article-title":"X-ModalNet: A semi-supervised deep cross-modal network for classification of remote sensing data","volume":"167","author":"Hong","year":"2020","journal-title":"ISPRS J. Photogram. Rem. Sens"},{"key":"B10","doi-asserted-by":"publisher","first-page":"413","DOI":"10.1007\/s12193-020-00352-w","article-title":"An audiovisual interface-based drumming system for multimodal human-robot interaction","volume":"15","author":"Ince","year":"2021","journal-title":"J. Multimodal User Interf"},{"key":"B11","doi-asserted-by":"publisher","first-page":"14","DOI":"10.14569\/IJACSA.2023.0140304","article-title":"\u201cDeep learning algorithm based wearable device for basketball stance recognition in basketball,\u201d","author":"Jiang","year":"2023","journal-title":"International Journal of Advanced Computer Science and Applications"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.21203\/rs.3.rs-2947413\/v1","author":"Khobdeh","year":"2023","journal-title":"Basketball action recognition based on the combination of yolo and a deep fuzzy lstm network"},{"key":"B13","unstructured":"Coderl: Mastering code generation through pretrained models and deep reinforcement learning2131421328\n            LeH.\n            WangY.\n            GotmareA. D.\n            SavareseS.\n            HoiS. C. H.\n          Adv. Neural Inf. Process. Syst352022"},{"key":"B14","doi-asserted-by":"publisher","first-page":"54","DOI":"10.12775\/JEHS.2021.11.07.005","article-title":"Application of artificial intelligence in basketball sport","volume":"11","author":"Li","year":"2021","journal-title":"J. Educ. Health Sport"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2021\/1430512","article-title":"Artificial intelligence and neural network-based shooting accuracy prediction analysis in basketball","volume":"2021","author":"Li","year":"2021","journal-title":"Mobile Inf. Syst"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1109\/TCSVT.2023.3288565","article-title":"\u201cEnergy-motion features aggregation network for players fine-grained action analysis in soccer videos","author":"Li","year":"2023"},{"key":"B17","doi-asserted-by":"publisher","first-page":"2225906","DOI":"10.1080\/08839514.2023.2225906","article-title":"Improved efficientdet algorithm for basketball players upper limb movement trajectory recognition","volume":"37","author":"Liang","year":"2023","journal-title":"Appl. Artif. Intell"},{"key":"B18","doi-asserted-by":"publisher","first-page":"5517","DOI":"10.1109\/CVPR52729.2023.00534","article-title":"\u201cLeapfrog diffusion model for stochastic trajectory prediction,\u201d","author":"Mao","year":"2023","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B19","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1007\/978-3-319-56991-8_32","article-title":"\u201cDeep reinforcement learning: an overview,\u201d","volume-title":"Proceedings of SAI Intelligent Systems Conference (IntelliSys) 2016","author":"Mousavi","year":"2018"},{"key":"B20","unstructured":"Reinforcement learning enhanced explainer for graph neural networks2252322533\n            ShanC.\n            ShenY.\n            ZhangY.\n            LiX.\n            LiD.\n          Adv. Neur. Inf. Process. Syst342021"},{"key":"B21","doi-asserted-by":"publisher","first-page":"85","DOI":"10.3390\/asi4040085","article-title":"An improved model for analyzing textual sentiment based on a deep neural network using multi-head attention mechanism","volume":"4","author":"Sharaf Al-deen","year":"2021","journal-title":"Appl. Syst. Innov"},{"key":"B22","doi-asserted-by":"publisher","first-page":"2393","DOI":"10.1109\/TII.2019.2936167","article-title":"End-to-end navigation strategy with deep reinforcement learning for mobile robots","volume":"16","author":"Shi","year":"2019","journal-title":"IEEE Trans. Ind. Inf"},{"key":"B23","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1007\/978-3-030-50801-2_4","article-title":"\u201cRobotics, automation, and the future of sports,\u201d","author":"Siegel","year":"2020","journal-title":"21st Century Sports: How Technologies Will Change Sports in the Digital Age"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2019.XV.073","author":"Singh","year":"2019","journal-title":"End-to-end robotic reinforcement learning without reward engineering"},{"key":"B25","doi-asserted-by":"publisher","first-page":"673","DOI":"10.1145\/3366423.3380149","article-title":"\u201cAdversarial attacks on graph neural networks via node injections: A hierarchical reinforcement learning approach,\u201d","author":"Sun","year":"2020","journal-title":"Proceedings of the Web Conference 2020"},{"key":"B26","doi-asserted-by":"publisher","first-page":"4418","DOI":"10.24963\/ijcai.2018\/614","article-title":"\u201cGet the point of my utterance! learning towards effective responses with multi-head attention mechanism,\u201d","author":"Tao","year":"2018","journal-title":"IJCAI"},{"key":"B27","doi-asserted-by":"crossref","DOI":"10.1088\/1742-6596\/2083\/4\/042003","article-title":"\u201cStudy and application of computer information big data in basketball vision system using high-definition camera motion data capture,\u201d","volume-title":"Journal of Physics: Conference Series","author":"Wei","year":"2021"},{"key":"B28","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1016\/B978-0-323-99992-2.00032-3","article-title":"\u201cBasketball,\u201d","volume-title":"The Youth Athlete","author":"William","year":"2023"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TGRS.2022.3228927","article-title":"Convolutional neural networks for multimodal remote sensing data classification","volume":"60","author":"Wu","year":"2022","journal-title":"IEEE Trans. Geosci. Rem. Sens"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TGRS.2023.3284671","article-title":"Extended vision transformer (exvit) for land use and land cover classification: A multimodal deep learning framework","volume":"61","author":"Yao","year":"2023","journal-title":"IEEE Trans. Geosci. Rem. Sens"},{"key":"B31","doi-asserted-by":"publisher","first-page":"9141","DOI":"10.18653\/v1\/2023.acl-long.508","article-title":"\u201cGrounded multimodal named entity recognition on social media,\u201d","author":"Yu","year":"2023","journal-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"},{"key":"B32","doi-asserted-by":"publisher","first-page":"6347","DOI":"10.1109\/JLT.2022.3195949","article-title":"Multi-span long-haul fiber transmission model based on cascaded neural networks with multi-head attention mechanism","volume":"40","author":"Zang","year":"2022","journal-title":"J. Lightw. Technol"},{"key":"B33","first-page":"1","article-title":"\u201cOverfitting and underfitting analysis for deep learning based end-to-end communication systems,\u201d","volume-title":"2019 11th international conference on wireless communications and signal processing (WCSP)","author":"Zhang","year":"2019"},{"key":"B34","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11782","article-title":"\u201cAn end-to-end deep learning architecture for graph classification,\u201d","volume":"32","author":"Zhang","year":"2018","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B35","doi-asserted-by":"publisher","first-page":"4689","DOI":"10.1609\/aaai.v36i4.20394","article-title":"\u201cAlphaholdem: High-performance artificial intelligence for heads-up no-limit poker via end-to-end reinforcement learning,\u201d","author":"Zhao","year":"2022","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B36","doi-asserted-by":"publisher","first-page":"1729881420909965","DOI":"10.1177\/1729881420909965","article-title":"Retracted: Design of basketball robot based on behavior-based fuzzy control","volume":"17","author":"Zhi","year":"2020","journal-title":"Int. J. Adv. Rob. Syst"},{"key":"B37","doi-asserted-by":"publisher","first-page":"377","DOI":"10.1504\/IJICT.2023.131223","article-title":"Emotion recognition algorithm of basketball players based on deep learning","volume":"22","author":"Zhou","year":"2023","journal-title":"Int. J. Inf. Commun. Technol"},{"key":"B38","doi-asserted-by":"crossref","first-page":"1317","DOI":"10.1109\/IPEC54454.2022.9777376","article-title":"\u201cPositioning of basketball robot target recognition system based on data mining algorithm,\u201d","volume-title":"2022 IEEE Asia-Pacific Conference on Image Processing, Electronics and Computers (IPEC)","author":"Zhu","year":"2022"},{"key":"B39","doi-asserted-by":"publisher","first-page":"43305","DOI":"10.1109\/ACCESS.2023.3269287","article-title":"Multi-agent deep-learning based comparative analysis of team sport trajectories","volume":"11","author":"Ziyi","year":"2023","journal-title":"IEEE Access"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1274543\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,10,13]],"date-time":"2023-10-13T13:23:01Z","timestamp":1697203381000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1274543\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,13]]},"references-count":39,"alternative-id":["10.3389\/fnbot.2023.1274543"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1274543","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,13]]},"article-number":"1274543"}}