{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,22]],"date-time":"2026-06-22T14:37:53Z","timestamp":1782139073056,"version":"3.54.5"},"reference-count":41,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T00:00:00Z","timestamp":1759881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>To enhance the obstacle avoidance performance and autonomous decision-making capabilities of robots in complex dynamic environments, this paper proposes an end-to-end intelligent obstacle avoidance method that integrates deep reinforcement learning, spatiotemporal attention mechanisms, and a Transformer-based architecture. Current mainstream robot obstacle avoidance methods often rely on system architectures with separated perception and decision-making modules, which suffer from issues such as fragmented feature transmission, insufficient environmental modeling, and weak policy generalization. To address these problems, this paper adopts Deep Q-Network (DQN) as the core of reinforcement learning, guiding the robot to autonomously learn optimal obstacle avoidance strategies through interaction with the environment, effectively handling continuous decision-making problems in dynamic and uncertain scenarios. To overcome the limitations of traditional perception mechanisms in modeling the temporal evolution of obstacles, a spatiotemporal attention mechanism is introduced, jointly modeling spatial positional relationships and historical motion trajectories to enhance the model's perception of critical obstacle areas and potential collision risks. Furthermore, an end-to-end Transformer-based perception-decision architecture is designed, utilizing multi-head self-attention to perform high-dimensional feature modeling on multi-modal input information (such as LiDAR and depth images), and generating action policies through a decoding module. This completely eliminates the need for manual feature engineering and intermediate state modeling, constructing an integrated learning process of perception and decision-making. Experiments conducted in several typical obstacle avoidance simulation environments demonstrate that the proposed method outperforms existing mainstream deep reinforcement learning approaches in terms of obstacle avoidance success rate, path optimization, and policy convergence speed. It exhibits good stability and generalization capabilities, showing broad application prospects for deployment in real-world complex environments.<\/jats:p>","DOI":"10.3389\/fnbot.2025.1646336","type":"journal-article","created":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T05:40:44Z","timestamp":1759902044000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["End-to-end robot intelligent obstacle avoidance method based on deep reinforcement learning with spatiotemporal transformer architecture"],"prefix":"10.3389","volume":"19","author":[{"given":"Yuwen","family":"Zhou","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weizhong","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2025,10,8]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"8174","DOI":"10.3390\/app13148174","article-title":"Dynamic obstacle avoidance and path planning through reinforcement learning","volume":"13","author":"Almazrouei","year":"2023","journal-title":"Appl. Sci"},{"key":"B2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.38094\/sgej1027","article-title":"The simultaneous localization and mapping (SLAM): an overview","volume":"1","author":"Alsadik","year":"2021","journal-title":"Surv. Geospat. Eng. J"},{"key":"B3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/ICECCT.2019.8869489","article-title":"\u201cA review on sift and surf for underwater image feature detection and matching,\u201d","volume-title":"2019 IEEE International Conference on Electrical, Computer and Communication Technologies (ICECCT)","author":"Ansari","year":"2019"},{"key":"B4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/IST59124.2023.10355706","article-title":"\u201cOv 2 slam on euroc mav datasets: a study of corner detector performance,\u201d","volume-title":"2023 IEEE International Conference on Imaging Systems and Techniques (IST)","author":"Burghoffer","year":"2023"},{"key":"B5","doi-asserted-by":"publisher","first-page":"4586","DOI":"10.1109\/TII.2025.3545050","article-title":"Fast finite-time bipartite formation with obstacle avoidance for time-delay multiagent systems: application in mobile robot swarm","volume":"21","author":"Chang","year":"2025","journal-title":"IEEE Trans. Ind. Inform"},{"key":"B6","first-page":"15084","article-title":"Decision transformer: reinforcement learning via sequence modeling","volume":"34","author":"Chen","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B7","doi-asserted-by":"publisher","first-page":"23660","DOI":"10.1609\/aaai.v39i22.34536","article-title":"Attributive reasoning for hallucination diagnosis of large language models","volume":"39","author":"Chen","year":"2025","journal-title":"Proc. AAAI Conf. Artif. Intell"},{"key":"B8","first-page":"98","article-title":"\u201cCab-kws: contrastive augmentation: an unsupervised learning approach for keyword spotting in speech technology,\u201d","volume-title":"International Conference on Pattern Recognition","author":"Dai","year":"2024"},{"key":"B9","first-page":"3164","article-title":"\u201cRobothor: an open simulation-to-real embodied ai platform,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Deitke","year":"2020"},{"key":"B10","doi-asserted-by":"publisher","first-page":"104907","DOI":"10.1016\/j.robot.2024.104907","article-title":"An algorithm for dynamic obstacle avoidance applied to uavs","volume":"186","author":"Enriquez","year":"2025","journal-title":"Rob. Auton. Syst"},{"key":"B11","first-page":"1006","article-title":"\u201cSemantic concept testing in autonomous driving by extraction of object-level annotations from carla,\u201d","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Gannamaneni","year":"2021"},{"key":"B12","doi-asserted-by":"publisher","first-page":"34","DOI":"10.1109\/TRO.2006.889486","article-title":"Improved techniques for grid mapping with rao-blackwellized particle filters","volume":"23","author":"Grisetti","year":"2007","journal-title":"IEEE Trans. Robot"},{"key":"B13","doi-asserted-by":"publisher","first-page":"5850","DOI":"10.1109\/LRA.2022.3161710","article-title":"Obstacle avoidance with dynamic avoidance risk region for mobile robots in dynamic environments","volume":"7","author":"Guo","year":"2022","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B14","doi-asserted-by":"publisher","first-page":"5871","DOI":"10.1109\/LRA.2025.3560842","article-title":"Seeing through pixel motion: learning obstacle avoidance from optical flow with one camera","volume":"10","author":"Hu","year":"2025","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1468","DOI":"10.3390\/s21041468","article-title":"Double deep q-learning and faster r-cnn-based autonomous vehicle navigation and obstacle avoidance in dynamic environment","volume":"21","author":"Issa","year":"2021","journal-title":"Sensors"},{"key":"B16","first-page":"1273","article-title":"Offline reinforcement learning as one big sequence modeling problem","volume":"34","author":"Janner","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B17","doi-asserted-by":"publisher","first-page":"126780","DOI":"10.1016\/j.eswa.2025.126780","article-title":"Performing task automation for surgical robot: a spatial-temporal varying primal-dual neural network with guided obstacle avoidance and null space optimization","volume":"273","author":"Jian","year":"2025","journal-title":"Expert Syst. Appl"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1007\/s41870-025-02500-5.","article-title":"Autonomous navigation of ROS2 based turtlebot3 in static and dynamic environments using intelligent approach","author":"Kashyap","year":"2025","journal-title":"Int. J. Inf. Technol"},{"key":"B19","doi-asserted-by":"publisher","first-page":"3573","DOI":"10.3390\/s24113573","article-title":"Obstacle avoidance and path planning methods for autonomous navigation of mobile robot","volume":"24","author":"Katona","year":"2024","journal-title":"Sensors"},{"key":"B20","first-page":"21675","article-title":"A combinatorial algorithm for approximating the optimal transport in the parallel and MPC settings","volume":"36","author":"Lahn","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B21","doi-asserted-by":"publisher","first-page":"121120","DOI":"10.1016\/j.oceaneng.2025.121120","article-title":"A moving obstacle avoidance strategy-based collaborative formation for the bionic multi-underwater spherical robot control system","volume":"329","author":"Li","year":"","journal-title":"Ocean Eng"},{"key":"B22","doi-asserted-by":"publisher","first-page":"9293","DOI":"10.1109\/TIE.2025.3536628","article-title":"Model predictive optimization and terminal sliding mode motion control for mobile robot with obstacle avoidance","volume":"72","author":"Li","year":"","journal-title":"IEEE Trans. Ind. Electron"},{"key":"B23","doi-asserted-by":"publisher","first-page":"2686","DOI":"10.1109\/LRA.2025.3534519","article-title":"Flexible active safety motion control for robotic obstacle avoidance: a CBF-guided mpc approach","volume":"10","author":"Liu","year":"2025","journal-title":"IEEE Robot. Autom. Lett"},{"key":"B24","doi-asserted-by":"publisher","first-page":"3660","DOI":"10.3390\/electronics11223660","article-title":"Research on path-planning algorithm integrating optimization a-star algorithm and artificial potential field method","volume":"11","author":"Liu","year":"2022","journal-title":"Electronics"},{"key":"B25","doi-asserted-by":"publisher","first-page":"107230","DOI":"10.1016\/j.cie.2021.107230","article-title":"Path planning optimization of indoor mobile robot based on adaptive ant colony algorithm","volume":"156","author":"Miao","year":"2021","journal-title":"Comput. Ind. Eng"},{"key":"B26","doi-asserted-by":"publisher","first-page":"101538","DOI":"10.1016\/j.iot.2025.101538","article-title":"Visual-based obstacle avoidance method using advanced cnn for mobile robots","volume":"31","author":"Misir","year":"2025","journal-title":"Internet Things"},{"key":"B27","doi-asserted-by":"publisher","first-page":"107608","DOI":"10.1016\/j.enggeo.2024.107608","article-title":"Grey relation analysis and multiple criteria decision analysis method model for suitability evaluation of underground space development","volume":"338","author":"Ni","year":"2024","journal-title":"Eng. Geol"},{"key":"B28","doi-asserted-by":"publisher","first-page":"107589","DOI":"10.1016\/j.margeo.2025.107589","article-title":"Global marine methane seepage: spatiotemporal patterns and ocean current control","volume":"487","author":"Ni","year":"2025","journal-title":"Mar. Geol"},{"key":"B29","article-title":"\u201cComputing all optimal partial transports,\u201d","volume-title":"International Conference on Learning Representations","author":"Phatak","year":"2023"},{"key":"B30","doi-asserted-by":"publisher","first-page":"211","DOI":"10.3390\/drones7030211","article-title":"Review of autonomous path planning algorithms for mobile robots","volume":"7","author":"Qin","year":"2023","journal-title":"Drones"},{"key":"B31","doi-asserted-by":"publisher","author":"Raghvendra","year":"2024","DOI":"10.48550\/arXiv.2405.03664"},{"key":"B32","doi-asserted-by":"publisher","first-page":"10031","DOI":"10.1109\/ACCESS.2022.3142859","article-title":"Particle swarm optimization: a comprehensive survey","volume":"10","author":"Shami","year":"2022","journal-title":"IEEE Access"},{"key":"B33","doi-asserted-by":"publisher","first-page":"100651","DOI":"10.1016\/j.cosrev.2024.100651","article-title":"Mobile robot localization: current challenges and future prospective","volume":"53","author":"Ullah","year":"2024","journal-title":"Comput. Sci. Rev"},{"key":"B34","doi-asserted-by":"publisher","first-page":"104272","DOI":"10.1016\/j.compind.2025.104272","article-title":"Dynamic obstacle avoidance control based on a novel dynamic window approach for agricultural robots","volume":"167","author":"Wang","year":"","journal-title":"Comput. Ind"},{"key":"B35","doi-asserted-by":"publisher","first-page":"00368504211037771","DOI":"10.1177\/00368504211037771","article-title":"Review of wheeled mobile robot collision avoidance under unknown environment","volume":"104","author":"Wang","year":"2021","journal-title":"Sci. Prog"},{"key":"B36","doi-asserted-by":"publisher","first-page":"8219","DOI":"10.1109\/TIE.2024.3522489","article-title":"Simultaneous planning and tracking framework for obstacle avoidance of autonomous mobile robots in dynamic scenarios","volume":"72","author":"Wang","year":"","journal-title":"IEEE Trans. Ind. Electron"},{"key":"B37","doi-asserted-by":"publisher","first-page":"9877","DOI":"10.3390\/app13179877","article-title":"Challenges and solutions for autonomous ground robot scene understanding and navigation in unstructured outdoor environments: a review","volume":"13","author":"Wijayathunga","year":"2023","journal-title":"Appl. Sci"},{"key":"B38","doi-asserted-by":"publisher","first-page":"102894","DOI":"10.1016\/j.rcim.2024.102894","article-title":"A real-time collision avoidance method for redundant dual-arm robots in an open operational environment","volume":"92","author":"Wu","year":"2025","journal-title":"Robot. Comput. Integr. Manuf"},{"key":"B39","doi-asserted-by":"publisher","first-page":"75742","DOI":"10.1109\/ACCESS.2022.3192026","article-title":"Fine segmentation on faces with masks based on a multistep iterative segmentation algorithm","volume":"10","author":"Zhang","year":"2022","journal-title":"IEEE Access"},{"key":"B40","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1145\/3570773.3570871","article-title":"\u201cOverview of filtering algorithms for autonomous mobile robots,\u201d","volume-title":"Proceedings of the 3rd International Symposium on Artificial Intelligence for Medicine Sciences","author":"Zhang","year":"2022"},{"key":"B41","first-page":"3961","article-title":"\u201cTowards open-set object detection and discovery,\u201d","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Zheng","year":"2022"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2025.1646336\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T05:40:46Z","timestamp":1759902046000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2025.1646336\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,8]]},"references-count":41,"alternative-id":["10.3389\/fnbot.2025.1646336"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2025.1646336","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,8]]},"article-number":"1646336"}}