{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T18:58:16Z","timestamp":1790708296084,"version":"4.1.0"},"reference-count":36,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,8,22]],"date-time":"2023-08-22T00:00:00Z","timestamp":1692662400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>Mobile robots are playing an increasingly significant role in social life and industrial production, such as searching and rescuing robots, autonomous exploration of sweeping robots, and so on. Improving the accuracy of autonomous navigation of mobile robots is a hot issue to be solved. However, traditional navigation methods are unable to realize crash-free navigation in an environment with dynamic obstacles, more and more scholars are gradually using autonomous navigation based on deep reinforcement learning (DRL) to replace overly conservative traditional methods. But on the other hand, DRL's training time is too long, and the lack of long-term memory easily leads the robot to a dead end, which makes its application in the actual scene more difficult. To shorten training time and prevent mobile robots from getting stuck and spinning around, we design a new robot autonomous navigation framework which combines the traditional global planning and the local planning based on DRL. Therefore, the entire navigation process can be transformed into first using traditional navigation algorithms to find the global path, then searching for several high-value landmarks on the global path, and then using the DRL algorithm to move the mobile robot toward the designated landmarks to complete the final navigation, which makes the robot training difficulty greatly reduced. Furthermore, in order to improve the lack of long-term memory in deep reinforcement learning, we design a feature extraction network containing memory modules to preserve the long-term dependence of input features. Through comparing our methods with traditional navigation methods and reinforcement learning based on end-to-end depth navigation methods, it shows that while the number of dynamic obstacles is large and obstacles are rapidly moving, our proposed method is, on average, 20% better than the second ranked method in navigation efficiency (navigation time and navigation paths' length), 34% better than the second ranked method in safety (collision times), 26.6% higher than the second ranked method in success rate, and shows strong robustness.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1200214","type":"journal-article","created":{"date-parts":[[2023,8,22]],"date-time":"2023-08-22T07:54:49Z","timestamp":1692690889000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":17,"title":["Deep reinforcement learning-aided autonomous navigation with landmark generators"],"prefix":"10.3389","volume":"17","author":[{"given":"Xuanzhi","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yankang","family":"Sun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuyang","family":"Xie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiang","family":"Bin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian","family":"Xiao","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,8,22]]},"reference":[{"key":"B1","doi-asserted-by":"crossref","DOI":"10.1109\/ICARA55094.2022.9738559","article-title":"\u201cA PRM approach to path planning with obstacle avoidance of an autonomous robot,\u201d","volume-title":"2022 8th International Conference on Automation, Robotics and Applications (ICARA)","author":"Alarabi","year":"2022"},{"key":"B2","doi-asserted-by":"crossref","DOI":"10.1109\/IVS.2017.7995717","article-title":"\u201cModified artificial potential field method for online path planning applications,\u201d","volume-title":"2017 IEEE Intelligent Vehicles Symposium (IV)","author":"Bounini","year":"2017"},{"key":"B3","doi-asserted-by":"crossref","DOI":"10.1109\/IROS.2017.8202312","article-title":"\u201cSocially aware motion planning with deep reinforcement learning,\u201d","volume-title":"2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Chen","year":""},{"key":"B4","first-page":"285","article-title":"\u201cDecentralized non-communicating multiagent collision avoidance with deep reinforcement learning,\u201d","volume-title":"2017 IEEE international conference on robotics and automation (ICRA)","author":"Chen","year":""},{"key":"B5","doi-asserted-by":"publisher","first-page":"7829","DOI":"10.1109\/ICRA48506.2021.9560951","article-title":"\u201cNavrep: Unsupervised representations for reinforcement learning of robot navigation in dynamic human environments,\u201d","author":"Dugas","year":"2021"},{"key":"B6","doi-asserted-by":"crossref","first-page":"3052","DOI":"10.1109\/IROS.2018.8593871","article-title":"\u201cMotion planning among dynamic, decision-making agents with deep reinforcement learning,\u201d","volume-title":"2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Everett","year":"2018"},{"key":"B7","doi-asserted-by":"publisher","first-page":"3133","DOI":"10.1109\/COMST.2019.2916583","article-title":"Applications of deep reinforcement learning in communications and networking: A survey","volume":"21","author":"Farhad","year":"2019","journal-title":"IEEE Commun. Surv. Tutorials"},{"key":"B8","doi-asserted-by":"crossref","first-page":"5113","DOI":"10.1109\/ICRA.2018.8461096","article-title":"\u201cPrm-rl: Long-range robotic navigation tasks by combining reinforcement learning and sampling-based planning,\u201d","volume-title":"2018 IEEE International Conference on Robotics and Automation (ICRA)","author":"Faust","year":"2018"},{"key":"B9","doi-asserted-by":"publisher","first-page":"547","DOI":"10.31209\/2019.100000110","article-title":"Applying probabilistic model checking to path planning in an intelligent transportation system using mobility trajectories and their statistical data","volume":"25","author":"Gao","year":"2019","journal-title":"Intell. Automat. Soft. Comput."},{"key":"B10","first-page":"315","article-title":"\u201cDeep sparse rectifier neural networks,\u201d","author":"Glorot","year":"2011"},{"key":"B11","doi-asserted-by":"publisher","first-page":"2259009","DOI":"10.1142\/S0218001422590091","article-title":"An improved a-star algorithm for complete coverage path planning of unmanned ships","volume":"36","author":"Guo","year":"2022","journal-title":"Int. J. Patt. Recogn. Artif. Intell."},{"key":"B12","doi-asserted-by":"publisher","first-page":"4520","DOI":"10.1109\/LRA.2021.3067851","article-title":"VR-ORCA: Variable responsibility optimal reciprocal collision avoidance","volume":"6","author":"Guo","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"B13","doi-asserted-by":"crossref","DOI":"10.1109\/IROS.2017.8202244","article-title":"\u201cDeep reinforcement learning for high precision assembly tasks,\u201d","volume-title":"2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Inoue","year":"2017"},{"key":"B14","doi-asserted-by":"publisher","first-page":"488","DOI":"10.3390\/math10030488","article-title":"A new hybrid based on long short-term memory network with spotted hyena optimization algorithm for multi-label text classification","volume":"10","author":"Khataei Maragheh","year":"2022","journal-title":"Mathematics"},{"key":"B15","doi-asserted-by":"publisher","first-page":"2064","DOI":"10.1109\/TNNLS.2019.2927869","article-title":"Deep reinforcement learning-based automatic exploration for navigation in unknown environment","volume":"31","author":"Li","year":"2019","journal-title":"IEEE Trans. Neur. Netw. Learn. Syst."},{"key":"B16","doi-asserted-by":"publisher","first-page":"666","DOI":"10.3906\/elk-1504-194","article-title":"Obstacle avoidance for mobile robot based on improved dynamic window approach","volume":"25","author":"Li","year":"2017","journal-title":"Turkish J. Electr. Eng. Comput. Sci."},{"key":"B17","doi-asserted-by":"publisher","first-page":"101360","DOI":"10.1016\/j.aei.2021.101360","article-title":"Deep reinforcement learning-based safe interaction for industrial human-robot collaboration using intrinsic reward function","volume":"49","author":"Liu","year":"2021","journal-title":"Adv. Eng. Inform."},{"key":"B18","doi-asserted-by":"publisher","first-page":"109675","DOI":"10.1016\/j.enbuild.2019.109675","article-title":"Study on deep reinforcement learning techniques for building energy consumption forecasting","volume":"208","author":"Liu","year":"2020","journal-title":"Energy Build."},{"key":"B19","doi-asserted-by":"crossref","first-page":"259","DOI":"10.1007\/978-1-4613-8997-2_20","volume-title":"Spatial Planning: A Configuration Space Approach. Autonomous robot vehicles","author":"Lozano-Perez","year":"1990"},{"key":"B20","first-page":"1928","article-title":"\u201cAsynchronous methods for deep reinforcement learning,\u201d","volume-title":"International Conference on Machine Learning","author":"Mnih","year":"2016"},{"key":"B21","doi-asserted-by":"crossref","DOI":"10.23919\/FUSION49465.2021.9626988","article-title":"\u201cFiltering and sensor optimization applied to angle-only navigation,\u201d","volume-title":"2021 IEEE 24th International Conference on Information Fusion (FUSION)","author":"Musso","year":"2021"},{"key":"B22","doi-asserted-by":"publisher","first-page":"35338","DOI":"10.1109\/ACCESS.2021.3062557","article-title":"Critical design and control issues of indoor autonomous mobile robots: a review","volume":"9","author":"Niloy","year":"2021","journal-title":"IEEE Access"},{"key":"B23","doi-asserted-by":"publisher","first-page":"115","DOI":"10.3390\/s16010115","article-title":"Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition","volume":"16","author":"Ord\u00f3\u00f1ez","year":"2016","journal-title":"Sensors"},{"key":"B24","doi-asserted-by":"publisher","first-page":"11202","DOI":"10.3390\/app112311202","article-title":"A novel k-means clustering algorithm with a noise algorithm for capturing urban hotspots","volume":"11","author":"Ran","year":"2021","journal-title":"Appl. Sci."},{"key":"B25","doi-asserted-by":"crossref","DOI":"10.1109\/IROS.2017.8206458","article-title":"\u201cKinodynamic trajectory optimization and control for car-like robots,\u201d","volume-title":"2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"R\u00f6smann","year":"2017"},{"key":"B26","first-page":"1986","article-title":"\u201cDynamic window based approach to mobile robot motion control in the presence of moving obstacles,\u201d","volume-title":"Proceedings IEEE International Conference on Robotics and Automation IEEE","author":"Seder","year":"2007"},{"key":"B27","doi-asserted-by":"publisher","first-page":"484","DOI":"10.1038\/nature16961","article-title":"Mastering the game of Go with deep neural networks and tree search","volume":"529","author":"Silver","year":"2016","journal-title":"Nature"},{"key":"B28","doi-asserted-by":"publisher","first-page":"58","DOI":"10.1145\/203330.203343","article-title":"Temporal difference learning and TD-Gammon","volume":"38","author":"Tesauro","year":"1995","journal-title":"Commun. ACM."},{"key":"B29","doi-asserted-by":"publisher","first-page":"5246","DOI":"10.1109\/TSG.2018.2879572","article-title":"Model-free real-time EV charging scheduling based on deep reinforcement learning","volume":"10","author":"Wan","year":"2018","journal-title":"IEEE Trans. Smart Grid"},{"key":"B30","doi-asserted-by":"publisher","first-page":"11777","DOI":"10.3390\/app112411777","article-title":"Fast-RRT: A RRT-based optimal path finding method","volume":"11","author":"Wu","year":"2021","journal-title":"Appl Sci."},{"key":"B31","doi-asserted-by":"publisher","first-page":"2835","DOI":"10.3390\/rs14122835","article-title":"A review of multi-sensor fusion slam systems based on 3D LIDAR","volume":"14","author":"Xu","year":"2022","journal-title":"Remote Sens."},{"key":"B32","doi-asserted-by":"publisher","first-page":"1994","DOI":"10.48550\/arXiv.2006.07569","article-title":"The power of predictions in online control","volume":"33","author":"Yu","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"B33","doi-asserted-by":"publisher","first-page":"8948","DOI":"10.3390\/s22228948","article-title":"An optimization-based motion planner for car-like logistics robots on narrow roads","volume":"22","author":"Yu","year":"2022","journal-title":"Sensors"},{"key":"B34","doi-asserted-by":"publisher","first-page":"3837","DOI":"10.3390\/s19183837","article-title":"Navigation in unknown dynamic environments based on deep reinforcement learning","volume":"19","author":"Zeng","year":"2019","journal-title":"Sensors"},{"key":"B35","doi-asserted-by":"publisher","first-page":"47824","DOI":"10.1109\/ACCESS.2020.2978077","article-title":"The experience-memory Q-learning algorithm for robot path planning in unknown environment","volume":"8","author":"Zhao","year":"2020","journal-title":"IEEE Access"},{"key":"B36","doi-asserted-by":"publisher","first-page":"674","DOI":"10.26599\/TST.2021.9010012","article-title":"Deep reinforcement learning based mobile robot navigation: A review","volume":"26","author":"Zhu","year":"2021","journal-title":"Tsinghua Sci. Technol."}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1200214\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,22]],"date-time":"2023-08-22T07:55:05Z","timestamp":1692690905000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1200214\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,22]]},"references-count":36,"alternative-id":["10.3389\/fnbot.2023.1200214"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1200214","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,22]]},"article-number":"1200214"}}