{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,24]],"date-time":"2026-07-24T15:15:05Z","timestamp":1784906105572,"version":"3.55.0"},"reference-count":38,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2024,9,12]],"date-time":"2024-09-12T00:00:00Z","timestamp":1726099200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and Technology of China","award":["2022YFB4703600"],"award-info":[{"award-number":["2022YFB4703600"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Autonomous decision-making is a hallmark of intelligent mobile robots and an essential element of autonomous navigation. The challenge is to enable mobile robots to complete autonomous navigation tasks in environments with mapless or low-precision maps, relying solely on low-precision sensors. To address this, we have proposed an innovative autonomous navigation algorithm called PEEMEF-DARC. This algorithm consists of three parts: Double Actors Regularized Critics (DARC), a priority-based excellence experience data collection mechanism, and a multi-source experience fusion strategy mechanism. The algorithm is capable of performing autonomous navigation tasks in unmapped and unknown environments without maps or prior knowledge. This algorithm enables autonomous navigation in unmapped and unknown environments without the need for maps or prior knowledge. Our enhanced algorithm improves the agent\u2019s exploration capabilities and utilizes regularization to mitigate the overestimation of state-action values. Additionally, the priority-based excellence experience data collection module and the multi-source experience fusion strategy module significantly reduce training time. Experimental results demonstrate that the proposed method excels in navigating the unmapped and unknown, achieving effective navigation without relying on maps or precise localization.<\/jats:p>","DOI":"10.3390\/s24185925","type":"journal-article","created":{"date-parts":[[2024,9,12]],"date-time":"2024-09-12T10:15:27Z","timestamp":1726136127000},"page":"5925","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Learning Autonomous Navigation in Unmapped and Unknown Environments"],"prefix":"10.3390","volume":"24","author":[{"given":"Naifeng","family":"He","sequence":"first","affiliation":[{"name":"College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhong","family":"Yang","sequence":"additional","affiliation":[{"name":"College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chunguang","family":"Bu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics, Shenyang Institute of Automation Chinese Academy of Sciences, Shenyang 110017, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoliang","family":"Fan","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics, Shenyang Institute of Automation Chinese Academy of Sciences, Shenyang 110017, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3959-8946","authenticated-orcid":false,"given":"Jiying","family":"Wu","sequence":"additional","affiliation":[{"name":"College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yaoyu","family":"Sui","sequence":"additional","affiliation":[{"name":"College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wenqiang","family":"Que","sequence":"additional","affiliation":[{"name":"College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,9,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"3092","DOI":"10.1109\/TASE.2023.3274924","article-title":"WiFi Similarity-Based Odometry","volume":"21","author":"Ismail","year":"2023","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Zhang, T., Zhang, H., Li, Y., Nakamura, Y., and Zhang, L. (August, January 31). Flowfusion: Dynamic dense rgb-d slam based on optical flow. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9197349"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1109\/TPAMI.2017.2658577","article-title":"Direct sparse odometry","volume":"40","author":"Engel","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3577","DOI":"10.1109\/TIE.2020.2982096","article-title":"DeepSLAM: A Robust Monocular SLAM System With Unsupervised Deep Learning","volume":"68","author":"Li","year":"2021","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lample, G., and Chaplot, D.S. (2017, January 4\u20139). Playing FPS games with deep reinforcement learning. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10827"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1140","DOI":"10.1126\/science.aar6404","article-title":"A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play","volume":"362","author":"Silver","year":"2018","journal-title":"Science"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Li, J., Monroe, W., Ritter, A., Galley, M., Gao, J., and Jurafsky, D. (2016). Deep reinforcement learning for dialogue generation. arXiv.","DOI":"10.18653\/v1\/D16-1127"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Ismail, J., and Ahmed, A. (2022). Improving a sequence-to-sequence nlp model using a reinforcement learning policy algorithm. arXiv.","DOI":"10.5121\/csit.2022.122317"},{"key":"ref_9","unstructured":"Lyu, J., Ma, X., Yan, J., and Li, X. (March, January 22). Efficient continuous control with double actors and regularized critics. Proceedings of the AAAI Conference on Artificial Intelligence, Online."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1109\/3477.499793","article-title":"Model-based learning for mobile robot navigation from the dynamical systems perspective","volume":"26","author":"Tani","year":"1996","journal-title":"IEEE Trans. Syst. Man Cybern. Part B (Cybern.)"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1109\/TSMCC.2007.897499","article-title":"Neurofuzzy-based approach to mobile robot navigation in unknown environments","volume":"37","author":"Zhu","year":"2007","journal-title":"IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.)"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1090","DOI":"10.1109\/LRA.2021.3056373","article-title":"A lifelong learning approach to mobile robot navigation","volume":"6","author":"Liu","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"8776","DOI":"10.1109\/TITS.2023.3269533","article-title":"Graph Relational Reinforcement Learning for Mobile Robot Navigation in Large-Scale Crowded Environments","volume":"24","author":"Liu","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1874","DOI":"10.1109\/TRO.2021.3075644","article-title":"Orb-slam3: An accurate open-source library for visual, visual\u2013inertial, and multimap slam","volume":"37","author":"Campos","year":"2021","journal-title":"IEEE Trans. Robot."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1255","DOI":"10.1109\/TRO.2017.2705103","article-title":"Orb-slam2: An open-source slam system for monocular, stereo, and rgb-d cameras","volume":"33","year":"2017","journal-title":"IEEE Trans. Robot."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1109\/TRO.2015.2463671","article-title":"ORB-SLAM: A versatile and accurate monocular SLAM system","volume":"31","author":"Montiel","year":"2015","journal-title":"IEEE Trans. Robot."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hess, W., Kohler, D., Rapp, H., and Andor, D. (2016, January 16\u201321). Real-time loop closure in 2D LIDAR SLAM. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden.","DOI":"10.1109\/ICRA.2016.7487258"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Shan, T., and Englot, B. (2018, January 1\u20135). Lego-loam: Lightweight and ground-optimized lidar odometry and mapping on variable terrain. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594299"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kim, G., and Kim, A. (2022, January 23\u201327). LT-mapper: A modular framework for lidar-based lifelong mapping. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9811916"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wang, J., R\u00fcnz, M., and Agapito, L. (2021, January 1\u20133). DSP-SLAM: Object oriented SLAM with deep shape priors. Proceedings of the 2021 International Conference on 3D Vision (3DV), London, UK.","DOI":"10.1109\/3DV53792.2021.00143"},{"key":"ref_21","unstructured":"Yue, J., Wen, W., Han, J., and Hsu, L.T. (2020). LiDAR data enrichment using deep learning based on high-resolution image: An approach to achieve high-performance LiDAR SLAM using Low-cost LiDAR. arXiv."},{"key":"ref_22","first-page":"1","article-title":"Visual SLAM algorithms: A survey from 2010 to 2016","volume":"9","author":"Taketomi","year":"2017","journal-title":"IPSJ Trans. Comput. Vis. Appl."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Henein, M., Zhang, J., Mahony, R., and Ila, V. (August, January 31). Dynamic SLAM: The need for speed. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9196895"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"7518","DOI":"10.1109\/LRA.2022.3183759","article-title":"The hilti slam challenge dataset","volume":"7","author":"Helmberger","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Bekta\u015f, K., and Bozma, H.I. (2022, January 23\u201327). Apf-rl: Safe mapless navigation in unknown environments. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9811537"},{"key":"ref_26","unstructured":"Surmann, H., Jestel, C., Marchel, R., Musberg, F., Elhadj, H., and Ardani, M. (2020). Deep reinforcement learning for real autonomous mobile robot navigation in indoor environments. arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"730","DOI":"10.1109\/LRA.2021.3133591","article-title":"Goal-driven autonomous exploration through deep reinforcement learning","volume":"7","author":"Cimurs","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Marchesini, E., and Farinelli, A. (2022, January 23\u201327). Enhancing deep reinforcement learning approaches for multi-robot navigation via single-robot evolutionary policy search. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9812341"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Lodel, M., Brito, B., Serra-G\u00f3mez, A., Ferranti, L., Babu\u0161ka, R., and Alonso-Mora, J. (2022, January 23\u201327). Where to look next: Learning viewpoint recommendations for informative trajectory planning. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9812190"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Lee, K., Kim, S., and Choi, J. (2023). Adaptive and Explainable Deployment of Navigation Skills via Hierarchical Deep Reinforcement Learning. arXiv.","DOI":"10.1109\/ICRA48891.2023.10160371"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Dawood, M., Dengler, N., de Heuvel, J., and Bennewitz, M. (June, January 29). Handling Sparse Rewards in Reinforcement Learning Using Model Predictive Control. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10161492"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Cui, Y., Lin, L., Huang, X., Zhang, D., Wang, Y., Jing, W., Chen, J., Xiong, R., and Wang, Y. (2022, January 23\u201327). Learning Observation-Based Certifiable Safe Policy for Decentralized Multi-Robot Navigation. Proceedings of the 2022 International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9811950"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Shen, Y., Li, W., and Lin, M.C. (2022, January 23\u201327). Inverse reinforcement learning with hybrid-weight trust-region optimization and curriculum learning for autonomous maneuvering. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.","DOI":"10.1109\/IROS47612.2022.9981103"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Xiong, Z., Eappen, J., Qureshi, A.H., and Jagannathan, S. (2022, January 23\u201327). Model-free neural lyapunov control for safe robot navigation. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.","DOI":"10.1109\/IROS47612.2022.9981632"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep reinforcement learning with double q-learning. Proceedings of the AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Marchesini, E., and Farinelli, A. (August, January 31). Discrete deep reinforcement learning for mapless navigation. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9196739"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Niu, H., Ji, Z., Arvin, F., Lennox, B., Yin, H., and Carrasco, J. (2021, January 11\u201314). Accelerated sim-to-real deep reinforcement learning: Learning collision avoidance from human player. Proceedings of the 2021 IEEE\/SICE International Symposium on System Integration (SII), Iwaki, Fukushima, Japan.","DOI":"10.1109\/IEEECONF49454.2021.9382693"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Marzari, L., Marchesini, E., and Farinelli, A. (2023). Online Safety Property Collection and Refinement for Safe Deep Reinforcement Learning in Mapless Navigation. arXiv.","DOI":"10.1109\/ICRA48891.2023.10161312"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/18\/5925\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:54:56Z","timestamp":1760111696000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/18\/5925"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,12]]},"references-count":38,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2024,9]]}},"alternative-id":["s24185925"],"URL":"https:\/\/doi.org\/10.3390\/s24185925","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,12]]}}}