{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T09:32:00Z","timestamp":1773307920678,"version":"3.50.1"},"reference-count":35,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2024,4,15]],"date-time":"2024-04-15T00:00:00Z","timestamp":1713139200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Scientific Research and Innovation Team Program of Sichuan University of Science and Engineering","award":["H92322"],"award-info":[{"award-number":["H92322"]}]},{"name":"Scientific Research and Innovation Team Program of Sichuan University of Science and Engineering","award":["Y2023085"],"award-info":[{"award-number":["Y2023085"]}]},{"name":"Graduate Innovation Fund of Sichuan University of Science &amp; Engineering","award":["H92322"],"award-info":[{"award-number":["H92322"]}]},{"name":"Graduate Innovation Fund of Sichuan University of Science &amp; Engineering","award":["Y2023085"],"award-info":[{"award-number":["Y2023085"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The swift advancements in robotics have rendered navigation an essential task for mobile robots. While map-based navigation methods depend on global environmental maps for decision-making, their efficacy in unfamiliar or dynamic settings falls short. Current deep reinforcement learning navigation strategies can navigate successfully without pre-existing map data, yet they grapple with issues like inefficient training, slow convergence, and infrequent rewards. To tackle these challenges, this study introduces an improved two-delay depth deterministic policy gradient algorithm (LP-TD3) for local planning navigation. Initially, the integration of the long\u2013short-term memory (LSTM) module with the Prioritized Experience Re-play (PER) mechanism into the existing TD3 framework was performed to optimize training and improve the efficiency of experience data utilization. Furthermore, the incorporation of an Intrinsic Curiosity Module (ICM) merges intrinsic with extrinsic rewards to tackle sparse reward problems and enhance exploratory behavior. Experimental evaluations using ROS and Gazebo simulators demonstrate that the proposed method outperforms the original on various performance metrics.<\/jats:p>","DOI":"10.3390\/s24082525","type":"journal-article","created":{"date-parts":[[2024,4,15]],"date-time":"2024-04-15T08:08:12Z","timestamp":1713168492000},"page":"2525","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Inspection Robot Navigation Based on Improved TD3 Algorithm"],"prefix":"10.3390","volume":"24","author":[{"given":"Bo","family":"Huang","sequence":"first","affiliation":[{"name":"School of Mechanical Engineering, Sichuan University of Science and Engineering, Zigong 643099, China"}]},{"given":"Jiacheng","family":"Xie","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Sichuan University of Science and Engineering, Zigong 643099, China"}]},{"given":"Jiawei","family":"Yan","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Sichuan University of Science and Engineering, Zigong 643099, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,4,15]]},"reference":[{"key":"ref_1","first-page":"96","article-title":"Mobile Robot Navigation and Obstacle Avoidance Techniques: A Review","volume":"2","author":"Pandey","year":"2017","journal-title":"Int. Robot. Autom. J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"582","DOI":"10.1016\/j.dt.2019.04.011","article-title":"A review: On path planning strategies for navigation of mobile robot","volume":"15","author":"Patle","year":"2019","journal-title":"Def. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, H.-Y., Lin, W.-M., and Chen, A.-X. (2018). Path Planning for the Mobile Robot: A Review. Symmetry, 10.","DOI":"10.3390\/sym10100450"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1109\/TSSC.1968.300136","article-title":"A Formal Basis for the Heuristic Determination of Minimum Cost Paths","volume":"4","author":"Hart","year":"1968","journal-title":"IEEE Trans. Syst. Sci. Cybern."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1007\/BF01386390","article-title":"A note on two problems in connexion with graphs","volume":"1","author":"Dijkstra","year":"1959","journal-title":"Numer. Math."},{"key":"ref_6","unstructured":"Kavraki, L., and Latombe, J.-C. (1994, January 8\u201313). Randomized preprocessing of configuration for fast path planning. Proceedings of the 1994 IEEE International Conference on Robotics and Automation, San Diego, CA, USA."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1186\/s10033-022-00775-4","article-title":"Hybrid Navigation System Based Autonomous Positioning and Path Planning for Mobile Robots","volume":"35","author":"Shentu","year":"2022","journal-title":"Chin. J. Mech. Eng."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li, Y., Li, J., Zhou, W., Yao, Q., Nie, J., and Qi, X. (2022). Robot Path Planning Navigation for Dense Planting Red Jujube Orchards Based on the Joint Improved A* and DWA Algorithms under Laser SLAM. Agriculture, 12.","DOI":"10.3390\/agriculture12091445"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1080\/02533839.2019.1694444","article-title":"Coarse grid partition to speed up A* robot navigation","volume":"43","author":"Wang","year":"2020","journal-title":"J. Chin. Inst. Eng."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3368961","article-title":"Autonomous Visual Navigation for Mobile Robots","volume":"53","author":"Yasuda","year":"2020","journal-title":"ACM Comput. Surv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1108\/SR-01-2019-0029","article-title":"Comparative analysis of range sensors for the robust autonomous navigation\u2014A review","volume":"40","author":"Singh","year":"2019","journal-title":"Sens. Rev."},{"key":"ref_12","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2023). Playing Atari with Deep Reinforcement Learning. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Van Hasselt, H., Guez, A., and Silver, D. (2016, January 12\u201317). Deep Reinforcement Learning with Double Q-Learning. Proceedings of the AAAI\u201916: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, Arizona.","DOI":"10.1609\/aaai.v30i1.10295"},{"key":"ref_14","unstructured":"Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2015). Continuous control with deep reinforcement learning. arXiv."},{"key":"ref_15","unstructured":"Mnih, V., Badia, A.P., Mirza, M., Graves, A., Lillicrap, T., Harley, T., Silver, D., and Kavukcuoglu, K. (2016, January 19\u201324). Asynchronous Methods for Deep Reinforcement Learning. Proceedings of the 33rd International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_16","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algorithms. arXiv."},{"key":"ref_17","unstructured":"Fujimoto, S., Hoof, H., and Meger, D. (2018, January 10\u201315). Addressing Function Approximation Error in Actor-Critic Methods. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_18","unstructured":"Chen, D., Peng, P., Huang, T., and Tian, Y. (2022). Deep Reinforcement Learning with Spiking Q-learning. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Wang, X., Ma, X., and Li, Z. (2023). Research on SLAM and Path Planning Method of Inspection Robot in Complex Scenarios. Electronics, 12.","DOI":"10.20944\/preprints202304.0219.v1"},{"key":"ref_20","unstructured":"Mirowski, P., Pascanu, R., Viola, F., Soyer, H., Ballard, A.J., Banino, A., Denil, M., Goroshin, R., Sifre, L., and Kavukcuoglu, K. (2016, January 2\u20134). Learning to Navigate in Complex Environments. Proceedings of the International Conference on Learning Representations, San Juan, Puerto Rico."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Tai, L., Paolo, G., and Liu, M. (2017, January 24\u201328). Virtual-to-real deep reinforcement learning: Continuous control of mobile robots for mapless navigation. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8202134"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Zhu, Y., Mottaghi, R., Kolve, E., Lim, J.J., Gupta, A., Li, F., and Farhadi, A. (June, January 29). Target-driven visual navigation in indoor scenes using deep reinforcement learning. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989381"},{"key":"ref_23","unstructured":"Xie, L., Wang, S., Markham, A., and Trigoni, N. (2017). Towards Monocular Vision based Obstacle Avoidance through Deep Reinforcement Learning. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Laina, I., Rupprecht, C., Belagiannis, V., Tombari, F., and Navab, N. (2016, January 25\u201328). Deeper depth prediction with fully convolutional residual networks. Proceedings of the 2016 Fourth International Conference on 3D Vision (3DV), Stanford, CA, USA.","DOI":"10.1109\/3DV.2016.32"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Gao, J., Ye, W., Guo, J., and Li, Z. (2020). Deep Reinforcement Learning for Indoor Mobile Robot Path Planning. Sensors, 20.","DOI":"10.3390\/s20195493"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1007\/s10514-020-09947-4","article-title":"Reinforcement based mobile robot path planning with improved dynamic window approach in unknown environment","volume":"45","author":"Chang","year":"2020","journal-title":"Auton. Robot."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1109\/100.580977","article-title":"The dynamic window approach to collision avoidance","volume":"4","author":"Fox","year":"1997","journal-title":"IEEE Robot. Autom. Mag."},{"key":"ref_28","unstructured":"Surmann, H., Jestel, C., Marchel, R., Musberg, F., Elhadj, H., and Ardani, M. (2020). Deep Reinforcement learning for real autonomous mobile robot navigation in indoor environments. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zhang, Q., Zhu, M., Zou, L., Li, M., and Zhang, Y. (2020). Learning Reward Function with Matching Network for Mapless Navigation. Sensors, 20.","DOI":"10.3390\/s20133664"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"730","DOI":"10.1109\/LRA.2021.3133591","article-title":"Goal-Driven Autonomous Exploration Through Deep Reinforcement Learning","volume":"7","author":"Cimurs","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_31","unstructured":"Kingma Diederik, P., and Ba, J. (2014). Adam: A Method for Stochastic Optimization. arXiv."},{"key":"ref_32","unstructured":"Bakker, B. (2001, January 3\u20138). Reinforcement Learning with Long Short-Term Memory. Proceedings of the NIPS\u201901: Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic, Vancouver, BC, Canada."},{"key":"ref_33","unstructured":"Schaul, T., Quan, J., Antonoglou, I., and Silver, D. (2015). Prioritized Experience Replay. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Park, M., Lee, S.Y., Hong, J.S., and Kwon, N.K. (2022). Deep Deterministic Policy Gradient-Based Autonomous Driving for Mobile Robots in Sparse Reward Environments. Sensors, 22.","DOI":"10.3390\/s22249574"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Pathak, D., Agrawal, P., Efros, A.A., and Darrell, T. (2017, January 21\u201326). Curiosity-Driven Exploration by Self-Supervised Prediction. Proceedings of the International Conference on Machine Learning, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.70"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/8\/2525\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:28:05Z","timestamp":1760106485000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/8\/2525"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,15]]},"references-count":35,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["s24082525"],"URL":"https:\/\/doi.org\/10.3390\/s24082525","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,15]]}}}