{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T02:20:23Z","timestamp":1787883623278,"version":"build-2784847793"},"reference-count":43,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,3,5]],"date-time":"2025-03-05T00:00:00Z","timestamp":1741132800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["52005443"],"award-info":[{"award-number":["52005443"]}]},{"name":"National Natural Science Foundation of China","award":["LQ21E050016"],"award-info":[{"award-number":["LQ21E050016"]}]},{"name":"Natural Science Foundation of Zhejiang Province","award":["52005443"],"award-info":[{"award-number":["52005443"]}]},{"name":"Natural Science Foundation of Zhejiang Province","award":["LQ21E050016"],"award-info":[{"award-number":["LQ21E050016"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>Dynamic path planning enables vehicles to autonomously navigate in unknown or continuously changing environments, thereby reducing reliance on fixed maps. Deep reinforcement learning (DRL), with its superior performance in handling high-dimensional state spaces and complex dynamic environments, has been widely applied to dynamic path planning. Traditional DRL methods are prone to capturing unnecessary noise information and irrelevant features during the training process, leading to instability and decreased adaptability of models in complex dynamic environments. To address this challenge, we propose a dynamic path-planning method based on our Causal State-Masking Twin-delayed Deep Deterministic Policy Gradient (CSM-TD3) algorithm. CSM-TD3 integrates a causal inference mechanism by introducing dynamic state masks and intervention mechanisms, allowing the policy network to focus on genuine causal features for decision optimization and thereby enhancing the convergence speed and generalization capabilities of the agent. Furthermore, causal state-masking DRL allows the system to learn the optimal mask configurations through backpropagation, enabling the model to adaptively adjust the causal features of interest. Extensive experimental results demonstrate that this method significantly enhances the convergence of the TD3 algorithm and effectively improves its performance in dynamic path planning.<\/jats:p>","DOI":"10.3390\/a18030146","type":"journal-article","created":{"date-parts":[[2025,3,5]],"date-time":"2025-03-05T03:18:22Z","timestamp":1741144702000},"page":"146","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Dynamic Path Planning for Vehicles Based on Causal State-Masking Deep Reinforcement Learning"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0412-4329","authenticated-orcid":false,"given":"Xia","family":"Hua","sequence":"first","affiliation":[{"name":"College of Mechanical Engineering, Zhejiang University of Technology, 18 Chaowang Road, Hangzhou 310014, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tengteng","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Mechanical Engineering, Zhejiang University of Technology, 18 Chaowang Road, Hangzhou 310014, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jun","family":"Cao","sequence":"additional","affiliation":[{"name":"College of Mechanical Engineering, Zhejiang University of Technology, 18 Chaowang Road, Hangzhou 310014, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1049\/iet-csr.2020.0004","article-title":"Survey of UAV motion planning","volume":"2","author":"Quan","year":"2020","journal-title":"IET Cyber-Syst. Robot."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"9745","DOI":"10.1109\/ACCESS.2018.2888617","article-title":"Path planning technologies for autonomous underwater vehicles-a review","volume":"7","author":"Li","year":"2018","journal-title":"IEEE Access"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"050020","DOI":"10.1063\/5.0215501","article-title":"Path planning techniques for autonomous vehicles","volume":"3144","author":"Wu","year":"2024","journal-title":"AIP Conf. Proc."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"898","DOI":"10.1016\/j.procs.2023.03.122","article-title":"Comparison of path planning methods for robot navigation in simulated agricultural environments","volume":"220","author":"Basoalto","year":"2023","journal-title":"Procedia Comput. Sci."},{"key":"ref_5","unstructured":"Feng, L., and Jia, J. (2011). Improved algorithm of RRT path planning based on comparison optimization. Jisuanji Gongcheng Yu Yingyong (Comput. Eng. Appl.), 47, Available online: https:\/\/api.semanticscholar.org\/CorpusID:62978159."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hou, J., Shi, L., Jiang, W., Luo, Z., and Yang, L. (2023, January 24\u201326). Dynamic path planning of mobile robots by combining improved informed-RRT* and VFH+ algorithms. Proceedings of the Eleventh International Symposium on Precision Mechanical Measurements, Guangzhou, China.","DOI":"10.1117\/12.3032931"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Liu, Y., Wang, C., Wu, H., and Wei, Y. (2023). Mobile Robot Path Planning Based on Kinematically Constrained A-Star Algorithm and DWA Fusion Algorithm. Mathematics, 11.","DOI":"10.3390\/math11214552"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"8867937","DOI":"10.1155\/2020\/8867937","article-title":"2D Lidar-Based SLAM and Path Planning for Indoor Rescue Using Mobile Robots","volume":"2020","author":"Zhang","year":"2020","journal-title":"J. Adv. Transp."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1016\/j.ifacol.2022.11.244","article-title":"A proposal of path planning for robots in warehouses by model predictive control without using global paths","volume":"55","author":"Ishihara","year":"2022","journal-title":"IFAC-PapersOnLine"},{"key":"ref_10","first-page":"50","article-title":"Prospects of AI assisted planning based on machine learning","volume":"24","author":"Huang","year":"2017","journal-title":"Urban Dev. Stud."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"778","DOI":"10.1007\/s41315-023-00298-8","article-title":"Performance analysis of path planning techniques for autonomous robots: A deep path planning analysis in 2D environments","volume":"7","author":"Rocha","year":"2023","journal-title":"Int. J. Intell. Robot. Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"567","DOI":"10.1007\/s42405-022-00461-8","article-title":"Fopid control of quadrotor based on neural networks optimization and path planning through machine learning and pso algorithm","volume":"23","author":"Mokhtari","year":"2022","journal-title":"Int. J. Aeronaut. Space Sci."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"182784","DOI":"10.1109\/ACCESS.2019.2959346","article-title":"A novel path planning method for articulated road roller using support vector machine and longest accessible path with course correction","volume":"7","author":"Xu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"128423","DOI":"10.1016\/j.neucom.2024.128423","article-title":"Recent progress, challenges and future prospects of applied deep reinforcement learning: A practical perspective in path planning","volume":"608","author":"Zhang","year":"2024","journal-title":"Neurocomputing"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Almazrouei, K., Kamel, I., and Rabie, T. (2023). Dynamic obstacle avoidance and path planning through reinforcement learning. Appl. Sci., 13.","DOI":"10.3390\/app13148174"},{"key":"ref_16","first-page":"1619","article-title":"Path planning of mobile robots based on memristor reinforcement learning in dynamic environment","volume":"35","author":"Yang","year":"2023","journal-title":"J. Syst. Simul."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.ifacol.2022.08.042","article-title":"Dynamic path planning for mobile robots with deep reinforcement learning","volume":"55","author":"Yang","year":"2022","journal-title":"IFAC-PapersOnLine"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ren, J., Huang, X., and Huang, R.N. (2022). Efficient deep reinforcement learning for optimal path planning. Electronics, 11.","DOI":"10.3390\/electronics11213628"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"9003","DOI":"10.1109\/TVT.2023.3348140","article-title":"An Efficient and Accurate A-star Algorithm for Autonomous Vehicle Path Planning","volume":"73","author":"Lin","year":"2023","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1007\/s44196-024-00571-z","article-title":"A Multiple Environment Available Path Planning Based on an Improved A* Algorithm","volume":"17","author":"Sun","year":"2024","journal-title":"Int. J. Comput. Intell. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Chen, G., Tan, Y., Zeng, G., Tang, F., and Zhou, W. (2024, January 10\u201312). Research on multipath planning problems based on an improved LPA* algorithm. Proceedings of the Seventh International Conference on Advanced Electronic Materials, Computers, and Software Engineering (AEMCSE 2024), Nanchang, China.","DOI":"10.1117\/12.3038338"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"tdac061","DOI":"10.1093\/tse\/tdac061","article-title":"Path-planning algorithms for self-driving vehicles based on improved RRT-Connect","volume":"5","author":"Li","year":"2023","journal-title":"Transp. Saf. Environ."},{"key":"ref_23","first-page":"111","article-title":"Path planning of mobile robot using improved RRT_Connect algorithm","volume":"42","author":"Shi","year":"2024","journal-title":"Sci. Technol. Rev."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1183","DOI":"10.1109\/LRA.2023.3236577","article-title":"Path re-planning design of a cobot in a dynamic environment based on current obstacle configuration","volume":"8","author":"Lee","year":"2023","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Chen, Z., Yu, J., Zhao, Z., Wang, X., and Chen, Y. (2023). A path-planning method considering environmental disturbance based on VPF-RRT. Drones, 7.","DOI":"10.3390\/drones7020145"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"7695","DOI":"10.1109\/LRA.2023.3322349","article-title":"A Homotopy Invariant Based on Convex Dissection Topology and a Distance Optimal Path Planning Algorithm","volume":"8","author":"Liu","year":"2023","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wen, Y., Wen, H., and Zhang, Z. (2021, January 8\u201310). Obstacle avoidance path planning of manipulator based on improved RRT algorithm. Proceedings of the 2021 International Conference on Computer, Control and Robotics (ICCCR), Shanghai, China.","DOI":"10.1109\/ICCCR49711.2021.9349398"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Tan, X., Han, L., Gong, H., and Wu, Q. (2023). Biologically inspired complete coverage path planning algorithm based on Q-learning. Sensors, 23.","DOI":"10.3390\/s23104647"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Zhang, P., Zhang, C.Y., and Gai, W.L. (2022, January 4\u20136). Research on path planning algorithm of unmanned ground platform based on reinforcement learning. Proceedings of the Third International Conference on Artificial Intelligence and Computer Engineering (ICAICE 2022), Wuhan, China.","DOI":"10.1117\/12.2671690"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"947","DOI":"10.1017\/S0263574723001613","article-title":"Reinforcement learning-based motion control for snake robots in complex environments","volume":"42","author":"Zhang","year":"2024","journal-title":"Robotica"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"44051","DOI":"10.1109\/ACCESS.2023.3273164","article-title":"B-APFDQN: A UAV path planning algorithm based on deep Q-network and artificial potential field","volume":"11","author":"Kong","year":"2023","journal-title":"IEEE Access"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"4489","DOI":"10.1109\/LRA.2023.3284354","article-title":"Robot navigation with reinforcement learned path generation and fine-tuned motion control","volume":"8","author":"Zhang","year":"2023","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Dan, H., and Peng, H. (2022, January 25\u201327). 3D path planning of UAV based on improved reinforcement learning. Proceedings of the Second International Conference on Electronic Information Engineering and Computer Communication (EIECC 2022), Xi\u2019an, China.","DOI":"10.1117\/12.2671556"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Jaramillo-Mart\u00ednez, R., Chavero-Navarrete, E., and Ibarra-P\u00e9rez, T. (2024). Reinforcement-Learning-Based Path Planning: A Reward Function Strategy. Appl. Sci., 14.","DOI":"10.3390\/app14177654"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2625","DOI":"10.1109\/TETCI.2024.3369485","article-title":"Path Planning for Unmanned Aerial Vehicle via Off-Policy Reinforcement Learning With Enhanced Exploration","volume":"8","author":"Wang","year":"2024","journal-title":"IEEE Trans. Emerg. Top. Comput. Intell."},{"key":"ref_36","first-page":"1674","article-title":"A spacecraft attitude maneuvering path planning method based on PIO-improved reinforcement learning","volume":"53","author":"Hua","year":"2021","journal-title":"Sci. Sin. Technol."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Feng, Z., Wang, G., Chang, Y., Shi, Y., Geng, J., and Zhu, C. (2023, January 14\u201316). Research on path planning and obstacle avoidance for unmanned platforms based on reinforcement learning. Proceedings of the International Conference on Electronic Materials and Information Engineering (EMIE 2023), Guangzhou, China.","DOI":"10.1117\/12.3010836"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"108865","DOI":"10.1016\/j.asoc.2022.108865","article-title":"Attitude control for hypersonic reentry vehicles: An efficient deep reinforcement learning method","volume":"123","author":"Liu","year":"2022","journal-title":"Appl. Soft Comput."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"112406","DOI":"10.1016\/j.knosys.2024.112406","article-title":"Causal deconfounding deep reinforcement learning for mobile robot motion planning","volume":"303","author":"Tang","year":"2024","journal-title":"Knowl.-Based Syst."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"C76","DOI":"10.1364\/JOCN.518685","article-title":"Deep-reinforcement-learning-based RMSCA for space division multiplexing networks with multi-core fibers","volume":"16","author":"Teng","year":"2024","journal-title":"J. Opt. Commun. Netw."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"6654","DOI":"10.1109\/TNNLS.2022.3212273","article-title":"Stochastic integrated actor\u2013critic for deep reinforcement learning","volume":"35","author":"Zheng","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"38017","DOI":"10.1109\/ACCESS.2024.3375083","article-title":"UAV path planning based on the average TD3 algorithm with prioritized experience replay","volume":"12","author":"Luo","year":"2024","journal-title":"IEEE Access"},{"key":"ref_43","first-page":"63","article-title":"A unified view of causal and non-causal feature selection","volume":"15","author":"Yu","year":"2021","journal-title":"ACM Trans. Knowl. Discov. Data (TKDD)"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/3\/146\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:47:22Z","timestamp":1760028442000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/18\/3\/146"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,5]]},"references-count":43,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,3]]}},"alternative-id":["a18030146"],"URL":"https:\/\/doi.org\/10.3390\/a18030146","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,5]]}}}