{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T07:17:06Z","timestamp":1777706226518,"version":"3.51.4"},"reference-count":23,"publisher":"SAGE Publications","issue":"3","license":[{"start":{"date-parts":[[2025,7,27]],"date-time":"2025-07-27T00:00:00Z","timestamp":1753574400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Intelligent &amp; Fuzzy Systems: Applications in Engineering and Technology"],"published-print":{"date-parts":[[2026,3]]},"abstract":"<jats:p>Excavator path planning in modern smart ports facilitates efficient and safe access to work areas by accurately identifying bulk storage locations. This approach not only enhances operational efficiency and reduces costs but also prioritizes the safety of large machinery movement. However, the bulk cargo areas of ports often contain irregular and dynamic obstacles, posing significant challenges to traditional path planning algorithms. This study addresses the issues of local optima and training instability in excavator path planning by introducing an enhanced deep reinforcement learning (DRL) algorithm, AS-TD3 (an improved TD3 algorithm). The algorithm is evaluated by simulating the port bulk area environment on a continuous map. By integrating the A* heuristic function with the reward mechanism of the TD3 algorithm, AS-TD3 enhances the global optimal solution discovery by accounting for distance, time, and state variations in path planning. The A* component provides an efficient heuristic search, while TD3 refines the decision-making process through reinforcement learning. Additionally, the epsilon-greedy strategy effectively balances exploration and exploitation, facilitating smooth convergence of the reward curve. Experimental results indicate that the AS-TD3 algorithm reduces the steps required to find the optimal path by 5.7% and accelerates convergence by 58.75%.<\/jats:p>","DOI":"10.1177\/18758967251358112","type":"journal-article","created":{"date-parts":[[2025,7,28]],"date-time":"2025-07-28T06:57:57Z","timestamp":1753685877000},"page":"788-802","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["Excavator Path Planning Based on Reinforcement Learning"],"prefix":"10.1177","volume":"50","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-0869-8955","authenticated-orcid":false,"given":"Lei","family":"Li","sequence":"first","affiliation":[{"name":"School of Engineering, Qufu Normal University, Rizhao, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-2107-1701","authenticated-orcid":false,"given":"Xinyu","family":"Dong","sequence":"additional","affiliation":[{"name":"School of Engineering, Qufu Normal University, Rizhao, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2025,7,27]]},"reference":[{"issue":"11","key":"e_1_3_2_2_1","first-page":"137","article-title":"Global dynamic path planning by integrating improved A* algorithm and dynamic window method","volume":"51","author":"Cheng C.-Q.","year":"2017","unstructured":"Cheng C.-Q., Hao X.-Y., Li J.-S. (2017). Global dynamic path planning by integrating improved A* algorithm and dynamic window method. Journal of Xi'an Jiaotong University, 51(11), 137\u2013143. https:\/\/doi.org\/10.7652\/xjtuxb201711019","journal-title":"Journal of Xi'an Jiaotong University"},{"key":"e_1_3_2_3_1","doi-asserted-by":"publisher","DOI":"10.3390\/s24051523"},{"key":"e_1_3_2_4_1","doi-asserted-by":"publisher","DOI":"10.3390\/machines10080707"},{"issue":"16","key":"e_1_3_2_5_1","first-page":"11","article-title":"Research on mobile robot path planning based on improved DDQN algorithm","volume":"20","author":"Fu Z.","year":"2024","unstructured":"Fu Z., Hu C., Li K., Wang X. (2024). Research on mobile robot path planning based on improved DDQN algorithm. Computer Knowledge and Technology, 20(16), 11\u201314. https:\/\/doi.org\/10.14004\/j.cnki.ckt.2024.0810","journal-title":"Computer Knowledge and Technology"},{"key":"e_1_3_2_6_1","unstructured":"Fujimoto S. van Hoof H. Meger D. (2024). Addressing function approximation error in actor-critic methods. Online Referencing. https:\/\/doi.org\/10.14004\/j.cnki.ckt.2024.0810"},{"key":"e_1_3_2_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.compeleceng.2024.109276"},{"key":"e_1_3_2_8_1","doi-asserted-by":"publisher","DOI":"10.3390\/app131810056"},{"key":"e_1_3_2_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-021-02303-8"},{"issue":"3","key":"e_1_3_2_10_1","first-page":"1573","article-title":"A path planning method based on the particle swarm optimization trained fuzzy neural network algorithm","volume":"24","author":"Liu X.-H.","unstructured":"Liu X.-H., Zhang D., Zhang J., Zhang T., Zhu H. (2021a). A path planning method based on the particle swarm optimization trained fuzzy neural network algorithm. Cluster Computing, 24(3), 1573\u20137543. https:\/\/doi.org\/10.1007\/s10586-021-03235-1","journal-title":"Cluster Computing"},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.1155\/2023\/3373873"},{"key":"e_1_3_2_12_1","doi-asserted-by":"publisher","DOI":"10.3390\/app14177654"},{"key":"e_1_3_2_13_1","volume-title":"Research on path planning and target detection of construction machinery Master's thesis","author":"Tian M.-Z.","year":"2023","unstructured":"Tian M.-Z. (2023). Research on path planning and target detection of construction machinery Master's thesis. Shandong University. https:\/\/doi.org\/10.27272\/d.cnki.gshdu.2023.003339"},{"key":"e_1_3_2_14_1","volume-title":"Path planning of unmanned construction machinery based on potential field ant colony algorithm","author":"Wang Z.-H.","year":"2021","unstructured":"Wang Z.-H. (2021). Path planning of unmanned construction machinery based on potential field ant colony algorithm. Master's thesis Shandong University. https:\/\/doi.org\/10.27272\/d.cnki.gshdu.2021.003805"},{"key":"e_1_3_2_15_1","volume-title":"Indoor mobile robot path planning and real-time obstacle avoidance based on improved SAC algorithm Master's thesis","author":"Wang P.-C.","year":"2023","unstructured":"Wang P.-C. (2023). Indoor mobile robot path planning and real-time obstacle avoidance based on improved SAC algorithm Master's thesis. Dalian Maritime University. https:\/\/doi.org\/10.26989\/d.cnki.gdlhu.2023.000938"},{"key":"e_1_3_2_16_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.displa.2024.102796"},{"key":"e_1_3_2_17_1","article-title":"Costmap A* Guided Reinforcement Learning Path Planning Method for Complex Environments Navigation","author":"Wang Y.","unstructured":"Wang Y., Shen B., Xiong L.(2024b). Costmap A* Guided Reinforcement Learning Path Planning Method for Complex Environments Navigation. Shanghai Jiaotong University (Science), (prepublish).","journal-title":"Shanghai Jiaotong University (Science)"},{"key":"e_1_3_2_18_1","doi-asserted-by":"publisher","DOI":"10.3390\/electronics12091998"},{"key":"e_1_3_2_19_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-40038-3"},{"key":"e_1_3_2_20_1","volume-title":"Research on path planning algorithm of mobile robot based on deep reinforcement learning Master's thesis","author":"Zhang L.","year":"2024","unstructured":"Zhang L. (2024). Research on path planning algorithm of mobile robot based on deep reinforcement learning Master's thesis. Henan University of Technology. https:\/\/doi.org\/10.27791\/d.cnki.ghegy.2024.000757"},{"issue":"03","key":"e_1_3_2_21_1","first-page":"365","article-title":"Mobile robot path planning algorithm based on improved deep dual-Q network","volume":"53","author":"Zhang L.","year":"2024","unstructured":"Zhang L., Mu Y.-S., Pan Q. (2024). Mobile robot path planning algorithm based on improved deep dual-Q network. Information and Control, 53(03), 365\u2013376. https:\/\/doi.org\/10.13976\/j.cnki.xk.2024.3090","journal-title":"Information and Control"},{"issue":"5","key":"e_1_3_2_22_1","article-title":"Improved reinforcement learning path planning algorithm integrating prior knowledge","volume":"18","author":"Zhen S.","year":"2023","unstructured":"Zhen S., Wang K., Zhang J. (2023). Improved reinforcement learning path planning algorithm integrating prior knowledge. PLoS One, 18(5), e0284942. https:\/\/doi.org\/10.1371\/journal.pone.0284942","journal-title":"PLoS One"},{"key":"e_1_3_2_23_1","doi-asserted-by":"publisher","DOI":"10.1142\/S0218213024400050"},{"issue":"3","key":"e_1_3_2_24_1","first-page":"131","article-title":"Artificial Intelligence Technology for Path Planning of Automated Earthwork Machinery","volume":"51","author":"Zhou C.","year":"2025","unstructured":"Zhou C., Wang Y., Li R. (2025). Artificial Intelligence Technology for Path Planning of Automated Earthwork Machinery. Journal of Field Robotics, 51(3), 131\u2013146. https:\/\/doi.org\/10.1002\/rob.22479","journal-title":"Journal of Field Robotics"}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems: Applications in Engineering and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/18758967251358112","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/18758967251358112","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/18758967251358112","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:46:21Z","timestamp":1777455981000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/18758967251358112"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,27]]},"references-count":23,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,3]]}},"alternative-id":["10.1177\/18758967251358112"],"URL":"https:\/\/doi.org\/10.1177\/18758967251358112","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,27]]}}}