{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T19:11:32Z","timestamp":1790709092258,"version":"4.1.0"},"reference-count":33,"publisher":"MDPI AG","issue":"14","license":[{"start":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T00:00:00Z","timestamp":1784246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["U24A20144"],"award-info":[{"award-number":["U24A20144"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"award":["U24A20144"],"award-info":[{"award-number":["U24A20144"]}],"id":[{"id":"https:\/\/ror.org\/01h0zpd94","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Autonomous Underwater Vehicles (AUVs) face significant challenges in path-following control due to strong environmental disturbances and model uncertainties. To address these issues, this paper proposes a model-free deep reinforcement learning framework, named ILLT (Improved LOS-LSTM-TD3), which integrates an integral line-of-sight (LOS) guidance law with the twin delayed deep deterministic policy gradient (TD3) algorithm. The framework treats the LOS look-ahead distance as a learnable optimization variable and incorporates an LSTM network to capture temporal motion dependencies. A progressive unfreezing transfer learning strategy, combined with attention-based feature\u2013current fusion, is designed to enhance domain adaptation under varying ocean currents. Simulation results demonstrate that ILLT reduces the average cross-track error by 48.5% compared to the baseline ILT algorithm and by 66.4% compared to traditional PID control, while achieving significantly faster convergence in target domains. Physical experiments in tank and lake environments further validate the algorithm\u2019s feasibility and robustness, with tracking errors approaching simulation results under moderate current conditions. These findings confirm the effectiveness of the proposed framework for underactuated AUV path-following tasks.<\/jats:p>","DOI":"10.3390\/s26144548","type":"journal-article","created":{"date-parts":[[2026,7,20]],"date-time":"2026-07-20T06:09:09Z","timestamp":1784527749000},"page":"4548","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Deep Reinforcement Learning-Based Path-Following Control for Underactuated Autonomous Underwater Vehicles"],"prefix":"10.3390","volume":"26","author":[{"given":"Xin","family":"Pan","sequence":"first","affiliation":[{"name":"College of Electrical Engineering, Naval University of Engineering, Wuhan 430033, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Huang","sequence":"additional","affiliation":[{"name":"College of Electrical Engineering, Naval University of Engineering, Wuhan 430033, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Liangjin","family":"Li","sequence":"additional","affiliation":[{"name":"College of Electrical Engineering, Naval University of Engineering, Wuhan 430033, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Song","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Electrical Engineering, Naval University of Engineering, Wuhan 430033, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,7,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"125017","DOI":"10.1063\/5.0300484","article-title":"Research on Complete Coverage Path Planning for Multiple Unmanned Underwater Vehicles Based on Deep Reinforcement Learning","volume":"15","author":"Pan","year":"2025","journal-title":"AIP Adv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1109\/TSMC.2022.3162862","article-title":"Advances in Line-of-Sight Guidance for Path Following of Autonomous Marine Vehicles: An Overview","volume":"53","author":"Gu","year":"2023","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"83911","DOI":"10.1109\/ACCESS.2024.3412816","article-title":"Three-Dimensional Path Following Control of Underactuated AUV Based on Nonlinear Disturbance Observer and Adaptive Line-of-Sight Guidance","volume":"12","author":"He","year":"2024","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"B\u00f8rhaug, E., Pavlov, A., and Pettersen, K.Y. (2008, January 9\u201311). Integral LOS control for path following of underactuated marine surface vessels in the presence of constant ocean currents. Proceedings of the 47th IEEE Conference on Decision and Control, Cancun, Mexico.","DOI":"10.1109\/CDC.2008.4739352"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"380","DOI":"10.3182\/20120919-3-IT-2046.00065","article-title":"Relative velocity control and integral LOS for path following of underactuated surface vessels","volume":"45","author":"Caharija","year":"2012","journal-title":"IFAC Proc. Vol."},{"key":"ref_6","first-page":"37","article-title":"Control for 3D path-following of underactuated autonomous underwater vehicle under current disturbance","volume":"51","author":"Yao","year":"2019","journal-title":"J. Harbin Inst. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2887","DOI":"10.1109\/TCST.2023.3259819","article-title":"An adaptive line-of-sight (ALOS) guidance law for path following of aircraft and marine craft","volume":"31","author":"Fossen","year":"2023","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"106139","DOI":"10.1016\/j.oceaneng.2019.106139","article-title":"ELOS-based path following control for underactuated surface vehicles with actuator dynamics","volume":"187","author":"Yu","year":"2019","journal-title":"Ocean Eng."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wu, S.P., Ye, H., Liu, W., Yang, X.F., Liu, Z.Q., and Zhang, H. (2024). An Improved ELOS Guidance Law for Path Following of Underactuated Unmanned Surface Vehicles. Sensors, 24.","DOI":"10.3390\/s24165384"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"120425","DOI":"10.1016\/j.oceaneng.2025.120425","article-title":"AUV preset performance approximate dynamic programming path following control based on fixed time guidance law","volume":"321","author":"Chai","year":"2025","journal-title":"Ocean Eng."},{"key":"ref_11","unstructured":"Li, Z.Y., Liu, X., Wang, T., and Geng, B. (2025, January 25\u201328). Energy-optimized Dynamic Light-of-sight Guidance Method for AUV Path Following. Proceedings of the 16th IFAC Conference on Control Applications in Marine Systems, Robotics and Vehicles CAMS 2025, Wuhan, China."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"He, L., Zhang, Y., Liu, Y., Bai, C.R., and Li, L. (2024, January 18\u201320). Backstepping Sliding Mode Control for Path Following of Underactuated AUV Affected by Ocean Currents. Proceedings of the 2024 IEEE International Conference on Unmanned Systems, Nanjing, China.","DOI":"10.1109\/ICUS61736.2024.10839892"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"114729","DOI":"10.1016\/j.oceaneng.2023.114729","article-title":"Improved indirect adaptive line-of-sight guidance law for path following of under-actuated AUV subject to big ocean currents","volume":"281","author":"Du","year":"2023","journal-title":"Ocean Eng."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Namgung, H. (2021). Local route planning for collision avoidance of maritime autonomous surface ships in compliance with COLREGs rules. Sustainability, 14.","DOI":"10.3390\/su14010198"},{"key":"ref_15","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, The MIT Press Cambridge. [2nd ed.]. A Bradford Book."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1568","DOI":"10.1109\/TIV.2024.3429500","article-title":"AUV Collision Avoidance Strategy Based on Fuzzy Reinforcement Learning","volume":"10","author":"Cao","year":"2025","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"7823","DOI":"10.1109\/ACCESS.2021.3049238","article-title":"Collision risk inference system for maritime autonomous surface ships using COLREGs rules compliant collision avoidance","volume":"9","author":"Namgung","year":"2021","journal-title":"IEEE Access"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"10786","DOI":"10.1109\/TCE.2025.3614074","article-title":"AUV Path Planning and Object Tracking Algorithm Based on Reinforcement Learning Under Point Cloud Observation","volume":"71","author":"Xi","year":"2025","journal-title":"IEEE Trans. Consum. Electron."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"19039","DOI":"10.1109\/TITS.2025.3599137","article-title":"Cooperative Multi-AUV Data Collection Method Based on Staged Deep Reinforcement Learning with Distributed Negotiation","volume":"26","author":"Zhang","year":"2025","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"8886","DOI":"10.1109\/TVT.2025.3634986","article-title":"Global Feature Enhanced Reinforcement Learning Method for Multi-AUV Cooperative Control","volume":"75","author":"Chen","year":"2025","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"115021","DOI":"10.1016\/j.oceaneng.2023.115021","article-title":"AUV 3D Docking Control Using Deep Reinforcement Learning","volume":"283","author":"Zhang","year":"2023","journal-title":"Ocean Eng."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1017\/S0373463326101428","article-title":"Path planning and collision avoidance technologies for maritime autonomous surface ships: A review of COLREGs compliance, algorithmic trends and the navigation-GPT framework","volume":"78","author":"Namgung","year":"2026","journal-title":"J. Navig."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Li, X.M., Geng, L.B., Liu, K.Z., and Zhao, Y.F. (2025). Model-Based Offline Reinforcement Learning for AUV Path-Following Under Unknown Ocean Currents with Limited Data. Drones, 9.","DOI":"10.3390\/drones9030201"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"32591","DOI":"10.1007\/s11071-025-11851-4","article-title":"Reinforcement Learning-Based Optimal 3D Path-Following Control Incorporating State Estimation for AUV Under Sparse Sensor Attacks and Measurement Disturbances","volume":"113","author":"Qiu","year":"2025","journal-title":"Nonlinear Dyn."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"893","DOI":"10.1109\/TIV.2023.3282681","article-title":"Neural Network Model-Based Reinforcement Learning Control for AUV 3-D Path Following","volume":"9","author":"Ma","year":"2024","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Pan, X.W., Chen, T.H., Jiang, D., Fang, Z., and Li, G.L. (2024, January 28\u201331). Deep Reinforcement Learning with Model Predictive Control for Path Following of Autonomous Underwater Vehicle. Proceedings of the 43rd Chinese Control Conference, Kunming, China.","DOI":"10.23919\/CCC63176.2024.10661914"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Fossen, T.I. (2011). Handbook of Marine Craft Hydrodynamics and Motion Control, Wiley.","DOI":"10.1002\/9781119994138"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1904","DOI":"10.1109\/TCST.2024.3377876","article-title":"Path-Following Control of Unmanned Underwater Vehicle Based on an Improved TD3 Deep Reinforcement Learning","volume":"32","author":"Fan","year":"2024","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1623","DOI":"10.1109\/TCST.2015.2504838","article-title":"Integral Line-of-Sight Guidance and Control of Underactuated Marine Vehicles: Theory, Simulations, and Experiments","volume":"24","author":"Caharija","year":"2016","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_30","unstructured":"Akan, M. (2024). Path Following of Autonomous Underwater Vehicles in the Presence of Unknown Disturbances. [Master\u2019s Thesis, Istanbul Technical University]."},{"key":"ref_31","first-page":"1","article-title":"Development of collision-case-based testing scenarios for validating autonomous ship collision-avoidance algorithms","volume":"77","author":"Lee","year":"2026","journal-title":"Brodogr. Int. J. Nav. Archit. Ocean Eng. Res. Dev."},{"key":"ref_32","first-page":"1","article-title":"Path Following Control for Under-actuated Unmanned Surface Vehicles Based on Improved TD3","volume":"45","author":"Qu","year":"2024","journal-title":"J. Shanghai Marit. Univ."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A flexible new technique for camera calibration","volume":"22","author":"Zhang","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/14\/4548\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T04:25:53Z","timestamp":1784607953000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/26\/14\/4548"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,17]]},"references-count":33,"journal-issue":{"issue":"14","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["s26144548"],"URL":"https:\/\/doi.org\/10.3390\/s26144548","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,17]]}}}