{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T23:50:15Z","timestamp":1768002615798,"version":"3.49.0"},"reference-count":41,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2024,8,31]],"date-time":"2024-08-31T00:00:00Z","timestamp":1725062400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Natural Science Foundation of China","award":["52375025"],"award-info":[{"award-number":["52375025"]}]},{"name":"MDPI Robotics","award":["52375025"],"award-info":[{"award-number":["52375025"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Autonomous underwater vehicles (AUVs) have been increasingly applied in marine environmental monitoring. Their outstanding capability of performing tasks without human intervention makes them a popular tool for environmental data collection, especially in unknown and remote regions. This paper addresses the path planning problem when AUVs are used to perform plume source tracing in an unknown environment. The goal of path planning is to locate the plume source efficiently. The path planning approach is developed using the Double Deep Q-Network (DDQN) algorithm in the deep reinforcement learning (DRL) framework. The AUV gains knowledge by interacting with the environment, and the optimal direction is extracted from the mapping obtained by a deep neural network. The proposed approach was tested by numerical simulation and on a real ground vehicle. In the numerical simulation, several initial sampling strategies were compared on the basis of survey efficiency. The results show that direct learning based on the interaction with the environment could be an appropriate survey strategy for plume source tracing problems. The comparison with the canonical lawnmower path used in practice showed that path planning using DRL algorithms could be potentially promising for large-scale environment exploration.<\/jats:p>","DOI":"10.3390\/robotics13090132","type":"journal-article","created":{"date-parts":[[2024,9,2]],"date-time":"2024-09-02T03:45:21Z","timestamp":1725248721000},"page":"132","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Adaptive Path Planning for Subsurface Plume Tracing with an Autonomous Underwater Vehicle"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9399-2381","authenticated-orcid":false,"given":"Zhiliang","family":"Wu","sequence":"first","affiliation":[{"name":"School of Mechanical Engineering, Tianjin University, Tianjin 300354, China"}]},{"given":"Shuozi","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Tianjin University, Tianjin 300354, China"}]},{"given":"Xusong","family":"Shao","sequence":"additional","affiliation":[{"name":"School of Mechanical Engineering, Tianjin University, Tianjin 300354, China"}]},{"given":"Fang","family":"Liu","sequence":"additional","affiliation":[{"name":"Yichang Research Institute of Testing Technology, Yichang 443003, China"}]},{"given":"Zefeng","family":"Bao","sequence":"additional","affiliation":[{"name":"China Automotive Technology and Research Center, Tianjin 300300, China"}]}],"member":"1968","published-online":{"date-parts":[[2024,8,31]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"28","DOI":"10.1016\/j.marpolbul.2016.06.027","article-title":"Environmental effects of the Deepwater Horizon oil spill: A review","volume":"110","author":"Beyer","year":"2016","journal-title":"Mar. Pollut. Bull."},{"key":"ref_2","unstructured":"Westerholm, D.A., and Rauch, S.D. (2016). Deepwater Horizon Oil Spill: Final Programmatic Damage Assessment and Restoration Plan and Final Programmatic Environmental Impact Statement. Deep. Horiz. Nat. Resour. Damage Assess. Trustees."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"484","DOI":"10.1002\/rob.20399","article-title":"A peak-capture algorithm used on an autonomous underwater vehicle in the 2010 Gulf of Mexico oil spill response scientific survey","volume":"28","author":"Zhang","year":"2011","journal-title":"J. Field Robot."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1298","DOI":"10.1021\/es103838p","article-title":"Fate of Dispersants Associated with the Deepwater Horizon Oil Spill","volume":"45","author":"Kujawinski","year":"2011","journal-title":"Environ. Sci. Technol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"7759","DOI":"10.1002\/2016JC012113","article-title":"Measurement and modeling of oil slick transport","volume":"121","author":"Jones","year":"2016","journal-title":"J. Geophys. Res. Ocean."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hwang, J., Bose, N., Nguyen, H.D., and Williams, G. (2020). Acoustic Search and Detection of Oil Plumes Using an Autonomous Underwater Vehicle. J. Mar. Sci. Eng., 8.","DOI":"10.3390\/jmse8080618"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1126\/science.1195223","article-title":"Tracking Hydrocarbon Plume Transport and Biodegradation at Deepwater Horizon","volume":"330","author":"Camilli","year":"2010","journal-title":"Science"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"451","DOI":"10.1016\/j.margeo.2014.03.012","article-title":"Autonomous Underwater Vehicles (AUVs): Their past, present and future contributions to the advancement of marine geoscience","volume":"352","author":"Wynn","year":"2014","journal-title":"Mar. Geol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"20229","DOI":"10.1073\/pnas.1101242108","article-title":"Composition and fate of gas and oil released to the water column during the Deepwater Horizon oil spill","volume":"109","author":"Reddy","year":"2012","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_10","unstructured":"DiPinto, L., Forth, H., Holmes, J., Kukulya, A., Conmy, R., and Garcia, O. (2019). Three-Dimensional Mapping of Dissolved Hydrocarbons and Oil Droplets Using a REMUS-600 Autonomous Underwater Vehicle. Report to Bureau of Safety and Environmental Enforcement, BSEE."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"292","DOI":"10.18178\/ijmerr.10.6.292-300","article-title":"Enhancement of AUV Autonomy Using Backseat Driver Control Architecture","volume":"10","author":"Hwang","year":"2021","journal-title":"Int. J. Mech. Eng. Robot. Res."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Gomez-Ibanez, D., Kukulya, A.L., Belani, A., Conmy, R.N., Sundaravadivelu, D., and DiPinto, L. (2022). Autonomous Water Sampler for Oil Spill Response. J. Mar. Sci. Eng., 10.","DOI":"10.3390\/jmse10040526"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1101\/SQB.1990.055.01.052","article-title":"Bacterial microprocessing","volume":"55","author":"Berg","year":"1990","journal-title":"Cold Spring Harb. Symp. Quant. Biol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"232","DOI":"10.3182\/20120919-3-IT-2046.00040","article-title":"Autonomous and Adaptive Underwater Plume Detection and Tracking with AUVs: Concepts, Methods, and Available Technology","volume":"45","author":"Petillo","year":"2016","journal-title":"IFAC Proc. Vol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1109\/JOE.2004.838066","article-title":"Chemical plume tracing via an autonomous underwater vehicle","volume":"30","author":"Farrell","year":"2005","journal-title":"IEEE J. Ocean. Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"292","DOI":"10.1109\/TRO.2006.870627","article-title":"Moth-inspired chemical plume tracing on an autonomous underwater vehicle","volume":"22","author":"Li","year":"2006","journal-title":"IEEE Trans. Robot."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1023\/A:1016275516949","article-title":"Integration of Flow and Chemical Sensing for Guidance of Autonomous Marine Robots in Turbulent Flows","volume":"2","author":"Grasso","year":"2002","journal-title":"Environ. Fluid Mech."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2515","DOI":"10.1109\/TNNLS.2018.2885374","article-title":"Plume Tracing via Model-Free Reinforcement Learning Method","volume":"30","author":"Hu","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"277","DOI":"10.1007\/s10514-006-7567-0","article-title":"Particle swarm-based olfactory guided search","volume":"20","author":"Marques","year":"2006","journal-title":"Auton Robot."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1016\/j.robot.2015.05.001","article-title":"Self-organized swarm robot for target search and trapping inspired by bacterial chemotaxis","volume":"72","author":"Yang","year":"2015","journal-title":"Robot. Auton. Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2302","DOI":"10.1109\/TCYB.2014.2306291","article-title":"Optimal Swarm Formation for Odor Plume Finding","volume":"44","author":"Marjovi","year":"2014","journal-title":"IEEE Trans. Cybern."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sampathkumar, A., Dugaev, D., Song, A., Hu, F., Peng, Z., and Zhang, F. (2022, January 14\u201316). Plume tracing simulations using multiple autonomous underwater vehicles. Proceedings of the 16th International Conference on Underwater Networks & Systems, Association for Computing Machinery, Boston, MA, USA.","DOI":"10.1145\/3567600.3568149"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1475","DOI":"10.1177\/0278364910377243","article-title":"Planning and implementing trajectories for autonomous underwater vehicles to track evolving ocean processes based on predictions from a regional ocean model","volume":"29","author":"Smith","year":"2010","journal-title":"Int. J. Robot. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"11889","DOI":"10.1109\/TITS.2023.3289173","article-title":"Multi-robot plume source localization by distributed quantum-inspired guidance with formation behavior","volume":"24","author":"Li","year":"2023","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"114596","DOI":"10.1016\/j.chaos.2024.114596","article-title":"Collective behavior of self-propelled particles with heterogeneity in both dynamics and delays","volume":"180","author":"Zhou","year":"2024","journal-title":"Chaos Solitons Fractals"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"101475","DOI":"10.1016\/j.swevo.2024.101475","article-title":"Automated design of action advising trigger conditions for multiagent reinforcement learning: A genetic programming-based approach","volume":"85","author":"Wang","year":"2024","journal-title":"Swarm Evol. Comput."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1068","DOI":"10.1109\/TSMCB.2006.874689","article-title":"Chemical Plume Source Localization","volume":"36","author":"Pang","year":"2006","journal-title":"IEEE Trans. Syst. Man Cybern. Part B (Cybern.)"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1729881419831874","DOI":"10.1177\/1729881419831874","article-title":"Underwater chemical plume tracing based on partially observable Markov decision process","volume":"16","author":"Yu","year":"2019","journal-title":"Int. J. Adv. Robot. Syst."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Marchant, R., and Ramos, F. (2012, January 7\u201312). Bayesian optimisation for Intelligent Environmental Monitoring. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6385653"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wang, L., and Pang, S. (2023). Autonomous underwater vehicle based chemical plume tracing via deep reinforcement learning methods. J. Mar. Sci. Eng., 11.","DOI":"10.3390\/jmse11020366"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Bao, Z., Li, Y., Shao, X., Wu, Z., and Li, Q. (2023, January 5\u201310). Adaptive path planning for plume detection with an underwater glider. Proceedings of the IFToMM World Congress on Mechanism and Machine Science, Tokyo, Japan.","DOI":"10.1007\/978-3-031-45770-8_78"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1109\/JOE.2023.3338694","article-title":"Adaptive Zigzag Mapping of a Patchy Field by a Long-Range Autonomous Underwater Vehicle","volume":"49","author":"Zhang","year":"2024","journal-title":"IEEE J. Ocean. Eng."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"110242","DOI":"10.1016\/j.oceaneng.2021.110242","article-title":"Informative path planning for anomaly detection in environment exploration and monitoring","volume":"243","author":"Blanchard","year":"2022","journal-title":"Ocean. Eng."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Bracco, A., Paris, C.B., Esbaugh, A.J., Frasier, K., Joye, S.B., Liu, G., Polzin, K.L., and Vaz, A.C. (2020). Transport, Fate and Impacts of the Deep Plume of Petroleum Hydrocarbons Formed During the Macondo Blowout. Front. Mar. Sci., 7.","DOI":"10.3389\/fmars.2020.542147"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-023-37841-x","article-title":"Widespread global disparities between modelled and observed mid-depth ocean currents","volume":"14","author":"Su","year":"2023","journal-title":"Nat. Commun."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"144029","DOI":"10.1016\/j.scitotenv.2020.144029","article-title":"A comprehensive review of plume source detection using unmanned vehicles for environmental sensing","volume":"762","author":"Lewis","year":"2021","journal-title":"Sci. Total Environ."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Holzbecher, E. (2012). Environmental Modeling Using MATLAB, Springer.","DOI":"10.1007\/978-3-642-22042-5"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1017\/S0373463307004183","article-title":"Chemical Plume Tracing and Odour Source Localisation by Autonomous Vehicles","volume":"60","author":"Naeem","year":"2007","journal-title":"J. Navig."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"13293","DOI":"10.1021\/es303197h","article-title":"Evolution of the Macondo Well Blowout: Simulating the Effects of the Circulation and Synthetic Dispersants on the Subsea Oil Transport","volume":"46","author":"Paris","year":"2012","journal-title":"Environ. Sci. Technol."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"108401","DOI":"10.1016\/j.engappai.2024.108401","article-title":"Entropy adjustment by interpolation for exploration in Proximal Policy Optimization (PPO)","volume":"113","author":"Boudlal","year":"2024","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1111\/2041-210X.12156","article-title":"Choosing between strategies for designing surveys: Autonomous underwater vehicles","volume":"5","author":"Foster","year":"2014","journal-title":"Methods Ecol. Evol."}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/13\/9\/132\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:46:24Z","timestamp":1760111184000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/13\/9\/132"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,31]]},"references-count":41,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2024,9]]}},"alternative-id":["robotics13090132"],"URL":"https:\/\/doi.org\/10.3390\/robotics13090132","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,31]]}}}