{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,14]],"date-time":"2026-05-14T02:19:47Z","timestamp":1778725187257,"version":"3.51.4"},"reference-count":33,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,3,29]],"date-time":"2023-03-29T00:00:00Z","timestamp":1680048000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"ORN","award":["N00014-20-1-2085"],"award-info":[{"award-number":["N00014-20-1-2085"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This work presents a framework that allows Unmanned Surface Vehicles (USVs) to avoid dynamic obstacles through initial training on an Unmanned Ground Vehicle (UGV) and cross-domain retraining on a USV. This is achieved by integrating a Deep Reinforcement Learning (DRL) agent that generates high-level control commands and leveraging a neural network based model predictive controller (NN-MPC) to reach target waypoints and reject disturbances. A Deep Q Network (DQN) utilized in this framework is trained in a ground environment using a Turtlebot robot and retrained in a water environment using the BREAM USV in the Gazebo simulator to avoid dynamic obstacles. The network is then validated in both simulation and real-world tests. The cross-domain learning largely decreases the training time (28%) and increases the obstacle avoidance performance (70 more reward points) compared to pure water domain training. This methodology shows that it is possible to leverage the data-rich and accessible ground environments to train DRL agent in data-poor and difficult-to-access marine environments. This will allow rapid and iterative agent development without further training due to the change in environment or vehicle dynamics.<\/jats:p>","DOI":"10.3390\/s23073572","type":"journal-article","created":{"date-parts":[[2023,3,30]],"date-time":"2023-03-30T01:31:30Z","timestamp":1680139890000},"page":"3572","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Dynamic Obstacle Avoidance for USVs Using Cross-Domain Deep Reinforcement Learning and Neural Network Model Predictive Controller"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3519-6405","authenticated-orcid":false,"given":"Jianwen","family":"Li","sequence":"first","affiliation":[{"name":"The School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2032-2296","authenticated-orcid":false,"given":"Jalil","family":"Chavez-Galaviz","sequence":"additional","affiliation":[{"name":"The School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8507-1868","authenticated-orcid":false,"given":"Kamyar","family":"Azizzadenesheli","sequence":"additional","affiliation":[{"name":"Nvidia Corporation, Santa Clara, CA 95051, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3285-8234","authenticated-orcid":false,"given":"Nina","family":"Mahmoudian","sequence":"additional","affiliation":[{"name":"The School of Mechanical Engineering, Purdue University, West Lafayette, IN 47907, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,29]]},"reference":[{"key":"ref_1","unstructured":"Cockcroft, A.N., and Lameijer, J.N.F. (2003). Guide to the Collision Avoidance Rules, Elsevier."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1292","DOI":"10.1007\/s00773-020-00787-6","article-title":"Path planning and collision avoidance for autonomous surface vehicles I: A review","volume":"26","author":"Vagale","year":"2021","journal-title":"J. Mar. Sci. Technol."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Zhang, H.Y., Lin, W.M., and Chen, A.X. (2018). Path planning for the mobile robot: A review. Symmetry, 10.","DOI":"10.3390\/sym10100450"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1002\/rob.20109","article-title":"Using interpolation to improve path planning: The Field D* algorithm","volume":"23","author":"Ferguson","year":"2006","journal-title":"J. Field Robot."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Sampedro, C., Bavle, H., Rodriguez-Ramos, A., de La Puente, P., and Campoy, P. (2018, January 1\u20135). Laser-based reactive navigation for multirotor aerial robots using deep reinforcement learning. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593706"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"987","DOI":"10.1002\/rob.21935","article-title":"Autonomous collision detection and avoidance for ARAGON USV: Development and field tests","volume":"37","author":"Han","year":"2020","journal-title":"J. Field Robot."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1224","DOI":"10.1080\/00423114.2018.1563706","article-title":"Optimal control of brakes and steering for autonomous collision avoidance using modified Hamiltonian algorithm","volume":"57","author":"Gao","year":"2019","journal-title":"Veh. Syst. Dyn."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"6001","DOI":"10.1109\/LRA.2020.3010730","article-title":"Nonlinear MPC for Collision Avoidance and Control of UAVs With Dynamic Obstacles","volume":"5","author":"Lindqvist","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Hagen, I.B., Kufoalor, D.K.M., Brekke, E.F., and Johansen, T.A. (2018, January 21\u201325). MPC-based Collision Avoidance Strategy for Existing Marine Vessel Guidance Systems. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8463182"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"177","DOI":"10.4173\/mic.2019.3.4","article-title":"Short-term ASV Collision Avoidance with Static and Moving Obstacles","volume":"40","author":"Eriksen","year":"2019","journal-title":"Model. Identif. Control. A Nor. Res. Bull."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"109380","DOI":"10.1016\/j.oceaneng.2021.109380","article-title":"Collision-avoidance navigation systems for Maritime Autonomous Surface Ships: A state of the art survey","volume":"235","author":"Zhang","year":"2021","journal-title":"Ocean Eng."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Choi, J., Park, K., Kim, M., and Seok, S. (2019, January 20\u201324). Deep reinforcement learning of navigation in a complex and crowded environment with a limited field of view. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8793979"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"165344","DOI":"10.1109\/ACCESS.2020.3022600","article-title":"COLREG-compliant collision avoidance for unmanned surface vehicle using deep reinforcement learning","volume":"8","author":"Meyer","year":"2020","journal-title":"IEEE Access"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1503","DOI":"10.1109\/LRA.2021.3058927","article-title":"Toward agile maneuvers in highly constrained spaces: Learning from hallucination","volume":"6","author":"Xiao","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Joshi, G., and Chowdhary, G. (2018, January 21\u201325). Cross-domain transfer in reinforcement learning using target apprentice. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8462977"},{"key":"ref_17","unstructured":"Shukla, Y., Thierauf, C., Hosseini, R., Tatiya, G., and Sinapov, J. (2022). ACuTE: Automatic Curriculum Transfer from Simple to Complex Environments. arXiv."},{"key":"ref_18","unstructured":"Gage, D.W. (1995). UGV History 101: A Brief History of Unmanned Ground Vehicle (UGV) Development Efforts, NAVAL COMMAND CONTROL AND OCEAN SURVEILLANCE CENTER RDT AND E DIV SAN DIEGO CA. Technical report."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.arcontrol.2016.04.018","article-title":"Unmanned surface vehicles: An overview of developments and challenges","volume":"41","author":"Liu","year":"2016","journal-title":"Annu. Rev. Control"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Fossen, T.I. (2011). Handbook of Marine Craft Hydrodynamics and Motion Control, John Wiley & Sons.","DOI":"10.1002\/9781119994138"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"739023","DOI":"10.3389\/frobt.2021.739023","article-title":"Robust ASV Navigation Through Ground to Water Cross-Domain Deep Reinforcement Learning","volume":"8","author":"Lambert","year":"2021","journal-title":"Front. Robot. AI"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"820","DOI":"10.1109\/TCST.2014.2338354","article-title":"Line-of-Sight Path Following for Dubins Paths With Adaptive Sideslip Compensation of Drift Forces","volume":"23","author":"Fossen","year":"2015","journal-title":"IEEE Trans. Control Syst. Technol."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"497","DOI":"10.2307\/2372560","article-title":"On curves of minimal length with a constraint on average curvature, and with prescribed initial and terminal positions and tangents","volume":"79","author":"Dubins","year":"1957","journal-title":"Am. J. Math."},{"key":"ref_24","first-page":"1","article-title":"Integral line-of-sight guidance and control of underactuated marine vehicles","volume":"24","author":"Caharija","year":"2014","journal-title":"IEEE Trans. Control. Syst. Technol."},{"key":"ref_25","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","article-title":"Long Short-Term Memory","volume":"9","author":"Hochreiter","year":"1997","journal-title":"Neural Comput."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Lambert, R., Page, B., Chavez, J., and Mahmoudian, N. (2020, January 5\u201330). A Low-Cost Autonomous Surface Vehicle for Multi-Vehicle Operations. Proceedings of the Global Oceans 2020: Singapore\u2013US Gulf Coast, Biloxi, MI, USA.","DOI":"10.1109\/IEEECONF38699.2020.9389236"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.ejcon.2021.01.008","article-title":"Continuous-time system identification with neural networks: Model structures and fitting criteria","volume":"59","author":"Forgione","year":"2021","journal-title":"Eur. J. Control"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Berger, E., Wheeler, R., and Ng, A.Y. (2009, January 12\u201317). ROS: An open-source Robot Operating System. Proceedings of the ICRA Workshop on Open Source Software, Kobe, Japan.","DOI":"10.1109\/MRA.2010.936956"},{"key":"ref_30","unstructured":"Hokuyo Automatic Co., Ltd. (2023, March 09). Hokuyo UTM-30LX-EW Scanning Laser Rangefinder. Available online: https:\/\/acroname.com\/store\/scanning-laser-rangefinder-r354-utm-30lx-ew."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Bingham, B., Aguero, C., McCarrin, M., Klamo, J., Malia, J., Allen, K., Lum, T., Rawson, M., and Waqar, R. (2019, January 27\u201331). Toward Maritime Robotic Simulation in Gazebo. Proceedings of the MTS\/IEEE OCEANS Conference, Seattle, WA, USA.","DOI":"10.23919\/OCEANS40490.2019.8962724"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2840","DOI":"10.1109\/TAC.2020.3009362","article-title":"Adaptive stochastic MPC under time-varying uncertainty","volume":"66","author":"Bujarbaruah","year":"2020","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"816","DOI":"10.1002\/rnc.1251","article-title":"An online active set strategy to overcome the limitations of explicit MPC","volume":"18","author":"Ferreau","year":"2008","journal-title":"Int. J. Robust Nonlinear Control. IFAC-Affil. J."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3572\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:06:06Z","timestamp":1760123166000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3572"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,29]]},"references-count":33,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["s23073572"],"URL":"https:\/\/doi.org\/10.3390\/s23073572","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,3,29]]}}}