{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T15:42:41Z","timestamp":1774453361446,"version":"3.50.1"},"reference-count":39,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,4,25]],"date-time":"2023-04-25T00:00:00Z","timestamp":1682380800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the National Key R&amp;D Program of China","award":["2020YFB1710200"],"award-info":[{"award-number":["2020YFB1710200"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>The avoidance of collisions among ships requires addressing various factors such as perception, decision-making, and control. These factors pose many challenges for autonomous collision avoidance. Traditional collision avoidance methods have encountered significant difficulties when used in autonomous collision avoidance. They are challenged to cope with the changing environment and harsh motion constraints. In the actual navigation of ships, it is necessary to carry out decision-making and control under the constraints of ship manipulation and risk. From the implementation process perspective, it is a typical sequential anthropomorphic decision-making problem. In order to solve the sequential decision problem, this paper improves DQN by setting a priority for sample collection and adopting non-uniform sampling, and it is applied to realize the intelligent collision avoidance of ships. It also verifies the performance of the algorithm in the simulation environment.<\/jats:p>","DOI":"10.3390\/a16050220","type":"journal-article","created":{"date-parts":[[2023,4,26]],"date-time":"2023-04-26T01:16:25Z","timestamp":1682471785000},"page":"220","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Improved DQN for Dynamic Obstacle Avoidance and Ship Path Planning"],"prefix":"10.3390","volume":"16","author":[{"given":"Xiao","family":"Yang","sequence":"first","affiliation":[{"name":"School of Computer Science and Technology, Harbin Engineering University, Nantong Street, Harbin 150001, China"}]},{"given":"Qilong","family":"Han","sequence":"additional","affiliation":[{"name":"School of Computer Science and Technology, Harbin Engineering University, Nantong Street, Harbin 150001, China"}]}],"member":"1968","published-online":{"date-parts":[[2023,4,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"109380","DOI":"10.1016\/j.oceaneng.2021.109380","article-title":"Collision-avoidance navigation systems for Maritime Autonomous Surface Ships: A state of the art survey","volume":"235","author":"Zhang","year":"2021","journal-title":"Ocean Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"112378","DOI":"10.1016\/j.oceaneng.2022.112378","article-title":"A human-like collision avoidance method for autonomous ship with attention-based deep reinforcement learning","volume":"264","author":"Jiang","year":"2022","journal-title":"Ocean Eng."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1550","DOI":"10.1016\/j.neucom.2017.09.088","article-title":"Efficient multi-task allocation and path planning for unmanned surface vehicle in support of ocean operations","volume":"275","author":"Liu","year":"2018","journal-title":"Neurocomputing"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wang, Z., Xiang, X., Yang, J., and Yang, S. (2017, January 18\u201320). Composite Astar and B-spline algorithm for path planning of autonomous underwater vehicle. Proceedings of the 2017 IEEE 7th International Conference on Underwater System Technology: Theory and Applications (USYS), Kuala Lumpur, Malaysia.","DOI":"10.1109\/USYS.2017.8309463"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.ifacol.2015.10.253","article-title":"Priority task approach for usvs\u2019 path following missions with obstacle avoidance and speed regulation","volume":"48","author":"Zereik","year":"2015","journal-title":"IFAC-PapersOnLine"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, Z., Yu, J., Zhao, Z., Wang, X., and Chen, Y. (2023). A Path-Planning Method Considering Environmental Disturbance Based on VPF-RRT. Drones, 7.","DOI":"10.3390\/drones7020145"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Szymak, P., and Praczyk, T. (2012, January 27\u201330). Using neural-evolutionary-fuzzy algorithm for anti-collision system of Unmanned Surface Vehicle. Proceedings of the 2012 17th International Conference on Methods & Models in Automation & Robotics (MMAR), Miedzyzdroje, Poland.","DOI":"10.1109\/MMAR.2012.6347873"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1016\/j.neucom.2022.05.006","article-title":"A deep reinforcement learning based method for real-time path planning and dynamic obstacle avoidance","volume":"497","author":"Chen","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"616","DOI":"10.1007\/s11227-021-03878-2","article-title":"An improved DQN path planning algorithm","volume":"78","author":"Li","year":"2022","journal-title":"J. Supercomput."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.neucom.2021.09.071","article-title":"Path planning and dynamic collision avoidance algorithm under COLREGs via deep reinforcement learning","volume":"468","author":"Xu","year":"2022","journal-title":"Neurocomputing"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"112809","DOI":"10.1016\/j.oceaneng.2022.112809","article-title":"Global path planning algorithm based on double DQN for multi-tasks amphibious unmanned surface vehicle","volume":"266","author":"Xiaofei","year":"2022","journal-title":"Ocean Eng."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Ru, J., Yu, H., Liu, H., Liu, J., Zhang, X., and Xu, H. (2022). A Bounded Near-Bottom Cruise Trajectory Planning Algorithm for Underwater Vehicles. J. Mar. Sci. Eng., 11.","DOI":"10.3390\/jmse11010007"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"100394","DOI":"10.1016\/j.iot.2021.100394","article-title":"A Tabu list strategy based DQN for AAV mobility in indoor single-path environment: Implementation and performance evaluation","volume":"14","author":"Saito","year":"2021","journal-title":"Internet Things"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1049\/trit.2020.0024","article-title":"Multi-robot path planning based on a deep reinforcement learning DQN algorithm","volume":"5","author":"Yang","year":"2020","journal-title":"CAAI Trans. Intell. Technol."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yi, C., and Qi, M. (2020, January 28\u201329). Research on virtual path planning based on improved DQN. Proceedings of the 2020 IEEE International Conference on Real-time Computing and Robotics (RCAR), Asahikawa, Japan.","DOI":"10.1109\/RCAR49640.2020.9303290"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"18433","DOI":"10.1109\/TITS.2022.3151826","article-title":"Fullest COLREGs evaluation using fuzzy logic for collaborative decision-making analysis of autonomous ships in complex situations","volume":"23","author":"Bakdi","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"108709","DOI":"10.1016\/j.oceaneng.2021.108709","article-title":"The hybrid path planning algorithm based on improved A* and artificial potential field for unmanned surface vehicle formations","volume":"223","author":"Sang","year":"2021","journal-title":"Ocean Eng."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"588","DOI":"10.1017\/S0373463318000796","article-title":"COLREGS-constrained real-time path planning for autonomous ships using modified artificial potential fields","volume":"72","author":"Lyu","year":"2019","journal-title":"J. Navig."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"102995","DOI":"10.1016\/j.apor.2021.102995","article-title":"Dynamic anti-collision A-star algorithm for multi-ship encounter situations","volume":"118","author":"He","year":"2022","journal-title":"Appl. Ocean Res."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhou, S., Wu, Z., and Ren, L. (2022). Ship Path Planning Based on Buoy Offset Historical Trajectory Data. J. Mar. Sci. Eng., 10.","DOI":"10.3390\/jmse10050674"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Bi, J., Gao, M., Zhang, W., Zhang, X., Bao, K., and Xin, Q. (2022). Research on Navigation Safety Evaluation of Coastal Waters Based on Dynamic Irregular Grid. J. Mar. Sci. Eng., 10.","DOI":"10.3390\/jmse10060733"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Yang, B., Yan, J., Cai, Z., Ding, Z., Li, D., Cao, Y., and Guo, L. (2021). A novel heuristic emergency path planning method based on vector grid map. ISPRS Int. J. Geo-Inf., 10.","DOI":"10.3390\/ijgi10060370"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"106495","DOI":"10.1016\/j.oceaneng.2019.106495","article-title":"Identifying factors influencing total-loss marine accidents in the world: Analysis and evaluation based on ship types and sea regions","volume":"191","author":"Chen","year":"2019","journal-title":"Ocean Eng."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1007\/s00773-015-0367-5","article-title":"The influence of route choice and operating conditions on fuel consumption and CO2 emission of ships","volume":"21","author":"Vettor","year":"2016","journal-title":"J. Mar. Sci. Technol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1175\/JTECH-D-11-00212.1","article-title":"Ship-mounted real-time surface observational system on board Indian vessels for validation and refinement of model forcing fields","volume":"30","author":"Harikumar","year":"2013","journal-title":"J. Atmos. Ocean. Technol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"111148","DOI":"10.1016\/j.oceaneng.2022.111148","article-title":"Formation control of a multi-AUVs system based on virtual structure and artificial potential field on SE (3)","volume":"253","author":"Zhen","year":"2022","journal-title":"Ocean Eng."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Shi, P., Liu, Z., and Liu, G. (2022, January 14\u201316). Local Path Planning of Unmanned Vehicles Based on Improved RRT Algorithm. Proceedings of the 2022 4th Asia Pacific Information Technology Conference, Bangkok, Thailand.","DOI":"10.1145\/3512353.3512387"},{"key":"ref_28","unstructured":"Rudin, N., Hoeller, D., Reist, P., and Hutter, M. (2022, January 14\u201318). Learning to walk in minutes using massively parallel deep reinforcement learning. Proceedings of the Conference on Robot Learning, PMLR, Auckland, NZ, USA."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"107660","DOI":"10.1016\/j.knosys.2021.107660","article-title":"A game-based deep reinforcement learning approach for energy-efficient computation in MEC systems","volume":"235","author":"Chen","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"401","DOI":"10.1016\/j.ejor.2021.07.016","article-title":"Deep reinforcement learning for inventory control: A roadmap","volume":"298","author":"Boute","year":"2022","journal-title":"Eur. J. Oper. Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"106487","DOI":"10.1016\/j.compbiomed.2022.106487","article-title":"Painless and accurate medical image analysis using deep reinforcement learning with task-oriented homogenized automatic pre-processing","volume":"153","author":"Yuan","year":"2023","journal-title":"Comput. Biol. Med."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1084763","DOI":"10.3389\/fmars.2022.1084763","article-title":"Collision avoidance for autonomous ship using deep reinforcement learning and prior-knowledge-based approximate representation","volume":"9","author":"Wang","year":"2023","journal-title":"Front. Mar. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Zhai, P., Zhang, Y., and Shaobo, W. (2022). Intelligent Ship Collision Avoidance Algorithm Based on DDQN with Prioritized Experience Replay under COLREGs. J. Mar. Sci. Eng., 10.","DOI":"10.3390\/jmse10050585"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Goswami, I., Das, P.K., Konar, A., and Janarthanan, R. (2010, January 1\u20134). Extended Q-learning algorithm for path-planning of a mobile robot. Proceedings of the Simulated Evolution and Learning: 8th International Conference, SEAL 2010, Kanpur, India.","DOI":"10.1007\/978-3-642-17298-4_40"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1141","DOI":"10.1109\/TSMCA.2012.2227719","article-title":"A deterministic improved Q-learning for path planning of a mobile robot","volume":"43","author":"Konar","year":"2013","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/j.cirpj.2022.11.003","article-title":"Deep reinforcement learning in smart manufacturing: A review and prospects","volume":"40","author":"Li","year":"2023","journal-title":"CIRP J. Manuf. Sci. Technol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"109902","DOI":"10.1016\/j.asoc.2022.109902","article-title":"Search and tracking strategy of autonomous surface underwater vehicle in oceanic eddies based on deep reinforcement learning","volume":"132","author":"Song","year":"2023","journal-title":"Appl. Soft Comput."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"210","DOI":"10.1080\/00207543.2021.1998695","article-title":"Anti-conflict AGV path planning in automated container terminals based on multi-agent reinforcement learning","volume":"61","author":"Hu","year":"2023","journal-title":"Int. J. Prod. Res."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Guo, S., Zhang, X., Du, Y., Zheng, Y., and Cao, Z. (2021). Path planning of coastal ships based on optimized DQN reward function. J. Mar. Sci. Eng., 9.","DOI":"10.3390\/jmse9020210"}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/5\/220\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:23:13Z","timestamp":1760124193000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/16\/5\/220"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,25]]},"references-count":39,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["a16050220"],"URL":"https:\/\/doi.org\/10.3390\/a16050220","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,25]]}}}