{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T06:48:22Z","timestamp":1777704502967,"version":"3.51.4"},"reference-count":43,"publisher":"SAGE Publications","issue":"5","license":[{"start":{"date-parts":[[2020,7,29]],"date-time":"2020-07-29T00:00:00Z","timestamp":1595980800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Intelligent &amp; Fuzzy Systems: Applications in Engineering and Technology"],"published-print":{"date-parts":[[2020,11,19]]},"abstract":"<jats:p>Human experience is regarded as an indispensable part of artificial intelligence in the process of controlling or decision making for autonomous cargo ships. In this paper, a novel Deep Q-Network-based (DQN) approach is proposed, which performs satisfactorily in controlling a cargo ship automatically without any human experience. At the very beginning, we use the model of KRISO Very Large Crude Carrier (KVLCC2) to describe a cargo ship. To manipulate this ship has to conquer great inertia and relatively insufficient driving force. Subsequently, customary waterways, regulations, conventions are described with Artificial Potential Field and value-functions in DQN. Based on this, the artificial intelligence of planning and controlling a cargo ship can be obtained by undertaking sufficient training, which can control the ship directly, while avoiding collisions, keeping its position in the middle of the route as much as possible. In simulation experiments, it is demonstrated that such an approach performs better than manual works and other traditional methods in most conditions, which makes the proposed method a promising solution in improving the autonomy level of cargo ships.<\/jats:p>","DOI":"10.3233\/jifs-200754","type":"journal-article","created":{"date-parts":[[2020,7,31]],"date-time":"2020-07-31T12:55:51Z","timestamp":1596200151000},"page":"7363-7379","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":10,"title":["Controlling a cargo ship without human experience using deep Q-network"],"prefix":"10.1177","volume":"39","author":[{"given":"Chen","family":"Chen","sequence":"first","affiliation":[{"name":"Wuhan University of Technology","place":["PR China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Feng","family":"Ma","sequence":"additional","affiliation":[{"name":"Wuhan University of Technology","place":["PR China"]},{"name":"National Engineering Research Centre for Water Transport Safety","place":["PR China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jialun","family":"Liu","sequence":"additional","affiliation":[{"name":"Wuhan University of Technology","place":["PR China"]},{"name":"National Engineering Research Centre for Water Transport Safety","place":["PR China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rudy R.","family":"Negenborn","sequence":"additional","affiliation":[{"name":"National Engineering Research Centre for Water Transport Safety","place":["PR China"]},{"name":"Delft University of Technology","place":["the Netherlands"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuanchang","family":"Liu","sequence":"additional","affiliation":[{"name":"University College London","place":["UK"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xinping","family":"Yan","sequence":"additional","affiliation":[{"name":"Wuhan University of Technology","place":["PR China"]},{"name":"National Engineering Research Centre for Water Transport Safety","place":["PR China"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2020,7,29]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.rtbm.2017.06.002"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-013-9370-z"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature16961"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1243\/14750902JEME80"},{"key":"e_1_3_1_7_2","first-page":"1","article-title":"Vehicle and mission control of the DELFIM autonomous surface craft","author":"Alves J.","year":"2006","unstructured":"AlvesJ., OliveiraP., OliveiraR., PascoalA., RufinoM., SebastiaoL., SilvestreC., Vehicle and mission control of the DELFIM autonomous surface craft, in: 2006 14th Mediterranean Conference on Control and Automation, IEEE, (2006), 1\u20136 .","journal-title":"2006 14th Mediterranean Conference on Control and Automation, IEEE"},{"key":"e_1_3_1_8_2","doi-asserted-by":"crossref","unstructured":"FossenT.I. Handbook of marine craft hydrodynamics and motion control John Wiley & Sons 2011.","DOI":"10.1002\/9781119994138"},{"key":"e_1_3_1_9_2","first-page":"106","article-title":"Survey on short-term technology developments and readiness levels for autonomous shipping","author":"van Cappelle L.E.","year":"2018","unstructured":"van CappelleL.E., ChenL. and NegenbornR.R., Survey on short-term technology developments and readiness levels for autonomous shipping, in: Proceedings of the 9th International Conference on Computational Logistics (ICCL 2018), Vietri sul Mare, Italy, 2018, pp. 106\u2013123 .","journal-title":"Proceedings of the 9th International Conference on Computational Logistics (ICCL 2018)"},{"key":"e_1_3_1_10_2","first-page":"219","article-title":"Survey on autonomous surface vessels: Part I \u2013 A new detailed definition of autonomy levels","author":"Schiaretti M.","year":"2017","unstructured":"SchiarettiM., ChenL., NegenbornR.R., Survey on autonomous surface vessels: Part I \u2013 A new detailed definition of autonomy levels, in: Proceedings of the 8th International Conference on Computational Logistics (ICCL 2017), Southampton, UK, 2017, pp. 219\u2013233.","journal-title":"Proceedings of the 8th International Conference on Computational Logistics (ICCL 2017)"},{"key":"e_1_3_1_11_2","first-page":"234","article-title":"Survey on autonomous surface vessels: Part II \u2013 Categorization of 60 prototypes and future applications","author":"Schiaretti M.","year":"2017","unstructured":"SchiarettiM., ChenL., NegenbornR.R., Survey on autonomous surface vessels: Part II \u2013 Categorization of 60 prototypes and future applications, in: Proceedings of the 8th International Conference on Computational Logistics (ICCL 2017), Southampton, UK, 2017, pp. 234\u2013252 .","journal-title":"Proceedings of the 8th International Conference on Computational Logistics (ICCL 2017)"},{"key":"e_1_3_1_12_2","first-page":"65","article-title":"Path Planning for Autonomous Inland Vessels Using A*BG","author":"Chen L.","year":"2016","unstructured":"ChenL., NegenbornR.R., LodewijksG., Path Planning for Autonomous Inland Vessels Using A*BG, in: Proceedings of the 7th International Conference on Computational Logistics (ICCL 2016), Lisbon, Portugal, 2016, pp. 65\u201379 .","journal-title":"Proceedings of the 7th International Conference on Computational Logistics (ICCL 2016)"},{"key":"e_1_3_1_13_2","first-page":"1","article-title":"A three-layered architecture for real time path planning and obstacle avoidance for surveillance USVs operating in harbour fields","author":"Casalino G.","year":"2009","unstructured":"CasalinoG., TurettaA., SimettiE., A three-layered architecture for real time path planning and obstacle avoidance for surveillance USVs operating in harbour fields, in: Oceans 2009-Euroe, IEEE, 2009, pp. 1\u20138 .","journal-title":"Oceans 2009-Euroe"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.3182\/20120919-3-IT-2046.00066"},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-62338-2_6"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.12716\/1001.11.01.09"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/MMAR.2018.8486119"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.arcontrol.2012.09.008"},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ifacol.2015.10.253"},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/MED.2016.7535964"},{"key":"e_1_3_1_21_2","first-page":"1","article-title":"A practical path planning and navigation algorithm for an unmanned surface vehicle using the fast marching algorithm","author":"Liu Y.","year":"2015","unstructured":"LiuY., SongR., BucknallR., A practical path planning and navigation algorithm for an unmanned surface vehicle using the fast marching algorithm, in: OCEANS 2015-Genova, IEEE, 2015, 1\u20137 .","journal-title":"OCEANS 2015-Genova"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.trc.2015.11.004"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.2478\/pomr-2018-0049"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00773-017-0486-2"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.3182\/20130626-3-AU-2035.00010"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2017.12.009"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00773-015-0355-9"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2019.106299"},{"key":"e_1_3_1_29_2","article-title":"Playing Atari with deep reinforcement learning","author":"Mnih V.","year":"2013","unstructured":"MnihV., KavukcuogluK., SilverD., GravesA., AntonoglouI., WierstraD., RiedmillerM., Playing Atari with deep reinforcement learning, in: Neural Informationrocessing Systems Deep Learning Workshop, 2013.","journal-title":"Neural Informationrocessing Systems Deep Learning Workshop"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793742"},{"key":"e_1_3_1_31_2","first-page":"2419","article-title":"Learning to navigate in cities without a map","author":"Mirowski P.","year":"2018","unstructured":"MirowskiP., GrimesM., MalinowskiM., HermannK.M., AndersonK., TeplyashinD., SimonyanK., ZissermanA., HadsellR., Learning to navigate in cities without a map, in: Advances in Neural Informationrocessing Systems, 2018, pp. 2419\u20132430. ;.","journal-title":"Advances in Neural Informationrocessing Systems"},{"key":"e_1_3_1_32_2","unstructured":"BojarskiM. Del TestaD. DworakowskiD. FirnerB. FleppB. GoyalP. JackelL.D. MonfortM. MullerU. and ZhangJ. End to end learning for self-driving cars (2016). arXiv preprint arXiv: 604.07316."},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2017.06.066"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2016.07.064"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.oceaneng.2017.04.008"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.3233\/JIFS-169564"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2018.05.016"},{"key":"e_1_3_1_38_2","article-title":"Dueling Network Architectures for Deep Reinforcement Learning","author":"Wang Z.","year":"2016","unstructured":"WangZ., De FreitasN. and LanctotM., Dueling Network Architectures for Deep Reinforcement Learning, in: 33rd International Conference on Machine Learning (ICML), 2016.","journal-title":"33rd International Conference on Machine Learning (ICML)"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2016.2604240"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1017\/S0373463300022384"},{"key":"e_1_3_1_41_2","article-title":"Free Sailing Manoeuvring Tests on KVLCC 1 and KVLCC 2. Technical Report","author":"Lee S.W.","year":"2007","unstructured":"LeeS.W., ToxopeusS.L., QuadvliegF.H.H.A., Free Sailing Manoeuvring Tests on KVLCC 1 and KVLCC 2. Technical Report. Maritime Research Institute Netherlands, 2007.","journal-title":"Maritime Research Institute Netherlands"},{"key":"e_1_3_1_42_2","unstructured":"Quadvlieg and BrouwerF.H.H.A. KVLCC2 Benchmark Data Including Uncertainty Analysis To Support Manoeuvring Predictions (2011)."},{"key":"e_1_3_1_43_2","unstructured":"BellemareM.G. VenessJ. BowlingM. Investigating Contingency Awareness Using Atari 2600 Games in: 2012."},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-018-0102-6"}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems: Applications in Engineering and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JIFS-200754","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.3233\/JIFS-200754","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JIFS-200754","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:41:22Z","timestamp":1777455682000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.3233\/JIFS-200754"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,7,29]]},"references-count":43,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2020,11,19]]}},"alternative-id":["10.3233\/JIFS-200754"],"URL":"https:\/\/doi.org\/10.3233\/jifs-200754","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,7,29]]}}}