{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T15:49:49Z","timestamp":1767973789113,"version":"3.49.0"},"reference-count":32,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2022,12,7]],"date-time":"2022-12-07T00:00:00Z","timestamp":1670371200000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002994","name":"Ministry of Knowledge Economy","doi-asserted-by":"publisher","award":["20213030020280"],"award-info":[{"award-number":["20213030020280"]}],"id":[{"id":"10.13039\/501100002994","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003052","name":"MOTIE","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100003052","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,1,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Offshore operation causes the dynamic motion of offshore cranes and payload by the ocean environment. The motion of the payload lowers the safety and efficiency of the work, which may increase the working time or cause accidents. Therefore, we design a control method for the crane using artificial intelligence to minimize the heave motion of the payload. Herein, reinforcement learning (RL), which calculates actions according to states, is applied. Furthermore, the deep deterministic policy gradient (DDPG) algorithm is used because the actions need to be determined in a continuous state. In the DDPG algorithm, the state is defined as the motion of the crane and speed of the wire rope, and the action is defined as the speed of the wire rope. In addition, the reward is calculated using the motion of the payload. In this study, the heave motion of the payload was reduced by developing an agent suitable for adjusting the length of the wire rope. The heave motion of the payload was compared in between the non-learning condition of the RL-based control and proportional integral differential (PID) control; and an average payload reduction rate of 30% was observed under RL-based control. The RL-based control performed better than the PID control under learned conditions.<\/jats:p>","DOI":"10.1093\/jcde\/qwac122","type":"journal-article","created":{"date-parts":[[2022,12,4]],"date-time":"2022-12-04T15:13:51Z","timestamp":1670166831000},"page":"414-424","source":"Crossref","is-referenced-by-count":9,"title":["Heave reduction of payload through crane control based on deep reinforcement learning using dual offshore cranes"],"prefix":"10.1093","volume":"10","author":[{"given":"Jun-Hyeok","family":"Bae","sequence":"first","affiliation":[{"name":"Department of Wind\u2022Ocean Power R&D, Green Energy Institute , Mokpo-city, Jeollanam-do, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ju-Hwan","family":"Cha","sequence":"additional","affiliation":[{"name":"Department of Naval Architecture and Ocean Engineering, Mokpo National University , Muan-gun, Jeollanam-do, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sol","family":"Ha","sequence":"additional","affiliation":[{"name":"Department of Naval Architecture and Ocean Engineering, Mokpo National University , Muan-gun, Jeollanam-do, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2022,12,6]]},"reference":[{"key":"2023022210395480000_","first-page":"2121","article-title":"Reinforcement learning control of a forestry crane manipulator","volume-title":"Proceedings of the 2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Andersson","year":"2021"},{"issue":"3","key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"344","DOI":"10.3390\/jmse9030344","article-title":"Experimental study on development of mooring simulator for multi floating cranes","volume":"9","author":"Bae","year":"2021","journal-title":"Journal of Marine Science and Engineering"},{"issue":"3","key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/j.cad.2011.06.010","article-title":"Development of a simulation framework and applications to new production processes in shipyards","volume":"44","author":"Cha","year":"2012","journal-title":"Computer-Aided Design"},{"issue":"1","key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1093\/jcde\/qwab068","article-title":"Minimize makespan of permutation flowshop using pointer network","volume":"9","author":"Cho","year":"2022","journal-title":"Journal of Computational Design and Engineering"},{"key":"2023022210395480000_","first-page":"1444","article-title":"Incorporation of ship motion prediction into active heave compensation for offshore crane operation","volume-title":"Proceedings of the 2020 15th IEEE Conference on Industrial Electronics and Applications","author":"Chu","year":"2020"},{"issue":"4","key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"1430","DOI":"10.1093\/jcde\/qwac063","article-title":"Automation of crane control for block lifting based on deep reinforcement learning","volume":"9","author":"Chun","year":"2022","journal-title":"Journal of Computational Design and Engineering"},{"key":"2023022210395480000_","article-title":"Deep reinforcement learning and transportation research: A comprehensive review","author":"Farazi","year":"2020"},{"issue":"19","key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"5493","DOI":"10.3390\/s20195493","article-title":"Deep reinforcement learning for indoor mobile robot path planning","volume":"20","author":"Gao","year":"2020","journal-title":"Sensors"},{"key":"2023022210395480000_","first-page":"3389","article-title":"Deep reinforcement learning for robotic manipulation with asynchronous off-policy updates","volume-title":"Proceedings of the IEEE international conference on robotics and automation","author":"Gu","year":"2017"},{"key":"2023022210395480000_","doi-asserted-by":"crossref","DOI":"10.1007\/1-84628-148-2","volume-title":"PID control","author":"Johnson","year":"2005"},{"issue":"8","key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"2712","DOI":"10.1007\/s12555-022-0025-8","article-title":"Data-driven modeling and adaptive predictive anti-swing control of overhead cranes","volume":"20","author":"Kim","year":"2022","journal-title":"International Journal of Control, Automation and Systems"},{"key":"2023022210395480000_","first-page":"1098","article-title":"ADRC-ESMPC active heave compensation control strategy for offshore cranes","volume-title":"Ships and Offshore Structures","author":"Li","year":"2020"},{"key":"2023022210395480000_","article-title":"Continuous control with deep reinforcement learning","author":"Lillicrap","year":"2015"},{"issue":"4","key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"1582","DOI":"10.1002\/asjc.1707","article-title":"Control for the new harsh sea conditions salvage crane based on modified fuzzy PID","volume":"20","author":"Liu","year":"2018","journal-title":"Asian Journal of Control"},{"key":"2023022210395480000_","first-page":"380","article-title":"Deep reinforcement learning for dynamic treatment regimes on medical registry data","volume-title":"Proceedings of the IEEE International Conference on Healthcare Informatics","author":"Liu","year":"2017"},{"key":"2023022210395480000_","article-title":"Impact control for offshore crane in load-landing operations using reinforcement learning","author":"Maamoun","year":"2022"},{"key":"2023022210395480000_","first-page":"1640","article-title":"Comprehensive review of deep reinforcement learning methods and applications in economics","volume-title":"Mathematics","author":"Mosavi","year":"2020"},{"key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.conengprac.2017.06.005","article-title":"Experimental validation of an active heave compensation system: Estimation, prediction and control","volume":"66","author":"Richter","year":"2017","journal-title":"Control Engineering Practice"},{"issue":"1","key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s43684-022-00040-4","article-title":"Nonlinear optimal control for the 4-DOF underactuated robotic tower crane","volume":"2","author":"Rigatos","year":"2022","journal-title":"Autonomous Intelligent Systems"},{"key":"2023022210395480000_","article-title":"Experimental and numerical analysis of installation process using dual floating crane vessel","volume-title":"Proceedings of the Thirteenth ISOPE Pacific\/Asia Offshore Mechanics Symposium","author":"Seo","year":"2018"},{"key":"2023022210395480000_","volume-title":"Computational dynamics","author":"Shabana","year":"1994"},{"key":"2023022210395480000_","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511610523","volume-title":"Dynamics of multibody systems","author":"Shabana","year":"2005","edition":"3rd ed."},{"key":"2023022210395480000_","first-page":"350","article-title":"Literature survey of statistical, deep and reinforce-ment learning in natural language processing","volume-title":"Proceedings of the International Conference on Computing, Communication and Automation","author":"Sharma","year":"2017"},{"key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"104769","DOI":"10.1016\/j.engappai.2022.104769","article-title":"Wind turbine pitch reinforcement learning control improved by PID regulator and learning observer","volume":"111","author":"Sierra-Garcia","year":"2022","journal-title":"Engineering Applications of Artificial Intelligence"},{"issue":"5","key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"823","DOI":"10.1103\/PhysRev.36.823","article-title":"On the theory of the Brownian motion","volume":"36","author":"Uhlenbeck","year":"1930","journal-title":"Physical Review"},{"key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1016\/j.oceaneng.2015.05.004","article-title":"A review of vertical motion heave compensation systems","volume":"104","author":"Woodacre","year":"2015","journal-title":"Ocean Engineering"},{"key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1109\/WRC-SARA.2018.8584227","article-title":"Motion control for biped robot via DDPG-based deep reinforcement learning","volume-title":"Proceedings of the 2018 WRC Symposium on Advanced Robotics and Automation","author":"Wu","year":"2018"},{"key":"2023022210395480000_","doi-asserted-by":"crossref","DOI":"10.1109\/TNNLS.2021.3115960","article-title":"Neuroadaptive control for complicated underactuated systems with simultaneous output and velocity constraints exerted on both actuated and unactuated states","author":"Yang","year":"2021","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"2023022210395480000_","first-page":"2711","article-title":"Action-decision networks for visual tracking with deep reinforcement learning","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Yun","year":"2017"},{"key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"2267","DOI":"10.1080\/00207179.2021.1906447","article-title":"Anti-sway control of offshore crane on the surface vessel using global sliding mode control","volume":"95","author":"Zanjani","year":"2022","journal-title":"International Journal of Control"},{"issue":"4","key":"2023022210395480000_","first-page":"373","article-title":"A comparative study of different active heave compensation approaches","volume":"10","author":"Zinage","year":"2020","journal-title":"Ocean Systems Engineering"},{"issue":"16","key":"2023022210395480000_","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1016\/j.ifacol.2021.10.088","article-title":"Deep reinforcement learning based controller for active heave compensation","volume":"54","author":"Zinage","year":"2021","journal-title":"IFAC-PapersOnLine"}],"container-title":["Journal of Computational Design and Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jcde\/advance-article-pdf\/doi\/10.1093\/jcde\/qwac122\/47646736\/qwac122.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jcde\/article-pdf\/10\/1\/414\/49294125\/qwac122.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jcde\/article-pdf\/10\/1\/414\/49294125\/qwac122.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,3,15]],"date-time":"2023-03-15T20:16:35Z","timestamp":1678911395000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jcde\/article\/10\/1\/414\/6880157"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,6]]},"references-count":32,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2023,1,11]]}},"URL":"https:\/\/doi.org\/10.1093\/jcde\/qwac122","relation":{},"ISSN":["2288-5048"],"issn-type":[{"value":"2288-5048","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,2]]},"published":{"date-parts":[[2022,12,6]]}}}