{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,25]],"date-time":"2026-01-25T03:49:53Z","timestamp":1769312993429,"version":"3.49.0"},"reference-count":38,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2022,7,23]],"date-time":"2022-07-23T00:00:00Z","timestamp":1658534400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002551","name":"Seoul National University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100002551","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100019211","name":"Institute of Engineering Research, Seoul National University","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100019211","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,8,27]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>In shipyards, blocks are controlled by connecting the crane and block with wires during block erection. During block lifting, if a block is not carefully controlled, it will cause damage. Block lifting using crane operation is performed by controlling the number of wires, hooks, and equalizers. Consequently, predicting stable block lifting is difficult. In this study, we proposed a control method to determine static equilibrium. Initially, an algorithm for finding the initial equilibrium state of the block (IES algorithm) was proposed, followed by deep reinforcement learning (DRL)-based method for block lifting. The position, orientation, angular velocity of the block, and hoisting speed of the wires were applied as the DRL state. The control input of the crane was calculated by deriving the hoisting speed of the wires. To verify the proposed method, comparative studies on the application of the IES algorithm were carried out, and further block movement was compared. Conclusively, the proposed method effectively increased block lifting safety.<\/jats:p>","DOI":"10.1093\/jcde\/qwac063","type":"journal-article","created":{"date-parts":[[2022,7,23]],"date-time":"2022-07-23T15:00:14Z","timestamp":1658588414000},"page":"1430-1448","source":"Crossref","is-referenced-by-count":8,"title":["Automation of crane control for block lifting based on deep reinforcement learning"],"prefix":"10.1093","volume":"9","author":[{"given":"Do-Hyun","family":"Chun","sequence":"first","affiliation":[{"name":"Department of Naval Architecture and Ocean Engineering, Seoul National University , Seoul 08826, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7972-6848","authenticated-orcid":false,"given":"Myung-Il","family":"Roh","sequence":"additional","affiliation":[{"name":"Department of the Naval Architecture and Ocean Engineering, and Research Institute of Marine Systems Engineering, Seoul National University , Seoul 08826, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hye-Won","family":"Lee","sequence":"additional","affiliation":[{"name":"Research Institute of Marine Systems Engineering, Seoul National University , Seoul 08826, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2022,7,23]]},"reference":[{"key":"2022082710080473300_bib1","doi-asserted-by":"crossref","first-page":"830","DOI":"10.15282\/ijame.7.2012.3.0068","article-title":"Tracking control of an underactuated gantry crane using an optimal feedback controller","volume":"7","author":"Bakhtiari-Nejad","year":"2013","journal-title":"International Journal of Automotive and Mechanical Engineering"},{"issue":"1","key":"2022082710080473300_bib2","first-page":"51","article-title":"Minimize makespan of permutation flowshop using pointer network","volume":"9","author":"Cho","year":"2022","journal-title":"Journal\u00a0of\u00a0Computational\u00a0Design and\u00a0Engineering"},{"key":"2022082710080473300_bib3","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1179\/0937725515Z.0000000004","article-title":"Integrated multi-domain system modelling and simulation for offshore crane operations","volume":"62","author":"Chu","year":"2015","journal-title":"Ship Technology Research"},{"key":"2022082710080473300_bib4","first-page":"297","article-title":"A study on the methods for finding initial equilibrium position of a lifting block for the safe erection","volume":"55","author":"Chun","year":"2018","journal-title":"Journal\u00a0of the\u00a0Society\u00a0of\u00a0Naval\u00a0Architects of\u00a0Korea"},{"key":"2022082710080473300_bib5","article-title":"A crane movement control for stability of block erection based on deep reinforcement learning","volume-title":"Proceedings of the International Federation of Automatic Control","author":"Chun","year":"2019"},{"key":"2022082710080473300_bib6","first-page":"2052","article-title":"Off-policy deep reinforcement learning without exploration","volume-title":"Proceedings of the 36th International Conference on Machine Learning","author":"Fujimoto","year":"2019"},{"key":"2022082710080473300_bib7","first-page":"1856","article-title":"Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Haarnoja","year":"2018"},{"key":"2022082710080473300_bib8","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.marstruc.2020.102841","article-title":"Time-domain structural analysis during block turnover and lifting using 2D flexible multibody dynamics","volume":"75","author":"Ham","year":"2021","journal-title":"Marine Structures"},{"key":"2022082710080473300_bib12","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1016\/j.oceaneng.2015.08.050","article-title":"Multibody dynamic analysis of a heavy load suspended by a floating crane with constraint-based wire rope","volume":"109","author":"Ham","year":"2015","journal-title":"Ocean Engineering"},{"key":"2022082710080473300_bib10","doi-asserted-by":"crossref","first-page":"98","DOI":"10.1016\/j.autcon.2015.10.007","article-title":"Simulation of load lifting with equalizers used in shipyards","volume":"61","author":"Ham","year":"2016","journal-title":"Automation in Construction"},{"key":"2022082710080473300_bib9","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1016\/j.advengsoft.2017.04.009","article-title":"Development and validation of a simulation-based safety evaluation program for a mega floating crane","volume":"112","author":"Ham","year":"2017","journal-title":"Advances in Engineering Software"},{"issue":"3","key":"2022082710080473300_bib13","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1016\/j.jcde.2017.12.005","article-title":"Integrated method of analysis, visualization, and hardware for ship motion simulation","volume":"5","author":"Ham","year":"2018","journal-title":"Journal of Computational Design and Engineering"},{"key":"2022082710080473300_bib11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.oceaneng.2019.106673","article-title":"Collision detection and response of multibody systems using a position difference method and non-interpenetration constraint in shipbuilding simulation","volume":"195","author":"Ham","year":"2020","journal-title":"Ocean Engineering"},{"key":"2022082710080473300_bib14","doi-asserted-by":"crossref","first-page":"1952","DOI":"10.1109\/TAC.2021.3071021","article-title":"Boundary feedback control of a nonhomogeneous wind turbine tower with exogenous disturbances","volume":"67","author":"Han","year":"2022","journal-title":"IEEE Transactions on Automatic Control"},{"key":"2022082710080473300_bib15","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1016\/j.automatica.2015.12.026","article-title":"Cooperative control of a nonuniform gantry crane with constrained tension","volume":"66","author":"He","year":"2016","journal-title":"Automatica"},{"key":"2022082710080473300_bib17","doi-asserted-by":"crossref","first-page":"1952","DOI":"10.1109\/TMECH.2017.2721553","article-title":"Boundary vibration control of variable length crane systems in two-dimensional space with output constraints","volume":"22","author":"He","year":"2017","journal-title":"IEEE\/ASME Transactions on Mechatronics"},{"key":"2022082710080473300_bib16","doi-asserted-by":"crossref","first-page":"104423","DOI":"10.1016\/j.conengprac.2020.104423","article-title":"Boundary vibration control of a floating wind turbine system with mooring lines","volume":"101","author":"He","year":"2020","journal-title":"Control Engineering Practice"},{"issue":"6","key":"2022082710080473300_bib18","first-page":"2231","article-title":"The effects of auto-tuned method in PID and PD control scheme for gantry crane system","volume":"4","author":"Hussien","year":"2015","journal-title":"International Journal of Soft Computing and Engineering"},{"key":"2022082710080473300_bib19","author":"ISO 19901-1","year":"2005"},{"issue":"5","key":"2022082710080473300_bib20","doi-asserted-by":"crossref","first-page":"657","DOI":"10.1093\/jcde\/qwaa049","article-title":"Machine tool architecture selection at the preliminary design stage: Application to hard material machining","volume":"7","author":"Lajili","year":"2020","journal-title":"Journal of Computational Design and Engineering"},{"key":"2022082710080473300_bib21","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.autcon.2016.03.013","article-title":"Block turnover simulation considering the interferences between the block and wire ropes in shipbuilding","volume":"67","author":"Lee","year":"2016","journal-title":"Automation in Construction"},{"key":"2022082710080473300_bib23","first-page":"237","article-title":"Controller design of a gantry crane for the safe erection of blocks in shipyards","volume-title":"Proceedings of 2019 Practical Design of Ships and Other Floating Structures (PRADS)","author":"Lee","year":"2019"},{"key":"2022082710080473300_bib24","doi-asserted-by":"crossref","first-page":"641","DOI":"10.1016\/j.ijnaoe.2021.08.001","article-title":"Prediction of ship power based on variation in deep feed-forward neural network","volume":"13","author":"Lee","year":"2021","journal-title":"International Journal of Naval Architecture and Ocean Engineering"},{"key":"2022082710080473300_bib22","doi-asserted-by":"crossref","first-page":"103573","DOI":"10.1016\/j.autcon.2021.103573","article-title":"Underactuated crane control for the automation of block erection in shipbuilding","volume":"124","author":"Lee","year":"2021","journal-title":"Automation in Construction"},{"key":"2022082710080473300_bib25","author":"Lew","year":"2003","journal-title":"Variational time integrators in computational solid mechanics"},{"key":"2022082710080473300_bib26","doi-asserted-by":"crossref","first-page":"597","DOI":"10.22201\/icat.16656423.2012.10.4.383","article-title":"Tracking control for an underactuated two-dimensional overhead crane","volume":"10","author":"Liu","year":"2012","journal-title":"Journal of Applied Research and Technology"},{"key":"2022082710080473300_bib27","doi-asserted-by":"crossref","first-page":"6252","DOI":"10.1109\/ICRA.2018.8461113","article-title":"Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning","volume-title":"Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA)","author":"Long","year":"2018"},{"key":"2022082710080473300_bib28","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1017\/S096249290100006X","article-title":"Discrete mechanics and variational integrators","volume":"10","author":"Marsden","year":"2001","journal-title":"Acta Numerica"},{"key":"2022082710080473300_bib29","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1016\/j.ijnaoe.2017.03.003","article-title":"Cooperative control system of the floating cranes for the dual lifting","volume":"10","author":"Nam","year":"2018","journal-title":"International Journal of Naval Architecture and Ocean Engineering"},{"key":"2022082710080473300_bib30","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1109\/TMECH.2010.2093907","article-title":"Sliding-mode antisway control of an offshore container crane","volume":"17","author":"Ngo","year":"2010","journal-title":"IEEE\/ASME Transactions on Mechatronics"},{"issue":"4","key":"2022082710080473300_bib31","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1093\/jcde\/qwaa035","article-title":"V-REP-based navigation of automated wheeled robot between obstacles using PSO-tuned feedforward neural network","volume":"7","author":"Pandey","year":"2020","journal-title":"Journal of Computational Design and Engineering"},{"key":"2022082710080473300_bib32","article-title":"Proximal policy optimization algorithms","author":"Schulman","year":"2017"},{"key":"2022082710080473300_bib33","volume-title":"Reinforcement learning: An introduction","author":"Sutton","year":"2018"},{"key":"2022082710080473300_bib34","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1016\/j.ijnaoe.2019.11.004","article-title":"Experimental and numerical study of autopilot using extended Kalman filter trained neural networks for surface vessels","volume":"12","author":"Wang","year":"2020","journal-title":"International Journal of Naval Architecture and Ocean Engineering"},{"key":"2022082710080473300_bib35","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1016\/S0167-2789(97)00051-1","article-title":"Mechanical integrators derived from a discrete variational principle","volume":"106","author":"Wendlandt","year":"1997","journal-title":"Physica D: Nonlinear Phenomena"},{"key":"2022082710080473300_bib36","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1016\/j.fss.2015.01.010","article-title":"Anti-sway tracking control of tower cranes with delayed uncertainty using a robust adaptive fuzzy control","volume":"290","author":"Wu","year":"2016","journal-title":"Fuzzy Sets and Systems"},{"key":"2022082710080473300_bib37","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1109\/TNNLS.2019.2910580","article-title":"Neural network-based adaptive antiswing control of an underactuated ship-mounted crane with roll motions and input dead zones","volume":"31","author":"Yang","year":"2019","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"2022082710080473300_bib38","doi-asserted-by":"crossref","first-page":"182","DOI":"10.17736\/ijope.2018.jc722","article-title":"Antisway control of a crane on an offshore support vessel based on the hardware-in-the-loop simulation","volume":"28","author":"Zhao","year":"2018","journal-title":"International Journal of Offshore and Polar Engineering"}],"container-title":["Journal of Computational Design and Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jcde\/advance-article-pdf\/doi\/10.1093\/jcde\/qwac063\/45052726\/qwac063.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jcde\/article-pdf\/9\/4\/1430\/45596081\/qwac063.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jcde\/article-pdf\/9\/4\/1430\/45596081\/qwac063.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,27]],"date-time":"2022-08-27T10:08:53Z","timestamp":1661594933000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jcde\/article\/9\/4\/1430\/6649210"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,23]]},"references-count":38,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,8,27]]}},"URL":"https:\/\/doi.org\/10.1093\/jcde\/qwac063","relation":{},"ISSN":["2288-5048"],"issn-type":[{"value":"2288-5048","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,8]]},"published":{"date-parts":[[2022,7,23]]}}}