{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T00:16:12Z","timestamp":1778112972254,"version":"3.51.4"},"reference-count":29,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T00:00:00Z","timestamp":1703116800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Mistra Digital Forest","award":["Grant DIA 2017\/14 #6"],"award-info":[{"award-number":["Grant DIA 2017\/14 #6"]}]},{"name":"Mistra Digital Forest","award":["HPC2N"],"award-info":[{"award-number":["HPC2N"]}]},{"name":"Swedish National Infrastructure for Computing at High Performance Computing Center North","award":["Grant DIA 2017\/14 #6"],"award-info":[{"award-number":["Grant DIA 2017\/14 #6"]}]},{"name":"Swedish National Infrastructure for Computing at High Performance Computing Center North","award":["HPC2N"],"award-info":[{"award-number":["HPC2N"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>We explore multi-log grasping using reinforcement learning and virtual visual servoing for automated forwarding in a simulated environment. Automation of forest processes is a major challenge, and many techniques regarding robot control pose different challenges due to the unstructured and harsh outdoor environment. Grasping multiple logs involves various problems of dynamics and path planning, where understanding the interaction between the grapple, logs, terrain, and obstacles requires visual information. To address these challenges, we separate image segmentation from crane control and utilise a virtual camera to provide an image stream from reconstructed 3D data. We use Cartesian control to simplify domain transfer to real-world applications. Because log piles are static, visual servoing using a 3D reconstruction of the pile and its surroundings is equivalent to using real camera data until the point of grasping. This relaxes the limits on computational resources and time for the challenge of image segmentation, and allows for data collection in situations where the log piles are not occluded. The disadvantage is the lack of information during grasping. We demonstrate that this problem is manageable and present an agent that is 95% successful in picking one or several logs from challenging piles of 2\u20135 logs.<\/jats:p>","DOI":"10.3390\/robotics13010003","type":"journal-article","created":{"date-parts":[[2023,12,21]],"date-time":"2023-12-21T08:16:02Z","timestamp":1703146562000},"page":"3","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Multi-Log Grasping Using Reinforcement Learning and Virtual Visual Servoing"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6266-4740","authenticated-orcid":false,"given":"Erik","family":"Wallin","sequence":"first","affiliation":[{"name":"Department of Physics, Ume\u00e5 University, 907 87 Ume\u00e5, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6565-3123","authenticated-orcid":false,"given":"Viktor","family":"Wiberg","sequence":"additional","affiliation":[{"name":"Department of Physics, Ume\u00e5 University, 907 87 Ume\u00e5, Sweden"},{"name":"Algoryx Simulation AB, Kuratorv\u00e4gen 2, 907 36 Ume\u00e5, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0787-4988","authenticated-orcid":false,"given":"Martin","family":"Servin","sequence":"additional","affiliation":[{"name":"Department of Physics, Ume\u00e5 University, 907 87 Ume\u00e5, Sweden"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,12,21]]},"reference":[{"key":"ref_1","first-page":"271","article-title":"The economic potential of semi-automated tele-extraction of roundwood in Sweden","volume":"33","author":"Fjeld","year":"2022","journal-title":"Int. J. For. Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"138","DOI":"10.1016\/S0924-2716(99)00008-8","article-title":"Processing of laser scanner data\u2014Algorithms and applications","volume":"54","author":"Axelsson","year":"1999","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_3","first-page":"219","article-title":"Terrain modelling and analysis using laser scanner data","volume":"34","author":"Elmqvist","year":"2001","journal-title":"Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1016\/j.jterra.2022.04.002","article-title":"Learning multiobjective rough terrain traversability","volume":"102","author":"Wallin","year":"2022","journal-title":"J. Terramech."},{"key":"ref_5","unstructured":"Lindroos, O., Mendoza-Trejo, O., La Hera, P., and Morales, D.O. (2019). Robotics and Automation for Improving Agriculture, Burleigh Dodds Science Publishing."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Caldera, S., Rassau, A., and Chai, D. (2018). Review of deep learning methods in robotic grasp detection. Multimodal Technol. Interact., 2.","DOI":"10.20944\/preprints201805.0484.v1"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1177\/0278364917710318","article-title":"Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection","volume":"37","author":"Levine","year":"2018","journal-title":"Int. J. Robot. Res."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1007\/s43154-020-00021-6","article-title":"A survey on learning-based robotic grasping","volume":"1","author":"Kleeberger","year":"2020","journal-title":"Curr. Robot. Rep."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1002\/rob.21496","article-title":"Increasing the level of automation in the forestry logging process with crane trajectory planning and control","volume":"31","author":"Westerberg","year":"2014","journal-title":"J. Field Robot."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"9525","DOI":"10.1109\/LRA.2022.3190808","article-title":"Nonlinear Model Learning for Compensation and Feedforward Control of Real-World Hydraulic Actuators Using Gaussian Processes","volume":"7","author":"Taheri","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Andersson, J., Bodin, K., Lindmark, D., Servin, M., and Wallin, E. (October, January 27). Reinforcement learning control of a forestry crane manipulator. Proceedings of the 2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic.","DOI":"10.1109\/IROS51168.2021.9636219"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhao, W., Queralta, J.P., and Westerlund, T. (2020, January 1\u20134). Sim-to-real transfer in deep reinforcement learning for robotics: A survey. Proceedings of the 2020 IEEE Symposium Series on Computational Intelligence (SSCI), Canberra, ACT, Australia.","DOI":"10.1109\/SSCI47803.2020.9308468"},{"key":"ref_13","unstructured":"Wiberg, V., Wallin, E., F\u00e4lldin, A., Semberg, T., Rossander, M., Wadbro, E., and Servin, M. (2023). Sim-to-real transfer of active suspension control using deep reinforcement learning. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Dhakate, R., Brommer, C., Bohm, C., Gietler, H., Weiss, S., and Steinbrener, J. (2022, January 23\u201327). Autonomous Control of Redundant Hydraulic Manipulator Using Reinforcement Learning with Action Feedback. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.","DOI":"10.1109\/IROS47612.2022.9981425"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1069","DOI":"10.1016\/j.conengprac.2010.05.015","article-title":"Semi-autonomous shared control of large-scale manipulator arms","volume":"18","author":"Hansson","year":"2010","journal-title":"Control Eng. Pract."},{"key":"ref_16","unstructured":"Ainetter, S., B\u00f6hm, C., Dhakate, R., Weiss, S., and Fraundorfer, F. (2021). Depth-aware object segmentation and grasp detection for robotic picking tasks. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Fortin, J.M., Gamache, O., Grondin, V., Pomerleau, F., and Gigu\u00e8re, P. (2022, January 23\u201327). Instance segmentation for autonomous log grasping in forestry operations. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.","DOI":"10.1109\/IROS47612.2022.9982286"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"La Hera, P., Trejo, O., Lindroos, O., Lideskog, H., Lindb\u00e4ck, T., Latif, S., Li, S., and Karlberg, M. (2023). Exploring the Feasibility of Autonomous Forestry Operations: Results from the First Experimental Unmanned Machine. Authorea.","DOI":"10.22541\/au.168492982.27477779\/v1"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ayoub, E., Levesque, P., and Sharf, I. (June, January 29). Grasp Planning with CNN for Log-loading Forestry Machine. Proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA), London, UK.","DOI":"10.1109\/ICRA48891.2023.10161562"},{"key":"ref_20","unstructured":"(2023, October 30). Algoryx Simulations. AGX Dynamics. Available online: https:\/\/www.algoryx.se\/agx-dynamics\/."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1145\/325165.325247","article-title":"An image synthesizer","volume":"19","author":"Perlin","year":"1985","journal-title":"Acm Siggraph Comput. Graph."},{"key":"ref_22","unstructured":"Cranab, AB (2022, October 08). Forwarder Cranes Brochure. Available online: https:\/\/www.cranab.com\/downloads\/Forwarder-Cranes\/Cranab-FC-brochure-EN.pdf."},{"key":"ref_23","unstructured":"Spong, M.W., and Vidyasagar, M. (2008). Robot Dynamics and Control, John Wiley & Sons."},{"key":"ref_24","unstructured":"Palmroth, M., Laitinen, S., Siltanen, V., and K\u00e4ppi, T. (2014). Method and System for Controlling the Crane of a Working Machine by Using Boom Tip Control. (WO2014118430A1), Patent, Available online: https:\/\/patents.google.com\/patent\/WO2014118430A1\/ko."},{"key":"ref_25","unstructured":"Sutton, R.S., and Barto, A.G. (2018). Reinforcement Learning: An Introduction, MIT Press."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"529","DOI":"10.1038\/nature14236","article-title":"Human-level control through deep reinforcement learning","volume":"518","author":"Mnih","year":"2015","journal-title":"Nature"},{"key":"ref_27","first-page":"1","article-title":"Stable-Baselines3: Reliable Reinforcement Learning Implementations","volume":"22","author":"Raffin","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_28","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal policy optimization algorithms. arXiv."},{"key":"ref_29","unstructured":"(2023, October 30). Stable baselines3. PPO Documentation. Available online: https:\/\/stable-baselines3.readthedocs.io\/en\/master\/modules\/ppo.html."}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/13\/1\/3\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:39:41Z","timestamp":1760132381000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/13\/1\/3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,21]]},"references-count":29,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["robotics13010003"],"URL":"https:\/\/doi.org\/10.3390\/robotics13010003","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,21]]}}}