{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T16:12:42Z","timestamp":1774541562341,"version":"3.50.1"},"reference-count":48,"publisher":"Wiley","issue":"8","license":[{"start":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T00:00:00Z","timestamp":1720396800000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100010664","name":"H2020 Future and Emerging Technologies","doi-asserted-by":"publisher","award":["863212"],"award-info":[{"award-number":["863212"]}],"id":[{"id":"10.13039\/100010664","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["advanced.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Advanced Intelligent Systems"],"published-print":{"date-parts":[[2024,8]]},"abstract":"<jats:p>Soft robots can adaptively interact with unstructured environments. However, nonlinear soft material properties challenge modeling and control. Learning\u2010based controllers that leverage efficient mechanical models are promising for solving complex interaction tasks. This article develops a closed\u2010loop pose\/force controller for a dexterous soft manipulator enabling dynamic pushing tasks using deep reinforcement learning. Force tests investigate the mechanical properties of a soft robot module, resulting in orthogonal forces of \u2009N. Then, the policy is trained in simulation leveraging a dynamic Cosserat rod model of the soft robot. Domain randomization mitigate the sim\u2010to\u2010real gap while careful reward engineering induced pose and force control even without explicit force inputs. Despite the approximate simulation, the sim\u2010to\u2010real transfer achieved an average reaching distance of \u2009mm\u2009(), an average orientation error of \u2009rad () and applied pushing forces up to \u2009N. Such performance is reasonable for the intended assistive tasks of the manipulator. The experiments uncovered that the soft robot interacting with the environment exhibited torsional and counter\u2010balancing movements. Although not explicitly enforced, they emerged from the mechanical intelligence of the manipulator. The results demonstrate the potential of soft robotic manipulation via reinforcement learning.<\/jats:p>","DOI":"10.1002\/aisy.202300899","type":"journal-article","created":{"date-parts":[[2024,7,8]],"date-time":"2024-07-08T07:25:45Z","timestamp":1720423545000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Pushing with Soft Robotic Arms via Deep Reinforcement Learning"],"prefix":"10.1002","volume":"6","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7545-8513","authenticated-orcid":false,"given":"Carlo","family":"Alessi","sequence":"first","affiliation":[{"name":"The BioRobotics Institute Scuola Superiore Sant\u2019Anna  Viale Rinaldo Piaggio, 34 56025 Pontedera PI Italy"},{"name":"Department of Excellence in Robotics &amp; AI Scuola Superiore Sant\u2019Anna  Piazza Martiri della Libert\u00e0 33 56127 Pisa PI Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7148-1612","authenticated-orcid":false,"given":"Diego","family":"Bianchi","sequence":"additional","affiliation":[{"name":"The BioRobotics Institute Scuola Superiore Sant\u2019Anna  Viale Rinaldo Piaggio, 34 56025 Pontedera PI Italy"},{"name":"Department of Excellence in Robotics &amp; AI Scuola Superiore Sant\u2019Anna  Piazza Martiri della Libert\u00e0 33 56127 Pisa PI Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6676-8472","authenticated-orcid":false,"given":"Gianni","family":"Stano","sequence":"additional","affiliation":[{"name":"Department of Mechanics Mathematics and Management Polytechnic University of Bari  Via Edoardo Orabona 4 70125 Bari BA Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9016-8039","authenticated-orcid":false,"given":"Matteo","family":"Cianchetti","sequence":"additional","affiliation":[{"name":"The BioRobotics Institute Scuola Superiore Sant\u2019Anna  Viale Rinaldo Piaggio, 34 56025 Pontedera PI Italy"},{"name":"Department of Excellence in Robotics &amp; AI Scuola Superiore Sant\u2019Anna  Piazza Martiri della Libert\u00e0 33 56127 Pisa PI Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8060-8080","authenticated-orcid":false,"given":"Egidio","family":"Falotico","sequence":"additional","affiliation":[{"name":"The BioRobotics Institute Scuola Superiore Sant\u2019Anna  Viale Rinaldo Piaggio, 34 56025 Pontedera PI Italy"},{"name":"Department of Excellence in Robotics &amp; AI Scuola Superiore Sant\u2019Anna  Piazza Martiri della Libert\u00e0 33 56127 Pisa PI Italy"}]}],"member":"311","published-online":{"date-parts":[[2024,7,8]]},"reference":[{"key":"e_1_2_7_2_1","doi-asserted-by":"publisher","DOI":"10.3389\/fbioe.2014.00003"},{"key":"e_1_2_7_3_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14543"},{"key":"e_1_2_7_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2022.3231360"},{"key":"e_1_2_7_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCS.2023.3253419"},{"key":"e_1_2_7_6_1","doi-asserted-by":"publisher","DOI":"10.1089\/soro.2017.0007"},{"key":"e_1_2_7_7_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202100165"},{"key":"e_1_2_7_8_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364910368147"},{"key":"e_1_2_7_9_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202200024"},{"key":"e_1_2_7_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2019.2955936"},{"key":"e_1_2_7_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCS.2023.3253421"},{"key":"e_1_2_7_12_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.201900171"},{"key":"e_1_2_7_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3146903"},{"key":"e_1_2_7_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3157369"},{"key":"e_1_2_7_15_1","author":"Bianchi D.","year":"2023","journal-title":"IEEE Robot. Autom. Mag."},{"key":"e_1_2_7_16_1","doi-asserted-by":"publisher","DOI":"10.1080\/01691864.2017.1395362"},{"key":"e_1_2_7_17_1","unstructured":"J.Allard S.Cotin F.Faure P.\u2010J.Bensoussan F.Poyer C.Duriez H.Delingette L.Grisoni inMMVR 15\u2010Medicine Meets Virtual Reality IOP Press Bristol2007 Vol.125 pp.13\u201318."},{"key":"e_1_2_7_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3154050"},{"key":"e_1_2_7_19_1","doi-asserted-by":"crossref","unstructured":"W.Zhao J. P.Queralta T.Westerlund in2020 IEEE Symposium Series on Computational Intelligence (SSCI) IEEE Piscataway NJ2020 pp.737\u2013744.","DOI":"10.1109\/SSCI47803.2020.9308468"},{"key":"e_1_2_7_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/s40964-020-00158-y"},{"key":"e_1_2_7_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2023.3266282"},{"key":"e_1_2_7_22_1","doi-asserted-by":"crossref","unstructured":"C.Alessi H.Hauser A.Lucantonio E.Falotico in2023 IEEE Inter. Conf. on Soft Robotics (RoboSoft) IEEE Piscataway NJ2023 pp.1\u20137.","DOI":"10.1109\/RoboSoft55895.2023.10121988"},{"key":"e_1_2_7_23_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364915584806"},{"key":"e_1_2_7_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2019.2958015"},{"key":"e_1_2_7_25_1","doi-asserted-by":"publisher","DOI":"10.1089\/soro.2019.0065"},{"key":"e_1_2_7_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3063698"},{"key":"e_1_2_7_27_1","doi-asserted-by":"crossref","unstructured":"X.Wang N.Rojas in2022 IEEE 5th Inter. Conf. on Soft Robotics (RoboSoft) IEEE Piscataway NJ2022 pp.247\u2013254.","DOI":"10.1109\/RoboSoft54090.2022.9762115"},{"key":"e_1_2_7_28_1","doi-asserted-by":"crossref","unstructured":"S.Satheeshbabu N. K.Uppalapati T.Fu G.Krishnan in2020 3rd IEEE Inter. Conf. on Soft Robotics (RoboSoft) IEEE Piscataway NJ2020 pp.497\u2013503.","DOI":"10.1109\/RoboSoft48309.2020.9116003"},{"key":"e_1_2_7_29_1","doi-asserted-by":"publisher","DOI":"10.1089\/soro.2014.0001"},{"key":"e_1_2_7_30_1","doi-asserted-by":"publisher","DOI":"10.1177\/1729881416687132"},{"key":"e_1_2_7_31_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-023-00669-7"},{"key":"e_1_2_7_32_1","doi-asserted-by":"crossref","unstructured":"C.Agabiti E.M\u00e9nager E.Falotico in2023 IEEE Inter. Conf. on Soft Robotics (RoboSoft) IEEE Piscataway NJ2023 pp.1\u20136.","DOI":"10.1109\/RoboSoft55895.2023.10122076"},{"key":"e_1_2_7_33_1","doi-asserted-by":"publisher","DOI":"10.3389\/frobt.2020.00008"},{"key":"e_1_2_7_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2023.3295236"},{"key":"e_1_2_7_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3185786"},{"key":"e_1_2_7_36_1","doi-asserted-by":"crossref","unstructured":"Z.Tang P.Wang W.Xin Z.Xie L.Kan M.Mohanakrishnan C.Laschi in2023 IEEE Inter. Conf. on Robotics and Automation (ICRA) IEEE Piscataway NJ2023 pp.982\u2013988.","DOI":"10.1109\/ICRA48891.2023.10160513"},{"key":"e_1_2_7_37_1","doi-asserted-by":"crossref","unstructured":"Z.Ding N. F.Lepora E.Johns in2020 IEEE Inter. Conf. on Robotics and Automation (ICRA) IEEE Piscataway NJ2020 pp.1639\u20131645.","DOI":"10.1109\/ICRA40945.2020.9197512"},{"key":"e_1_2_7_38_1","doi-asserted-by":"crossref","unstructured":"R.Lal R.Swaminathan L.Seenivasan L.Qiu H.Ren in2021 IEEE Inter. Conf. on Development and Learning (ICDL) IEEE Piscataway NJ2021 pp.1\u20136.","DOI":"10.1109\/ICDL49984.2021.9515617"},{"key":"e_1_2_7_39_1","doi-asserted-by":"crossref","unstructured":"Y.Li X.Wang K.\u2010W.Kwok in2022 IEEE\/RSJ Inter. Conf. on Intelligent Robots and Systems (IROS) IEEE Piscataway NJ2022 pp.7074\u20137081.","DOI":"10.1109\/IROS47612.2022.9981335"},{"key":"e_1_2_7_40_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.addma.2023.103463"},{"key":"e_1_2_7_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.addma.2023.103509"},{"key":"e_1_2_7_42_1","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.171628"},{"key":"e_1_2_7_43_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-019-12759-5"},{"key":"e_1_2_7_44_1","unstructured":"T.Preclik C.Popa U.R\u00fcde inProceedings of Multibody Dynamics Brussels BelgiumJuly2011 Vol.2011."},{"key":"e_1_2_7_45_1","unstructured":"J.Schulman F.Wolski P.Dhariwal A.Radford O.Klimov arXiv preprint arXiv:1707.06347 2017."},{"key":"e_1_2_7_46_1","first-page":"12 348","volume":"22","author":"Raffin A.","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_2_7_47_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364919887447"},{"key":"e_1_2_7_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3144788"},{"key":"e_1_2_7_49_1","unstructured":"B.Mehta M.Diaz F.Golemo C. J.Pal L.Paull inConf. on Robot Learning PMLRNovember2020 pp.1162\u20131176."}],"container-title":["Advanced Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aisy.202300899","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,7]],"date-time":"2025-10-07T18:50:38Z","timestamp":1759863038000},"score":1,"resource":{"primary":{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202300899"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,8]]},"references-count":48,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["10.1002\/aisy.202300899"],"URL":"https:\/\/doi.org\/10.1002\/aisy.202300899","archive":["Portico"],"relation":{},"ISSN":["2640-4567","2640-4567"],"issn-type":[{"value":"2640-4567","type":"print"},{"value":"2640-4567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,8]]},"assertion":[{"value":"2024-01-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"2300899"}}