{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T02:47:10Z","timestamp":1779245230734,"version":"3.51.4"},"reference-count":62,"publisher":"Wiley","issue":"9","license":[{"start":{"date-parts":[[2023,6,22]],"date-time":"2023-06-22T00:00:00Z","timestamp":1687392000000},"content-version":"am","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2023,6,22]],"date-time":"2023-06-22T00:00:00Z","timestamp":1687392000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["N00014-19-1-2373"],"award-info":[{"award-number":["N00014-19-1-2373"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["N00014- 22-1-2569"],"award-info":[{"award-number":["N00014- 22-1-2569"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["1830881"],"award-info":[{"award-number":["1830881"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["advanced.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Advanced Intelligent Systems"],"published-print":{"date-parts":[[2023,9]]},"abstract":"<jats:p>Inspired by the unique neurophysiology of the octopus, a hierarchical framework is proposed that simplifies the coordination of multiple soft arms by decomposing control into high\u2010level decision\u2010making, low\u2010level motor activation, and local reflexive behaviors via sensory feedback. When evaluated in the illustrative problem of a model octopus foraging for food, this hierarchical decomposition results in significant improvements relative to end\u2010to\u2010end methods. Performance is achieved through a mixed\u2010modes approach, whereby qualitatively different tasks are addressed via complementary control schemes. Herein, model\u2010free reinforcement learning is employed for high\u2010level decision\u2010making, while model\u2010based energy shaping takes care of arm\u2010level motor execution. To render the pairing computationally tenable, a novel neural network energy shaping (NN\u2010ES) controller is developed, achieving accurate motions with time\u2010to\u2010solutions 200 times faster than previous attempts. The hierarchical framework is then successfully deployed in increasingly challenging foraging scenarios, including an arena littered with obstacles in 3D space, demonstrating the viability of the approach.<\/jats:p>","DOI":"10.1002\/aisy.202300088","type":"journal-article","created":{"date-parts":[[2023,6,24]],"date-time":"2023-06-24T19:05:36Z","timestamp":1687633536000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["Hierarchical Control and Learning of a Foraging CyberOctopus"],"prefix":"10.1002","volume":"5","author":[{"given":"Chia-Hsien","family":"Shih","sequence":"first","affiliation":[{"name":"Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign  Urbana IL 61801 USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Noel","family":"Naughton","sequence":"additional","affiliation":[{"name":"Beckman Institute for Advanced Science and Technology University of Illinois at Urbana-Champaign  Urbana IL 61801 USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Udit","family":"Halder","sequence":"additional","affiliation":[{"name":"Coordinated Science Laboratory University of Illinois at Urbana-Champaign  Urbana IL 61801 USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Heng-Sheng","family":"Chang","sequence":"additional","affiliation":[{"name":"Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign  Urbana IL 61801 USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seung Hyun","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign  Urbana IL 61801 USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rhanor","family":"Gillette","sequence":"additional","affiliation":[{"name":"Department of Molecular and Integrative Physiology University of Illinois at Urbana-Champaign  Urbana IL 61801 USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Prashant G.","family":"Mehta","sequence":"additional","affiliation":[{"name":"Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign  Urbana IL 61801 USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2129-379X","authenticated-orcid":false,"given":"Mattia","family":"Gazzola","sequence":"additional","affiliation":[{"name":"Department of Mechanical Science and Engineering University of Illinois at Urbana-Champaign  Urbana IL 61801 USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2023,6,22]]},"reference":[{"key":"e_1_2_10_2_1","doi-asserted-by":"publisher","DOI":"10.1089\/soro.2016.0080"},{"key":"e_1_2_10_3_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-019-12759-5"},{"key":"e_1_2_10_4_1","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.aau1532"},{"key":"e_1_2_10_5_1","unstructured":"C.Della Santina C.Duriez D.Rus arXiv preprint arXiv:2110.013582021 1\u201369."},{"key":"e_1_2_10_6_1","doi-asserted-by":"publisher","DOI":"10.1089\/soro.2017.0007"},{"key":"e_1_2_10_7_1","doi-asserted-by":"publisher","DOI":"10.1080\/11762320802557865"},{"key":"e_1_2_10_8_1","doi-asserted-by":"publisher","DOI":"10.1163\/156855301317198160"},{"key":"e_1_2_10_9_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41563-020-0699-3"},{"key":"e_1_2_10_10_1","doi-asserted-by":"publisher","DOI":"10.1242\/jeb.176289"},{"key":"e_1_2_10_11_1","doi-asserted-by":"crossref","unstructured":"X.Lu W.Xu X.Li in2015 6th Int. Conf. on Automation Robotics and Applications (ICARA) Queenstown New Zealand2015 pp.332\u2013336 https:\/\/doi.org\/10.1109\/ICARA.2015.7081169.","DOI":"10.1109\/ICARA.2015.7081169"},{"key":"e_1_2_10_12_1","doi-asserted-by":"publisher","DOI":"10.1089\/soro.2018.0082"},{"key":"e_1_2_10_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2966407"},{"key":"e_1_2_10_14_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202100092"},{"key":"e_1_2_10_15_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-018-08173-y"},{"key":"e_1_2_10_16_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1107976"},{"key":"e_1_2_10_17_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(02)00243-9"},{"key":"e_1_2_10_18_1","doi-asserted-by":"publisher","DOI":"10.1088\/1748-3190\/11\/5\/056012"},{"key":"e_1_2_10_19_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-26276-x"},{"key":"e_1_2_10_20_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1255718"},{"key":"e_1_2_10_21_1","doi-asserted-by":"crossref","unstructured":"T.Li K.Nakajima M.Calisti C.Laschi R.Pfeifer 2013 IEEE Int. Conf. on Mechatronics and Automation Karlsruhe Germany 2013 1504\u20131511 https:\/\/doi.org\/10.1109\/ICRA.2013.6630770.","DOI":"10.1109\/ICRA.2013.6630770"},{"key":"e_1_2_10_22_1","doi-asserted-by":"publisher","DOI":"10.1088\/1748-3190\/10\/3\/035003"},{"key":"e_1_2_10_23_1","unstructured":"M. D.Grissom V.Chitrakaran D.Dienno M.Csencits M.Pritts B.Jones W.McMahan D.Dawson C.Rahn I.Walker Unmanned Systems Technology VIII Vol.6230 May 2006 p.62301F."},{"key":"e_1_2_10_24_1","doi-asserted-by":"publisher","DOI":"10.1088\/1748-3190\/abf6b9"},{"key":"e_1_2_10_25_1","doi-asserted-by":"crossref","unstructured":"Y.Sakuhara H.Shimizu K.Ito in2020 IEEE 10th Int. Conf. on Intelligent Systems (IS) Varna Bulgaria2020 pp.463\u2013468 https:\/\/doi.org\/10.1109\/IS48319.2020.9199967.","DOI":"10.1109\/IS48319.2020.9199967"},{"key":"e_1_2_10_26_1","doi-asserted-by":"crossref","unstructured":"H.-S.Chang U.Halder E.Gribkova A.Tekinalp N.Naughton M.Gazzola P. G.Mehta In2021 60th IEEE Conference on Decision and Control (CDC) Austin TX2021pp.1383\u20131390 https:\/\/doi.org\/10.1109\/CDC45484.2021.9683318.","DOI":"10.1109\/CDC45484.2021.9683318"},{"key":"e_1_2_10_27_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1365-2435.2005.01033.x"},{"key":"e_1_2_10_28_1","doi-asserted-by":"publisher","DOI":"10.4003\/0740-2783-24.1.13"},{"key":"e_1_2_10_29_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-020-77873-7"},{"key":"e_1_2_10_30_1","doi-asserted-by":"publisher","DOI":"10.1038\/scientificamericanmind0117-62"},{"key":"e_1_2_10_31_1","first-page":"144","volume":"112","author":"Young J.","year":"1971","journal-title":"J. Anat."},{"key":"e_1_2_10_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2012.09.001"},{"key":"e_1_2_10_33_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.1060976"},{"key":"e_1_2_10_34_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jembe.2013.02.016"},{"key":"e_1_2_10_35_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2020.08.037"},{"key":"e_1_2_10_36_1","doi-asserted-by":"crossref","unstructured":"H.-S.Chang U.Halder C.-H.Shih A.Tekinalp T.Parthasarathy E.Gribkova G.Chowdhary R.Gillette M.Gazzola P. G.Mehta in2020 59th IEEE Conf. on Decision and Control (CDC) Jeju Korea (South) 2020 pp.3913\u20133920 https:\/\/doi.org\/10.1109\/CDC42340.2020.9304408.","DOI":"10.1109\/CDC42340.2020.9304408"},{"key":"e_1_2_10_37_1","doi-asserted-by":"publisher","DOI":"10.1098\/rsos.171628"},{"key":"e_1_2_10_38_1","doi-asserted-by":"publisher","DOI":"10.1002\/jmor.10548"},{"key":"e_1_2_10_39_1","unstructured":"A.Tekinalp N.Naughton S.-H.Kim U.Halder R.Gillette P. G.Mehta W.Kier M.Gazzola arXiv preprint arXiv:2304.084132023."},{"key":"e_1_2_10_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-4147-6"},{"key":"e_1_2_10_41_1","doi-asserted-by":"publisher","DOI":"10.1089\/soro.2018.0088"},{"key":"e_1_2_10_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3063698"},{"key":"e_1_2_10_43_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1907051116"},{"key":"e_1_2_10_44_1","doi-asserted-by":"publisher","DOI":"10.1002\/adfm.201801145"},{"key":"e_1_2_10_45_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202000237"},{"key":"e_1_2_10_46_1","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.123.208003"},{"key":"e_1_2_10_47_1","doi-asserted-by":"publisher","DOI":"10.1098\/rsif.2013.0816"},{"key":"e_1_2_10_48_1","doi-asserted-by":"publisher","DOI":"10.1523\/JNEUROSCI.18-15-05976.1998"},{"key":"e_1_2_10_49_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2006.02.069"},{"key":"e_1_2_10_50_1","doi-asserted-by":"publisher","DOI":"10.1098\/rspa.2022.0593"},{"key":"e_1_2_10_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/37.915398"},{"key":"e_1_2_10_52_1","doi-asserted-by":"publisher","DOI":"10.1080\/00207170210135939"},{"key":"e_1_2_10_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/9.895562"},{"key":"e_1_2_10_54_1","volume-title":"The Oxford Handbook Of Invertebrate Neurobiology","author":"Levy G.","year":"2019"},{"key":"e_1_2_10_55_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cub.2022.10.010"},{"key":"e_1_2_10_56_1","volume-title":"The Traveling Salesman Problem","author":"Applegate D. L.","year":"2011"},{"key":"e_1_2_10_57_1","volume-title":"The Traveling Salesman Problem and its Variations","author":"Gutin G.","year":"2006"},{"key":"e_1_2_10_58_1","doi-asserted-by":"publisher","DOI":"10.1016\/0304-3975(77)90012-3"},{"key":"e_1_2_10_59_1","doi-asserted-by":"publisher","DOI":"10.3390\/ani10030457"},{"key":"e_1_2_10_60_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00199609"},{"key":"e_1_2_10_61_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1469-185X.1996.tb00746.x"},{"key":"e_1_2_10_62_1","unstructured":"J.Schulman F.Wolski P.Dhariwal A.Radford O.Klimov arXiv preprint arXiv:1707.063472017 pp.1\u201312."},{"key":"e_1_2_10_63_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42254-022-00481-z"}],"container-title":["Advanced Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aisy.202300088","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aisy.202300088","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,7]],"date-time":"2025-10-07T16:43:11Z","timestamp":1759855391000},"score":1,"resource":{"primary":{"URL":"https:\/\/advanced.onlinelibrary.wiley.com\/doi\/10.1002\/aisy.202300088"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,22]]},"references-count":62,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2023,9]]}},"alternative-id":["10.1002\/aisy.202300088"],"URL":"https:\/\/doi.org\/10.1002\/aisy.202300088","archive":["Portico"],"relation":{},"ISSN":["2640-4567","2640-4567"],"issn-type":[{"value":"2640-4567","type":"print"},{"value":"2640-4567","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,22]]},"assertion":[{"value":"2023-02-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"2300088"}}