{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T15:15:21Z","timestamp":1761664521309,"version":"3.41.2"},"reference-count":35,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,2,15]],"date-time":"2023-02-15T00:00:00Z","timestamp":1676419200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["52025054"],"award-info":[{"award-number":["52025054"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["52105016"],"award-info":[{"award-number":["52105016"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>As astronauts perform on-orbit servicing of extravehicular activity (EVA) without the help of the space station\u2019s robotic arms, it will be rather difficult and labor-consuming to maintain the appropriate position in case of impact. In order to solve this problem, we propose the development of a wearable robotic limb system for astronaut assistance and a variable damping control method for maintaining the astronaut\u2019s position. The requirements of the astronaut\u2019s impact-resisting ability during EVA were analyzed, including the capabilities of deviation resistance, fast return, oscillation resistance, and accurate return. To meet these needs, the system of the astronaut with robotic limbs was modeled and simplified. In combination with this simplified model and a reinforcement learning algorithm, a variable damping controller for the end of the robotic limb was obtained, which can regulate the dynamic performance of the robot end to resist oscillation after impact. A weightless simulation environment for the astronaut with robotic limbs was constructed. The simulation results demonstrate that the proposed method can meet the recommended requirements for maintaining an astronaut\u2019s position during EVA. No matter how the damping coefficient was set, the fixed damping control method failed to meet all four requirements at the same time. In comparison to the fixed damping control method, the variable damping controller proposed in this paper fully satisfied all the impact-resisting requirements by itself. It could prevent excessive deviation from the original position and was able to achieve a fast return to the starting point. The maximum deviation displacement was reduced by 39.3% and the recovery time was cut by 17.7%. Besides, it also had the ability to prevent reciprocating oscillation and return to the original position accurately.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1093718","type":"journal-article","created":{"date-parts":[[2023,2,15]],"date-time":"2023-02-15T09:11:08Z","timestamp":1676452268000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Reinforcement learning based variable damping control of wearable robotic limbs for maintaining astronaut pose during extravehicular activity"],"prefix":"10.3389","volume":"17","author":[{"given":"Sikai","family":"Zhao","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tianjiao","family":"Zheng","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dongbao","family":"Sui","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanhe","family":"Zhu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,2,15]]},"reference":[{"journal-title":"Skybot F-850 will spend a week on the ISS charming astronauts with its sense of humor.","year":"2019","author":"Ackerman","key":"B1"},{"key":"B2","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1016\/j.actaastro.2014.12.004","article-title":"The first decade of commercial space tourism.","volume":"108","author":"Chang","year":"2015","journal-title":"Acta Astronaut."},{"key":"B3","doi-asserted-by":"publisher","first-page":"883562","DOI":"10.3389\/fnbot.2022.883562","article-title":"Deep reinforcement learning based trajectory planning under uncertain constraints.","volume":"16","author":"Chen","year":"2022","journal-title":"Front. Neurorobot."},{"key":"B4","doi-asserted-by":"publisher","first-page":"4831","DOI":"10.1126\/scirobotics.aan4831","article-title":"Robotic space exploration agents.","volume":"2","author":"Chien","year":"2017","journal-title":"Sci. Robot."},{"key":"B5","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1146\/annurev-statistics-031219-041220","article-title":"Q-Learning: Theory and applications.","volume":"7","author":"Clifton","year":"2020","journal-title":"Ann. Rev. Stat. Appl."},{"key":"B6","doi-asserted-by":"publisher","first-page":"829437","DOI":"10.3389\/fnbot.2022.829437","article-title":"Reinforcement learning with vision-proprioception model for robot planar pushing.","volume":"16","author":"Cong","year":"2022","journal-title":"Front. Neurorobot."},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5979830","article-title":"Robonaut 2 \u2013 the first humanoid robot in space","author":"Diftler","year":"2011","journal-title":"Proceedings of the 2011 IEEE international conference on robotics and automation"},{"key":"B8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.paerosci.2014.03.002","article-title":"A review of space robotics technologies for on-orbit servicing.","volume":"68","author":"Flores-Abad","year":"2014","journal-title":"Prog. Aerosp. Sci."},{"journal-title":"NASA spacewalker Anne McClain.","year":"2019","author":"Garcia","key":"B9"},{"journal-title":"NASA\u2019s ironman-like exoskeleton could give astronauts, paraplegics improved mobility and strength.","year":"2013","author":"Hall","key":"B10"},{"key":"B11","doi-asserted-by":"publisher","first-page":"565702","DOI":"10.3389\/fnbot.2020.565702","article-title":"Reinforcement Q-learning control with reward shaping function for swing phase control in a semi-active prosthetic knee.","volume":"14","author":"Hutabarat","year":"2020","journal-title":"Front. Neurorobot."},{"key":"B12","doi-asserted-by":"publisher","first-page":"5594","DOI":"10.1126\/scirobotics.aan5594","article-title":"Robotics for space and marine sciences.","volume":"2","author":"Jacobstein","year":"2017","journal-title":"Sci. Robot."},{"key":"B13","doi-asserted-by":"publisher","first-page":"4545","DOI":"10.1126\/scirobotics.aan4545","article-title":"A robotic device using gecko-inspired adhesives can grasp and manipulate large objects in microgravity.","volume":"2","author":"Jiang","year":"2017","journal-title":"Sci. Robot."},{"key":"B14","doi-asserted-by":"publisher","first-page":"4383","DOI":"10.1126\/scirobotics.aan4383","article-title":"Exploration telepresence a strategy for optimizing scientific research at remote space destinations.","volume":"2","author":"Lester","year":"2017","journal-title":"Sci. Robot."},{"key":"B15","doi-asserted-by":"publisher","first-page":"163","DOI":"10.3390\/aerospace9030163","article-title":"Constrained motion planning of 7-DOF space manipulator via deep reinforcement learning combined with artificial potential field.","volume":"9","author":"Li","year":"2022","journal-title":"Aerospace"},{"key":"B16","doi-asserted-by":"publisher","first-page":"817723","DOI":"10.3389\/fbioe.2021.817723","article-title":"Self-tuning control of manipulator positioning based on fuzzy PID and PSO algorithm.","volume":"9","author":"Liu","year":"2021","journal-title":"Front. Bioeng. Biotechnol."},{"key":"B17","doi-asserted-by":"crossref","DOI":"10.1109\/AERO47225.2020.9172268","article-title":"Design of an AR visor display system for extravehicular activity operations","author":"McHenry","year":"2020","journal-title":"Proceedings of the 2020 IEEE aerospace conference"},{"journal-title":"STS-112 spacewalk.","year":"2014","author":"Mohon","key":"B18"},{"key":"B19","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1016\/j.mechmachtheory.2006.04.004","article-title":"Singularity analysis of the Canadarm2.","volume":"42","author":"Nokleby","year":"2007","journal-title":"Mech. Mach. Theory"},{"key":"B20","doi-asserted-by":"publisher","first-page":"103","DOI":"10.3389\/fnbot.2019.00103","article-title":"Constrained deep Q-learning gradually approaching ordinary q-learning.","volume":"13","author":"Ohnishi","year":"2019","journal-title":"Front. Neurorobot."},{"key":"B21","doi-asserted-by":"publisher","first-page":"559","DOI":"10.1016\/j.actaastro.2021.09.012","article-title":"An in-line damper for tethers-in-space oscillations dissipation.","volume":"189","author":"Olivieri","year":"2021","journal-title":"Acta Astronaut."},{"key":"B22","doi-asserted-by":"publisher","first-page":"530","DOI":"10.1016\/j.actaastro.2021.09.007","article-title":"Modularity for the future in space robotics: A review.","volume":"189","author":"Post","year":"2021","journal-title":"Acta Astronaut."},{"key":"B23","doi-asserted-by":"publisher","first-page":"21","DOI":"10.1016\/j.actaastro.2021.01.034","article-title":"Impact modeling and reactionless control for post-capturing and maneuvering of orbiting objects using a multi-arm space robot.","volume":"182","author":"Raina","year":"2021","journal-title":"Acta Astronaut."},{"key":"B24","article-title":"V-REP: A versatile and scalable robot simulation framework","author":"Rohmer","year":"2013","journal-title":"Proceedings of the IEEE\/RSJ international conference on intelligent robots and systems"},{"key":"B25","doi-asserted-by":"publisher","first-page":"1160","DOI":"10.1111\/ssqu.12469","article-title":"The international space station: Collaboration, utilization, and commercialization*.","volume":"98","author":"Ruttley","year":"2017","journal-title":"Soc. Sci. Q."},{"key":"B26","doi-asserted-by":"publisher","first-page":"1353","DOI":"10.1016\/j.asr.2019.12.006","article-title":"Study on impact energy absorption and adhesion of biomimetic buffer system for space robots.","volume":"65","author":"Su","year":"2020","journal-title":"Adv. Space Res."},{"key":"B27","doi-asserted-by":"publisher","first-page":"909023","DOI":"10.3389\/fbioe.2022.909023","article-title":"Improved multi-stream convolutional block attention module for sEMG-based gesture recognition.","volume":"10","author":"Wang","year":"2022","journal-title":"Front. Bioeng. Biotechnol."},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2022.3144569","article-title":"Robust adaptive learning control of space robot for target capturing using neural network.","author":"Wang","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"B29","doi-asserted-by":"publisher","first-page":"138","DOI":"10.1016\/j.actaastro.2012.04.038","article-title":"Space tourism: Its history, future and importance.","volume":"92","author":"Webber","year":"2013","journal-title":"Acta Astronaut."},{"key":"B30","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1109\/tmech.2019.2952552","article-title":"Effective capture of nongraspable objects for space robots using geometric cage pairs.","volume":"25","author":"Zhang","year":"","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"B31","doi-asserted-by":"publisher","first-page":"2449","DOI":"10.1007\/s11071-020-05615-5","article-title":"Adaptive robust decoupling control of multi-arm space robots using time-delay estimation technique.","volume":"100","author":"Zhang","year":"","journal-title":"Nonlinear Dyn."},{"key":"B32","doi-asserted-by":"publisher","first-page":"852408","DOI":"10.3389\/fbioe.2022.852408","article-title":"Time optimal trajectory planning based on improved sparrow search algorithm.","volume":"10","author":"Zhang","year":"2022","journal-title":"Front. Bioeng. Biotechnol."},{"key":"B33","doi-asserted-by":"publisher","first-page":"832829","DOI":"10.3389\/fbioe.2022.832829","article-title":"A Tandem robotic arm inverse kinematic solution based on an improved particle swarm algorithm.","volume":"10","author":"Zhao","year":"2022","journal-title":"Front. Bioeng. Biotechnol."},{"key":"B34","doi-asserted-by":"publisher","first-page":"6305","DOI":"10.3390\/s21186305","article-title":"Modular robotic limbs for astronaut activities assistance.","volume":"21","author":"Zhao","year":"2021","journal-title":"Sensors (Basel)"},{"key":"B35","doi-asserted-by":"publisher","first-page":"308","DOI":"10.1109\/tro.2007.894685","article-title":"Evolved and designed self-reproducing modular robotics.","volume":"23","author":"Zykov","year":"2007","journal-title":"IEEE Trans. Robot."}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1093718\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,15]],"date-time":"2023-02-15T09:11:17Z","timestamp":1676452277000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1093718\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,15]]},"references-count":35,"alternative-id":["10.3389\/fnbot.2023.1093718"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1093718","relation":{},"ISSN":["1662-5218"],"issn-type":[{"type":"electronic","value":"1662-5218"}],"subject":[],"published":{"date-parts":[[2023,2,15]]},"article-number":"1093718"}}