{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T04:35:42Z","timestamp":1787027742805,"version":"build-2736575974"},"reference-count":32,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2023,8,15]],"date-time":"2023-08-15T00:00:00Z","timestamp":1692057600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"JSPS Grant-in-Aid for Scientific Research on Innovative Areas \u201cHyper-Adaptability\u201d project","award":["22H04764"],"award-info":[{"award-number":["22H04764"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Biomimetics"],"abstract":"<jats:p>The lack of intuitive controllability remains a primary challenge in enabling transhumeral amputees to control a prosthesis for arm reaching with residual limb kinematics. Recent advancements in prosthetic arm control have focused on leveraging the predictive capabilities of artificial neural networks (ANNs) to automate elbow joint motion and wrist pronation\u2013supination during target reaching tasks. However, large quantities of human motion data collected from different subjects for various activities of daily living (ADL) tasks are required to train these ANNs. For example, the reaching motion can be altered when the height of the desk is changed; however, it is cumbersome to conduct human experiments for all conditions. This paper proposes a framework for cloning motion datasets using deep reinforcement learning (DRL) to cater to training data requirements. DRL algorithms have been demonstrated to create human-like synergistic motion in humanoid agents to handle redundancy and optimize movements. In our study, we collected real motion data from six individuals performing multi-directional arm reaching tasks in the horizontal plane. We generated synthetic motion data that mimicked similar arm reaching tasks by utilizing a physics simulation and DRL-based arm manipulation. We then trained a CNN-LSTM network with different configurations of training motion data, including DRL, real, and hybrid datasets, to test the efficacy of the cloned motion data. The results of our evaluation showcase the effectiveness of the cloned motion data in training the ANN to predict natural elbow motion accurately across multiple subjects. Furthermore, motion data augmentation through combining real and cloned motion datasets has demonstrated the enhanced robustness of the ANN by supplementing and diversifying the limited training data. These findings have significant implications for creating synthetic dataset resources for various arm movements and fostering strategies for automatized prosthetic elbow motion.<\/jats:p>","DOI":"10.3390\/biomimetics8040367","type":"journal-article","created":{"date-parts":[[2023,8,15]],"date-time":"2023-08-15T11:09:44Z","timestamp":1692097784000},"page":"367","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Transhumeral Arm Reaching Motion Prediction through Deep Reinforcement Learning-Based Synthetic Motion Cloning"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1896-9592","authenticated-orcid":false,"given":"Muhammad Hannan","family":"Ahmed","sequence":"first","affiliation":[{"name":"Department of Robotics, Graduate School of Engineering, Tohoku University, Sendai 980-8579, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5326-7847","authenticated-orcid":false,"given":"Kyo","family":"Kutsuzawa","sequence":"additional","affiliation":[{"name":"Department of Robotics, Graduate School of Engineering, Tohoku University, Sendai 980-8579, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6179-5706","authenticated-orcid":false,"given":"Mitsuhiro","family":"Hayashibe","sequence":"additional","affiliation":[{"name":"Department of Robotics, Graduate School of Engineering, Tohoku University, Sendai 980-8579, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,8,15]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"492","DOI":"10.1177\/0309364613506913","article-title":"The DEKA Arm: Its features, functionality, and evolution during the Veterans Affairs Study to optimize the DEKA Arm","volume":"38","author":"Resnik","year":"2014","journal-title":"Prosthetics Orthot. Int."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"153","DOI":"10.3389\/fneur.2018.00153","article-title":"Initial Clinical Evaluation of the Modular Prosthetic Limb","volume":"9","author":"Perry","year":"2018","journal-title":"Front. Neurol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"6878","DOI":"10.1109\/ACCESS.2018.2791583","article-title":"A review of non-invasive sensory feedback methods for transradial prosthetic hands","volume":"6","author":"Alici","year":"2018","journal-title":"IEEE Access"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2029","DOI":"10.1016\/j.apmr.2012.03.011","article-title":"Characterization of compensatory trunk movements during prosthetic upper limb reaching tasks","volume":"93","author":"Metzger","year":"2012","journal-title":"Arch. Phys. Med. Rehabil."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1055\/s-0035-1544166","article-title":"Targeted muscle reinnervation and advanced prosthetic arms","volume":"29","author":"Cheesborough","year":"2015","journal-title":"Semin. Plastic Surg."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1109\/TMRB.2021.3072253","article-title":"Quantifying Prosthetic and Intact Limb Use in Upper Limb Amputees via Egocentric Video: An Unsupervised, At-Home Study","volume":"3","author":"Spiers","year":"2021","journal-title":"IEEE Trans. Med. Robot. Bionics"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1109\/MSPEC.2022.9915629","article-title":"The Bionic-Hand Arms Race: High-Tech Hands are Complicated, Costly, and Often Impractical","volume":"59","author":"Young","year":"2022","journal-title":"IEEE Spectr."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"65","DOI":"10.3389\/fnbot.2019.00065","article-title":"Reachy, a 3D-Printed Human-Like Robotic Arm as a Testbed for Human\u2013Robot Control Strategies","volume":"13","author":"Cheesborough","year":"2019","journal-title":"Front. Neurorobot."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"573","DOI":"10.1109\/TCDS.2017.2697904","article-title":"Synergetic learning control paradigm for redundant robot to enhance error-energy index","volume":"10","author":"Hayashibe","year":"2018","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ahmed, M.H., Chai, J., Shimoda, S., and Hayashibe, M. (2023). Synergy-Space Recurrent Neural Network for Transferable Forearm Motion Prediction from Residual Limb Motion. Sensors, 23.","DOI":"10.3390\/s23094188"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1109\/TMRB.2020.2970065","article-title":"Assessment of an Automatic Prosthetic Elbow Control Strategy Using Residual Limb Motion for Transhumeral Amputated Individuals with Socket or Osseointegrated Prostheses","volume":"2","author":"Merad","year":"2020","journal-title":"IEEE Trans. Med. Robot. Bionics"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"S445","DOI":"10.1080\/10255842.2020.1714976","article-title":"Improving the control of prostheses in arm amputees with approaches based on motor coordination","volume":"22","author":"Merad","year":"2019","journal-title":"Comput. Methods Biomech. Biomed. Eng."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1080\/00222895.1984.10735316","article-title":"Neuromotor Synergies as a Basis for Coordinated Intentional Action","volume":"16","author":"Lee","year":"1984","journal-title":"J. Mot. Behav."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"10105","DOI":"10.1523\/JNEUROSCI.18-23-10105.1998","article-title":"Postural Hand Synergies for Tool Use","volume":"18","author":"Santello","year":"1998","journal-title":"J. Neurosci."},{"key":"ref_15","first-page":"42","article-title":"Control of reaching movements by muscle synergy combinations","volume":"7","author":"Lacquaniti","year":"2013","journal-title":"Front. Comput. Neurosci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3076","DOI":"10.1073\/pnas.0500199102","article-title":"Shared and specific muscle synergies in natural motor behaviors","volume":"102","author":"Bizzi","year":"2005","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"48","DOI":"10.3389\/fncom.2013.00048","article-title":"Common muscle synergies for balance and walking","volume":"7","author":"Chvatal","year":"2013","journal-title":"Front. Comput. Neurosci."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"211721","DOI":"10.1098\/rsos.211721","article-title":"Motor synergy generalization framework for new targets in multi-planar and multi-directional reaching task","volume":"9","author":"Kutsuzawa","year":"2022","journal-title":"R. Soc. Open Sci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"482","DOI":"10.1109\/TNSRE.2005.858458","article-title":"Automatic determination of synergies by radial basis function artificial neural networks for the control of a neural prosthesis","volume":"13","author":"Iftime","year":"2005","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Popovic, D.B., Popovic, M.B., and Sinkjaer, T. (2006, January 17\u201318). Life-like Control for Neural Prostheses: \u201cProximal Controls Distal\u201d. Proceedings of the 2005 IEEE Engineering in Medicine and Biology 27th Annual Conference, Shanghai, China.","DOI":"10.1109\/IEMBS.2005.1616283"},{"key":"ref_21","first-page":"573","article-title":"Intuitive prosthetic control using upper limb inter-joint coordinations and IMU-based shoulder angles measurement: A pilot study","volume":"10","author":"Merad","year":"2018","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Merad, M., Roby-Brami, A., and Jarrass\u00e9, N. (2016, January 26\u201329). Towards the implementation of natural prosthetic elbow motion using upper limb joint coordination. Proceedings of the 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), Singapore.","DOI":"10.1109\/BIOROB.2016.7523729"},{"key":"ref_23","first-page":"573","article-title":"Estimation of prosthetic arm motions using stump arm kinematics","volume":"10","author":"Dasanayake","year":"2018","journal-title":"IEEE Trans. Cogn. Dev. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.jelekin.2017.06.001","article-title":"Estimation of distal arm joint angles from EMG and shoulder orientation for transhumeral prostheses","volume":"35","author":"Akhtar","year":"2017","journal-title":"J. Electromyogr. Kinesiol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"498","DOI":"10.1109\/TMRB.2021.3056924","article-title":"Synergy Emergence in Deep Reinforcement Learning for Full-Dimensional Arm Manipulation","volume":"3","author":"Han","year":"2021","journal-title":"IEEE Trans. Med. Robot. Bionics"},{"key":"ref_26","first-page":"7791","article-title":"Control of Fast-Reaching Movements by Muscle Synergy Combinations","volume":"26","author":"Portone","year":"2006","journal-title":"IEEE Trans. Med. Robot. Bionics"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1109\/LRA.2020.2968067","article-title":"Motor Synergy Development in High-Performing Deep Reinforcement Learning Algorithms","volume":"5","author":"Chai","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Todorov, E., Erez, T., and Tassa, Y. (2012, January 7\u201312). MuJoCo: A physics engine for model-based control. Proceedings of the IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6386109"},{"key":"ref_29","unstructured":"Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. (2018, January 10\u201315). Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor. Proceedings of the 35th International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"877","DOI":"10.1007\/s00500-021-06238-7","article-title":"Skeleton-based human activity recognition using ConvLSTM and guided feature learning","volume":"26","author":"Yadav","year":"2022","journal-title":"Soft Comput."},{"key":"ref_31","unstructured":"Pascanu, R., Gulcehre, C., Cho, K., and Bengio, Y. (2014, January 14\u201316). How to Construct Deep Recurrent Neural Networks. Proceedings of the International Conference on Learning Representations (ICLR), Banff, AB, Canada."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1217","DOI":"10.1109\/JPROC.2008.922591","article-title":"Prediction of Distal Arm Posture in 3-D Space from Shoulder Movements for Control of Upper Limb Prostheses","volume":"96","author":"Kaliki","year":"2008","journal-title":"Proc. IEEE"}],"container-title":["Biomimetics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-7673\/8\/4\/367\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T20:33:58Z","timestamp":1760128438000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-7673\/8\/4\/367"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,15]]},"references-count":32,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2023,8]]}},"alternative-id":["biomimetics8040367"],"URL":"https:\/\/doi.org\/10.3390\/biomimetics8040367","relation":{},"ISSN":["2313-7673"],"issn-type":[{"value":"2313-7673","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,8,15]]}}}