{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,8]],"date-time":"2026-01-08T01:16:19Z","timestamp":1767834979769,"version":"3.49.0"},"reference-count":36,"publisher":"MDPI AG","issue":"16","license":[{"start":{"date-parts":[[2021,8,5]],"date-time":"2021-08-05T00:00:00Z","timestamp":1628121600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Research Foundation of Korea","award":["NRF-2019R1A2C1003713"],"award-info":[{"award-number":["NRF-2019R1A2C1003713"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Anthropomorphic robotic hands are designed to attain dexterous movements and flexibility much like human hands. Achieving human-like object manipulation remains a challenge especially due to the control complexity of the anthropomorphic robotic hand with a high degree of freedom. In this work, we propose a deep reinforcement learning (DRL) to train a policy using a synergy space for generating natural grasping and relocation of variously shaped objects using an anthropomorphic robotic hand. A synergy space is created using a continuous normalizing flow network with point clouds of haptic areas, representing natural hand poses obtained from human grasping demonstrations. The DRL policy accesses the synergistic representation and derives natural hand poses through a deep regressor for object grasping and relocation tasks. Our proposed synergy-based DRL achieves an average success rate of 88.38% for the object manipulation tasks, while the standard DRL without synergy space only achieves 50.66%. Qualitative results show the proposed synergy-based DRL policy produces human-like finger placements over the surface of each object including apple, banana, flashlight, camera, lightbulb, and hammer.<\/jats:p>","DOI":"10.3390\/s21165301","type":"journal-article","created":{"date-parts":[[2021,8,5]],"date-time":"2021-08-05T09:35:32Z","timestamp":1628156132000},"page":"5301","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Object Manipulation with an Anthropomorphic Robotic Hand via Deep Reinforcement Learning with a Synergy Space of Natural Hand Poses"],"prefix":"10.3390","volume":"21","author":[{"given":"Patricio","family":"Rivera","sequence":"first","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Korea"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0077-8528","authenticated-orcid":false,"given":"Edwin","family":"Valarezo A\u00f1azco","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Korea"}]},{"given":"Tae-Seong","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Electronics and Information Convergence Engineering, Kyung Hee University, Yongin 17104, Korea"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Billard, A., and Kragic, D. (2019). Trends and Challenges in Robot Manipulation. Science, 364.","DOI":"10.1126\/science.aat8414"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"477","DOI":"10.1613\/jair.3229","article-title":"A Probabilistic Framework for Learning Kinematic Models of Articulated Objects","volume":"41","author":"Sturm","year":"2011","journal-title":"JAIR"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1108\/01439910510573237","article-title":"Shadow Delivers First Hand","volume":"32","author":"Kochan","year":"2005","journal-title":"Ind. Robot."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kumar, V., Xu, Z., and Todorov, E. (2013, January 6\u201310). Fast, Strong and Compliant Pneumatic Actuation for Dexterous Tendon-Driven Hands. Proceedings of the 2013 IEEE International Conference on Robotics and Automation, Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6630771"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1177\/0278364915592961","article-title":"A Novel Type of Compliant and Underactuated Robotic Hand for Dexterous Grasping","volume":"35","author":"Deimel","year":"2016","journal-title":"Int. J. Robot. Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1177\/0278364919887447","article-title":"Learning Dexterous In-Hand Manipulation","volume":"39","author":"Andrychowicz","year":"2020","journal-title":"Int. J. Robot. Res."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zeng, A., Yu, K.-T., Song, S., Suo, D., Walker, E., Rodriguez, A., and Xiao, J. (June, January 29). Multi-View Self-Supervised Deep Learning for 6D Pose Estimation in the Amazon Picking Challenge. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989165"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zeng, A., Song, S., Yu, K.-T., Donlon, E., Hogan, F.R., Bauza, M., Ma, D., Taylor, O., Liu, M., and Romo, E. (2018, January 21\u201326). Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, QLD, Australia.","DOI":"10.1109\/ICRA.2018.8461044"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Van Hoof, H., Hermans, T., Neumann, G., and Peters, J. (2015, January 3\u20135). Learning Robot In-Hand Manipulation with Tactile Features. Proceedings of the 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids), Seoul, Korea.","DOI":"10.1109\/HUMANOIDS.2015.7363524"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Kumra, S., and Kanan, C. (2017, January 24\u201328). Robotic Grasp Detection Using Deep Convolutional Neural Networks. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8202237"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"178450","DOI":"10.1109\/ACCESS.2020.3027923","article-title":"Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations","volume":"8","author":"Mohammed","year":"2020","journal-title":"IEEE Access"},{"key":"ref_12","first-page":"1","article-title":"A Review of Robot Learning for Manipulation: Challenges, Representations, and Algorithms","volume":"22","author":"Kroemer","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Hester, T., Vecerik, M., Pietquin, O., Lanctot, M., Schaul, T., Piot, B., Horgan, D., Quan, J., Sendonaris, A., and Dulac-Arnold, G. (2017). Deep Q-Learning from Demonstrations. arXiv.","DOI":"10.1609\/aaai.v32i1.11757"},{"key":"ref_14","unstructured":"Subramanian, K., Isbell, C.L., and Thomaz, A.L. (2016, January 9\u201313). Exploration from Demonstration for Interactive Reinforcement Learning. Proceedings of the International Conference on Agents and Multiagent Systems, Singapore."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Rajeswaran, A., Kumar, V., Gupta, A., Vezzani, G., Schulman, J., Todorov, E., and Levine, S. (2017). Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations. arXiv.","DOI":"10.15607\/RSS.2018.XIV.049"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2710","DOI":"10.1109\/TSMC.2019.2901955","article-title":"Learning to Grasp Familiar Objects Based on Experience and Objects\u2019 Shape Affordance","volume":"49","author":"Liu","year":"2019","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1533","DOI":"10.1109\/LRA.2019.2896485","article-title":"Learning From Humans How to Grasp: A Data-Driven Architecture for Autonomous Grasping With Anthropomorphic Soft Hands","volume":"4","author":"Santina","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Ji, S.-Q., Huang, M.-B., and Huang, H.-P. (2019). Robot Intelligent Grasp of Unknown Objects Based on Multi-Sensor Information. Sensors, 19.","DOI":"10.3390\/s19071595"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"955","DOI":"10.1080\/01691864.2018.1509018","article-title":"Hierarchical Reinforcement Learning of Multiple Grasping Strategies with Human Instructions","volume":"32","author":"Osa","year":"2018","journal-title":"Adv. Robot."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"10105","DOI":"10.1523\/JNEUROSCI.18-23-10105.1998","article-title":"Postural Hand Synergies for Tool Use","volume":"18","author":"Santello","year":"1998","journal-title":"J. Neurosci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1007\/s11370-018-0262-0","article-title":"Hand-Arm Autonomous Grasping: Synergistic Motions to Enhance the Learning Process","volume":"12","author":"Ficuciello","year":"2019","journal-title":"Intell. Serv. Robot."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"3153","DOI":"10.1098\/rstb.2011.0152","article-title":"Modelling Natural and Artificial Hands with Synergies","volume":"366","author":"Bicchi","year":"2011","journal-title":"Phil. Trans. R. Soc. B"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"768","DOI":"10.1177\/0278364913518998","article-title":"Adaptive Synergies for the Design and Control of the Pisa\/IIT SoftHand","volume":"33","author":"Catalano","year":"2014","journal-title":"Int. J. Robot. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1030","DOI":"10.1080\/01691864.2017.1365011","article-title":"Grasp and Dexterous Manipulation of Multi-Fingered Robotic Hands: A Review from a Control View Point","volume":"31","author":"Ozawa","year":"2017","journal-title":"Adv. Robot."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.plrev.2016.02.001","article-title":"Hand Synergies: Integration of Robotics and Neuroscience for Understanding the Control of Biological and Artificial Hands","volume":"17","author":"Santello","year":"2016","journal-title":"Phys. Life Rev."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Todorov, E., Erez, T., and Tassa, Y. (2012, January 7\u201312). MuJoCo: A Physics Engine for Model-Based Control. Proceedings of the 2012 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Vilamoura-Algarve, Portugal.","DOI":"10.1109\/IROS.2012.6386109"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1144","DOI":"10.1109\/TII.2018.2841043","article-title":"Synergy-Based Control of Underactuated Anthropomorphic Hands","volume":"15","author":"Ficuciello","year":"2019","journal-title":"IEEE Trans. Ind. Inf."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhou, J., Chen, X., Chang, U., Liu, Y., Chen, Y., and Wang, Z. (2019, January 14\u201318). A Grasping Component Mapping Approach for Soft Robotic End-Effector Control. Proceedings of the 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft), Seoul, Korea.","DOI":"10.1109\/ROBOSOFT.2019.8722773"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Brahmbhatt, S., Ham, C., Kemp, C.C., and Hays, J. (2019, January 15\u201320). ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00891"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Brahmbhatt, S., Handa, A., Hays, J., and Fox, D. (2019, January 3\u20138). ContactGrasp: Functional Multi-Finger Grasp Synthesis from Contact. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macau, China.","DOI":"10.1109\/IROS40897.2019.8967960"},{"key":"ref_31","first-page":"4743","article-title":"Improved Variational Inference with Inverse Autoregressive Flow","volume":"29","author":"Kingma","year":"2016","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_32","unstructured":"Rezende, D.J., and Mohamed, S. (2015, January 6\u201311). Variational Inference with Normalizing Flows. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Charles, R.Q., Su, H., Kaichun, M., and Guibas, L.J. (2017, January 21\u201326). PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.16"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Yang, G., Huang, X., Hao, Z., Liu, M.-Y., Belongie, S., and Hariharan, B. (November, January 27). PointFlow: 3D Point Cloud Generation with Continuous Normalizing Flows. Proceedings of the 2019 IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00464"},{"key":"ref_35","unstructured":"Chen, R.T.Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D. (2018, January 3\u20138). Neural Ordinary Differential Equations. Proceedings of the Advances in Neural Information Processing Systems, Montr\u00e9al, QC, Canada."},{"key":"ref_36","unstructured":"Kakade, S.M. (2001, January 9\u201314). A Natural Policy Gradient. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/16\/5301\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:41:10Z","timestamp":1760164870000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/16\/5301"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,5]]},"references-count":36,"journal-issue":{"issue":"16","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["s21165301"],"URL":"https:\/\/doi.org\/10.3390\/s21165301","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,8,5]]}}}