{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,9]],"date-time":"2026-02-09T11:26:08Z","timestamp":1770636368512,"version":"3.49.0"},"reference-count":78,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NSERC Discovery Grants Program","award":["RGPIN-06797 and RGPAS-522723"],"award-info":[{"award-number":["RGPIN-06797 and RGPAS-522723"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:p>\n            Learning dexterous manipulation skills is a long-standing challenge in computer graphics and robotics, especially when the task involves complex and delicate interactions between the hands, tools and objects. In this paper, we focus on chopsticks-based object relocation tasks, which are common yet demanding. The key to successful chopsticks skills is steady gripping of the sticks that also supports delicate maneuvers. We automatically discover physically valid chopsticks holding poses by Bayesian Optimization (BO) and Deep Reinforcement Learning (DRL), which works for multiple gripping styles and hand morphologies without the need of example data. Given as input the discovered gripping poses and desired objects to be moved, we build physics-based hand controllers to accomplish relocation tasks in two stages. First, kinematic trajectories are synthesized for the chopsticks and hand in a motion planning stage. The key components of our motion planner include a grasping model to select suitable chopsticks configurations for grasping the object, and a trajectory optimization module to generate collision-free chopsticks trajectories. Then we train physics-based hand controllers through DRL again to track the desired kinematic trajectories produced by the motion planner. We demonstrate the capabilities of our framework by relocating objects of various shapes and sizes, in diverse gripping styles and holding positions for multiple hand morphologies. Our system achieves faster learning speed and better control robustness, when compared to vanilla systems that attempt to learn chopstick-based skills without a gripping pose optimization module and\/or without a kinematic motion planner. Our code and models are available at this link.\n            <jats:sup>1<\/jats:sup>\n          <\/jats:p>","DOI":"10.1145\/3528223.3530057","type":"journal-article","created":{"date-parts":[[2022,7,22]],"date-time":"2022-07-22T21:06:27Z","timestamp":1658523987000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":27,"title":["Learning to use chopsticks in diverse gripping styles"],"prefix":"10.1145","volume":"41","author":[{"given":"Zeshi","family":"Yang","sequence":"first","affiliation":[{"name":"Simon Fraser University, Canada and CFCS Peking University, China"}]},{"given":"Kangkang","family":"Yin","sequence":"additional","affiliation":[{"name":"Simon Fraser University, Canada"}]},{"given":"Libin","family":"Liu","sequence":"additional","affiliation":[{"name":"CFCS Peking University, China"}]}],"member":"320","published-online":{"date-parts":[[2022,7,22]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Ilge Akkaya Marcin Andrychowicz Maciek Chociej Mateusz Litwin Bob McGrew Arthur Petron Alex Paino Matthias Plappert Glenn Powell Raphael Ribas et al. 2019. Solving rubik's cube with a robot hand. arXiv preprint arXiv:1910.07113 (2019)."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cag.2013.04.007"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0097-8493(98)00122-8"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356536"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/1921427.1921443"},{"key":"e_1_2_2_6_1","first-page":"10","article-title":"Preference galleries for material design","volume":"105","author":"Brochu Eric","year":"2007","unstructured":"Eric Brochu, Abhijeet Ghosh, and Nando de Freitas. 2007. Preference galleries for material design. SIGGRAPH Posters 105, 10.1145 (2007), 1280720--1280834.","journal-title":"SIGGRAPH Posters"},{"key":"e_1_2_2_7_1","volume-title":"The pincer chopsticks: The investigation of a new utensil in pinching function. Applied ergonomics 38, 3","author":"Chang Bao-Chi","year":"2007","unstructured":"Bao-Chi Chang, Biing-Shiun Huang, Ching-Kong Chen, and Shyh-Jen Wang. 2007. The pincer chopsticks: The investigation of a new utensil in pinching function. Applied ergonomics 38, 3 (2007), 385--390."},{"key":"e_1_2_2_8_1","volume-title":"Conference on Robot Learning. PMLR.","author":"Chen Tao","year":"2021","unstructured":"Tao Chen, Jie Xu, and Pulkit Agrawal. 2021. A system for general in-hand object re-orientation. In Conference on Robot Learning. PMLR."},{"key":"e_1_2_2_9_1","volume-title":"Proceedings of the 2003 ACM SIGGRAPH\/Eurographics Symposium on Computer Animation. 110--119","author":"ElKoura George","year":"2003","unstructured":"George ElKoura and Karan Singh. 2003. Handrix: animating the human hand. In Proceedings of the 2003 ACM SIGGRAPH\/Eurographics Symposium on Computer Animation. 110--119."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01146"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364919872545"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS45743.2020.9340947"},{"key":"e_1_2_2_13_1","volume-title":"since","year":"2012","unstructured":"GPy. since 2012. GPy: A Gaussian process framework in python. http:\/\/github.com\/SheffieldML\/GPy."},{"key":"e_1_2_2_14_1","first-page":"1809","article-title":"Entropy search for information-efficient global optimization","volume":"13","author":"Hennig Philipp","year":"2012","unstructured":"Philipp Hennig and Christian J Schuler. 2012. Entropy search for information-efficient global optimization. The Journal of Machine Learning Research 13, 1 (2012), 1809--1837.","journal-title":"The Journal of Machine Learning Research"},{"key":"e_1_2_2_15_1","doi-asserted-by":"crossref","unstructured":"Daniel Holden Jun Saito Taku Komura and Thomas Joyce. 2015. Learning motion manifolds with convolutional autoencoders. In SIGGRAPH Asia 2015 Technical Briefs. 1--4.","DOI":"10.1145\/2820903.2820918"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9561733"},{"key":"e_1_2_2_17_1","volume-title":"Nadia Magnenat Thalmann, and Daniel Thalmann","author":"Huang Zhiyong","year":"1995","unstructured":"Zhiyong Huang, Ronan Boulic, Nadia Magnenat Thalmann, and Daniel Thalmann. 1995. A multi-sensor approach for grasping and 3D interaction. In Computer graphics. Elsevier, 235--253."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024197"},{"key":"e_1_2_2_19_1","volume-title":"A novel technique improves surgeon performance and eliminates arm collision in robotic single-incision laparoscopic surgery. Surgical endoscopy 24, 6","author":"Joseph Rohan A","year":"2010","unstructured":"Rohan A Joseph, Alvin C Goh, Sebastian P Cuevas, Michael A Donovan, Matthew G Kauffman, Nilson A Salas, Brian Miles, Barbara L Bass, and Brian J Dunkin. 2010. \"Chopstick\" surgery: A novel technique improves surgeon performance and eliminates arm collision in robotic single-incision laparoscopic surgery. Surgical endoscopy 24, 6 (2010), 1331--1335."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV50981.2020.00043"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS45743.2020.9341529"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9561662"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CGI.2000.852318"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2014.2316022"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-021-27261-0"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/192161.192266"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392444"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141969"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2010.2103055"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/1632592.1632616"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531365"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3083723"},{"key":"e_1_2_2_33_1","volume-title":"since","year":"2021","unstructured":"Macro. since 2021. Ten thousand ways to use chopsticks. https:\/\/marcosticks.org\/poster-ten-thousand-ways-to-use-chopsticks\/."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392474"},{"key":"e_1_2_2_35_1","volume-title":"a research-toolkit for particle swarm optimization in Python. Journal of Open Source Software 3","author":"Miranda Lester James V.","year":"2018","unstructured":"Lester James V. Miranda. 2018. PySwarms, a research-toolkit for particle swarm optimization in Python. Journal of Open Source Software 3 (2018)."},{"key":"e_1_2_2_36_1","volume-title":"Proceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Animation. 137--144","author":"Mordatch Igor","year":"2012","unstructured":"Igor Mordatch, Zoran Popovi\u0107, and Emanuel Todorov. 2012. Contact-invariant optimization for hand manipulation. In Proceedings of the ACM SIGGRAPH\/Eurographics Symposium on Computer Animation. 137--144."},{"key":"e_1_2_2_37_1","first-page":"467","article-title":"A study on ways of holding chopsticks","volume":"29","author":"Mukai Yukiko","year":"1978","unstructured":"Yukiko Mukai and Keiko Hashimoto. 1978. A study on ways of holding chopsticks. Journal of Home Economics of Japan 29, 7 (1978), 467--473.","journal-title":"Journal of Home Economics of Japan"},{"key":"e_1_2_2_38_1","volume-title":"Conference on Robot Learning. PMLR, 1101--1112","author":"Nagabandi Anusha","year":"2020","unstructured":"Anusha Nagabandi, Kurt Konolige, Sergey Levine, and Vikash Kumar. 2020. Deep dynamics models for learning dexterous manipulation. In Conference on Robot Learning. PMLR, 1101--1112."},{"key":"e_1_2_2_39_1","first-page":"19","article-title":"Relationship between mothers and children on how to hold chopsticks and concerns about chopsticks in japanese kindergarten","volume":"12","author":"Osera Tomoko","year":"2018","unstructured":"Tomoko Osera, Chihiro Yamamoto, Rika Senke, Misako Kobayashi, Setsuko Tsutie, and Nobutaka Kurihara. 2018. Relationship between mothers and children on how to hold chopsticks and concerns about chopsticks in japanese kindergarten. Journal of Japanese Society of Shokuiku 12, 1 (2018), 19--25.","journal-title":"Journal of Japanese Society of Shokuiku"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356501"},{"key":"e_1_2_2_41_1","unstructured":"Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in pytorch. (2017)."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201311"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8460528"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073602"},{"key":"e_1_2_2_45_1","volume-title":"MCP: Learning composable hierarchical control with multiplicative compositional policies. Advances in Neural Information Processing Systems 32","author":"Peng Xue Bin","year":"2019","unstructured":"Xue Bin Peng, Michael Chang, Grace Zhang, Pieter Abbeel, and Sergey Levine. 2019. MCP: Learning composable hierarchical control with multiplicative compositional policies. Advances in Neural Information Processing Systems 32 (2019)."},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2020.XVI.064"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459670"},{"key":"e_1_2_2_48_1","volume-title":"Data-efficient deep reinforcement learning for dexterous manipulation. arXiv preprint arXiv:1704.03073","author":"Popov Ivaylo","year":"2017","unstructured":"Ivaylo Popov, Nicolas Heess, Timothy Lillicrap, Roland Hafner, Gabriel Barth-Maron, Matej Vecerik, Thomas Lampe, Yuval Tassa, Tom Erez, and Martin Riedmiller. 2017. Data-efficient deep reinforcement learning for dexterous manipulation. arXiv preprint arXiv:1704.03073 (2017)."},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1002\/rcs.346"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8461237"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2018.XIV.049"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIE.2008.2008753"},{"key":"e_1_2_2_53_1","volume-title":"Summer School on Machine Learning","author":"Rasmussen Carl Edward","unstructured":"Carl Edward Rasmussen. 2003. Gaussian processes in machine learning. In Summer School on Machine Learning. Springer, 63--71."},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487604"},{"key":"e_1_2_2_55_1","volume-title":"International conference on machine learning. PMLR","author":"Schulman John","year":"2015","unstructured":"John Schulman, Sergey Levine, Pieter Abbeel, Michael Jordan, and Philipp Moritz. 2015. Trust region policy optimization. In International conference on machine learning. PMLR, 1889--1897."},{"key":"e_1_2_2_56_1","volume-title":"Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347","author":"Schulman John","year":"2017","unstructured":"John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. 2017. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)."},{"key":"e_1_2_2_57_1","volume-title":"Proceedings of the 27th annual international conference on machine learning. 1015--1022","author":"Srinivas Niranjan","year":"2010","unstructured":"Niranjan Srinivas, Andreas Krause, Sham M Kakade, and Matthias Seeger. 2010. Gaussian process optimization in the bandit setting: No regret and experimental design. In Proceedings of the 27th annual international conference on machine learning. 1015--1022."},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58548-8_34"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2018.XIV.010"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6907751"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2012.6386109"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1162\/pres.1996.5.4.393"},{"key":"e_1_2_2_63_1","volume-title":"Kevin A Smith, and Joshua B Tenenbaum.","author":"Toussaint Marc A","year":"2018","unstructured":"Marc A Toussaint, Kelsey Rebecca Allen, Kevin A Smith, and Joshua B Tenenbaum. 2018. Differentiable physics and stable modes for tool-use and manipulation planning. (2018)."},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2462000"},{"key":"e_1_2_2_65_1","volume-title":"Computer Graphics Forum","author":"Wheatland Nkenge","unstructured":"Nkenge Wheatland, Yingying Wang, Huaguang Song, Michael Neff, Victor Zordan, and Sophie J\u00f6rg. 2015. State of the art in hand and finger modeling and animation. In Computer Graphics Forum, Vol. 34. Wiley Online Library, 735--760."},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356499"},{"key":"e_1_2_2_67_1","volume-title":"Learning to manipulate deformable objects without demonstrations. arXiv preprint arXiv:1910.13439","author":"Wu Yilin","year":"2019","unstructured":"Yilin Wu, Wilson Yan, Thanard Kurutach, Lerrel Pinto, and Pieter Abbeel. 2019. Learning to manipulate deformable objects without demonstrations. arXiv preprint arXiv:1910.13439 (2019)."},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/IWAIT.2018.8369662"},{"key":"e_1_2_2_69_1","first-page":"260","article-title":"Effect of parental training on table manners and the way of holding chopsticks","volume":"43","author":"Yamauchi Tomoko","year":"2010","unstructured":"Tomoko Yamauchi, Atsumi Koide, Atsuko Yamamoto, and Kazuko Oba. 2010. Effect of parental training on table manners and the way of holding chopsticks. Journal of Cookery Science of Japan 43, 4 (2010), 260--264.","journal-title":"Journal of Cookery Science of Japan"},{"key":"e_1_2_2_70_1","volume-title":"ROBOTIK 2012; 7th German Conference on Robotics. VDE, 1--6.","author":"Yamazaki Akira","year":"2012","unstructured":"Akira Yamazaki and Ryosuke Masuda. 2012. Autonomous foods handling by chopsticks for meal assistant robot. In ROBOTIK 2012; 7th German Conference on Robotics. VDE, 1--6."},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3451254"},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185537"},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459817"},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2020.2988642"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00886"},{"key":"e_1_2_2_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459830"},{"key":"e_1_2_2_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417868"},{"key":"e_1_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508412"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530057","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530057","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:25Z","timestamp":1750186945000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530057"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":78,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10.1145\/3528223.3530057"],"URL":"https:\/\/doi.org\/10.1145\/3528223.3530057","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7]]},"assertion":[{"value":"2022-07-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}