{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T22:05:37Z","timestamp":1776204337414,"version":"3.50.1"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T00:00:00Z","timestamp":1731974400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS-1911224"],"award-info":[{"award-number":["IIS-1911224"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2024,12,19]]},"abstract":"<jats:p>We present a hard-real-time multi-resolution mesh shape deformation technique for skeleton-driven soft-body characters. Producing mesh deformations at multiple levels of detail is very important in many applications in computer graphics. Our work targets applications where the multi-resolution shapes must be generated at fast speeds (\"hard-real-time\", e.g., a few milliseconds at most and preferably under 1 millisecond), as commonly needed in computer games, virtual reality and Metaverse applications. We assume that the character mesh is driven by a skeleton, and that high-quality character shapes are available in a set of training poses originating from a high-quality (slow) rig such as volumetric FEM simulation. Our method combines multi-resolution analysis, mesh partition of unity, and neural networks, to learn the pre-skinning shape deformations in an arbitrary character pose. Combined with linear blend skinning, this makes it possible to reconstruct the training shapes, as well as interpolate and extrapolate them. Crucially, we simultaneously achieve this at hard real-time rates and at multiple mesh resolution levels. Our technique makes it possible to trade deformation quality for memory and computation speed, to accommodate the strict requirements of modern real-time systems. Furthermore, we propose memory layout and code improvements to boost computation speeds. Previous methods for realtime approximations of quality shape deformations did not focus on hard real-time, or did not investigate the multi-resolution aspect of the problem. Compared to a \"naive\" approach of separately processing each hierarchical level of detail, our method offers a substantial memory reduction as well as computational speedups. It also makes it possible to construct the shape progressively level by level and interrupt the computation at any time, enabling graceful degradation of the deformation detail. We demonstrate our technique on several examples, including a stylized human character, human hands, and an inverse-kinematics-driven quadruped animal.<\/jats:p>","DOI":"10.1145\/3687985","type":"journal-article","created":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T15:46:04Z","timestamp":1732031164000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Multi-Resolution Real-Time Deep Pose-Space Deformation"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6445-9465","authenticated-orcid":false,"given":"Mianlun","family":"Zheng","sequence":"first","affiliation":[{"name":"University of Southern California, Los Angeles, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6371-9018","authenticated-orcid":false,"given":"Jernej","family":"Barbic","sequence":"additional","affiliation":[{"name":"University of Southern California, Los Angeles, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,11,19]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1137\/0731051"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1002\/nme.6385"},{"key":"e_1_2_1_3_1","volume-title":"Applications of Machine Learning for Character Animation","author":"Bailey Stephen Wells","unstructured":"Stephen Wells Bailey. 2020. Applications of Machine Learning for Character Animation. University of California, Berkeley."},{"key":"e_1_2_1_4_1","volume-title":"O'Brien","author":"Bailey Stephen W.","year":"2020","unstructured":"Stephen W. Bailey, Dalton Omens, Paul Dilorenzo, and James F. O'Brien. 2020. Fast and Deep Facial Deformations. ACM Transactions on Graphics (SIGGRAPH 2020) 39, 4, Article 94 (2020)."},{"key":"e_1_2_1_5_1","volume-title":"O'Brien","author":"Bailey Stephen W.","year":"2018","unstructured":"Stephen W. Bailey, Dave Otte, Paul Dilorenzo, and James F. O'Brien. 2018. Fast and Deep Deformation Approximations. ACM Transactions on Graphics (SIGGRAPH 2018) 37, 4, Article 119 (2018)."},{"key":"e_1_2_1_6_1","volume-title":"PBNS: Physically based neural simulator for unsupervised garment pose-space deformation. ACM Trans. on Graphics (SIGGRAPH Asia 2021) 40, 6","author":"Bertiche Hugo","year":"2021","unstructured":"Hugo Bertiche, Meysam Madadi, and Sergio Escalera. 2021. PBNS: Physically based neural simulator for unsupervised garment pose-space deformation. ACM Trans. on Graphics (SIGGRAPH Asia 2021) 40, 6 (2021)."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1090\/dimacs\/067\/01"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.gmod.2005.01.004"},{"key":"e_1_2_1_9_1","volume-title":"Proc. of the Symp. on Comp. Animation","author":"Capell Steve","year":"2002","unstructured":"Steve Capell, Seth Green, Brian Curless, Tom Duchamp, and Zoran Popovi\u0107. 2002. A Multiresolution Framework for Dynamic Deformations. In Proc. of the Symp. on Comp. Animation 2002. 41--48."},{"key":"e_1_2_1_10_1","unstructured":"Dan Casas Jes\u00fas P\u00e9rez Miguel A Otaduy Cristian Romero et al. 2021. Learning Contact Corrections for Handle-Based Subspace Dynamics. (2021)."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14107"},{"key":"e_1_2_1_12_1","volume-title":"Proc. of ACM SIGGRAPH","author":"Debunne Gilles","year":"2001","unstructured":"Gilles Debunne, Mathieu Desbrun, Marie-Paule Cani, and Alan H. Barr. 2001. Dynamic Real-Time Deformations Using Space & Time Adaptive Sampling. In Proc. of ACM SIGGRAPH 2001. 31--36."},{"key":"e_1_2_1_13_1","volume-title":"Proc. of Euro. Conf. on Comp. Vision (ECCV). 612--628","author":"Deng Boyang","year":"2020","unstructured":"Boyang Deng, John P Lewis, Timothy Jeruzalski, Gerard Pons-Moll, Geoffrey Hinton, Mohammad Norouzi, and Andrea Tagliasacchi. 2020. NASA: Neural articulated shape approximation. In Proc. of Euro. Conf. on Comp. Vision (ECCV). 612--628."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13645"},{"key":"e_1_2_1_15_1","volume-title":"Proc. of ACM SIGGRAPH 93","author":"Funkhouser T. A.","unstructured":"T. A. Funkhouser and C. H. S\u00e9quin. 1993. Adaptive Display Algorithm for Interactive Frame Rates During Visualization of Complex Virtual Environments. In Proc. of ACM SIGGRAPH 93. 247--254."},{"key":"e_1_2_1_16_1","volume-title":"Proc. of ACM SIGGRAPH 97","author":"Garland Michael","unstructured":"Michael Garland and Paul S. Heckbert. 1997. Surface simplification using quadric error metrics. In Proc. of ACM SIGGRAPH 97. 209--216."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566578"},{"key":"e_1_2_1_18_1","volume-title":"Real-time deep dynamic characters. ACM Trans. on Graphics (SIGGRAPH 2021) 40, 4","author":"Habermann Marc","year":"2021","unstructured":"Marc Habermann, Lingjie Liu, Weipeng Xu, Michael Zollhoefer, Gerard Pons-Moll, and Christian Theobalt. 2021. Real-time deep dynamic characters. ACM Trans. on Graphics (SIGGRAPH 2021) 40, 4 (2021), 1--16."},{"key":"e_1_2_1_19_1","unstructured":"Hang Si. 2011. TetGen: A Quality Tetrahedral Mesh Generator and a 3D Delaunay Triangulator."},{"key":"e_1_2_1_20_1","volume-title":"Symp. on Computer Animation (SCA). 1--12","author":"Holden Daniel","year":"2019","unstructured":"Daniel Holden, Bang Chi Duong, Sayantan Datta, and Derek Nowrouzezahrai. 2019. Subspace neural physics: Fast data-driven interactive simulation. In Symp. on Computer Animation (SCA). 1--12."},{"key":"e_1_2_1_21_1","volume-title":"Proc. of ACM SIGGRAPH 96","author":"Hoppe Hugues","year":"1996","unstructured":"Hugues Hoppe. 1996. Progressive meshes. In Proc. of ACM SIGGRAPH 96. 99--108."},{"key":"e_1_2_1_22_1","volume-title":"Skinning: Real-time Shape Deformation. In ACM SIGGRAPH 2014 Courses.","author":"Jacobson Alec","unstructured":"Alec Jacobson, Zhigang Deng, Ladislav Kavan, and John P. Lewis. 2014. Skinning: Real-time Shape Deformation. In ACM SIGGRAPH 2014 Courses."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/588272.588278"},{"key":"e_1_2_1_24_1","volume-title":"Proc. of Symp. on Interactive 3D Graphics and Games. 49--56","author":"Kavan Ladislav","year":"2009","unstructured":"Ladislav Kavan, Steven Collins, and Carol O'Sullivan. 2009. Automatic linearization of nonlinear skinning. In Proc. of Symp. on Interactive 3D Graphics and Games. 49--56."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409625.1409627"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964988"},{"key":"e_1_2_1_27_1","volume-title":"Adam: A Method for Stochastic Optimization. In Int. Conf. on Learning Representations (ICLR).","author":"Kingma Diederik","year":"2015","unstructured":"Diederik Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Int. Conf. on Learning Representations (ICLR)."},{"key":"e_1_2_1_28_1","volume-title":"EigenSkin: Real Time Large Deformation Character Skinning in Hardware. In In Symp. on Computer Animation (SCA).","author":"Kry P. G.","unstructured":"P. G. Kry, D. L. James, and D. K. Pai. 2002. EigenSkin: Real Time Large Deformation Character Skinning in Hardware. In In Symp. on Computer Animation (SCA)."},{"key":"e_1_2_1_29_1","volume-title":"Robust and accurate skeletal rigging from mesh sequences. ACM Trans. on Graphics (SIGGRAPH 2014) 33, 4","author":"Le Binh Huy","year":"2014","unstructured":"Binh Huy Le and Zhigang Deng. 2014. Robust and accurate skeletal rigging from mesh sequences. ACM Trans. on Graphics (SIGGRAPH 2014) 33, 4 (2014), 1--10."},{"key":"e_1_2_1_30_1","volume-title":"Direct delta mush skinning and variants. ACM Trans. on Graphics (SIGGRAPH 2019) 38, 4","author":"Le Binh Huy","year":"2019","unstructured":"Binh Huy Le and John-Peter Lewis. 2019. Direct delta mush skinning and variants. ACM Trans. on Graphics (SIGGRAPH 2019) 38, 4 (2019), 113--1."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344862"},{"key":"e_1_2_1_32_1","volume-title":"Learning skeletal articulations with neural blend shapes. ACM Trans. on Graphics (SIGGRAPH 2021) 40, 4","author":"Li Peizhuo","year":"2021","unstructured":"Peizhuo Li, Kfir Aberman, Rana Hanocka, Libin Liu, Olga Sorkine-Hornung, and Baoquan Chen. 2021. Learning skeletal articulations with neural blend shapes. ACM Trans. on Graphics (SIGGRAPH 2021) 40, 4 (2021), 1--15."},{"key":"e_1_2_1_33_1","volume-title":"Mirela Ben-Chen, and Alec Jacobson.","author":"Derek Liu Hsueh-Ti","year":"2021","unstructured":"Hsueh-Ti Derek Liu, Jiayi Eris Zhang, Mirela Ben-Chen, and Alec Jacobson. 2021. Surface multigrid via intrinsic prolongation. ACM Transactions on Graphics (SIGGRAPH 2021) 40, 4, Article 80 (2021), 13 pages."},{"key":"e_1_2_1_34_1","volume-title":"Neuroskinning: Automatic skin binding for production characters with deep graph networks. ACM Trans. on Graphics (SIGGRAPH 2019) 38, 4","author":"Liu Lijuan","year":"2019","unstructured":"Lijuan Liu, Youyi Zheng, Di Tang, Yi Yuan, Changjie Fan, and Kun Zhou. 2019. Neuroskinning: Automatic skin binding for production characters with deep graph networks. ACM Trans. on Graphics (SIGGRAPH 2019) 38, 4 (2019), 1--12."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818013"},{"key":"e_1_2_1_36_1","first-page":"1745","article-title":"NNWarp: Neural network-based nonlinear deformation","volume":"26","author":"Luo Ran","year":"2018","unstructured":"Ran Luo, Tianjia Shao, Huamin Wang, Weiwei Xu, Xiang Chen, Kun Zhou, and Yin Yang. 2018. NNWarp: Neural network-based nonlinear deformation. IEEE Trans. on Visualization and Computer Graphics (TVCG) 26, 4 (2018), 1745--1759.","journal-title":"IEEE Trans. on Visualization and Computer Graphics (TVCG)"},{"key":"e_1_2_1_37_1","doi-asserted-by":"crossref","unstructured":"A. McAdams Y. Zhu A. Selle M. Empey R. Tamstorf J. Teran and E. Sifakis. 2011. Efficient elasticity for character skinning with contact and collisions. ACM Trans. on Graphics (SIGGRAPH 2011) 30 4 (2011).","DOI":"10.1145\/2010324.1964932"},{"key":"e_1_2_1_38_1","unstructured":"Mixamo. 2024. Adobe Mixamo project https:\/\/www.mixamo.com."},{"key":"e_1_2_1_39_1","volume-title":"Multi-level partition of unity implicits. ACM Trans. on Graphics (SIGGRAPH 2003) 22, 3","author":"Ohtake Yutaka","year":"2003","unstructured":"Yutaka Ohtake, Alexander Belyaev, Marc Alexa, Greg Turk, and Hans-Peter Seidel. 2003. Multi-level partition of unity implicits. ACM Trans. on Graphics (SIGGRAPH 2003) 22, 3 (2003), 463--470."},{"key":"e_1_2_1_40_1","volume-title":"6-DoF Haptic Rendering Using Contact Levels of Detail and Haptic Textures. Ph. D. Dissertation. Department of Comp. Science","author":"Otaduy Miguel A.","unstructured":"Miguel A. Otaduy. 2004. 6-DoF Haptic Rendering Using Contact Levels of Detail and Haptic Textures. Ph. D. Dissertation. Department of Comp. Science, University of North Carolina at Chapel Hill."},{"key":"e_1_2_1_41_1","volume-title":"Adaptive Deformations with Fast Tight Bounds. In Symp. on Computer Animation (SCA). 181--190","author":"Otaduy Miguel A.","year":"2007","unstructured":"Miguel A. Otaduy, Daniel Germann, Stephane Redon, and Markus Gross. 2007. Adaptive Deformations with Fast Tight Bounds. In Symp. on Computer Animation (SCA). 181--190."},{"key":"e_1_2_1_42_1","volume-title":"PyTorch: An Imperative Style","author":"Paszke Adam","year":"1912","unstructured":"Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. arXiv preprint arXiv:1912.01703 (2019)."},{"key":"e_1_2_1_43_1","volume-title":"Contact-centric deformation learning. ACM Trans. on Graphics (SIGGRAPH 2022) 41, 4","author":"Romero Cristian","year":"2022","unstructured":"Cristian Romero, Dan Casas, Maurizio M Chiaramonte, and Miguel A Otaduy. 2022. Contact-centric deformation learning. ACM Trans. on Graphics (SIGGRAPH 2022) 41, 4 (2022), 1--11."},{"key":"e_1_2_1_44_1","volume-title":"Computer Graphics Forum","author":"Romero Cristian","unstructured":"Cristian Romero, Miguel A Otaduy, Dan Casas, and Jesus Perez. 2020. Modeling and estimation of nonlinear skin mechanics for animated avatars. In Computer Graphics Forum, Vol. 39. Wiley Online Library, 77--88."},{"key":"e_1_2_1_45_1","volume-title":"Computer Graphics Forum","author":"Santesteban Igor","unstructured":"Igor Santesteban, Elena Garces, Miguel A Otaduy, and Dan Casas. 2020. SoftSMPL: Data-driven Modeling of Nonlinear Soft-tissue Dynamics for Parametric Humans. In Computer Graphics Forum, Vol. 39. Wiley Online Library, 65--75."},{"key":"e_1_2_1_46_1","volume-title":"Computer Graphics Forum","author":"Santesteban Igor","unstructured":"Igor Santesteban, Miguel A Otaduy, and Dan Casas. 2019. Learning-based animation of clothing for virtual try-on. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 355--366."},{"key":"e_1_2_1_47_1","volume-title":"Accurate face rig approximation with deep differential subspace reconstruction. ACM Trans. on Graphics (SIGGRAPH 2020) 39, 4","author":"Song Steven L","year":"2020","unstructured":"Steven L Song, Weiqi Shi, and Michael Reed. 2020. Accurate face rig approximation with deep differential subspace reconstruction. ACM Trans. on Graphics (SIGGRAPH 2020) 39, 4 (2020), 34--1."},{"key":"e_1_2_1_48_1","volume-title":"Signed Distance Fields for Polygon Soup Meshes. Graphics Interface 2014","author":"Xu Hongyi","year":"2014","unstructured":"Hongyi Xu and Jernej Barbi\u010d. 2014. Signed Distance Fields for Polygon Soup Meshes. Graphics Interface 2014 (2014)."},{"key":"e_1_2_1_49_1","volume-title":"Rignet: Neural rigging for articulated characters. ACM Trans. on Graphics (SIGGRAPH 2020) 39, 4","author":"Xu Zhan","year":"2020","unstructured":"Zhan Xu, Yang Zhou, Evangelos Kalogerakis, Chris Landreth, and Karan Singh. 2020. Rignet: Neural rigging for articulated characters. ACM Trans. on Graphics (SIGGRAPH 2020) 39, 4 (2020), 14."},{"key":"e_1_2_1_50_1","volume-title":"Progressive simulation for cloth quasistatics. ACM Transactions on Graphics (SIGGRAPH Asia 2022) 41, 6","author":"Zhang Jiayi Eris","year":"2022","unstructured":"Jiayi Eris Zhang, J\u00e9r\u00e9mie Dumas, Yun Fei, Alec Jacobson, Doug L James, and Danny M Kaufman. 2022. Progressive simulation for cloth quasistatics. ACM Transactions on Graphics (SIGGRAPH Asia 2022) 41, 6 (2022), 1--16."},{"key":"e_1_2_1_51_1","volume-title":"Simulation of Hand Anatomy Using Medical Imaging. ACM Trans. on Graphics (SIGGRAPH Asia 2022) 41, 6","author":"Zheng Mianlun","year":"2022","unstructured":"Mianlun Zheng, Bohan Wang, Jingtao Huang, and Jernej Barbi\u010d. 2022. Simulation of Hand Anatomy Using Medical Imaging. ACM Trans. on Graphics (SIGGRAPH Asia 2022) 41, 6 (2022)."},{"key":"e_1_2_1_52_1","volume-title":"Proceedings of the IEEE\/CVF Conf. on Computer Vision and Pattern Recognition. 5932--5940","author":"Zheng Mianlun","year":"2021","unstructured":"Mianlun Zheng, Yi Zhou, Duygu Ceylan, and Jernej Barbi\u010d. 2021. A deep emulator for secondary motion of 3d characters. In Proceedings of the IEEE\/CVF Conf. on Computer Vision and Pattern Recognition. 5932--5940."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1731047.1731054"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3687985","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3687985","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:09:58Z","timestamp":1750295398000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3687985"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,19]]},"references-count":53,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,12,19]]}},"alternative-id":["10.1145\/3687985"],"URL":"https:\/\/doi.org\/10.1145\/3687985","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,19]]},"assertion":[{"value":"2024-11-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}