{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:13:49Z","timestamp":1750220029057,"version":"3.41.0"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2022,11,30]],"date-time":"2022-11-30T00:00:00Z","timestamp":1669766400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"United States National Science Foundation","award":["IIS-1911224"],"award-info":[{"award-number":["IIS-1911224"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:p>Precision modeling of the hand internal musculoskeletal anatomy has been largely limited to individual poses, and has not been connected into continuous volumetric motion of the hand anatomy actuating across the hand's entire range of motion. This is for a good reason, as hand anatomy and its motion are extremely complex and cannot be predicted merely from the anatomy in a single pose. We give a method to simulate the volumetric shape of hand's musculoskeletal organs to any pose in the hand's range of motion, producing external hand shapes and internal organ shapes that match ground truth optical scans and medical images (MRI) in multiple scanned poses. We achieve this by combining MRI images in multiple hand poses with FEM multibody nonlinear elastoplastic simulation. Our system models bones, muscles, tendons, joint ligaments and fat as separate volumetric organs that mechanically interact through contact and attachments, and whose shape matches medical images (MRI) in the MRI-scanned hand poses. The match to MRI is achieved by incorporating pose-space deformation and plastic strains into the simulation. We show how to do this in a non-intrusive manner that still retains all the simulation benefits, namely the ability to prescribe realistic material properties, generalize to arbitrary poses, preserve volume and obey contacts and attachments. We use our method to produce volumetric renders of the internal anatomy of the human hand in motion, and to compute and render highly realistic hand surface shapes. We evaluate our method by comparing it to optical scans, and demonstrate that we qualitatively and quantitatively substantially decrease the error compared to previous work. We test our method on five complex hand sequences, generated either using keyframe animation or performance animation using modern hand tracking techniques.<\/jats:p>","DOI":"10.1145\/3550454.3555486","type":"journal-article","created":{"date-parts":[[2022,11,30]],"date-time":"2022-11-30T21:19:07Z","timestamp":1669843147000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Simulation of Hand Anatomy Using Medical Imaging"],"prefix":"10.1145","volume":"41","author":[{"given":"Mianlun","family":"Zheng","sequence":"first","affiliation":[{"name":"University of Southern California"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bohan","family":"Wang","sequence":"additional","affiliation":[{"name":"University of Southern California, Massachusetts Institute of Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jingtao","family":"Huang","sequence":"additional","affiliation":[{"name":"University of Southern California"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jernej","family":"Barbi\u010d","sequence":"additional","affiliation":[{"name":"University of Southern California"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2022,11,30]]},"reference":[{"unstructured":"3dMD. 2022. 3dMDHands. https:\/\/3dmd.com\/.","key":"e_1_2_2_1_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_2_1","DOI":"10.1145\/3450626.3459769"},{"volume-title":"Optimal Step Nonrigid ICP Algorithms for Surface Registration. In Conf. on Computer Vision and Pattern Recognition (CVPR).","author":"Amberg B.","unstructured":"B. Amberg, S. Romdhani, and T. Vetter. 2007. Optimal Step Nonrigid ICP Algorithms for Surface Registration. In Conf. on Computer Vision and Pattern Recognition (CVPR).","key":"e_1_2_2_3_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_4_1","DOI":"10.1145\/3340260"},{"unstructured":"Artelys. 2019. Knitro. https:\/\/www.artelys.com\/solvers\/knitro\/.","key":"e_1_2_2_5_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_6_1","DOI":"10.1145\/1360612.1360662"},{"doi-asserted-by":"publisher","key":"e_1_2_2_7_1","DOI":"10.1145\/2601097.2601116"},{"volume-title":"University of Dundee","author":"Erolin C.","unstructured":"C. Erolin. 2019. Hand Anatomy. University of Dundee, Centre for Anatomy and Human Identification. https:\/\/sketchfab.com\/anatomy_dundee\/collections\/hand-anatomy.","key":"e_1_2_2_8_1"},{"volume-title":"Physically Based Rigging for Deformable Characters. In Symp. on Computer Animation (SCA). 301--310","author":"Capell S.","unstructured":"S. Capell, M. Burkhart, B. Curless, T. Duchamp, and Z. Popovi\u0107. 2005. Physically Based Rigging for Deformable Characters. In Symp. on Computer Animation (SCA). 301--310.","key":"e_1_2_2_9_1"},{"volume-title":"Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society. 8009--8012","author":"Chen H.","unstructured":"H. Chen, C. Chen, T. Yang, L. Kuo, IM Jou, F. Su, and Y. Sun. 2011. Model-based segmentation of flexor tendons from magnetic resonance images of finger joints. In Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society. 8009--8012.","key":"e_1_2_2_10_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_11_1","DOI":"10.1016\/j.ijsolstr.2010.07.001"},{"key":"e_1_2_2_12_1","article-title":"Anatomy Transfer","volume":"32","author":"Dicko A. H.","year":"2013","unstructured":"A. H. Dicko, T. Liu, B. Gilles, L. Kavan, F. Faure, O. Palombi, and M.P. Cani. 2013. Anatomy Transfer. ACM Trans. on Graphics (SIGGRAPH) 32, 6 (2013), 188:1--188:8.","journal-title":"ACM Trans. on Graphics (SIGGRAPH)"},{"doi-asserted-by":"publisher","key":"e_1_2_2_13_1","DOI":"10.1016\/j.jbiomech.2017.04.030"},{"doi-asserted-by":"publisher","key":"e_1_2_2_14_1","DOI":"10.1016\/j.mri.2012.05.001"},{"doi-asserted-by":"publisher","key":"e_1_2_2_15_1","DOI":"10.1016\/j.jbiomech.2009.10.020"},{"doi-asserted-by":"publisher","key":"e_1_2_2_16_1","DOI":"10.1016\/j.jbiomech.2017.12.024"},{"volume-title":"IEEE World Haptics Conf. 239--244","author":"Garre C.","unstructured":"C. Garre, F. Hern\u00e1ndez, A. Gracia, and M. A. Otaduy. 2011. Interactive simulation of a deformable hand for haptic rendering. In IEEE World Haptics Conf. 239--244.","key":"e_1_2_2_17_1"},{"unstructured":"M. Hadwiger J. M. Kniss C. Rezk-salama D. Weiskopf and K. Engel. 2006. Real-Time Volume Graphics. A. K. Peters Ltd.","key":"e_1_2_2_18_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_19_1","DOI":"10.1145\/3072959.3073664"},{"volume-title":"Skinning: Real-time Shape Deformation. In ACM SIGGRAPH 2014 Courses.","author":"Jacobson A.","unstructured":"A. Jacobson, Z. Deng, L. Kavan, and J. P. Lewis. 2014. Skinning: Real-time Shape Deformation. In ACM SIGGRAPH 2014 Courses.","key":"e_1_2_2_20_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_21_1","DOI":"10.1145\/2980179.2982438"},{"key":"e_1_2_2_22_1","volume-title":"The Upper Limb","author":"Kapandji A.I.","unstructured":"A.I. Kapandji. 2009. The physiology of the joints, 6th Edition, Vol. 1: The Upper Limb. Elsevier Exclusive.","edition":"6"},{"doi-asserted-by":"publisher","key":"e_1_2_2_23_1","DOI":"10.1145\/1409625.1409627"},{"volume-title":"Conf. on Computer Vision and Pattern Recognition (CVPR). 20492--20501","author":"Keller M.","unstructured":"M. Keller, S. Zuffi, M. J. Black, and S. Pujades. 2022. OSSO: Obtaining Skeletal Shape from Outside. In Conf. on Computer Vision and Pattern Recognition (CVPR). 20492--20501.","key":"e_1_2_2_24_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_25_1","DOI":"10.1145\/2019627.2019640"},{"volume-title":"EigenSkin: Real Time Large Deformation Character Skinning in Hardware. In In Symp. on Computer Animation (SCA).","author":"Kry P. G.","unstructured":"P. G. Kry, D. L. James, and D. K. Pai. 2002. EigenSkin: Real Time Large Deformation Character Skinning in Hardware. In In Symp. on Computer Animation (SCA).","key":"e_1_2_2_26_1"},{"volume-title":"Position and Orientation Based Cosserat Rods. In Symp. on Computer Animation (SCA).","author":"Kugelstadt T.","unstructured":"T. Kugelstadt and E. Sch\u00f6mer. 2016. Position and Orientation Based Cosserat Rods. In Symp. on Computer Animation (SCA).","key":"e_1_2_2_27_1"},{"doi-asserted-by":"crossref","unstructured":"C. Kuok T. Yang B. Tsai I. Jou M. Horng F. Su Y. Sun et al. 2020. Segmentation of finger tendon and synovial sheath in ultrasound image using deep convolutional neural network. Biomedical engineering online 19 1 (2020) 1--25.","key":"e_1_2_2_28_1","DOI":"10.1186\/s12938-020-00768-1"},{"volume-title":"Symp. on Computer Animation (SCA). 355--363","author":"Kurihara T.","unstructured":"T. Kurihara and N. Miyata. 2004. Modeling deformable human hands from medical images. In Symp. on Computer Animation (SCA). 355--363.","key":"e_1_2_2_29_1"},{"unstructured":"Leal Allan. 2018. autodiff. https:\/\/github.com\/autodiff\/autodiff\/.","key":"e_1_2_2_30_1"},{"unstructured":"LeapMotion. 2017. https:\/\/www.leapmotion.com.","key":"e_1_2_2_31_1"},{"doi-asserted-by":"crossref","unstructured":"S. Lee R. Yu J. Park M. Aanjaneya E. Sifakis and J. Lee. 2018. Dexterous manipulation and control with volumetric muscles. ACM Transactions on Graphics (SIGGRAPH 2018) 37 4 (2018) 57:1--57:13.","key":"e_1_2_2_32_1","DOI":"10.1145\/3197517.3201330"},{"doi-asserted-by":"publisher","key":"e_1_2_2_33_1","DOI":"10.1145\/1559755.1559756"},{"doi-asserted-by":"publisher","key":"e_1_2_2_34_1","DOI":"10.1145\/344779.344862"},{"key":"e_1_2_2_35_1","volume-title":"PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging. arXiv preprint arXiv:2106.10893","author":"Li Y.","year":"2021","unstructured":"Y. Li, M. Wu, Y. Zhang, L. Xu, and J. Yu. 2021. PIANO: A Parametric Hand Bone Model from Magnetic Resonance Imaging. arXiv preprint arXiv:2106.10893 (2021)."},{"doi-asserted-by":"crossref","unstructured":"Y. Li L. Zhang Z. Qiu Y. Jiang N. Li Y. Ma Y. Zhang L. Xu and J. Yu. 2022. NIMBLE: a non-rigid hand model with bones and muscles. ACM Transactions on Graphics (SIGGRAPH 2022) 41 4 (2022) 1--16.","key":"e_1_2_2_36_1","DOI":"10.1145\/3528223.3530079"},{"key":"e_1_2_2_37_1","first-page":"215","article-title":"Simulation and control of skeleton-driven soft body characters","volume":"32","author":"Liu L.","year":"2013","unstructured":"L. Liu, K. Yin, B. Wang, and B. Guo. 2013. Simulation and control of skeleton-driven soft body characters. ACM Trans. on Graphics (SIGGRAPH Asia) 32, 6 (2013), 215.","journal-title":"ACM Trans. on Graphics (SIGGRAPH Asia)"},{"doi-asserted-by":"publisher","key":"e_1_2_2_38_1","DOI":"10.1145\/2816795.2818013"},{"unstructured":"C. Lugaresi J. Tang H. Nash C. McClanahan E. Uboweja M. Hays F. Zhang C. Chang M. G. Yong J. Lee W. Chang W. Hua M. Georg and M Grundmann. 2019. MediaPipe: A Framework for Building Perception Pipelines.","key":"e_1_2_2_39_1"},{"doi-asserted-by":"crossref","unstructured":"A. McAdams Y. Zhu A. Selle M. Empey R. Tamstorf J. Teran and E. Sifakis. 2011. Efficient elasticity for character skinning with contact and collisions. ACM Trans. on Graphics (SIGGRAPH 2011) 30 4 (2011).","key":"e_1_2_2_40_1","DOI":"10.1145\/1964921.1964932"},{"volume-title":"IEEE Int. Conf. on Intelligent Robots and Systems. 2750--2755","author":"Miyata N.","unstructured":"N. Miyata, M. Kouch, M. Mochimaru, and T. Kurihara. 2005. Finger joint kinematics from MR images. In IEEE Int. Conf. on Intelligent Robots and Systems. 2750--2755.","key":"e_1_2_2_41_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_42_1","DOI":"10.1111\/cgf.14185"},{"doi-asserted-by":"publisher","key":"e_1_2_2_43_1","DOI":"10.1145\/1186822.1073216"},{"key":"e_1_2_2_44_1","volume-title":"HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization. In Proc. of the European Conf. on Computer Vision (ECCV)","author":"Qian N.","year":"2020","unstructured":"N. Qian, J. Wang, F. Mueller, F. Bernard, V. Golyanik, and C. Theobalt. 2020. HTML: A Parametric Hand Texture Model for 3D Hand Reconstruction and Personalization. In Proc. of the European Conf. on Computer Vision (ECCV). Springer."},{"key":"e_1_2_2_45_1","volume-title":"Real-Time Weighted Pose-Space Deformation on the GPU. In Proc. of Eurographics","volume":"25","author":"Rhee T.","unstructured":"T. Rhee, J.P. Lewis, and U. Neumann. 2006. Real-Time Weighted Pose-Space Deformation on the GPU. In Proc. of Eurographics, Vol. 25."},{"doi-asserted-by":"publisher","key":"e_1_2_2_46_1","DOI":"10.1145\/3130800.3130883"},{"doi-asserted-by":"publisher","key":"e_1_2_2_47_1","DOI":"10.1016\/j.cag.2018.10.013"},{"volume-title":"Segmentation of bone structures in Magnetic Resonance Images for human hand skeletal kinematics modelling. Master's thesis","author":"Rusu A.","unstructured":"A. Rusu. 2011. Segmentation of bone structures in Magnetic Resonance Images for human hand skeletal kinematics modelling. Master's thesis. German Aerospace Center.","key":"e_1_2_2_48_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_49_1","DOI":"10.1145\/2766987"},{"doi-asserted-by":"publisher","key":"e_1_2_2_50_1","DOI":"10.1145\/2766957"},{"doi-asserted-by":"crossref","unstructured":"E. Sifakis I. Neverov and R. Fedkiw. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. on Graphics (SIGGRAPH 2005) 24 3 (2005) 417--425.","key":"e_1_2_2_51_1","DOI":"10.1145\/1073204.1073208"},{"doi-asserted-by":"publisher","key":"e_1_2_2_52_1","DOI":"10.1145\/3180491"},{"doi-asserted-by":"publisher","key":"e_1_2_2_54_1","DOI":"10.1145\/1360612.1360682"},{"volume-title":"Weta Digital: Tissue Muscle and Fat Simulation System.","year":"2013","unstructured":"Tissue. 2013. Weta Digital: Tissue Muscle and Fat Simulation System.","key":"e_1_2_2_55_1"},{"doi-asserted-by":"crossref","unstructured":"R. Vaillant G. Guennebaud L. Barthe B. Wyvill and M.P. Cani. 2014. Robust Iso-surface Tracking for Interactive Character Skinning. ACM Trans. on Graphics (SIGGRAPH Asia 2014) 33 6 (2014) 189:1--137:11.","key":"e_1_2_2_56_1","DOI":"10.1145\/2661229.2661264"},{"volume-title":"Conf. on Motion and Vibration Control (MOVIC).","author":"van der Smagt P.","unstructured":"P. van der Smagt and G. Stillfried. 2008. Using MRI data to compute a hand kinematic model. In Conf. on Motion and Vibration Control (MOVIC).","key":"e_1_2_2_57_1"},{"doi-asserted-by":"publisher","key":"e_1_2_2_58_1","DOI":"10.1145\/1559755.1559762"},{"doi-asserted-by":"crossref","unstructured":"B. Wang G. Matcuk and J. Barbi\u010d. 2019. Hand Modeling and Simulation Using Stabilized Magnetic Resonance Imaging. ACM Trans. on Graphics (SIGGRAPH 2019) 38 4 (2019).","key":"e_1_2_2_59_1","DOI":"10.1145\/3306346.3322983"},{"unstructured":"B. Wang G. Matcuk and J. Barbi\u010d. 2020. Hand MRI dataset. http:\/\/www.jernejbarbic.com\/hand-mri-dataset.","key":"e_1_2_2_60_1"},{"key":"e_1_2_2_61_1","article-title":"Modeling of Personalized Anatomy using Plastic Strains","volume":"40","author":"Wang B.","year":"2021","unstructured":"B. Wang, G. Matcuk, and J. Barbi\u010d. 2021. Modeling of Personalized Anatomy using Plastic Strains. ACM Trans. on Graphics (TOG) 40, 2 (2021).","journal-title":"ACM Trans. on Graphics (TOG)"},{"doi-asserted-by":"publisher","key":"e_1_2_2_62_1","DOI":"10.1114\/1.180"},{"unstructured":"Wrap3. 2018. Nonlinear Iterative Closest Point mesh registration software. https:\/\/www.russian3dscanner.com.","key":"e_1_2_2_63_1"},{"key":"e_1_2_2_64_1","volume-title":"Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control. Critical reviews in biomedical engineering 17, 4","author":"Zajac FE","year":"1989","unstructured":"FE Zajac. 1989. Muscle and tendon: properties, models, scaling, and application to biomechanics and motor control. Critical reviews in biomedical engineering 17, 4 (1989), 359--411."},{"unstructured":"Zygote. 2016. Zygote body. http:\/\/www.zygotebody.com.","key":"e_1_2_2_65_1"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550454.3555486","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3550454.3555486","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:51:43Z","timestamp":1750182703000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550454.3555486"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,30]]},"references-count":64,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["10.1145\/3550454.3555486"],"URL":"https:\/\/doi.org\/10.1145\/3550454.3555486","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"type":"print","value":"0730-0301"},{"type":"electronic","value":"1557-7368"}],"subject":[],"published":{"date-parts":[[2022,11,30]]},"assertion":[{"value":"2022-11-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}