{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,8,15]],"date-time":"2025-08-15T01:28:59Z","timestamp":1755221339421,"version":"3.43.0"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"4","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Comput. Graph. Interact. Tech."],"published-print":{"date-parts":[[2025,8,31]]},"abstract":"<jats:p>We introduce a self-supervised neural approach for real-time, physically-principled facial rigs that incorporate soft tissue deformations and contact interactions. Traditional artist-crafted blendshapes lack the capacity to produce realistic tissue deformations, while physics-based models, despite their accuracy, are computationally prohibitive for interactive applications. Our method addresses these limitations by learning a neural map from a set of rig controls to corresponding deformations that minimize the mechanical energy of an anatomically-based face model, including soft tissue, skull, and teeth layers. This self-supervised framework eliminates the need for predefined simulation data and supports unsupervised handling of collisions between rigid and soft bodies, streamlining the computational process. We demonstrate that, for the first time, our approach achieves real-time performance of physics-based face rigs with complex non-linear deformations and contact handling.<\/jats:p>","DOI":"10.1145\/3747866","type":"journal-article","created":{"date-parts":[[2025,8,8]],"date-time":"2025-08-08T15:33:31Z","timestamp":1754667211000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["NeuRiPhy: Neural Baking of Physics-Based Deformations for Facial Rigs 59"],"prefix":"10.1145","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-4781-5093","authenticated-orcid":false,"given":"Davide","family":"Corigliano","sequence":"first","affiliation":[{"name":"Department of Computer Science","place":["Zurich, Switzerland"]},{"name":"ETH Zurich","place":["Zurich, Switzerland"]}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-1798-4068","authenticated-orcid":false,"given":"Daniel","family":"Peter","sequence":"additional","affiliation":[{"name":"Align Technology","place":["Zurich, Switzerland"]}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-2089-0150","authenticated-orcid":false,"given":"Niko Benjamin","family":"Huber","sequence":"additional","affiliation":[{"name":"Align Technology","place":["Zurich, Switzerland"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8086-7664","authenticated-orcid":false,"given":"Bernhard","family":"Thomaszewski","sequence":"additional","affiliation":[{"name":"Department of Computer Science","place":["Zurich, Switzerland"]},{"name":"ETH Zurich","place":["Zurich, Switzerland"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7494-8660","authenticated-orcid":false,"given":"Barbara","family":"Solenthaler","sequence":"additional","affiliation":[{"name":"Department of Computer Science","place":["Zurich, Switzerland"]},{"name":"ETH Zurich","place":["Zurich, Switzerland"]}]}],"member":"320","published-online":{"date-parts":[[2025,8,8]]},"reference":[{"key":"e_1_3_3_2_1","doi-asserted-by":"publisher","unstructured":"Stephen\u00a0W. Bailey Dalton Omens Paul Dilorenzo and James\u00a0F. O\u2019Brien. 2020. Fast and deep facial deformations. ACM Transactions on Graphics 39 (7 2020). Issue 4. 10.1145\/3386569.3392397\/SUPPLFILE\/A94-BAILEY.MP4","DOI":"10.1145\/3386569.3392397\/SUPPLFILE\/A94-BAILEY.MP4"},{"key":"e_1_3_3_3_1","doi-asserted-by":"publisher","unstructured":"Stephen\u00a0W. Bailey Dave Otte Paul Dilorenzo and James\u00a0F. O\u2019Brien. 2018. Fast and deep deformation approximations. ACM Transactions on Graphics 37 (2018). Issue 4. 10.1145\/3197517.3201300","DOI":"10.1145\/3197517.3201300"},{"key":"e_1_3_3_4_1","doi-asserted-by":"publisher","unstructured":"Vincent Barrielle Nicolas Stoiber and C\u00e9dric Cagniart. 2016. BlendForces: A dynamic framework for facial animation. Computer Graphics Forum 35 (5 2016) 341\u2013352. Issue 2. 10.1111\/CGF.12836","DOI":"10.1111\/CGF.12836"},{"key":"e_1_3_3_5_1","doi-asserted-by":"publisher","DOI":"10.2312\/SCA\/SCA08\/057-066"},{"key":"e_1_3_3_6_1","doi-asserted-by":"publisher","unstructured":"Sofien Bouaziz Sebastian Martin Tiantian Liu Ladislav Kavan and Mark Pauly. 2014. Projective dynamics: fusing constraint projections for fast simulation. ACM Trans. Graph. 33 4 Article 59 (July 2014) 11\u00a0pages. 10.1145\/2601097.2601116","DOI":"10.1145\/2601097.2601116"},{"key":"e_1_3_3_7_1","doi-asserted-by":"publisher","unstructured":"Byungkuk Choi Haekwang Eom Benjamin Mouscadet Stephen Cullingford Kurt Ma Stefanie Gassel Suzi Kim Andrew Moffat Millicent Maier Marco Revelant Joe Letteri and Karan Singh. 2022. Animatomy: an Animator-centric Anatomically Inspired System for 3D Facial Modeling Animation and Transfer. Proceedings - SIGGRAPH Asia 2022 Conference Papers (11 2022). 10.1145\/3550469.3555398","DOI":"10.1145\/3550469.3555398"},{"key":"e_1_3_3_8_1","doi-asserted-by":"crossref","unstructured":"Daniel\u00a0K. Dorda Daniel Peter Dominik Borer Niko\u00a0Benjamin Huber Irena Sailer Markus\u00a0H. Gross Barbara Solenthaler and Bernhard Thomaszewski. 2022. Differentiable Simulation for Outcome-Driven Orthognathic Surgery Planning. Eurographics Symposium on Computer Animation 41 (Sep 2022) 53\u201361.","DOI":"10.1111\/cgf.14623"},{"key":"e_1_3_3_9_1","doi-asserted-by":"publisher","unstructured":"Y. Du Y. Li S. Coros and B. Thomaszewski. 2024. Robust and Artefact-Free Deformable Contact with Smooth Surface Representations. Computer Graphics Forum 43 8 (2024) e15187. 10.1111\/cgf.15187 arXiv:https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/cgf.15187","DOI":"10.1111\/cgf.15187"},{"key":"e_1_3_3_10_1","doi-asserted-by":"crossref","unstructured":"Bernhard Egger William\u00a0AP Smith Ayush Tewari Stefanie Wuhrer Michael Zollhoefer Thabo Beeler Florian Bernard Timo Bolkart Adam Kortylewski Sami Romdhani et\u00a0al. 2020. 3d morphable face models\u2014past present and future. ACM Transactions on Graphics (ToG) 39 5 (2020) 1\u201338.","DOI":"10.1145\/3395208"},{"key":"e_1_3_3_11_1","doi-asserted-by":"publisher","unstructured":"Lawson Fulton Vismay Modi David Duvenaud David\u00a0I.W. Levin and Alec Jacobson. 2019. Latent-space Dynamics for Reduced Deformable Simulation. Computer Graphics Forum 38 (5 2019) 379\u2013391. Issue 2. 10.1111\/CGF.13645","DOI":"10.1111\/CGF.13645"},{"key":"e_1_3_3_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3641519.3657408"},{"key":"e_1_3_3_13_1","doi-asserted-by":"publisher","unstructured":"Fabian Hahn Sebastian Martin Bernhard Thomaszewski Robert Sumner Stelian Coros and Markus Gross. 2012. Rig-space physics. ACM Transactions on Graphics 31 (7 2012). Issue 4. 10.1145\/2185520.2185568","DOI":"10.1145\/2185520.2185568"},{"key":"e_1_3_3_14_1","doi-asserted-by":"publisher","unstructured":"Fabian Hahn Bernhard Thomaszewski Stelian Coros Robert\u00a0W. Sumner and Markus Gross. 2013. Efficient simulation of secondary motion in rig-space. Proceedings - SCA 2013: 12th ACM SIGGRAPH \/ Eurographics Symposium on Computer Animation (2013) 165\u2013172. 10.1145\/2485895.2485918","DOI":"10.1145\/2485895.2485918"},{"key":"e_1_3_3_15_1","doi-asserted-by":"publisher","unstructured":"Yushan Han Yizhou Chen Carmichael Ong Jingyu Chen Jennifer Hicks and Joseph Teran. 2024. A Neural Network Model for Efficient Musculoskeletal-Driven Skin Deformation. ACM Transactions on Graphics 43 (7 2024). Issue 4. 10.1145\/3658135","DOI":"10.1145\/3658135"},{"key":"e_1_3_3_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3309486.3340245"},{"key":"e_1_3_3_17_1","doi-asserted-by":"publisher","unstructured":"Alexandru\u00a0Eugen Ichim Petr Kadle\u010dek Ladislav Kavan and Mark Pauly. 2017. Phace: Physics-based face modeling and animation. ACM Transactions on Graphics 36 (2017). Issue 4. 10.1145\/3072959.3073664","DOI":"10.1145\/3072959.3073664"},{"key":"e_1_3_3_18_1","doi-asserted-by":"crossref","unstructured":"Alec Jacobson Ilya Baran Jovan Popovi\u0107 and Olga Sorkine. 2011. Bounded Biharmonic Weights for Real-Time Deformation. ACM Transactions on Graphics (proceedings of ACM SIGGRAPH) 30 4 (2011) 78:1\u201378:8.","DOI":"10.1145\/2010324.1964973"},{"key":"e_1_3_3_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3532720.3535628"},{"key":"e_1_3_3_20_1","doi-asserted-by":"publisher","unstructured":"Yeara Kozlov Derek Bradley Moritz B\u00e4cher Bernhard Thomaszewski Thabo Beeler and Markus Gross. 2017. Enriching Facial Blendshape Rigs with Physical Simulation. Computer Graphics Forum 36 (5 2017) 75\u201384. Issue 2. 10.1111\/CGF.13108","DOI":"10.1111\/CGF.13108"},{"key":"e_1_3_3_21_1","doi-asserted-by":"publisher","unstructured":"J.\u00a0P. Lewis and Ken\u00a0Ichi Anjyo. 2010. Direct-manipulation blendshapes. IEEE Computer Graphics and Applications 30 (2010) 42\u201350. Issue 4. 10.1109\/MCG.2010.41","DOI":"10.1109\/MCG.2010.41"},{"key":"e_1_3_3_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344862"},{"key":"e_1_3_3_23_1","doi-asserted-by":"crossref","unstructured":"Minchen Li Zachary Ferguson Teseo Schneider Timothy Langlois Denis Zorin Daniele Panozzo Chenfanfu Jiang and Danny\u00a0M. Kaufman. 2020. Incremental Potential Contact: Intersection- and Inversion-free Large Deformation Dynamics. ACM Trans. Graph. (SIGGRAPH) 39 4 Article 59 (2020).","DOI":"10.1145\/3386569.3392425"},{"key":"e_1_3_3_24_1","doi-asserted-by":"publisher","unstructured":"Shugao Ma Tomas Simon Jason Saragih Dawei Wang Yuecheng Li Fernando De\u00a0La Torre and Yaser Sheikh. 2021. Pixel Codec Avatars. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2021) 64\u201373. 10.1109\/CVPR46437.2021.00013","DOI":"10.1109\/CVPR46437.2021.00013"},{"key":"e_1_3_3_25_1","doi-asserted-by":"publisher","unstructured":"Aleka McAdams Andrew Selle Mark Empey Rasmus Tamstorf Joseph Teran Eftychios Sifakis Yongning Zhu Aleka McAdams Joseph Teran and Eftychios Sifakis. 2011. Efficient elasticity for character skinning with contact and collisions. ACM Transactions on Graphics 30 (7 2011) 1\u201312. Issue 4. 10.1145\/2010324.1964932","DOI":"10.1145\/2010324.1964932"},{"key":"e_1_3_3_26_1","doi-asserted-by":"publisher","unstructured":"Hyojoon Park Sangeetha\u00a0Grama Srinivasan Matthew Cong Doyub Kim Byungsoo Kim Jonathan Swartz Ken Museth and Eftychios Sifakis. 2024. Near-realtime Facial Animation by Deep 3D Simulation Super-Resolution. ACM Transactions on Graphics 43 (10 2024) 1\u201320. Issue 5. 10.1145\/3670687","DOI":"10.1145\/3670687"},{"key":"e_1_3_3_27_1","doi-asserted-by":"publisher","unstructured":"Dafei Qin Jun Saito Noam Aigerman Thibault Groueix and Taku Komura. 2023. Neural Face Rigging for Animating and Retargeting Facial Meshes in the Wild. Proceedings - SIGGRAPH 2023 Conference Papers (7 2023). 10.1145\/3588432.3591556","DOI":"10.1145\/3588432.3591556"},{"key":"e_1_3_3_28_1","doi-asserted-by":"publisher","unstructured":"Zesong Qiu Yuwei Li Dongming He Qixuan Zhang Longwen Zhang Yinghao Zhang Jingya Wang Lan Xu Xudong Wang Yuyao Zhang and Jingyi Yu. 2022. Sculptor: Skeleton-consistent face creation using a learned parametric generator. ACM Transactions on Graphics 41 (11 2022). Issue 6. 10.1145\/3550454.3555462","DOI":"10.1145\/3550454.3555462"},{"key":"e_1_3_3_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3388767.3407340"},{"key":"e_1_3_3_30_1","doi-asserted-by":"crossref","unstructured":"Cristian Romero Dan Casas Maurizio\u00a0M. Chiaramonte and Miguel\u00a0A. Otaduy. 2022. Contact-Centric Deformation Learning. ACM Transactions on Graphics (Proc. of ACM SIGGRAPH) 41 4 (2022).","DOI":"10.1145\/3528223.3530182"},{"key":"e_1_3_3_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3610548.3618229"},{"key":"e_1_3_3_32_1","doi-asserted-by":"publisher","unstructured":"Cristian Romero Dan Casas Jes\u00fas P\u00e9rez and Miguel Otaduy. 2021a. Learning Contact Corrections for Handle-Based Subspace Dynamics. ACM Trans. Graph. 40 4 Article 59 (jul 2021) 12\u00a0pages. 10.1145\/3450626.3459875","DOI":"10.1145\/3450626.3459875"},{"key":"e_1_3_3_33_1","doi-asserted-by":"crossref","unstructured":"Cristian Romero Dan Casas Jes\u00fas P\u00e9rez and Miguel\u00a0A. Otaduy. 2021b. Learning Contact Corrections for Handle-Based Subspace Dynamics. ACM Transactions on Graphics (Proc. of ACM SIGGRAPH) 40 4 (2021).","DOI":"10.1145\/3476576.3476703"},{"key":"e_1_3_3_34_1","doi-asserted-by":"publisher","unstructured":"Nicholas Sharp Cristian Romero Alec Jacobson Etienne Vouga Paul Kry David\u00a0I.W. Levin and Justin Solomon. 2023. Data-Free Learning of Reduced-Order Kinematics. Proceedings - SIGGRAPH 2023 Conference Papers (7 2023). 10.1145\/3588432.3591521","DOI":"10.1145\/3588432.3591521"},{"key":"e_1_3_3_35_1","doi-asserted-by":"publisher","unstructured":"Siyuan Shen Yin Yang Tianjia Shao He Wang Chenfanfu Jiang Lei Lan and Kun Zhou. 2021. High-Order Differentiable Autoencoder for Nonlinear Model Reduction. ACM Trans. Graph. 40 4 Article 59 (jul 2021) 15\u00a0pages. 10.1145\/3450626.3459754","DOI":"10.1145\/3450626.3459754"},{"key":"e_1_3_3_36_1","doi-asserted-by":"publisher","unstructured":"Eftychios Sifakis Igor Neverov and Ronald Fedkiw. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. (7 2005) 417\u2013425. 10.1145\/1186822.1073208","DOI":"10.1145\/1186822.1073208"},{"key":"e_1_3_3_37_1","unstructured":"Vincent Sitzmann Julien N.\u00a0P. Martel Alexander\u00a0W. Bergman David\u00a0B. Lindell and Gordon Wetzstein. 2020. Implicit Neural Representations with Periodic Activation Functions. CoRR abs\/2006.09661 (2020). arXiv:https:\/\/arXiv.org\/abs\/2006.09661https:\/\/arxiv.org\/abs\/2006.09661"},{"key":"e_1_3_3_38_1","doi-asserted-by":"publisher","unstructured":"Breannan Smith Fernando\u00a0De Goes and Theodore Kim. 2018. Stable Neo-Hookean Flesh Simulation. ACM Trans. Graph. 37 2 Article 59 (March 2018) 15\u00a0pages. 10.1145\/3180491","DOI":"10.1145\/3180491"},{"key":"e_1_3_3_39_1","doi-asserted-by":"publisher","unstructured":"Steven\u00a0L. Song Weiqi Shi and Michael Reed. 2020. Accurate face rig approximation with deep differential subspace reconstruction. ACM Trans. Graph. 39 4 Article 59 (Aug. 2020) 12\u00a0pages. 10.1145\/3386569.3392491","DOI":"10.1145\/3386569.3392491"},{"key":"e_1_3_3_40_1","doi-asserted-by":"publisher","unstructured":"Sangeetha\u00a0Grama Srinivasan Qisi Wang Junior Rojas Gergely Kl\u00e1r Ladislav Kavan and Eftychios Sifakis. 2021. Learning active quasistatic physics-based models from data. ACM Transactions on Graphics 40 (7 2021). Issue 4. 10.1145\/3450626.3459883","DOI":"10.1145\/3450626.3459883"},{"key":"e_1_3_3_41_1","doi-asserted-by":"publisher","unstructured":"Nicolas Wagner Mario Botsch and Ulrich Schwanecke. 2023. SoftDECA: Computationally Efficient Physics-Based Facial Animations. Proceedings - MIG 2023: 16th ACM SIGGRAPH Conference on Motion Interaction and Games (11 2023). 10.1145\/3623264.3624439","DOI":"10.1145\/3623264.3624439"},{"key":"e_1_3_3_42_1","doi-asserted-by":"publisher","unstructured":"Nicolas Wagner Ulrich Schwanecke and Mario Botsch. 2024. SparseSoftDECA \u2014 Efficient high-resolution physics-based facial animation from sparse landmarks. Computers and Graphics (Pergamon) 119 (4 2024). 10.1016\/J.CAG.2024.103903","DOI":"10.1016\/J.CAG.2024.103903"},{"key":"e_1_3_3_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52733.2024.02185"},{"key":"e_1_3_3_44_1","doi-asserted-by":"publisher","unstructured":"Lingchen Yang Byungsoo Kim Gaspard Zoss Baran G\u00f6zc\u00fc Markus Gross and Barbara Solenthaler. 2022. Implicit neural representation for physics-driven actuated soft bodies. ACM Transactions on Graphics 41 (7 2022). Issue 4. 10.1145\/3528223.3530156","DOI":"10.1145\/3528223.3530156"},{"key":"e_1_3_3_45_1","doi-asserted-by":"publisher","unstructured":"Lingchen Yang Gaspard Zoss Prashanth Chandran Markus Gross Barbara Solenthaler Eftychios Sifakis and Derek Bradley. 2024. Learning a Generalized Physical Face Model From Data. ACM Trans. Graph. 43 4 Article 59 (July 2024) 14\u00a0pages. 10.1145\/3658189","DOI":"10.1145\/3658189"},{"key":"e_1_3_3_46_1","doi-asserted-by":"publisher","unstructured":"Jiayi\u00a0Eris Zhang Seungbae Bang David I.\u00a0W. Levin and Alec Jacobson. 2020. Complementary dynamics. ACM Trans. Graph. 39 6 Article 59 (Nov. 2020) 11\u00a0pages. 10.1145\/3414685.3417819","DOI":"10.1145\/3414685.3417819"},{"key":"e_1_3_3_47_1","doi-asserted-by":"publisher","unstructured":"Longwen Zhang Zijun Zhao Xinzhou Cong Qixuan Zhang Shuqi Gu Yuchong Gao Rui Zheng Wei Yang Lan Xu and Jingyi Yu. 2023. HACK: Learning a Parametric Head and Neck Model for High-fidelity Animation. ACM Transactions on Graphics 42 (8 2023). Issue 4. 10.1145\/3592093","DOI":"10.1145\/3592093"},{"key":"e_1_3_3_48_1","doi-asserted-by":"publisher","unstructured":"Mianlun Zheng Yi Zhou Duygu Ceylan and Jernej Barbi\u010d. 2021. A Deep Emulator for Secondary Motion of 3D Characters. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2021) 5928\u20135936. 10.1109\/CVPR46437.2021.00587","DOI":"10.1109\/CVPR46437.2021.00587"}],"container-title":["Proceedings of the ACM on Computer Graphics and Interactive Techniques"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3747866","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,8]],"date-time":"2025-08-08T16:25:21Z","timestamp":1754670321000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3747866"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,8]]},"references-count":47,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,8,31]]}},"alternative-id":["10.1145\/3747866"],"URL":"https:\/\/doi.org\/10.1145\/3747866","relation":{},"ISSN":["2577-6193"],"issn-type":[{"value":"2577-6193","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,8]]},"assertion":[{"value":"2025-08-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}