{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,2]],"date-time":"2026-06-02T09:28:13Z","timestamp":1780392493749,"version":"3.54.1"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,5,11]],"date-time":"2024-05-11T00:00:00Z","timestamp":1715385600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Comput. Graph. Interact. Tech."],"published-print":{"date-parts":[[2024,5,11]]},"abstract":"<jats:p>3D rendering of dynamic face captures is a challenging problem, and it demands improvements on several fronts---photorealism, efficiency, compatibility, and configurability. We present a novel representation that enables high-quality volumetric rendering of an actor's dynamic facial performances with minimal compute and memory footprint. It runs natively on commodity graphics soft- and hardware, and allows for a graceful trade-off between quality and efficiency. Our method utilizes recent advances in neural rendering, particularly learning discrete radiance manifolds to sparsely sample the scene to model volumetric effects. We achieve efficient modeling by learning a single set of manifolds for the entire dynamic sequence, while implicitly modeling appearance changes as temporal canonical texture. We export a single layered mesh and view-independent RGBA texture video that is compatible with legacy graphics renderers without additional ML integration. We demonstrate our method by rendering dynamic face captures of real actors in a game engine, at comparable photorealism to state-of-the-art neural rendering techniques at previously unseen frame rates.<\/jats:p>","DOI":"10.1145\/3651304","type":"journal-article","created":{"date-parts":[[2024,5,13]],"date-time":"2024-05-13T13:39:44Z","timestamp":1715607584000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["FaceFolds: Meshed Radiance Manifolds for Efficient Volumetric Rendering of Dynamic Faces"],"prefix":"10.1145","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7944-0305","authenticated-orcid":false,"given":"Safa C.","family":"Medin","sequence":"first","affiliation":[{"name":"MIT, USA and Google, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1427-7612","authenticated-orcid":false,"given":"Gengyan","family":"Li","sequence":"additional","affiliation":[{"name":"ETH Zurich, Switzerland and Google, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2471-9776","authenticated-orcid":false,"given":"Ruofei","family":"Du","sequence":"additional","affiliation":[{"name":"Google, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-5005-8110","authenticated-orcid":false,"given":"Stephan","family":"Garbin","sequence":"additional","affiliation":[{"name":"Google, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-2986-8210","authenticated-orcid":false,"given":"Philip","family":"Davidson","sequence":"additional","affiliation":[{"name":"Google, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9166-4758","authenticated-orcid":false,"given":"Gregory W.","family":"Wornell","sequence":"additional","affiliation":[{"name":"MIT, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8077-1205","authenticated-orcid":false,"given":"Thabo","family":"Beeler","sequence":"additional","affiliation":[{"name":"Google, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7906-4004","authenticated-orcid":false,"given":"Abhimitra","family":"Meka","sequence":"additional","affiliation":[{"name":"Google, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,5,13]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00264"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01620"},{"key":"e_1_2_2_3_1","volume-title":"Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. CVPR","author":"Barron Jonathan T.","year":"2022","unstructured":"Jonathan T. Barron, Ben Mildenhall, Dor Verbin, Pratul P. Srinivasan, and Peter Hedman. 2022. Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields. CVPR (2022)."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.01804"},{"key":"e_1_2_2_5_1","unstructured":"Alexander W. Bergman Petr Kellnhofer Wang Yifan Eric R. Chan David B. Lindell and Gordon Wetzstein. 2023. Generative Neural Articulated Radiance Fields. arXiv:2206.14314 [cs.CV]"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","unstructured":"Volker Blanz and Thomas Vetter. 1999. A Morphable Model for the Synthesis of 3D Faces. In SIGGRAPH. https:\/\/doi.org\/10.1145\/311535.311556","DOI":"10.1145\/311535.311556"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","unstructured":"Guy E Blelloch. 1990. Prefix Sums and Their Applications. (1990). https:\/\/doi.org\/10.1109\/ISTCS.1995.377028","DOI":"10.1109\/ISTCS.1995.377028"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01363"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00315"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530143"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01565"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","unstructured":"Zhiqin Chen Thomas Funkhouser Peter Hedman and Andrea Tagliasacchi. 2023. MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures. In The Conference on Computer Vision and Pattern Recognition (CVPR). https:\/\/doi.org\/10.1109\/CVPR52729.2023.01590","DOI":"10.1109\/CVPR52729.2023.01590"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766945"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/378456.378468"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","unstructured":"Yu Deng Baoyuan Wang and Heung-Yeung Shum. 2023. Learning Detailed Radiance Manifolds for High-Fidelity and 3D-Consistent Portrait Synthesis From Monocular Image. https:\/\/doi.org\/10.1109\/CVPR52729.2023.02017 arXiv:2211.13901 [cs.CV]","DOI":"10.1109\/CVPR52729.2023.02017"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3592460"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3190834.3190843"},{"key":"e_1_2_2_18_1","article-title":"Montage4D: Real-time Seamless Fusion and Stylization of Multiview Video Textures","volume":"8","author":"Du Ruofei","year":"2019","unstructured":"Ruofei Du, Ming Chuang, Wayne Chang, Hugues Hoppe, and Amitabh Varshney. 2019. Montage4D: Real-time Seamless Fusion and Stylization of Multiview Video Textures. Journal of Computer Graphics Techniques 8, 1 (17 Jan. 2019), 1--34. https:\/\/www.jcgt.org\/published\/0008\/01\/01\/","journal-title":"Journal of Computer Graphics Techniques"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3618399"},{"key":"e_1_2_2_20_1","volume-title":"Barron","author":"Duckworth Daniel","year":"2023","unstructured":"Daniel Duckworth, Peter Hedman, Christian Reiser, Peter Zhizhin, Jean-Fran\u00e7ois Thibert, Mario Lu\u010di\u0107, Richard Szeliski, and Jonathan T. Barron. 2023. SMERF: Streamable Memory Efficient Radiance Fields for Real-Time Large-Scene Exploration. arXiv:2312.07541 [cs.CV]"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3395208"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2011.01888.x"},{"key":"e_1_2_2_23_1","unstructured":"Stephan J. Garbin Marek Kowalski Virginia Estellers Stanislaw Szymanowicz Shideh Rezaeifar Jingjing Shen Matthew Johnson and Julien Valentin. 2022. VolTeMorph: Realtime Controllable and Generalisable Animation of Volumetric Representations. arXiv:2208.00949 [cs.GR]"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01408"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275073"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01810"},{"key":"e_1_2_2_27_1","volume-title":"Implicit Geometric Regularization for Learning Shapes. ArXiv Preprint ArXiv:2002.10099","author":"Gropp Amos","year":"2020","unstructured":"Amos Gropp, Lior Yariv, Niv Haim, Matan Atzmon, and Yaron Lipman. 2020. Implicit Geometric Regularization for Learning Shapes. ArXiv Preprint ArXiv:2002.10099 (2020). https:\/\/arxiv.org\/pdf\/2002.10099"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356571"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","unstructured":"Peter Hedman Pratul P. Srinivasan Ben Mildenhall Jonathan T. Barron and Paul Debevec. 2021. Baking Neural Radiance Fields for Real-Time View Synthesis. In 10.1109\/ICCV48922.2021.00582. 5855--5864. https:\/\/doi.org\/10.1109\/ICCV48922.2021.00582","DOI":"10.1109\/ICCV48922.2021.00582"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00047"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487228.2487237x26amp"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3592433"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-20086-1_20"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201283"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1412.6980"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3592455"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530130"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130813"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00817"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/3495724.3497037"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323020"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459863"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i2.20091"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","unstructured":"Ben Mildenhall Pratul P. Srinivasan Matthew Tancik Jonathan T. Barron Ravi Ramamoorthi and Ren Ng. 2020. NeRF: Representing Scenes As Neural Radiance Fields for View Synthesis. In ECCV. https:\/\/doi.org\/10.1145\/3503250","DOI":"10.1145\/3503250"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3503250"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530127"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00356"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2984511.2984517"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00581"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478513.3480487"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2312.02069"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/3592426"},{"key":"e_1_2_2_53_1","doi-asserted-by":"crossref","unstructured":"Shunsuke Saito Gabriel Schwartz Tomas Simon Junxuan Li and Giljoo Nam. 2023. Relightable Gaussian Codec Avatars. (2023). arXiv:2312.03704 [cs.GR]","DOI":"10.1109\/CVPR52733.2024.00021"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00542"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02011"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.5555\/3495724.3496356"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2106.10689"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/tip.2003.819861"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3023573"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2210.06465"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2207.11243"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV51070.2023.00209"},{"key":"e_1_2_2_63_1","doi-asserted-by":"crossref","unstructured":"Yu-Jie Yuan Yang-Tian Sun Yu-Kun Lai Yuewen Ma Rongfei Jia and Lin Gao. 2022. NeRF-Editing: Geometry Editing of Neural Radiance Fields. arXiv:2205.04978 [cs.GR]","DOI":"10.1109\/CVPR52688.2022.01781"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58610-_31"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00068"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01318"},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02017"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201323"},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00444"}],"container-title":["Proceedings of the ACM on Computer Graphics and Interactive Techniques"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3651304","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3651304","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,23]],"date-time":"2025-08-23T01:26:14Z","timestamp":1755912374000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3651304"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,11]]},"references-count":69,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,5,11]]}},"alternative-id":["10.1145\/3651304"],"URL":"https:\/\/doi.org\/10.1145\/3651304","relation":{},"ISSN":["2577-6193"],"issn-type":[{"value":"2577-6193","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,11]]},"assertion":[{"value":"2024-05-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}