{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T06:07:18Z","timestamp":1784268438516,"version":"3.55.0"},"reference-count":68,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2023,11,30]],"date-time":"2023-11-30T00:00:00Z","timestamp":1701302400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2022YFF0902200"],"award-info":[{"award-number":["2022YFF0902200"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"name":"NSFC project","award":["62125107, and 61827805"],"award-info":[{"award-number":["62125107, and 61827805"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2024,2,29]]},"abstract":"<jats:p>The problem of modeling an animatable 3D human head avatar under lightweight setups is of significant importance but has not been well solved. Existing 3D representations either perform well in the realism of portrait images synthesis or the accuracy of expression control, but not both. To address the problem, we introduce a novel hybrid explicit-implicit 3D representation, Facial Model Conditioned Neural Radiance Field, which integrates the expressiveness of NeRF and the prior information from the parametric template. At the core of our representation, a synthetic-renderings-based condition method is proposed to fuse the prior information from the parametric model into the implicit field without constraining its topological flexibility. Besides, based on the hybrid representation, we properly overcome the inconsistent shape issue presented in existing methods and improve the animation stability. Moreover, by adopting an overall GAN-based architecture using an image-to-image translation network, we achieve high-resolution, realistic and view-consistent synthesis of dynamic head appearance. Experiments demonstrate that our method can achieve state-of-the-art performance for 3D head avatar animation compared with previous methods.<\/jats:p>","DOI":"10.1145\/3626316","type":"journal-article","created":{"date-parts":[[2023,10,2]],"date-time":"2023-10-02T08:24:29Z","timestamp":1696235069000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":43,"title":["HAvatar: High-fidelity Head Avatar via Facial Model Conditioned Neural Radiance Field"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8976-7723","authenticated-orcid":false,"given":"Xiaochen","family":"Zhao","sequence":"first","affiliation":[{"name":"Tsinghua University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6674-9327","authenticated-orcid":false,"given":"Lizhen","family":"Wang","sequence":"additional","affiliation":[{"name":"Tsinghua University &amp; NNKosmos Technology, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2966-9501","authenticated-orcid":false,"given":"Jingxiang","family":"Sun","sequence":"additional","affiliation":[{"name":"Tsinghua University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8633-4551","authenticated-orcid":false,"given":"Hongwen","family":"Zhang","sequence":"additional","affiliation":[{"name":"Tsinghua University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3426-1634","authenticated-orcid":false,"given":"Jinli","family":"Suo","sequence":"additional","affiliation":[{"name":"Tsinghua University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3215-0225","authenticated-orcid":false,"given":"Yebin","family":"Liu","sequence":"additional","affiliation":[{"name":"Tsinghua University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,11,30]]},"reference":[{"key":"e_1_3_3_2_1","doi-asserted-by":"crossref","unstructured":"ACM Trans. Graph. 2016 35 4 Real-time facial animation with image-based dynamic avatars","DOI":"10.1145\/2897824.2925873"},{"key":"e_1_3_3_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01972"},{"key":"e_1_3_3_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311556"},{"key":"e_1_3_3_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10590-1_20"},{"key":"e_1_3_3_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459806"},{"issue":"4","key":"e_1_3_3_7_1","first-page":"1","article-title":"Authentic volumetric avatars from a phone scan","volume":"41","year":"2022","unstructured":"Chen Cao, Tomas Simon, Jin Kyu Kim, Gabe Schwartz, Michael Zollhoefer, Shun-Suke Saito, Stephen Lombardi, Shih-En Wei, Danielle Belko, Shoou-I Yu, Yaser Sheikh, and Jason Saragih. 2022. Authentic volumetric avatars from a phone scan. ACM Trans. Graph. 41, 4 (2022), 1\u201319.","journal-title":"ACM Trans. Graph."},{"key":"e_1_3_3_8_1","article-title":"FaceWarehouse: A 3D facial expression database for visual computing","author":"Cao Chen","year":"2014","unstructured":"Chen Cao, Yanlin Weng, Shun Zhou, Yiying Tong, and Kun Zhou. 2014. FaceWarehouse: A 3D facial expression database for visual computing. IEEE Trans. Visualiz. Comput. Graph. 20, 3 (2014), 413\u2013425.","journal-title":"IEEE Trans. Visualiz. Comput. Graph."},{"key":"e_1_3_3_9_1","first-page":"16123","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201922)","year":"2022","unstructured":"Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas J. Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, and Gordon Wetzstein. 2022. Efficient geometry-aware 3D generative adversarial networks. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201922). 16123\u201316133."},{"key":"e_1_3_3_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01139"},{"key":"e_1_3_3_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01353"},{"key":"e_1_3_3_12_1","article-title":"MeshGAN: Non-linear 3D morphable models of faces","author":"Cheng Shiyang","year":"2019","unstructured":"Shiyang Cheng, Michael Bronstein, Yuxiang Zhou, Irene Kotsia, Maja Pantic, and Stefanos Zafeiriou. 2019. MeshGAN: Non-linear 3D morphable models of faces. Arxiv Preprint Arxiv:1903.10384 (2019).","journal-title":"Arxiv Preprint Arxiv:1903.10384"},{"key":"e_1_3_3_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01967"},{"key":"e_1_3_3_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TBIOM.2021.3049576"},{"key":"e_1_3_3_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459936"},{"key":"e_1_3_3_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00854"},{"key":"e_1_3_3_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459836"},{"issue":"9","key":"e_1_3_3_18_1","first-page":"4879","article-title":"Fast-GANFIT: Generative adversarial network for high fidelity 3D face reconstruction","volume":"44","author":"Gecer Baris","year":"2021","unstructured":"Baris Gecer, Stylianos Ploumpis, Irene Kotsia, and Stefanos Zafeiriou. 2021. Fast-GANFIT: Generative adversarial network for high fidelity 3D face reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. 44, 9 (2021), 4879\u20134893.","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"e_1_3_3_19_1","article-title":"Generative adversarial nets","volume":"27","author":"Goodfellow Ian","year":"2014","unstructured":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27 (2014).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_3_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01810"},{"key":"e_1_3_3_21_1","article-title":"StyleNeRF: A style-based 3D-aware generator for high-resolution image synthesis","author":"Gu Jiatao","year":"2021","unstructured":"Jiatao Gu, Lingjie Liu, Peng Wang, and Christian Theobalt. 2021. StyleNeRF: A style-based 3D-aware generator for high-resolution image synthesis. Arxiv Preprint Arxiv:2110.08985 (2021).","journal-title":"Arxiv Preprint Arxiv:2110.08985"},{"key":"e_1_3_3_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00573"},{"key":"e_1_3_3_23_1","article-title":"HeadNeRF: A real-time NeRF-based parametric head model","author":"Hong Yang","year":"2022","unstructured":"Yang Hong, Bo Peng, Haiyao Xiao, Ligang Liu, and Juyong Zhang. 2022. HeadNeRF: A real-time NeRF-based parametric head model. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR\u201922).","journal-title":"Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR\u201922)"},{"key":"e_1_3_3_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3092817"},{"key":"e_1_3_3_25_1","article-title":"Progressive growing of GANs for improved quality, stability, and variation","author":"Karras Tero","year":"2017","unstructured":"Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017. Progressive growing of GANs for improved quality, stability, and variation. Arxiv Preprint Arxiv:1710.10196 (2017).","journal-title":"Arxiv Preprint Arxiv:1710.10196"},{"key":"e_1_3_3_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.2970919"},{"key":"e_1_3_3_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"e_1_3_3_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00427"},{"key":"e_1_3_3_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201283"},{"key":"e_1_3_3_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/FG47880.2020.00048"},{"key":"e_1_3_3_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3125598"},{"key":"e_1_3_3_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778769"},{"key":"e_1_3_3_33_1","unstructured":"Ruilong Li Karl Bladin Yajie Zhao Chinmay Chinara Owen Ingraham Pengda Xiang Xinglei Ren Pratusha Prasad Bipin Kishore Jun Xing et\u00a0al. 2020. Learning formation of physically-based face attributes. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition . 3410\u20133419."},{"key":"e_1_3_3_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130813"},{"key":"e_1_3_3_35_1","first-page":"arXiv\u20132012","article-title":"Real-time high-resolution background matting","author":"Lin Shanchuan","year":"2020","unstructured":"Shanchuan Lin, Andrey Ryabtsev, Soumyadip Sengupta, Brian Curless, Steve Seitz, and Ira Kemelmacher-Shlizerman. 2020. Real-time high-resolution background matting. Arxiv (2020), arXiv\u20132012.","journal-title":"Arxiv"},{"key":"e_1_3_3_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201401"},{"key":"e_1_3_3_37_1","article-title":"Neural volumes: Learning dynamic renderable volumes from images","author":"Lombardi Stephen","year":"2019","unstructured":"Stephen Lombardi, Tomas Simon, Jason Saragih, Gabriel Schwartz, Andreas Lehrmann, and Yaser Sheikh. 2019. Neural volumes: Learning dynamic renderable volumes from images. Arxiv Preprint Arxiv:1906.07751 (2019).","journal-title":"Arxiv Preprint Arxiv:1906.07751"},{"key":"e_1_3_3_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459863"},{"key":"e_1_3_3_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/37402.37422"},{"key":"e_1_3_3_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01149"},{"key":"e_1_3_3_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00013"},{"key":"e_1_3_3_42_1","first-page":"3481","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Mescheder Lars","year":"2018","unstructured":"Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. 2018. Which training methods for GANs do actually converge? In Proceedings of the International Conference on Machine Learning. PMLR, 3481\u20133490."},{"key":"e_1_3_3_43_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-19784-0_11"},{"key":"e_1_3_3_44_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58452-8_24"},{"key":"e_1_3_3_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356568"},{"key":"e_1_3_3_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275075"},{"key":"e_1_3_3_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508417"},{"key":"e_1_3_3_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01129"},{"key":"e_1_3_3_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00025"},{"key":"e_1_3_3_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00581"},{"key":"e_1_3_3_51_1","article-title":"HyperNeRF: A higher-dimensional representation for topologically varying neural radiance fields","author":"Park Keunhong","year":"2021","unstructured":"Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B. Goldman, Ricardo Martin-Brualla, and Steven M. Seitz. 2021b. HyperNeRF: A higher-dimensional representation for topologically varying neural radiance fields. Arxiv Preprint Arxiv:2106.13228 (2021).","journal-title":"Arxiv Preprint Arxiv:2106.13228"},{"key":"e_1_3_3_52_1","article-title":"Accelerating 3D deep learning with PyTorch3D","author":"Ravi Nikhila","year":"2020","unstructured":"Nikhila Ravi, Jeremy Reizenstein, David Novotny, Taylor Gordon, Wan-Yen Lo, Justin Johnson, and Georgia Gkioxari. 2020. Accelerating 3D deep learning with PyTorch3D. Arxiv (2020).","journal-title":"Arxiv"},{"key":"e_1_3_3_53_1","article-title":"Pivotal tuning for latent-based editing of real images","author":"Roich Daniel","year":"2021","unstructured":"Daniel Roich, Ron Mokady, Amit H. Bermano, and Daniel Cohen-Or. 2021. Pivotal tuning for latent-based editing of real images. ACM Trans. Graph. 42, 1, Article 6 (2023), 13 pages.","journal-title":"ACM Trans. Graph."},{"key":"e_1_3_3_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00270"},{"key":"e_1_3_3_55_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58517-4_42"},{"key":"e_1_3_3_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323035"},{"key":"e_1_3_3_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00122"},{"key":"e_1_3_3_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00767"},{"key":"e_1_3_3_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/1185657.1185864"},{"key":"e_1_3_3_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528233.3530753"},{"key":"e_1_3_3_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01969"},{"key":"e_1_3_3_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00565"},{"key":"e_1_3_3_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01573"},{"key":"e_1_3_3_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00773"},{"key":"e_1_3_3_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00068"},{"key":"e_1_3_3_66_1","first-page":"2492","article-title":"Multiview neural surface reconstruction by disentangling geometry and appearance","volume":"33","author":"Yariv Lior","year":"2020","unstructured":"Lior Yariv, Yoni Kasten, Dror Moran, Meirav Galun, Matan Atzmon, Basri Ronen, and Yaron Lipman. 2020. Multiview neural surface reconstruction by disentangling geometry and appearance. Adv. Neural Inf. Process. Syst. 33 (2020), 2492\u20132502.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_3_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01261"},{"key":"e_1_3_3_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00955"},{"key":"e_1_3_3_69_1","article-title":"IM Avatar: Implicit morphable head avatars from videos","author":"Zheng Yufeng","year":"2022","unstructured":"Yufeng Zheng, Victoria Fern\u00e1ndez Abrevaya, Xu Chen, Marcel C. B\u00fchler, Michael J. Black, and Otmar Hilliges. 2022. IM Avatar: Implicit morphable head avatars from videos. In Proceedings of the Computer Vision and Pattern Recognition (CVPR\u201922).","journal-title":"Proceedings of the Computer Vision and Pattern Recognition (CVPR\u201922)"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626316","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3626316","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:53:37Z","timestamp":1750287217000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626316"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,30]]},"references-count":68,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,2,29]]}},"alternative-id":["10.1145\/3626316"],"URL":"https:\/\/doi.org\/10.1145\/3626316","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,11,30]]},"assertion":[{"value":"2022-08-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-22","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}