{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T15:18:39Z","timestamp":1778080719160,"version":"3.51.4"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,7,19]],"date-time":"2021-07-19T00:00:00Z","timestamp":1626652800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-sa\/4.0\/"}],"funder":[{"name":"Qualcomm Innovation Fellowship"},{"name":"Ronald L. Graham chair"},{"name":"Facebook Distinguished Faculty Award"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2021,8,31]]},"abstract":"<jats:p>\n            We present a method for building high-fidelity animatable 3D face models that can be posed and rendered with novel lighting environments in real-time. Our main insight is that relightable models trained to produce an image lit from a single light direction can generalize to natural illumination conditions but are computationally expensive to render. On the other hand, efficient, high-fidelity face models trained with point-light data do not generalize to novel lighting conditions. We leverage the strengths of each of these two approaches. We first train an expensive but\n            <jats:italic>generalizable model<\/jats:italic>\n            on point-light illuminations, and use it to generate a training set of high-quality synthetic face images under natural illumination conditions. We then train an\n            <jats:italic>efficient model<\/jats:italic>\n            on this augmented dataset, reducing the generalization ability requirements. As the efficacy of this approach hinges on the quality of the synthetic data we can generate, we present a study of lighting pattern combinations for dynamic captures and evaluate their suitability for learning generalizable relightable models. Towards achieving the best possible quality, we present a novel approach for generating dynamic relightable faces that exceeds state-of-the-art performance. Our method is capable of capturing subtle lighting effects and can even generate compelling near-field relighting despite being trained exclusively with far-field lighting data. Finally, we motivate the utility of our model by animating it with images captured from VR-headset mounted cameras, demonstrating the first system for face-driven interactions in VR that uses a photorealistic relightable face model.\n          <\/jats:p>","DOI":"10.1145\/3450626.3459829","type":"journal-article","created":{"date-parts":[[2021,7,20]],"date-time":"2021-07-20T00:04:27Z","timestamp":1626739467000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":73,"title":["Deep relightable appearance models for animatable faces"],"prefix":"10.1145","volume":"40","author":[{"given":"Sai","family":"Bi","sequence":"first","affiliation":[{"name":"University of California"}]},{"given":"Stephen","family":"Lombardi","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}]},{"given":"Shunsuke","family":"Saito","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}]},{"given":"Tomas","family":"Simon","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}]},{"given":"Shih-En","family":"Wei","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}]},{"given":"Kevyn","family":"Mcphail","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}]},{"given":"Ravi","family":"Ramamoorthi","sequence":"additional","affiliation":[{"name":"University of California"}]},{"given":"Yaser","family":"Sheikh","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}]},{"given":"Jason","family":"Saragih","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}]}],"member":"320","published-online":{"date-parts":[[2021,7,19]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"crossref","unstructured":"Oleg Alexander Mike Rogers William Lambeth Matt Chiang and Paul Debevec. 2009. The digital emily project: photoreal facial modeling and animation. In Acm siggraph 2009 courses. 1--15.  Oleg Alexander Mike Rogers William Lambeth Matt Chiang and Paul Debevec. 2009. The digital emily project: photoreal facial modeling and animation. In Acm siggraph 2009 courses. 1--15.","DOI":"10.1145\/1667239.1667251"},{"key":"e_1_2_2_2_1","doi-asserted-by":"crossref","unstructured":"P. B\u00e9rard D. Bradley M. Gross and T. Beeler. 2019. Practical Person-Specific Eye Rigging. Computer Graphics Forum 38 (2019).  P. B\u00e9rard D. Bradley M. Gross and T. Beeler. 2019. Practical Person-Specific Eye Rigging. Computer Graphics Forum 38 (2019).","DOI":"10.1111\/cgf.13650"},{"key":"e_1_2_2_3_1","volume-title":"The mathematics of radiative transfer. Number 50","author":"Busbridge Ida Winifred","unstructured":"Ida Winifred Busbridge . 1960. The mathematics of radiative transfer. Number 50 . University Press. Ida Winifred Busbridge. 1960. The mathematics of radiative transfer. Number 50. University Press."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766943"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344855"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417767"},{"key":"e_1_2_2_7_1","volume-title":"Learning to predict indoor illumination from a single image. arXiv preprint arXiv:1704.00090","author":"Gardner Marc-Andr\u00e9","year":"2017","unstructured":"Marc-Andr\u00e9 Gardner , Kalyan Sunkavalli , Ersin Yumer , Xiaohui Shen , Emiliano Gambaretto , Christian Gagn\u00e9 , and Jean-Fran\u00e7ois Lalonde . 2017. Learning to predict indoor illumination from a single image. arXiv preprint arXiv:1704.00090 ( 2017 ). Marc-Andr\u00e9 Gardner, Kalyan Sunkavalli, Ersin Yumer, Xiaohui Shen, Emiliano Gambaretto, Christian Gagn\u00e9, and Jean-Fran\u00e7ois Lalonde. 2017. Learning to predict indoor illumination from a single image. arXiv preprint arXiv:1704.00090 (2017)."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508380"},{"key":"e_1_2_2_9_1","doi-asserted-by":"crossref","unstructured":"Abhijeet Ghosh Tim Hawkins Pieter Peers Sune Frederiksen and Paul Debevec. 2008. Practical modeling and acquisition of layered facial reflectance. In ACM SIGGRAPH Asia 2008 papers. 1--10.  Abhijeet Ghosh Tim Hawkins Pieter Peers Sune Frederiksen and Paul Debevec. 2008. Practical modeling and acquisition of layered facial reflectance. In ACM SIGGRAPH Asia 2008 papers. 1--10.","DOI":"10.1145\/1457515.1409092"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275073"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.103"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356571"},{"key":"e_1_2_2_13_1","unstructured":"David Ha Andrew Dai and Quoc V Le. 2016. Hypernetworks. NIPS (2016).  David Ha Andrew Dai and Quoc V Le. 2016. Hypernetworks. NIPS (2016)."},{"key":"e_1_2_2_14_1","doi-asserted-by":"crossref","unstructured":"C. Hern\u00e1ndez G. Vogiatzis G. J. Brostow B. Stenger and R. Cipolla. 2007. Non-Rigid Photometric Stereo with Colored Lights. In ICCV (Rio de Janeiro Brazil).  C. Hern\u00e1ndez G. Vogiatzis G. J. Brostow B. Stenger and R. Cipolla. 2007. Non-Rigid Photometric Stereo with Colored Lights. In ICCV (Rio de Janeiro Brazil).","DOI":"10.1109\/ICCV.2007.4408939"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383319"},{"key":"e_1_2_2_16_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_2_17_1","volume-title":"Weinberger (Eds.)","volume":"27","author":"Kingma Durk P","year":"2014","unstructured":"Durk P Kingma , Shakir Mohamed , Danilo Jimenez Rezende , and Max Welling . 2014 . Semi-supervised Learning with Deep Generative Models. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q . Weinberger (Eds.) , Vol. 27 . Curran Associates, Inc., 3581--3589. https:\/\/proceedings.neurips.cc\/paper\/ 2014\/file\/d523773c6b194f37b938d340d5d02232-Paper.pdf Durk P Kingma, Shakir Mohamed, Danilo Jimenez Rezende, and Max Welling. 2014. Semi-supervised Learning with Deep Generative Models. In Advances in Neural Information Processing Systems, Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Q. Weinberger (Eds.), Vol. 27. Curran Associates, Inc., 3581--3589. https:\/\/proceedings.neurips.cc\/paper\/2014\/file\/d523773c6b194f37b938d340d5d02232-Paper.pdf"},{"key":"e_1_2_2_18_1","volume-title":"Proceedings of the 2nd International Conference on Learning Representations","author":"Kingma Diederik P","year":"2013","unstructured":"Diederik P Kingma and Max Welling . 2013 . Auto-encoding variational bayes . Proceedings of the 2nd International Conference on Learning Representations (2013). Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. Proceedings of the 2nd International Conference on Learning Representations (2013)."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201401"},{"key":"e_1_2_2_20_1","first-page":"10","article-title":"Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination","volume":"2007","author":"Ma Wan-Chun","year":"2007","unstructured":"Wan-Chun Ma , Tim Hawkins , Pieter Peers , Charles-Felix Chabert , Malte Weiss , and Paul E Debevec . 2007 . Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination . Rendering Techniques 2007 , 9 (2007), 10 . Wan-Chun Ma, Tim Hawkins, Pieter Peers, Charles-Felix Chabert, Malte Weiss, and Paul E Debevec. 2007. Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination. Rendering Techniques 2007, 9 (2007), 10.","journal-title":"Rendering Techniques"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323027"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417814"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275075"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392493"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3089269.3089276"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.578"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323008"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417821"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323030"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073258"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141987"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/800248.807402"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1731047.1731055"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980233"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275101"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201313"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201364"},{"key":"e_1_2_2_38_1","volume-title":"Freeman","author":"Zhang Xiuming","year":"2020","unstructured":"Xiuming Zhang , Sean Fanello , Yun-Ta Tsai , Tiancheng Sun , Tianfan Xue , Rohit Pandey , Sergio Orts-Escolano , Philip Davidson , Christoph Rhemann , Paul Debevec , Jonathan T. Barron , Ravi Ramamoorthi , and William T . Freeman . 2020 . Neural Light Transport for Relighting and View Synthesis. ACM Transactions on Graphics (TOG) ( 2020). Xiuming Zhang, Sean Fanello, Yun-Ta Tsai, Tiancheng Sun, Tianfan Xue, Rohit Pandey, Sergio Orts-Escolano, Philip Davidson, Christoph Rhemann, Paul Debevec, Jonathan T. Barron, Ravi Ramamoorthi, and William T. Freeman. 2020. Neural Light Transport for Relighting and View Synthesis. ACM Transactions on Graphics (TOG) (2020)."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3450626.3459829","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3450626.3459829","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:20Z","timestamp":1750191440000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3450626.3459829"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,19]]},"references-count":38,"aliases":["10.1145\/3476576.3476647"],"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,8,31]]}},"alternative-id":["10.1145\/3450626.3459829"],"URL":"https:\/\/doi.org\/10.1145\/3450626.3459829","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,19]]},"assertion":[{"value":"2021-07-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}