{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,6]],"date-time":"2026-05-06T15:18:39Z","timestamp":1778080719110,"version":"3.51.4"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2019,7,12]],"date-time":"2019-07-12T00:00:00Z","timestamp":1562889600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2019,8,31]]},"abstract":"<jats:p>We present a novel technique to relight images of human faces by learning a model of facial reflectance from a database of 4D reflectance field data of several subjects in a variety of expressions and viewpoints. Using our learned model, a face can be relit in arbitrary illumination environments using only two original images recorded under spherical color gradient illumination. The output of our deep network indicates that the color gradient images contain the information needed to estimate the full 4D reflectance field, including specular reflections and high frequency details. While capturing spherical color gradient illumination still requires a special lighting setup, reduction to just two illumination conditions allows the technique to be applied to dynamic facial performance capture. We show side-by-side comparisons which demonstrate that the proposed system outperforms the state-of-the-art techniques in both realism and speed.<\/jats:p>","DOI":"10.1145\/3306346.3323027","type":"journal-article","created":{"date-parts":[[2019,7,12]],"date-time":"2019-07-12T19:04:08Z","timestamp":1562958248000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":79,"title":["Deep reflectance fields"],"prefix":"10.1145","volume":"38","author":[{"given":"Abhimitra","family":"Meka","sequence":"first","affiliation":[{"name":"MPI Informatics, Saarland Informatics Campus, Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christian","family":"H\u00e4ne","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rohit","family":"Pandey","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Zollh\u00f6fer","sequence":"additional","affiliation":[{"name":"Stanford University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sean","family":"Fanello","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Graham","family":"Fyffe","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Adarsh","family":"Kowdle","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xueming","family":"Yu","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jay","family":"Busch","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jason","family":"Dourgarian","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Denny","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sofien","family":"Bouaziz","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peter","family":"Lincoln","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matt","family":"Whalen","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Geoff","family":"Harvey","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan","family":"Taylor","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shahram","family":"Izadi","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrea","family":"Tagliasacchi","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul","family":"Debevec","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christian","family":"Theobalt","sequence":"additional","affiliation":[{"name":"MPI Informatics, Saarland Informatics Campus"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Julien","family":"Valentin","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christoph","family":"Rhemann","sequence":"additional","affiliation":[{"name":"Google"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,7,12]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980257"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2014.2377712"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925962"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766924"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311556"},{"key":"e_1_2_2_6_1","volume-title":"Photographic Image Synthesis with Cascaded Refinement Networks. ICCV","author":"Chen Qifeng","year":"2017","unstructured":"Qifeng Chen and Vladlen Koltun . 2017. Photographic Image Synthesis with Cascaded Refinement Networks. ICCV ( 2017 ). Qifeng Chen and Vladlen Koltun. 2017. Photographic Image Synthesis with Cascaded Refinement Networks. ICCV (2017)."},{"key":"e_1_2_2_7_1","volume-title":"SIGGRAPH Asia.","author":"Debevec Paul","unstructured":"Paul Debevec . 2012. The Light Stages and Their Applications to Photoreal Digital Actors . In SIGGRAPH Asia. Singapore . Paul Debevec. 2012. The Light Stages and Their Applications to Photoreal Digital Actors. In SIGGRAPH Asia. Singapore."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344855"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.5555\/2383894.2383917"},{"key":"e_1_2_2_10_1","doi-asserted-by":"crossref","unstructured":"S. M. Ali Eslami Danilo Jimenez Rezende Frederic Besse Fabio Viola Ari S. Morcos Marta Garnelo Avraham Ruderman Andrei A. Rusu Ivo Danihelka Karol Gregor David P. Reichert Lars Buesing Theophane Weber Oriol Vinyals Dan Rosenbaum Neil Rabinowitz Helen King Chloe Hillier Matt Botvinick Daan Wierstra Koray Kavukcuoglu and Demis Hassabis. 2018. Neural scene representation and rendering. Science 360 6394 (2018).  S. M. Ali Eslami Danilo Jimenez Rezende Frederic Besse Fabio Viola Ari S. Morcos Marta Garnelo Avraham Ruderman Andrei A. Rusu Ivo Danihelka Karol Gregor David P. Reichert Lars Buesing Theophane Weber Oriol Vinyals Dan Rosenbaum Neil Rabinowitz Helen King Chloe Hillier Matt Botvinick Daan Wierstra Koray Kavukcuoglu and Demis Hassabis. 2018. Neural scene representation and rendering. Science 360 6394 (2018).","DOI":"10.1126\/science.aar6170"},{"key":"e_1_2_2_11_1","doi-asserted-by":"crossref","unstructured":"Graham Fyffe and Paul Debevec. 2015. Single-Shot Reflectance Measurement from Polarized Color Gradient Illumination. In ICCP.  Graham Fyffe and Paul Debevec. 2015. Single-Shot Reflectance Measurement from Polarized Color Gradient Illumination. In ICCP.","DOI":"10.1109\/ICCPHOT.2015.7168375"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1599301.1599381"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508380"},{"key":"e_1_2_2_14_1","doi-asserted-by":"crossref","unstructured":"Pablo Garrido Michael Zollhoefer Dan Casas Levi Valgaerts Kiran Varanasi Patrick Perez and Christian Theobalt. 2016. Reconstruction of Personalized 3D Face Rigs from Monocular Video. (2016).  Pablo Garrido Michael Zollhoefer Dan Casas Levi Valgaerts Kiran Varanasi Patrick Perez and Christian Theobalt. 2016. Reconstruction of Personalized 3D Face Rigs from Monocular Video. (2016).","DOI":"10.1145\/2890493"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024163"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275073"},{"key":"e_1_2_2_17_1","volume-title":"Animatable Facial Reflectance Fields. Rendering Techniques","author":"Hawkins Tim","year":"2004","unstructured":"Tim Hawkins , Andreas Wenger , Chris Tchou , Andrew Gardner , Fredrik G\u00f6ransson , and Paul E Debevec . 2004. Animatable Facial Reflectance Fields. Rendering Techniques ( 2004 ). Tim Hawkins, Andreas Wenger, Chris Tchou, Andrew Gardner, Fredrik G\u00f6ransson, and Paul E Debevec. 2004. Animatable Facial Reflectance Fields. Rendering Techniques (2004)."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766931"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766974"},{"key":"e_1_2_2_20_1","volume-title":"Image-to-Image Translation with Conditional Adversarial Networks. arxiv","author":"Isola Phillip","year":"2016","unstructured":"Phillip Isola , Jun-Yan Zhu , Tinghui Zhou , and Alexei A Efros . 2016. Image-to-Image Translation with Conditional Adversarial Networks. arxiv ( 2016 ). Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2016. Image-to-Image Translation with Conditional Adversarial Networks. arxiv (2016)."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275104"},{"key":"e_1_2_2_22_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba . 2014 . Adam : A Method for Stochastic Optimization. CoRR ( 2014). Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR (2014)."},{"key":"e_1_2_2_23_1","volume-title":"PatchFCN for Intracranial Hemorrhage Detection. arXiv preprint arXiv:1806.03265","author":"Kuo Weicheng","year":"2018","unstructured":"Weicheng Kuo , Christian H\u00e4ne , Esther Yuh , Pratik Mukherjee , and Jitendra Malik . 2018. PatchFCN for Intracranial Hemorrhage Detection. arXiv preprint arXiv:1806.03265 ( 2018 ). Weicheng Kuo, Christian H\u00e4ne, Esther Yuh, Pratik Mukherjee, and Jitendra Malik. 2018. PatchFCN for Intracranial Hemorrhage Detection. arXiv preprint arXiv:1806.03265 (2018)."},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12047"},{"key":"e_1_2_2_25_1","volume-title":"ECCV (Lecture Notes in Computer Science)","author":"Li Zhengqin","unstructured":"Zhengqin Li , Kalyan Sunkavalli , and Manmohan Chandraker . 2018a. Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image . In ECCV (Lecture Notes in Computer Science) . Springer . Zhengqin Li, Kalyan Sunkavalli, and Manmohan Chandraker. 2018a. Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image. In ECCV (Lecture Notes in Computer Science). Springer."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275055"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201401"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"e_1_2_2_29_1","volume-title":"Proceedings of the Eurographics Conference on Rendering Techniques (EGSR'07)","author":"Ma Wan-Chun","year":"2007","unstructured":"Wan-Chun Ma , Tim Hawkins , Pieter Peers , Charles-Felix Chabert , Malte Weiss , and Paul Debevec . 2007 . Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination . In Proceedings of the Eurographics Conference on Rendering Techniques (EGSR'07) . Wan-Chun Ma, Tim Hawkins, Pieter Peers, Charles-Felix Chabert, Malte Weiss, and Paul Debevec. 2007. Rapid Acquisition of Specular and Diffuse Normal Maps from Polarized Spherical Gradient Illumination. In Proceedings of the Eurographics Conference on Rendering Techniques (EGSR'07)."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275099"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2017.2734425"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00661"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13225"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275017"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1275808.1276442"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766899"},{"key":"e_1_2_2_37_1","volume-title":"U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI","author":"Ronneberger Olaf","year":"2015","unstructured":"Olaf Ronneberger , Philipp Fischer , and Thomas Brox . 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI ( 2015 ). Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. MICCAI (2015)."},{"key":"e_1_2_2_38_1","volume-title":"Photorealistic Facial Texture Inference Using Deep Neural Networks","author":"Saito Shunsuke","unstructured":"Shunsuke Saito , Lingyu Wei , Liwen Hu , Koki Nagano , and Hao Li. 2017. Photorealistic Facial Texture Inference Using Deep Neural Networks . In CVPR. IEEE Computer Society , 2326--2335. Shunsuke Saito, Lingyu Wei, Liwen Hu, Koki Nagano, and Hao Li. 2017. Photorealistic Facial Texture Inference Using Deep Neural Networks. In CVPR. IEEE Computer Society, 2326--2335."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601137"},{"key":"e_1_2_2_40_1","unstructured":"Zhixin Shu Sunil Hadap Eli Shechtman Kalyan Sunkavalli Sylvain Paris and Dimitris Samaras. 2017. Portrait Lighting Transfer Using a Mass Transport Approach. ACM Trans. Graph. (2017).  Zhixin Shu Sunil Hadap Eli Shechtman Kalyan Sunkavalli Sylvain Paris and Dimitris Samaras. 2017. Portrait Lighting Transfer Using a Mass Transport Approach. ACM Trans. Graph. (2017)."},{"key":"e_1_2_2_41_1","unstructured":"K. Simonyan and A. Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR abs\/1409.1556 (2014).  K. Simonyan and A. Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR abs\/1409.1556 (2014)."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2007.1006"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2929464.2929475"},{"key":"e_1_2_2_44_1","unstructured":"Zhen Wen Zicheng Liu and T. S. Huang. 2003. Face relighting with radiance environment maps. In CVPR.  Zhen Wen Zicheng Liu and T. S. Huang. 2003. Face relighting with radiance environment maps. In CVPR."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1186822.1073258"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201313"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201364"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073627"},{"key":"e_1_2_2_49_1","volume-title":"The unreasonable effectiveness of deep features as a perceptual metric. CVPR","author":"Zhang Richard","year":"2018","unstructured":"Richard Zhang , Phillip Isola , Alexei A Efros , Eli Shechtman , and Oliver Wang . 2018. The unreasonable effectiveness of deep features as a perceptual metric. CVPR ( 2018 ). Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. CVPR (2018)."},{"key":"e_1_2_2_50_1","unstructured":"Jun-Yan Zhu Taesung Park Phillip Isola and Alexei A Efros. 2017. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. In ICCV.  Jun-Yan Zhu Taesung Park Phillip Isola and Alexei A Efros. 2017. Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks. In ICCV."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3306346.3323027","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3306346.3323027","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:25:52Z","timestamp":1750206352000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3306346.3323027"}},"subtitle":["high-quality facial reflectance field inference from color gradient illumination"],"short-title":[],"issued":{"date-parts":[[2019,7,12]]},"references-count":50,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,8,31]]}},"alternative-id":["10.1145\/3306346.3323027"],"URL":"https:\/\/doi.org\/10.1145\/3306346.3323027","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,12]]},"assertion":[{"value":"2019-07-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}