{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T22:28:49Z","timestamp":1776119329320,"version":"3.50.1"},"reference-count":44,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,7,30]],"date-time":"2018-07-30T00:00:00Z","timestamp":1532908800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Adobe"},{"DOI":"10.13039\/100007065","name":"Nvidia","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100007065","id-type":"DOI","asserted-by":"crossref"}]},{"name":"ANRT CIFRE scholarship between Inria and Optis"},{"DOI":"10.13039\/100015599","name":"Toyota Research Institute","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100015599","id-type":"DOI","asserted-by":"crossref"}]},{"name":"EU"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,8,31]]},"abstract":"<jats:p>\n            Texture, highlights, and shading are some of many visual cues that allow humans to perceive material appearance in single pictures. Yet, recovering spatially-varying bi-directional reflectance distribution functions (SVBRDFs) from a single image based on such cues has challenged researchers in computer graphics for decades. We tackle lightweight appearance capture by training a deep neural network to automatically extract and make sense of these visual cues. Once trained, our network is capable of recovering per-pixel normal, diffuse albedo, specular albedo and specular roughness from a single picture of a flat surface lit by a hand-held flash. We achieve this goal by introducing several innovations on training data acquisition and network design. For training, we leverage a large dataset of artist-created, procedural SVBRDFs which we sample and render under multiple lighting directions. We further amplify the data by material mixing to cover a wide diversity of shading effects, which allows our network to work across many material classes. Motivated by the observation that distant regions of a material sample often offer complementary visual cues, we design a network that combines an encoder-decoder convolutional track for local feature extraction with a fully-connected track for\n            <jats:italic>global feature<\/jats:italic>\n            extraction and propagation. Many important material effects are view-dependent, and as such ambiguous when observed in a single image. We tackle this challenge by defining the loss as a differentiable SVBRDF similarity metric that compares the\n            <jats:italic>renderings<\/jats:italic>\n            of the predicted maps against renderings of the ground truth from several lighting and viewing directions. Combined together, these novel ingredients bring clear improvement over state of the art methods for single-shot capture of spatially varying BRDFs.\n          <\/jats:p>","DOI":"10.1145\/3197517.3201378","type":"journal-article","created":{"date-parts":[[2018,7,31]],"date-time":"2018-07-31T15:56:23Z","timestamp":1533052583000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":221,"title":["Single-image SVBRDF capture with a rendering-aware deep network"],"prefix":"10.1145","volume":"37","author":[{"given":"Valentin","family":"Deschaintre","sequence":"first","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur"}]},{"given":"Miika","family":"Aittala","sequence":"additional","affiliation":[{"name":"MIT CSAIL"}]},{"given":"Fredo","family":"Durand","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur"}]},{"given":"George","family":"Drettakis","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur"}]},{"given":"Adrien","family":"Bousseau","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur"}]}],"member":"320","published-online":{"date-parts":[[2018,7,30]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Mart\u00edn Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Ian Goodfellow Andrew Harp Geoffrey Irving Michael Isard Yangqing Jia Rafal Jozefowicz Lukasz Kaiser Manjunath Kudlur Josh Levenberg Dan Man\u00e9 Rajat Monga Sherry Moore Derek Murray Chris Olah Mike Schuster Jonathon Shlens Benoit Steiner Ilya Sutskever Kunal Talwar Paul Tucker Vincent Vanhoucke Vijay Vasudevan Fernanda Vi\u00e9gas Oriol Vinyals Pete Warden Martin Wattenberg Martin Wicke Yuan Yu and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. (2015). https:\/\/www.tensorflow.org\/ Software available from tensorflow.org.  Mart\u00edn Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Ian Goodfellow Andrew Harp Geoffrey Irving Michael Isard Yangqing Jia Rafal Jozefowicz Lukasz Kaiser Manjunath Kudlur Josh Levenberg Dan Man\u00e9 Rajat Monga Sherry Moore Derek Murray Chris Olah Mike Schuster Jonathon Shlens Benoit Steiner Ilya Sutskever Kunal Talwar Paul Tucker Vincent Vanhoucke Vijay Vasudevan Fernanda Vi\u00e9gas Oriol Vinyals Pete Warden Martin Wattenberg Martin Wicke Yuan Yu and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. (2015). https:\/\/www.tensorflow.org\/ Software available from tensorflow.org."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925917"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766967"},{"key":"e_1_2_2_4_1","volume-title":"https:\/\/share.allegorithmic.com\/","author":"Share Substance","year":"2018","unstructured":"Allegorithmic. 2018. Substance Share . ( 2018 ). https:\/\/share.allegorithmic.com\/ Allegorithmic. 2018. Substance Share. (2018). https:\/\/share.allegorithmic.com\/"},{"key":"e_1_2_2_5_1","volume-title":"Technical Report","author":"Ashikhmin Michael","unstructured":"Michael Ashikhmin and Simon Premoze . 2007. Distribution-based BRDFs. Technical Report . University of Utah. Michael Ashikhmin and Simon Premoze. 2007. Distribution-based BRDFs. Technical Report. University of Utah."},{"key":"e_1_2_2_6_1","volume-title":"Photographic Image Synthesis with Cascaded Refinement Networks. In International Conference on Computer Vision (ICCV).","author":"Chen Qifeng","year":"2017","unstructured":"Qifeng Chen and Vladlen Koltun . 2017 . Photographic Image Synthesis with Cascaded Refinement Networks. In International Conference on Computer Vision (ICCV). Qifeng Chen and Vladlen Koltun. 2017. Photographic Image Synthesis with Cascaded Refinement Networks. In International Conference on Computer Vision (ICCV)."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/357290.357293"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661283"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024180"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778835"},{"key":"e_1_2_2_11_1","volume-title":"Proc. IEEE Workshop on Identifying Objects Across Variations in Lighting: Psychophysics and Computation","author":"Dror Ron O.","year":"2001","unstructured":"Ron O. Dror , Edward H. Adelson , and Alan S. Willsky . 2001. Recognition of Surface Reflectance Properties from a Single Image under Unknown Real-World Illumination . Proc. IEEE Workshop on Identifying Objects Across Variations in Lighting: Psychophysics and Computation ( 2001 ). Ron O. Dror, Edward H. Adelson, and Alan S. Willsky. 2001. Recognition of Surface Reflectance Properties from a Single Image under Unknown Real-World Illumination. Proc. IEEE Workshop on Identifying Objects Across Variations in Lighting: Psychophysics and Computation (2001)."},{"key":"e_1_2_2_12_1","volume-title":"Abhijeet Ghosh, Cornelia Denk, and Mashhuda Glencross.","author":"Guarnera Dar'ya","year":"2016","unstructured":"Dar'ya Guarnera , Giuseppe Claudio Guarnera , Abhijeet Ghosh, Cornelia Denk, and Mashhuda Glencross. 2016 . BRDF Representation and Acquisition. Computer Graphics Forum ( 2016). Dar'ya Guarnera, Giuseppe Claudio Guarnera, Abhijeet Ghosh, Cornelia Denk, and Mashhuda Glencross. 2016. BRDF Representation and Acquisition. Computer Graphics Forum (2016)."},{"key":"e_1_2_2_13_1","volume-title":"Deep Residual Learning for Image Recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"He Kaiming","year":"2016","unstructured":"Kaiming He , Xiangyu Zhang , Shaoqing Ren , and Jian Sun . 2016 . Deep Residual Learning for Image Recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_14_1","volume-title":"Reflectance Capture Using Univariate Sampling of BRDFs. In IEEE International Conference on Computer Vision (ICCV).","author":"Hui Z.","unstructured":"Z. Hui , K. Sunkavalli , J. Y. Lee , S. Hadap , J. Wang , and A. C. Sankaranarayanan . 2017 . Reflectance Capture Using Univariate Sampling of BRDFs. In IEEE International Conference on Computer Vision (ICCV). Z. Hui, K. Sunkavalli, J. Y. Lee, S. Hadap, J. Wang, and A. C. Sankaranarayanan. 2017. Reflectance Capture Using Univariate Sampling of BRDFs. In IEEE International Conference on Computer Vision (ICCV)."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925974"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13220"},{"key":"e_1_2_2_17_1","volume-title":"Image-to-image Translation with Conditional Adversarial Networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Isola Phillip","year":"2017","unstructured":"Phillip Isola , Jun-Yan Zhu , Tinghui Zhou , and Alexei A Efros . 2017 . Image-to-image Translation with Conditional Adversarial Networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image Translation with Conditional Adversarial Networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_18_1","unstructured":"Wenzel Jakob. 2010. Mitsuba renderer. (2010). http:\/\/www.mitsuba-renderer.org.  Wenzel Jakob. 2010. Mitsuba renderer. (2010). http:\/\/www.mitsuba-renderer.org."},{"key":"e_1_2_2_19_1","volume-title":"International Conference on Learning Representations (ICLR).","author":"Karras Tero","year":"2018","unstructured":"Tero Karras , Timo Aila , Samuli Laine , and Jaakko Lehtinen . 2018 . Progressive Growing of GANs for Improved Quality, Stability, and Variation . In International Conference on Learning Representations (ICLR). Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive Growing of GANs for Improved Quality, Stability, and Variation. In International Conference on Learning Representations (ICLR)."},{"key":"e_1_2_2_20_1","volume-title":"Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations (ICLR).","author":"Diederik","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015 . Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations (ICLR). Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations (ICLR)."},{"key":"e_1_2_2_21_1","unstructured":"G\u00fcnter Klambauer Thomas Unterthiner Andreas Mayr and Sepp Hochreiter. 2017. Self-Normalizing Neural Networks. In Advances in Neural Information Processing Systems (NIPS). 972--981.  G\u00fcnter Klambauer Thomas Unterthiner Andreas Mayr and Sepp Hochreiter. 2017. Self-Normalizing Neural Networks. In Advances in Neural Information Processing Systems (NIPS). 972--981."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/636886.636891"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073641"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.248"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2430318"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.342"},{"key":"e_1_2_2_27_1","doi-asserted-by":"crossref","unstructured":"K. Rematas S. Georgoulis T. Ritschel E. Gavves M. Fritz L. Van Gool and T. Tuytelaars. 2017. Reflectance and Natural Illumination from Single-Material Specular Objects Using Deep Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) (2017).  K. Rematas S. Georgoulis T. Ritschel E. Gavves M. Fritz L. Van Gool and T. Tuytelaars. 2017. Reflectance and Natural Illumination from Single-Material Specular Objects Using Deep Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI) (2017).","DOI":"10.1109\/TPAMI.2017.2742999"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964940"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46475-6_7"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12719"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130894"},{"key":"e_1_2_2_32_1","first-page":"234","article-title":"U-Net: Convolutional Networks for Biomedical Image Segmentation","volume":"9351","author":"Ronneberger O.","year":"2015","unstructured":"O. Ronneberger , P. Fischer , and T. Brox . 2015 . U-Net: Convolutional Networks for Biomedical Image Segmentation . In Medical Image Computing and Computer-Assisted Intervention (MICCAI) (LNCS) , Vol. 9351. 234 -- 241 . O. Ronneberger, P.Fischer, and T. Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI) (LNCS), Vol. 9351. 234--241.","journal-title":"Medical Image Computing and Computer-Assisted Intervention (MICCAI) (LNCS)"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.308"},{"key":"e_1_2_2_34_1","volume-title":"MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction. In IEEE International Conference on Computer Vision (ICCV).","author":"Tewari Ayush","year":"2017","unstructured":"Ayush Tewari , Michael Zoll\u00f6fer , Hyeongwoo Kim , Pablo Garrido , Florian Bernard , Patrick Perez , and Theobalt Christian . 2017 . MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction. In IEEE International Conference on Computer Vision (ICCV). Ayush Tewari, Michael Zoll\u00f6fer, Hyeongwoo Kim, Pablo Garrido, Florian Bernard, Patrick Perez, and Theobalt Christian. 2017. MoFA: Model-based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction. In IEEE International Conference on Computer Vision (ICCV)."},{"key":"e_1_2_2_35_1","volume-title":"Improved Texture Networks: Maximizing Quality and Diversity in Feed-Forward Stylization and Texture Synthesis. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Ulyanov Dmitry","year":"2017","unstructured":"Dmitry Ulyanov , Andrea Vedaldi , and Victor Lempitsky . 2017 . Improved Texture Networks: Maximizing Quality and Diversity in Feed-Forward Stylization and Texture Synthesis. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2017. Improved Texture Networks: Maximizing Quality and Diversity in Feed-Forward Stylization and Texture Synthesis. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_36_1","volume-title":"Proc. of Eurographics Conference on Rendering Techniques (EGSR).","author":"Walter Bruce","unstructured":"Bruce Walter , Stephen R. Marschner , Hongsong Li , and Kenneth E. Torrance . 2007. Microfacet Models for Refraction Through Rough Surfaces . In Proc. of Eurographics Conference on Rendering Techniques (EGSR). Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. 2007. Microfacet Models for Refraction Through Rough Surfaces. In Proc. of Eurographics Conference on Rendering Techniques (EGSR)."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024206"},{"key":"e_1_2_2_38_1","volume-title":"Non-local Neural Networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Wang Xiaolong","year":"2018","unstructured":"Xiaolong Wang , Ross B. Girshick , Abhinav Gupta , and Kaiming He . 2018 . Non-local Neural Networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Xiaolong Wang, Ross B. Girshick, Abhinav Gupta, and Kaiming He. 2018. Non-local Neural Networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10578-9_11"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982396"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073703"},{"key":"e_1_2_2_42_1","volume-title":"Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Zhang Yinda","unstructured":"Yinda Zhang , Shuran Song , Ersin Yumer , Manolis Savva , Joon-Young Lee , Hailin Jin . and Thomas A. Funkhouser . 2017a . Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Yinda Zhang, Shuran Song, Ersin Yumer, Manolis Savva, Joon-Young Lee, Hailin Jin. and Thomas A. Funkhouser. 2017a. Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_43_1","volume-title":"Pyramid Scene Parsing Network. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Zhao Hengshuang","year":"2017","unstructured":"Hengshuang Zhao , Jianping Shi , Xiaojuan Qi , Xiaogang Wang , and Jiaya Jia . 2017 . Pyramid Scene Parsing Network. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Hengshuang Zhao, Jianping Shi, Xiaojuan Qi, Xiaogang Wang, and Jiaya Jia. 2017. Pyramid Scene Parsing Network. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2006.170"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201378","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3197517.3201378","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:39:45Z","timestamp":1750210785000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201378"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,30]]},"references-count":44,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,8,31]]}},"alternative-id":["10.1145\/3197517.3201378"],"URL":"https:\/\/doi.org\/10.1145\/3197517.3201378","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,30]]},"assertion":[{"value":"2018-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}