{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T22:28:52Z","timestamp":1776119332346,"version":"3.50.1"},"reference-count":37,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2017,7,20]],"date-time":"2017-07-20T00:00:00Z","timestamp":1500508800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["IIS-1350323"],"award-info":[{"award-number":["IIS-1350323"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2017,8,31]]},"abstract":"<jats:p>We present a convolutional neural network (CNN) based solution for modeling physically plausible spatially varying surface reflectance functions (SVBRDF) from a single photograph of a planar material sample under unknown natural illumination. Gathering a sufficiently large set of labeled training pairs consisting of photographs of SVBRDF samples and corresponding reflectance parameters, is a difficult and arduous process. To reduce the amount of required labeled training data, we propose to leverage the appearance information embedded in unlabeled images of spatially varying materials to self-augment the training process. Starting from an initial approximative network obtained from a small set of labeled training pairs, we estimate provisional model parameters for each unlabeled training exemplar. Given this provisional reflectance estimate, we then synthesize a novel temporary<jats:italic>labeled<\/jats:italic>training pair by rendering the exact corresponding image under a new lighting condition. After refining the network using these additional training samples, we re-estimate the provisional model parameters for the unlabeled data and repeat the self-augmentation process until convergence. We demonstrate the efficacy of the proposed network structure on spatially varying wood, metals, and plastics, as well as thoroughly validate the effectiveness of the self-augmentation training process.<\/jats:p>","DOI":"10.1145\/3072959.3073641","type":"journal-article","created":{"date-parts":[[2017,7,21]],"date-time":"2017-07-21T12:24:07Z","timestamp":1500639847000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":126,"title":["Modeling surface appearance from a single photograph using self-augmented convolutional neural networks"],"prefix":"10.1145","volume":"36","author":[{"given":"Xiao","family":"Li","sequence":"first","affiliation":[{"name":"University of Science and Technology of China &amp; Microsoft Research Asia"}]},{"given":"Yue","family":"Dong","sequence":"additional","affiliation":[{"name":"Microsoft Research Asia"}]},{"given":"Pieter","family":"Peers","sequence":"additional","affiliation":[{"name":"College of William &amp; Mary"}]},{"given":"Xin","family":"Tong","sequence":"additional","affiliation":[{"name":"Microsoft Research Asia"}]}],"member":"320","published-online":{"date-parts":[[2017,7,20]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925917"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2014.2377712"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2462002"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.37"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024180"},{"key":"e_1_2_2_6_1","volume-title":"Digital Modeling of Material Appearance","author":"Dorsey Julie","unstructured":"Julie Dorsey , Holly Rushmeier , and Franois Sillion . 2008. Digital Modeling of Material Appearance . Morgan Kaufmann Publishers Inc . Julie Dorsey, Holly Rushmeier, and Franois Sillion. 2008. Digital Modeling of Material Appearance. Morgan Kaufmann Publishers Inc."},{"key":"e_1_2_2_7_1","doi-asserted-by":"crossref","unstructured":"Adrien Gaidon Qiao Wang Yohann Cabon and Eleonora Vig. 2016. Virtual Worlds as Proxy for Multi-object Tracking Analysis. In CVPR. 4340--4349. Adrien Gaidon Qiao Wang Yohann Cabon and Eleonora Vig. 2016. Virtual Worlds as Proxy for Multi-object Tracking Analysis. In CVPR. 4340--4349.","DOI":"10.1109\/CVPR.2016.470"},{"key":"e_1_2_2_8_1","unstructured":"Ian J. Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative Adversarial Nets. In NIPS. 2672--2680. Ian J. Goodfellow Jean Pouget-Abadie Mehdi Mirza Bing Xu David Warde-Farley Sherjil Ozair Aaron Courville and Yoshua Bengio. 2014. Generative Adversarial Nets. In NIPS. 2672--2680."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.254"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10584-0_23"},{"key":"e_1_2_2_11_1","unstructured":"HDRLabs. 2016. sIBL Archive. http:\/\/www.hdrlabs.com\/sibl\/archive.html. (2016). HDRLabs. 2016. sIBL Archive. http:\/\/www.hdrlabs.com\/sibl\/archive.html. (2016)."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2003.1211400"},{"key":"e_1_2_2_13_1","volume-title":"Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In ICML. 448--456.","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy . 2015 . Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In ICML. 448--456. Sergey Ioffe and Christian Szegedy. 2015. Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. In ICML. 448--456."},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2647868.2654889"},{"key":"e_1_2_2_15_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba . 2015 . Adam : A Method for Stochastic Optimization. In ICLR. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR."},{"key":"e_1_2_2_16_1","volume-title":"Kingma and Max Welling","author":"Diederik","year":"2014","unstructured":"Diederik P. Kingma and Max Welling . 2014 . Auto-Encoding Variational Bayes. In ICLR. Diederik P. Kingma and Max Welling. 2014. Auto-Encoding Variational Bayes. In ICLR."},{"key":"e_1_2_2_17_1","unstructured":"Ming-Yu Liu and Oncel Tuzel. 2016. Coupled Generative Adversarial Networks. In NIPS. 469--477. Ming-Yu Liu and Oncel Tuzel. 2016. Coupled Generative Adversarial Networks. In NIPS. 469--477."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33783-3_42"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2430318"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-87536-9_99"},{"key":"e_1_2_2_21_1","volume-title":"Yu","author":"Narihira Takuya","year":"2015","unstructured":"Takuya Narihira , Michael Maire , and Stella X . Yu . 2015 . Direct Intrinsics : Learning Albedo-Shading Decomposition by Convolutional Regression. In ICCV. 2992--2992. Takuya Narihira, Michael Maire, and Stella X. Yu. 2015. Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression. In ICCV. 2992--2992."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33718-5_38"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2450734"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.488"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15549-9_4"},{"key":"e_1_2_2_26_1","doi-asserted-by":"crossref","unstructured":"Olaf Ronneberger Philipp Fischer and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In MICCAI Part III. 234--241. Olaf Ronneberger Philipp Fischer and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In MICCAI Part III. 234--241.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_2_27_1","unstructured":"German Ros Laura Sellart Joanna Materzynska David Vazquez and Antonio Lopez. 2016. The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes. In CVPR. 3234--3243. German Ros Laura Sellart Joanna Materzynska David Vazquez and Antonio Lopez. 2016. The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes. In CVPR. 3234--3243."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2015.39"},{"key":"e_1_2_2_29_1","unstructured":"Kihyuk Sohn Xinchen Yan and Honglak Lee. 2015. Learning Structured Output Representation Using Deep Conditional Generative Models. In NIPS. 3483--3491. Kihyuk Sohn Xinchen Yan and Honglak Lee. 2015. Learning Structured Output Representation Using Deep Conditional Generative Models. In NIPS. 3483--3491."},{"key":"e_1_2_2_30_1","volume-title":"Hinton","author":"Tang Yichuan","year":"2012","unstructured":"Yichuan Tang , Ruslan Salakhutdinov , and Geoffrey E . Hinton . 2012 . Deep Lambertian Networks. In ICML. 1623--1630. Yichuan Tang, Ruslan Salakhutdinov, and Geoffrey E. Hinton. 2012. Deep Lambertian Networks. In ICML. 1623--1630."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2629500"},{"key":"e_1_2_2_32_1","unstructured":"VRay. 2016. VRay: Material Library. http:\/\/www.vray-materials.de\/. (2016). VRay. 2016. VRay: Material Library. http:\/\/www.vray-materials.de\/. (2016)."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024206"},{"key":"e_1_2_2_34_1","doi-asserted-by":"crossref","unstructured":"Ting-Chun Wang Manmohan Chandraker Alexei Efros and Ravi Ramamoorthi. 2016. SVBRDF-invariant shape and reflectance estimation from light-field cameras. In CVPR. 5451--5459. Ting-Chun Wang Manmohan Chandraker Alexei Efros and Ravi Ramamoorthi. 2016. SVBRDF-invariant shape and reflectance estimation from light-field cameras. In CVPR. 5451--5459.","DOI":"10.1109\/CVPR.2016.588"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/142920.134078"},{"key":"e_1_2_2_36_1","doi-asserted-by":"crossref","unstructured":"Michael Weinmann and Reinhard Klein. 2015. Advances in Geometry and Reflectance Acquisition. In ACM SIGGRAPH Asia Course Notes. Michael Weinmann and Reinhard Klein. 2015. Advances in Geometry and Reflectance Acquisition. In ACM SIGGRAPH Asia Course Notes.","DOI":"10.1145\/2818143.2818165"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982396"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3072959.3073641","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3072959.3073641","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3072959.3073641","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,24]],"date-time":"2025-06-24T17:07:58Z","timestamp":1750784878000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3072959.3073641"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,7,20]]},"references-count":37,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2017,8,31]]}},"alternative-id":["10.1145\/3072959.3073641"],"URL":"https:\/\/doi.org\/10.1145\/3072959.3073641","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,7,20]]},"assertion":[{"value":"2017-07-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}