{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T06:07:05Z","timestamp":1784268425049,"version":"3.55.0"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2018,12,4]],"date-time":"2018-12-04T00:00:00Z","timestamp":1543881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,12,31]]},"abstract":"<jats:p>Reconstructing shape and reflectance properties from images is a highly under-constrained problem, and has previously been addressed by using specialized hardware to capture calibrated data or by assuming known (or highly constrained) shape or reflectance. In contrast, we demonstrate that we can recover non-Lambertian, spatially-varying BRDFs and complex geometry belonging to any arbitrary shape class, from a single RGB image captured under a combination of unknown environment illumination and flash lighting. We achieve this by training a deep neural network to regress shape and reflectance from the image. Our network is able to address this problem because of three novel contributions: first, we build a large-scale dataset of procedurally generated shapes and real-world complex SVBRDFs that approximate real world appearance well. Second, single image inverse rendering requires reasoning at multiple scales, and we propose a cascade network structure that allows this in a tractable manner. Finally, we incorporate an in-network rendering layer that aids the reconstruction task by handling global illumination effects that are important for real-world scenes. Together, these contributions allow us to tackle the entire inverse rendering problem in a holistic manner and produce state-of-the-art results on both synthetic and real data.<\/jats:p>","DOI":"10.1145\/3272127.3275055","type":"journal-article","created":{"date-parts":[[2018,11,28]],"date-time":"2018-11-28T19:16:10Z","timestamp":1543432570000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":294,"title":["Learning to reconstruct shape and spatially-varying reflectance from a single image"],"prefix":"10.1145","volume":"37","author":[{"given":"Zhengqin","family":"Li","sequence":"first","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zexiang","family":"Xu","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ravi","family":"Ramamoorthi","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kalyan","family":"Sunkavalli","sequence":"additional","affiliation":[{"name":"Adobe Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manmohan","family":"Chandraker","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,12,4]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925917"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766967"},{"key":"e_1_2_2_3_1","doi-asserted-by":"crossref","unstructured":"Aayush Bansal Bryan Russell and Abhinav Gupta. 2016. Marr Revisited: 2D-3D Model Alignment via Surface Normal Prediction. In CVPR.  Aayush Bansal Bryan Russell and Abhinav Gupta. 2016. Marr Revisited: 2D-3D Model Alignment via Surface Normal Prediction. In CVPR.","DOI":"10.1109\/CVPR.2016.642"},{"key":"e_1_2_2_4_1","volume-title":"illumination, and reflectance from shading. PAMI 37, 8","author":"Barron Jonathan T","year":"2015","unstructured":"Jonathan T Barron and Jitendra Malik . 2015. Shape , illumination, and reflectance from shading. PAMI 37, 8 ( 2015 ). Jonathan T Barron and Jitendra Malik. 2015. Shape, illumination, and reflectance from shading. PAMI 37, 8 (2015)."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46487-9_38"},{"key":"e_1_2_2_6_1","doi-asserted-by":"crossref","unstructured":"Sean Bell Paul Upchurch Noah Snavely and Kavita Bala. 2015. Material Recognition in the Wild with the Materials in Context Database. In CVPR.  Sean Bell Paul Upchurch Noah Snavely and Kavita Bala. 2015. Material Recognition in the Wild with the Materials in Context Database. In CVPR.","DOI":"10.1109\/CVPR.2015.7298970"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311556"},{"key":"e_1_2_2_8_1","doi-asserted-by":"crossref","unstructured":"Manmohan Chandraker. 2014. On shape and material recovery from motion. In ECCV.  Manmohan Chandraker. 2014. On shape and material recovery from motion. In ECCV.","DOI":"10.1007\/978-3-319-10584-0_14"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.299"},{"key":"e_1_2_2_10_1","volume-title":"Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012","author":"Chang Angel X","year":"2015","unstructured":"Angel X Chang , Thomas Funkhouser , Leonidas Guibas , Pat Hanrahan , Qixing Huang , Zimo Li , Silvio Savarese , Manolis Savva , Shuran Song , Hao Su , 2015 . Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012 (2015). Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. 2015. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012 (2015)."},{"key":"e_1_2_2_11_1","volume-title":"Radiosity and realistic image synthesis","author":"Cohen Michael F","unstructured":"Michael F Cohen and John R Wallace . 1993. Radiosity and realistic image synthesis . Elsevier . Michael F Cohen and John R Wallace. 1993. Radiosity and realistic image synthesis. Elsevier."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344855"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201378"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.304"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130891"},{"key":"e_1_2_2_16_1","doi-asserted-by":"crossref","unstructured":"Stamatios Georgoulis Konstantinos Rematas Tobias Ritschel Mario Fritz Tinne Tuytelaars and Luc Van Gool. 2017. What is around the camera?. In ICCV.  Stamatios Georgoulis Konstantinos Rematas Tobias Ritschel Mario Fritz Tinne Tuytelaars and Luc Van Gool. 2017. What is around the camera?. In ICCV.","DOI":"10.1109\/ICCV.2017.553"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2015.10"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.102"},{"key":"e_1_2_2_19_1","unstructured":"Kaiming He Xiangyu Zhang Shaoqing Ren and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR.  Kaiming He Xiangyu Zhang Shaoqing Ren and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR."},{"key":"e_1_2_2_20_1","doi-asserted-by":"crossref","unstructured":"Yannick Hold-Geoffroy Kalyan Sunkavalli Sunil Hadap Emiliano Gambaretto and Jean-Fran\u00e7ois Lalonde. 2017. Deep Outdoor Illumination Estimation. In CVPR.  Yannick Hold-Geoffroy Kalyan Sunkavalli Sunil Hadap Emiliano Gambaretto and Jean-Fran\u00e7ois Lalonde. 2017. Deep Outdoor Illumination Estimation. In CVPR.","DOI":"10.1109\/CVPR.2017.255"},{"key":"e_1_2_2_21_1","doi-asserted-by":"crossref","unstructured":"Z. Hui and A. C. Sankaranarayanan. 2017. Shape and Spatially-Varying Reflectance Estimation from Virtual Exemplars. PAMI 39 10 (2017).  Z. Hui and A. C. Sankaranarayanan. 2017. Shape and Spatially-Varying Reflectance Estimation from Virtual Exemplars. PAMI 39 10 (2017).","DOI":"10.1109\/TPAMI.2016.2623613"},{"key":"e_1_2_2_22_1","volume-title":"Sankaranarayanan","author":"Hui Zhuo","year":"2017","unstructured":"Zhuo Hui , Kalyan Sunkavalli , Joon-Young Lee , Sunil Hadap , Jian Wang , and Aswin C . Sankaranarayanan . 2017 . Reflectance capture using univariate sampling of BRDFs. In ICCV. Zhuo Hui, Kalyan Sunkavalli, Joon-Young Lee, Sunil Hadap, Jian Wang, and Aswin C. Sankaranarayanan. 2017. Reflectance capture using univariate sampling of BRDFs. In ICCV."},{"key":"e_1_2_2_23_1","doi-asserted-by":"crossref","unstructured":"Eddy Ilg Nikolaus Mayer Tonmoy Saikia Margret Keuper Alexey Dosovitskiy and Thomas Brox. 2017. FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks. In CVPR.  Eddy Ilg Nikolaus Mayer Tonmoy Saikia Margret Keuper Alexey Dosovitskiy and Thomas Brox. 2017. FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks. In CVPR.","DOI":"10.1109\/CVPR.2017.179"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13220"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995510"},{"key":"e_1_2_2_26_1","volume-title":"SIGGRAPH 2013 Courses: Physically Based Shading Theory Practice","author":"Karis Brian","year":"2013","unstructured":"Brian Karis and Epic Games . 2013 . Real shading in Unreal Engine 4 . SIGGRAPH 2013 Courses: Physically Based Shading Theory Practice (2013). Brian Karis and Epic Games. 2013. Real shading in Unreal Engine 4. SIGGRAPH 2013 Courses: Physically Based Shading Theory Practice (2013)."},{"key":"e_1_2_2_27_1","volume-title":"Interactive BRDF Estimation for Mixed-Reality Applications. WSCG 20, 1","author":"Knecht Martin","year":"2012","unstructured":"Martin Knecht , Georg Tanzmeister , Christoph Traxler , and Michael Wimmer . 2012. Interactive BRDF Estimation for Mixed-Reality Applications. WSCG 20, 1 ( 2012 ). Martin Knecht, Georg Tanzmeister, Christoph Traxler, and Michael Wimmer. 2012. Interactive BRDF Estimation for Mixed-Reality Applications. WSCG 20, 1 (2012)."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073641"},{"key":"e_1_2_2_29_1","unstructured":"Zhengqin Li Kalyan Sunkavalli and Manmohan Chandraker. 2018. Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image. In ECCV.  Zhengqin Li Kalyan Sunkavalli and Manmohan Chandraker. 2018. Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image. In ECCV."},{"key":"e_1_2_2_30_1","doi-asserted-by":"crossref","unstructured":"Z. Li Z. Xu R. Ramamoorthi and M. Chandraker. 2017b. Robust Energy Minimization for BRDF-Invariant Shape from Light Fields. In CVPR.  Z. Li Z. Xu R. Ramamoorthi and M. Chandraker. 2017b. Robust Energy Minimization for BRDF-Invariant Shape from Light Fields. In CVPR.","DOI":"10.1109\/CVPR.2017.69"},{"key":"e_1_2_2_31_1","unstructured":"Guilin Liu Duygu Ceylan Ersin Yumer Jimei Yang and Jyh-Ming Lien. 2017. Material Editing using a Physically Based Rendering Network. ICCV.  Guilin Liu Duygu Ceylan Ersin Yumer Jimei Yang and Jyh-Ming Lien. 2017. Material Editing using a Physically Based Rendering Network. ICCV."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130884"},{"key":"e_1_2_2_33_1","volume-title":"Kenneth E Torrance, and Donald P Greenberg.","author":"Marschner Stephen R","year":"1999","unstructured":"Stephen R Marschner , Stephen H Westin , Eric PF Lafortune , Kenneth E Torrance, and Donald P Greenberg. 1999 . Image-based BRDF measurement including human skin. In Rendering Techniques . Stephen R Marschner, Stephen H Westin, Eric PF Lafortune, Kenneth E Torrance, and Donald P Greenberg. 1999. Image-based BRDF measurement including human skin. In Rendering Techniques."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882343"},{"key":"e_1_2_2_35_1","volume-title":"LIME: Live Intrinsic Material Estimation. In CVPR.","author":"Meka Abhimitra","year":"2018","unstructured":"Abhimitra Meka , Maxim Maximov , Michael Zollhoefer , Avishek Chatterjee , Hans-Peter Seidel , Christian Richardt , and Christian Theobalt . 2018 . LIME: Live Intrinsic Material Estimation. In CVPR. Abhimitra Meka, Maxim Maximov, Michael Zollhoefer, Avishek Chatterjee, Hans-Peter Seidel, Christian Richardt, and Christian Theobalt. 2018. LIME: Live Intrinsic Material Estimation. In CVPR."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13225"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00115695"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141977"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073226"},{"key":"e_1_2_2_40_1","doi-asserted-by":"crossref","unstructured":"Alejandro Newell Kaiyu Yang and Jia Deng. 2016. Stacked Hourglass Networks for Human Pose Estimation. In ECCV.  Alejandro Newell Kaiyu Yang and Jia Deng. 2016. Stacked Hourglass Networks for Human Pose Estimation. In ECCV.","DOI":"10.1007\/978-3-319-46484-8_29"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1882261.1866165"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2450734"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383317"},{"key":"e_1_2_2_44_1","doi-asserted-by":"crossref","unstructured":"Konstantinos Rematas Tobias Ritschel Mario Fritz Efstratios Gavves and Tinne Tuytelaars. 2016. Deep reflectance maps. In CVPR.  Konstantinos Rematas Tobias Ritschel Mario Fritz Efstratios Gavves and Tinne Tuytelaars. 2016. Deep reflectance maps. In CVPR.","DOI":"10.1109\/CVPR.2016.488"},{"key":"e_1_2_2_45_1","doi-asserted-by":"crossref","unstructured":"Kosta Ristovski Vladan Radosavljevic Slobodan Vucetic and Zoran Obradovic. 2013. Continuous Conditional Random Fields for Efficient Regression in Large Fully Connected Graphs.. In AAAI.   Kosta Ristovski Vladan Radosavljevic Slobodan Vucetic and Zoran Obradovic. 2013. Continuous Conditional Random Fields for Efficient Regression in Large Fully Connected Graphs.. In AAAI.","DOI":"10.1609\/aaai.v27i1.8685"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12719"},{"key":"e_1_2_2_47_1","doi-asserted-by":"crossref","unstructured":"O. Ronneberger P.Fischer and T. Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In MICCAI.  O. Ronneberger P.Fischer and T. Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In MICCAI.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_2_48_1","volume-title":"Jacobs","author":"Sengupta Soumyadip","year":"2018","unstructured":"Soumyadip Sengupta , Angjoo Kanazawa , Carlos D. Castillo , and David W . Jacobs . 2018 . SfSNet: Learning Shape , Refectance and Illuminance of Faces in the Wild. In CVPR. Soumyadip Sengupta, Angjoo Kanazawa, Carlos D. Castillo, and David W. Jacobs. 2018. SfSNet: Learning Shape, Refectance and Illuminance of Faces in the Wild. In CVPR."},{"key":"e_1_2_2_49_1","doi-asserted-by":"crossref","unstructured":"Jian Shi Yue Dong Hao Su and Stella X Yu. 2017. Learning Non-Lambertian Object Intrinsics Across ShapeNet Categories. In CVPR.  Jian Shi Yue Dong Hao Su and Stella X Yu. 2017. Learning Non-Lambertian Object Intrinsics Across ShapeNet Categories. In CVPR.","DOI":"10.1109\/CVPR.2017.619"},{"key":"e_1_2_2_50_1","doi-asserted-by":"crossref","unstructured":"Z. Shu E. Yumer S. Hadap K. Sunkavalli E. Shechtman and D. Samaras. 2017. Neural Face Editing with Intrinsic Image Disentangling. In CVPR.  Z. Shu E. Yumer S. Hadap K. Sunkavalli E. Shechtman and D. Samaras. 2017. Neural Face Editing with Intrinsic Image Disentangling. In CVPR.","DOI":"10.1109\/CVPR.2017.578"},{"key":"e_1_2_2_51_1","doi-asserted-by":"crossref","unstructured":"A. Tewari M. Zollhofer H. Kim P. Garrido F. Bernard P. Perez and C. Theobalt. 2018. MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction. In ICCV.  A. Tewari M. Zollhofer H. Kim P. Garrido F. Bernard P. Perez and C. Theobalt. 2018. MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction. In ICCV.","DOI":"10.1109\/ICCV.2017.401"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.214"},{"key":"e_1_2_2_53_1","volume-title":"SVBRDF-Invariant Shape and Reflectance Estimation from Light-Field Cameras. PAMI","author":"Wang Ting-Chun","year":"2017","unstructured":"Ting-Chun Wang , Manmohan Chandraker , Alexei Efros , and Ravi Ramamoorthi . 2017. SVBRDF-Invariant Shape and Reflectance Estimation from Light-Field Cameras. PAMI ( 2017 ). Ting-Chun Wang, Manmohan Chandraker, Alexei Efros, and Ravi Ramamoorthi. 2017. SVBRDF-Invariant Shape and Reflectance Estimation from Light-Field Cameras. PAMI (2017)."},{"key":"e_1_2_2_54_1","doi-asserted-by":"crossref","unstructured":"S. E. Wei V. Ramakrishna T. Kanade and Y. Sheikh. 2016. Convolutional Pose Machines. In CVPR.  S. E. Wei V. Ramakrishna T. Kanade and Y. Sheikh. 2016. Convolutional Pose Machines. In CVPR.","DOI":"10.1109\/CVPR.2016.511"},{"key":"e_1_2_2_55_1","volume-title":"Photometric Method For Determining Surface Orientation From Multiple Images. Optical Engineering 19","author":"Woodham Robert J.","year":"1980","unstructured":"Robert J. Woodham . 1980. Photometric Method For Determining Surface Orientation From Multiple Images. Optical Engineering 19 ( 1980 ). Robert J. Woodham. 1980. Photometric Method For Determining Surface Orientation From Multiple Images. Optical Engineering 19 (1980)."},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12600"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201313"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311559"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3272127.3275055","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3272127.3275055","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:44:04Z","timestamp":1750207444000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3272127.3275055"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,12,4]]},"references-count":58,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2018,12,31]]}},"alternative-id":["10.1145\/3272127.3275055"],"URL":"https:\/\/doi.org\/10.1145\/3272127.3275055","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,12,4]]},"assertion":[{"value":"2018-12-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}