{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T02:33:18Z","timestamp":1784687598656,"version":"3.55.0"},"reference-count":43,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,7,30]],"date-time":"2018-07-30T00:00:00Z","timestamp":1532908800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,8,31]]},"abstract":"<jats:p>\n            We present an image-based relighting method that can synthesize scene appearance under novel, distant illumination from the visible hemisphere, from only five images captured under pre-defined directional lights. Our method uses a deep convolutional neural network to regress the relit image from these five images; this\n            <jats:italic>relighting network<\/jats:italic>\n            is trained on a large synthetic dataset comprised of procedurally generated shapes with real-world reflectances. We show that by combining a custom-designed\n            <jats:italic>sampling network<\/jats:italic>\n            with the relighting network, we can jointly learn both the optimal input light directions and the relighting function. We present an extensive evaluation of our network, including an empirical analysis of reconstruction quality, optimal lighting configurations for different scenarios, and alternative network architectures. We demonstrate, on both synthetic and real scenes, that our method is able to reproduce complex, high-frequency lighting effects like specularities and cast shadows, and outperforms other image-based relighting methods that require an order of magnitude more images.\n          <\/jats:p>","DOI":"10.1145\/3197517.3201313","type":"journal-article","created":{"date-parts":[[2018,7,31]],"date-time":"2018-07-31T15:56:23Z","timestamp":1533052583000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":135,"title":["Deep image-based relighting from optimal sparse samples"],"prefix":"10.1145","volume":"37","author":[{"given":"Zexiang","family":"Xu","sequence":"first","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kalyan","family":"Sunkavalli","sequence":"additional","affiliation":[{"name":"Adobe Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sunil","family":"Hadap","sequence":"additional","affiliation":[{"name":"Adobe Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ravi","family":"Ramamoorthi","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,7,30]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Marr Revisited: 2D-3D Model Alignment via Surface Normal Prediction. CVPR","author":"Bansal Aayush","year":"2016","unstructured":"Aayush Bansal , Bryan Russell , and Abhinav Gupta . 2016. Marr Revisited: 2D-3D Model Alignment via Surface Normal Prediction. CVPR ( 2016 ). Aayush Bansal, Bryan Russell, and Abhinav Gupta. 2016. Marr Revisited: 2D-3D Model Alignment via Surface Normal Prediction. CVPR (2016)."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2003.1177153"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008005721484"},{"key":"e_1_2_2_4_1","volume-title":"ACM SIGGRAPH 2012 Courses.","author":"Burley Brett","year":"2012","unstructured":"Brett Burley . 2012 . Physically-based shading at Disney . In ACM SIGGRAPH 2012 Courses. Brett Burley. 2012. Physically-based shading at Disney. In ACM SIGGRAPH 2012 Courses."},{"key":"e_1_2_2_5_1","unstructured":"Ayan Chakrabarti. 2016. Learning sensor multiplexing design through back-propagation. In Advances in Neural Information Processing Systems. 3081--3089.   Ayan Chakrabarti. 2016. Learning sensor multiplexing design through back-propagation. In Advances in Neural Information Processing Systems. 3081--3089."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2481415"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344855"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.304"},{"key":"e_1_2_2_9_1","volume-title":"Relighting Human Locomotion with Flowed Reflectance Fields. Rendering techniques 2006","author":"Einarsson Per","year":"2006","unstructured":"Per Einarsson , Charles-Felix Chabert , Andrew Jones , Wan-Chun Ma , Bruce Lamond , Tim Hawkins , Mark T Bolas , Sebastian Sylwan , and Paul E Debevec . 2006. Relighting Human Locomotion with Flowed Reflectance Fields. Rendering techniques 2006 ( 2006 ), 17th. Per Einarsson, Charles-Felix Chabert, Andrew Jones, Wan-Chun Ma, Bruce Lamond, Tim Hawkins, Mark T Bolas, Sebastian Sylwan, and Paul E Debevec. 2006. Relighting Human Locomotion with Flowed Reflectance Fields. Rendering techniques 2006 (2006), 17th."},{"key":"e_1_2_2_10_1","doi-asserted-by":"crossref","unstructured":"John Flynn Ivan Neulander James Philbin and Noah Snavely. 2016. DeepStereo: Learning to Predict New Views From the World's Imagery. In CVPR.  John Flynn Ivan Neulander James Philbin and Noah Snavely. 2016. DeepStereo: Learning to Predict New Views From the World's Imagery. In CVPR.","DOI":"10.1109\/CVPR.2016.595"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1243980.1243984"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130891"},{"key":"e_1_2_2_13_1","doi-asserted-by":"crossref","unstructured":"Stamatios Georgoulis Konstantinos Rematas Tobias Ritschel Mario Fritz Tinne Tuytelaars and Luc Van Gool. 2017. What Is Around The Camera?. In ICCV.  Stamatios Georgoulis Konstantinos Rematas Tobias Ritschel Mario Fritz Tinne Tuytelaars and Luc Van Gool. 2017. What Is Around The Camera?. In ICCV.","DOI":"10.1109\/ICCV.2017.553"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.102"},{"key":"e_1_2_2_15_1","doi-asserted-by":"crossref","unstructured":"Yannick Hold-Geoffroy Kalyan Sunkavalli Sunil Hadap Emiliano Gambaretto and Jean-Fran\u00e7ois Lalonde. 2017. Deep Outdoor Illumination Estimation. In CVPR.  Yannick Hold-Geoffroy Kalyan Sunkavalli Sunil Hadap Emiliano Gambaretto and Jean-Fran\u00e7ois Lalonde. 2017. Deep Outdoor Illumination Estimation. In CVPR.","DOI":"10.1109\/CVPR.2017.255"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2623613"},{"key":"e_1_2_2_17_1","unstructured":"Wenzel Jakob. 2010. Mitsuba renderer. (2010). http:\/\/www.mitsuba-renderer.org.  Wenzel Jakob. 2010. Mitsuba renderer. (2010). http:\/\/www.mitsuba-renderer.org."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980251"},{"key":"e_1_2_2_19_1","unstructured":"Eric P Lafortune and Yves D Willems. 1993. Bi-directional path tracing. (1993).  Eric P Lafortune and Yves D Willems. 1993. Bi-directional path tracing. (1993)."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073641"},{"key":"e_1_2_2_21_1","unstructured":"Guilin Liu Duygu Ceylan Ersin Yumer Jimei Yang and Jyh-Ming Lien. 2017. Material Editing Using a Physically Based Rendering Network. In ICCV. 2261--2269.  Guilin Liu Duygu Ceylan Ersin Yumer Jimei Yang and Jyh-Ming Lien. 2017. Material Editing Using a Physically Based Rendering Network. In ICCV. 2261--2269."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276454"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383320"},{"key":"e_1_2_2_24_1","volume-title":"Eurographics Workshop on Rendering, Alexander Keller and Henrik Wann Jensen (Eds.).","author":"Matusik Wojciech","year":"2004","unstructured":"Wojciech Matusik , Matthew Loper , and Hanspeter Pfister . 2004 . Progressively-Refined Reflectance Functions from Natural Illumination . In Eurographics Workshop on Rendering, Alexander Keller and Henrik Wann Jensen (Eds.). Wojciech Matusik, Matthew Loper, and Hanspeter Pfister. 2004. Progressively-Refined Reflectance Functions from Natural Illumination. In Eurographics Workshop on Rendering, Alexander Keller and Henrik Wann Jensen (Eds.)."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1027411.1027414"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882280"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818085"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2450734"},{"key":"e_1_2_2_29_1","volume-title":"Proceedings of the Sixteenth Eurographics conference on Rendering Techniques. Eurographics Association, 173--182","author":"Peers Pieter","year":"2005","unstructured":"Pieter Peers and Philip Dutr\u00e9 . 2005 . Inferring reflectance functions from wavelet noise . In Proceedings of the Sixteenth Eurographics conference on Rendering Techniques. Eurographics Association, 173--182 . Pieter Peers and Philip Dutr\u00e9. 2005. Inferring reflectance functions from wavelet noise. In Proceedings of the Sixteenth Eurographics conference on Rendering Techniques. Eurographics Association, 173--182."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1477926.1477929"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSAA.18.002448"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33783-3_43"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.488"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766899"},{"key":"e_1_2_2_35_1","volume-title":"U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI) (LNCS)","volume":"9351","author":"Ronneberger O.","unstructured":"O. Ronneberger , P. Fischer , and T. Brox . 2015 . U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI) (LNCS) , Vol. 9351 . Springer, 234--241. O. Ronneberger, P.Fischer, and T. Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI) (LNCS), Vol. 9351. Springer, 234--241."},{"key":"e_1_2_2_36_1","volume-title":"Integrated High-Quality Acquisition of Geometry and Appearance for Cultural Heritage","author":"Schwartz Christopher","unstructured":"Christopher Schwartz , Michael Weinmann , Roland Ruiters , and Reinhard Klein . 2011. Integrated High-Quality Acquisition of Geometry and Appearance for Cultural Heritage . In VAST. Eurographics Association , 25--32. Christopher Schwartz, Michael Weinmann, Roland Ruiters, and Reinhard Klein. 2011. Integrated High-Quality Acquisition of Geometry and Appearance for Cultural Heritage. In VAST. Eurographics Association, 25--32."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007975506780"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882281"},{"key":"e_1_2_2_39_1","volume-title":"Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows. In ECCV. 251--264.","author":"Sunkavalli Kalyan","year":"2010","unstructured":"Kalyan Sunkavalli , Todd Zickler , and Hanspeter Pfister . 2010 . Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows. In ECCV. 251--264. Kalyan Sunkavalli, Todd Zickler, and Hanspeter Pfister. 2010. Visibility Subspaces: Uncalibrated Photometric Stereo with Shadows. In ECCV. 251--264."},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/2383847.2383874"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531335"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1117\/12.7972479"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982396"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201313","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3197517.3201313","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:39:44Z","timestamp":1750210784000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201313"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,30]]},"references-count":43,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,8,31]]}},"alternative-id":["10.1145\/3197517.3201313"],"URL":"https:\/\/doi.org\/10.1145\/3197517.3201313","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,30]]},"assertion":[{"value":"2018-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}