{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,5]],"date-time":"2026-08-05T17:58:35Z","timestamp":1785952715941,"version":"3.56.0"},"reference-count":57,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2019,7,12]],"date-time":"2019-07-12T00:00:00Z","timestamp":1562889600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100011199","name":"FP7 Ideas: European Research Council","doi-asserted-by":"publisher","award":["FUNGRAPH -- ERC Advanced Grant no 788065"],"award-info":[{"award-number":["FUNGRAPH -- ERC Advanced Grant no 788065"]}],"id":[{"id":"10.13039\/100011199","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000780","name":"European Commission","doi-asserted-by":"crossref","award":["EMOTIVE -- H2020 project No 727188"],"award-info":[{"award-number":["EMOTIVE -- H2020 project No 727188"]}],"id":[{"id":"10.13039\/501100000780","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2019,8,31]]},"abstract":"<jats:p>\n            We propose the first learning-based algorithm that can relight images in a plausible and controllable manner given multiple views of an outdoor scene. In particular, we introduce a\n            <jats:italic>geometry-aware<\/jats:italic>\n            neural network that utilizes multiple geometry cues (normal maps, specular direction, etc.) and source and target shadow masks computed from a noisy\n            <jats:italic>proxy geometry<\/jats:italic>\n            obtained by multi-view stereo. Our model is a three-stage pipeline: two subnetworks refine the source and target shadow masks, and a third performs the final relighting. Furthermore, we introduce a novel representation for the shadow masks, which we call\n            <jats:italic>RGB shadow images.<\/jats:italic>\n            They reproject the colors from all views into the shadowed pixels and enable our network to cope with inacuraccies in the proxy and the non-locality of the shadow casting interactions. Acquiring large-scale multi-view relighting datasets for real scenes is challenging, so we train our network on photorealistic synthetic data. At train time, we also compute a noisy stereo-based geometric proxy, this time from the synthetic renderings. This allows us to bridge the gap between the real and synthetic domains. Our model generalizes well to real scenes. It can alter the illumination of drone footage, image-based renderings, textured mesh reconstructions, and even internet photo collections.\n          <\/jats:p>","DOI":"10.1145\/3306346.3323013","type":"journal-article","created":{"date-parts":[[2019,7,12]],"date-time":"2019-07-12T19:04:08Z","timestamp":1562958248000},"page":"1-14","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":91,"title":["Multi-view relighting using a geometry-aware network"],"prefix":"10.1145","volume":"38","author":[{"given":"Julien","family":"Philip","sequence":"first","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur and Inria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Micha\u00ebl","family":"Gharbi","sequence":"additional","affiliation":[{"name":"Adobe"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tinghui","family":"Zhou","sequence":"additional","affiliation":[{"name":"UC Berkeley"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexei A.","family":"Efros","sequence":"additional","affiliation":[{"name":"UC Berkeley"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"George","family":"Drettakis","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur and Inria"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,7,12]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Mart\u00edn Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Ian Goodfellow Andrew Harp Geoffrey Irving Michael Isard Yangqing Jia Rafal Jozefowicz Lukasz Kaiser Manjunath Kudlur Josh Levenberg Dan Man\u00e9 Rajat Monga Sherry Moore Derek Murray Chris Olah Mike Schuster Jonathon Shlens Benoit Steiner Ilya Sutskever Kunal Talwar Paul Tucker Vincent Vanhoucke Vijay Vasudevan Fernanda Vi\u00e9gas Oriol Vinyals Pete Warden Martin Wattenberg Martin Wicke Yuan Yu and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. http:\/\/tensorflow.org\/  Mart\u00edn Abadi Ashish Agarwal Paul Barham Eugene Brevdo Zhifeng Chen Craig Citro Greg S. Corrado Andy Davis Jeffrey Dean Matthieu Devin Sanjay Ghemawat Ian Goodfellow Andrew Harp Geoffrey Irving Michael Isard Yangqing Jia Rafal Jozefowicz Lukasz Kaiser Manjunath Kudlur Josh Levenberg Dan Man\u00e9 Rajat Monga Sherry Moore Derek Murray Chris Olah Mike Schuster Jonathon Shlens Benoit Steiner Ilya Sutskever Kunal Talwar Paul Tucker Vincent Vanhoucke Vijay Vasudevan Fernanda Vi\u00e9gas Oriol Vinyals Pete Warden Martin Wattenberg Martin Wicke Yuan Yu and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. http:\/\/tensorflow.org\/"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383309"},{"key":"e_1_2_2_3_1","volume-title":"Photographic Image Synthesis with Cascaded Refinement Networks. CoRR abs\/1707.09405","author":"Chen Qifeng","year":"2017","unstructured":"Qifeng Chen and Vladlen Koltun . 2017. Photographic Image Synthesis with Cascaded Refinement Networks. CoRR abs\/1707.09405 ( 2017 ). arXiv:1707.09405 http:\/\/arxiv.org\/abs\/1707.09405 Qifeng Chen and Vladlen Koltun. 2017. Photographic Image Synthesis with Cascaded Refinement Networks. CoRR abs\/1707.09405 (2017). arXiv:1707.09405 http:\/\/arxiv.org\/abs\/1707.09405"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/38.988744"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344855"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2756549"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2006.18"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2007.4408933"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2732407"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995725"},{"key":"e_1_2_2_11_1","volume-title":"Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. CoRR","author":"He Kaiming","year":"2015","unstructured":"Kaiming He , Xiangyu Zhang , Shaoqing Ren , and Jian Sun . 2015. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. CoRR ( 2015 ). Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification. CoRR (2015)."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.255"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185591"},{"key":"e_1_2_2_15_1","doi-asserted-by":"crossref","unstructured":"Phillip Isola Jun-Yan Zhu Tinghui Zhou and Alexei A Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. In CVPR.  Phillip Isola Jun-Yan Zhu Tinghui Zhou and Alexei A Efros. 2017. Image-to-Image Translation with Conditional Adversarial Networks. In CVPR.","DOI":"10.1109\/CVPR.2017.632"},{"key":"e_1_2_2_16_1","unstructured":"Wenzel Jakob. 2010. Mitsuba renderer. http:\/\/www.mitsuba-renderer.org.  Wenzel Jakob. 2010. Mitsuba renderer. http:\/\/www.mitsuba-renderer.org."},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46475-6_43"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275104"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024191"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601209"},{"key":"e_1_2_2_21_1","volume-title":"Adam: A method for stochastic optimization. ICLR","author":"Kingma Diederik","year":"2015","unstructured":"Diederik Kingma and Jimmy Ba . 2015 . Adam: A method for stochastic optimization. ICLR (2015). Diederik Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. ICLR (2015)."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409060.1409069"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366221"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01267-0_11"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2009.5459163"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618477"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/1888028.1888053"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSA.61.000001"},{"key":"e_1_2_2_29_1","volume-title":"Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm\u00e4ssan","author":"Lehtinen Jaakko","year":"2018","unstructured":"Jaakko Lehtinen , Jacob Munkberg , Jon Hasselgren , Samuli Laine , Tero Karras , Miika Aittala , and Timo Aila . 2018 . Noise2Noise: Learning Image Restoration without Clean Data . In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm\u00e4ssan , Stockholm, Sweden, July 10--15 , 2018. 2971--2980. Jaakko Lehtinen, Jacob Munkberg, Jon Hasselgren, Samuli Laine, Tero Karras, Miika Aittala, and Timo Aila. 2018. Noise2Noise: Learning Image Restoration without Clean Data. In Proceedings of the 35th International Conference on Machine Learning, ICML 2018, Stockholmsm\u00e4ssan, Stockholm, Sweden, July 10--15, 2018. 2971--2980."},{"key":"e_1_2_2_30_1","unstructured":"Ming-Yu Liu Thomas Breuel and Jan Kautz. 2017. Unsupervised image-to-image translation networks. In Advances in Neural Information Processing Systems. 700--708.   Ming-Yu Liu Thomas Breuel and Jan Kautz. 2017. Unsupervised image-to-image translation networks. In Advances in Neural Information Processing Systems. 700--708."},{"key":"e_1_2_2_31_1","volume-title":"Rendering Techniques 99","author":"Loscos C\u00e9line","unstructured":"C\u00e9line Loscos , Marie-Claude Frasson , George Drettakis , Bruce Walter , Xavier Granier , and Pierre Poulin . 1999. Interactive virtual relighting and remodeling of real scenes . In Rendering Techniques 99 . Springer , 329--340. C\u00e9line Loscos, Marie-Claude Frasson, George Drettakis, Bruce Walter, Xavier Granier, and Pierre Poulin. 1999. Interactive virtual relighting and remodeling of real scenes. In Rendering Techniques 99. Springer, 329--340."},{"key":"e_1_2_2_32_1","volume-title":"Deep photo style transfer. CoRR, abs\/1703.07511 2","author":"Luan Fujun","year":"2017","unstructured":"Fujun Luan , Sylvain Paris , Eli Shechtman , and Kavita Bala . 2017. Deep photo style transfer. CoRR, abs\/1703.07511 2 ( 2017 ). Fujun Luan, Sylvain Paris, Eli Shechtman, and Kavita Bala. 2017. Deep photo style transfer. CoRR, abs\/1703.07511 2 (2017)."},{"key":"e_1_2_2_33_1","volume-title":"Color and Imaging Conference","volume":"1997","author":"Marschner Stephen R","year":"1997","unstructured":"Stephen R Marschner and Donald P Greenberg . 1997 . Inverse lighting for photography . In Color and Imaging Conference , Vol. 1997 . Society for Imaging Science and Technology, 262--265. Stephen R Marschner and Donald P Greenberg. 1997. Inverse lighting for photography. In Color and Imaging Conference, Vol. 1997. Society for Imaging Science and Technology, 262--265."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882315"},{"key":"e_1_2_2_35_1","volume-title":"Willems","author":"Masselus Vincent","year":"2004","unstructured":"Vincent Masselus , Pieter Peers , Philip Dutr\u00e9 , and Yves D . Willems . 2004 . Smooth reconstruction and compact representation of reflectance functions for image-based relighting. In Rendering Techniques 2004. Eurographics Association , Norrkoping, Sweden, 287--298. Vincent Masselus, Pieter Peers, Philip Dutr\u00e9, and Yves D. Willems. 2004. Smooth reconstruction and compact representation of reflectance functions for image-based relighting. In Rendering Techniques 2004. Eurographics Association, Norrkoping, Sweden, 287--298."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2007.30"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276442"},{"key":"e_1_2_2_38_1","volume-title":"DeshadowNet: A Multi-Context Embedding Deep Network for Shadow Removal. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Qu Liangqiong","unstructured":"Liangqiong Qu , Jiandong Tian , Shengfeng He , Yandong Tang , and Rynson W. H. Lau . 2017 . DeshadowNet: A Multi-Context Embedding Deep Network for Shadow Removal. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Liangqiong Qu, Jiandong Tian, Shengfeng He, Yandong Tang, and Rynson W. H. Lau. 2017. DeshadowNet: A Multi-Context Embedding Deep Network for Shadow Removal. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_39_1","unstructured":"CapturingReality RealityCapture. 2016. RealityCapture. \\protect{http:\/\/https:\/\/www.capturingreality.com\/}  CapturingReality RealityCapture. 2016. RealityCapture. \\protect{http:\/\/https:\/\/www.capturingreality.com\/}"},{"key":"e_1_2_2_40_1","volume-title":"U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-assisted Intervention (MICCAI)","author":"Ronneberger Olaf","year":"2015","unstructured":"Olaf Ronneberger , Philipp Fischer , and Thomas Brox . 2015 . U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-assisted Intervention (MICCAI) . Springer , 234--241. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In Medical Image Computing and Computer-assisted Intervention (MICCAI). Springer, 234--241."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2011.10.001"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508419"},{"key":"e_1_2_2_43_1","volume-title":"Computer Graphics Forum","author":"Shor Yael","unstructured":"Yael Shor and Dani Lischinski . 2008. The shadow meets the mask: Pyramid-based shadow removal . In Computer Graphics Forum , Vol. 27 . Wiley Online Library , 577--586. Yael Shor and Dani Lischinski. 2008. The shadow meets the mask: Pyramid-based shadow removal. In Computer Graphics Forum, Vol. 27. Wiley Online Library, 577--586."},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.578"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141964"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/1029949.1029977"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276504"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/1186223.1186323"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00192"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2008.244"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2001.937606"},{"key":"e_1_2_2_52_1","volume-title":"Huang","author":"Wen Zhen","year":"2003","unstructured":"Zhen Wen , Zicheng Liu , and Thomas S . Huang . 2003 . Face Relighting with Radiance Environment Maps. In CVPR. Zhen Wen, Zicheng Liu, and Thomas S. Huang. 2003. Face Relighting with Radiance Environment Maps. In CVPR."},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073258"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/1243980.1243982"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201313"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311559"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.244"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3306346.3323013","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3306346.3323013","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:25:52Z","timestamp":1750206352000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3306346.3323013"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,12]]},"references-count":57,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,8,31]]}},"alternative-id":["10.1145\/3306346.3323013"],"URL":"https:\/\/doi.org\/10.1145\/3306346.3323013","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,12]]},"assertion":[{"value":"2019-07-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}