{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T06:14:08Z","timestamp":1784268848890,"version":"3.55.0"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2019,7,12]],"date-time":"2019-07-12T00:00:00Z","timestamp":1562889600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["1617234, 1703957"],"award-info":[{"award-number":["1617234, 1703957"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Powell-Bundle Fellowship"},{"name":"UC San Diego Center for Visual Computing"},{"name":"Adobe"},{"name":"Ronald L. Graham Chair"},{"name":"ONR","award":["N000141712687"],"award-info":[{"award-number":["N000141712687"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2019,8,31]]},"abstract":"<jats:p>The goal of light transport acquisition is to take images from a sparse set of lighting and viewing directions, and combine them to enable arbitrary relighting with changing view. While relighting from sparse images has received significant attention, there has been relatively less progress on view synthesis from a sparse set of \"photometric\" images---images captured under controlled conditions, lit by a single directional source; we use a spherical gantry to position the camera on a sphere surrounding the object. In this paper, we synthesize novel viewpoints across a wide range of viewing directions (covering a 60\u00b0 cone) from a sparse set of just six viewing directions. While our approach relates to previous view synthesis and image-based rendering techniques, those methods are usually restricted to much smaller baselines, and are captured under environment illumination. At our baselines, input images have few correspondences and large occlusions; however we benefit from structured photometric images. Our method is based on a deep convolutional network trained to directly synthesize new views from the six input views. This network combines 3D convolutions on a plane sweep volume with a novel per-view per-depth plane attention map prediction network to effectively aggregate multi-view appearance. We train our network with a large-scale synthetic dataset of 1000 scenes with complex geometry and material properties. In practice, it is able to synthesize novel viewpoints for captured real data and reproduces complex appearance effects like occlusions, view-dependent specularities and hard shadows. Moreover, the method can also be combined with previous relighting techniques to enable changing both lighting and view, and applied to computer vision problems like multiview stereo from sparse image sets.<\/jats:p>","DOI":"10.1145\/3306346.3323007","type":"journal-article","created":{"date-parts":[[2019,7,12]],"date-time":"2019-07-12T19:04:08Z","timestamp":1562958248000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":67,"title":["Deep view synthesis from sparse photometric images"],"prefix":"10.1145","volume":"38","author":[{"given":"Zexiang","family":"Xu","sequence":"first","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sai","family":"Bi","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kalyan","family":"Sunkavalli","sequence":"additional","affiliation":[{"name":"Adobe Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sunil","family":"Hadap","sequence":"additional","affiliation":[{"name":"Amazon"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Su","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ravi","family":"Ramamoorthi","sequence":"additional","affiliation":[{"name":"University of California"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,7,12]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"illumination, and reflectance from shading","author":"Barron Jonathan T","year":"2015","unstructured":"Jonathan T Barron and Jitendra Malik . 2015. Shape , illumination, and reflectance from shading . IEEE transactions on pattern analysis and machine intelligence (TPAMI) 37, 8 ( 2015 ), 1670--1687. Jonathan T Barron and Jitendra Malik. 2015. Shape, illumination, and reflectance from shading. IEEE transactions on pattern analysis and machine intelligence (TPAMI) 37, 8 (2015), 1670--1687."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073610"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383309"},{"key":"e_1_2_2_4_1","volume-title":"Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012","author":"Chang Angel X","year":"2015","unstructured":"Angel X Chang , Thomas Funkhouser , Leonidas Guibas , Pat Hanrahan , Qixing Huang , Zimo Li , Silvio Savarese , Manolis Savva , Shuran Song , Hao Su , 2015 . Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012 (2015). Angel X Chang, Thomas Funkhouser, Leonidas Guibas, Pat Hanrahan, Qixing Huang, Zimo Li, Silvio Savarese, Manolis Savva, Shuran Song, Hao Su, et al. 2015. Shapenet: An information-rich 3d model repository. arXiv preprint arXiv:1512.03012 (2015)."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487228.2487238"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2011.01981.x"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3203192"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/166117.166153"},{"key":"e_1_2_2_9_1","volume-title":"Computer Graphics Forum","author":"D\u0105ba\u0142a Lukasz","unstructured":"Lukasz D\u0105ba\u0142a , Matthias Ziegler , Piotr Didyk , Frederik Zilly , Joachim Keinert , Karol Myszkowski , H-P Seidel , Przemyslaw Rokita , and Tobias Ritschel . 2016. Efficient Multi-image Correspondences for On-line Light Field Video Processing . In Computer Graphics Forum , Vol. 35 . Wiley Online Library , 401--410. Lukasz D\u0105ba\u0142a, Matthias Ziegler, Piotr Didyk, Frederik Zilly, Joachim Keinert, Karol Myszkowski, H-P Seidel, Przemyslaw Rokita, and Tobias Ritschel. 2016. Efficient Multi-image Correspondences for On-line Light Field Video Processing. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 401--410."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2005.37"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344855"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/237170.237191"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201378"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.304"},{"key":"e_1_2_2_15_1","volume-title":"Marcus Magnor, Philippe Bekaert, Edilson De Aguiar, Naveed Ahmed, Christian Theobalt, and Anita Sellent.","author":"Eisemann Martin","year":"2008","unstructured":"Martin Eisemann , Bert De Decker , Marcus Magnor, Philippe Bekaert, Edilson De Aguiar, Naveed Ahmed, Christian Theobalt, and Anita Sellent. 2008 . Floating textures. In Computer graphics forum, Vol. 27 . Wiley Online Library , 409--418. Martin Eisemann, Bert De Decker, Marcus Magnor, Philippe Bekaert, Edilson De Aguiar, Naveed Ahmed, Christian Theobalt, and Anita Sellent. 2008. Floating textures. In Computer graphics forum, Vol. 27. Wiley Online Library, 409--418."},{"key":"e_1_2_2_16_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5515--5524","author":"Flynn John","year":"2016","unstructured":"John Flynn , Ivan Neulander , James Philbin , and Noah Snavely . 2016 . Deepstereo: Learning to predict new views from the world's imagery . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5515--5524 . John Flynn, Ivan Neulander, James Philbin, and Noah Snavely. 2016. Deepstereo: Learning to predict new views from the world's imagery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 5515--5524."},{"key":"e_1_2_2_17_1","unstructured":"Ryo Furukawa Hiroshi Kawasaki Katsushi Ikeuchi and Masao Sakauchi. 2002. Appearance Based Object Modeling using Texture Database: Acquisition Compression and Rendering.. In Rendering Techniques. 257--266.   Ryo Furukawa Hiroshi Kawasaki Katsushi Ikeuchi and Masao Sakauchi. 2002. Appearance Based Object Modeling using Texture Database: Acquisition Compression and Rendering.. In Rendering Techniques. 257--266."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/2354409.2354978"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/237170.237200"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275084"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778836"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00298"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.573"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980251"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/237170.237199"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073641"},{"key":"e_1_2_2_27_1","unstructured":"Zhengqin Li Kalyan Sunkavalli and Manmohan Chandraker. 2018a. Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image. In ECCV.  Zhengqin Li Kalyan Sunkavalli and Manmohan Chandraker. 2018a. Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Image. In ECCV."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275055"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383320"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275017"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.82"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1477926.1477929"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130855"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.488"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46487-9_31"},{"key":"e_1_2_2_36_1","volume-title":"VAST","volume":"2011","author":"Schwartz Christopher","year":"2011","unstructured":"Christopher Schwartz , Michael Weinmann , Roland Ruiters , and Reinhard Klein . 2011 . Integrated High-Quality Acquisition of Geometry and Appearance for Cultural Heritage .. In VAST , Vol. 2011 . 25--32. Christopher Schwartz, Michael Weinmann, Roland Ruiters, and Reinhard Klein. 2011. Integrated High-Quality Acquisition of Geometry and Appearance for Cultural Heritage.. In VAST, Vol. 2011. 25--32."},{"key":"e_1_2_2_37_1","doi-asserted-by":"crossref","unstructured":"Sudipta Sinha Drew Steedly and Rick Szeliski. 2009. Piecewise planar stereo for image-based rendering. (2009).  Sudipta Sinha Drew Steedly and Rick Szeliski. 2009. Piecewise planar stereo for image-based rendering. (2009).","DOI":"10.1109\/ICCV.2009.5459417"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.246"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_10"},{"key":"e_1_2_2_40_1","volume-title":"Single-view to multi-view: Reconstructing unseen views with a convolutional network. CoRR abs\/1511.06702 1, 2","author":"Tatarchenko Maxim","year":"2015","unstructured":"Maxim Tatarchenko , Alexey Dosovitskiy , and Thomas Brox . 2015. Single-view to multi-view: Reconstructing unseen views with a convolutional network. CoRR abs\/1511.06702 1, 2 ( 2015 ), 2. Maxim Tatarchenko, Alexey Dosovitskiy, and Thomas Brox. 2015. Single-view to multi-view: Reconstructing unseen views with a convolutional network. CoRR abs\/1511.06702 1, 2 (2015), 2."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2653101"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2818143.2818165"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1561\/0600000022"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141987"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344925"},{"key":"e_1_2_2_46_1","volume-title":"Photometric method for determining surface orientation from multiple images. Optical engineering 19, 1","author":"Woodham Robert J","year":"1980","unstructured":"Robert J Woodham . 1980. Photometric method for determining surface orientation from multiple images. Optical engineering 19, 1 ( 1980 ), 191139. Robert J Woodham. 1980. Photometric method for determining surface orientation from multiple images. Optical engineering 19, 1 (1980), 191139."},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01261-8_1"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980248"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982396"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201313"},{"key":"e_1_2_2_51_1","unstructured":"Jimei Yang Scott E Reed Ming-Hsuan Yang and Honglak Lee. 2015. Weakly-supervised disentangling with recurrent transformations for 3d view synthesis. In Advances in Neural Information Processing Systems. 1099--1107.   Jimei Yang Scott E Reed Ming-Hsuan Yang and Honglak Lee. 2015. Weakly-supervised disentangling with recurrent transformations for 3d view synthesis. In Advances in Neural Information Processing Systems. 1099--1107."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13640-016-0129-2"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01237-3_47"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601134"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201323"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_18"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980247"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.195"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3306346.3323007","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3306346.3323007","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3306346.3323007","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:25:52Z","timestamp":1750206352000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3306346.3323007"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7,12]]},"references-count":58,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2019,8,31]]}},"alternative-id":["10.1145\/3306346.3323007"],"URL":"https:\/\/doi.org\/10.1145\/3306346.3323007","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,7,12]]},"assertion":[{"value":"2019-07-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}