{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T06:05:50Z","timestamp":1784268350692,"version":"3.55.0"},"reference-count":119,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T00:00:00Z","timestamp":1690329600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:p>We present a neural rendering-based method called NeRO for reconstructing the geometry and the BRDF of reflective objects from multiview images captured in an unknown environment. Multiview reconstruction of reflective objects is extremely challenging because specular reflections are view-dependent and thus violate the multiview consistency, which is the cornerstone for most multiview reconstruction methods. Recent neural rendering techniques can model the interaction between environment lights and the object surfaces to fit the view-dependent reflections, thus making it possible to reconstruct reflective objects from multiview images. However, accurately modeling environment lights in the neural rendering is intractable, especially when the geometry is unknown. Most existing neural rendering methods, which can model environment lights, only consider direct lights and rely on object masks to reconstruct objects with weak specular reflections. Therefore, these methods fail to reconstruct reflective objects, especially when the object mask is not available and the object is illuminated by indirect lights. We propose a two-step approach to tackle this problem. First, by applying the split-sum approximation and the integrated directional encoding to approximate the shading effects of both direct and indirect lights, we are able to accurately reconstruct the geometry of reflective objects without any object masks. Then, with the object geometry fixed, we use more accurate sampling to recover the environment lights and the BRDF of the object. Extensive experiments demonstrate that our method is capable of accurately reconstructing the geometry and the BRDF of reflective objects from only posed RGB images without knowing the environment lights and the object masks. Codes and datasets are available at https:\/\/github.com\/liuyuan-pal\/NeRO.<\/jats:p>","DOI":"10.1145\/3592134","type":"journal-article","created":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T15:47:45Z","timestamp":1690386465000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":135,"title":["NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Images"],"prefix":"10.1145","volume":"42","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2933-5667","authenticated-orcid":false,"given":"Yuan","family":"Liu","sequence":"first","affiliation":[{"name":"University of Hong Kong, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-8245-1860","authenticated-orcid":false,"given":"Peng","family":"Wang","sequence":"additional","affiliation":[{"name":"University of Hong Kong, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3335-6623","authenticated-orcid":false,"given":"Cheng","family":"Lin","sequence":"additional","affiliation":[{"name":"Tencent Games, Shenzhen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3386-8805","authenticated-orcid":false,"given":"Xiaoxiao","family":"Long","sequence":"additional","affiliation":[{"name":"University of Hong Kong, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6049-4458","authenticated-orcid":false,"given":"Jiepeng","family":"Wang","sequence":"additional","affiliation":[{"name":"University of Hong Kong, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4301-1474","authenticated-orcid":false,"given":"Lingjie","family":"Liu","sequence":"additional","affiliation":[{"name":"Max Planck Institute for Informatics, Saarbruecken, Germany"},{"name":"University of Pennsylvania, Philadelphia, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2729-5860","authenticated-orcid":false,"given":"Taku","family":"Komura","sequence":"additional","affiliation":[{"name":"University of Hong Kong, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2284-3952","authenticated-orcid":false,"given":"Wenping","family":"Wang","sequence":"additional","affiliation":[{"name":"Texas A&amp;M University, College Station, United States of America"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,7,26]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"SAL: Sign agnostic learning of shapes from raw data. In CVPR.","author":"Atzmon Matan","year":"2020","unstructured":"Matan Atzmon and Yaron Lipman. 2020. SAL: Sign agnostic learning of shapes from raw data. In CVPR."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2014.2377712"},{"key":"e_1_2_2_3_1","doi-asserted-by":"crossref","unstructured":"Jonathan T Barron Ben Mildenhall Dor Verbin Pratul P Srinivasan and Peter Hedman. 2022. Mip-NeRF 360: Unbounded anti-aliased neural radiance fields. In CVPR.","DOI":"10.1109\/CVPR52688.2022.00539"},{"key":"e_1_2_2_4_1","doi-asserted-by":"crossref","unstructured":"Jonathan T Barron and Ben Poole. 2016. The fast bilateral solver. In ECCV.","DOI":"10.1007\/978-3-319-46487-9_38"},{"key":"e_1_2_2_5_1","volume-title":"Neural reflectance fields for appearance acquisition. arXiv preprint arXiv:2008.03824","author":"Bi Sai","year":"2020","unstructured":"Sai Bi, Zexiang Xu, Pratul Srinivasan, Ben Mildenhall, Kalyan Sunkavalli, Milo\u0161 Ha\u0161an, Yannick Hold-Geoffroy, David Kriegman, and Ravi Ramamoorthi. 2020. Neural reflectance fields for appearance acquisition. arXiv preprint arXiv:2008.03824 (2020)."},{"key":"e_1_2_2_6_1","volume-title":"d.]. Deep reflectance volumes: Relightable reconstructions from multi-view photometric images","author":"Bi Sai","unstructured":"Sai Bi, Zexiang Xu, Kalyan Sunkavalli, Milo\u0161 Ha\u0161an, Yannick Hold-Geoffroy, David Kriegman, and Ravi Ramamoorthi. [n. d.]. Deep reflectance volumes: Relightable reconstructions from multi-view photometric images. In ECCV. Springer."},{"key":"e_1_2_2_7_1","doi-asserted-by":"crossref","unstructured":"Michael Bleyer Christoph Rhemann and Carsten Rother. 2011. Patchmatch stereo-stereo matching with slanted support windows. In BMVC.","DOI":"10.5244\/C.25.14"},{"key":"e_1_2_2_8_1","doi-asserted-by":"crossref","unstructured":"Mark Boss Raphael Braun Varun Jampani Jonathan T Barron Ce Liu and Hendrik Lensch. 2021a. NerD: Neural reflectance decomposition from image collections. In CVPR.","DOI":"10.1109\/ICCV48922.2021.01245"},{"key":"e_1_2_2_9_1","volume-title":"SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections. In NeurIPS.","author":"Boss Mark","year":"2022","unstructured":"Mark Boss, Andreas Engelhardt, Abhishek Kar, Yuanzhen Li, Deqing Sun, Jonathan T Barron, Hendrik Lensch, and Varun Jampani. 2022. SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections. In NeurIPS."},{"key":"e_1_2_2_10_1","unstructured":"Mark Boss Varun Jampani Raphael Braun Ce Liu Jonathan Barron and Hendrik Lensch. 2021b. Neural-PIL: Neural pre-integrated lighting for reflectance decomposition. In NeurIPS."},{"key":"e_1_2_2_11_1","doi-asserted-by":"crossref","unstructured":"Neill DF Campbell George Vogiatzis Carlos Hern\u00e1ndez and Roberto Cipolla. 2008. Using multiple hypotheses to improve depth-maps for multi-view stereo. In ECCV.","DOI":"10.1007\/978-3-540-88682-2_58"},{"key":"e_1_2_2_12_1","unstructured":"Pierre Charbonnier Laure Blanc-Feraud Gilles Aubert and Michel Barlaud. 1994. Two deterministic half-quadratic regularization algorithms for computed imaging. In ICIP."},{"key":"e_1_2_2_13_1","volume-title":"Learning to predict 3d objects with an interpolation-based differentiable renderer. NeurIPS 32","author":"Chen Wenzheng","year":"2019","unstructured":"Wenzheng Chen, Huan Ling, Jun Gao, Edward Smith, Jaakko Lehtinen, Alec Jacobson, and Sanja Fidler. 2019. Learning to predict 3d objects with an interpolation-based differentiable renderer. NeurIPS 32 (2019)."},{"key":"e_1_2_2_14_1","volume-title":"Sameh Khamis, Or Litany, and Sanja Fidler.","author":"Chen Wenzheng","year":"2021","unstructured":"Wenzheng Chen, Joey Litalien, Jun Gao, Zian Wang, Clement Fuji Tsang, Sameh Khamis, Or Litany, and Sanja Fidler. 2021. DIB-R++: Learning to predict lighting and material with a hybrid differentiable renderer. NeurIPS (2021)."},{"key":"e_1_2_2_15_1","doi-asserted-by":"crossref","unstructured":"Zhaoxi Chen and Ziwei Liu. 2022. Relighting4d: Neural relightable human from videos. In ECCV.","DOI":"10.1007\/978-3-031-19781-9_35"},{"key":"e_1_2_2_16_1","volume-title":"Ravi Ramamoorthi, and Hao Su.","author":"Cheng Shuo","year":"2020","unstructured":"Shuo Cheng, Zexiang Xu, Shilin Zhu, Zhuwen Li, Li Erran Li, Ravi Ramamoorthi, and Hao Su. 2020. Deep stereo using adaptive thin volume representation with uncertainty awareness. In CVPR."},{"key":"e_1_2_2_17_1","doi-asserted-by":"crossref","unstructured":"Ziang Cheng Hongdong Li Yuta Asano Yinqiang Zheng and Imari Sato. 2021. Multiview 3D Reconstruction of a Texture-less Smooth Surface of Unknown Generic Reflectance. In CVPR.","DOI":"10.1109\/CVPR46437.2021.01596"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/357290.357293"},{"key":"e_1_2_2_19_1","doi-asserted-by":"crossref","unstructured":"Fran\u00e7ois Darmon B\u00e9n\u00e9dicte Bascle Jean-Cl\u00e9ment Devaux Pascal Monasse and Mathieu Aubry. 2022. Improving neural implicit surfaces geometry with patch warping. In CVPR.","DOI":"10.1109\/CVPR52688.2022.00616"},{"key":"e_1_2_2_20_1","volume-title":"PANDORA: Polarization-Aided Neural Decomposition Of Radiance. In ECCV.","author":"Dave Akshat","year":"2022","unstructured":"Akshat Dave, Yongyi Zhao, and Ashok Veeraraghavan. 2022. PANDORA: Polarization-Aided Neural Decomposition Of Radiance. In ECCV."},{"key":"e_1_2_2_21_1","volume-title":"DIP: Differentiable Interreflection-aware Physics-based Inverse Rendering. arXiv preprint arXiv:2212.04705","author":"Deng Youming","year":"2022","unstructured":"Youming Deng, Xueting Li, Sifei Liu, and Ming-Hsuan Yang. 2022. DIP: Differentiable Interreflection-aware Physics-based Inverse Rendering. arXiv preprint arXiv:2212.04705 (2022)."},{"key":"e_1_2_2_22_1","volume-title":"Multi-view intrinsic images of outdoors scenes with an application to relighting. ACM Transactions on Graphics (ToG)","author":"Duch\u00eane Sylvain","year":"2015","unstructured":"Sylvain Duch\u00eane, Clement Riant, Gaurav Chaurasia, Jorge Lopez-Moreno, Pierre-Yves Laffont, Stefan Popov, Adrien Bousseau, and George Drettakis. 2015. Multi-view intrinsic images of outdoors scenes with an application to relighting. ACM Transactions on Graphics (ToG) (2015)."},{"key":"e_1_2_2_23_1","unstructured":"Qiancheng Fu Qingshan Xu Yew-Soon Ong and Wenbing Tao. 2022. Geo-Neus: Geometry-Consistent Neural Implicit Surfaces Learning for Multi-view Reconstruction. In NeurIPS."},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.161"},{"key":"e_1_2_2_25_1","doi-asserted-by":"crossref","unstructured":"David Gallup Jan-Michael Frahm Philippos Mordohai Qingxiong Yang and Marc Pollefeys. 2007. Real-time plane-sweeping stereo with multiple sweeping directions. In CVPR.","DOI":"10.1109\/CVPR.2007.383245"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417767"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323042"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913491297"},{"key":"e_1_2_2_29_1","doi-asserted-by":"crossref","unstructured":"Clement Godard Peter Hedman Wenbin Li and Gabriel J Brostow. 2015. Multi-view reconstruction of highly specular surfaces in uncontrolled environments. In 3DV.","DOI":"10.1109\/3DV.2015.10"},{"key":"e_1_2_2_30_1","unstructured":"Amos Gropp Lior Yariv Niv Haim Matan Atzmon and Yaron Lipman. 2020. Implicit Geometric Regularization for Learning Shapes. In ICML."},{"key":"e_1_2_2_31_1","doi-asserted-by":"crossref","unstructured":"Haoyu Guo Sida Peng Haotong Lin Qianqian Wang Guofeng Zhang Hujun Bao and Xiaowei Zhou. 2022b. Neural 3D Scene Reconstruction with the Manhattan-world Assumption. In CVPR.","DOI":"10.1109\/CVPR52688.2022.00543"},{"key":"e_1_2_2_32_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3414685.3417779","article-title":"MaterialGAN: reflectance capture using a generative SVBRDF model","volume":"39","author":"Guo Yu","year":"2020","unstructured":"Yu Guo, Cameron Smith, Milo\u0161 Ha\u0161an, Kalyan Sunkavalli, and Shuang Zhao. 2020. MaterialGAN: reflectance capture using a generative SVBRDF model. ToG 39, 6 (2020), 1--13.","journal-title":"ToG"},{"key":"e_1_2_2_33_1","unstructured":"Yuan-Chen Guo Di Kang Linchao Bao Yu He and Song-Hai Zhang. 2022a. NeRFRen: Neural radiance fields with reflections. In CVPR."},{"key":"e_1_2_2_34_1","doi-asserted-by":"crossref","unstructured":"Kai Han Kwan-Yee K Wong Dirk Schnieders and Miaomiao Liu. 2016. Mirror surface reconstruction under an uncalibrated camera. In CVPR.","DOI":"10.1109\/CVPR.2016.196"},{"key":"e_1_2_2_35_1","volume-title":"Multiple view geometry in computer vision","author":"Hartley Richard","unstructured":"Richard Hartley and Andrew Zisserman. 2003. Multiple view geometry in computer vision. Cambridge university press."},{"key":"e_1_2_2_36_1","unstructured":"Jon Hasselgren Nikolai Hofmann and Jacob Munkberg. 2022. Shape Light & Material Decomposition from Images using Monte Carlo Rendering and Denoising. In NeurIPS."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.156"},{"key":"e_1_2_2_38_1","doi-asserted-by":"crossref","unstructured":"Eldar Insafutdinov Dylan Campbell Jo\u00e3o F Henriques and Andrea Vedaldi. 2022. SNeS: Learning Probably Symmetric Neural Surfaces from Incomplete Data. In ECCV. 367--383.","DOI":"10.1007\/978-3-031-19824-3_22"},{"key":"e_1_2_2_39_1","doi-asserted-by":"crossref","unstructured":"Rasmus Jensen Anders Dahl George Vogiatzis Engin Tola and Henrik Aan\u00e6s. 2014. Large scale multi-view stereopsis evaluation. In CVPR.","DOI":"10.1109\/CVPR.2014.59"},{"key":"e_1_2_2_40_1","doi-asserted-by":"crossref","unstructured":"Achuta Kadambi Vage Taamazyan Boxin Shi and Ramesh Raskar. 2015. Polarized 3D: High-quality depth sensing with polarization cues. In ICCV.","DOI":"10.1109\/ICCV.2015.385"},{"key":"e_1_2_2_41_1","doi-asserted-by":"crossref","unstructured":"James T Kajiya. 1986. The rendering equation. In SIGGRAPH.","DOI":"10.1145\/15922.15902"},{"key":"e_1_2_2_42_1","first-page":"1","article-title":"Real shading in unreal engine 4","volume":"4","author":"Karis Brian","year":"2013","unstructured":"Brian Karis and Epic Games. 2013. Real shading in unreal engine 4. Proc. Physically Based Shading Theory Practice 4, 3 (2013), 1.","journal-title":"Proc. Physically Based Shading Theory Practice"},{"key":"e_1_2_2_43_1","doi-asserted-by":"crossref","unstructured":"Hiroharu Kato Yoshitaka Ushiku and Tatsuya Harada. 2018. Neural 3d mesh renderer. In CVPR.","DOI":"10.1109\/CVPR.2018.00411"},{"key":"e_1_2_2_44_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3550454.3555497"},{"key":"e_1_2_2_46_1","doi-asserted-by":"crossref","unstructured":"Zhengfei Kuang Kyle Olszewski Menglei Chai Zeng Huang Panos Achlioptas and Sergey Tulyakov. 2022. NeROIC: Neural Rendering of Objects from Online Image Collections. In SIGGRAPH.","DOI":"10.1145\/3528223.3530177"},{"key":"e_1_2_2_47_1","volume-title":"Vox-Surf: Voxel-based implicit surface representation","author":"Li Hai","year":"2022","unstructured":"Hai Li, Xingrui Yang, Hongjia Zhai, Yuqian Liu, Hujun Bao, and Guofeng Zhang. 2022. Vox-Surf: Voxel-based implicit surface representation. IEEE Transactions on Visualization and Computer Graphics (2022)."},{"key":"e_1_2_2_48_1","unstructured":"Junxuan Li and Hongdong Li. 2022a. Neural Reflectance for Shape Recovery with Shadow Handling. In CVPR."},{"key":"e_1_2_2_49_1","unstructured":"Junxuan Li and Hongdong Li. 2022b. Self-calibrating photometric stereo by neural inverse rendering. In ECCV."},{"key":"e_1_2_2_50_1","unstructured":"Zhengqin Li Mohammad Shafiei Ravi Ramamoorthi Kalyan Sunkavalli and Manmohan Chandraker. 2020. Inverse rendering for complex indoor scenes: Shape spatially-varying lighting and svbrdf from a single image. In CVPR."},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275055"},{"key":"e_1_2_2_52_1","unstructured":"Shichen Liu Tianye Li Weikai Chen and Hao Li. 2019. Soft rasterizer: A differentiable renderer for image-based 3d reasoning. In CVPR."},{"key":"e_1_2_2_53_1","doi-asserted-by":"crossref","unstructured":"Yang Liu Alexandros Neophytou Sunando Sengupta and Eric Sommerlade. 2021. Relighting images in the wild with a self-supervised siamese auto-encoder. In CVPR.","DOI":"10.1109\/WACV48630.2021.00008"},{"key":"e_1_2_2_54_1","volume-title":"Sparseneus: Fast generalizable neural surface reconstruction from sparse views. In ECCV.","author":"Long Xiaoxiao","year":"2022","unstructured":"Xiaoxiao Long, Cheng Lin, Peng Wang, Taku Komura, and Wenping Wang. 2022. Sparseneus: Fast generalizable neural surface reconstruction from sparse views. In ECCV."},{"key":"e_1_2_2_55_1","unstructured":"Linjie Lyu Ayush Tewari Thomas Leimk\u00fchler Marc Habermann and Christian Theobalt. 2022. Neural Radiance Transfer Fields for Relightable Novel-view Synthesis with Global Illumination. In ECCV."},{"key":"e_1_2_2_56_1","doi-asserted-by":"crossref","unstructured":"B Mildenhall PP Srinivasan M Tancik JT Barron R Ramamoorthi and R Ng. 2020. NeRF: Representing scenes as neural radiance fields for view synthesis. In ECCV.","DOI":"10.1007\/978-3-030-58452-8_24"},{"key":"e_1_2_2_57_1","doi-asserted-by":"crossref","unstructured":"Jacob Munkberg Jon Hasselgren Tianchang Shen Jun Gao Wenzheng Chen Alex Evans Thomas M\u00fcller and Sanja Fidler. 2022. Extracting Triangular 3D Models Materials and Lighting From Images. In CVPR.","DOI":"10.1109\/CVPR52688.2022.00810"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275017"},{"key":"e_1_2_2_59_1","doi-asserted-by":"crossref","unstructured":"Thomas Nestmeyer Jean-Fran\u00e7ois Lalonde Iain Matthews and Andreas Lehrmann. 2020. Learning physics-guided face relighting under directional light. In CVPR.","DOI":"10.1109\/CVPR42600.2020.00517"},{"key":"e_1_2_2_60_1","volume-title":"Directional reflectance and emissivity of an opaque surface. Applied optics 4, 7","author":"Nicodemus Fred E","year":"1965","unstructured":"Fred E Nicodemus. 1965. Directional reflectance and emissivity of an opaque surface. Applied optics 4, 7 (1965), 767--775."},{"key":"e_1_2_2_61_1","doi-asserted-by":"crossref","unstructured":"Michael Niemeyer Lars Mescheder Michael Oechsle and Andreas Geiger. 2020. Differentiable volumetric rendering: Learning implicit 3d representations without 3d supervision. In CVPR.","DOI":"10.1109\/CVPR42600.2020.00356"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356498"},{"key":"e_1_2_2_63_1","volume-title":"Unisurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In ICCV.","author":"Oechsle Michael","year":"2021","unstructured":"Michael Oechsle, Songyou Peng, and Andreas Geiger. 2021. Unisurf: Unifying neural implicit surfaces and radiance fields for multi-view reconstruction. In ICCV."},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00025"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323013"},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3469842"},{"key":"e_1_2_2_67_1","unstructured":"Poly Heaven. 2022. Poly Heaven. https:\/\/polyhaven.com\/."},{"key":"e_1_2_2_68_1","unstructured":"Stefan Rahmann and Nikos Canterakis. 2001. Reconstruction of specular surfaces using polarization imaging. In CVPR."},{"key":"e_1_2_2_69_1","doi-asserted-by":"crossref","unstructured":"Christian Richardt Douglas Orr Ian Davies Antonio Criminisi and Neil A Dodgson. 2010. Real-time spatiotemporal stereo matching using the dual-cross-bilateral grid. In ECCV.","DOI":"10.1007\/978-3-642-15558-1_37"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3384535"},{"key":"e_1_2_2_71_1","unstructured":"Stefan Roth and Michael J Black. 2006. Specular flow and the recovery of surface structure. In CVPR."},{"key":"e_1_2_2_72_1","doi-asserted-by":"crossref","unstructured":"Viktor Rudnev Mohamed Elgharib William Smith Lingjie Liu Vladislav Golyanik and Christian Theobalt. 2022. NeRF for outdoor scene relighting. In ECCV.","DOI":"10.1007\/978-3-031-19787-1_35"},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1014573219977"},{"key":"e_1_2_2_74_1","doi-asserted-by":"crossref","unstructured":"Carolin Schmitt Simon Donne Gernot Riegler Vladlen Koltun and Andreas Geiger. 2020. On joint estimation of pose geometry and svBRDF from a handheld scanner. In CVPR.","DOI":"10.1109\/CVPR42600.2020.00355"},{"key":"e_1_2_2_75_1","doi-asserted-by":"crossref","unstructured":"Johannes Lutz Sch\u00f6nberger and Jan-Michael Frahm. 2016. Structure-from-Motion Revisited. In CVPR.","DOI":"10.1109\/CVPR.2016.445"},{"key":"e_1_2_2_76_1","doi-asserted-by":"crossref","unstructured":"Johannes Lutz Sch\u00f6nberger Enliang Zheng Marc Pollefeys and Jan-Michael Frahm. 2016. Pixelwise View Selection for Unstructured Multi-View Stereo. In ECCV).","DOI":"10.1007\/978-3-319-46487-9_31"},{"key":"e_1_2_2_77_1","unstructured":"Tianchang Shen Jun Gao Kangxue Yin Ming-Yu Liu and Sanja Fidler. 2021. Deep Marching Tetrahedra: a hybrid representation for high-resolution 3d shape synthesis. In NeurIPS."},{"key":"e_1_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508419"},{"key":"e_1_2_2_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185596"},{"key":"e_1_2_2_80_1","volume-title":"Scene representation networks: Continuous 3d-structure-aware neural scene representations. Advances in Neural Information Processing Systems 32","author":"Sitzmann Vincent","year":"2019","unstructured":"Vincent Sitzmann, Michael Zollh\u00f6fer, and Gordon Wetzstein. 2019. Scene representation networks: Continuous 3d-structure-aware neural scene representations. Advances in Neural Information Processing Systems 32 (2019)."},{"key":"e_1_2_2_81_1","unstructured":"Christoph Strecha Rik Fransens and Luc Van Gool. 2006. Combined depth and outlier estimation in multi-view stereo. In CVPR."},{"key":"e_1_2_2_82_1","unstructured":"Jiaming Sun Xi Chen Qianqian Wang Zhengqi Li Hadar Averbuch-Elor Xiaowei Zhou and Noah Snavely. 2022. Neural 3D reconstruction in the wild. In SIGGRAPH."},{"key":"e_1_2_2_83_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14022"},{"key":"e_1_2_2_84_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14507"},{"key":"e_1_2_2_85_1","volume-title":"3d reconstruction of mirror-type objects using efficient ray coding","author":"Tin Siu-Kei","unstructured":"Siu-Kei Tin, Jinwei Ye, Mahdi Nezamabadi, and Can Chen. 2016. 3d reconstruction of mirror-type objects using efficient ray coding. In ICCP. IEEE, 1--11."},{"key":"e_1_2_2_86_1","volume-title":"ORCa: Glossy Objects as Radiance Field Cameras. arXiv preprint arXiv:2212.04531","author":"Tiwary Kushagra","year":"2022","unstructured":"Kushagra Tiwary, Askhat Dave, Nikhil Behari, Tzofi Klinghoffer, Ashok Veeraraghavan, and Ramesh Raskar. 2022. ORCa: Glossy Objects as Radiance Field Cameras. arXiv preprint arXiv:2212.04531 (2022)."},{"key":"e_1_2_2_87_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSA.57.001105"},{"key":"e_1_2_2_88_1","doi-asserted-by":"crossref","unstructured":"Dor Verbin Peter Hedman Ben Mildenhall Todd Zickler Jonathan T Barron and Pratul P Srinivasan. 2022. Ref-NeRF: Structured view-dependent appearance for neural radiance fields. In CVPR.","DOI":"10.1109\/CVPR52688.2022.00541"},{"key":"e_1_2_2_89_1","doi-asserted-by":"crossref","unstructured":"Fangjinhua Wang Silvano Galliani Christoph Vogel Pablo Speciale and Marc Pollefeys. 2021a. PatchMatchNet: Learned multi-view patchmatch stereo. In CVPR.","DOI":"10.1109\/CVPR46437.2021.01397"},{"key":"e_1_2_2_90_1","volume-title":"Neuris: Neural reconstruction of indoor scenes using normal priors. In ECCV.","author":"Wang Jiepeng","year":"2022","unstructured":"Jiepeng Wang, Peng Wang, Xiaoxiao Long, Christian Theobalt, Taku Komura, Lingjie Liu, and Wenping Wang. 2022c. Neuris: Neural reconstruction of indoor scenes using normal priors. In ECCV."},{"key":"e_1_2_2_91_1","unstructured":"Peng Wang Lingjie Liu Yuan Liu Christian Theobalt Taku Komura and Wenping Wang. 2021b. NeuS: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. In NeurIPS."},{"key":"e_1_2_2_92_1","volume-title":"NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction. arXiv preprint arXiv:2212.05231","author":"Wang Yiming","year":"2022","unstructured":"Yiming Wang, Qin Han, Marc Habermann, Kostas Daniilidis, Christian Theobalt, and Lingjie Liu. 2022a. NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstruction. arXiv preprint arXiv:2212.05231 (2022)."},{"key":"e_1_2_2_93_1","unstructured":"Yiqun Wang Ivan Skorokhodov and Peter Wonka. 2022b. HF-NeuS: Improved Surface Reconstruction Using High-Frequency Details. In NeurIPS."},{"key":"e_1_2_2_94_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2003.819861"},{"key":"e_1_2_2_95_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201319"},{"key":"e_1_2_2_96_1","doi-asserted-by":"crossref","unstructured":"Felix Wimbauer Shangzhe Wu and Christian Rupprecht. 2022. De-rendering 3D Objects in the Wild. In CVPR.","DOI":"10.1109\/CVPR52688.2022.01794"},{"key":"e_1_2_2_97_1","unstructured":"Shihao Wu Hui Huang Tiziano Portenier Matan Sela Daniel Cohen-Or Ron Kimmel and Matthias Zwicker. 2018. Specular-to-diffuse translation for multi-view reconstruction. In ECCV."},{"key":"e_1_2_2_98_1","volume-title":"Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction. arXiv preprint arXiv:2208.12697","author":"Wu Tong","year":"2022","unstructured":"Tong Wu, Jiaqi Wang, Xingang Pan, Xudong Xu, Christian Theobalt, Ziwei Liu, and Dahua Lin. 2022. Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction. arXiv preprint arXiv:2208.12697 (2022)."},{"key":"e_1_2_2_99_1","unstructured":"Jianfeng Yan Zizhuang Wei Hongwei Yi Mingyu Ding Runze Zhang Yisong Chen Guoping Wang and Yu-Wing Tai. 2020. Dense hybrid recurrent multi-view stereo net with dynamic consistency checking. In ECCV."},{"key":"e_1_2_2_100_1","doi-asserted-by":"crossref","unstructured":"Jiayu Yang Wei Mao Jose M Alvarez and Miaomiao Liu. 2020. Cost volume pyramid based depth inference for multi-view stereo. In CVPR.","DOI":"10.1109\/CVPR42600.2020.00493"},{"key":"e_1_2_2_101_1","volume-title":"Ps-nerf: Neural inverse rendering for multi-view photometric stereo. In ECCV.","author":"Yang Wenqi","year":"2022","unstructured":"Wenqi Yang, Guanying Chen, Chaofeng Chen, Zhenfang Chen, and Kwan-Yee K Wong. 2022a. Ps-nerf: Neural inverse rendering for multi-view photometric stereo. In ECCV."},{"key":"e_1_2_2_102_1","unstructured":"Wenqi Yang Guanying Chen Chaofeng Chen Zhenfang Chen and Kwan-Yee K Wong. 2022b. S3-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpoint. In NeurIPS."},{"key":"e_1_2_2_103_1","doi-asserted-by":"crossref","unstructured":"Yao Yao Zixin Luo Shiwei Li Tian Fang and Long Quan. 2018. MVSNet: Depth inference for unstructured multi-view stereo. In ECCV.","DOI":"10.1007\/978-3-030-01237-3_47"},{"key":"e_1_2_2_104_1","doi-asserted-by":"crossref","unstructured":"Yao Yao Jingyang Zhang Jingbo Liu Yihang Qu Tian Fang David McKinnon Yanghai Tsin and Long Quan. 2022. NeILF: Neural incident light field for physically-based material estimation. In ECCV.","DOI":"10.1007\/978-3-031-19821-2_40"},{"key":"e_1_2_2_105_1","unstructured":"Lior Yariv Jiatao Gu Yoni Kasten and Yaron Lipman. 2021. Volume rendering of neural implicit surfaces. In NeurIPS."},{"key":"e_1_2_2_106_1","unstructured":"Lior Yariv Yoni Kasten Dror Moran Meirav Galun Matan Atzmon Basri Ronen and Yaron Lipman. 2020. Multiview neural surface reconstruction by disentangling geometry and appearance. In NeurIPS."},{"key":"e_1_2_2_107_1","volume-title":"Intrinsicnerf: Learning intrinsic neural radiance fields for editable novel view synthesis. arXiv preprint arXiv:2210.00647","author":"Ye Weicai","year":"2022","unstructured":"Weicai Ye, Shuo Chen, Chong Bao, Hujun Bao, Marc Pollefeys, Zhaopeng Cui, and Guofeng Zhang. 2022. Intrinsicnerf: Learning intrinsic neural radiance fields for editable novel view synthesis. arXiv preprint arXiv:2210.00647 (2022)."},{"key":"e_1_2_2_108_1","doi-asserted-by":"crossref","unstructured":"Ye Yu Abhimitra Meka Mohamed Elgharib Hans-Peter Seidel Christian Theobalt and William AP Smith. 2020. Self-supervised outdoor scene relighting. In ECCV.","DOI":"10.1007\/978-3-030-58542-6_6"},{"key":"e_1_2_2_109_1","volume-title":"Inverserendernet: Learning single image inverse rendering. In CVPR.","author":"Yu Ye","year":"2019","unstructured":"Ye Yu and William AP Smith. 2019. Inverserendernet: Learning single image inverse rendering. In CVPR."},{"key":"e_1_2_2_110_1","unstructured":"Jason Zhang Gengshan Yang Shubham Tulsiani and Deva Ramanan. 2021c. NeRS: Neural reflectance surfaces for sparse-view 3d reconstruction in the wild. In NeurIPS."},{"key":"e_1_2_2_111_1","volume-title":"IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images. In CVPR.","author":"Zhang Kai","year":"2022","unstructured":"Kai Zhang, Fujun Luan, Zhengqi Li, and Noah Snavely. 2022a. IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images. In CVPR."},{"key":"e_1_2_2_112_1","doi-asserted-by":"crossref","unstructured":"Kai Zhang Fujun Luan Qianqian Wang Kavita Bala and Noah Snavely. 2021a. PhySG: Inverse rendering with spherical gaussians for physics-based material editing and relighting. In CVPR.","DOI":"10.1109\/CVPR46437.2021.00541"},{"key":"e_1_2_2_113_1","volume-title":"Analyzing and improving neural radiance fields. arXiv preprint arXiv:2010.07492","author":"Zhang Kai","year":"2020","unstructured":"Kai Zhang, Gernot Riegler, Noah Snavely, and Vladlen Koltun. 2020. Nerf++: Analyzing and improving neural radiance fields. arXiv preprint arXiv:2010.07492 (2020)."},{"key":"e_1_2_2_114_1","doi-asserted-by":"crossref","unstructured":"Richard Zhang Phillip Isola Alexei A Efros Eli Shechtman and Oliver Wang. 2018. The unreasonable effectiveness of deep features as a perceptual metric. In CVPR.","DOI":"10.1109\/CVPR.2018.00068"},{"key":"e_1_2_2_115_1","doi-asserted-by":"crossref","unstructured":"Xiuming Zhang Pratul P Srinivasan Boyang Deng Paul Debevec William T Freeman and Jonathan T Barron. 2021b. NeRFactor: Neural factorization of shape and reflectance under an unknown illumination. In SIGGRAPH.","DOI":"10.1145\/3478513.3480496"},{"key":"e_1_2_2_116_1","doi-asserted-by":"crossref","unstructured":"Yuanqing Zhang Jiaming Sun Xingyi He Huan Fu Rongfei Jia and Xiaowei Zhou. 2022b. Modeling Indirect Illumination for Inverse Rendering. In CVPR.","DOI":"10.1109\/CVPR52688.2022.01809"},{"key":"e_1_2_2_117_1","doi-asserted-by":"crossref","unstructured":"Boming Zhao Bangbang Yang Zhenyang Li Zuoyue Li Guofeng Zhang Jiashu Zhao Dawei Yin Zhaopeng Cui and Hujun Bao. 2022b. Factorized and controllable neural re-rendering of outdoor scene for photo extrapolation. In ACM MM.","DOI":"10.1145\/3503161.3548125"},{"key":"e_1_2_2_118_1","doi-asserted-by":"crossref","unstructured":"Fuqiang Zhao Yuheng Jiang Kaixin Yao Jiakai Zhang Liao Wang Haizhao Dai Yuhui Zhong Yingliang Zhang Minye Wu Lan Xu et al. 2022a. Human performance modeling and rendering via neural animated mesh. In SIGGRAPH Asia.","DOI":"10.1145\/3550454.3555451"},{"key":"e_1_2_2_119_1","volume-title":"Neural Relightable Participating Media Rendering. NeurIPS","author":"Zheng Quan","year":"2021","unstructured":"Quan Zheng, Gurprit Singh, and Hans-Peter Seidel. 2021. Neural Relightable Participating Media Rendering. NeurIPS (2021)."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3592134","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3592134","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:37:46Z","timestamp":1750178266000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3592134"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,26]]},"references-count":119,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["10.1145\/3592134"],"URL":"https:\/\/doi.org\/10.1145\/3592134","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,26]]},"assertion":[{"value":"2023-07-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}