{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,21]],"date-time":"2026-03-21T02:09:03Z","timestamp":1774058943643,"version":"3.50.1"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T00:00:00Z","timestamp":1573171200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2019,12,31]]},"abstract":"<jats:p>We propose a novel framework that automatically learns the lighting patterns for efficient, joint acquisition of unknown reflectance and shape. The core of our framework is a deep neural network, with a shared linear encoder that directly corresponds to the lighting patterns used in physical acquisition, as well as non-linear decoders that output per-pixel normal and diffuse \/ specular information from photographs. We exploit the diffuse and normal information from multiple views to reconstruct a detailed 3D shape, and then fit BRDF parameters to the diffuse \/ specular information, producing texture maps as reflectance results. We demonstrate the effectiveness of the framework with physical objects that vary considerably in reflectance and shape, acquired with as few as 16 ~ 32 lighting patterns that correspond to 7 ~ 15 seconds of per-view acquisition time. Our framework is useful for optimizing the efficiency in both novel and existing setups, as it can automatically adapt to various factors, including the geometry \/ the lighting layout of the device and the properties of appearance.<\/jats:p>","DOI":"10.1145\/3355089.3356492","type":"journal-article","created":{"date-parts":[[2019,11,8]],"date-time":"2019-11-08T20:27:58Z","timestamp":1573244878000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":52,"title":["Learning efficient illumination multiplexing for joint capture of reflectance and shape"],"prefix":"10.1145","volume":"38","author":[{"given":"Kaizhang","family":"Kang","sequence":"first","affiliation":[{"name":"Zhejiang University, China"}]},{"given":"Cihui","family":"Xie","sequence":"additional","affiliation":[{"name":"Zhejiang University, China"}]},{"given":"Chengan","family":"He","sequence":"additional","affiliation":[{"name":"Yale University"}]},{"given":"Mingqi","family":"Yi","sequence":"additional","affiliation":[{"name":"Zhejiang University, China"}]},{"given":"Minyi","family":"Gu","sequence":"additional","affiliation":[{"name":"Zhejiang University, China"}]},{"given":"Zimin","family":"Chen","sequence":"additional","affiliation":[{"name":"Zhejiang University, China"}]},{"given":"Kun","family":"Zhou","sequence":"additional","affiliation":[{"name":"Zhejiang University and ZJU-FaceUnity Joint Lab of Intelligent Graphics, China"}]},{"given":"Hongzhi","family":"Wu","sequence":"additional","affiliation":[{"name":"Zhejiang University, China"}]}],"member":"320","published-online":{"date-parts":[[2019,11,8]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461978"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766967"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073610"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601180"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/300776.300778"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201378"},{"key":"e_1_2_2_7_1","volume-title":"Deep appearance modeling: A survey. Visual Informatics","author":"Dong Yue","year":"2019","unstructured":"Yue Dong. 2019. Deep appearance modeling: A survey. Visual Informatics (2019)."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778835"},{"key":"e_1_2_2_9_1","unstructured":"Mark Fiala. 2005. ARTag a fiducial marker system using digital techniques. In CVPR."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882342"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2009.01493.x"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12867"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778836"},{"key":"e_1_2_2_14_1","doi-asserted-by":"crossref","unstructured":"Satoshi Ikehata. 2018. CNN-PS: CNN-based photometric stereo for general non-convex surfaces. In ECCV.","DOI":"10.1007\/978-3-030-01267-0_1"},{"key":"e_1_2_2_15_1","volume-title":"The Rendering Equation (SIGGRAPH '86)","author":"Kajiya James T.","year":"1986","unstructured":"James T. Kajiya. 1986. The Rendering Equation (SIGGRAPH '86). 143--150."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201279"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487228.2487237"},{"key":"e_1_2_2_18_1","doi-asserted-by":"crossref","unstructured":"Alex Kendall Hayk Martirosyan Saumitro Dasgupta Peter Henry Ryan Kennedy Abraham Bachrach and Adam Bry. 2017. End-To-End Learning of Geometry and Context for Deep Stereo Regression. In ICCV.","DOI":"10.1109\/ICCV.2017.17"},{"key":"e_1_2_2_19_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR."},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1531326.1531360"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1141949"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/636886.636891"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073641"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275055"},{"key":"e_1_2_2_25_1","volume-title":"Practical Physically-based Shading in Film and Game Production. In ACM SIGGRAPH 2012 Courses. Article 10","author":"McAuley Stephen","year":"2012","unstructured":"Stephen McAuley, Stephen Hill, Naty Hoffman, Yoshiharu Gotanda, Brian Smits, Brent Burley, and Adam Martinez. 2012. Practical Physically-based Shading in Film and Game Production. In ACM SIGGRAPH 2012 Courses. Article 10, 7 pages."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2049662.2049669"},{"key":"e_1_2_2_27_1","volume-title":"Diego Gutierrez, and Min H Kim.","author":"Nam Giljoo","year":"2018","unstructured":"Giljoo Nam, Joo Ho Lee, Diego Gutierrez, and Min H Kim. 2018. Practical SVBRDF acquisition of 3D objects with unstructured flash photography. In SIGGRAPH Asia Technical Papers. 267."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073226"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818085"},{"key":"e_1_2_2_30_1","unstructured":"Daniel Scharstein and Richard Szeliski. 2003. High-accuracy stereo depth maps using structured light. In CVPR."},{"key":"e_1_2_2_31_1","doi-asserted-by":"crossref","unstructured":"Johannes Lutz Sch\u00f6nberger Enliang Zheng Marc Pollefeys and Jan-Michael Frahm. 2016. Pixelwise View Selection for Unstructured Multi-View Stereo. In ECCV.","DOI":"10.1007\/978-3-319-46487-9_31"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461944"},{"key":"e_1_2_2_33_1","volume-title":"Torrance","author":"Walter Bruce","year":"2007","unstructured":"Bruce Walter, Stephen R. Marschner, Hongsong Li, and Kenneth E. Torrance. 2007. Microfacet Models for Refraction through Rough Surfaces. In Rendering Techniques (Proc. EGWR)."},{"key":"e_1_2_2_34_1","doi-asserted-by":"crossref","unstructured":"Michael Weinmann and Reinhard Klein. 2015. Advances in Geometry and Reflectance Acquisition. In SIGGRAPH Asia Courses. Article 1 71 pages.","DOI":"10.1145\/2818143.2818165"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1561\/0600000022"},{"key":"e_1_2_2_36_1","volume-title":"Photometric method for determining surface orientation from multiple images. Optical engineering 19, 1","author":"Woodham Robert J","year":"1980","unstructured":"Robert J Woodham. 1980. Photometric method for determining surface orientation from multiple images. Optical engineering 19, 1 (1980), 191139."},{"key":"e_1_2_2_37_1","first-page":"8","article-title":"Simultaneous Localization and Appearance Estimation with a Consumer RGB-D Camera","volume":"22","author":"Wu Hongzhi","year":"2016","unstructured":"Hongzhi Wu, Zhaotian Wang, and Kun Zhou. 2016. Simultaneous Localization and Appearance Estimation with a Consumer RGB-D Camera. IEEE TVCG 22, 8 (Aug 2016), 2012--2023.","journal-title":"IEEE TVCG"},{"key":"e_1_2_2_38_1","unstructured":"Shihao Wu Hui Huang Tiziano Portenier Matan Sela Daniel Cohen-Or Ron Kimmel and Matthias Zwicker. 2018. Specular-to-Diffuse Translation for Multi-View Reconstruction. In ECCV."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179"},{"key":"e_1_2_2_40_1","doi-asserted-by":"crossref","unstructured":"Yao Yao Zixin Luo Shiwei Li Tian Fang and Long Quan. 2018. MVSNet: Depth Inference for Unstructured Multi-view Stereo. In ECCV.","DOI":"10.1007\/978-3-030-01237-3_47"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1057432.1057439"},{"key":"e_1_2_2_42_1","unstructured":"Zhenglong Zhou Zhe Wu and Ping Tan. 2013. Multi-view photometric stereo with spatially varying isotropic materials. In CVPR."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3355089.3356492","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3355089.3356492","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:44:40Z","timestamp":1750203880000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3355089.3356492"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,8]]},"references-count":42,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2019,12,31]]}},"alternative-id":["10.1145\/3355089.3356492"],"URL":"https:\/\/doi.org\/10.1145\/3355089.3356492","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,8]]},"assertion":[{"value":"2019-11-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}