{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T16:18:27Z","timestamp":1782317907627,"version":"3.54.5"},"reference-count":36,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2020,11,27]],"date-time":"2020-11-27T00:00:00Z","timestamp":1606435200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2020,12,31]]},"abstract":"<jats:p>Capturing the 3D geometry of transparent objects is a challenging task, ill-suited for general-purpose scanning and reconstruction techniques, since these cannot handle specular light transport phenomena. Existing state-of-the-art methods, designed specifically for this task, either involve a complex setup to reconstruct complete refractive ray paths, or leverage a data-driven approach based on synthetic training data. In either case, the reconstructed 3D models suffer from over-smoothing and loss of fine detail. This paper introduces a novel, high precision, 3D acquisition and reconstruction method for solid transparent objects. Using a static background with a coded pattern, we establish a mapping between the camera view rays and locations on the background. Differentiable tracing of refractive ray paths is then used to directly optimize a 3D mesh approximation of the object, while simultaneously ensuring silhouette consistency and smoothness. Extensive experiments and comparisons demonstrate the superior accuracy of our method.<\/jats:p>","DOI":"10.1145\/3414685.3417815","type":"journal-article","created":{"date-parts":[[2020,11,27]],"date-time":"2020-11-27T21:51:05Z","timestamp":1606513865000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":52,"title":["Differentiable refraction-tracing for mesh reconstruction of transparent objects"],"prefix":"10.1145","volume":"39","author":[{"given":"Jiahui","family":"Lyu","sequence":"first","affiliation":[{"name":"Shenzhen University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bojian","family":"Wu","sequence":"additional","affiliation":[{"name":"Alibaba Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dani","family":"Lischinski","sequence":"additional","affiliation":[{"name":"The Hebrew University of Jerusalem"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Cohen-Or","sequence":"additional","affiliation":[{"name":"Shenzhen University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hui","family":"Huang","sequence":"additional","affiliation":[{"name":"Shenzhen University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,11,27]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073693"},{"key":"e_1_2_2_2_1","volume-title":"Proc. Int. Conf. on Computer Vision","author":"Ben-Ezra Moshe","year":"2003","unstructured":"Moshe Ben-Ezra and Shree K. Nayar . 2003. What Does Motion Reveal About Transparency? Proc. Int. Conf. on Computer Vision ( 2003 ), 1025--1032. Moshe Ben-Ezra and Shree K. Nayar. 2003. What Does Motion Reveal About Transparency? Proc. Int. Conf. on Computer Vision (2003), 1025--1032."},{"key":"e_1_2_2_3_1","volume-title":"Principles of optics: electromagnetic theory of propagation, interference and diffraction of light","author":"Born Max","unstructured":"Max Born and Emil Wolf . 2013. Principles of optics: electromagnetic theory of propagation, interference and diffraction of light . Elsevier . Max Born and Emil Wolf. 2013. Principles of optics: electromagnetic theory of propagation, interference and diffraction of light. Elsevier."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344844"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360686"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2010.01753.x"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00411"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008191222954"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-007-0049-9"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.1177"},{"key":"e_1_2_2_11_1","article-title":"Differentiable Monte Carlo Ray Tracing through Edge Sampling","volume":"37","author":"Li Tzu-Mao","year":"2018","unstructured":"Tzu-Mao Li , Miika Aittala , Fr\u00e9do Durand , and Jaakko Lehtinen . 2018 . Differentiable Monte Carlo Ray Tracing through Edge Sampling . ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37 , 6 (2018), 222:1--222:11. Tzu-Mao Li, Miika Aittala, Fr\u00e9do Durand, and Jaakko Lehtinen. 2018. Differentiable Monte Carlo Ray Tracing through Edge Sampling. ACM Trans. Graph. (Proc. SIGGRAPH Asia) 37, 6 (2018), 222:1--222:11.","journal-title":"ACM Trans. Graph. (Proc. SIGGRAPH Asia)"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00134"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00780"},{"key":"e_1_2_2_14_1","volume-title":"Proc. Int. Conf. on Computer Vision","author":"Morris Nigel JW","year":"2007","unstructured":"Nigel JW Morris and Kiriakos N. Kutulakos . 2007. Reconstructing the surface of inhomogeneous transparent scenes by scatter-trace photography . Proc. Int. Conf. on Computer Vision ( 2007 ), 1--8. Nigel JW Morris and Kiriakos N. Kutulakos. 2007. Reconstructing the surface of inhomogeneous transparent scenes by scatter-trace photography. Proc. Int. Conf. on Computer Vision (2007), 1--8."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2011.24"},{"key":"e_1_2_2_16_1","article-title":"Mitsuba 2: a retargetable forward and inverse renderer","volume":"38","author":"Nimier-David Merlin","year":"2019","unstructured":"Merlin Nimier-David , Delio Vicini , Tizian Zeltner , and Wenzel Jakob . 2019 . Mitsuba 2: a retargetable forward and inverse renderer . ACM Trans. on Graphics (Proc. of SIGGRAPH Asia) 38 , 6 (2019), 203:1--203:17. Merlin Nimier-David, Delio Vicini, Tizian Zeltner, and Wenzel Jakob. 2019. Mitsuba 2: a retargetable forward and inverse renderer. ACM Trans. on Graphics (Proc. of SIGGRAPH Asia) 38, 6 (2019), 203:1--203:17.","journal-title":"ACM Trans. on Graphics (Proc. of SIGGRAPH Asia)"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2011.01876.x"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778803"},{"key":"e_1_2_2_19_1","volume-title":"Automatic differentiation in PyTorch. NIPS-W","author":"Paszke Adam","year":"2017","unstructured":"Adam Paszke , Sam Gross , Soumith Chintala , Gregory Chanan , Edward Yang , Zachary DeVito , Zeming Lin , Alban Desmaison , Luca Antiga , and Adam Lerer . 2017. Automatic differentiation in PyTorch. NIPS-W ( 2017 ). Adam Paszke, Sam Gross, Soumith Chintala, Gregory Chanan, Edward Yang, Zachary DeVito, Zeming Lin, Alban Desmaison, Luca Antiga, and Adam Lerer. 2017. Automatic differentiation in PyTorch. NIPS-W (2017)."},{"key":"e_1_2_2_20_1","volume-title":"Proc. Eurographics Workshop on Rendering","author":"Peers Pieter","year":"2003","unstructured":"Pieter Peers and Philip Dutr\u00e9 . 2003 . Wavelet environment matting . Proc. Eurographics Workshop on Rendering (2003), 157--166. Pieter Peers and Philip Dutr\u00e9. 2003. Wavelet environment matting. Proc. Eurographics Workshop on Rendering (2003), 157--166."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2009.96"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/964965.808606"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.403"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.704"},{"key":"e_1_2_2_25_1","volume-title":"Proc. IEEE Int. Conf. on Robotics & Automation","author":"Sajjan Shreeyak","year":"2020","unstructured":"Shreeyak Sajjan , Matthew Moore , Mike Pan , Ganesh Nagaraja , Johnny Chung Lee , Andy Zeng , and Shuran Song . 2020 . ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation . Proc. IEEE Int. Conf. on Robotics & Automation (2020). Shreeyak Sajjan, Matthew Moore, Mike Pan, Ganesh Nagaraja, Johnny Chung Lee, Andy Zeng, and Shuran Song. 2020. ClearGrasp: 3D Shape Estimation of Transparent Objects for Manipulation. Proc. IEEE Int. Conf. on Robotics & Automation (2020)."},{"key":"e_1_2_2_26_1","volume-title":"Proc. IEEE Conf. on Computer Vision & Pattern Recognition","author":"Shan Qi","year":"2012","unstructured":"Qi Shan , Sameer Agarwal , and Brian Curless . 2012 . Refractive height fields from single and multiple images . Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2012), 286--293. Qi Shan, Sameer Agarwal, and Brian Curless. 2012. Refractive height fields from single and multiple images. Proc. IEEE Conf. on Computer Vision & Pattern Recognition (2012), 286--293."},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2019.00111"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.475"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.5555\/2383894.2383902"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2015.7350870"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126367"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201286"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995472"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_16"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.888718"},{"key":"e_1_2_2_36_1","volume-title":"Salesin","author":"Zongker Douglas E.","year":"1999","unstructured":"Douglas E. Zongker , Dawn M. Werner , Brian Curless , and David H . Salesin . 1999 . Environment Matting and Compositing. ACM Trans. on Graphics (Proc. of SIGGRAPH) ( 1999), 205--214. Douglas E. Zongker, Dawn M. Werner, Brian Curless, and David H. Salesin. 1999. Environment Matting and Compositing. ACM Trans. on Graphics (Proc. of SIGGRAPH) (1999), 205--214."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3414685.3417815","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3414685.3417815","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:03:14Z","timestamp":1750197794000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3414685.3417815"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,27]]},"references-count":36,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2020,12,31]]}},"alternative-id":["10.1145\/3414685.3417815"],"URL":"https:\/\/doi.org\/10.1145\/3414685.3417815","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,27]]},"assertion":[{"value":"2020-11-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}