{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T13:46:09Z","timestamp":1781271969082,"version":"3.54.1"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2018,7,30]],"date-time":"2018-07-30T00:00:00Z","timestamp":1532908800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,8,31]]},"abstract":"<jats:p>Planar reflective surfaces such as glass and mirrors are notoriously hard to reconstruct for most current 3D scanning techniques. When treated na\u00efvely, they introduce duplicate scene structures, effectively destroying the reconstruction altogether. Our key insight is that an easy to identify structure attached to the scanner---in our case an AprilTag---can yield reliable information about the existence and the geometry of glass and mirror surfaces in a scene. We introduce a fully automatic pipeline that allows us to reconstruct the geometry and extent of planar glass and mirror surfaces while being able to distinguish between the two. Furthermore, our system can automatically segment observations of multiple reflective surfaces in a scene based on their estimated planes and locations. In the proposed setup, minimal additional hardware is needed to create high-quality results. We demonstrate this using reconstructions of several scenes with a variety of real mirrors and glass.<\/jats:p>","DOI":"10.1145\/3197517.3201319","type":"journal-article","created":{"date-parts":[[2018,7,31]],"date-time":"2018-07-31T15:56:23Z","timestamp":1533052583000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":51,"title":["Reconstructing scenes with mirror and glass surfaces"],"prefix":"10.1145","volume":"37","author":[{"given":"Thomas","family":"Whelan","sequence":"first","affiliation":[{"name":"Facebook Reality Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Goesele","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs and TU Darmstadt"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steven J.","family":"Lovegrove","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julian","family":"Straub","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Simon","family":"Green","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard","family":"Szeliski","sequence":"additional","affiliation":[{"name":"Facebook"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steven","family":"Butterfield","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shobhit","family":"Verma","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard","family":"Newcombe","sequence":"additional","affiliation":[{"name":"Facebook Reality Labs"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,7,30]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"Sameer Agarwal Keir Mierle and Others. 2018. Ceres Solver http:\/\/ceres-solver.org. (2018). Sameer Agarwal Keir Mierle and Others. 2018. Ceres Solver http:\/\/ceres-solver.org. (2018)."},{"key":"e_1_2_2_2_1","volume-title":"Single Image Reflection Suppression. In CVPR","author":"Arvanitopoulos N.","year":"2017","unstructured":"N. Arvanitopoulos , R. Achanta , and S. S\u00fcsstrunk . 2017 . Single Image Reflection Suppression. In CVPR 2017 . 1752--1760. N. Arvanitopoulos, R. Achanta, and S. S\u00fcsstrunk. 2017. Single Image Reflection Suppression. In CVPR 2017. 1752--1760."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2014.85"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995346"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/146370.146374"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2017.00081"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.182"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.261"},{"key":"e_1_2_2_9_1","volume-title":"Detecting Specular Surfaces on Natural Images. In CVPR","author":"DelPozo A.","year":"2007","unstructured":"A. DelPozo and S. Savarese . 2007 . Detecting Specular Surfaces on Natural Images. In CVPR 2007 . A. DelPozo and S. Savarese. 2007. Detecting Specular Surfaces on Natural Images. In CVPR 2007."},{"key":"e_1_2_2_10_1","volume-title":"CVPR","author":"Ding Yuanyuan","year":"2008","unstructured":"Yuanyuan Ding and Jingyi Yu . 2008 . Recovering shape characteristics on near-flat specular surfaces . In CVPR 2008. Yuanyuan Ding and Jingyi Yu. 2008. Recovering shape characteristics on near-flat specular surfaces. In CVPR 2008."},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2658577"},{"key":"e_1_2_2_12_1","doi-asserted-by":"crossref","unstructured":"A. Fasano M. Callieri P. Cignoni and R. Scopigno. 2003. Exploiting mirrors for laser stripe 3D scanning. In 3DIM 2003. 243--250. A. Fasano M. Callieri P. Cignoni and R. Scopigno. 2003. Exploiting mirrors for laser stripe 3D scanning. In 3DIM 2003. 243--250.","DOI":"10.1109\/IM.2003.1240256"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2013.6630875"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2015.10"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2010.01753.x"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.94"},{"key":"e_1_2_2_17_1","volume-title":"Glass Confidence Maps Building Based on Neural Networks Using Laser Range-Finders for Mobile Robots. In IEEE\/SICE International Symposium on System Integration.","author":"Jiang Jun","year":"2017","unstructured":"Jun Jiang , Renato Miyagusuku , Atsushi Yamashita , and Hajime Asama . 2017 . Glass Confidence Maps Building Based on Neural Networks Using Laser Range-Finders for Mobile Robots. In IEEE\/SICE International Symposium on System Integration. Jun Jiang, Renato Miyagusuku, Atsushi Yamashita, and Hajime Asama. 2017. Glass Confidence Maps Building Based on Neural Networks Using Laser Range-Finders for Mobile Robots. In IEEE\/SICE International Symposium on System Integration."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2015.2459891"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2006.153"},{"key":"e_1_2_2_20_1","first-page":"109","article-title":"Mirror identification and correction of 3D point clouds. The International Archives of Photogrammetry","volume":"40","author":"K\u00e4shammer P.-F.","year":"2015","unstructured":"P.-F. K\u00e4shammer and A. N\u00fcchter . 2015 . Mirror identification and correction of 3D point clouds. The International Archives of Photogrammetry , Remote Sensing and Spatial Information Sciences 40 , 5 (2015), 109 . P.-F. K\u00e4shammer and A. N\u00fcchter. 2015. Mirror identification and correction of 3D point clouds. The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences 40, 5 (2015), 109.","journal-title":"Remote Sensing and Spatial Information Sciences"},{"key":"e_1_2_2_21_1","volume-title":"ICRA","author":"Klank U","year":"2011","unstructured":"U Klank , D. Carton , and M. Beetz . 2011. Transparent object detection and reconstruction on a mobile platform . In ICRA 2011 . 5971--5978. U Klank, D. Carton, and M. Beetz. 2011. Transparent object detection and reconstruction on a mobile platform. In ICRA 2011. 5971--5978."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2016.10.014"},{"key":"e_1_2_2_23_1","volume-title":"ICAR","author":"Koch R.","year":"2017","unstructured":"R. Koch , S. May , and A. N\u00fcchter . 2017a. Effective distinction of transparent and specular reflective objects in point clouds of a multi-echo laser scanner . In ICAR 2017 . 566--571. R. Koch, S. May, and A. N\u00fcchter. 2017a. Effective distinction of transparent and specular reflective objects in point clouds of a multi-echo laser scanner. In ICAR 2017. 566--571."},{"key":"e_1_2_2_24_1","volume-title":"Jordan","author":"Kulis Brian","year":"2011","unstructured":"Brian Kulis and Michael I . Jordan . 2011 . Revisiting k-means: New algorithms via Bayesian nonparametrics. arXiv preprint arXiv.1111.0352 (2011). Brian Kulis and Michael I. Jordan. 2011. Revisiting k-means: New algorithms via Bayesian nonparametrics. arXiv preprint arXiv.1111.0352 (2011)."},{"key":"e_1_2_2_25_1","volume-title":"Fast Normalized Cross-Correlation. In Vision Interface '95","author":"Lewis J. P.","year":"1995","unstructured":"J. P. Lewis . 1995 . Fast Normalized Cross-Correlation. In Vision Interface '95 . Canadian Image Processing and Pattern Recognition Society. J. P. Lewis. 1995. Fast Normalized Cross-Correlation. In Vision Interface '95. Canadian Image Processing and Pattern Recognition Society."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2014.2353622"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/37402.37422"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2004.1261080"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2015.2463671"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508374"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5979561"},{"key":"e_1_2_2_33_1","volume-title":"Camera Pose Estimation Using Images of Planar Mirror Reflections. In ECCV","author":"Rodrigues Rui","year":"2010","unstructured":"Rui Rodrigues , Jo\u00e3o P. Barreto , and Urbano Nunes . 2010 . Camera Pose Estimation Using Images of Planar Mirror Reflections. In ECCV 2010. 382--395. Rui Rodrigues, Jo\u00e3o P. Barreto, and Urbano Nunes. 2010. Camera Pose Estimation Using Images of Planar Mirror Reflections. In ECCV 2010. 382--395."},{"key":"e_1_2_2_34_1","volume-title":"CVPR","author":"Shih YiChang","year":"2015","unstructured":"YiChang Shih , D. Krishnan , F. Durand , and W. T. Freeman . 2015. Reflection removal using ghosting cues . In CVPR 2015 . 3193--3201. YiChang Shih, D. Krishnan, F. Durand, and W. T. Freeman. 2015. Reflection removal using ghosting cues. In CVPR 2015. 3193--3201."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185596"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298630"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.gmod.2004.11.002"},{"key":"e_1_2_2_39_1","volume-title":"TVSeg - Interactive Total Variation Based Image Segmentation. In BMVC","author":"Unger Markus","year":"2008","unstructured":"Markus Unger , Thomas Pock , Werner Trobin , Daniel Cremers , and Horst Bischof . 2008 b. TVSeg - Interactive Total Variation Based Image Segmentation. In BMVC 2008. Markus Unger, Thomas Pock, Werner Trobin, Daniel Cremers, and Horst Bischof. 2008b. TVSeg - Interactive Total Variation Based Image Segmentation. In BMVC 2008."},{"key":"e_1_2_2_40_1","volume-title":"IROS","author":"Wang J.","year":"2016","unstructured":"J. Wang and E. Olson . 2016. AprilTag 2: Efficient and robust fiducial detection . In IROS 2016 . 4193--4198. J. Wang and E. Olson. 2016. AprilTag 2: Efficient and robust fiducial detection. In IROS 2016. 4193--4198."},{"key":"e_1_2_2_41_1","volume-title":"Sing Bing Kang, and Jingyi Yu","author":"Wang Qiaosong","year":"2015","unstructured":"Qiaosong Wang , Haiting Lin , Yi Ma , Sing Bing Kang, and Jingyi Yu . 2015 . Automatic Layer Separation using Light Field Imaging. CoRR abs\/1506.04721 (2015). arXiv:1506.04721 http:\/\/arxiv.org\/abs\/1506.04721 Qiaosong Wang, Haiting Lin, Yi Ma, Sing Bing Kang, and Jingyi Yu. 2015. Automatic Layer Separation using Light Field Imaging. CoRR abs\/1506.04721 (2015). arXiv:1506.04721 http:\/\/arxiv.org\/abs\/1506.04721"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40602-7_1"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766940"},{"key":"e_1_2_2_44_1","volume-title":"ICRA","author":"Yang Shao-Wen","year":"2008","unstructured":"Shao-Wen Yang and Chieh-Chih Wang . 2008 . Dealing with laser scanner failure: Mirrors and windows . In ICRA 2008. 3009--3015. Shao-Wen Yang and Chieh-Chih Wang. 2008. Dealing with laser scanner failure: Mirrors and windows. In ICRA 2008. 3009--3015."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/TMECH.2010.2040113"},{"key":"e_1_2_2_46_1","unstructured":"Y. Zhang M. Ye D. Manocha and R. Yang. 2017. 3D Reconstruction in the Presence of Glass and Mirrors by Acoustic and Visual Fusion. PAMI (2017). Y. Zhang M. Ye D. Manocha and R. Yang. 2017. 3D Reconstruction in the Presence of Glass and Mirrors by Acoustic and Visual Fusion. PAMI (2017)."},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.888718"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201319","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3197517.3201319","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,6]],"date-time":"2025-07-06T04:06:59Z","timestamp":1751774819000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3197517.3201319"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,7,30]]},"references-count":46,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2018,8,31]]}},"alternative-id":["10.1145\/3197517.3201319"],"URL":"https:\/\/doi.org\/10.1145\/3197517.3201319","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,7,30]]},"assertion":[{"value":"2018-07-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}