{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,29]],"date-time":"2026-07-29T14:22:55Z","timestamp":1785334975231,"version":"3.55.0"},"reference-count":97,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["CNS-2038612, IIS-1900821"],"award-info":[{"award-number":["CNS-2038612, IIS-1900821"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Zillow Group","award":["Gift"],"award-info":[{"award-number":["Gift"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:p>We describe a novel approach to decompose a single panorama of an empty indoor environment into four appearance components: specular, direct sunlight, diffuse and diffuse ambient without direct sunlight. Our system is weakly supervised by automatically generated semantic maps (with floor, wall, ceiling, lamp, window and door labels) that have shown success on perspective views and are trained for panoramas using transfer learning without any further annotations. A GAN-based approach supervised by coarse information obtained from the semantic map extracts specular reflection and direct sunlight regions on the floor and walls. These lighting effects are removed via a similar GAN-based approach and a semantic-aware inpainting step. The appearance decomposition enables multiple applications including sun direction estimation, virtual furniture insertion, floor material replacement, and sun direction change, providing an effective tool for virtual home staging. We demonstrate the effectiveness of our approach on a large and recently released dataset of panoramas of empty homes.<\/jats:p>","DOI":"10.1145\/3528223.3530148","type":"journal-article","created":{"date-parts":[[2022,7,22]],"date-time":"2022-07-22T21:06:27Z","timestamp":1658523987000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["Semantically supervised appearance decomposition for virtual staging from a single panorama"],"prefix":"10.1145","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0953-1444","authenticated-orcid":false,"given":"Tiancheng","family":"Zhi","sequence":"first","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2225-8796","authenticated-orcid":false,"given":"Bowei","family":"Chen","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8555-2952","authenticated-orcid":false,"given":"Ivaylo","family":"Boyadzhiev","sequence":"additional","affiliation":[{"name":"Zillow Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2016-2915","authenticated-orcid":false,"given":"Sing Bing","family":"Kang","sequence":"additional","affiliation":[{"name":"Zillow Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4566-5930","authenticated-orcid":false,"given":"Martial","family":"Hebert","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0389-1921","authenticated-orcid":false,"given":"Srinivasa G.","family":"Narasimhan","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,7,22]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766967"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2015.09.001"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.190"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2014.2377712"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01477-5"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601206"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01245"},{"key":"e_1_2_2_8_1","volume-title":"Neural-pil: Neural pre-integrated lighting for reflectance decomposition. Advances in Neural Information Processing Systems 34","author":"Boss Mark","year":"2021","unstructured":"Mark Boss, Varun Jampani, Raphael Braun, Ce Liu, Jonathan Barron, and Hendrik Lensch. 2021b. Neural-pil: Neural pre-integrated lighting for reflectance decomposition. Advances in Neural Information Processing Systems 34 (2021)."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00404"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2699184"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3129537"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00217"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201378"},{"key":"e_1_2_2_14_1","volume-title":"Computer graphics forum","author":"Deschaintre Valentin","unstructured":"Valentin Deschaintre, Miika Aittala, Fr\u00e9do Durand, George Drettakis, and Adrien Bousseau. 2019. Flexible svbrdf capture with a multi-image deep network. In Computer graphics forum, Vol. 38. Wiley Online Library, 1--13."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00497"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130816"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.351"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00766"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323042"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00727"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130891"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00707"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2742999"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW50498.2020.00328"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2009.5459428"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01225-0_17"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00709"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.255"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00767"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530088"},{"key":"e_1_2_2_31_1","unstructured":"Wenzel Jakob. 2010. Mitsuba renderer. http:\/\/www.mitsuba-renderer.org."},{"key":"e_1_2_2_32_1","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems. 5938--5948","author":"Janner Michael","year":"2017","unstructured":"Michael Janner, Jiajun Wu, Tejas D Kulkarni, Ilker Yildirim, and Joshua B Tenenbaum. 2017. Self-supervised intrinsic image decomposition. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 5938--5948."},{"key":"e_1_2_2_33_1","volume-title":"Detecting specular reflections and cast shadows to estimate reflectance and illumination of dynamic indoor scenes","author":"Jiddi Salma","year":"2020","unstructured":"Salma Jiddi, Philippe Robert, and Eric Marchand. 2020. Detecting specular reflections and cast shadows to estimate reflectance and illumination of dynamic indoor scenes. IEEE Transactions on Visualization and Computer Graphics (2020)."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46475-6_43"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2070781.2024191"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2602146"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00380"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.192"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.12"},{"key":"e_1_2_2_40_1","volume-title":"Adam: A method for stochastic optimization. In ICLR.","author":"Kingma Diederik P","year":"2015","unstructured":"Diederik P Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In ICLR."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-011-0501-8"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-009-0291-4"},{"key":"e_1_2_2_43_1","doi-asserted-by":"crossref","unstructured":"John Lambert Zhuang Liu Ozan Sener James Hays and Vladlen Koltun. 2020. MSeg: A Composite Dataset for Multi-domain Semantic Segmentation. In Computer Vision and Pattern Recognition (CVPR).","DOI":"10.1109\/CVPR42600.2020.00295"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.1106"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00362"},{"key":"e_1_2_2_46_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 10591--10600","author":"Li Junxuan","year":"2021","unstructured":"Junxuan Li, Hongdong Li, and Yasuyuki Matsushita. 2021a. Lighting, Reflectance and Geometry Estimation From 360deg Panoramic Stereo. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 10591--10600."},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.346"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00255"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_23"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00942"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01219-9_5"},{"key":"e_1_2_2_52_1","unstructured":"Zhengqin Li Ting-Wei Yu Shen Sang Sarah Wang Sai Bi Zexiang Xu Hong-Xing Yu Kalyan Sunkavalli Milo\u0161 Ha\u0161an Ravi Ramamoorthi and Manmohan Chandraker. 2021b. OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets. 7190--7199 pages."},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-20205-7_1"},{"key":"e_1_2_2_54_1","doi-asserted-by":"crossref","first-page":"932","DOI":"10.1109\/TIP.2014.2378032","article-title":"Estimation of sunlight direction using 3D object models","volume":"24","author":"Liu Yang","year":"2014","unstructured":"Yang Liu, Theo Gevers, and Xueqing Li. 2014. Estimation of sunlight direction using 3D object models. IEEE Transactions on Image Processing 24, 3 (2014), 932--942.","journal-title":"IEEE Transactions on Image Processing"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00331"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298965"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58452-8_24"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58542-6_24"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00149"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58598-3_26"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00869"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00659"},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.5555\/136809.136817"},{"key":"e_1_2_2_65_1","volume-title":"Real-time highlight removal using intensity ratio. Applied optics 52, 19","author":"Shen Hui-Liang","year":"2013","unstructured":"Hui-Liang Shen and Zhi-Huan Zheng. 2013. Real-time highlight removal using intensity ratio. Applied optics 52, 19 (2013), 4483--4493."},{"key":"e_1_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.619"},{"key":"e_1_2_2_67_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3193--3201","author":"Shih YiChang","year":"2015","unstructured":"YiChang Shih, Dilip Krishnan, Fredo Durand, and William T Freeman. 2015. Reflection removal using ghosting cues. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3193--3201."},{"key":"e_1_2_2_68_1","volume-title":"Neural Face Editing with Intrinsic Image Disentangling. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Shu Z.","year":"2017","unstructured":"Z. Shu, Ersin Yumer, Sunil Hadap, Kalyan Sunkavalli, E. Shechtman, and D. Samaras. 2017. Neural Face Editing with Intrinsic Image Disentangling. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), 5444--5453."},{"key":"e_1_2_2_69_1","unstructured":"Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In ICLR."},{"key":"e_1_2_2_70_1","volume-title":"HDR Environment Map Estimation for Real-Time Augmented Reality. arXiv preprint arXiv:2011.10687","author":"Somanath Gowri","year":"2020","unstructured":"Gowri Somanath and Daniel Kurz. 2020. HDR Environment Map Estimation for Real-Time Augmented Reality. arXiv preprint arXiv:2011.10687 (2020)."},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00810"},{"key":"e_1_2_2_72_1","first-page":"529","article-title":"Learning spherical convolution for fast features from 360 imagery","volume":"30","author":"Su Yu-Chuan","year":"2017","unstructured":"Yu-Chuan Su and Kristen Grauman. 2017. Learning spherical convolution for fast features from 360 imagery. Advances in Neural Information Processing Systems 30 (2017), 529--539.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_2_73_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00967"},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00114"},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00259"},{"key":"e_1_2_2_76_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2005.36"},{"key":"e_1_2_2_77_1","volume-title":"Advances in Neural Information Processing Systems","author":"Tappen Marshall","year":"2002","unstructured":"Marshall Tappen, William Freeman, and Edward Adelson. 2003. Recovering Intrinsic Images from a Single Image. In Advances in Neural Information Processing Systems, S. Becker, S. Thrun, and K. Obermayer (Eds.), Vol. 15. MIT Press. https:\/\/proceedings.neurips.cc\/paper\/2002\/file\/fa2431bf9d65058fe34e9713e32d60e6-Paper.pdf"},{"key":"e_1_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58539-6_6"},{"key":"e_1_2_2_79_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00502"},{"key":"e_1_2_2_80_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2016.7532311"},{"key":"e_1_2_2_81_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01276"},{"key":"e_1_2_2_82_1","unstructured":"Jingdong Wang Ke Sun Tianheng Cheng Borui Jiang Chaorui Deng Yang Zhao Dong Liu Yadong Mu Mingkui Tan Xinggang Wang et al. 2020. Deep high-resolution representation learning for visual recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020)."},{"key":"e_1_2_2_83_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW54120.2021.00217"},{"key":"e_1_2_2_84_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01231"},{"key":"e_1_2_2_85_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2018.00032"},{"key":"e_1_2_2_86_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00837"},{"key":"e_1_2_2_87_1","doi-asserted-by":"crossref","unstructured":"Zhongqi Wu Chuanqing Zhuang Jian Shi Jun Xiao and Jianwei Guo. 2020. Deep Specular Highlight Removal for Single Real-world Image. In SIGGRAPH Asia 2020 Posters. 1--2.","DOI":"10.1145\/3415264.3425454"},{"key":"e_1_2_2_88_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15561-1_7"},{"key":"e_1_2_2_89_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311559"},{"key":"e_1_2_2_90_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982432"},{"key":"e_1_2_2_91_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01040"},{"key":"e_1_2_2_92_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00541"},{"key":"e_1_2_2_93_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00068"},{"key":"e_1_2_2_94_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00503"},{"key":"e_1_2_2_95_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58545-7_30"},{"key":"e_1_2_2_96_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.544"},{"key":"e_1_2_2_97_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00219"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530148","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530148","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530148","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:48Z","timestamp":1750186968000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530148"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":97,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10.1145\/3528223.3530148"],"URL":"https:\/\/doi.org\/10.1145\/3528223.3530148","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7]]},"assertion":[{"value":"2022-07-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}