{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,1]],"date-time":"2026-08-01T17:18:58Z","timestamp":1785604738137,"version":"3.56.0"},"reference-count":56,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2022,7,1]],"date-time":"2022-07-01T00:00:00Z","timestamp":1656633600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"IITP of Korea","award":["2017-0-00072"],"award-info":[{"award-number":["2017-0-00072"]}]},{"name":"MSRA"},{"DOI":"10.13039\/100004358","name":"Samsung Electronics","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100004358","id-type":"DOI","asserted-by":"crossref"}]},{"name":"MSIT\/IITP of Korea","award":["RS-2022-00155620"],"award-info":[{"award-number":["RS-2022-00155620"]}]},{"name":"Samsung Research Funding Center","award":["SRFC-IT2001-04"],"award-info":[{"award-number":["SRFC-IT2001-04"]}]},{"name":"NIRCH of Korea","award":["2021A02P02-001"],"award-info":[{"award-number":["2021A02P02-001"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,7]]},"abstract":"<jats:p>\n            Omnidirectional videos capture environmental scenes effectively, but they have rarely been used for geometry reconstruction. In this work, we propose an egocentric 3D reconstruction method that can acquire scene geometry with high accuracy from a short egocentric omnidirectional video. To this end, we first estimate per-frame depth using a spherical disparity network. We then fuse per-frame depth estimates into a novel\n            <jats:italic>spherical binoctree<\/jats:italic>\n            data structure that is specifically designed to tolerate spherical depth estimation errors. By subdividing the spherical space into binary tree and octree nodes that represent spherical frustums adaptively, the spherical binoctree effectively enables egocentric surface geometry reconstruction for environmental scenes while simultaneously assigning high-resolution nodes for closely observed surfaces. This allows to reconstruct an entire scene from a short video captured with a small camera trajectory. Experimental results validate the effectiveness and accuracy of our approach for reconstructing the 3D geometry of environmental scenes from short egocentric omnidirectional video inputs. We further demonstrate various applications using a conventional omnidirectional camera, including novel-view synthesis, object insertion, and relighting of scenes using reconstructed 3D models with texture.\n          <\/jats:p>","DOI":"10.1145\/3528223.3530074","type":"journal-article","created":{"date-parts":[[2022,7,22]],"date-time":"2022-07-22T21:06:27Z","timestamp":1658523987000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":23,"title":["Egocentric scene reconstruction from an omnidirectional video"],"prefix":"10.1145","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7327-3681","authenticated-orcid":false,"given":"Hyeonjoong","family":"Jang","sequence":"first","affiliation":[{"name":"KAIST, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9899-6365","authenticated-orcid":false,"given":"Andr\u00e9as","family":"Meuleman","sequence":"additional","affiliation":[{"name":"KAIST, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2632-0048","authenticated-orcid":false,"given":"Dahyun","family":"Kang","sequence":"additional","affiliation":[{"name":"KAIST, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6670-6263","authenticated-orcid":false,"given":"Donggun","family":"Kim","sequence":"additional","affiliation":[{"name":"KAIST, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6716-9845","authenticated-orcid":false,"given":"Christian","family":"Richardt","sequence":"additional","affiliation":[{"name":"University of Bath, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5078-4005","authenticated-orcid":false,"given":"Min H.","family":"Kim","sequence":"additional","affiliation":[{"name":"KAIST, South Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,7,22]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417770"},{"key":"e_1_2_2_2_1","volume-title":"Blender - a 3D modelling and rendering package","author":"Community Blender Online","unstructured":"Blender Online Community. 2022. Blender - a 3D modelling and rendering package. Blender Foundation. https:\/\/www.blender.org\/"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","unstructured":"Brian Curless and Marc Levoy. 1996. A volumetric method for building complex models from range images. In SIGGRAPH. 303--312. 10.1145\/237170.237269","DOI":"10.1145\/237170.237269"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2019.00018"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130828"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201384"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982420"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2016.05.305"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.1166"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","unstructured":"Sunghoon Im Hyowon Ha Fran\u00e7ois Rameau Hae-Gon Jeon Gyeongmin Choe and In So Kweon. 2016. All-around Depth from Small Motion with A Spherical Panoramic Camera. In ECCV. 10.1007\/978-3-319-46487-9_10","DOI":"10.1007\/978-3-319-46487-9_10"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","unstructured":"Shahram Izadi David Kim Otmar Hilliges David Molyneaux Richard Newcombe Push-meet Kohli Jamie Shotton Steve Hodges Dustin Freeman Andrew Davison and Andrew Fitzgibbon. 2011. KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In UIST. 559--568. 10.1145\/2047196.2047270","DOI":"10.1145\/2047196.2047270"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3058957"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","unstructured":"Lei Jin Yanyu Xu Jia Zheng Junfei Zhang Rui Tang Shugong Xu Jingyi Yu and Shenghua Gao. 2020. Geometric Structure Based and Regularized Depth Estimation From 360 Indoor Imagery. In CVPR. 886--895. 10.1109\/CVPR42600.2020.00097","DOI":"10.1109\/CVPR42600.2020.00097"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1007971901577"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487228.2487237"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-013-0616-1"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","unstructured":"Ren Komatsu Hiromitsu Fujii Yusuke Tamura Atsushi Yamashita and Hajime Asama. 2020. 360\u00b0 Depth Estimation from Multiple Fisheye Images with Origami Crown Representation of Icosahedron. In IROS. 10.1109\/IROS45743.2020.9340981","DOI":"10.1109\/IROS45743.2020.9340981"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","unstructured":"Tilman K\u00fchner and Julius K\u00fcmmerle. 2020. Large-Scale Volumetric Scene Reconstruction using LiDAR. In ICRA. 6261--6267. 10.1109\/ICRA40945.2020.9197388","DOI":"10.1109\/ICRA40945.2020.9197388"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/VR.2019.8798016"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00135"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2008.2006736"},{"key":"e_1_2_2_22_1","doi-asserted-by":"crossref","unstructured":"Vadim Litvinov and Maxime Lhuillier. 2013. Incremental Solid Modeling from Sparse and Omnidirectional Structure-from-Motion Data. In BMVC.","DOI":"10.5244\/C.27.61"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/37401.37422"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392377"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566590"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073645"},{"key":"e_1_2_2_27_1","unstructured":"Morgan McGuire. 2017. Computer Graphics Archive. https:\/\/casual-effects.com\/data"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01126"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-56414-2_5"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508374"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275031"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356555"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","unstructured":"Giovanni Pintore Marco Agus Eva Almansa Jens Schneider and Enrico Gobbetti. 2021. SliceNet: Deep Dense Depth Estimation From a Single Indoor Panorama Using a Slice-Based Representation. In CVPR. 11531--11540. 10.1109\/CVPR46437.2021.01137","DOI":"10.1109\/CVPR46437.2021.01137"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:VISI.0000025798.50602.3a"},{"key":"e_1_2_2_35_1","volume-title":"Signal-Specialized Parametrization. In Eurographics Workshop on Rendering. 87--98","author":"Sander Pedro V.","year":"2002","unstructured":"Pedro V. Sander, Steven J. Gortler, John Snyder, and Hugues Hoppe. 2002. Signal-Specialized Parametrization. In Eurographics Workshop on Rendering. 87--98."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2005.00843.x"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.445"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","unstructured":"Johannes L. Sch\u00f6nberger Enliang Zheng Jan-Michael Frahm and Marc Pollefeys. 2016. Pixelwise View Selection for Unstructured Multi-View Stereo. In ECCV. 501--518. 10.1007\/978-3-319-46487-9_31","DOI":"10.1007\/978-3-319-46487-9_31"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2898757"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3343031.3350539"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","unstructured":"Cheng Sun Min Sun and Hwann-Tzong Chen. 2021. HoHoNet: 360 Indoor Holistic Understanding with Latent Horizontal Features. In CVPR. 2573--2582. 10.1109\/CVPR46437.2021.00260","DOI":"10.1109\/CVPR46437.2021.00260"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58536-5_24"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","unstructured":"Fu-En Wang Hou-Ning Hu Hsien-Tzu Cheng Juan-Ting Lin Shang-Ta Yang Meng-Li Shih Hung-Kuo Chu and Min Sun. 2018. Self-Supervised Learning of Depth and Camera Motion from 360\u00b0 Videos. In ACCV. 10.1007\/978-3-030-20873-8_4","DOI":"10.1007\/978-3-030-20873-8_4"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","unstructured":"Fu-En Wang Yu-Hsuan Yeh Min Sun Wei-Chen Chiu and Yi-Hsuan Tsai. 2020b. BiFuse: Monocular 360 Depth Estimation via Bi-Projection Fusion. In CVPR. 462--471. 10.1109\/CVPR42600.2020.00054","DOI":"10.1109\/CVPR42600.2020.00054"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","unstructured":"Ning-Hsu Wang Bolivar Solarte Yi-Hsuan Tsai Wei-Chen Chiu and Min Sun. 2020a. 360SD-Net: 360\u00b0 Stereo Depth Estimation with Learnable Cost Volume. In ICRA. 582--588. 10.1109\/ICRA40945.2020.9196975","DOI":"10.1109\/ICRA40945.2020.9196975"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","unstructured":"Katja Wolff Changil Kim Henning Zimmer Christopher Schroers Mario Botsch Olga Sorkine-Hornung and Alexander Sorkine-Hornung. 2016. Point Cloud Noise and Outlier Removal for Image-Based 3D Reconstruction. In 3DV. 118--127. 10.1109\/3DV.2016.20","DOI":"10.1109\/3DV.2016.20"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","unstructured":"Changhee Won Jongbin Ryu and Jongwoo Lim. 2019a. OmniMVS: End-to-End Learning for Omnidirectional Stereo Matching. In ICCV. 8986--8995. 10.1109\/ICCV.2019.00908","DOI":"10.1109\/ICCV.2019.00908"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","unstructured":"Changhee Won Jongbin Ryu and Jongwoo Lim. 2019b. SweepNet: Wide-baseline Omnidirectional Depth Estimation. In ICRA. 10.1109\/ICRA.2019.8793823","DOI":"10.1109\/ICRA.2019.8793823"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","unstructured":"Changhee Won Hochang Seok Zhaopeng Cui Marc Pollefeys and Jongwoo Lim. 2020. OmniSLAM: Omnidirectional Localization and Dense Mapping for Wide-baseline Multi-camera Systems. In ICRA. 559--566. 10.1109\/ICRA40945.2020.9196695","DOI":"10.1109\/ICRA40945.2020.9196695"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.gmod.2012.09.002"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","unstructured":"Wei Zeng Sezer Karaoglu and Theo Gevers. 2020. Joint 3D Layout and Depth Prediction from a Single Indoor Panorama Image. In ECCV. 10.1007\/978-3-030-58517-4_39","DOI":"10.1007\/978-3-030-58517-4_39"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","unstructured":"Jianing Zhang Tianyi Zhu Anke Zhang Xiaoyun Yuan Zihan Wang Sebastian Beetschen Lan Xu Xing Lin Qionghai Dai and Lu Fang. 2020. Multiscale-VR: Multiscale Gigapixel 3D Panoramic Videography for Virtual Reality. In ICCP. 10.1109\/ICCP48838.2020.9105244","DOI":"10.1109\/ICCP48838.2020.9105244"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1057432.1057439"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601134"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","unstructured":"Nikolaos Zioulis Antonis Karakottas Dimitrios Zarpalas Federico Alvarez and Petros Daras. 2019. Spherical View Synthesis for Self-Supervised 360\u00b0 Depth Estimation. In 3DV. 690--699. 10.1109\/3DV.2019.00081","DOI":"10.1109\/3DV.2019.00081"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","unstructured":"Nikolaos Zioulis Antonis Karakottas Dimitrios Zarpalas and Petros Daras. 2018. OmniDepth: Dense Depth Estimation for Indoors Spherical Panoramas. In ECCV. 448--465. 10.1007\/978-3-030-01231-1_28","DOI":"10.1007\/978-3-030-01231-1_28"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530074","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3528223.3530074","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T19:02:25Z","timestamp":1750186945000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3528223.3530074"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7]]},"references-count":56,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,7]]}},"alternative-id":["10.1145\/3528223.3530074"],"URL":"https:\/\/doi.org\/10.1145\/3528223.3530074","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,7]]},"assertion":[{"value":"2022-07-22","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}