{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T17:43:09Z","timestamp":1777657389258,"version":"3.51.4"},"reference-count":27,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2018,12,4]],"date-time":"2018-12-04T00:00:00Z","timestamp":1543881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2018,12,31]]},"abstract":"<jats:p>Current AR systems only track sparse geometric features but do not compute depth for all pixels. For this reason, most AR effects are pure overlays that can never be occluded by real objects. We present a novel algorithm that propagates sparse depth to every pixel in near realtime. The produced depth maps are spatio-temporally smooth but exhibit sharp discontinuities at depth edges. This enables AR effects that can fully interact with and be occluded by the real scene. Our algorithm uses a video and a sparse SLAM reconstruction as input. It starts by estimating soft depth edges from the gradient of optical flow fields. Because optical flow is unreliable near occlusions we compute forward and backward flow fields and fuse the resulting depth edges using a novel reliability measure. We then localize the depth edges by thinning and aligning them with image edges. Finally, we optimize the propagated depth smoothly but encourage discontinuities at the recovered depth edges. We present results for numerous real-world examples and demonstrate the effectiveness for several occlusion-aware AR video effects. To quantitatively evaluate our algorithm we characterize the properties that make depth maps desirable for AR applications, and present novel evaluation metrics that capture how well these are satisfied. Our results compare favorably to a set of competitive baseline algorithms in this context.<\/jats:p>","DOI":"10.1145\/3272127.3275083","type":"journal-article","created":{"date-parts":[[2018,11,28]],"date-time":"2018-11-28T19:16:10Z","timestamp":1543432570000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":75,"title":["Fast depth densification for occlusion-aware augmented reality"],"prefix":"10.1145","volume":"37","author":[{"given":"Aleksander","family":"Holynski","sequence":"first","affiliation":[{"name":"University of Washington"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Johannes","family":"Kopf","sequence":"additional","affiliation":[{"name":"Facebook"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,12,4]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2980257"},{"key":"e_1_2_1_2_1","volume-title":"The Fast Bilateral Solver. European Conference on Computer Vision (ECCV)","author":"Barron Jonathan T","year":"2016","unstructured":"Jonathan T Barron and Ben Poole . 2016 . The Fast Bilateral Solver. European Conference on Computer Vision (ECCV) (2016), 617--632. Jonathan T Barron and Ben Poole. 2016. The Fast Bilateral Solver. European Conference on Computer Vision (ECCV) (2016), 617--632."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818107"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.1986.4767851"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.298"},{"key":"e_1_2_1_6_1","volume-title":"Direct Sparse Odometry. Transactions on Pattern Analysis and Machine Intelligence (TPAMI)","author":"Engel Jakob","year":"2018","unstructured":"Jakob Engel , Vladlen Koltun , and Daniel Cremers . 2018. Direct Sparse Odometry. Transactions on Pattern Analysis and Machine Intelligence (TPAMI) ( 2018 ). Jakob Engel, Vladlen Koltun, and Daniel Cremers. 2018. Direct Sparse Odometry. Transactions on Pattern Analysis and Machine Intelligence (TPAMI) (2018)."},{"key":"e_1_2_1_7_1","volume-title":"LSD-SLAM: Large-Scale Direct Monocular SLAM. European Conference on Computer Vision (ECCV)","author":"Engel Jakob","year":"2014","unstructured":"Jakob Engel , Thomas Sch\u00f6ps , and Daniel Cremers . 2014 . LSD-SLAM: Large-Scale Direct Monocular SLAM. European Conference on Computer Vision (ECCV) (2014), 834--849. Jakob Engel, Thomas Sch\u00f6ps, and Daniel Cremers. 2014. LSD-SLAM: Large-Scale Direct Monocular SLAM. European Conference on Computer Vision (ECCV) (2014), 834--849."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1561\/0600000052"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25367-6_15"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_29"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015780"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6907233"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360692"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126513"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2018.00155"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2014.2361034"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015777"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2015.2463671"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.5555\/1927006.1927046"},{"key":"e_1_2_1_20_1","doi-asserted-by":"crossref","unstructured":"Johannes Lutz Sch\u00f6nberger Enliang Zheng Marc Pollefeys and Jan-Michael Frahm. 2016. Pixelwise View Selection for Unstructured Multi-View Stereo. (2016).  Johannes Lutz Sch\u00f6nberger Enliang Zheng Marc Pollefeys and Jan-Michael Frahm. 2016. Pixelwise View Selection for Unstructured Multi-View Stereo. (2016).","DOI":"10.1007\/978-3-319-46487-9_31"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.19"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.511"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/1926258.1926261"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/1141911.1142005"},{"key":"e_1_2_1_25_1","volume-title":"Just-in-Time Reconstruction: Inpainting Sparse Maps using Single View Depth Predictors as Priors. arXiv preprint arXiv:1805.04239","author":"Weerasekera Chamara Saroj","year":"2018","unstructured":"Chamara Saroj Weerasekera , Thanuja Dharmasiri , Ravi Garg , Tom Drummond , and Ian Reid . 2018. Just-in-Time Reconstruction: Inpainting Sparse Maps using Single View Depth Predictors as Priors. arXiv preprint arXiv:1805.04239 ( 2018 ). Chamara Saroj Weerasekera, Thanuja Dharmasiri, Ravi Garg, Tom Drummond, and Ian Reid. 2018. Just-in-Time Reconstruction: Inpainting Sparse Maps using Single View Depth Predictors as Priors. arXiv preprint arXiv:1805.04239 (2018)."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.52"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00026"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3272127.3275083","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3272127.3275083","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T00:44:05Z","timestamp":1750207445000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3272127.3275083"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,12,4]]},"references-count":27,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2018,12,31]]}},"alternative-id":["10.1145\/3272127.3275083"],"URL":"https:\/\/doi.org\/10.1145\/3272127.3275083","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,12,4]]},"assertion":[{"value":"2018-12-04","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}