{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T11:31:19Z","timestamp":1790681479740,"version":"4.1.0"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2015,7,27]],"date-time":"2015-07-27T00:00:00Z","timestamp":1437955200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2015,7,27]]},"abstract":"<jats:p>Many compelling video processing effects can be achieved if per-pixel depth information and 3D camera calibrations are known. However, the success of such methods is highly dependent on the accuracy of this \"scene-space\" information. We present a novel, sampling-based framework for processing video that enables high-quality scene-space video effects in the presence of inevitable errors in depth and camera pose estimation. Instead of trying to improve the explicit 3D scene representation, the key idea of our method is to exploit the high redundancy of approximate scene information that arises due to most scene points being visible multiple times across many frames of video. Based on this observation, we propose a novel pixel gathering and filtering approach. The gathering step is general and collects pixel samples in scene-space, while the filtering step is application-specific and computes a desired output video from the gathered sample sets. Our approach is easily parallelizable and has been implemented on GPU, allowing us to take full advantage of large volumes of video data and facilitating practical runtimes on HD video using a standard desktop computer. Our generic scene-space formulation is able to comprehensively describe a multitude of video processing applications such as denoising, deblurring, super resolution, object removal, computational shutter functions, and other scene-space camera effects. We present results for various casually captured, hand-held, moving, compressed, monocular videos depicting challenging scenes recorded in uncontrolled environments.<\/jats:p>","DOI":"10.1145\/2766920","type":"journal-article","created":{"date-parts":[[2015,7,28]],"date-time":"2015-07-28T12:26:38Z","timestamp":1438086398000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":32,"title":["Sampling based scene-space video processing"],"prefix":"10.1145","volume":"34","author":[{"given":"Felix","family":"Klose","sequence":"first","affiliation":[{"name":"Disney Research Zurich and TU Braunschweig"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Oliver","family":"Wang","sequence":"additional","affiliation":[{"name":"Disney Research Zurich"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jean-Charles","family":"Bazin","sequence":"additional","affiliation":[{"name":"Disney Research Zurich"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marcus","family":"Magnor","sequence":"additional","affiliation":[{"name":"TU Braunschweig"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alexander","family":"Sorkine-Hornung","sequence":"additional","affiliation":[{"name":"Disney Research Zurich"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2015,7,27]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2003.1175093"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/2629645"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.5555\/2383847.2383889"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185560"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/2319074.2321415"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.161"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964964"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778832"},{"key":"e_1_2_2_9_1","unstructured":"Google 2015. Project Tango. https:\/\/www.google.com\/atap\/projecttango\/#project.  Google 2015. Project Tango. https:\/\/www.google.com\/atap\/projecttango\/#project."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33718-5_49"},{"key":"e_1_2_2_11_1","doi-asserted-by":"crossref","unstructured":"Gupta A. Bhat P. Dontcheva M. Curless B. Deussen O. and Cohen M. 2009. Enhancing and experiencing spacetime resolution with videos and stills. In ICCP.  Gupta A. Bhat P. Dontcheva M. Curless B. Deussen O. and Cohen M. 2009. Enhancing and experiencing spacetime resolution with videos and stills. In ICCP.","DOI":"10.1109\/ICCPHOT.2009.5559006"},{"key":"e_1_2_2_12_1","unstructured":"Infognition 2015. Infognition superresolution plugin. http:\/\/www.infognition.com\/super_resolution\/.  Infognition 2015. Infognition superresolution plugin. http:\/\/www.infognition.com\/super_resolution\/."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.147"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601209"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-009-0233-1"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276497"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601195"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12285"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185530"},{"key":"e_1_2_2_20_1","doi-asserted-by":"crossref","unstructured":"Lipski C. Klose F. and Magnor M. A. 2014. Correspondence and depth-image based rendering a hybrid approach for free-viewpoint video. T-CSVT.  Lipski C. Klose F. and Magnor M. A. 2014. Correspondence and depth-image based rendering a hybrid approach for free-viewpoint video. T-CSVT.","DOI":"10.1109\/TCSVT.2014.2302379"},{"key":"e_1_2_2_21_1","doi-asserted-by":"crossref","unstructured":"Newcombe R. A. and Davison A. J. 2010. Live dense reconstruction with a single moving camera. In CVPR.  Newcombe R. A. and Davison A. J. 2010. Live dense reconstruction with a single moving camera. In CVPR.","DOI":"10.1109\/CVPR.2010.5539794"},{"key":"e_1_2_2_22_1","doi-asserted-by":"crossref","unstructured":"\u00d6ztireli A. C. Guennebaud G. and Gross M. 2009. Feature preserving point set surfaces based on non-linear kernel regression. CGF (Eurographics).  \u00d6ztireli A. C. Guennebaud G. and Gross M. 2009. Feature preserving point set surfaces based on non-linear kernel regression. CGF (Eurographics).","DOI":"10.1111\/j.1467-8659.2009.01388.x"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1281500.1281602"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2008.29"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2012.03003.x"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1014573219977"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.19"},{"key":"e_1_2_2_28_1","doi-asserted-by":"crossref","unstructured":"Shum H. Chan S. and Kang S. B. 2007. Image-based rendering. Springer.   Shum H. Chan S. and Kang S. B. 2007. Image-based rendering. Springer.","DOI":"10.1109\/MSP.2007.4317461"},{"key":"e_1_2_2_29_1","unstructured":"Sun J. Xu Z. and Shum H. 2008. Image super-resolution using gradient profile prior. In CVPR.  Sun J. Xu Z. and Shum H. 2008. Image super-resolution using gradient profile prior. In CVPR."},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.72"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.15"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2005.537"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073259"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2009.47"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.52"},{"key":"e_1_2_2_36_1","unstructured":"Zhang L. Vaddadi S. Jin H. and Nayar S. K. 2009. Multiple view image denoising. In CVPR.  Zhang L. Vaddadi S. Jin H. and Nayar S. K. 2009. Multiple view image denoising. In CVPR."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1015706.1015766"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383300"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2766920","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2766920","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T18:56:02Z","timestamp":1750272962000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2766920"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,7,27]]},"references-count":38,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2015,7,27]]}},"alternative-id":["10.1145\/2766920"],"URL":"https:\/\/doi.org\/10.1145\/2766920","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,7,27]]},"assertion":[{"value":"2015-07-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}