{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:47:02Z","timestamp":1750308422007,"version":"3.41.0"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2021,4,26]],"date-time":"2021-04-26T00:00:00Z","timestamp":1619395200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Comput. Graph. Interact. Tech."],"published-print":{"date-parts":[[2021,4,26]]},"abstract":"<jats:p>Image-based rendering (IBR) provides a rich toolset for free-viewpoint navigation in captured scenes. Many methods exist, usually with an emphasis either on image quality or rendering speed. In this paper we identify common IBR artifacts and combine the strengths of different algorithms to strike a good balance in the speed\/quality tradeoff. First, we address the problem of visible color seams that arise from blending casually-captured input images by explicitly treating view-dependent effects. Second, we compensate for geometric reconstruction errors by refining per-view information using a novel clustering and filtering approach. Finally, we devise a practical hybrid IBR algorithm, which locally identifies and utilizes the rendering method best suited for an image region while retaining interactive rates. We compare our method against classical and modern (neural) approaches in indoor and outdoor scenes and demonstrate superiority in quality and\/or speed.<\/jats:p>","DOI":"10.1145\/3451260","type":"journal-article","created":{"date-parts":[[2021,4,29]],"date-time":"2021-04-29T04:17:37Z","timestamp":1619669857000},"page":"1-20","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Hybrid Image-based Rendering for Free-view Synthesis"],"prefix":"10.1145","volume":"4","author":[{"given":"Siddhant","family":"Prakash","sequence":"first","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur and Inria, Sophia-Antipolis, France"}]},{"given":"Thomas","family":"Leimk\u00fchler","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur and Inria, Sophia-Antipolis, France"}]},{"given":"Simon","family":"Rodriguez","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur and Inria, Sophia-Antipolis, France"}]},{"given":"George","family":"Drettakis","sequence":"additional","affiliation":[{"name":"Universit\u00e9 C\u00f4te d'Azur and Inria, Sophia-Antipolis, France"}]}],"member":"320","published-online":{"date-parts":[[2021,4,28]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Interactive Digital Photomontage. In ACM SIGGRAPH 2004 Papers","author":"Agarwala Aseem","year":"2004","unstructured":"Aseem Agarwala , Mira Dontcheva , Maneesh Agrawala , Steven Drucker , Alex Colburn , Brian Curless , David Salesin , and Michael Cohen . 2004 . Interactive Digital Photomontage. In ACM SIGGRAPH 2004 Papers ( Los Angeles, California) (SIGGRAPH '04). Association for Computing Machinery, New York, NY, USA, 294--302. https:\/\/doi.org\/10.1145\/1 186562.1015718 10.1145\/1186562.1015718 Aseem Agarwala, Mira Dontcheva, Maneesh Agrawala, Steven Drucker, Alex Colburn, Brian Curless, David Salesin, and Michael Cohen. 2004. Interactive Digital Photomontage. In ACM SIGGRAPH 2004 Papers (Los Angeles, California) (SIGGRAPH '04). Association for Computing Machinery, New York, NY, USA, 294--302. https:\/\/doi.org\/10.1145\/1186562.1015718"},{"key":"e_1_2_2_2_1","unstructured":"Sebastien Bonopera Peter Hedman Jerome Esnault Siddhant Prakash Simon Rodriguez Theo Thonat Mehdi Benadel Gaurav Chaurasia Julien Philip and George Drettakis. 2020. sibr: A System for Image Based Rendering. https:\/\/sibr.gitlabpages.inria.fr\/  Sebastien Bonopera Peter Hedman Jerome Esnault Siddhant Prakash Simon Rodriguez Theo Thonat Mehdi Benadel Gaurav Chaurasia Julien Philip and George Drettakis. 2020. sibr: A System for Image Based Rendering. https:\/\/sibr.gitlabpages.inria.fr\/"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383309"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2015.59"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487228.2487238"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/218380.218395"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/166117.166153"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/893689"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2008.01138.x"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00247"},{"key":"e_1_2_2_11_1","volume-title":"Ambient Point Clouds for View Interpolation. In ACM SIGGRAPH 2010 Papers","author":"Goesele Michael","year":"2010","unstructured":"Michael Goesele , Jens Ackermann , Simon Fuhrmann , Carsten Haubold , Ronny Klowsky , Drew Steedly , and Richard Szeliski . 2010 . Ambient Point Clouds for View Interpolation. In ACM SIGGRAPH 2010 Papers ( Los Angeles, California) (SIGGRAPH '10). Association for Computing Machinery, New York, NY, USA, Article 95, 6 pages. https:\/\/doi.org\/10.1145\/ 1833349.1778832 10.1145\/1833349.1778832 Michael Goesele, Jens Ackermann, Simon Fuhrmann, Carsten Haubold, Ronny Klowsky, Drew Steedly, and Richard Szeliski. 2010. Ambient Point Clouds for View Interpolation. In ACM SIGGRAPH 2010 Papers (Los Angeles, California) (SIGGRAPH '10). Association for Computing Machinery, New York, NY, USA, Article 95, 6 pages. https:\/\/doi.org\/10.1145\/1833349.1778832"},{"key":"e_1_2_2_12_1","volume-title":"Multi-View Stereo for Community Photo Collections. In 2007 IEEE 11th International Conference on Computer Vision. 1--8. https:\/\/doi.org\/10","author":"Goesele M.","year":"2007","unstructured":"M. Goesele , N. Snavely , B. Curless , H. Hoppe , and S. M. Seitz . 2007 . Multi-View Stereo for Community Photo Collections. In 2007 IEEE 11th International Conference on Computer Vision. 1--8. https:\/\/doi.org\/10 .1109\/ICCV. 2007 .4408933 10.1109\/ICCV.2007.4408933 M. Goesele, N. Snavely, B. Curless, H. Hoppe, and S. M. Seitz. 2007. Multi-View Stereo for Community Photo Collections. In 2007 IEEE 11th International Conference on Computer Vision. 1--8. https:\/\/doi.org\/10.1109\/ICCV.2007.4408933"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.55"},{"volume-title":"The Lumigraph. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96)","author":"Gortler Steven J.","key":"e_1_2_2_14_1","unstructured":"Steven J. Gortler , Radek Grzeszczuk , Richard Szeliski , and Michael F. Cohen . 1996 . The Lumigraph. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96) . Association for Computing Machinery, New York, NY, USA, 43--54. https:\/\/doi.org\/10.1145\/237170.237200 10.1145\/237170.237200 Steven J. Gortler, Radek Grzeszczuk, Richard Szeliski, and Michael F. Cohen. 1996. The Lumigraph. In Proceedings of the 23rd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH '96). Association for Computing Machinery, New York, NY, USA, 43--54. https:\/\/doi.org\/10.1145\/237170.237200"},{"key":"e_1_2_2_15_1","volume-title":"Non-rigid dense correspondence with applications for image enhancement. ACM transactions on graphics (TOG) 30, 4","author":"HaCohen Yoav","year":"2011","unstructured":"Yoav HaCohen , Eli Shechtman , Dan B Goldman , and Dani Lischinski . 2011. Non-rigid dense correspondence with applications for image enhancement. ACM transactions on graphics (TOG) 30, 4 ( 2011 ), 1--10. Yoav HaCohen, Eli Shechtman, Dan B Goldman, and Dani Lischinski. 2011. Non-rigid dense correspondence with applications for image enhancement. ACM transactions on graphics (TOG) 30, 4 (2011), 1--10."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461997"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275084"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982420"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130824"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995693"},{"key":"e_1_2_2_21_1","unstructured":"Markus Kettunen Erik H\u00e4rk\u00f6nen and Jaakko Lehtinen. 2019. E-LPIPS: Robust Perceptual Image Similarity via Random Transformation Ensembles. arXiv:1906.03973 [cs.CV]  Markus Kettunen Erik H\u00e4rk\u00f6nen and Jaakko Lehtinen. 2019. E-LPIPS: Robust Perceptual Image Similarity via Random Transformation Ensembles. arXiv:1906.03973 [cs.CV]"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.70732"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073599"},{"key":"e_1_2_2_24_1","volume-title":"Modeling and Visualization (VMV 2020) and the 10th Eurographics Workshop on Visual Computing for Biology and Medicine (VCBM","author":"Koltun Vladlen","year":"2020","unstructured":"Vladlen Koltun . 2020 . Towards Photorealism. (2020). https:\/\/youtu.be\/Rd0nBO6--bM 42nd German Conference on Pattern Recognition (DAGM GCPR 2020), the 25th International Symposium on Vision , Modeling and Visualization (VMV 2020) and the 10th Eurographics Workshop on Visual Computing for Biology and Medicine (VCBM 2020). Vladlen Koltun. 2020. Towards Photorealism. (2020). https:\/\/youtu.be\/Rd0nBO6--bM 42nd German Conference on Pattern Recognition (DAGM GCPR 2020), the 25th International Symposium on Vision, Modeling and Visualization (VMV 2020) and the 10th Eurographics Workshop on Visual Computing for Biology and Medicine (VCBM 2020)."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508369"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882264"},{"key":"e_1_2_2_27_1","volume-title":"Seamless Mosaicing of Image-Based Texture Maps. In 2007 IEEE Conference on Computer Vision and Pattern Recognition. 1--6. https:\/\/doi.org\/10","author":"Lempitsky V.","year":"2007","unstructured":"V. Lempitsky and D. Ivanov . 2007 . Seamless Mosaicing of Image-Based Texture Maps. In 2007 IEEE Conference on Computer Vision and Pattern Recognition. 1--6. https:\/\/doi.org\/10 .1109\/CVPR. 2007 .383078 10.1109\/CVPR.2007.383078 V. Lempitsky and D. Ivanov. 2007. Seamless Mosaicing of Image-Based Texture Maps. In 2007 IEEE Conference on Computer Vision and Pattern Recognition. 1--6. https:\/\/doi.org\/10.1109\/CVPR.2007.383078"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/237170.237199"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323020"},{"key":"e_1_2_2_30_1","volume-title":"Structural Similarity-Based Object Tracking in Video Sequences. In 2006 9th International Conference on Information Fusion. 1--6. https:\/\/doi.org\/10","author":"Loza A.","year":"2006","unstructured":"A. Loza , L. Mihaylova , N. Canagarajah , and D. Bull . 2006 . Structural Similarity-Based Object Tracking in Video Sequences. In 2006 9th International Conference on Information Fusion. 1--6. https:\/\/doi.org\/10 .1109\/ICIF. 2006 .301574 10.1109\/ICIF.2006.301574 A. Loza, L. Mihaylova, N. Canagarajah, and D. Bull. 2006. Structural Similarity-Based Object Tracking in Video Sequences. In 2006 9th International Conference on Information Fusion. 1--6. https:\/\/doi.org\/10.1109\/ICIF.2006.301574"},{"key":"e_1_2_2_31_1","doi-asserted-by":"crossref","unstructured":"Ricardo Martin-Brualla Noha Radwan Mehdi S. M. Sajjadi Jonathan T. Barron Alexey Dosovitskiy and Daniel Duckworth. 2021. NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections. arXiv:2008.02268 [cs.CV]  Ricardo Martin-Brualla Noha Radwan Mehdi S. M. Sajjadi Jonathan T. Barron Alexey Dosovitskiy and Daniel Duckworth. 2021. NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections. arXiv:2008.02268 [cs.CV]","DOI":"10.1109\/CVPR46437.2021.00713"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/218380.218398"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00704"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322980"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58452-8_24"},{"volume-title":"Reproducible Research in Pattern Recognition","author":"Moulon Pierre","key":"e_1_2_2_36_1","unstructured":"Pierre Moulon , Pascal Monasse , Romuald Perrot , and Renaud Marlet . 2017. OpenMVG: Open Multiple View Geometry . In Reproducible Research in Pattern Recognition , Bertrand Kerautret, Miguel Colom, and Pascal Monasse (Eds.). Springer International Publishing , Cham , 60--74. Pierre Moulon, Pascal Monasse, Romuald Perrot, and Renaud Marlet. 2017. OpenMVG: Open Multiple View Geometry. In Reproducible Research in Pattern Recognition, Bertrand Kerautret, Miguel Colom, and Pascal Monasse (Eds.). Springer International Publishing, Cham, 60--74."},{"key":"e_1_2_2_37_1","volume-title":"Poisson Image Editing. In ACM SIGGRAPH 2003 Papers. Association for Computing Machinery","author":"P\u00e9rez Patrick","year":"2003","unstructured":"Patrick P\u00e9rez , Michel Gangnet , and Andrew Blake . 2003 . Poisson Image Editing. In ACM SIGGRAPH 2003 Papers. Association for Computing Machinery , New York, NY, USA, 313--318. https:\/\/doi.org\/10.1145\/1 201775.882269 10.1145\/1201775.882269 Patrick P\u00e9rez, Michel Gangnet, and Andrew Blake. 2003. Poisson Image Editing. In ACM SIGGRAPH 2003 Papers. Association for Computing Machinery, New York, NY, USA, 313--318. https:\/\/doi.org\/10.1145\/1201775.882269"},{"key":"e_1_2_2_38_1","unstructured":"Capturing Reality. 2018. RealityCapture reconstruction software. https:\/\/www.capturingreality.com\/Product.  Capturing Reality. 2018. RealityCapture reconstruction software. https:\/\/www.capturingreality.com\/Product."},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58529-7_37"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3384535"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46487-9_31"},{"volume-title":"Image-based rendering","author":"Shum Heung-Yeung","key":"e_1_2_2_42_1","unstructured":"Heung-Yeung Shum , Shing-Chow Chan , and Sing Bing Kang . 2008. Image-based rendering . Springer Science & Business Media . Heung-Yeung Shum, Shing-Chow Chan, and Sing Bing Kang. 2008. Image-based rendering. Springer Science & Business Media."},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185596"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00254"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1179352.1141964"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00026"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14022"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323035"},{"key":"e_1_2_2_49_1","volume-title":"IGNOR: Image-guided Neural Object Rendering. CoRR abs\/1811.10720","author":"Thies Justus","year":"2018","unstructured":"Justus Thies , Michael Zollh\u00f6fer , Christian Theobalt , Marc Stamminger , and Matthias Nie\u00dfner . 2018 . IGNOR: Image-guided Neural Object Rendering. CoRR abs\/1811.10720 (2018). arXiv:1811.10720 http:\/\/arxiv.org\/abs\/1811.10720 Justus Thies, Michael Zollh\u00f6fer, Christian Theobalt, Marc Stamminger, and Matthias Nie\u00dfner. 2018. IGNOR: Image-guided Neural Object Rendering. CoRR abs\/1811.10720 (2018). arXiv:1811.10720 http:\/\/arxiv.org\/abs\/1811.10720"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_54"},{"key":"e_1_2_2_51_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2980179.2982432","article-title":"Emptying, refurnishing, and relighting indoor spaces","volume":"35","author":"Zhang Edward","year":"2016","unstructured":"Edward Zhang , Michael F Cohen , and Brian Curless . 2016 . Emptying, refurnishing, and relighting indoor spaces . ACM Transactions on Graphics (TOG) 35 , 6 (2016), 1 -- 14 . Edward Zhang, Michael F Cohen, and Brian Curless. 2016. Emptying, refurnishing, and relighting indoor spaces. ACM Transactions on Graphics (TOG) 35, 6 (2016), 1--14.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_52_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2601097.2601134","article-title":"Color map optimization for 3D reconstruction with consumer depth cameras","volume":"33","author":"Zhou Qian-Yi","year":"2014","unstructured":"Qian-Yi Zhou and Vladlen Koltun . 2014 . Color map optimization for 3D reconstruction with consumer depth cameras . ACM Transactions on Graphics (TOG) 33 , 4 (2014), 1 -- 10 . Qian-Yi Zhou and Vladlen Koltun. 2014. Color map optimization for 3D reconstruction with consumer depth cameras. ACM Transactions on Graphics (TOG) 33, 4 (2014), 1--10.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_53_1","volume-title":"Stereo Magnification: Learning View Synthesis using Multiplane Images. arXiv:1805.09817 [cs.CV]","author":"Zhou Tinghui","year":"2018","unstructured":"Tinghui Zhou , Richard Tucker , John Flynn , Graham Fyffe , and Noah Snavely . 2018 . Stereo Magnification: Learning View Synthesis using Multiplane Images. arXiv:1805.09817 [cs.CV] Tinghui Zhou, Richard Tucker, John Flynn, Graham Fyffe, and Noah Snavely. 2018. Stereo Magnification: Learning View Synthesis using Multiplane Images. arXiv:1805.09817 [cs.CV]"}],"container-title":["Proceedings of the ACM on Computer Graphics and Interactive Techniques"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3451260","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3451260","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T17:49:26Z","timestamp":1750268966000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3451260"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,26]]},"references-count":53,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,4,26]]}},"alternative-id":["10.1145\/3451260"],"URL":"https:\/\/doi.org\/10.1145\/3451260","relation":{},"ISSN":["2577-6193"],"issn-type":[{"type":"electronic","value":"2577-6193"}],"subject":[],"published":{"date-parts":[[2021,4,26]]},"assertion":[{"value":"2021-04-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}