{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T20:31:02Z","timestamp":1777149062816,"version":"3.51.4"},"reference-count":79,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2022,11,30]],"date-time":"2022-11-30T00:00:00Z","timestamp":1669766400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"ERC","award":["788065"],"award-info":[{"award-number":["788065"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2022,12]]},"abstract":"<jats:p>View-dependent effects such as reflections pose a substantial challenge for image-based and neural rendering algorithms. Above all, curved reflectors are particularly hard, as they lead to highly non-linear reflection flows as the camera moves. We introduce a new point-based representation to compute Neural Point Catacaustics allowing novel-view synthesis of scenes with curved reflectors, from a set of casually-captured input photos. At the core of our method is a neural warp field that models catacaustic trajectories of reflections, so complex specular effects can be rendered using efficient point splatting in conjunction with a neural renderer. One of our key contributions is the explicit representation of reflections with a reflection point cloud which is displaced by the neural warp field, and a primary point cloud which is optimized to represent the rest of the scene. After a short manual annotation step, our approach allows interactive high-quality renderings of novel views with accurate reflection flow. Additionally, the explicit representation of reflection flow supports several forms of scene manipulation in captured scenes, such as reflection editing, cloning of specular objects, reflection tracking across views, and comfortable stereo viewing. We provide the source code and other supplemental material on https:\/\/repo-sam.inria.fr\/fungraph\/neural_catacaustics\/<\/jats:p>","DOI":"10.1145\/3550454.3555497","type":"journal-article","created":{"date-parts":[[2022,11,30]],"date-time":"2022-11-30T21:19:07Z","timestamp":1669843147000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":70,"title":["Neural Point Catacaustics for Novel-View Synthesis of Reflections"],"prefix":"10.1145","volume":"41","author":[{"given":"Georgios","family":"Kopanas","sequence":"first","affiliation":[{"name":"Inria &amp; Universit\u00e9 C\u00f4te d'Azur, France"}]},{"given":"Thomas","family":"Leimk\u00fchler","sequence":"additional","affiliation":[{"name":"Max-Planck-Institut f\u00fcr Informatik, Germany"}]},{"given":"Gilles","family":"Rainer","sequence":"additional","affiliation":[{"name":"Inria &amp; Universit\u00e9 C\u00f4te d'Azur, France"}]},{"given":"Cl\u00e9ment","family":"Jambon","sequence":"additional","affiliation":[{"name":"Inria &amp; Universit\u00e9 C\u00f4te d'Azur and Ecole Polytechnique, France"}]},{"given":"George","family":"Drettakis","sequence":"additional","affiliation":[{"name":"Inria &amp; Universit\u00e9 C\u00f4te d'Azur, France"}]}],"member":"320","published-online":{"date-parts":[[2022,11,30]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58542-6_42"},{"key":"e_1_2_2_2_1","volume-title":"Srinivasan","author":"Barron Jonathan T.","year":"2021","unstructured":"Jonathan T. Barron, Ben Mildenhall, Matthew Tancik, Peter Hedman, Ricardo Martin-Brualla, and Pratul P. Srinivasan. 2021. Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields. ICCV (2021)."},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417827"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/360349.360353"},{"key":"e_1_2_2_5_1","volume-title":"Curves and Singularities: a geometrical introduction to singularity theory","author":"Bruce James William","unstructured":"James William Bruce and PJ Giblin. 1992. Curves and Singularities: a geometrical introduction to singularity theory. Cambridge university press."},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311553"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.565"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/380666.380670"},{"key":"e_1_2_2_9_1","volume-title":"Computer Graphics Forum","author":"D\u0105ba\u0142a \u0141ukasz","unstructured":"\u0141ukasz D\u0105ba\u0142a, Petr Kellnhofer, Tobias Ritschel, Piotr Didyk, Krzysztof Templin, Karol Myszkowski, Przemyslaw Rokita, and H-P Seidel. 2014. Manipulating refractive and reflective binocular disparity. In Computer Graphics Forum, Vol. 33. Wiley Online Library, 53--62."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/253284.253308"},{"key":"e_1_2_2_11_1","doi-asserted-by":"crossref","unstructured":"David H Douglas and Thomas K Peucker. 1973. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: the international journal for geographic information and geovisualization 10 2 (1973) 112--122.","DOI":"10.3138\/FM57-6770-U75U-7727"},{"key":"e_1_2_2_12_1","first-page":"8","article-title":"Accurate interactive specular reflections on curved objects. In Vision Modeling and Visualization (VMV 2005). Berlin","volume":"2005","author":"Estalella Pau","year":"2005","unstructured":"Pau Estalella, Ignacio Martin, George Drettakis, Dani Tost, Olivier Devillers, and Fr\u00e9d\u00e9ric Cazals. 2005. Accurate interactive specular reflections on curved objects. In Vision Modeling and Visualization (VMV 2005). Berlin: Akademische Verl.-Ges. Aka, 2005., 8.","journal-title":"Akademische Verl.-Ges. Aka"},{"key":"e_1_2_2_13_1","volume-title":"Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling.","author":"Feng Wanquan","year":"2022","unstructured":"Wanquan Feng, Jin Li, Hongrui Cai, Xiaonan Luo, and Juyong Zhang. 2022. Neural Points: Point Cloud Representation with Neural Fields for Arbitrary Upsampling. (2022)."},{"key":"e_1_2_2_14_1","volume-title":"Andries Van Dam, Steven K Feiner, John F Hughes, and J Hughes.","author":"Foley James D","year":"1996","unstructured":"James D Foley, Foley Dan Van, Andries Van Dam, Steven K Feiner, John F Hughes, and J Hughes. 1996. Computer graphics: principles and practice. Vol. 12110. Addison-Wesley Professional."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01408"},{"key":"e_1_2_2_16_1","first-page":"121","article-title":"Reflections on spheres and cylinders of revolution","volume":"3","author":"Glaeser Georg","year":"1999","unstructured":"Georg Glaeser. 1999. Reflections on spheres and cylinders of revolution. Journal for Geometry and Graphics 3, 2 (1999), 121--139.","journal-title":"Journal for Geometry and Graphics"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2007.4408933"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.1986.276658"},{"key":"e_1_2_2_19_1","volume-title":"Point-based graphics","author":"Gross Markus","unstructured":"Markus Gross and Hanspeter Pfister. 2007. Point-based graphics. Elsevier."},{"key":"e_1_2_2_20_1","volume-title":"NeRFReN: Neural Radiance Fields with Reflections. arXiv preprint arXiv:2111.15234","author":"Guo Yuan-Chen","year":"2021","unstructured":"Yuan-Chen Guo, Di Kang, Linchao Bao, Yu He, and Song-Hai Zhang. 2021. NeRFReN: Neural Radiance Fields with Reflections. arXiv preprint arXiv:2111.15234 (2021)."},{"key":"e_1_2_2_21_1","article-title":"Theory of systems of rays","author":"Hamilton William Rowan","year":"1828","unstructured":"William Rowan Hamilton. 1828. Theory of systems of rays. The Transactions of the Royal Irish Academy (1828), 69--174.","journal-title":"The Transactions of the Royal Irish Academy (1828), 69--174."},{"key":"e_1_2_2_22_1","first-page":"B18","volume-title":"Deep Blending for Free-Viewpoint Image-Based Rendering. ACM Transactions on Graphics (SIGGRAPH Asia Conference Proceedings) 37","author":"Hedman Peter","year":"2018","unstructured":"Peter Hedman, Julien Philip, True Price, Jan-Michael Frahm, George Drettakis, and Gabriel Brostow. 2018. Deep Blending for Free-Viewpoint Image-Based Rendering. ACM Transactions on Graphics (SIGGRAPH Asia Conference Proceedings) 37, 6 (November 2018). http:\/\/www-sop.inria.fr\/reves\/Basilic\/2018\/HPPFDB18"},{"key":"e_1_2_2_23_1","volume-title":"Baking Neural Radiance Fields for Real-Time View Synthesis. ICCV","author":"Hedman Peter","year":"2021","unstructured":"Peter Hedman, Pratul P. Srinivasan, Ben Mildenhall, Jonathan T. Barron, and Paul Debevec. 2021. Baking Neural Radiance Fields for Real-Time View Synthesis. ICCV (2021)."},{"key":"e_1_2_2_24_1","volume-title":"Stereoscopic Displays and Virtual Reality Systems VIII","author":"Jones Graham R","unstructured":"Graham R Jones, Delman Lee, Nicolas S Holliman, and David Ezra. 2001. Controlling perceived depth in stereoscopic images. In Stereoscopic Displays and Virtual Reality Systems VIII, Vol. 4297. SPIE, 42--53."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1080\/00927872.2012.759956"},{"key":"e_1_2_2_26_1","volume-title":"Computer Graphics Forum","author":"Kopanas Georgios","unstructured":"Georgios Kopanas, Julien Philip, Thomas Leimk\u00fchler, and George Drettakis. 2021. Point-Based Neural Rendering with Per-View Optimization. In Computer Graphics Forum, Vol. 40. Wiley Online Library, 29--43."},{"key":"e_1_2_2_27_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2508363.2508369","article-title":"Image-based rendering in the gradient domain","volume":"32","author":"Kopf Johannes","year":"2013","unstructured":"Johannes Kopf, Fabian Langguth, Daniel Scharstein, Richard Szeliski, and Michael Goesele. 2013. Image-based rendering in the gradient domain. ACM Transactions on Graphics (TOG) 32, 6 (2013), 1--9.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.2352\/J.ImagingSci.Technol.2009.53.3.030201"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778812"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00149"},{"key":"e_1_2_2_31_1","volume-title":"A catalog of special plane curves","author":"Lawrence J Dennis","unstructured":"J Dennis Lawrence. 2013. A catalog of special plane curves. Courier Corporation."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2007.1106"},{"key":"e_1_2_2_33_1","unstructured":"Gerrit Lochmann Bernhard Reinert Tobias Ritschel Stefan M\u00fcller and Hans-Peter Seidel. 2014. Real-time Reflective and Refractive Novel-view Synthesis.. In VMV. 9--16."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIF.2006.301574"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00704"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58452-8_24"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/133994.134082"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530127"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356498"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280929"},{"key":"e_1_2_2_41_1","volume-title":"Neural Point Light Fields. arXiv preprint arXiv:2112.01473","author":"Ost Julian","year":"2021","unstructured":"Julian Ost, Issam Laradji, Alejandro Newell, Yuval Bahat, and Felix Heide. 2021. Neural Point Light Fields. arXiv preprint arXiv:2112.01473 (2021)."},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00581"},{"key":"e_1_2_2_43_1","volume-title":"Free-viewpoint Indoor Neural Relighting from Multi-view Stereo. ACM Transactions on Graphics","author":"Philip Julien","year":"2021","unstructured":"Julien Philip, S\u00e9bastien Morgenthaler, Micha\u00ebl Gharbi, and George Drettakis. 2021. Free-viewpoint Indoor Neural Relighting from Multi-view Stereo. ACM Transactions on Graphics (2021). http:\/\/www-sop.inria.fr\/reves\/Basilic\/2021\/PMGD21"},{"key":"e_1_2_2_44_1","volume-title":"Computational geometry","author":"Preparata Franco P","unstructured":"Franco P Preparata and Michael Ian Shamos. 1985. Convex hulls: Basic algorithms. In Computational geometry. Springer, 95--149."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276472"},{"key":"e_1_2_2_46_1","unstructured":"Capturing Reality. 2018. RealityCapture reconstruction software. https:\/\/www.capturingreality.com\/Product."},{"key":"e_1_2_2_47_1","volume-title":"Computer Graphics Forum","author":"Ren Liu","unstructured":"Liu Ren, Hanspeter Pfister, and Matthias Zwicker. 2002. Object space EWA surface splatting: A hardware accelerated approach to high quality point rendering. In Computer Graphics Forum, Vol. 21. Wiley Online Library, 461--470."},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01204"},{"key":"e_1_2_2_49_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1618452.1618475","article-title":"Interactive reflection editing","volume":"28","author":"Ritschel Tobias","year":"2009","unstructured":"Tobias Ritschel, Makoto Okabe, Thorsten Thorm\u00e4hlen, and Hans-Peter Seidel. 2009. Interactive reflection editing. ACM Transactions on Graphics (TOG) 28, 5 (2009), 1--7.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_50_1","first-page":"D20","volume-title":"Glossy Probe Reprojection for Interactive Global Illumination. ACM Transactions on Graphics (SIGGRAPH Asia Conference Proceedings) 39","author":"Rodriguez Simon","year":"2020","unstructured":"Simon Rodriguez, Thomas Leimk\u00fchler, Siddhant Prakash, Chris Wyman, Peter Shirley, and George Drettakis. 2020a. Glossy Probe Reprojection for Interactive Global Illumination. ACM Transactions on Graphics (SIGGRAPH Asia Conference Proceedings) 39, 6 (December 2020). http:\/\/www-sop.inria.fr\/reves\/Basilic\/2020\/RLPWSD20"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3384535"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.290"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530122"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5539826"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461980"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1167\/11.8.11"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185596"},{"key":"e_1_2_2_58_1","unstructured":"Vincent Sitzmann Michael Zollh\u00f6fer and Gordon Wetzstein. 2019. Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations. In Advances in Neural Information Processing Systems."},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2000.855826"},{"key":"e_1_2_2_60_1","volume-title":"Computer Graphics Forum","author":"Szirmay-Kalos L\u00e1szl\u00f3","unstructured":"L\u00e1szl\u00f3 Szirmay-Kalos, Tam\u00e1s Umenhoffer, Gustavo Patow, L\u00e1szl\u00f3 Sz\u00e9csi, and Mateu Sbert. 2009. Specular effects on the gpu: State of the art. In Computer Graphics Forum, Vol. 28. Wiley Online Library, 1586--1617."},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185588"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14022"},{"key":"e_1_2_2_63_1","unstructured":"Ayush Tewari Justus Thies Ben Mildenhall Pratul Srinivasan Edgar Tretschk Yifan Wang Christoph Lassner Vincent Sitzmann Ricardo Martin-Brualla Stephen Lombardi et al. 2021. Advances in neural rendering. arXiv preprint arXiv:2111.05849 (2021)."},{"key":"e_1_2_2_64_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323035"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01272"},{"key":"e_1_2_2_66_1","volume-title":"Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields. arXiv preprint arXiv:2112.03907","author":"Verbin Dor","year":"2021","unstructured":"Dor Verbin, Peter Hedman, Ben Mildenhall, Todd Zickler, Jonathan T Barron, and Pratul P Srinivasan. 2021. Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields. arXiv preprint arXiv:2112.03907 (2021)."},{"key":"e_1_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCP51581.2021.9466270"},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201319"},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/800249.807419"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01261-8_6"},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00749"},{"key":"e_1_2_2_72_1","volume-title":"NeX: Real-time View Synthesis with Neural Basis Expansion. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Wizadwongsa Suttisak","year":"2021","unstructured":"Suttisak Wizadwongsa, Pakkapon Phongthawee, Jiraphon Yenphraphai, and Supasorn Suwajanakorn. 2021. NeX: Real-time View Synthesis with Neural Basis Expansion. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_73_1","volume-title":"Eulerian video magnification for revealing subtle changes in the world. ACM transactions on graphics (TOG) 31, 4","author":"Wu Hao-Yu","year":"2012","unstructured":"Hao-Yu Wu, Michael Rubinstein, Eugene Shih, John Guttag, Fr\u00e9do Durand, and William Freeman. 2012. Eulerian video magnification for revealing subtle changes in the world. ACM transactions on graphics (TOG) 31, 4 (2012), 1--8."},{"key":"e_1_2_2_74_1","volume-title":"Neural Fields in Visual Computing and Beyond. arXiv preprint arXiv:2111.11426","author":"Xie Yiheng","year":"2021","unstructured":"Yiheng Xie, Towaki Takikawa, Shunsuke Saito, Or Litany, Shiqin Yan, Numair Khan, Federico Tombari, James Tompkin, Vincent Sitzmann, and Srinath Sridhar. 2021. Neural Fields in Visual Computing and Beyond. arXiv preprint arXiv:2111.11426 (2021)."},{"key":"e_1_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459849"},{"key":"e_1_2_2_76_1","unstructured":"Qiangeng Xu Zexiang Xu Julien Philip Sai Bi Zhixin Shu Kalyan Sunkavalli and Ulrich Neumann. 2022. Point-NeRF: Point-based Neural Radiance Fields. (2022)."},{"key":"e_1_2_2_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356513"},{"key":"e_1_2_2_78_1","unstructured":"Alex Yu Ruilong Li Matthew Tancik Hao Li Ren Ng and Angjoo Kanazawa. 2021. PlenOctrees for Real-time Rendering of Neural Radiance Fields. In ICCV."},{"key":"e_1_2_2_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383300"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550454.3555497","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3550454.3555497","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:51:43Z","timestamp":1750182703000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3550454.3555497"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,30]]},"references-count":79,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,12]]}},"alternative-id":["10.1145\/3550454.3555497"],"URL":"https:\/\/doi.org\/10.1145\/3550454.3555497","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,30]]},"assertion":[{"value":"2022-11-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}