{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T02:40:10Z","timestamp":1774924810523,"version":"3.50.1"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,7,19]],"date-time":"2021-07-19T00:00:00Z","timestamp":1626652800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"UC San Diego Center for Visual Computing"},{"name":"NSF","award":["1703957, 1764078"],"award-info":[{"award-number":["1703957, 1764078"]}]},{"name":"Google Ph.D. Fellowships"},{"name":"Ronald L. Graham Chair"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2021,8,31]]},"abstract":"<jats:p>Path guiding is a promising technique to reduce the variance of path tracing. Although existing online path guiding algorithms can eventually learn good sampling distributions given a large amount of time and samples, the speed of learning becomes a major bottleneck. In this paper, we accelerate the learning of sampling distributions by training a light-weight neural network offline to reconstruct from sparse samples. Uniquely, we design our neural network to directly operate convolutions on a sparse quadtree, which regresses a high-quality hierarchical sampling distribution. Our approach can reconstruct reasonably accurate sampling distributions faster, allowing for efficient path guiding and rendering. In contrast to the recent offline neural path guiding techniques that reconstruct low-resolution 2D images for sampling, our novel hierarchical framework enables more fine-grained directional sampling with less memory usage, effectively advancing the practicality and efficiency of neural path guiding. In addition, we take advantage of hybrid bidirectional samples including both path samples and photons, as we have found this more robust to different light transport scenarios compared to using only one type of sample as in previous work. Experiments on diverse testing scenes demonstrate that our approach often improves rendering results with better visual quality and lower errors. Our framework can also provide the proper balance of speed, memory cost, and robustness.<\/jats:p>","DOI":"10.1145\/3450626.3459810","type":"journal-article","created":{"date-parts":[[2021,7,20]],"date-time":"2021-07-20T00:04:27Z","timestamp":1626739467000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Hierarchical neural reconstruction for path guiding using hybrid path and photon samples"],"prefix":"10.1145","volume":"40","author":[{"given":"Shilin","family":"Zhu","sequence":"first","affiliation":[{"name":"University of California San Diego"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zexiang","family":"Xu","sequence":"additional","affiliation":[{"name":"Adobe Research"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tiancheng","family":"Sun","sequence":"additional","affiliation":[{"name":"University of California San Diego"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandr","family":"Kuznetsov","sequence":"additional","affiliation":[{"name":"University of California San Diego"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mark","family":"Meyer","sequence":"additional","affiliation":[{"name":"Pixar Animation Studios"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Henrik Wann","family":"Jensen","sequence":"additional","affiliation":[{"name":"University of California San Diego and Luxion"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hao","family":"Su","sequence":"additional","affiliation":[{"name":"University of California San Diego"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ravi","family":"Ramamoorthi","sequence":"additional","affiliation":[{"name":"University of California San Diego"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,7,19]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Computer Graphics Forum","author":"Bako Steve","unstructured":"Steve Bako , Mark Meyer , Tony DeRose , and Pradeep Sen . 2019. Offline Deep Importance Sampling for Monte Carlo Path Tracing . In Computer Graphics Forum , Vol. 38 . Wiley Online Library , 527--542. Steve Bako, Mark Meyer, Tony DeRose, and Pradeep Sen. 2019. Offline Deep Importance Sampling for Monte Carlo Path Tracing. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 527--542."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073708"},{"key":"e_1_2_2_3_1","unstructured":"Benedikt Bitterli. 2016. Rendering resources. https:\/\/benedikt-bitterli.me\/resources\/.  Benedikt Bitterli. 2016. Rendering resources. https:\/\/benedikt-bitterli.me\/resources\/."},{"key":"e_1_2_2_4_1","unstructured":"LLC Blend Swap. 2016. Blend swap.  LLC Blend Swap. 2016. Blend swap."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073601"},{"key":"e_1_2_2_6_1","volume-title":"Quadtree Generating Networks: Efficient Hierarchical Scene Parsing with Sparse Convolutions. In The IEEE Winter Conference on Applications of Computer Vision.","author":"Chitta Kashyap","year":"2020","unstructured":"Kashyap Chitta , Jose M Alvarez , and Martial Hebert . 2020 . Quadtree Generating Networks: Efficient Hierarchical Scene Parsing with Sparse Convolutions. In The IEEE Winter Conference on Applications of Computer Vision. Kashyap Chitta, Jose M Alvarez, and Martial Hebert. 2020. Quadtree Generating Networks: Efficient Hierarchical Scene Parsing with Sparse Convolutions. In The IEEE Winter Conference on Applications of Computer Vision."},{"key":"e_1_2_2_7_1","volume-title":"Computer Graphics Forum","author":"Diolatzis Stavros","unstructured":"Stavros Diolatzis , Adrien Gruson , Wenzel Jakob , Derek Nowrouzezahrai , and George Drettakis . 2020. Practical Product Path Guiding Using Linearly Transformed Cosines . In Computer Graphics Forum , Vol. 39 . Wiley Online Library , 23--33. Stavros Diolatzis, Adrien Gruson, Wenzel Jakob, Derek Nowrouzezahrai, and George Drettakis. 2020. Practical Product Path Guiding Using Linearly Transformed Cosines. In Computer Graphics Forum, Vol. 39. Wiley Online Library, 23--33."},{"key":"e_1_2_2_8_1","unstructured":"TM Evermotion. 2012. Evermotion 3d models.  TM Evermotion. 2012. Evermotion 3d models."},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366211"},{"key":"e_1_2_2_10_1","volume-title":"Sparse 3D convolutional neural networks. arXiv preprint arXiv:1505.02890","author":"Graham Ben","year":"2015","unstructured":"Ben Graham . 2015. Sparse 3D convolutional neural networks. arXiv preprint arXiv:1505.02890 ( 2015 ). Ben Graham. 2015. Sparse 3D convolutional neural networks. arXiv preprint arXiv:1505.02890 (2015)."},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00961"},{"key":"e_1_2_2_12_1","volume-title":"Submanifold sparse convolutional networks. arXiv preprint arXiv:1706.01307","author":"Graham Benjamin","year":"2017","unstructured":"Benjamin Graham and Laurens van der Maaten . 2017. Submanifold sparse convolutional networks. arXiv preprint arXiv:1706.01307 ( 2017 ). Benjamin Graham and Laurens van der Maaten. 2017. Submanifold sparse convolutional networks. arXiv preprint arXiv:1706.01307 (2017)."},{"key":"e_1_2_2_13_1","volume-title":"Eurographics Symposium on Rendering","volume":"2018","author":"Guo Jerry","year":"2018","unstructured":"Jerry Guo , Pablo Bauszat , Jacco Bikker , and Elmar Eisemann . 2018 . Primary sample space path guiding . In Eurographics Symposium on Rendering , Vol. 2018 . The Eurographics Association, 73--82. Jerry Guo, Pablo Bauszat, Jacco Bikker, and Elmar Eisemann. 2018. Primary sample space path guiding. In Eurographics Symposium on Rendering, Vol. 2018. The Eurographics Association, 73--82."},{"key":"e_1_2_2_14_1","doi-asserted-by":"crossref","unstructured":"Toshiya Hachisuka Shinji Ogaki and Henrik Wann Jensen. 2008. Progressive photon mapping. In ACM SIGGRAPH Asia 2008 papers. 1--8.  Toshiya Hachisuka Shinji Ogaki and Henrik Wann Jensen. 2008. Progressive photon mapping. In ACM SIGGRAPH Asia 2008 papers. 1--8.","DOI":"10.1145\/1457515.1409083"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366210"},{"key":"e_1_2_2_16_1","volume-title":"Computer Graphics Forum","author":"Herholz Sebastian","unstructured":"Sebastian Herholz , Oskar Elek , Ji\u0159\u00ed Vorba , Hendrik Lensch , and Jaroslav K\u0159iv\u00e1nek . 2016. Product importance sampling for light transport path guiding . In Computer Graphics Forum , Vol. 35 . Wiley Online Library , 67--77. Sebastian Herholz, Oskar Elek, Ji\u0159\u00ed Vorba, Hendrik Lensch, and Jaroslav K\u0159iv\u00e1nek. 2016. Product importance sampling for light transport path guiding. In Computer Graphics Forum, Vol. 35. Wiley Online Library, 67--77."},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3368313"},{"key":"e_1_2_2_18_1","unstructured":"Wenzel Jakob. 2010. Mitsuba renderer. http:\/\/www.mitsuba-renderer.org.  Wenzel Jakob. 2010. Mitsuba renderer. http:\/\/www.mitsuba-renderer.org."},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-7091-9430-0_31"},{"key":"e_1_2_2_20_1","volume-title":"Rendering Techniques' 96","author":"Jensen Henrik Wann","unstructured":"Henrik Wann Jensen . 1996. Global illumination using photon maps . In Rendering Techniques' 96 . Springer , 21--30. Henrik Wann Jensen. 1996. Global illumination using photon maps. In Rendering Techniques' 96. Springer, 21--30."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/15922.15902"},{"key":"e_1_2_2_22_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1966394.1966404"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601219"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01231-1_34"},{"key":"e_1_2_2_26_1","unstructured":"Eric P Lafortune and Yves D Willems. 1993. Bi-directional path tracing. (1993).  Eric P Lafortune and Yves D Willems. 1993. Bi-directional path tracing. (1993)."},{"key":"e_1_2_2_27_1","first-page":"1","article-title":"Grass: Generative recursive autoencoders for shape structures","volume":"36","author":"Li Jun","year":"2017","unstructured":"Jun Li , Kai Xu , Siddhartha Chaudhuri , Ersin Yumer , Hao Zhang , and Leonidas Guibas . 2017 . Grass: Generative recursive autoencoders for shape structures . ACM Transactions on Graphics (TOG) 36 , 4 (2017), 1 -- 14 . Jun Li, Kai Xu, Siddhartha Chaudhuri, Ersin Yumer, Hao Zhang, and Leonidas Guibas. 2017. Grass: Generative recursive autoencoders for shape structures. ACM Transactions on Graphics (TOG) 36, 4 (2017), 1--14.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_28_1","first-page":"1","article-title":"Grains: Generative recursive autoencoders for indoor scenes","volume":"38","author":"Li Manyi","year":"2019","unstructured":"Manyi Li , Akshay Gadi Patil , Kai Xu , Siddhartha Chaudhuri , Owais Khan , Ariel Shamir , Changhe Tu , Baoquan Chen , Daniel Cohen-Or , and Hao Zhang . 2019 . Grains: Generative recursive autoencoders for indoor scenes . ACM Transactions on Graphics (TOG) 38 , 2 (2019), 1 -- 16 . Manyi Li, Akshay Gadi Patil, Kai Xu, Siddhartha Chaudhuri, Owais Khan, Ariel Shamir, Changhe Tu, Baoquan Chen, Daniel Cohen-Or, and Hao Zhang. 2019. Grains: Generative recursive autoencoders for indoor scenes. ACM Transactions on Graphics (TOG) 38, 2 (2019), 1--16.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356527"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3305366.3328091"},{"key":"e_1_2_2_31_1","volume-title":"Computer Graphics Forum","author":"M\u00fcller Thomas","unstructured":"Thomas M\u00fcller , Markus Gross , and Jan Nov\u00e1k . 2017. Practical path guiding for efficient light-transport simulation . In Computer Graphics Forum , Vol. 36 . Wiley Online Library , 91--100. Thomas M\u00fcller, Markus Gross, and Jan Nov\u00e1k. 2017. Practical path guiding for efficient light-transport simulation. In Computer Graphics Forum, Vol. 36. Wiley Online Library, 91--100."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341156"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417804"},{"key":"e_1_2_2_34_1","volume-title":"Austin Robison, et al","author":"Parker Steven G","year":"2010","unstructured":"Steven G Parker , James Bigler , Andreas Dietrich , Heiko Friedrich , Jared Hoberock , David Luebke , David McAllister , Morgan McGuire , Keith Morley , Austin Robison, et al . 2010 . OptiX: a general purpose ray tracing engine. Acm transactions on graphics (tog) 29, 4 (2010), 1--13. Steven G Parker, James Bigler, Andreas Dietrich, Heiko Friedrich, Jared Hoberock, David Luebke, David McAllister, Morgan McGuire, Keith Morley, Austin Robison, et al. 2010. OptiX: a general purpose ray tracing engine. Acm transactions on graphics (tog) 29, 4 (2010), 1--13."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392441"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.701"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_2_38_1","volume-title":"Lensch","author":"Ruppert Lukas","year":"2020","unstructured":"Lukas Ruppert , Sebastian Herholz , and Hendrik P. A . Lensch . 2020 . Robust Fitting of Parallax-Aware Mixtures for Path Guiding. ACM Transactions on Graphics (TOG) ( 2020). Lukas Ruppert, Sebastian Herholz, and Hendrik P. A. Lensch. 2020. Robust Fitting of Parallax-Aware Mixtures for Path Guiding. ACM Transactions on Graphics (TOG) (2020)."},{"key":"e_1_2_2_39_1","volume-title":"Rendering Techniques' 95","author":"Shirley Peter","unstructured":"Peter Shirley , Bretton Wade , Philip M Hubbard , David Zareski , Bruce Walter , and Donald P Greenberg . 1995. Global illumination via density-estimation . In Rendering Techniques' 95 . Springer , 219--230. Peter Shirley, Bretton Wade, Philip M Hubbard, David Zareski, Bruce Walter, and Donald P Greenberg. 1995. Global illumination via density-estimation. In Rendering Techniques' 95. Springer, 219--230."},{"key":"e_1_2_2_40_1","unstructured":"Turbo Squid. 2020. 3D Models Plugins Textures and more at Turbo Squid.  Turbo Squid. 2020. 3D Models Plugins Textures and more at Turbo Squid."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.230"},{"key":"e_1_2_2_42_1","volume-title":"Cg trader. URL http:\/\/www.cgtrader.com 4","author":"Trader CG","year":"2020","unstructured":"CG Trader . 2020. Cg trader. URL http:\/\/www.cgtrader.com 4 ( 2020 ). CG Trader. 2020. Cg trader. URL http:\/\/www.cgtrader.com 4 (2020)."},{"key":"e_1_2_2_43_1","volume-title":"Robust Monte Carlo methods for light transport simulation","author":"Veach Eric","unstructured":"Eric Veach . 1997. Robust Monte Carlo methods for light transport simulation . Vol. 1610 . Stanford University PhD thesis. Eric Veach. 1997. Robust Monte Carlo methods for light transport simulation. Vol. 1610. Stanford University PhD thesis."},{"key":"e_1_2_2_44_1","volume-title":"Photorealistic Rendering Techniques","author":"Veach Eric","unstructured":"Eric Veach and Leonidas Guibas . 1995a. Bidirectional estimators for light transport . In Photorealistic Rendering Techniques . Springer , 145--167. Eric Veach and Leonidas Guibas. 1995a. Bidirectional estimators for light transport. In Photorealistic Rendering Techniques. Springer, 145--167."},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/218380.218498"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201388"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3305366.3328091"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601203"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925912"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073608"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW50498.2020.00141"},{"key":"e_1_2_2_52_1","article-title":"Adaptive O-CNN: A Patch-based Deep Representation of 3D Shapes","volume":"37","author":"Wang Peng-Shuai","year":"2018","unstructured":"Peng-Shuai Wang , Chun-Yu Sun , Yang Liu , and Xin Tong . 2018 . Adaptive O-CNN: A Patch-based Deep Representation of 3D Shapes . ACM Transactions on Graphics (SIGGRAPH Asia) 37 , 6 (2018). Peng-Shuai Wang, Chun-Yu Sun, Yang Liu, and Xin Tong. 2018. Adaptive O-CNN: A Patch-based Deep Representation of 3D Shapes. ACM Transactions on Graphics (SIGGRAPH Asia) 37, 6 (2018).","journal-title":"ACM Transactions on Graphics (SIGGRAPH Asia)"},{"key":"e_1_2_2_53_1","volume-title":"Computer Graphics Forum","author":"Zheng Quan","unstructured":"Quan Zheng and Matthias Zwicker . 2019. Learning to importance sample in primary sample space . In Computer Graphics Forum , Vol. 38 . Wiley Online Library , 169--179. Quan Zheng and Matthias Zwicker. 2019. Learning to importance sample in primary sample space. In Computer Graphics Forum, Vol. 38. Wiley Online Library, 169--179."},{"key":"e_1_2_2_54_1","volume-title":"Hao Su, and Ravi Ramamoorthi.","author":"Zhu Shilin","year":"2020","unstructured":"Shilin Zhu , Zexiang Xu , Henrik Wann Jensen , Hao Su, and Ravi Ramamoorthi. 2020 a. Deep Kernel Density Estimation for Photon Mapping. In Computer Graphics Forum, Vol. 39 . Wiley-Blackwell . Shilin Zhu, Zexiang Xu, Henrik Wann Jensen, Hao Su, and Ravi Ramamoorthi. 2020a. Deep Kernel Density Estimation for Photon Mapping. In Computer Graphics Forum, Vol. 39. Wiley-Blackwell."},{"key":"e_1_2_2_55_1","volume-title":"Hao Su, and Ravi Ramamoorthi.","author":"Zhu Shilin","year":"2020","unstructured":"Shilin Zhu , Zexiang Xu , Tiancheng Sun , Alexandr Kuznetsov , Mark Meyer , Henrik Wann Jensen , Hao Su, and Ravi Ramamoorthi. 2020 b. Photon-Driven Neural Path Guiding . arXiv preprint arXiv:2010.01775 (2020). Shilin Zhu, Zexiang Xu, Tiancheng Sun, Alexandr Kuznetsov, Mark Meyer, Henrik Wann Jensen, Hao Su, and Ravi Ramamoorthi. 2020b. Photon-Driven Neural Path Guiding. arXiv preprint arXiv:2010.01775 (2020)."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3450626.3459810","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3450626.3459810","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3450626.3459810","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:20Z","timestamp":1750191440000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3450626.3459810"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,19]]},"references-count":55,"aliases":["10.1145\/3476576.3476578"],"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,8,31]]}},"alternative-id":["10.1145\/3450626.3459810"],"URL":"https:\/\/doi.org\/10.1145\/3450626.3459810","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,19]]},"assertion":[{"value":"2021-07-19","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}