{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,10]],"date-time":"2026-06-10T10:10:39Z","timestamp":1781086239723,"version":"3.54.1"},"reference-count":44,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,6,23]],"date-time":"2024-06-23T00:00:00Z","timestamp":1719100800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2024,6,30]]},"abstract":"<jats:p>We present a complete system for real-time rendering of scenes with complex appearance previously reserved for offline use. This is achieved with a combination of algorithmic and system level innovations.<\/jats:p>\n          <jats:p\/>\n          <jats:p>Our appearance model utilizes learned hierarchical textures that are interpreted using neural decoders, which produce reflectance values and importance-sampled directions. To best utilize the modeling capacity of the decoders, we equip the decoders with two graphics priors. The first prior\u2014transformation of directions into learned shading frames\u2014facilitates accurate reconstruction of mesoscale effects. The second prior\u2014a microfacet sampling distribution\u2014allows the neural decoder to perform importance sampling efficiently. The resulting appearance model supports anisotropic sampling and level-of-detail rendering, and allows baking deeply layered material graphs into a compact unified neural representation.<\/jats:p>\n          <jats:p>By exposing hardware accelerated tensor operations to ray tracing shaders, we show that it is possible to inline and execute the neural decoders efficiently inside a real-time path tracer. We analyze scalability with increasing number of neural materials and propose to improve performance using code optimized for coherent and divergent execution. Our neural material shaders can be over an order of magnitude faster than non-neural layered materials. This opens up the door for using film-quality visuals in real-time applications such as games and live previews.<\/jats:p>","DOI":"10.1145\/3659577","type":"journal-article","created":{"date-parts":[[2024,4,20]],"date-time":"2024-04-20T10:23:54Z","timestamp":1713608634000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":28,"title":["Real-time Neural Appearance Models"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6434-465X","authenticated-orcid":false,"given":"Tizian","family":"Zeltner","sequence":"first","affiliation":[{"name":"NVIDIA, Z\u00fcrich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-2978-2130","authenticated-orcid":false,"given":"Fabrice","family":"Rousselle","sequence":"additional","affiliation":[{"name":"NVIDIA, Z\u00fcrich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4146-187X","authenticated-orcid":false,"given":"Andrea","family":"Weidlich","sequence":"additional","affiliation":[{"name":"NVIDIA, Montreal, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-9870-6307","authenticated-orcid":false,"given":"Petrik","family":"Clarberg","sequence":"additional","affiliation":[{"name":"NVIDIA, Lund, Sweden"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8320-9584","authenticated-orcid":false,"given":"Jan","family":"Nov\u00e1k","sequence":"additional","affiliation":[{"name":"NVIDIA, Prague, Czech Republic"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8799-7119","authenticated-orcid":false,"given":"Benedikt","family":"Bitterli","sequence":"additional","affiliation":[{"name":"NVIDIA, Redmond, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7586-1735","authenticated-orcid":false,"given":"Alex","family":"Evans","sequence":"additional","affiliation":[{"name":"NVIDIA, London, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-5793-9287","authenticated-orcid":false,"given":"Tom\u00e1\u0161","family":"Davidovi\u010d","sequence":"additional","affiliation":[{"name":"NVIDIA, Prague, Czech Republic"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-3515-7309","authenticated-orcid":false,"given":"Simon","family":"Kallweit","sequence":"additional","affiliation":[{"name":"NVIDIA, Z\u00fcrich, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-6526-0922","authenticated-orcid":false,"given":"Aaron","family":"Lefohn","sequence":"additional","affiliation":[{"name":"NVIDIA, Redmond, United States"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,6,23]]},"reference":[{"issue":"1","key":"e_1_3_4_2_1","first-page":"1","article-title":"Improved shader and texture level of detail using ray cones","volume":"10","author":"Akenine-M\u00f6ller Tomas","year":"2021","unstructured":"Tomas Akenine-M\u00f6ller, Cyril Crassin, Jakub Boksansky, Laurent Belcour, Alexey Panteleev, and Oli Wright. 2021. Improved shader and texture level of detail using ray cones. Journal of Computer Graphics Techniques 10, 1(2021), 1\u201324. Retrieved from http:\/\/jcgt.org\/published\/0010\/01\/01\/","journal-title":"Journal of Computer Graphics Techniques"},{"key":"e_1_3_4_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3406183"},{"key":"e_1_3_4_4_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14449"},{"key":"e_1_3_4_5_1","doi-asserted-by":"publisher","unstructured":"Yaoyi Bai Songyin Wu Zheng Zeng Beibei Wang and Ling-Qi Yan. 2022. BSDF Importance Baking: A Lightweight Neural Solution to Importance Sampling Parametric BSDFs. DOI:10.48550\/ARXIV.2210.13681","DOI":"10.48550\/ARXIV.2210.13681"},{"key":"e_1_3_4_6_1","doi-asserted-by":"publisher","unstructured":"Steve Bako Pradeep Sen and Anton Kaplanyan. 2023. Deep appearance prefiltering. ACM Transactions on Graphics 42 2 Article 23 (Jan 2023) 23 pages. 10.1145\/3570327","DOI":"10.1145\/3570327"},{"key":"e_1_3_4_7_1","article-title":"Research Advances Toward Real-Time Path Tracing","author":"Clarberg Petrik","year":"2022","unstructured":"Petrik Clarberg, Simon Kallweit, Craig Kolb, Pawel Kozlowski, Yong He, Lifan Wu, and Edward Liu. 2022a. Research Advances Toward Real-Time Path Tracing. Game Developers Conference (GDC).","journal-title":"Game Developers Conference (GDC)"},{"key":"e_1_3_4_8_1","article-title":"Real-Time Path Tracing and Beyond","author":"Clarberg Petrik","year":"2022","unstructured":"Petrik Clarberg, Simon Kallweit, Craig Kolb, Pawel Kozlowski, Yong He, Lifan Wu, Edward Liu, Benedikt Bitterli, and Matt Pharr. 2022b. Real-Time Path Tracing and Beyond. HPG 2022 Keynote.","journal-title":"HPG 2022 Keynote"},{"key":"e_1_3_4_9_1","unstructured":"Laurent Dinh Jascha Sohl-Dickstein and Samy Bengio. 2017. Density estimation using Real NVP. In Proceedings of the International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=HkpbnH9lx"},{"key":"e_1_3_4_10_1","volume-title":"Photorealistic Surface Rendering with Microfacet Theory","author":"Dupuy Jonathan","year":"2015","unstructured":"Jonathan Dupuy. 2015. Photorealistic Surface Rendering with Microfacet Theory. Ph. D. Dissertation. Universit\u00e9 Claude Bernard - Lyon I ; Universit\u00e9 de Montr\u00e9al."},{"key":"e_1_3_4_11_1","doi-asserted-by":"publisher","unstructured":"Jonathan Dupuy Eric Heitz Jean-Claude Iehl Pierre Poulin Fabrice Neyret and Victor Ostromoukhov. 2013. Linear efficient antialiased displacement and reflectance mapping. ACM Transactions on Graphics 32 6 Article 211 (2013) 11 pages. 10.1145\/2508363.2508422","DOI":"10.1145\/2508363.2508422"},{"key":"e_1_3_4_12_1","doi-asserted-by":"publisher","unstructured":"Jonathan Dupuy and Wenzel Jakob. 2018. An adaptive parameterization for efficient material acquisition and rendering. ACM Transactions on Graphics 37 6 Article 274 (2018) 14 pages. 10.1145\/3272127.3275059","DOI":"10.1145\/3272127.3275059"},{"key":"e_1_3_4_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528233.3530732"},{"key":"e_1_3_4_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3550454.3555487"},{"key":"e_1_3_4_15_1","doi-asserted-by":"publisher","unstructured":"Yong He Kayvon Fatahalian and Theresa Foley. 2018. Slang: Language mechanisms for extensible real-time shading systems. ACM Transactions on Graphics 37 4 Article 141 (2018) 13 pages. 10.1145\/3197517.3201380","DOI":"10.1145\/3197517.3201380"},{"key":"e_1_3_4_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3305366.3328085"},{"key":"e_1_3_4_17_1","unstructured":"Simon Kallweit Petrik Clarberg Craig Kolb Tom\u00e1\u0161 Davidovi\u010d Kai-Hwa Yao Theresa Foley Yong He Lifan Wu Lucy Chen Tomas Akenine-M\u00f6ller Chris Wyman Cyril Crassin and Nir Benty. 2022. The Falcor Rendering Framework (version 5.2). Retrieved 01-May-2024 from https:\/\/github.com\/NVIDIAGameWorks\/Falcor"},{"key":"e_1_3_4_18_1","doi-asserted-by":"publisher","unstructured":"Alexandr Kuznetsov Milo\u0161 Ha\u0161an Zexiang Xu Ling-Qi Yan Bruce Walter Nima Khademi Kalantari Steve Marschner and Ravi Ramamoorthi. 2019. Learning generative models for rendering specular microgeometry. ACM Transactions on Graphics 38 6 Article 225 (2019) 14 pages. 10.1145\/3355089.3356525","DOI":"10.1145\/3355089.3356525"},{"key":"e_1_3_4_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459795"},{"key":"e_1_3_4_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528233.3530721"},{"key":"e_1_3_4_21_1","doi-asserted-by":"publisher","unstructured":"Samuli Laine Tero Karras and Timo Aila. 2013. Megakernels considered harmful: Wavefront path tracing on GPUs. In Proceedings of the 5th High-Performance Graphics Conference (Anaheim California). Association for Computing Machinery New York NY USA 137\u2013143. 10.1145\/2492045.2492060","DOI":"10.1145\/2492045.2492060"},{"key":"e_1_3_4_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882343"},{"key":"e_1_3_4_23_1","doi-asserted-by":"crossref","unstructured":"Ben Mildenhall Pratul P. Srinivasan Matthew Tancik Jonathan T. Barron Ravi Ramamoorthi and Ren Ng. 2020. NeRF: Representing scenes as neural radiance fields for view synthesis. In Computer Vision \u2013 ECCV 2020. Springer International Publishing Cham 405\u2013421.","DOI":"10.1007\/978-3-030-58452-8_24"},{"key":"e_1_3_4_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530127"},{"key":"e_1_3_4_25_1","doi-asserted-by":"publisher","unstructured":"Thomas M\u00fcller Brian McWilliams Fabrice Rousselle Markus Gross and Jan Nov\u00e1k. 2019. Neural importance sampling. ACM Transactions on Graphics 38 5 Article 145 (2019) 19 pages. 10.1145\/3341156","DOI":"10.1145\/3341156"},{"key":"e_1_3_4_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417804"},{"key":"e_1_3_4_27_1","doi-asserted-by":"publisher","unstructured":"Oliver Nalbach Elena Arabadzhiyska Dushyant Mehta Hans-Peter Seidel and Tobias Ritschel. 2017. Deep shading: Convolutional neural networks for screen space shading. Computer Graphics Forum 36 4 (2017) 65\u201378. 10.1111\/cgf.13225","DOI":"10.1111\/cgf.13225"},{"key":"e_1_3_4_28_1","doi-asserted-by":"publisher","unstructured":"Marc Olano and Dan Baker. 2010. LEAN mapping. In Proceedings of the 2010 ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games (Washington D.C.). Association for Computing Machinery New York NY USA 181\u2013188. 10.1145\/1730804.1730834","DOI":"10.1145\/1730804.1730834"},{"key":"e_1_3_4_29_1","volume-title":"Physically Based Rendering, Third Edition: From Theory to Implementation","author":"Pharr Matt","year":"2016","unstructured":"Matt Pharr, Wenzel Jakob, and Greg Humphreys. 2016. Physically Based Rendering, Third Edition: From Theory to Implementation. Morgan Kaufmann."},{"key":"e_1_3_4_30_1","doi-asserted-by":"publisher","unstructured":"Gilles Rainer Abhijeet Ghosh Wenzel Jakob and Tim Weyrich. 2020. Unified neural encoding of BTFs. Computer Graphics Forum 39 2 (2020) 167\u2013178. 10.1111\/cgf.13921","DOI":"10.1111\/cgf.13921"},{"key":"e_1_3_4_31_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13633"},{"key":"e_1_3_4_32_1","unstructured":"Daniel Rebain Mark J. Matthews Kwang Moo Yi Gopal Sharma Dmitry Lagun and Andrea Tagliasacchi. 2023. Attention beats concatenation for conditioning neural fields. Transactions on Machine Learning Research (2023). https:\/\/openreview.net\/forum?id=GzqdMrFQsE"},{"key":"e_1_3_4_33_1","first-page":"11","volume-title":"Proceedings of the Eurographics Workshop on Rendering Techniques","author":"Rusinkiewicz Szymon M.","year":"1998","unstructured":"Szymon M. Rusinkiewicz. 1998. A new change of variables for efficient BRDF representation. In Proceedings of the Eurographics Workshop on Rendering Techniques. Springer, Springer Vienna, Vienna, 11\u201322."},{"key":"e_1_3_4_34_1","unstructured":"Doug Smythe and Jonathan Stone. 2021. MaterialX: An Open Standard for Network-Based CG Object Looks Version 1.38. Retrieved 01-May-2024 from https:\/\/materialx.org\/assets\/MaterialX.v1.38.Spec.pdf"},{"key":"e_1_3_4_35_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14335"},{"key":"e_1_3_4_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323035"},{"key":"e_1_3_4_37_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSA.65.000531"},{"key":"e_1_3_4_38_1","doi-asserted-by":"publisher","unstructured":"Karthik Vaidyanathan Marco Salvi Bartlomiej Wronski Tomas Akenine-M\u00f6ller Pontus Ebelin and Aaron Lefohn. 2023. Random-access neural compression of material textures. ACM Transactions on Graphics 42 4 Article 88 (2023) 25 pages. 10.1145\/3592407","DOI":"10.1145\/3592407"},{"key":"e_1_3_4_39_1","doi-asserted-by":"publisher","unstructured":"Dietger van Antwerpen. 2011. Improving SIMD efficiency for parallel Monte Carlo light transport on the GPU. In Proceedings of the ACM SIGGRAPH Symposium on High Performance Graphics (Vancouver British Columbia Canada). Association for Computing Machinery New York NY USA 41\u201350. 10.1145\/2018323.2018330","DOI":"10.1145\/2018323.2018330"},{"key":"e_1_3_4_40_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems Vol. 30. Curran Associates Inc."},{"key":"e_1_3_4_41_1","doi-asserted-by":"publisher","unstructured":"Eric Veach and Leonidas J. Guibas. 1995. Optimally combining sampling techniques for Monte Carlo rendering. In Proceedings of the 22nd Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH\u201995). Association for Computing Machinery New York NY USA 419\u2013428. 10.1145\/218380.218498","DOI":"10.1145\/218380.218498"},{"key":"e_1_3_4_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201388"},{"key":"e_1_3_4_43_1","unstructured":"Bruce Walter Stephen R. Marschner Hongsong Li and Kenneth E. Torrance. 2007. Microfacet models for refraction through rough surfaces. In Proceedings of the 18th Eurographics Conference on Rendering Techniques (Grenoble France) (EGSR\u201907). Eurographics Association Goslar DEU 195\u2013206."},{"key":"e_1_3_4_44_1","doi-asserted-by":"publisher","unstructured":"Bing Xu Liwen Wu Milo\u0161 Ha\u0161an Fujun Luan Iliyan Georgiev Zexiang Xu and Ravi Ramamoorthi. 2023. NeuSample: Importance sampling for neural materials. In ACM SIGGRAPH 2023 Conference Proceedings (Los Angeles CA USA). Association for Computing Machinery New York NY USA Article 41 (2023) 10 pages. 10.1145\/3588432.3591524","DOI":"10.1145\/3588432.3591524"},{"key":"e_1_3_4_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3490385"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3659577","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3659577","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:42Z","timestamp":1750291422000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3659577"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,23]]},"references-count":44,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,6,30]]}},"alternative-id":["10.1145\/3659577"],"URL":"https:\/\/doi.org\/10.1145\/3659577","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,23]]},"assertion":[{"value":"2023-06-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-03-11","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}