{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T18:21:35Z","timestamp":1773771695665,"version":"3.50.1"},"reference-count":40,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2020,8,12]],"date-time":"2020-08-12T00:00:00Z","timestamp":1597190400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2020,8,31]]},"abstract":"<jats:p>We present a technique for adaptively partitioning neural scene representations. Our method disentangles lighting, material, and geometric information yielding a scene representation that preserves the orthogonality of these components, improves interpretability of the model, and allows compositing new scenes by mixing components of existing ones. The proposed adaptive partitioning respects the uneven entropy of individual components and permits compressing the scene representation to lower its memory footprint and potentially reduce the evaluation cost of the model. Furthermore, the partitioned representation enables an in-depth analysis of existing image generators. We compare the flow of information through individual partitions, and by contrasting it to the impact of additional inputs (G-buffer), we are able to identify the roots of undesired visual artifacts, and propose one possible solution to remedy the poor performance. We also demonstrate the benefits of complementing traditional forward renderers by neural representations and synthesis, e.g. to infer expensive shading effects, and show how these could improve production rendering in the future if developed further.<\/jats:p>","DOI":"10.1145\/3386569.3392475","type":"journal-article","created":{"date-parts":[[2020,8,12]],"date-time":"2020-08-12T11:44:27Z","timestamp":1597232667000},"update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":20,"title":["Compositional neural scene representations for shading inference"],"prefix":"10.1145","volume":"39","author":[{"given":"Jonathan","family":"Granskog","sequence":"first","affiliation":[{"name":"ETH Zurich &amp; NVIDIA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fabrice","family":"Rousselle","sequence":"additional","affiliation":[{"name":"NVIDIA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marios","family":"Papas","sequence":"additional","affiliation":[{"name":"DisneyResearch|Studios"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jan","family":"Nov\u00e1k","sequence":"additional","affiliation":[{"name":"NVIDIA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2020,8,12]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"International Conference on Learning Representations.","author":"Ancona Marco","year":"2018","unstructured":"Marco Ancona, Enea Ceolini, Cengiz \u00d6ztireli, and Markus Gross. 2018. Towards better understanding of gradient-based attribution methods for Deep Neural Networks. In International Conference on Learning Representations."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.50"},{"key":"e_1_2_2_3_1","volume-title":"Advances in Neural Information Processing Systems 29. Curran Associates","author":"Chen Xi","unstructured":"Xi Chen, Yan Duan, Rein Houthooft, John Schulman, Ilya Sutskever, and Pieter Abbeel. 2016. InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets. In Advances in Neural Information Processing Systems 29. Curran Associates, Inc., 2172--2180."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/280814.280864"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344855"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13765"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359786"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","unstructured":"S. M. Ali Eslami Danilo Jimenez Rezende Frederic Besse Fabio Viola Ari S. Morcos Marta Garnelo Avraham Ruderman Andrei A. Rusu Ivo Danihelka Karol Gregor David P. Reichert Lars Buesing Theophane Weber Oriol Vinyals Dan Rosenbaum Neil Rabinowitz Helen King Chloe Hillier Matt Botvinick Daan Wierstra Koray Kavukcuoglu and Demis Hassabis. 2018. Neural scene representation and rendering. Science 360 6394 (2018) 1204--1210. 10.1126\/science.aar6170","DOI":"10.1126\/science.aar6170"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13783"},{"key":"e_1_2_2_10_1","volume-title":"Proc. ICLR.","author":"Higgins Irina","year":"2017","unstructured":"Irina Higgins, Lo\u00efc Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew M Botvinick, Shakir Mohamed, and Alexander Lerchner. 2017. beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework. In Proc. ICLR."},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130880"},{"key":"e_1_2_2_12_1","volume-title":"International Conference on Learning Representations.","author":"Karras Tero","year":"2018","unstructured":"Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive Growing of GANs for Improved Quality, Stability, and Variation. In International Conference on Learning Representations."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201283"},{"key":"e_1_2_2_14_1","volume-title":"Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations.","author":"Diederik","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In International Conference on Learning Representations."},{"key":"e_1_2_2_15_1","volume-title":"Advances in Neural Information Processing Systems 28. Curran Associates","author":"Kulkarni Tejas D","unstructured":"Tejas D Kulkarni, William F. Whitney, Pushmeet Kohli, and Josh Tenenbaum. 2015. Deep Convolutional Inverse Graphics Network. In Advances in Neural Information Processing Systems 28. Curran Associates, Inc., 2539--2547."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323020"},{"key":"e_1_2_2_17_1","doi-asserted-by":"crossref","unstructured":"Wojciech Matusik. 2003. A data-driven reflectance model. Ph.D. Dissertation. Massachusetts Institute of Technology.","DOI":"10.1145\/1201775.882343"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341156"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13225"},{"key":"e_1_2_2_20_1","volume-title":"Image Analysis","author":"Nguyen-Ha Phong","unstructured":"Phong Nguyen-Ha, Lam Huynh, Esa Rahtu, and Janne Heikkil\u00e4. 2019. Predicting Novel Views Using Generative Adversarial Query Network. In Image Analysis. Springer International Publishing, Cham, 16--27."},{"key":"e_1_2_2_21_1","volume-title":"The IEEE International Conference on Computer Vision (ICCV).","author":"Nguyen-Phuoc Thu","year":"2019","unstructured":"Thu Nguyen-Phuoc, Chuan Li, Lucas Theis, Christian Richardt, and Yong-Liang Yang. 2019. HoloGAN: Unsupervised Learning of 3D Representations From Natural Images. In The IEEE International Conference on Computer Vision (ICCV)."},{"key":"e_1_2_2_22_1","volume-title":"Advances in Neural Information Processing Systems 31. Curran Associates","author":"Nguyen-Phuoc Thu H","unstructured":"Thu H Nguyen-Phuoc, Chuan Li, Stephen Balaban, and Yongliang Yang. 2018. RenderNet: A deep convolutional network for differentiable rendering from 3D shapes. In Advances in Neural Information Processing Systems 31. Curran Associates, Inc., 7891--7901."},{"key":"e_1_2_2_23_1","unstructured":"Weili Nie Tero Karras Animesh Garg Shoubhik Debhath Anjul Patney Ankit B. Patel and Anima Anandkumar. 2020. Semi-Supervised StyleGAN for Disentanglement Learning. arXiv:cs.CV\/2003.03461"},{"key":"e_1_2_2_24_1","volume-title":"Transformable Bottleneck Networks. The IEEE International Conference on Computer Vision (ICCV) (Nov","author":"Olszewski Kyle","year":"2019","unstructured":"Kyle Olszewski, Sergey Tulyakov, Oliver Woodford, Hao Li, and Linjie Luo. 2019. Transformable Bottleneck Networks. The IEEE International Conference on Computer Vision (ICCV) (Nov 2019)."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323013"},{"key":"e_1_2_2_26_1","volume-title":"PieAPP: Perceptual Image-Error Assessment Through Pairwise Preference. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Prashnani Ekta","year":"2018","unstructured":"Ekta Prashnani, Hong Cai, Yasamin Mostofi, and Pradeep Sen. 2018. PieAPP: Perceptual Image-Error Assessment Through Pairwise Preference. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_27_1","doi-asserted-by":"crossref","unstructured":"Konstantinos Rematas and Vittorio Ferrari. 2019. Neural Voxel Renderer: Learning an Accurate and Controllable Rendering Tool. arXiv:cs.CV\/1912.04591","DOI":"10.1109\/CVPR42600.2020.00546"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2462009"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_2_30_1","unstructured":"Dan Rosenbaum Frederic Besse Fabio Viola Danilo J. Rezende and S. M. Ali Eslami. 2018. Learning models for visual 3D localization with implicit mapping. arXiv:cs.CV\/1807.03149"},{"key":"e_1_2_2_31_1","volume-title":"Proceedings of the 34th International Conference on Machine Learning","volume":"70","author":"Shrikumar Avanti","year":"2017","unstructured":"Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning Important Features Through Propagating Activation Differences. In Proceedings of the 34th International Conference on Machine Learning, Vol. 70. PMLR, Sydney, Australia, 3145--3153."},{"key":"e_1_2_2_32_1","volume-title":"Advances in Neural Information Processing Systems 32. Curran Associates","author":"Sitzmann Vincent","unstructured":"Vincent Sitzmann, Michael Zollh\u00f6fer, and Gordon Wetzstein. 2019. Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 1119--1130."},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46478-7_20"},{"key":"e_1_2_2_34_1","doi-asserted-by":"crossref","unstructured":"Ayush Tewari Ohad Fried Justus Thies Vincent Sitzmann Stephen Lombardi Kalyan Sunkavalli Ricardo Martin-Brualla Tomas Simon Jason Saragih Matthias Nie\u00dfner Rohit Pandey Sean Fanello Gordon Wetzstein Jun-Yan Zhu Christian Theobalt Maneesh Agrawala Eli Shechtman Dan B Goldman and Michael Zollh\u00f6fer. 2020. State of the Art on Neural Rendering. arXiv:cs.CV\/2004.03805","DOI":"10.1111\/cgf.14022"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323035"},{"key":"e_1_2_2_36_1","volume-title":"Advances in Neural Information Processing Systems 32. Curran Associates","author":"Tobin Joshua","unstructured":"Joshua Tobin, Wojciech Zaremba, and Pieter Abbeel. 2019. Geometry-Aware Neural Rendering. In Advances in Neural Information Processing Systems 32. Curran Associates, Inc., 11555--11565."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322974"},{"key":"e_1_2_2_38_1","volume-title":"Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Wagner J\u00f6rg","year":"2019","unstructured":"J\u00f6rg Wagner, Jan Mathias K\u00f6hler, Tobias Gindele, Leon Hetzel, Jakob Thadd\u00e4us Wiedemer, and Sven Behnke. 2019. Interpretable and Fine-Grained Visual Explanations for Convolutional Neural Networks. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_39_1","volume-title":"Zeiler and Rob Fergus","author":"Matthew","year":"2014","unstructured":"Matthew D. Zeiler and Rob Fergus. 2014. Visualizing and Understanding Convolutional Networks. In Computer Vision - ECCV 2014. Springer International Publishing, Cham, 818--833."},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13628"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3386569.3392475","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3386569.3392475","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T05:39:23Z","timestamp":1750829963000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3386569.3392475"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,8,12]]},"references-count":40,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2020,8,31]]}},"alternative-id":["10.1145\/3386569.3392475"],"URL":"https:\/\/doi.org\/10.1145\/3386569.3392475","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,8,12]]},"assertion":[{"value":"2020-08-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}