{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,5]],"date-time":"2025-12-05T12:23:59Z","timestamp":1764937439478,"version":"3.41.0"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2022,7,25]],"date-time":"2022-07-25T00:00:00Z","timestamp":1658707200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Comput. Graph. Interact. Tech."],"published-print":{"date-parts":[[2022,7,25]]},"abstract":"<jats:p>Recent advances in ray tracing hardware bring real-time path tracing into reach, and ray traced soft shadows, glossy reflections, and diffuse global illumination are now common features in games. Nonetheless, ray budgets are still limited. This results in undersampling, which manifests as aliasing and noise. Prior work addresses these issues separately. While temporal supersampling methods based on neural networks have gained a wide use in modern games due to their better robustness, neural denoising remains challenging because of its higher computational cost.<\/jats:p>\n          <jats:p>We introduce a novel neural network architecture for real-time rendering that combines supersampling and denoising, thus lowering the cost compared to two separate networks. This is achieved by sharing a single low-precision feature extractor with multiple higher-precision filter stages. To reduce cost further, our network takes low-resolution inputs and reconstructs a high-resolution denoised supersampled output. Our technique produces temporally stable high-fidelity results that significantly outperform state-of-the-art real-time statistical or analytical denoisers combined with TAA or neural upsampling to the target resolution.<\/jats:p>","DOI":"10.1145\/3543870","type":"journal-article","created":{"date-parts":[[2022,7,27]],"date-time":"2022-07-27T23:35:58Z","timestamp":1658964958000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":15,"title":["Temporally Stable Real-Time Joint Neural Denoising and Supersampling"],"prefix":"10.1145","volume":"5","author":[{"given":"Manu Mathew","family":"Thomas","sequence":"first","affiliation":[{"name":"Intel Corporation, USA and University of California, Santa Cruz, USA"}]},{"given":"Gabor","family":"Liktor","sequence":"additional","affiliation":[{"name":"Intel Corporation, USA"}]},{"given":"Christoph","family":"Peters","sequence":"additional","affiliation":[{"name":"Intel Corporation, USA"}]},{"given":"Sungye","family":"Kim","sequence":"additional","affiliation":[{"name":"Intel Corporation, USA"}]},{"given":"Karthik","family":"Vaidyanathan","sequence":"additional","affiliation":[{"name":"Intel Corporation (now NVIDIA), USA"}]},{"given":"Angus G.","family":"Forbes","sequence":"additional","affiliation":[{"name":"University of California, Santa Cruz, USA"}]}],"member":"320","published-online":{"date-parts":[[2022,7,27]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073708"},{"key":"e_1_2_2_2_1","article-title":"Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder","volume":"36","author":"Alla Chaitanya Chakravarty R.","year":"2017","unstructured":"Chakravarty R. Alla Chaitanya , Anton S. Kaplanyan , Christoph Schied , Marco Salvi , Aaron Lefohn , Derek Nowrouzezahrai , and Timo Aila . 2017 . Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder . ACM Trans. Graph. 36 , 4, Article 98 (jul 2017), 12 pages. https:\/\/doi.org\/10.1145\/3072959.3073601 Chakravarty R. Alla Chaitanya, Anton S. Kaplanyan, Christoph Schied, Marco Salvi, Aaron Lefohn, Derek Nowrouzezahrai, and Timo Aila. 2017. Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder. ACM Trans. Graph. 36, 4, Article 98 (jul 2017), 12 pages. https:\/\/doi.org\/10.1145\/3072959.3073601","journal-title":"ACM Trans. Graph."},{"key":"e_1_2_2_3_1","volume-title":"Game Developers Conference.","author":"Chowdhury Hisham","year":"2022","unstructured":"Hisham Chowdhury , Kawiak, Rense Robert , de Boer, Gabriel Ferreira , and Lucas Xavier . 2022 . Intel XeSS - an AI based Super Sampling solution for Real-time Rendering. (2022) . In Game Developers Conference. Hisham Chowdhury, Kawiak, Rense Robert, de Boer, Gabriel Ferreira, and Lucas Xavier. 2022. Intel XeSS - an AI based Super Sampling solution for Real-time Rendering. (2022). In Game Developers Conference."},{"key":"e_1_2_2_4_1","volume-title":"Multi-task learning with deep neural networks: A survey. arXiv preprint arXiv:2009.09796","author":"Crawshaw Michael","year":"2020","unstructured":"Michael Crawshaw . 2020. Multi-task learning with deep neural networks: A survey. arXiv preprint arXiv:2009.09796 ( 2020 ). Michael Crawshaw. 2020. Multi-task learning with deep neural networks: A survey. arXiv preprint arXiv:2009.09796 (2020)."},{"volume-title":"Proceedings of the Conference on High Performance Graphics","author":"Dammertz Holger","key":"e_1_2_2_5_1","unstructured":"Holger Dammertz , Daniel Sewtz , Johannes Hanika , and Hendrik P. A. Lensch . 2010. Edge-Avoiding \u00c0-Trous Wavelet Transform for Fast Global Illumination Filtering . In Proceedings of the Conference on High Performance Graphics ( Saarbrucken, Germany) (HPG '10). Eurographics Association, 67--75. Holger Dammertz, Daniel Sewtz, Johannes Hanika, and Hendrik P. A. Lensch. 2010. Edge-Avoiding \u00c0-Trous Wavelet Transform for Fast Global Illumination Filtering. In Proceedings of the Conference on High Performance Graphics (Saarbrucken, Germany) (HPG '10). Eurographics Association, 67--75."},{"key":"e_1_2_2_6_1","volume-title":"Rui Wang, Yuchi Huo, and Hujun Bao.","author":"Games Epic","year":"2021","unstructured":"Epic Games . 2018. Unreal Engine 4.19 : Screen percentage with temporal upsample. https:\/\/docs.unrealengine.com\/4.27\/enUS\/RenderingAndGraphics\/ScreenPercentage\/ Hangming Fan , Rui Wang, Yuchi Huo, and Hujun Bao. 2021 . Real-time Monte Carlo Denoising with Weight Sharing Kernel Prediction Network. Computer Graphics Forum ( 2021). https:\/\/doi.org\/10.1111\/cgf.14338 Epic Games. 2018. Unreal Engine 4.19: Screen percentage with temporal upsample. https:\/\/docs.unrealengine.com\/4.27\/enUS\/RenderingAndGraphics\/ScreenPercentage\/ Hangming Fan, Rui Wang, Yuchi Huo, and Hujun Bao. 2021. Real-time Monte Carlo Denoising with Weight Sharing Kernel Prediction Network. Computer Graphics Forum (2021). https:\/\/doi.org\/10.1111\/cgf.14338"},{"key":"e_1_2_2_7_1","volume-title":"ImageNet-trained CNNs are biased towards texture","author":"Geirhos Robert","year":"1811","unstructured":"Robert Geirhos , Patricia Rubisch , Claudio Michaelis , Matthias Bethge , Felix A Wichmann , and Wieland Brendel . 2018. ImageNet-trained CNNs are biased towards texture ; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv: 1811 .12231 (2018). Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A Wichmann, and Wieland Brendel. 2018. ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv preprint arXiv:1811.12231 (2018)."},{"key":"e_1_2_2_8_1","volume-title":"Neural Temporal Adaptive Sampling and Denoising. Computer Graphics Forum","author":"Hasselgren Jon","year":"2020","unstructured":"Jon Hasselgren , Jacob Munkberg , Marco Salvi , Anjul Patney , and Aaron Lefohn . 2020a. Neural Temporal Adaptive Sampling and Denoising. Computer Graphics Forum ( 2020 ). https:\/\/doi.org\/10.1111\/cgf.13919 Jon Hasselgren, Jacob Munkberg, Marco Salvi, Anjul Patney, and Aaron Lefohn. 2020a. Neural Temporal Adaptive Sampling and Denoising. Computer Graphics Forum (2020). https:\/\/doi.org\/10.1111\/cgf.13919"},{"volume-title":"Computer Graphics Forum","author":"Hasselgren Jon","key":"e_1_2_2_9_1","unstructured":"Jon Hasselgren , Jacob Munkberg , Marco Salvi , Anjul Patney , and Aaron Lefohn . 2020b. Neural temporal adaptive sampling and denoising . In Computer Graphics Forum , Vol. 39 . Wiley Online Library , 147--155. Jon Hasselgren, Jacob Munkberg, Marco Salvi, Anjul Patney, and Aaron Lefohn. 2020b. Neural temporal adaptive sampling and denoising. In Computer Graphics Forum, Vol. 39. Wiley Online Library, 147--155."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459793"},{"key":"e_1_2_2_11_1","unstructured":"Intel Corporation. 2019. Intel Open Image Denoise. https:\/\/www.openimagedenoise.org\/  Intel Corporation. 2019. Intel Open Image Denoise. https:\/\/www.openimagedenoise.org\/"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46475-6_43"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766977"},{"key":"e_1_2_2_14_1","unstructured":"Brian Karis. 2014. High quality temporal supersampling. (2014). In ACM SIGGRAPH Courses: Advances in RealTime Rendering in Games.  Brian Karis. 2014. High quality temporal supersampling. (2014). In ACM SIGGRAPH Courses: Advances in RealTime Rendering in Games."},{"key":"e_1_2_2_15_1","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition. 7482--7491","author":"Kendall Alex","year":"2018","unstructured":"Alex Kendall , Yarin Gal , and Roberto Cipolla . 2018 . Multi-task learning using uncertainty to weigh losses for scene geometry and semantics . In Proceedings of the IEEE conference on computer vision and pattern recognition. 7482--7491 . Alex Kendall, Yarin Gal, and Roberto Cipolla. 2018. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In Proceedings of the IEEE conference on computer vision and pattern recognition. 7482--7491."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00656"},{"key":"e_1_2_2_17_1","volume-title":"Game Developers Conference.","author":"Liu Edward","year":"2020","unstructured":"Edward Liu . 2020 . DLSS 2.0 - Image Reconstruction for Real-Time Rendering with Deep learning. (2020) . In Game Developers Conference. Edward Liu. 2020. DLSS 2.0 - Image Reconstruction for Real-Time Rendering with Deep learning. (2020). In Game Developers Conference."},{"key":"e_1_2_2_18_1","doi-asserted-by":"crossref","unstructured":"Edward Liu Ignacio Llamas Juan Ca\u00f1ada and Patrick Kelly. 2019. Cinematic Rendering in UE4 with Real-Time Ray Tracing and Denoising. In Ray Tracing Gems Eric Haines and Tomas Akenine-M\u00f6ller (Eds.). Apress Media 289--319. https:\/\/doi.org\/10.1007\/978-1-4842-4427-2  Edward Liu Ignacio Llamas Juan Ca\u00f1ada and Patrick Kelly. 2019. Cinematic Rendering in UE4 with Real-Time Ray Tracing and Denoising. In Ray Tracing Gems Eric Haines and Tomas Akenine-M\u00f6ller (Eds.). Apress Media 289--319. https:\/\/doi.org\/10.1007\/978-1-4842-4427-2","DOI":"10.1007\/978-1-4842-4427-2_19"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366182"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461947"},{"key":"e_1_2_2_21_1","volume-title":"Eurographics Symposium on Rendering - DL-only Track","author":"Meng Xiaoxu","year":"2020","unstructured":"Xiaoxu Meng , Quan Zheng , Amitabh Varshney , Gurprit Singh , and Matthias Zwicker . 2020. Real-time Monte Carlo Denoising with the Neural Bilateral Grid . In Eurographics Symposium on Rendering - DL-only Track , Carsten Dachsbacher and Matt Pharr (Eds.). The Eurographics Association . https:\/\/doi.org\/10.2312\/sr. 2020 1133 Xiaoxu Meng, Quan Zheng, Amitabh Varshney, Gurprit Singh, and Matthias Zwicker. 2020. Real-time Monte Carlo Denoising with the Neural Bilateral Grid. In Eurographics Symposium on Rendering - DL-only Track, Carsten Dachsbacher and Matt Pharr (Eds.). The Eurographics Association. https:\/\/doi.org\/10.2312\/sr.20201133"},{"volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV). 10912--10922","author":"Roberts Mike","key":"e_1_2_2_22_1","unstructured":"Mike Roberts , Jason Ramapuram , Anurag Ranjan , Atulit Kumar , Miguel Angel Bautista , Nathan Paczan , Russ Webb , and Joshua M. Susskind . 2021. Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding . In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV). 10912--10922 . Mike Roberts, Jason Ramapuram, Anurag Ranjan, Atulit Kumar, Miguel Angel Bautista, Nathan Paczan, Russ Webb, and Joshua M. Susskind. 2021. Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV). 10912--10922."},{"key":"e_1_2_2_23_1","doi-asserted-by":"crossref","unstructured":"Olaf Ronneberger Philipp Fischer and Thomas Brox. 2015a. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv:1505.04597 [cs.CV]  Olaf Ronneberger Philipp Fischer and Thomas Brox. 2015a. U-Net: Convolutional Networks for Biomedical Image Segmentation. arXiv:1505.04597 [cs.CV]","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3105762.3105770"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3233301"},{"key":"e_1_2_2_27_1","volume-title":"Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman . 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 ( 2014 ). Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)."},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN48605.2020.9207102"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3414685.3417786"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201388"},{"key":"e_1_2_2_31_1","volume-title":"Proceedings of the European conference on computer vision (ECCV) workshops. 0--0.","author":"Wang Xintao","year":"2018","unstructured":"Xintao Wang , Ke Yu , Shixiang Wu , Jinjin Gu , Yihao Liu , Chao Dong , Yu Qiao , and Chen Change Loy . 2018 . Esrgan: Enhanced super-resolution generative adversarial networks . In Proceedings of the European conference on computer vision (ECCV) workshops. 0--0. Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. 2018. Esrgan: Enhanced super-resolution generative adversarial networks. In Proceedings of the European conference on computer vision (ECCV) workshops. 0--0."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2019.2956697"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392376"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355089.3356547"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816814"},{"key":"e_1_2_2_36_1","volume-title":"A Survey of Temporal Antialiasing Techniques. Computer Graphics Forum 39","author":"Yang Lei","year":"2020","unstructured":"Lei Yang , Shiqiu Liu , and Marco Salvi . 2020. A Survey of Temporal Antialiasing Techniques. Computer Graphics Forum 39 ( 2020 ). Lei Yang, Shiqiu Liu, and Marco Salvi. 2020. A Survey of Temporal Antialiasing Techniques. Computer Graphics Forum 39 (2020)."},{"key":"e_1_2_2_37_1","volume-title":"Temporally Reliable Motion Vectors for Real-time Ray Tracing. Computer Graphics Forum 40, 2","author":"Zeng Zheng","year":"2021","unstructured":"Zheng Zeng , Shiqiu Liu , Jinglei Yang , Lu Wang , and Ling-Qi Yan . 2021. Temporally Reliable Motion Vectors for Real-time Ray Tracing. Computer Graphics Forum 40, 2 ( 2021 ). https:\/\/doi.org\/10.1111\/cgf.142616 Zheng Zeng, Shiqiu Liu, Jinglei Yang, Lu Wang, and Ling-Qi Yan. 2021. Temporally Reliable Motion Vectors for Real-time Ray Tracing. Computer Graphics Forum 40, 2 (2021). https:\/\/doi.org\/10.1111\/cgf.142616"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2021.3070203"}],"container-title":["Proceedings of the ACM on Computer Graphics and Interactive Techniques"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3543870","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3543870","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T17:49:40Z","timestamp":1750268980000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3543870"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,7,25]]},"references-count":38,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2022,7,25]]}},"alternative-id":["10.1145\/3543870"],"URL":"https:\/\/doi.org\/10.1145\/3543870","relation":{},"ISSN":["2577-6193"],"issn-type":[{"type":"electronic","value":"2577-6193"}],"subject":[],"published":{"date-parts":[[2022,7,25]]},"assertion":[{"value":"2022-07-27","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}