{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T16:45:56Z","timestamp":1779295556704,"version":"3.51.4"},"reference-count":46,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T00:00:00Z","timestamp":1731974400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2024,12,19]]},"abstract":"<jats:p>\n            Real-time rendering has been embracing ever-demanding effects, such as ray tracing. However, rendering such effects in high resolution and high frame rate remains challenging. Frame extrapolation methods, which do not introduce additional latency as opposed to frame interpolation methods such as DLSS 3 and FSR 3, boost the frame rate by generating future frames based on previous frames. However, it is a more challenging task because of the lack of information in the disocclusion regions and complex future motions, and recent methods also have a high engine integration cost due to requiring G-buffers as input. We propose a\n            <jats:italic>G-buffer free<\/jats:italic>\n            frame extrapolation method, GFFE, with a novel heuristic framework and an efficient neural network, to plausibly generate new frames in real time without introducing additional latency. We analyze the motion of dynamic fragments and different types of disocclusions, and design the corresponding modules of the extrapolation block to handle them. After that, a light-weight shading correction network is used to correct shading and improve overall quality. GFFE achieves comparable or better results than previous interpolation and G-buffer dependent extrapolation methods, with more efficient performance and easier integration.\n          <\/jats:p>","DOI":"10.1145\/3687923","type":"journal-article","created":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T15:46:04Z","timestamp":1732031164000},"page":"1-15","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["GFFE: G-buffer Free Frame Extrapolation for Low-latency Real-time Rendering"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-9581-2506","authenticated-orcid":false,"given":"Songyin","family":"Wu","sequence":"first","affiliation":[{"name":"University of California Santa Barbara, Santa Barbara, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-6539-7103","authenticated-orcid":false,"given":"Deepak","family":"Vembar","sequence":"additional","affiliation":[{"name":"Intel Corporation, Portland, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-7496-8586","authenticated-orcid":false,"given":"Anton","family":"Sochenov","sequence":"additional","affiliation":[{"name":"Intel Corporation, Seattle, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3629-1754","authenticated-orcid":false,"given":"Selvakumar","family":"Panneer","sequence":"additional","affiliation":[{"name":"Intel Corporation, Seattle, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0219-2192","authenticated-orcid":false,"given":"Sungye","family":"Kim","sequence":"additional","affiliation":[{"name":"Intel (now AMD), Folsom, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8376-6719","authenticated-orcid":false,"given":"Anton","family":"Kaplanyan","sequence":"additional","affiliation":[{"name":"Intel Corporation, Seattle, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9379-094X","authenticated-orcid":false,"given":"Ling-Qi","family":"Yan","sequence":"additional","affiliation":[{"name":"University of California Santa Barbara, Santa Barbara, United States of America"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,11,19]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"AMD. 2021. AMD FidelityFX\u2122 Super Resolution. https:\/\/www.amd.com\/en\/technologies\/fidelityfx-super-resolution Accessed: 2023-05-23."},{"key":"e_1_2_1_2_1","unstructured":"AMD. 2022. AMD FidelityFX\u2122 Super Resolution 3. https:\/\/gpuopen.com\/fidelityfx-super-resolution-3\/ https:\/\/gpuopen.com\/fidelityfx-super-resolution-3\/ [Accessed: 2024-01-24]."},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Dmitry Andreev. 2010. Real-time frame rate up-conversion for video games: or how to get from 30 to 60 fps for\" free\". In ACM SIGGRAPH Talks. 1--1.","DOI":"10.1145\/1837026.1837047"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00382"},{"key":"e_1_2_1_5_1","volume-title":"Computer graphics forum","author":"Bowles Huw","unstructured":"Huw Bowles, Kenny Mitchell, Robert W Sumner, Jeremy Moore, and Markus Gross. 2012. Iterative image warping. In Computer graphics forum, Vol. 31. Wiley Online Library, 237--246."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478513.3480553"},{"key":"e_1_2_1_7_1","article-title":"Kernel-Based Frame Interpolation for Spatio-Temporally Adaptive Rendering. In ACM","author":"Briedis Karlis Martins","year":"2023","unstructured":"Karlis Martins Briedis, Abdelaziz Djelouah, Rapha\u00ebl Ortiz, Mark Meyer, Markus Gross, and Christopher Schroers. 2023. Kernel-Based Frame Interpolation for Spatio-Temporally Adaptive Rendering. In ACM Trans. Graph. (SIGGRAPH). 1--11.","journal-title":"Trans. Graph. (SIGGRAPH). 1--11."},{"key":"e_1_2_1_8_1","volume-title":"Proc. international conference on image processing","volume":"2","author":"Charbonnier Pierre","year":"1994","unstructured":"Pierre Charbonnier, Laure Blanc-Feraud, Gilles Aubert, and Michel Barlaud. 1994. Two deterministic half-quadratic regularization algorithms for computed imaging. In Proc. international conference on image processing, Vol. 2. 168--172."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2009.01641.x"},{"key":"e_1_2_1_10_1","unstructured":"Epic Games. 2022. Unreal Engine. https:\/\/www.unrealengine.com"},{"key":"e_1_2_1_11_1","volume-title":"Article 278","author":"Guo Jie","year":"2021","unstructured":"Jie Guo, Xihao Fu, Liqiang Lin, Hengjun Ma, Yanwen Guo, Shiqiu Liu, and Ling-Qi Yan. 2021. ExtraNet: Real-Time Extrapolated Rendering for Low-Latency Temporal Supersampling. ACM Trans. Graph., Article 278 (2021)."},{"key":"e_1_2_1_12_1","volume-title":"Computer Graphics Forum","author":"Guo Yu-Xiao","unstructured":"Yu-Xiao Guo, Guojun Chen, Yue Dong, and Xin Tong. 2022. Classifier Guided Temporal Supersampling for Real-time Rendering. In Computer Graphics Forum, Vol. 41. Wiley Online Library, 237--246."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.123"},{"key":"e_1_2_1_14_1","volume-title":"Proc. IEEE CVPR. 6121--6131","author":"Hu Xiaotao","year":"2023","unstructured":"Xiaotao Hu, Zhewei Huang, Ailin Huang, Jun Xu, and Shuchang Zhou. 2023. A dynamic multi-scale voxel flow network for video prediction. In Proc. IEEE CVPR. 6121--6131."},{"key":"e_1_2_1_15_1","volume-title":"European Conference on Comp. Vision. 624--642","author":"Huang Zhewei","year":"2022","unstructured":"Zhewei Huang, Tianyuan Zhang, Wen Heng, Boxin Shi, and Shuchang Zhou. 2022. Real-time intermediate flow estimation for video frame interpolation. In European Conference on Comp. Vision. 624--642."},{"key":"e_1_2_1_16_1","unstructured":"Intel. 2022. Intel\u00ae Arc\u2122- Xe Super Sampling. https:\/\/www.intel.com\/content\/www\/us\/en\/products\/docs\/discrete-gpus\/arc\/technology\/xess.html Accessed: 2023-05-23."},{"key":"e_1_2_1_17_1","volume-title":"Proc. IEEE CVPR. 1578--1587","author":"Jin Xin","year":"2023","unstructured":"Xin Jin, Longhai Wu, Jie Chen, Youxin Chen, Jayoon Koo, and Cheul-hee Hahm. 2023. A Unified Pyramid Recurrent Network for Video Frame Interpolation. In Proc. IEEE CVPR. 1578--1587."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073609"},{"key":"e_1_2_1_19_1","unstructured":"Simon Kallweit Petrik Clarberg Craig Kolb Tom'a\u0161 Davidovi\u010d Kai-Hwa Yao Theresa Foley Yong He Lifan Wu Lucy Chen Tomas Akenine-M\u00f6ller Chris Wyman Cyril Crassin and Nir Benty. 2022. The Falcor Rendering Framework. https:\/\/github.com\/NVIDIAGameWorks\/Falcor https:\/\/github.com\/NVIDIAGameWorks\/Falcor."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3406187"},{"key":"e_1_2_1_21_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_1_22_1","volume-title":"Proc. IEEE CVPR. 1969--1978","author":"Kong Lingtong","year":"2022","unstructured":"Lingtong Kong, Boyuan Jiang, Donghao Luo, Wenqing Chu, Xiaoming Huang, Ying Tai, Chengjie Wang, and Jie Yang. 2022. Ifrnet: Intermediate feature refine network for efficient frame interpolation. In Proc. IEEE CVPR. 1969--1978."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3190859"},{"key":"e_1_2_1_24_1","unstructured":"Zhan Li Carl S Marshall Deepak S Vembar and Feng Liu. 2022. Future Frame Synthesis for Fast Monte Carlo Rendering. In Graph. Interface."},{"key":"e_1_2_1_25_1","volume-title":"Nvidia GPU Tech. Conf. (GTC).","author":"Liu Edward","year":"2020","unstructured":"Edward Liu. 2020. DLSS 2.0-Image reconstruction for real-time rendering with deep learning. In Nvidia GPU Tech. Conf. (GTC)."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459831"},{"key":"e_1_2_1_27_1","volume-title":"Proc. Symp. Interactive 3D Graphics. 7--ff.","author":"Mark William R","year":"1997","unstructured":"William R Mark, Leonard McMillan, and Gary Bishop. 1997. Post-rendering 3D warping. In Proc. Symp. Interactive 3D Graphics. 7--ff."},{"key":"e_1_2_1_28_1","volume-title":"Proc. AAAI conference on artificial intelligence","volume":"32","author":"Meister Simon","year":"2018","unstructured":"Simon Meister, Junhwa Hur, and Stefan Roth. 2018. Unflow: Unsupervised learning of optical flow with a bidirectional census loss. In Proc. AAAI conference on artificial intelligence, Vol. 32."},{"key":"e_1_2_1_29_1","unstructured":"NVIDIA. 2021. NVIDIA TensorRT. https:\/\/developer.nvidia.com\/tensorrt"},{"key":"e_1_2_1_30_1","unstructured":"NVIDIA. 2022. NVIDIA DLSS 3: AI-Powered Performance Multiplier Boosts Frame Rates By Up To 4X. https:\/\/www.nvidia.com\/en-us\/geforce\/news\/dlss3-ai-powered-neural-graphics-innovations\/ https:\/\/www.nvidia.com\/en-us\/geforce\/news\/dlss3-ai-powered-neural-graphics-innovations\/ [Accessed: 2024-01-24]."},{"key":"e_1_2_1_31_1","unstructured":"Oculus. 2016. Asynchronous Spacewarp. https:\/\/developer.oculus.com\/blog\/asynchronous-spacewarp\/"},{"key":"e_1_2_1_32_1","volume-title":"European Conference on Computer Vision. Springer, 250--266","author":"Reda Fitsum","year":"2022","unstructured":"Fitsum Reda, Janne Kontkanen, Eric Tabellion, Deqing Sun, Caroline Pantofaru, and Brian Curless. 2022. Film: Frame interpolation for large motion. In European Conference on Computer Vision. Springer, 250--266."},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1111\/cgf.13032","article-title":"Proxy-guided image-based rendering for mobile devices","volume":"35","author":"Reinert Bernhard","year":"2016","unstructured":"Bernhard Reinert, Johannes Kopf, Tobias Ritschel, Eduardo Cuervo, David Chu, and Hans-Peter Seidel. 2016. Proxy-guided image-based rendering for mobile devices. In Comp. Graph. Forum, Vol. 35. 353--362.","journal-title":"Comp. Graph. Forum"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2017.2657078"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3355088.3365170"},{"key":"e_1_2_1_36_1","volume-title":"A Case Study of First Person Aiming at Low Latency for Esports. CoRR abs\/2105.10498","author":"Spjut Josef B.","year":"2021","unstructured":"Josef B. Spjut, Ben Boudaoud, and Joohwan Kim. 2021. A Case Study of First Person Aiming at Low Latency for Esports. CoRR abs\/2105.10498 (2021). arXiv:2105.10498 https:\/\/arxiv.org\/abs\/2105.10498"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2993369.2993375"},{"key":"e_1_2_1_38_1","article-title":"ExtraSS: A Framework for Joint Spatial Super Sampling and Frame Extrapolation. In ACM","author":"Wu Songyin","year":"2023","unstructured":"Songyin Wu, Sungye Kim, Zheng Zeng, Deepak Vembar, Sangeeta Jha, Anton Kaplanyan, and Ling-Qi Yan. 2023a. ExtraSS: A Framework for Joint Spatial Super Sampling and Frame Extrapolation. In ACM Trans. Graph. (SIGGRAPH Asia). Article 92.","journal-title":"Trans. Graph. (SIGGRAPH Asia). Article 92."},{"key":"e_1_2_1_39_1","article-title":"Adaptive Recurrent Frame Prediction with Learnable Motion Vectors. In ACM","author":"Wu Zhizhen","year":"2023","unstructured":"Zhizhen Wu, Chenyu Zuo, Yuchi Huo, Yazhen Yuan, Yifan Peng, Guiyang Pu, Rui Wang, and Hujun Bao. 2023b. Adaptive Recurrent Frame Prediction with Learnable Motion Vectors. In ACM Trans. Graph. (SIGGRAPH Asia). Article 10.","journal-title":"Trans. Graph. (SIGGRAPH Asia). Article 10."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392376"},{"key":"e_1_2_1_41_1","article-title":"Image-based bidirectional scene reprojection. In ACM","author":"Yang Lei","year":"2011","unstructured":"Lei Yang, Yu-Chiu Tse, Pedro V Sander, Jason Lawrence, Diego Nehab, Hugues Hoppe, and Clara L Wilkins. 2011. Image-based bidirectional scene reprojection. In ACM Trans. Graph. (SIGGRAPH Asia). 1--10.","journal-title":"Trans. Graph. (SIGGRAPH Asia). 1--10."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3641519.3657424"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.142616"},{"key":"e_1_2_1_44_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 5682--5692","author":"Zhang Guozhen","year":"2023","unstructured":"Guozhen Zhang, Yuhan Zhu, Haonan Wang, Youxin Chen, Gangshan Wu, and Limin Wang. 2023. Extracting motion and appearance via inter-frame attention for efficient video frame interpolation. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 5682--5692."},{"key":"e_1_2_1_45_1","volume-title":"Proc. IEEE CVPR.","author":"Zhang Richard","year":"2018","unstructured":"Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In Proc. IEEE CVPR."},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 22169--22179","author":"Zhou Kun","year":"2023","unstructured":"Kun Zhou, Wenbo Li, Xiaoguang Han, and Jiangbo Lu. 2023. Exploring motion ambiguity and alignment for high-quality video frame interpolation. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 22169--22179."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3687923","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3687923","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:09:57Z","timestamp":1750295397000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3687923"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,19]]},"references-count":46,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,12,19]]}},"alternative-id":["10.1145\/3687923"],"URL":"https:\/\/doi.org\/10.1145\/3687923","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,19]]},"assertion":[{"value":"2024-11-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}