{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T16:45:54Z","timestamp":1779295554089,"version":"3.51.4"},"reference-count":39,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2021,12,1]],"date-time":"2021-12-01T00:00:00Z","timestamp":1638316800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:p>\n            The demand for creating rendered content continues to drastically grow. As it often is extremely computationally expensive and thus costly to render high-quality computer-generated images, there is a high incentive to reduce this computational burden. Recent advances in learning-based frame interpolation methods have shown exciting progress but still have not achieved the production-level quality which would be required to render fewer pixels and achieve savings in rendering times and costs. Therefore, in this paper we propose a method specifically targeted to achieve high-quality frame interpolation for rendered content. In this setting, we assume that we have full input for every\n            <jats:italic>n<\/jats:italic>\n            -th frame in addition to auxiliary feature buffers that are cheap to evaluate (e.g. depth, normals, albedo) for every frame. We propose solutions for leveraging such auxiliary features to obtain better motion estimates, more accurate occlusion handling, and to correctly reconstruct non-linear motion between keyframes. With this, our method is able to significantly push the state-of-the-art in frame interpolation for rendered content and we are able to obtain production-level quality results.\n          <\/jats:p>","DOI":"10.1145\/3478513.3480553","type":"journal-article","created":{"date-parts":[[2021,12,10]],"date-time":"2021-12-10T18:28:45Z","timestamp":1639160925000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":22,"title":["Neural frame interpolation for rendered content"],"prefix":"10.1145","volume":"40","author":[{"given":"Karlis Martins","family":"Briedis","sequence":"first","affiliation":[{"name":"Studios, Switzerland and ETH Z\u00fcrich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Abdelaziz","family":"Djelouah","sequence":"additional","affiliation":[{"name":"Studios, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mark","family":"Meyer","sequence":"additional","affiliation":[{"name":"Pixar Animation Studios"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ian","family":"McGonigal","sequence":"additional","affiliation":[{"name":"Industrial Light &amp; Magic, United Kingdom"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Markus","family":"Gross","sequence":"additional","affiliation":[{"name":"Studios, Switzerland and ETH Z\u00fcrich, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christopher","family":"Schroers","sequence":"additional","affiliation":[{"name":"Studios, Switzerland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,12,10]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-010-0390-2"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073708"},{"key":"e_1_2_2_3_1","volume-title":"Depth-Aware Video Frame Interpolation. In IEEE Conference on Computer Vision and Pattern Recognition.","author":"Bao Wenbo","year":"2019","unstructured":"Wenbo Bao , Wei-Sheng Lai , Chao Ma , Xiaoyun Zhang , Zhiyong Gao , and Ming-Hsuan Yang . 2019 . Depth-Aware Video Frame Interpolation. In IEEE Conference on Computer Vision and Pattern Recognition. Wenbo Bao, Wei-Sheng Lai, Chao Ma, Xiaoyun Zhang, Zhiyong Gao, and Ming-Hsuan Yang. 2019. Depth-Aware Video Frame Interpolation. In IEEE Conference on Computer Vision and Pattern Recognition."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33783-3_44"},{"key":"e_1_2_2_5_1","volume-title":"Zheng Liu, Juwei Lu, Jin Tang, and Konstantinos N. Plataniotis.","author":"Chi Zhixiang","year":"2020","unstructured":"Zhixiang Chi , Rasoul Mohammadi Nasiri , Zheng Liu, Juwei Lu, Jin Tang, and Konstantinos N. Plataniotis. 2020 . All at Once : Temporally Adaptive Multi-frame Interpolation with Advanced Motion Modeling. In Computer Vision - ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing , Cham, 107--123. Zhixiang Chi, Rasoul Mohammadi Nasiri, Zheng Liu, Juwei Lu, Jin Tang, and Konstantinos N. Plataniotis. 2020. All at Once: Temporally Adaptive Multi-frame Interpolation with Advanced Motion Modeling. In Computer Vision - ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing, Cham, 107--123."},{"key":"e_1_2_2_6_1","doi-asserted-by":"crossref","unstructured":"Myungsub Choi Heewon Kim Bohyung Han Ning Xu and Kyoung Mu Lee. 2020. Channel Attention Is All You Need for Video Frame Interpolation. In AAAI.  Myungsub Choi Heewon Kim Bohyung Han Ning Xu and Kyoung Mu Lee. 2020. Channel Attention Is All You Need for Video Frame Interpolation. In AAAI.","DOI":"10.1609\/aaai.v34i07.6693"},{"key":"e_1_2_2_7_1","volume-title":"4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http:\/\/arxiv.org\/abs\/1511","author":"Clevert Djork-Arn\u00e9","year":"2016","unstructured":"Djork-Arn\u00e9 Clevert , Thomas Unterthiner , and Sepp Hochreiter . 2016 . Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) . In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http:\/\/arxiv.org\/abs\/1511 .07289 Djork-Arn\u00e9 Clevert, Thomas Unterthiner, and Sepp Hochreiter. 2016. Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs). In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings, Yoshua Bengio and Yann LeCun (Eds.). http:\/\/arxiv.org\/abs\/1511.07289"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.316"},{"key":"e_1_2_2_9_1","doi-asserted-by":"crossref","unstructured":"D. Fourure R. Emonet E. Fromont D. Muselet A. Tremeau and C. Wolf. 2017. Residual conv-deconv grid network for semantic segmentation. arXiv preprint arXiv:1707.07958 (2017).  D. Fourure R. Emonet E. Fromont D. Muselet A. Tremeau and C. Wolf. 2017. Residual conv-deconv grid network for semantic segmentation. arXiv preprint arXiv:1707.07958 (2017).","DOI":"10.5244\/C.31.181"},{"key":"e_1_2_2_10_1","unstructured":"Junhwa Hur and Stefan Roth. 2019. Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation. In CVPR.  Junhwa Hur and Stefan Roth. 2019. Iterative Residual Refinement for Joint Optical Flow and Occlusion Estimation. In CVPR."},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.179"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00938"},{"key":"e_1_2_2_13_1","volume-title":"FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation.","author":"Kalluri Tarun","year":"2021","unstructured":"Tarun Kalluri , Deepak Pathak , Manmohan Chandraker , and Du Tran . 2021 . FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation. (2021). Tarun Kalluri, Deepak Pathak, Manmohan Chandraker, and Du Tran. 2021. FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation. (2021)."},{"key":"e_1_2_2_14_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014). Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00536"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-66823-5_3"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46466-4_26"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00059"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298747"},{"key":"e_1_2_2_21_1","volume-title":"Context-Aware Synthesis for Video Frame Interpolation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","author":"Niklaus Simon","year":"2018","unstructured":"Simon Niklaus and Feng Liu . 2018 . Context-Aware Synthesis for Video Frame Interpolation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Simon Niklaus and Feng Liu. 2018. Context-Aware Synthesis for Video Frame Interpolation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)."},{"key":"e_1_2_2_22_1","volume-title":"Softmax Splatting for Video Frame Interpolation. In IEEE Conference on Computer Vision and Pattern Recognition.","author":"Niklaus Simon","year":"2020","unstructured":"Simon Niklaus and Feng Liu . 2020 . Softmax Splatting for Video Frame Interpolation. In IEEE Conference on Computer Vision and Pattern Recognition. Simon Niklaus and Feng Liu. 2020. Softmax Splatting for Video Frame Interpolation. In IEEE Conference on Computer Vision and Pattern Recognition."},{"key":"e_1_2_2_23_1","volume-title":"IEEE Conference on Computer Vision and Pattern Recognition.","author":"Niklaus Simon","year":"2017","unstructured":"Simon Niklaus , Long Mai , and Feng Liu . 2017 a. Video Frame Interpolation via Adaptive Convolution . In IEEE Conference on Computer Vision and Pattern Recognition. Simon Niklaus, Long Mai, and Feng Liu. 2017a. Video Frame Interpolation via Adaptive Convolution. In IEEE Conference on Computer Vision and Pattern Recognition."},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.37"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV48630.2021.00114"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.438"},{"key":"e_1_2_2_27_1","volume-title":"BMBC: Bilateral Motion Estimation with Bilateral Cost","author":"Park Junheum","year":"2020","unstructured":"Junheum Park , Keunsoo Ko , Chul Lee , and Chang-Su Kim . 2020 . BMBC: Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation. In Computer Vision - ECCV 2020 , Andrea Vedaldi, Horst Bischof , Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing , Cham, 109--125. Junheum Park, Keunsoo Ko, Chul Lee, and Chang-Su Kim. 2020. BMBC: Bilateral Motion Estimation with Bilateral Cost Volume for Video Interpolation. In Computer Vision - ECCV 2020, Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing, Cham, 109--125."},{"key":"e_1_2_2_28_1","volume-title":"U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI) (LNCS","volume":"241","author":"Ronneberger O.","year":"2015","unstructured":"O. Ronneberger , P. Fischer , and T. Brox . 2015 . U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI) (LNCS , Vol. 9351). Springer, 234-- 241 . http:\/\/lmb.informatik.uni-freiburg.de\/Publications\/ 2015 \/RFB15a (available on arXiv:1505.04597 [cs.CV]). O. Ronneberger, P. Fischer, and T. Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention (MICCAI) (LNCS, Vol. 9351). Springer, 234--241. http:\/\/lmb.informatik.uni-freiburg.de\/Publications\/2015\/RFB15a (available on arXiv:1505.04597 [cs.CV])."},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00931"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58536-5_24"},{"key":"e_1_2_2_31_1","volume-title":"Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022","author":"Ulyanov Dmitry","year":"2016","unstructured":"Dmitry Ulyanov , Andrea Vedaldi , and Victor Lempitsky . 2016. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 ( 2016 ). Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. 2016. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022 (2016)."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3197517.3201388"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392376"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3454434"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-018-01144-2"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.142616"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58595-2_29"},{"key":"e_1_2_2_38_1","doi-asserted-by":"crossref","unstructured":"Richard Zhang Phillip Isola Alexei A Efros Eli Shechtman and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In CVPR.  Richard Zhang Phillip Isola Alexei A Efros Eli Shechtman and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In CVPR.","DOI":"10.1109\/CVPR.2018.00068"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.5555\/2858834.2858848"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478513.3480553","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3478513.3480553","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:11:40Z","timestamp":1750191100000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3478513.3480553"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,12]]},"references-count":39,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["10.1145\/3478513.3480553"],"URL":"https:\/\/doi.org\/10.1145\/3478513.3480553","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,12]]},"assertion":[{"value":"2021-12-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}