{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T06:02:59Z","timestamp":1784268179701,"version":"3.55.0"},"reference-count":61,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T00:00:00Z","timestamp":1690329600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:p>\n            Representing human performance at high-fidelity is an essential building block in diverse applications, such as film production, computer games or videoconferencing. To close the gap to production-level quality, we introduce HumanRF\n            <jats:sup>1<\/jats:sup>\n            , a 4D dynamic neural scene representation that captures full-body appearance in motion from multi-view video input, and enables playback from novel, unseen viewpoints. Our novel representation acts as a dynamic video encoding that captures fine details at high compression rates by factorizing space-time into a temporal matrix-vector decomposition. This allows us to obtain temporally coherent reconstructions of human actors for long sequences, while representing high-resolution details even in the context of challenging motion. While most research focuses on synthesizing at resolutions of 4MP or lower, we address the challenge of operating at 12MP. To this end, we introduce ActorsHQ, a novel multi-view dataset that provides 12MP footage from 160 cameras for 16 sequences with high-fidelity, per-frame mesh reconstructions\n            <jats:sup>2<\/jats:sup>\n            . We demonstrate challenges that emerge from using such high-resolution data and show that our newly introduced HumanRF effectively leverages this data, making a significant step towards production-level quality novel view synthesis.\n          <\/jats:p>","DOI":"10.1145\/3592415","type":"journal-article","created":{"date-parts":[[2023,7,26]],"date-time":"2023-07-26T15:47:45Z","timestamp":1690386465000},"page":"1-12","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":150,"title":["HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion"],"prefix":"10.1145","volume":"42","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3086-8922","authenticated-orcid":false,"given":"Mustafa","family":"I\u015f\u0131k","sequence":"first","affiliation":[{"name":"Synthesia, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-8404-7908","authenticated-orcid":false,"given":"Martin","family":"R\u00fcnz","sequence":"additional","affiliation":[{"name":"Synthesia, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5928-515X","authenticated-orcid":false,"given":"Markos","family":"Georgopoulos","sequence":"additional","affiliation":[{"name":"Synthesia, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4753-4811","authenticated-orcid":false,"given":"Taras","family":"Khakhulin","sequence":"additional","affiliation":[{"name":"Synthesia, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-5223-6031","authenticated-orcid":false,"given":"Jonathan","family":"Starck","sequence":"additional","affiliation":[{"name":"Synthesia, London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6947-1092","authenticated-orcid":false,"given":"Lourdes","family":"Agapito","sequence":"additional","affiliation":[{"name":"University College London (UCL), London, United Kingdom"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6093-5199","authenticated-orcid":false,"given":"Matthias","family":"Nie\u00dfner","sequence":"additional","affiliation":[{"name":"Technical University of Munich, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,7,26]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3386569.3392485"},{"key":"e_1_2_2_2_1","volume-title":"HexPlane: A Fast Representation for Dynamic Scenes. CVPR","author":"Cao Ang","year":"2023","unstructured":"Ang Cao and Justin Johnson. 2023. HexPlane: A Fast Representation for Dynamic Scenes. CVPR (2023)."},{"key":"e_1_2_2_3_1","doi-asserted-by":"crossref","unstructured":"Joel Carranza Christian Theobalt Marcus A. Magnor and Hans-Peter Seidel. 2003. Free-viewpoint video of human actors. In ACM Transactions on Graphics (TOG).","DOI":"10.1145\/1201775.882309"},{"key":"e_1_2_2_4_1","volume-title":"TensoRF: Tensorial Radiance Fields. In European Conference on Computer Vision (ECCV).","author":"Chen Anpei","year":"2022","unstructured":"Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su. 2022b. TensoRF: Tensorial Radiance Fields. In European Conference on Computer Vision (ECCV)."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01139"},{"key":"e_1_2_2_6_1","volume-title":"Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. arXiv preprint arXiv:2208.00277","author":"Chen Zhiqin","year":"2022","unstructured":"Zhiqin Chen, Thomas Funkhouser, Peter Hedman, and Andrea Tagliasacchi. 2022a. Mobilenerf: Exploiting the polygon rasterization pipeline for efficient neural field rendering on mobile architectures. arXiv preprint arXiv:2208.00277 (2022)."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766945"},{"key":"e_1_2_2_8_1","volume-title":"Robust estimation of a location parameter in the presence of asymmetry. The Annals of Statistics","author":"Collins John R","year":"1976","unstructured":"John R Collins. 1976. Robust estimation of a location parameter in the presence of asymmetry. The Annals of Statistics (1976), 68--85."},{"key":"e_1_2_2_9_1","unstructured":"Inc. Epic Games. 2022. RealityCapture. https:\/\/www.capturingreality.com Accessed: 2023-01-12."},{"key":"e_1_2_2_10_1","volume-title":"Fast Dynamic Radiance Fields with Time-Aware Neural Voxels. arXiv preprint arXiv:2205.15285","author":"Fang Jiemin","year":"2022","unstructured":"Jiemin Fang, Taoran Yi, Xinggang Wang, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Matthias Nie\u00dfner, and Qi Tian. 2022a. Fast Dynamic Radiance Fields with Time-Aware Neural Voxels. arXiv preprint arXiv:2205.15285 (2022)."},{"key":"e_1_2_2_11_1","volume-title":"Fast Dynamic Radiance Fields with Time-Aware Neural Voxels. In SIGGRAPH Asia 2022 Conference Papers.","author":"Fang Jiemin","year":"2022","unstructured":"Jiemin Fang, Taoran Yi, Xinggang Wang, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Matthias Nie\u00dfner, and Qi Tian. 2022b. Fast Dynamic Radiance Fields with Time-Aware Neural Voxels. In SIGGRAPH Asia 2022 Conference Papers."},{"key":"e_1_2_2_12_1","volume-title":"Benjamin Recht, and Angjoo Kanazawa.","author":"Fridovich-Keil Sara","year":"2023","unstructured":"Sara Fridovich-Keil, Giacomo Meanti, Frederik Rahb\u00e6k Warburg, Benjamin Recht, and Angjoo Kanazawa. 2023. K-Planes: Explicit Radiance Fields in Space, Time, and Appearance. In CVPR."},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00542"},{"key":"e_1_2_2_14_1","volume-title":"Proceedings of the Asian Conference on Computer Vision. 3757--3775","author":"Guo Xiang","year":"2022","unstructured":"Xiang Guo, Guanying Chen, Yuchao Dai, Xiaoqing Ye, Jiadai Sun, Xiao Tan, and Errui Ding. 2022a. Neural Deformable Voxel Grid for Fast Optimization of Dynamic View Synthesis. In Proceedings of the Asian Conference on Computer Vision. 3757--3775."},{"key":"e_1_2_2_15_1","volume-title":"Proceedings of the Asian Conference on Computer Vision (ACCV).","author":"Guo Xiang","year":"2022","unstructured":"Xiang Guo, Guanying Chen, Yuchao Dai, Xiaoqing Ye, Jiadai Sun, Xiao Tan, and Errui Ding. 2022b. Neural Deformable Voxel Grid for Fast Optimization of Dynamic View Synthesis. In Proceedings of the Asian Conference on Computer Vision (ACCV)."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459749"},{"key":"e_1_2_2_17_1","volume-title":"6m: Large scale datasets and predictive methods for 3d human sensing in natural environments","author":"Ionescu Catalin","year":"2013","unstructured":"Catalin Ionescu, Dragos Papava, Vlad Olaru, and Cristian Sminchisescu. 2013. Human3. 6m: Large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE transactions on pattern analysis and machine intelligence 36, 7 (2013), 1325--1339."},{"key":"e_1_2_2_18_1","volume-title":"Virtualized reality: Constructing virtual worlds from real scenes","author":"Kanade Takeo","year":"1997","unstructured":"Takeo Kanade, Peter Rander, and PJ Narayanan. 1997. Virtualized reality: Constructing virtual worlds from real scenes. IEEE multimedia 4, 1 (1997), 34--47."},{"key":"e_1_2_2_19_1","volume-title":"International journal of computer vision 38, 3","author":"Kutulakos Kiriakos N","year":"2000","unstructured":"Kiriakos N Kutulakos and Steven M Seitz. 2000. A theory of shape by space carving. International journal of computer vision 38, 3 (2000), 199--218."},{"key":"e_1_2_2_20_1","volume-title":"Tava: Template-free animatable volumetric actors. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27","author":"Li Ruilong","year":"2022","unstructured":"Ruilong Li, Julian Tanke, Minh Vo, Michael Zollh\u00f6fer, J\u00fcrgen Gall, Angjoo Kanazawa, and Christoph Lassner. 2022b. Tava: Template-free animatable volumetric actors. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part XXXII. Springer, 419--436."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00544"},{"key":"e_1_2_2_22_1","volume-title":"Toward a practical perceptual video quality metric. The Netflix Tech Blog 6, 2","author":"Li Zhi","year":"2016","unstructured":"Zhi Li, Anne Aaron, Ioannis Katsavounidis, Anush Moorthy, and Megha Manohara. 2016. Toward a practical perceptual video quality metric. The Netflix Tech Blog 6, 2 (2016)."},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00643"},{"key":"e_1_2_2_24_1","volume-title":"DynIBaR: Neural Dynamic Image-Based Rendering. arXiv preprint arXiv:2211.11082","author":"Li Zhengqi","year":"2022","unstructured":"Zhengqi Li, Qianqian Wang, Forrester Cole, Richard Tucker, and Noah Snavely. 2022c. DynIBaR: Neural Dynamic Image-Based Rendering. arXiv preprint arXiv:2211.11082 (2022)."},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV51458.2022.00319"},{"key":"e_1_2_2_26_1","volume-title":"Jussi Keppo, Ying Shan, Xiaohu Qie, and Mike Zheng Shou.","author":"Liu Jia-Wei","year":"2022","unstructured":"Jia-Wei Liu, Yan-Pei Cao, Weijia Mao, Wenqiao Zhang, David Junhao Zhang, Jussi Keppo, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. 2022. DeVRF: Fast Deformable Voxel Radiance Fields for Dynamic Scenes. Advances in Neural Information Processing Systems."},{"key":"e_1_2_2_27_1","first-page":"1","article-title":"Neural actor: Neural free-view synthesis of human actors with pose control","volume":"40","author":"Liu Lingjie","year":"2021","unstructured":"Lingjie Liu, Marc Habermann, Viktor Rudnev, Kripasindhu Sarkar, Jiatao Gu, and Christian Theobalt. 2021. Neural actor: Neural free-view synthesis of human actors with pose control. ACM Transactions on Graphics (TOG) 40, 6 (2021), 1--16.","journal-title":"ACM Transactions on Graphics (TOG)"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323020"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3450626.3459863"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818013"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530086"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/2945.468400"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2017.00064"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00459"},{"key":"e_1_2_2_35_1","doi-asserted-by":"crossref","unstructured":"Ben Mildenhall Pratul P. Srinivasan Matthew Tancik Jonathan T. Barron Ravi Ramamoorthi and Ren Ng. 2020. NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis. In ECCV.","DOI":"10.1007\/978-3-030-58452-8_24"},{"key":"e_1_2_2_36_1","unstructured":"Thomas M\u00fcller. 2021. tiny-cuda-nn. https:\/\/github.com\/NVlabs\/tiny-cuda-nn Accessed: 2022-10-21."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530127"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00571"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00025"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00581"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3478513.3480487"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00894"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01018"},{"key":"e_1_2_2_44_1","volume-title":"Tensor4D: Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering. arXiv preprint arXiv:2211.11610","author":"Shao Ruizhi","year":"2022","unstructured":"Ruizhi Shao, Zerong Zheng, Hanzhang Tu, Boning Liu, Hongwen Zhang, and Yebin Liu. 2022. Tensor4D: Efficient Neural 4D Decomposition for High-fidelity Dynamic Reconstruction and Rendering. arXiv preprint arXiv:2211.11610 (2022)."},{"key":"e_1_2_2_45_1","volume-title":"NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields. arXiv preprint arXiv:2210.15947","author":"Song Liangchen","year":"2022","unstructured":"Liangchen Song, Anpei Chen, Zhong Li, Zhang Chen, Lele Chen, Junsong Yuan, Yi Xu, and Andreas Geiger. 2022. NeRFPlayer: A Streamable Dynamic Scene Representation with Decomposed Neural Radiance Fields. arXiv preprint arXiv:2210.15947 (2022)."},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2007.68"},{"key":"e_1_2_2_47_1","first-page":"12278","article-title":"A-nerf: Articulated neural radiance fields for learning human shape, appearance, and pose","volume":"34","author":"Su Shih-Yang","year":"2021","unstructured":"Shih-Yang Su, Frank Yu, Michael Zollh\u00f6fer, and Helge Rhodin. 2021. A-nerf: Articulated neural radiance fields for learning human shape, appearance, and pose. Advances in Neural Information Processing Systems 34 (2021), 12278--12291.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_2_48_1","doi-asserted-by":"crossref","unstructured":"Cheng Sun Min Sun and Hwann-Tzong Chen. 2022. Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction. In CVPR.","DOI":"10.1109\/CVPR52688.2022.00538"},{"key":"e_1_2_2_49_1","volume-title":"Compressible-composable NeRF via Rank-residual Decomposition. arXiv preprint arXiv:2205.14870","author":"Tang Jiaxiang","year":"2022","unstructured":"Jiaxiang Tang, Xiaokang Chen, Jingbo Wang, and Gang Zeng. 2022. Compressible-composable NeRF via Rank-residual Decomposition. arXiv preprint arXiv:2205.14870 (2022)."},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01316"},{"key":"e_1_2_2_51_1","volume-title":"Arah: Animatable","author":"Wang Shaofei","year":"2022","unstructured":"Shaofei Wang, Katja Schwarz, Andreas Geiger, and Siyu Tang. 2022a. Arah: Animatable volume rendering of articulated human sdfs. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23--27, 2022, Proceedings, Part XXXII. Springer, 1--19."},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00565"},{"key":"e_1_2_2_53_1","volume-title":"Image quality assessment: from error visibility to structural similarity","author":"Wang Zhou","year":"2004","unstructured":"Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4 (2004), 600--612."},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01542"},{"key":"e_1_2_2_55_1","volume-title":"Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance. Advances in Neural Information Processing Systems 33","author":"Yariv Lior","year":"2020","unstructured":"Lior Yariv, Yoni Kasten, Dror Moran, Meirav Galun, Matan Atzmon, Basri Ronen, and Yaron Lipman. 2020. Multiview Neural Surface Reconstruction by Disentangling Geometry and Appearance. Advances in Neural Information Processing Systems 33 (2020)."},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00570"},{"key":"e_1_2_2_57_1","volume-title":"NeuVV: Neural Volumetric Videos with Immersive Rendering and Editing. arXiv preprint arXiv:2202.06088","author":"Zhang Jiakai","year":"2022","unstructured":"Jiakai Zhang, Liao Wang, Xinhang Liu, Fuqiang Zhao, Minzhang Li, Haizhao Dai, Boyuan Zhang, Wei Yang, Lan Xu, and Jingyi Yu. 2022. NeuVV: Neural Volumetric Videos with Immersive Rendering and Editing. arXiv preprint arXiv:2202.06088 (2022)."},{"key":"e_1_2_2_58_1","doi-asserted-by":"crossref","unstructured":"Richard Zhang Phillip Isola Alexei A Efros Eli Shechtman and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In CVPR.","DOI":"10.1109\/CVPR.2018.00068"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3550454.3555451"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01543"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00783"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3592415","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3592415","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:48:59Z","timestamp":1750182539000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3592415"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,26]]},"references-count":61,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["10.1145\/3592415"],"URL":"https:\/\/doi.org\/10.1145\/3592415","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,26]]},"assertion":[{"value":"2023-07-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}