{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T17:01:19Z","timestamp":1777568479127,"version":"3.51.4"},"reference-count":121,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T00:00:00Z","timestamp":1731974400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2024,12,19]]},"abstract":"<jats:p>\n            High-quality real-time rendering using user-affordable capture rigs is an essential property of human performance capture systems for real-world applications. However, state-of-the-art performance capture methods may not yield satisfactory rendering results under a very sparse (e.g., four) capture setting. Specifically, neural radiance field (NeRF)-based methods and 3D Gaussian Splatting (3DGS)-based methods tend to produce local geometry errors for unseen performers, while occupancy field (PIFu)-based methods often produce unrealistic rendering results. In this paper, we propose a novel generalizable neural approach to reconstruct and render the performers from very sparse RGBD streams in high quality. The core of our method is a novel point-based generalizable human (PGH) representation conditioned on the pixel-aligned RGBD features. The PGH representation learns a\n            <jats:italic>surface implicit function<\/jats:italic>\n            for the regression of surface points and a\n            <jats:italic>Gaussian implicit function<\/jats:italic>\n            for parameterizing the radiance fields of the regressed surface points with 2D Gaussian surfels, and uses surfel splatting for fast rendering. We learn this hybrid human representation via two novel networks. First, we propose a novel point-regressing network (PRNet) with a depth-guided point cloud initialization (DPI) method to regress an accurate surface point cloud based on the denoised depth information. Second, we propose a novel neural blending-based surfel splatting network (SPNet) to render high-quality geometries and appearances in novel views based on the regressed surface points and high-resolution RGBD features of adjacent views. Our method produces free-view human performance videos of 1K resolution at 12 fps on average. Experiments on two benchmarks show that our method outperforms state-of-the-art human performance capture methods.\n          <\/jats:p>","DOI":"10.1145\/3687993","type":"journal-article","created":{"date-parts":[[2024,11,19]],"date-time":"2024-11-19T15:46:04Z","timestamp":1732031164000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Gaussian Surfel Splatting for Live Human Performance Capture"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-5191-9348","authenticated-orcid":false,"given":"Zheng","family":"Dong","sequence":"first","affiliation":[{"name":"State Key Laboratory of CAD &amp; CG, Zhejiang University, Hang Zhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5855-3810","authenticated-orcid":false,"given":"Ke","family":"Xu","sequence":"additional","affiliation":[{"name":"City University of Hong Kong, Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-4791-0263","authenticated-orcid":false,"given":"Yaoan","family":"Gao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of CAD &amp; CG, Zhejiang University, Hang Zhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2662-0334","authenticated-orcid":false,"given":"Hujun","family":"Bao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of CAD &amp; CG, Zhejiang University, Hang Zhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3756-3539","authenticated-orcid":false,"given":"Weiwei","family":"Xu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of CAD &amp; CG, Zhejiang University, Hang Zhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8957-8129","authenticated-orcid":false,"given":"Rynson W. H.","family":"Lau","sequence":"additional","affiliation":[{"name":"City University of Hong Kong, Hong Kong, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,11,19]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Gaussian Shell Maps for Efficient 3D Human Generation. arXiv:2311.17857","author":"Abdal Rameen","year":"2023","unstructured":"Rameen Abdal, Wang Yifan, Zifan Shi, Yinghao Xu, Ryan Po, Zhengfei Kuang, Qifeng Chen, Dit-Yan Yeung, and Gordon Wetzstein. 2023. Gaussian Shell Maps for Efficient 3D Human Generation. arXiv:2311.17857 (2023)."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-25082-8_3"},{"key":"e_1_2_1_3_1","volume-title":"JIFF: Jointly-aligned Implicit Face Function for High Quality Single View Clothed Human Reconstruction. In IEEE Conf. Comput. Vis. Pattern Recog.","author":"Cao Yukang","year":"2022","unstructured":"Yukang Cao, Guanying Chen, Kai Han, Wenqi Yang, and Kwan-Yee K Wong. 2022. JIFF: Jointly-aligned Implicit Face Function for High Quality Single View Clothed Human Reconstruction. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_4_1","unstructured":"Kennard Chan Guosheng Lin Haiyu Zhao and Weisi Lin. 2022a. S-PIFu: Integrating Parametric Human Models with PIFu for Single-view Clothed Human Reconstruction. In Adv. Neural Inform. Process. Syst."},{"key":"e_1_2_1_5_1","volume-title":"IntegratedPIFu: Integrated Pixel Aligned Implicit Function for Single-view Human Reconstruction. In Eur. Conf. Comput. Vis.","author":"Chan Kennard Yanting","year":"2022","unstructured":"Kennard Yanting Chan, Guosheng Lin, Haiyu Zhao, and Weisi Lin. 2022b. IntegratedPIFu: Integrated Pixel Aligned Implicit Function for Single-view Human Reconstruction. In Eur. Conf. Comput. Vis."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01386"},{"key":"e_1_2_1_7_1","volume-title":"Animatable Neural Radiance Fields from Monocular RGB Videos. arXiv:2106.13629","author":"Chen Jianchuan","year":"2021","unstructured":"Jianchuan Chen, Ying Zhang, Di Kang, Xuefei Zhe, Linchao Bao, Xu Jia, and Huchuan Lu. 2021b. Animatable Neural Radiance Fields from Monocular RGB Videos. arXiv:2106.13629 (2021)."},{"key":"e_1_2_1_8_1","volume-title":"Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis. arXiv:2204.11798","author":"Cheng Wei","year":"2022","unstructured":"Wei Cheng, Su Xu, Jingtan Piao, Chen Qian, Wayne Wu, Kwan-Yee Lin, and Hongsheng Li. 2022. Generalizable Neural Performer: Learning Robust Radiance Fields for Human Novel View Synthesis. arXiv:2204.11798 (2022)."},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Alvaro Collet Ming Chuang Pat Sweeney Don Gillett Dennis Evseev David Calabrese Hugues Hoppe Adam Kirk and Steve Sullivan. 2015. High-Quality Streamable Free-Viewpoint Video. ACM Trans. Graph. (2015).","DOI":"10.1145\/2766945"},{"key":"e_1_2_1_10_1","volume-title":"ACM SIGGRAPH Conference Proceedings.","author":"Dai Pinxuan","year":"2024","unstructured":"Pinxuan Dai, Jiamin Xu, Wenxiang Xie, Xinguo Liu, Huamin Wang, and Weiwei Xu. 2024. High-quality Surface Reconstruction using Gaussian Surfels. In ACM SIGGRAPH Conference Proceedings."},{"key":"e_1_2_1_11_1","volume-title":"ACM SIGGRAPH Conference Proceedings.","author":"Aguiar Edilson De","year":"2008","unstructured":"Edilson De Aguiar, Carsten Stoll, Christian Theobalt, Naveed Ahmed, Hans-Peter Seidel, and Sebastian Thrun. 2008. Performance capture from sparse multi-view video. In ACM SIGGRAPH Conference Proceedings."},{"key":"e_1_2_1_12_1","unstructured":"Zheng Dong Ke Xu Ziheng Duan Hujun Bao Weiwei Xu and Rynson Lau. 2022. Geometry-aware Two-scale PIFu Representation for Human Reconstruction. In Adv. Neural Inform. Process. Syst."},{"key":"e_1_2_1_13_1","volume-title":"SAILOR: Synergizing Radiance and Occupancy Fields for Live Human Performance Capture. ACM Trans. Graph.","author":"Dong Zheng","year":"2023","unstructured":"Zheng Dong, Ke Xu, Yaoan Gao, Qilin Sun, Hujun Bao, Weiwei Xu, and Rynson WH Lau. 2023. SAILOR: Synergizing Radiance and Occupancy Fields for Live Human Performance Capture. ACM Trans. Graph. (2023)."},{"key":"e_1_2_1_14_1","unstructured":"Mingsong Dou Philip L. Davidson S. Fanello S. Khamis Adarsh Kowdle Christoph Rhemann Vladimir Tankovich and Shahram Izadi. 2017. Motion2fusion: real-time volumetric performance capture. ACM Trans. Graph. (2017)."},{"key":"e_1_2_1_15_1","volume-title":"Adarsh Kowdle, Sergio Orts Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, et al.","author":"Dou Mingsong","year":"2016","unstructured":"Mingsong Dou, Sameh Khamis, Yury Degtyarev, Philip Davidson, Sean Ryan Fanello, Adarsh Kowdle, Sergio Orts Escolano, Christoph Rhemann, David Kim, Jonathan Taylor, et al. 2016. Fusion4d: Real-time performance capture of challenging scenes. ACM Trans. Graph. (2016)."},{"key":"e_1_2_1_16_1","volume-title":"FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction. In Adv. Neural Inform. Process. Syst.","author":"Feng Qiao","year":"2022","unstructured":"Qiao Feng, Yebin Liu, Yu-Kun Lai, Jingyu Yang, and Kun Li. 2022. FOF: Learning Fourier Occupancy Field for Monocular Real-time Human Reconstruction. In Adv. Neural Inform. Process. Syst."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01201"},{"key":"e_1_2_1_18_1","volume-title":"Plenoxels: Radiance Fields without Neural Networks. In CVPR.","author":"Yu Fridovich-Keil","year":"2022","unstructured":"Fridovich-Keil and Yu, Matthew Tancik, Qinhong Chen, Benjamin Recht, and Angjoo Kanazawa. 2022. Plenoxels: Radiance Fields without Neural Networks. In CVPR."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00854"},{"key":"e_1_2_1_20_1","volume-title":"Mps-nerf: Generalizable 3d human rendering from multiview images","author":"Gao Xiangjun","year":"2022","unstructured":"Xiangjun Gao, Jiaolong Yang, Jongyoo Kim, Sida Peng, Zicheng Liu, and Xin Tong. 2022. Mps-nerf: Generalizable 3d human rendering from multiview images. IEEE Trans. Pattern Anal. Mach. Intell. (2022)."},{"key":"e_1_2_1_21_1","doi-asserted-by":"crossref","unstructured":"Kaiwen Guo Peter Lincoln Philip Davidson Jay Busch Xueming Yu Matt Whalen Geoff Harvey Sergio Orts-Escolano Rohit Pandey Jason Dourgarian et al. 2019. The relightables: Volumetric performance capture of humans with realistic relighting. ACM Trans. Graph. (2019).","DOI":"10.1145\/3355089.3356571"},{"key":"e_1_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Marc Habermann Lingjie Liu Weipeng Xu Michael Zollhoefer Gerard Pons-Moll and Christian Theobalt. 2021. Real-time deep dynamic characters. ACM Trans. Graph. (2021).","DOI":"10.1145\/3476576.3476653"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3311970"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00510"},{"key":"e_1_2_1_25_1","unstructured":"Tong He John Collomosse Hailin Jin and Stefano Soatto. 2020. Geo-PIFu: Geometry and Pixel Aligned Implicit Functions for Single-view Human Reconstruction. In NeurIPS."},{"key":"e_1_2_1_26_1","unstructured":"Tong He Yuanlu Xu Shunsuke Saito Stefano Soatto and Tony Tung. 2021. ARCH+ +: Animation-ready clothed human reconstruction revisited. In ICCV."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00060"},{"key":"e_1_2_1_28_1","volume-title":"Int. Conf. Comput. Vis.","author":"Hu Shoukang","year":"2023","unstructured":"Shoukang Hu, Fangzhou Hong, Liang Pan, Haiyi Mei, Lei Yang, and Ziwei Liu. 2023. Sherf: Generalizable human nerf from a single image. In Int. Conf. Comput. Vis."},{"key":"e_1_2_1_29_1","unstructured":"Shoukang Hu and Ziwei Liu. 2024. GauHuman: Articulated Gaussian Splatting for Real-Time 3D Human Rendering."},{"key":"e_1_2_1_30_1","volume-title":"ACM SIGGRAPH Conference Proceedings.","author":"Huang Binbin","year":"2024","unstructured":"Binbin Huang, Zehao Yu, Anpei Chen, Andreas Geiger, and Shenghua Gao. 2024. 2D Gaussian Splatting for Geometrically Accurate Radiance Fields. In ACM SIGGRAPH Conference Proceedings."},{"key":"e_1_2_1_31_1","volume-title":"Arch: Animatable reconstruction of clothed humans. In CVPR.","author":"Huang Zeng","year":"2020","unstructured":"Zeng Huang, Yuanlu Xu, Christoph Lassner, Hao Li, and Tony Tung. 2020. Arch: Animatable reconstruction of clothed humans. In CVPR."},{"key":"e_1_2_1_32_1","doi-asserted-by":"crossref","unstructured":"Mustafa I\u015f\u0131k Martin R\u00fcnz Markos Georgopoulos Taras Khakhulin Jonathan Starck Lourdes Agapito and Matthias Nie\u00dfner. 2023. HumanRF: High-Fidelity Neural Radiance Fields for Humans in Motion. ACM Trans. Graph. (2023).","DOI":"10.1145\/3592415"},{"key":"e_1_2_1_33_1","volume-title":"Proc. of SIGGRAPH.","author":"Jiakai Zhang","year":"2021","unstructured":"Zhang Jiakai, Liu Xinhang, Ye Xinyi, Zhao Fuqiang, Zhang Yanshun, Wu Minye, Zhang Yingliang, Xu Lan, and Yu Jingyi. 2021. Editable Free-Viewpoint Video using a Layered Neural Representation. In Proc. of SIGGRAPH."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-19824-3_24"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00868"},{"key":"e_1_2_1_36_1","doi-asserted-by":"crossref","unstructured":"Bernhard Kerbl Georgios Kopanas Thomas Leimk\u00fchler and George Drettakis. 2023. 3D Gaussian Splatting for Real-Time Radiance Field Rendering. ACM Trans. Graph. (2023).","DOI":"10.1145\/3592433"},{"key":"e_1_2_1_37_1","volume-title":"3D Gaussian Splatting as Markov Chain Monte Carlo. arXiv:2404.09591","author":"Kheradmand Shakiba","year":"2024","unstructured":"Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Jeff Tseng, Hossam Isack, Abhishek Kar, Andrea Tagliasacchi, and Kwang Moo Yi. 2024. 3D Gaussian Splatting as Markov Chain Monte Carlo. arXiv:2404.09591 (2024)."},{"key":"e_1_2_1_38_1","volume-title":"You Only Train Once: Multi-Identity Free-Viewpoint Neural Human Rendering from Monocular Videos. arXiv:2303.05835","author":"Kim Jaehyeok","year":"2023","unstructured":"Jaehyeok Kim, Dongyoon Wee, and Dan Xu. 2023. You Only Train Once: Multi-Identity Free-Viewpoint Neural Human Rendering from Monocular Videos. arXiv:2303.05835 (2023)."},{"key":"e_1_2_1_39_1","volume-title":"Adam: A method for stochastic optimization. arXiv:1412.6980","author":"Kingma Diederik","year":"2014","unstructured":"Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv:1412.6980 (2014)."},{"key":"e_1_2_1_40_1","volume-title":"Int. J. Comput. Vis.","author":"Kiriakos Kutulakos","year":"2000","unstructured":"Kutulakos Kiriakos and Seitz Steven. 2000. A Theory of Shape by Space Carving. Int. J. Comput. Vis. (2000)."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00530"},{"key":"e_1_2_1_42_1","volume-title":"HUGS: Human Gaussian Splats. In IEEE Conf. Comput. Vis. Pattern Recog.","author":"Kocabas Muhammed","year":"2024","unstructured":"Muhammed Kocabas, Rick Chang, James Gabriel, Oncel Tuzel, and Anurag Ranjan. 2024. HUGS: Human Gaussian Splats. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_43_1","unstructured":"Youngjoong Kwon Dahun Kim Duygu Ceylan and Henry Fuchs. 2021. Neural human performer: Learning generalizable radiance fields for human performance rendering. In Adv. Neural Inform. Process. Syst."},{"key":"e_1_2_1_44_1","unstructured":"Youngjoong Kwon Dahun Kim Duygu Ceylan and Henry Fuchs. 2023. Neural image-based avatars: Generalizable radiance fields for human avatar modeling. In ICLR."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.273735"},{"key":"e_1_2_1_46_1","volume-title":"IEEE Conf. Comput. Vis. Pattern Recog.","author":"Lei Jiahui","year":"2024","unstructured":"Jiahui Lei, Yufu Wang, Georgios Pavlakos, Lingjie Liu, and Kostas Daniilidis. 2024. Gart: Gaussian articulated template models. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58592-1_4"},{"key":"e_1_2_1_48_1","doi-asserted-by":"crossref","unstructured":"Yue Li Marc Habermann Bernhard Thomaszewski Stelian Coros Thabo Beeler and Christian Theobalt. 2021. Deep Physics-aware Inference of Cloth Deformation for Monocular Human Performance Capture. In 3DV.","DOI":"10.1109\/3DV53792.2021.00047"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00142"},{"key":"e_1_2_1_50_1","volume-title":"Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling. In IEEE Conf. Comput. Vis. Pattern Recog.","author":"Li Zhe","year":"2024","unstructured":"Zhe Li, Zerong Zheng, Lizhen Wang, and Yebin Liu. 2024. Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_51_1","unstructured":"Haotong Lin Sida Peng Zhen Xu Yunzhi Yan Qing Shuai Hujun Bao and Xiaowei Zhou. 2022. Efficient Neural Radiance Fields with Learned Depth-Guided Sampling. In ACM SIGGRAPH Asia."},{"key":"e_1_2_1_52_1","unstructured":"Lingjie Liu Marc Habermann Viktor Rudnev Kripasindhu Sarkar Jiatao Gu and Christian Theobalt. 2021. Neural actor: Neural free-view synthesis of human actors with pose control. ACM Trans. Graph. (2021)."},{"key":"e_1_2_1_53_1","doi-asserted-by":"crossref","unstructured":"Xian Liu Xiaohang Zhan Jiaxiang Tang Ying Shan Gang Zeng Dahua Lin Xihui Liu and Ziwei Liu. 2024. HumanGaussian: Text-Driven 3D Human Generation with Gaussian Splatting.","DOI":"10.1109\/CVPR52733.2024.00635"},{"key":"e_1_2_1_54_1","volume-title":"A point-cloud-based multiview stereo algorithm for free-viewpoint video","author":"Liu Yebin","year":"2009","unstructured":"Yebin Liu, Qionghai Dai, and Wenli Xu. 2009. A point-cloud-based multiview stereo algorithm for free-viewpoint video. IEEE Trans. Vis. Comput. Graph. (2009)."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818013"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-19784-0_11"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58452-8_24"},{"key":"e_1_2_1_58_1","volume-title":"IEEE Conf. Comput. Vis. Pattern Recog.","author":"Moreau Arthur","year":"2024","unstructured":"Arthur Moreau, Jifei Song, Helisa Dhamo, Richard Shaw, Yiren Zhou, and Eduardo P\u00e9rez-Pellitero. 2024. Human gaussian splatting: Real-time rendering of animatable avatars. IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_59_1","doi-asserted-by":"crossref","unstructured":"Thomas M\u00fcller Fabrice Rousselle Jan Nov\u00e1k and Alexander Keller. 2021. Real-time Neural Radiance Caching for Path Tracing. ACM Trans. Graph. (2021).","DOI":"10.1145\/3476576.3476579"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298631"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298631"},{"key":"e_1_2_1_62_1","volume-title":"Kinectfusion: Real-time dense surface mapping and tracking","author":"Newcombe Richard A","year":"2011","unstructured":"Richard A Newcombe, Shahram Izadi, Otmar Hilliges, David Molyneaux, David Kim, Andrew J Davison, Pushmeet Kohi, Jamie Shotton, Steve Hodges, and Andrew Fitzgibbon. 2011. Kinectfusion: Real-time dense surface mapping and tracking. In IEEE ISMAR."},{"key":"e_1_2_1_63_1","doi-asserted-by":"crossref","unstructured":"Anqi Pang Xin Chen Haimin Luo Minye Wu Jingyi Yu and Lan Xu. 2021. Few-shot neural human performance rendering from sparse RGBD videos. In IJCAI.","DOI":"10.24963\/ijcai.2021\/130"},{"key":"e_1_2_1_64_1","volume-title":"Nerfies: Deformable Neural Radiance Fields. In Int. Conf. Comput. Vis.","author":"Park Keunhong","year":"2021","unstructured":"Keunhong Park, Utkarsh Sinha, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Steven M. Seitz, and Ricardo Martin-Brualla. 2021a. Nerfies: Deformable Neural Radiance Fields. In Int. Conf. Comput. Vis."},{"key":"e_1_2_1_65_1","volume-title":"Seitz","author":"Park Keunhong","year":"2021","unstructured":"Keunhong Park, Utkarsh Sinha, Peter Hedman, Jonathan T. Barron, Sofien Bouaziz, Dan B Goldman, Ricardo Martin-Brualla, and Steven M. Seitz. 2021b. HyperNeRF: A Higher-Dimensional Representation for Topologically Varying Neural Radiance Fields. ACM Trans. Graph. (2021)."},{"key":"e_1_2_1_66_1","unstructured":"Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in Pytorch. In Adv. Neural Inform. Process. Syst."},{"key":"e_1_2_1_67_1","volume-title":"IntrinsicNGP: Intrinsic Coordinate based Hash Encoding for Human NeRF. arXiv:2302.14683","author":"Peng Bo","year":"2023","unstructured":"Bo Peng, Jun Hu, Jingtao Zhou, Xuan Gao, and Juyong Zhang. 2023. IntrinsicNGP: Intrinsic Coordinate based Hash Encoding for Human NeRF. arXiv:2302.14683 (2023)."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01405"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00894"},{"key":"e_1_2_1_70_1","volume-title":"ANIM: Accurate Neural Implicit Model for Human Reconstruction from a single RGB-D image. In CVPR.","author":"Pesavento Marco","year":"2024","unstructured":"Marco Pesavento, Yuanlu Xu, Nikolaos Sarafianos, Robert Maier, Ziyan Wang, Chun-Han Yao, Marco Volino, Edmond Boyer, Adrian Hilton, and Tony Tung. 2024. ANIM: Accurate Neural Implicit Model for Human Reconstruction from a single RGB-D image. In CVPR."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01018"},{"key":"e_1_2_1_72_1","volume-title":"IEEE Conf. Comput. Vis. Pattern Recog.","author":"Qian Shenhan","year":"2024","unstructured":"Shenhan Qian, Tobias Kirschstein, Liam Schoneveld, Davide Davoli, Simon Giebenhain, and Matthias Nie\u00dfner. 2024. Gaussianavatars: Photorealistic head avatars with rigged 3d Gaussians. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01550"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528233.3530740"},{"key":"e_1_2_1_75_1","volume-title":"Rig3DGS: Creating Controllable Portraits from Casual Monocular Videos. arXiv:2402.03723","author":"Rivero Alfredo","year":"2024","unstructured":"Alfredo Rivero, ShahRukh Athar, Zhixin Shu, and Dimitris Samaras. 2024. Rig3DGS: Creating Controllable Portraits from Casual Monocular Videos. arXiv:2402.03723 (2024)."},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00239"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00016"},{"key":"e_1_2_1_78_1","volume-title":"ACM SIGGRAPH Asia Conference Proceedings.","author":"Shao Ruizhi","year":"2022","unstructured":"Ruizhi Shao, Liliang Chen, Zerong Zheng, Hongwen Zhang, Yuxiang Zhang, Han Huang, Yandong Guo, and Yebin Liu. 2022a. FloRen: Real-Time High-Quality Human Performance Rendering via Appearance Flow Using Sparse RGB Cameras. In ACM SIGGRAPH Asia Conference Proceedings."},{"key":"e_1_2_1_79_1","volume-title":"DoubleField: Bridging the Neural Surface and Radiance Fields for High-fidelity Human Reconstruction and Rendering. In IEEE Conf. Comput. Vis. Pattern Recog.","author":"Shao Ruizhi","year":"2022","unstructured":"Ruizhi Shao, Hongwen Zhang, He Zhang, Mingjia Chen, Yanpei Cao, Tao Yu, and Yebin Liu. 2022b. DoubleField: Bridging the Neural Surface and Radiance Fields for High-fidelity Human Reconstruction and Rendering. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_80_1","volume-title":"SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting. In IEEE Conf. Comput. Vis. Pattern Recog.","author":"Shao Zhijing","year":"2024","unstructured":"Zhijing Shao, Zhaolong Wang, Zhuang Li, Duotun Wang, Xiangru Lin, Yu Zhang, Mingming Fan, and Zeyu Wang. 2024. SplattingAvatar: Realistic Real-Time Human Avatars with Mesh-Embedded Gaussian Splatting. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-20086-1_7"},{"key":"e_1_2_1_82_1","volume-title":"A-nerf: Articulated neural radiance fields for learning human shape, appearance, and pose. Adv. Neural Inform. Process. Syst.","author":"Su Shih-Yang","year":"2021","unstructured":"Shih-Yang Su, Frank Yu, Michael Zollh\u00f6fer, and Helge Rhodin. 2021. A-nerf: Articulated neural radiance fields for learning human shape, appearance, and pose. Adv. Neural Inform. Process. Syst. (2021)."},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58548-8_15"},{"key":"e_1_2_1_84_1","volume-title":"LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation. arXiv:2402.05054","author":"Tang Jiaxiang","year":"2024","unstructured":"Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu. 2024. LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creation. arXiv:2402.05054 (2024)."},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01272"},{"key":"e_1_2_1_86_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Adv. Neural Inform. Process. Syst."},{"key":"e_1_2_1_87_1","doi-asserted-by":"crossref","unstructured":"Daniel Vlasic Ilya Baran Wojciech Matusik and Jovan Popovi\u0107. 2008. Articulated Mesh Animation from Multi-View Silhouettes. ACM Trans. Graph. (2008).","DOI":"10.1145\/1399504.1360696"},{"key":"e_1_2_1_88_1","doi-asserted-by":"crossref","unstructured":"Daniel Vlasic Pieter Peers Ilya Baran Paul Debevec Jovan Popovi\u0107 Szymon Rusinkiewicz and Wojciech Matusik. 2009. Dynamic shape capture using multi-view photometric stereo. In ACM SIGGRAPH Asia.","DOI":"10.1145\/1661412.1618520"},{"key":"e_1_2_1_89_1","volume-title":"Deep High-Resolution Representation Learning for Visual Recognition","author":"Wang Jingdong","year":"2021","unstructured":"Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, Wenyu Liu, and Bin Xiao. 2021a. Deep High-Resolution Representation Learning for Visual Recognition. IEEE Trans. Pattern Anal. Mach. Intell. (2021)."},{"key":"e_1_2_1_90_1","volume-title":"IButter: Neural Interactive Bullet Time Generator for Human Free-Viewpoint Rendering. In ACM Int. Conf. Multimedia.","author":"Wang Liao","year":"2021","unstructured":"Liao Wang, Ziyu Wang, Pei Lin, Yuheng Jiang, Xin Suo, Minye Wu, Lan Xu, and Jingyi Yu. 2021c. IButter: Neural Interactive Bullet Time Generator for Human Free-Viewpoint Rendering. In ACM Int. Conf. Multimedia."},{"key":"e_1_2_1_91_1","volume-title":"Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-Time. In IEEE Conf. Comput. Vis. Pattern Recog.","author":"Wang Liao","year":"2022","unstructured":"Liao Wang, Jiakai Zhang, Xinhang Liu, Fuqiang Zhao, Yanshun Zhang, Yingliang Zhang, Minye Wu, Jingyi Yu, and Lan Xu. 2022. Fourier PlenOctrees for Dynamic Radiance Field Rendering in Real-Time. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_92_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00466"},{"key":"e_1_2_1_93_1","volume-title":"Learning ViewSynthesis for Desktop Telepresence with Few RGBD Cameras","author":"Wang Shengze","year":"2024","unstructured":"Shengze Wang, Ziheng Wang, Ryan Schmelzle, Liujie Zheng, YoungJoong Kwon, Roni Sengupta, and Henry Fuchs. 2024. Learning ViewSynthesis for Desktop Telepresence with Few RGBD Cameras. IEEE TVCG (2024)."},{"key":"e_1_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00061"},{"key":"e_1_2_1_95_1","volume-title":"HumanNeRF: Free-Viewpoint Rendering of Moving People From Monocular Video. In IEEE Conf. Comput. Vis. Pattern Recog.","author":"Weng Chung-Yi","year":"2022","unstructured":"Chung-Yi Weng, Brian Curless, Pratul P. Srinivasan, Jonathan T. Barron, and Ira Kemelmacher-Shlizerman. 2022. HumanNeRF: Free-Viewpoint Rendering of Moving People From Monocular Video. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_96_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01234-2_1"},{"key":"e_1_2_1_97_1","volume-title":"IEEE Conf. Comput. Vis. Pattern Recog.","author":"Wu Guanjun","year":"2024","unstructured":"Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Wang Xinggang. 2024. 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_98_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00175"},{"key":"e_1_2_1_99_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00930"},{"key":"e_1_2_1_100_1","doi-asserted-by":"crossref","unstructured":"D. Xiang F. Prada C. Wu and J. Hodgins. 2020. MonoClothCap: Towards Temporally Coherent Clothing Capture from Monocular RGB Video. In 3DV.","DOI":"10.1109\/3DV50981.2020.00042"},{"key":"e_1_2_1_101_1","volume-title":"IEEE Conf. Comput. Vis. Pattern Recog.","author":"Xiu Yuliang","unstructured":"Yuliang Xiu, Jinlong Yang, Dimitrios Tzionas, and Michael J. Black. 2022. ICON: Implicit Clothed humans Obtained from Normals. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_102_1","unstructured":"Weipeng Xu Avishek Chatterjee Michael Zollh\u00f6fer Helge Rhodin Dushyant Mehta Hans-Peter Seidel and Christian Theobalt. 2018. MonoPerfCap: Human Performance Capture From Monocular Video. ACM Trans. Graph. (2018)."},{"key":"e_1_2_1_103_1","volume-title":"IEEE Conf. Comput. Vis. Pattern Recog.","author":"Xu Yuelang","year":"2024","unstructured":"Yuelang Xu, Benwang Chen, Zhe Li, Hongwen Zhang, Lizhen Wang, Zerong Zheng, and Yebin Liu. 2024a. Gaussian Head Avatar: Ultra High-fidelity Head Avatar via Dynamic Gaussians. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_104_1","volume-title":"IEEE Conf. Comput. Vis. Pattern Recog.","author":"Xu Zhen","year":"2024","unstructured":"Zhen Xu, Sida Peng, Haotong Lin, Guangzhao He, Jiaming Sun, Yujun Shen, Hujun Bao, and Xiaowei Zhou. 2024b. 4K4D: Real-Time 4D View Synthesis at 4K Resolution. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_105_1","volume-title":"Yu Chen, and Gim Hee Lee.","author":"Yan Zhiwen","year":"2023","unstructured":"Zhiwen Yan, Weng Fei Low, Yu Chen, and Gim Hee Lee. 2023. Multi-Scale 3D Gaussian Splatting for Anti-Aliased Rendering. arXiv:2311.17089 (2023)."},{"key":"e_1_2_1_106_1","volume-title":"Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. arXiv:2309.13101","author":"Yang Ziyi","year":"2023","unstructured":"Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin. 2023. Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction. arXiv:2309.13101 (2023)."},{"key":"e_1_2_1_107_1","volume-title":"GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation. arXiv:2403.14621","author":"Yinghao Xu","year":"2024","unstructured":"Xu Yinghao, Shi Zifan, Yifan Wang, Chen Hansheng, Yang Ceyuan, Peng Sida, Shen Yujun, and Wetzstein Gordon. 2024. GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generation. arXiv:2403.14621 (2024)."},{"key":"e_1_2_1_108_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00455"},{"key":"e_1_2_1_109_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00569"},{"key":"e_1_2_1_110_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00761"},{"key":"e_1_2_1_111_1","volume-title":"IEEE Conf. Comput. Vis. Pattern Recog.","author":"Yu Zehao","year":"2024","unstructured":"Zehao Yu, Anpei Chen, Binbin Huang, Torsten Sattler, and Andreas Geiger. 2024a. Mip-Splatting: Alias-free 3D Gaussian Splatting. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_112_1","volume-title":"Gaussian Opacity Fields: Efficient High-quality Compact Surface Reconstruction in Unbounded Scenes. arXiv:2404.10772","author":"Yu Zehao","year":"2024","unstructured":"Zehao Yu, Torsten Sattler, and Andreas Geiger. 2024b. Gaussian Opacity Fields: Efficient High-quality Compact Surface Reconstruction in Unbounded Scenes. arXiv:2404.10772 (2024)."},{"key":"e_1_2_1_113_1","volume-title":"NeuVV: Neural Volumetric Videos with Immersive Rendering and Editing. arXiv:2202.06088","author":"Zhang Jiakai","year":"2022","unstructured":"Jiakai Zhang, Liao Wang, Xinhang Liu, Fuqiang Zhao, Minzhang Li, Haizhao Dai, Boyuan Zhang, Wei Yang, Lan Xu, and Jingyi Yu. 2022. NeuVV: Neural Volumetric Videos with Immersive Rendering and Editing. arXiv:2202.06088 (2022)."},{"key":"e_1_2_1_114_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00068"},{"key":"e_1_2_1_115_1","doi-asserted-by":"crossref","unstructured":"Fuqiang Zhao Yuheng Jiang Kaixin Yao Jiakai Zhang Liao Wang Haizhao Dai Yuhui Zhong Yingliang Zhang Minye Wu Lan Xu and Jingyi Yu. 2022a. Human Performance Modeling and Rendering via Neural Animated Mesh. ACM Trans. Graph. (2022).","DOI":"10.1145\/3550454.3555451"},{"key":"e_1_2_1_116_1","volume-title":"HumanNeRF: Efficiently Generated Human Radiance Field From Sparse Inputs. In IEEE Conf. Comput. Vis. Pattern Recog.","author":"Zhao Fuqiang","year":"2022","unstructured":"Fuqiang Zhao, Wei Yang, Jiakai Zhang, Pei Lin, Yingliang Zhang, Jingyi Yu, and Lan Xu. 2022b. HumanNeRF: Efficiently Generated Human Radiance Field From Sparse Inputs. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_117_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01544"},{"key":"e_1_2_1_118_1","volume-title":"IEEE Conf. Comput. Vis. Pattern Recog.","author":"Zheng Shunyuan","year":"2024","unstructured":"Shunyuan Zheng, Boyao Zhou, Ruizhi Shao, Boning Liu, Shengping Zhang, Liqiang Nie, and Yebin Liu. 2024. GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis. In IEEE Conf. Comput. Vis. Pattern Recog."},{"key":"e_1_2_1_119_1","volume-title":"PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction","author":"Zheng Zerong","year":"2021","unstructured":"Zerong Zheng, Tao Yu, Yebin Liu, and Dai Qionghai. 2021. PaMIR: Parametric Model-Conditioned Implicit Representation for Image-based Human Reconstruction. IEEE Trans. Pattern Anal. Mach. Intell. (2021)."},{"key":"e_1_2_1_120_1","volume-title":"HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting. arXiv:2402.06149","author":"Zhou Zhenglin","year":"2024","unstructured":"Zhenglin Zhou, Fan Ma, Hehe Fan, and Yi Yang. 2024. HeadStudio: Text to Animatable Head Avatars with 3D Gaussian Splatting. arXiv:2402.06149 (2024)."},{"key":"e_1_2_1_121_1","volume-title":"Drivable 3D Gaussian Avatars. arXiv:2311.08581","author":"Zielonka Wojciech","year":"2023","unstructured":"Wojciech Zielonka, Timur Bagautdinov, Shunsuke Saito, Michael Zollh\u00f6fer, Justus Thies, and Javier Romero. 2023. Drivable 3D Gaussian Avatars. arXiv:2311.08581 (2023)."}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3687993","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3687993","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:09:58Z","timestamp":1750295398000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3687993"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,19]]},"references-count":121,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,12,19]]}},"alternative-id":["10.1145\/3687993"],"URL":"https:\/\/doi.org\/10.1145\/3687993","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,19]]},"assertion":[{"value":"2024-11-19","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}