{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T15:48:21Z","timestamp":1780501701955,"version":"3.54.1"},"reference-count":101,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2019,3,13]],"date-time":"2019-03-13T00:00:00Z","timestamp":1552435200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100010663","name":"European Research Council","doi-asserted-by":"publisher","award":["770784"],"award-info":[{"award-number":["770784"]}],"id":[{"id":"10.13039\/100010663","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2019,4,30]]},"abstract":"<jats:p>We present the first real-time human performance capture approach that reconstructs dense, space-time coherent deforming geometry of entire humans in general everyday clothing from just a single RGB video. We propose a novel two-stage analysis-by-synthesis optimization whose formulation and implementation are designed for high performance. In the first stage, a skinned template model is jointly fitted to background subtracted input video, 2D and 3D skeleton joint positions found using a deep neural network, and a set of sparse facial landmark detections. In the second stage, dense non-rigid 3D deformations of skin and even loose apparel are captured based on a novel real-time capable algorithm for non-rigid tracking using dense photometric and silhouette constraints. Our novel energy formulation leverages automatically identified material regions on the template to model the differing non-rigid deformation behavior of skin and apparel. The two resulting non-linear optimization problems per frame are solved with specially tailored data-parallel Gauss-Newton solvers. To achieve real-time performance of over 25Hz, we design a pipelined parallel architecture using the CPU and two commodity GPUs. Our method is the first real-time monocular approach for full-body performance capture. Our method yields comparable accuracy with off-line performance capture techniques while being orders of magnitude faster.<\/jats:p>","DOI":"10.1145\/3311970","type":"journal-article","created":{"date-parts":[[2019,3,14]],"date-time":"2019-03-14T17:11:58Z","timestamp":1552583518000},"page":"1-17","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":212,"title":["LiveCap"],"prefix":"10.1145","volume":"38","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3899-7515","authenticated-orcid":false,"given":"Marc","family":"Habermann","sequence":"first","affiliation":[{"name":"Max Planck Institute for Informatics"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9548-5108","authenticated-orcid":false,"given":"Weipeng","family":"Xu","sequence":"additional","affiliation":[{"name":"Max Planck Institute for Informatics"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michael","family":"Zollh\u00f6fer","sequence":"additional","affiliation":[{"name":"Stanford University, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gerard","family":"Pons-Moll","sequence":"additional","affiliation":[{"name":"Max Planck Institute for Informatics"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christian","family":"Theobalt","sequence":"additional","affiliation":[{"name":"Max Planck Institute for Informatics"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,3,13]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298623"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073207"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-88688-4_2"},{"key":"e_1_2_2_4_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201907)","author":"Balan Alexandru O."},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2392759"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.265"},{"key":"e_1_2_2_7_1","volume-title":"Proceedings of the European Conference on Computer Vision (ECCV\u201916)","author":"Bogo Federica"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1007\/11744047_49"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2009.32"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5539814"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766943"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/882262.882309"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00371-013-0775-7"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766945"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360697"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130801"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925969"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206755"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.168"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2890493"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.715"},{"key":"e_1_2_2_22_1","volume-title":"Proceedings of the 2009 IEEE 12th International Conference on Computer Vision (ICCV 09)","author":"Guan Peng"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00762"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2018.00074"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3083722"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-34263-9_6"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5539853"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.141"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-01811-4_9"},{"key":"e_1_2_2_30_1","volume-title":"Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201916)","author":"Huang C.-H."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2017.00055"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_22"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2047196.2047270"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1882261.1866174"},{"key":"e_1_2_2_35_1","unstructured":"Hanbyul Joo Tomas Simon and Yaser Sheikh. 2018. Total capture: A 3D deformation model for tracking faces hands and bodies. arXiv:1801.01615.  Hanbyul Joo Tomas Simon and Yaser Sheikh. 2018. Total capture: A 3D deformation model for tracking faces hands and bodies. arXiv:1801.01615."},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/2980179.2982438"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00744"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1230100.1230107"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073685"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275062"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1572741.1572749"},{"key":"e_1_2_2_42_1","volume-title":"Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201917)","author":"Lassner Christoph"},{"key":"e_1_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.336"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995424"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818013"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/344779.344951"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073596"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.216727"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.504"},{"key":"e_1_2_2_50_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201915)","author":"Newcombe Richard A."},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2984511.2984517"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360695"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1006\/cviu.2000.0891"},{"key":"e_1_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073711"},{"key":"e_1_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766993"},{"key":"e_1_2_2_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.501"},{"key":"e_1_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073679"},{"key":"e_1_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46454-1_31"},{"key":"e_1_2_2_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2016.25"},{"key":"e_1_2_2_61_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.134"},{"key":"e_1_2_2_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2634212"},{"key":"e_1_2_2_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130800.3130883"},{"key":"e_1_2_2_64_1","volume-title":"Video Pop-Up: Monocular 3D Reconstruction of Dynamic Scenes","author":"Russell Chris"},{"key":"e_1_2_2_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.158"},{"key":"e_1_2_2_66_1","volume-title":"Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. 1034--1041","author":"Saragih J. M."},{"key":"e_1_2_2_67_1","volume-title":"Proceedings of the International Conference on 3D Body Scanning Technologies. 406--413","author":"Sekine M."},{"key":"e_1_2_2_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2004.1315063"},{"key":"e_1_2_2_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.581"},{"key":"e_1_2_2_70_1","doi-asserted-by":"publisher","DOI":"10.5555\/1965841.1965850"},{"key":"e_1_2_2_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2007.68"},{"key":"e_1_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.284"},{"key":"e_1_2_2_73_1","volume-title":"Robust articulated-ICP for real-time hand tracking. Computer Graphics Forum 34, 5","author":"Tagliasacchi Andrea","year":"2015"},{"key":"e_1_2_2_74_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.425"},{"key":"e_1_2_2_75_1","volume-title":"Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201916)","author":"Tekin B."},{"key":"e_1_2_2_76_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.603"},{"key":"e_1_2_2_77_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01234-2_2"},{"key":"e_1_2_2_78_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360696"},{"key":"e_1_2_2_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618520"},{"key":"e_1_2_2_80_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46478-7_17"},{"key":"e_1_2_2_81_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00371-005-0346-7"},{"key":"e_1_2_2_82_1","doi-asserted-by":"publisher","DOI":"10.1145\/2366145.2366207"},{"key":"e_1_2_2_83_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126465"},{"key":"e_1_2_2_84_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508418"},{"key":"e_1_2_2_85_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33765-9_54"},{"key":"e_1_2_2_86_1","doi-asserted-by":"publisher","DOI":"10.1145\/3181973"},{"key":"e_1_2_2_87_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_27"},{"key":"e_1_2_2_88_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33709-3_59"},{"key":"e_1_2_2_89_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.301"},{"key":"e_1_2_2_90_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.111"},{"key":"e_1_2_2_91_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.104"},{"key":"e_1_2_2_92_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00761"},{"key":"e_1_2_2_93_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.582"},{"key":"e_1_2_2_94_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661286"},{"key":"e_1_2_2_95_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.92"},{"key":"e_1_2_2_96_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601134"},{"key":"e_1_2_2_97_1","doi-asserted-by":"publisher","DOI":"10.1145\/1778765.1778863"},{"key":"e_1_2_2_98_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.51"},{"key":"e_1_2_2_99_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.537"},{"key":"e_1_2_2_100_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2005.11.005"},{"key":"e_1_2_2_101_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601165"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3311970","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3311970","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:54:28Z","timestamp":1750204468000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3311970"}},"subtitle":["Real-Time Human Performance Capture From Monocular Video"],"short-title":[],"issued":{"date-parts":[[2019,3,13]]},"references-count":101,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2019,4,30]]}},"alternative-id":["10.1145\/3311970"],"URL":"https:\/\/doi.org\/10.1145\/3311970","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,3,13]]},"assertion":[{"value":"2018-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-03-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}