{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T06:04:36Z","timestamp":1784268276822,"version":"3.55.0"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2016,7,11]],"date-time":"2016-07-11T00:00:00Z","timestamp":1468195200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2016,7,11]]},"abstract":"<jats:p>We contribute a new pipeline for live multi-view performance capture, generating temporally coherent high-quality reconstructions in real-time. Our algorithm supports both incremental reconstruction, improving the surface estimation over time, as well as parameterizing the nonrigid scene motion. Our approach is highly robust to both large frame-to-frame motion and topology changes, allowing us to reconstruct extremely challenging scenes. We demonstrate advantages over related real-time techniques that either deform an online generated template or continually fuse depth data nonrigidly into a single reference model. Finally, we show geometric reconstruction results on par with offline methods which require orders of magnitude more processing time and many more RGBD cameras.<\/jats:p>","DOI":"10.1145\/2897824.2925969","type":"journal-article","created":{"date-parts":[[2016,7,11]],"date-time":"2016-07-11T16:04:33Z","timestamp":1468253073000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":427,"title":["Fusion4D"],"prefix":"10.1145","volume":"35","author":[{"given":"Mingsong","family":"Dou","sequence":"first","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sameh","family":"Khamis","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yury","family":"Degtyarev","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philip","family":"Davidson","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sean Ryan","family":"Fanello","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adarsh","family":"Kowdle","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sergio Orts","family":"Escolano","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christoph","family":"Rhemann","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Kim","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jonathan","family":"Taylor","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pushmeet","family":"Kohli","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vladimir","family":"Tankovich","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shahram","family":"Izadi","sequence":"additional","affiliation":[{"name":"Microsoft Research"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2016,7,11]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964970"},{"key":"e_1_2_2_2_1","first-page":"1","article-title":"Patchmatch stereo: Stereo matching with slanted support windows","volume":"11","author":"Bleyer M.","year":"2011","journal-title":"Proc. BMVC"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.265"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2185520.2185549"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360698"},{"key":"e_1_2_2_6_1","volume-title":"Proc. CVPR.","author":"Cagniart C."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1016\/0262-8856(92)90066-C"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2461940"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766945"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/237170.237269"},{"key":"e_1_2_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360697"},{"key":"e_1_2_2_12_1","volume-title":"-M","author":"Dou M.","year":"2013"},{"key":"e_1_2_2_13_1","doi-asserted-by":"crossref","unstructured":"Dou M. Taylor J. Fuchs H. Fitzgibbon A. and Izadi S. 2015. 3d scanning deformable objects with a single rgbd sensor. In CVPR. Dou M. Taylor J. Fuchs H. Fitzgibbon A. and Izadi S. 2015. 3d scanning deformable objects with a single rgbd sensor. In CVPR .","DOI":"10.1109\/CVPR.2015.7298647"},{"key":"e_1_2_2_14_1","unstructured":"Engels C. Stew\u00e9nius H. and Nist\u00e9r D. 2006. Bundle adjustment rules. Photogrammetric computer vision 2 124--131. Engels C. Stew\u00e9nius H. and Nist\u00e9r D. 2006. Bundle adjustment rules. Photogrammetric computer vision 2 124--131."},{"key":"e_1_2_2_15_1","volume-title":"-P","author":"Gall J.","year":"2009"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.353"},{"key":"e_1_2_2_17_1","unstructured":"Kr\u00e4henb\u00fch P. and Koltun V. 2011. Efficient inference in fully connected crfs with gaussian edge potentials. NIPS. Kr\u00e4henb\u00fch P. and Koltun V. 2011. Efficient inference in fully connected crfs with gaussian edge potentials. NIPS ."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008191222954"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618521"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1023\/B:VISI.0000029664.99615.94"},{"key":"e_1_2_2_21_1","volume-title":"Proc. SGP, 173--182","author":"Mitra N. J."},{"key":"e_1_2_2_22_1","first-page":"98","article-title":"The uncanny valley {from the field}. Robotics & Automation Magazine","volume":"19","author":"Mori M.","year":"2012","journal-title":"IEEE"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"e_1_2_2_24_1","volume-title":"Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time. In CVPR, 343--352.","author":"Newcombe R. A.","year":"2015"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0818-9"},{"key":"e_1_2_2_26_1","volume-title":"Proc. ISMAR, IEEE, 83--88","author":"Pradeep V."},{"key":"e_1_2_2_27_1","volume-title":"Epicflow: Edge-preserving interpolation of correspondences for optical flow. CVPR.","author":"Revaud J.","year":"2015"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2005.104"},{"key":"e_1_2_2_29_1","doi-asserted-by":"crossref","unstructured":"Rusinkiewicz S. and Levoy M. 2001. Efficient variants of the icp algorithm. In 3DIM 145--152. Rusinkiewicz S. and Levoy M. 2001. Efficient variants of the icp algorithm. In 3DIM 145--152.","DOI":"10.1109\/IM.2001.924423"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.377"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2010.09.005"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2007.68"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126338"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276478"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2159516.2159517"},{"key":"e_1_2_2_36_1","doi-asserted-by":"crossref","unstructured":"Theobalt C. de Aguiar E. Stoll C. Seidel H.-P. and Thrun S. 2010. Performance capture from multi-view video. In Image and Geometry Processing for 3D-Cinematography R. Ronfard and G. Taubin Eds. Springer 127ff. Theobalt C. de Aguiar E. Stoll C. Seidel H.-P. and Thrun S. 2010. Performance capture from multi-view video. In Image and Geometry Processing for 3D-Cinematography R. Ronfard and G. Taubin Eds. Springer 127ff.","DOI":"10.1007\/978-3-642-12392-4_6"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33715-4_3"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1399504.1360696"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618520"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1516522.1516526"},{"key":"e_1_2_2_41_1","doi-asserted-by":"crossref","unstructured":"Wang S. Fanello S. R. Rhemann C. Izadi S. and Kohli P. 2016. The global patch collider. CVPR. Wang S. Fanello S. R. Rhemann C. Izadi S. and Kohli P. 2016. The global patch collider. CVPR .","DOI":"10.1109\/CVPR.2016.21"},{"key":"e_1_2_2_42_1","volume-title":"Proc. Pacific Graphics, 629--638","author":"Waschb\u00fcsch M."},{"key":"e_1_2_2_43_1","doi-asserted-by":"crossref","unstructured":"Wei L. Huang Q. Ceylan D. Vouga E. and Li H. 2015. Dense human body correspondences using convolutional networks. arXiv preprint arXiv:1511.05904. Wei L. Huang Q. Ceylan D. Vouga E. and Li H. 2015. Dense human body correspondences using convolutional networks. arXiv preprint arXiv:1511.05904 .","DOI":"10.1109\/CVPR.2016.171"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2013.175"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.301"},{"key":"e_1_2_2_46_1","doi-asserted-by":"crossref","unstructured":"Ye M. Zhang Q. Wang L. Zhu J. Yang R. and Gall J. 2013. A survey on human motion analysis from depth data. In Time-of-Flight and Depth Imaging. Sensors Algorithms and Applications. Springer 149--187. Ye M. Zhang Q. Wang L. Zhu J. Yang R. and Gall J. 2013. A survey on human motion analysis from depth data. In Time-of-Flight and Depth Imaging. Sensors Algorithms and Applications . Springer 149--187.","DOI":"10.1007\/978-3-642-44964-2_8"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10602-1_50"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.26"},{"key":"e_1_2_2_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.92"},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601165"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2897824.2925969","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2897824.2925969","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:55:04Z","timestamp":1750222504000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2897824.2925969"}},"subtitle":["real-time performance capture of challenging scenes"],"short-title":[],"issued":{"date-parts":[[2016,7,11]]},"references-count":50,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2016,7,11]]}},"alternative-id":["10.1145\/2897824.2925969"],"URL":"https:\/\/doi.org\/10.1145\/2897824.2925969","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,7,11]]},"assertion":[{"value":"2016-07-11","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}