{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T15:54:19Z","timestamp":1782834859975,"version":"3.54.5"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2017,11,20]],"date-time":"2017-11-20T00:00:00Z","timestamp":1511136000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2017,12,31]]},"abstract":"<jats:p>We present Motion2Fusion, a state-of-the-art 360 performance capture system that enables *real-time* reconstruction of arbitrary non-rigid scenes. We provide three major contributions over prior work: 1) a new non-rigid fusion pipeline allowing for far more faithful reconstruction of high frequency geometric details, avoiding the over-smoothing and visual artifacts observed previously. 2) a high speed pipeline coupled with a machine learning technique for 3D correspondence field estimation reducing tracking errors and artifacts that are attributed to fast motions. 3) a backward and forward non-rigid alignment strategy that more robustly deals with topology changes but is still free from scene priors. Our novel performance capture system demonstrates real-time results nearing 3x speed-up from previous state-of-the-art work on the exact same GPU hardware. Extensive quantitative and qualitative comparisons show more precise geometric and texturing results with less artifacts due to fast motions or topology changes than prior art.<\/jats:p>","DOI":"10.1145\/3130800.3130801","type":"journal-article","created":{"date-parts":[[2017,11,22]],"date-time":"2017-11-22T16:25:08Z","timestamp":1511367908000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":157,"title":["Motion2fusion"],"prefix":"10.1145","volume":"36","author":[{"given":"Mingsong","family":"Dou","sequence":"first","affiliation":[{"name":"perceptiveIO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Philip","family":"Davidson","sequence":"additional","affiliation":[{"name":"perceptiveIO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sean Ryan","family":"Fanello","sequence":"additional","affiliation":[{"name":"perceptiveIO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sameh","family":"Khamis","sequence":"additional","affiliation":[{"name":"perceptiveIO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Adarsh","family":"Kowdle","sequence":"additional","affiliation":[{"name":"perceptiveIO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christoph","family":"Rhemann","sequence":"additional","affiliation":[{"name":"perceptiveIO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vladimir","family":"Tankovich","sequence":"additional","affiliation":[{"name":"perceptiveIO"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shahram","family":"Izadi","sequence":"additional","affiliation":[{"name":"perceptiveIO"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2017,11,20]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.457"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276467"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766943"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2461912.2462012"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/508357.508360"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766945"},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766945"},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/237170.237269"},{"key":"e_1_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925969"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601223"},{"key":"e_1_2_2_11_1","volume-title":"Hyperdepth: Learning depth from structured light without matching. In CVPR.","author":"Fanello Sean Ryan","year":"2016"},{"key":"e_1_2_2_12_1","doi-asserted-by":"crossref","unstructured":"Sean Ryan Fanello Julien Valentin Adarsh Kowdle Christoph Rhemann Vladimir Tankovich Carlo Ciliberto Philip Davidson and Shahram Izadi. 2017a. Low Compute and Fully Parallel Computer Vision with HashMatch. In ICCV. Sean Ryan Fanello Julien Valentin Adarsh Kowdle Christoph Rhemann Vladimir Tankovich Carlo Ciliberto Philip Davidson and Shahram Izadi. 2017a. Low Compute and Fully Parallel Computer Vision with HashMatch. In ICCV.","DOI":"10.1109\/ICCV.2017.418"},{"key":"e_1_2_2_13_1","doi-asserted-by":"crossref","unstructured":"Sean Ryan Fanello Julien Valentin Christoph Rhemann Adarsh Kowdle Vladimir Tankovich Philip Davidson and Shahram Izadi. 2017b. UltraStereo: Efficient Learning-based Matching for Active Stereo Systems. In CVPR. Sean Ryan Fanello Julien Valentin Christoph Rhemann Adarsh Kowdle Vladimir Tankovich Philip Davidson and Shahram Izadi. 2017b. UltraStereo: Efficient Learning-based Matching for Active Stereo Systems. In CVPR.","DOI":"10.1109\/CVPR.2017.692"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.316"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.353"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3083722"},{"key":"e_1_2_2_17_1","doi-asserted-by":"crossref","unstructured":"Matthias Innmann Michael Zollh\u00f6fer Matthias Nie\u00dfner Christian Theobalt and Marc Stamminger. 2016. VolumeDeform: Real-time volumetric non-rigid reconstruction. In ECCV. 362--379. Matthias Innmann Michael Zollh\u00f6fer Matthias Nie\u00dfner Christian Theobalt and Marc Stamminger. 2016. VolumeDeform: Real-time volumetric non-rigid reconstruction. In ECCV. 362--379.","DOI":"10.1007\/978-3-319-46484-8_22"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/SMI.2006.31"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1230100.1230107"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2006.68"},{"key":"e_1_2_2_21_1","unstructured":"Marius Leordeanu Martial Hebert and Rahul Sukthankar. 2009. An Integer Projected Fixed Point Method for Graph Matching and MAP Inference. In NIPS. Marius Leordeanu Martial Hebert and Rahul Sukthankar. 2009. An Integer Projected Fixed Point Method for Graph Matching and MAP Inference. In NIPS."},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/566654.566590"},{"key":"e_1_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618521"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508407"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3072959.3073596"},{"key":"e_1_2_2_26_1","unstructured":"Mark Meyer Mathieu Desbrun Peter Schr\u00f6der and Alan H Barr. 2002. Discrete differential-geometry operators for triangulated 2-manifolds. Visualization and mathematics 3 2 (2002) 52--58. Mark Meyer Mathieu Desbrun Peter Schr\u00f6der and Alan H Barr. 2002. Discrete differential-geometry operators for triangulated 2-manifolds. Visualization and mathematics 3 2 (2002) 52--58."},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298631"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2984511.2984517"},{"key":"e_1_2_2_30_1","doi-asserted-by":"crossref","unstructured":"Charles Ruizhongtai Qi Hao Su Kaichun Mo and Leonidas J. Guibas. 2016. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In CVPR. Charles Ruizhongtai Qi Hao Su Kaichun Mo and Leonidas J. Guibas. 2016. PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation. In CVPR.","DOI":"10.1109\/CVPR.2017.16"},{"key":"e_1_2_2_31_1","unstructured":"Ali Rahimi and Benjamin Recht. 2007. Random Features for Large-scale Kernel Machines. In NIPS. 5. Ali Rahimi and Benjamin Recht. 2007. Random Features for Large-scale Kernel Machines. In NIPS. 5."},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383307"},{"key":"e_1_2_2_33_1","doi-asserted-by":"crossref","unstructured":"Alla Sheffer and John C Hart. 2002. Seamster: inconspicuous low-distortion texture seam layout. In Visualization. 291--298. Alla Sheffer and John C Hart. 2002. Seamster: inconspicuous low-distortion texture seam layout. In Visualization. 291--298.","DOI":"10.1109\/VISUAL.2002.1183787"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2398356.2398381"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1007\/s003710050082"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276478"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.605"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925965"},{"key":"e_1_2_2_39_1","doi-asserted-by":"crossref","unstructured":"C. Theobalt E. de Aguiar C. Stoll H.-P. Seidel and S. Thrun. 2010. Performance Capture from Multi-view Video. In Image and Geometry Processing for 3D-Cinematography R. Ronfard and G. Taubin (Eds.). Springer 127ff. C. Theobalt E. de Aguiar C. Stoll H.-P. Seidel and S. Thrun. 2010. Performance Capture from Multi-view Video. In Image and Geometry Processing for 3D-Cinematography R. Ronfard and G. Taubin (Eds.). Springer 127ff.","DOI":"10.1007\/978-3-642-12392-4_6"},{"key":"e_1_2_2_40_1","doi-asserted-by":"crossref","unstructured":"J. Thies M. Zollh\u00f6fer M. Stamminger C. Theobalt and M. Nie\u00dfner. 2016. Face2Face: Real-time Face Capture and Reenactment of RGB Videos. In CVPR. J. Thies M. Zollh\u00f6fer M. Stamminger C. Theobalt and M. Nie\u00dfner. 2016. Face2Face: Real-time Face Capture and Reenactment of RGB Videos. In CVPR.","DOI":"10.1109\/CVPR.2016.262"},{"key":"e_1_2_2_41_1","doi-asserted-by":"crossref","unstructured":"Shenlong Wang Sean Ryan Fanello Christoph Rhemann Shahram Izadi and Pushmeet Kohli. 2016. The Global Patch Collider. In CVPR. 127--135. Shenlong Wang Sean Ryan Fanello Christoph Rhemann Shahram Izadi and Pushmeet Kohli. 2016. The Global Patch Collider. In CVPR. 127--135.","DOI":"10.1109\/CVPR.2016.21"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.acha.2007.12.002"},{"key":"e_1_2_2_43_1","volume-title":"2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1275--1283","author":"Xie Jin","year":"2015"},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.301"},{"key":"e_1_2_2_45_1","volume-title":"Time-of-Flight and Depth Imaging. Sensors, Algorithms, and Applications","author":"Ye Mao"},{"key":"e_1_2_2_46_1","doi-asserted-by":"crossref","unstructured":"Sergey Zagoruyko and Nikos Komodakis. 2015. Learning to Compare Image Patches via Convolutional Neural Networks. In CVPR. 4353--4361. Sergey Zagoruyko and Nikos Komodakis. 2015. Learning to Compare Image Patches via Convolutional Neural Networks. In CVPR. 4353--4361.","DOI":"10.1109\/CVPR.2015.7299064"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2008.245"},{"key":"e_1_2_2_48_1","doi-asserted-by":"crossref","unstructured":"Jure \u017dbontar and Yann LeCun. 2015. Computing the stereo matching cost with a convolutional neural network. In CVPR. 1592--1599. Jure \u017dbontar and Yann LeCun. 2015. Computing the stereo matching cost with a convolutional neural network. In CVPR. 1592--1599.","DOI":"10.1109\/CVPR.2015.7298767"},{"key":"e_1_2_2_49_1","unstructured":"F. Zhou and F. De la Torre. 2012. Factorized graph matching. In CVPR. 127--134. F. Zhou and F. De la Torre. 2012. Factorized graph matching. In CVPR. 127--134."},{"key":"e_1_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/1057432.1057439"},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601165"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3130800.3130801","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3130800.3130801","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,27]],"date-time":"2025-06-27T12:06:07Z","timestamp":1751025967000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3130800.3130801"}},"subtitle":["real-time volumetric performance capture"],"short-title":[],"issued":{"date-parts":[[2017,11,20]]},"references-count":51,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2017,12,31]]}},"alternative-id":["10.1145\/3130800.3130801"],"URL":"https:\/\/doi.org\/10.1145\/3130800.3130801","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,11,20]]},"assertion":[{"value":"2017-11-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}