{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T03:41:39Z","timestamp":1777520499676,"version":"3.51.4"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2017,6,1]],"date-time":"2017-06-01T00:00:00Z","timestamp":1496275200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"NSFC","doi-asserted-by":"crossref","award":["61671268, 61522111 and 61531014"],"award-info":[{"award-number":["61671268, 61522111 and 61531014"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National key foundation for exploring scientific instrument","award":["2013YQ140517"],"award-info":[{"award-number":["2013YQ140517"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2017,6,30]]},"abstract":"<jats:p>This article proposes a real-time method that uses a single-view RGB-D input (a depth sensor integrated with a color camera) to simultaneously reconstruct a casual scene with a detailed geometry model, surface albedo, per-frame non-rigid motion, and per-frame low-frequency lighting, without requiring any template or motion priors. The key observation is that accurate scene motion can be used to integrate temporal information to recover the precise appearance, whereas the intrinsic appearance can help to establish true correspondence in the temporal domain to recover motion. Based on this observation, we first propose a shading-based scheme to leverage appearance information for motion estimation. Then, using the reconstructed motion, a volumetric albedo fusing scheme is proposed to complete and refine the intrinsic appearance of the scene by incorporating information from multiple frames. Since the two schemes are iteratively applied during recording, the reconstructed appearance and motion become increasingly more accurate. In addition to the reconstruction results, our experiments also show that additional applications can be achieved, such as relighting, albedo editing, and free-viewpoint rendering of a dynamic scene, since geometry, appearance, and motion are all reconstructed by our technique.<\/jats:p>","DOI":"10.1145\/3083722","type":"journal-article","created":{"date-parts":[[2017,6,5]],"date-time":"2017-06-05T12:50:00Z","timestamp":1496667000000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":60,"title":["Real-Time Geometry, Albedo, and Motion Reconstruction Using a Single RGB-D Camera"],"prefix":"10.1145","volume":"36","author":[{"given":"Kaiwen","family":"Guo","sequence":"first","affiliation":[{"name":"Tsinghua University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Feng","family":"Xu","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, P.R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tao","family":"Yu","sequence":"additional","affiliation":[{"name":"Beihang University and Tsinghua University, Beijing, P.R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaoyang","family":"Liu","sequence":"additional","affiliation":[{"name":"Tsinghua University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qionghai","family":"Dai","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, P.R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yebin","family":"Liu","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, P.R. China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2017,6]]},"reference":[{"key":"e_1_2_2_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360698"},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5539814"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766945"},{"key":"e_1_2_2_4_1","volume-title":"SIGGRAPH","author":"Curless Brian","unstructured":"Brian Curless and Marc Levoy . 1996. A volumetric method for building complex models from range images . In SIGGRAPH . ACM , New York, NY , 303--312. DOI:https:\/\/doi.org\/10.1145\/237170.237269 10.1145\/237170.237269 Brian Curless and Marc Levoy. 1996. A volumetric method for building complex models from range images. In SIGGRAPH. ACM, New York, NY, 303--312. DOI:https:\/\/doi.org\/10.1145\/237170.237269"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360697"},{"key":"e_1_2_2_6_1","unstructured":"Julie Dorsey Holly Rushmeier and Fran\u00e7ois Sillion. 2010. Digital Modeling of Material Appearance. Morgan Kaufmann. Julie Dorsey Holly Rushmeier and Fran\u00e7ois Sillion. 2010. Digital Modeling of Material Appearance. Morgan Kaufmann."},{"key":"e_1_2_2_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897824.2925969"},{"key":"e_1_2_2_8_1","doi-asserted-by":"crossref","unstructured":"Mingsong Dou Jonathan Taylor Henry Fuchs Andrew Fitzgibbon and Shahram Izadi. 2015. 3D scanning deformable objects with a single RGB-D sensor. In CVPR. 493--501. Mingsong Dou Jonathan Taylor Henry Fuchs Andrew Fitzgibbon and Shahram Izadi. 2015. 3D scanning deformable objects with a single RGB-D sensor. In CVPR. 493--501.","DOI":"10.1109\/CVPR.2015.7298647"},{"key":"e_1_2_2_9_1","volume-title":"Debevec","author":"Einarsson Per","year":"2006","unstructured":"Per Einarsson , Charles-Felix Chabert , Andrew Jones , Wan-Chun Ma , Bruce Lamond , Tim Hawkins , Mark T. Bolas , Sebastian Sylwan , and Paul E . Debevec . 2006 . Relighting human locomotion with flowed reflectance fields. Rendering Techniques 2006 (2006), Vol. 17 . Per Einarsson, Charles-Felix Chabert, Andrew Jones, Wan-Chun Ma, Bruce Lamond, Tim Hawkins, Mark T. Bolas, Sebastian Sylwan, and Paul E. Debevec. 2006. Relighting human locomotion with flowed reflectance fields. Rendering Techniques 2006 (2006), Vol. 17."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.353"},{"key":"e_1_2_2_11_1","first-page":"1","article-title":"Robust non-rigid motion tracking and surface reconstruction using L0 regularization","volume":"99","author":"Guo Kaiwen","year":"2017","unstructured":"Kaiwen Guo , Feng Xu , Yangang Wang , Yebin Liu , and Qionghai Dai . 2017 . Robust non-rigid motion tracking and surface reconstruction using L0 regularization . IEEE Transactions on Visualization and Computer Graphics PP , 99 (2017), 1 -- 1 . Kaiwen Guo, Feng Xu, Yangang Wang, Yebin Liu, and Qionghai Dai. 2017. Robust non-rigid motion tracking and surface reconstruction using L0 regularization. IEEE Transactions on Visualization and Computer Graphics PP, 99 (2017), 1--1.","journal-title":"IEEE Transactions on Visualization and Computer Graphics PP"},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126241"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-10605-2_26"},{"key":"e_1_2_2_14_1","doi-asserted-by":"crossref","unstructured":"Matthias Innmann Michael Zollh\u00f6fer Matthias Nie\u00dfner Christian Theobalt and Marc Stamminger. 2016. VolumeDeform: Real-time volumetric non-rigid reconstruction. In ECCV. Matthias Innmann Michael Zollh\u00f6fer Matthias Nie\u00dfner Christian Theobalt and Marc Stamminger. 2016. VolumeDeform: Real-time volumetric non-rigid reconstruction. In ECCV.","DOI":"10.1007\/978-3-319-46484-8_22"},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.381"},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12047"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618521"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508407"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2009.5459161"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2009.88"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2013.47"},{"key":"e_1_2_2_22_1","volume-title":"A Mathematical Introduction to Robotic Manipulation","author":"Murray Richard M.","unstructured":"Richard M. Murray , Zexiang Li , S. Shankar Sastry , and S Shankara Sastry . 1994. A Mathematical Introduction to Robotic Manipulation . CRC Press . Richard M. Murray, Zexiang Li, S. Shankar Sastry, and S Shankara Sastry. 1994. A Mathematical Introduction to Robotic Manipulation. CRC Press."},{"key":"e_1_2_2_23_1","volume-title":"Seitz","author":"Newcombe Richard A.","year":"2015","unstructured":"Richard A. Newcombe , Dieter Fox , and Steven M . Seitz . 2015 . DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time. In CVPR. Richard A. Newcombe, Dieter Fox, and Steven M. Seitz. 2015. DynamicFusion: Reconstruction and tracking of non-rigid scenes in real-time. In CVPR."},{"key":"e_1_2_2_24_1","unstructured":"Bj\u00f6rn Nutti \u00c5sa Kronander Mattias Nilsing Kristofer Maad Cristina Svensson and Hao Li. 2014. Depth sensor-based realtime tumor tracking for accurate radiation therapy. In Eurographics (Short Papers). Citeseer 1--4. Bj\u00f6rn Nutti \u00c5sa Kronander Mattias Nilsing Kristofer Maad Cristina Svensson and Hao Li. 2014. Depth sensor-based realtime tumor tracking for accurate radiation therapy. In Eurographics (Short Papers). Citeseer 1--4."},{"key":"e_1_2_2_25_1","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5407--5416","author":"Or-El Roy","year":"2015","unstructured":"Roy Or-El , Guy Rosman , Aaron Wetzler , Ron Kimmel , and Alfred M Bruckstein . 2015 . Rgbd-fusion: Real-time high precision depth recovery . In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5407--5416 . Roy Or-El, Guy Rosman, Aaron Wetzler, Ron Kimmel, and Alfred M Bruckstein. 2015. Rgbd-fusion: Real-time high precision depth recovery. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5407--5416."},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/383259.383271"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/IM.2001.924423"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409060.1409063"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2007.68"},{"key":"e_1_2_2_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.249"},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2007.1006"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/2816795.2818056"},{"key":"e_1_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360696"},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618520"},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1516522.1516526"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2012.03227.x"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508418"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126358"},{"key":"e_1_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33765-9_54"},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2661229.2661232"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2015.2498617"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.12600"},{"key":"e_1_2_2_43_1","volume-title":"Towards building an RGBD-M scanner. arXiv Preprint arXiv:1603.03875","author":"Wu Zhe","year":"2016","unstructured":"Zhe Wu , Sai-Kit Yeung , and Ping Tan . 2016. Towards building an RGBD-M scanner. arXiv Preprint arXiv:1603.03875 ( 2016 ). Zhe Wu, Sai-Kit Yeung, and Ping Tan. 2016. Towards building an RGBD-M scanner. arXiv Preprint arXiv:1603.03875 (2016)."},{"key":"e_1_2_2_44_1","doi-asserted-by":"publisher","DOI":"10.5555\/2964398.2964460"},{"key":"e_1_2_2_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.301"},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.92"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2766887"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601165"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3083722","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3083722","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:56:38Z","timestamp":1750290998000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3083722"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,6]]},"references-count":48,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2017,6,30]]}},"alternative-id":["10.1145\/3083722"],"URL":"https:\/\/doi.org\/10.1145\/3083722","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,6]]},"assertion":[{"value":"2016-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-06-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}