{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T14:03:37Z","timestamp":1772719417628,"version":"3.50.1"},"reference-count":70,"publisher":"Association for Computing Machinery (ACM)","issue":"1s","license":[{"start":{"date-parts":[[2018,3,26]],"date-time":"2018-03-26T00:00:00Z","timestamp":1522022400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Fonds National de la Recherche (FNR), Luxembourg","award":["C11\/BM\/1204105\/FAVE\/Ottersten"],"award-info":[{"award-number":["C11\/BM\/1204105\/FAVE\/Ottersten"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2018,3,31]]},"abstract":"<jats:p>In this article, we discuss enhanced full 360\u00b0 3D reconstruction of dynamic scenes containing non-rigidly deforming objects using data acquired from commodity depth or 3D cameras. Several approaches for enhanced and full 3D reconstruction of non-rigid objects have been proposed in the literature. These approaches suffer from several limitations due to requirement of a template, inability to tackle large local deformations and topology changes, inability to tackle highly noisy and low-resolution data, and inability to produce online results. We target online and template-free enhancement of the quality of noisy and low-resolution full 3D reconstructions of dynamic non-rigid objects. For this purpose, we propose a view-independent recursive and dynamic multi-frame 3D super-resolution scheme for noise removal and resolution enhancement of 3D measurements. The proposed scheme tracks the position and motion of each 3D point at every timestep by making use of the current acquisition and the result of the previous iteration. The effects of system blur due to per-point tracking are subsequently tackled by introducing a novel and efficient multi-level 3D bilateral total variation regularization. These characteristics enable the proposed scheme to handle large deformations and topology changes accurately. A thorough evaluation of the proposed scheme on both real and simulated data is carried out. The results show that the proposed scheme improves upon the performance of the state-of-the-art methods and is able to accurately enhance the quality of low-resolution and highly noisy 3D reconstructions while being robust to large local deformations.<\/jats:p>","DOI":"10.1145\/3177756","type":"journal-article","created":{"date-parts":[[2018,3,27]],"date-time":"2018-03-27T12:06:45Z","timestamp":1522152405000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["Full 3D Reconstruction of Non-Rigidly Deforming Objects"],"prefix":"10.1145","volume":"14","author":[{"given":"Hassan","family":"Afzal","sequence":"first","affiliation":[{"name":"University of Luxembourg, Luxembourg"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Djamila","family":"Aouada","sequence":"additional","affiliation":[{"name":"University of Luxembourg, Luxembourg"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bruno","family":"Mirbach","sequence":"additional","affiliation":[{"name":"IEE S.A., Luxembourg"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bj\u00f6rn","family":"Ottersten","sequence":"additional","affiliation":[{"name":"University of Luxembourg, Luxembourg"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,3,26]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-23117-4_61"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR.2014.425"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/3DV.2014.114"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2003.1175093"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1073204.1073207"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33712-3_6"},{"key":"e_1_2_1_7_1","volume-title":"Proc. 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS\u201914)","author":"Aouada Djamila","unstructured":"Djamila Aouada , Kassem Al Ismaeil , Kedija Kedir Idris , and Bj\u00f6rn E. Ottersten . 2014. Surface UP-SR for an improved face recognition using low resolution depth cameras . In Proc. 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS\u201914) . IEEE, 107--112. Djamila Aouada, Kassem Al Ismaeil, Kedija Kedir Idris, and Bj\u00f6rn E. Ottersten. 2014. Surface UP-SR for an improved face recognition using low resolution depth cameras. In Proc. 11th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS\u201914). IEEE, 107--112."},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 10th International Conference on Computer Vision Theory and Applications (VISAPP\u201915)","volume":"1","author":"Aouada Djamila","unstructured":"Djamila Aouada , Kassem Al Ismaeil , and Bj\u00f6rn E. Ottersten . 2015. Patch-based statistical performance analysis of upsampling for precise super-resolution . In Proceedings of the 10th International Conference on Computer Vision Theory and Applications (VISAPP\u201915) , Vol. 1 . 186--193. Djamila Aouada, Kassem Al Ismaeil, and Bj\u00f6rn E. Ottersten. 2015. Patch-based statistical performance analysis of upsampling for precise super-resolution. In Proceedings of the 10th International Conference on Computer Vision Theory and Applications (VISAPP\u201915), Vol. 1. 186--193."},{"key":"e_1_2_1_9_1","unstructured":"Asus. 2010. Xtion PRO LIVE. (2010). https:\/\/www.asus.com\/3D-Sensor\/Xtion_PRO_LIVE.  Asus. 2010. Xtion PRO LIVE. (2010). https:\/\/www.asus.com\/3D-Sensor\/Xtion_PRO_LIVE."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276467"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2014.2337258"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2016.2601059"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/34.24792"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/1888089.1888114"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1020145606604"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2009.01384.x"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-37484-5_12"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2012.190"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3054739"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360697"},{"key":"e_1_2_1_21_1","volume-title":"Proceeding of the 2014 IEEE Virtual Reality Conference. IEEE, 39--44","author":"Dou Mingsong","unstructured":"Mingsong Dou and H. Fuchs . 2014. Temporally enhanced 3D capture of room-sized dynamic scenes with commodity depth cameras . In Proceeding of the 2014 IEEE Virtual Reality Conference. IEEE, 39--44 . Mingsong Dou and H. Fuchs. 2014. Temporally enhanced 3D capture of room-sized dynamic scenes with commodity depth cameras. In Proceeding of the 2014 IEEE Virtual Reality Conference. IEEE, 39--44."},{"key":"e_1_2_1_22_1","volume-title":"IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2013","author":"Dou Mingsong","year":"2013","unstructured":"Mingsong Dou , Henry Fuchs , and Jan Michael Frahm . 2013 . Scanning and tracking dynamic objects with commodity depth cameras . In IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2013 , Adelaide, Australia October 1--4 , 2013. IEEE, 99--106. Mingsong Dou, Henry Fuchs, and Jan Michael Frahm. 2013. Scanning and tracking dynamic objects with commodity depth cameras. In IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2013, Adelaide, Australia October 1--4, 2013. IEEE, 99--106."},{"key":"e_1_2_1_23_1","volume-title":"Proc. of the International Conference on Intelligent Robots and Systems (IROS\u201913)","author":"Freese M.","year":"2013","unstructured":"M. Freese , E. Rohmer , and S. P. N. Singh . 2013 . V-REP: A versatile and scalable robot simulation framework . In Proc. of the International Conference on Intelligent Robots and Systems (IROS\u201913) . IEEE, 1321--1326. M. Freese, E. Rohmer, and S. P. N. Singh. 2013. V-REP: A versatile and scalable robot simulation framework. In Proc. of the International Conference on Intelligent Robots and Systems (IROS\u201913). IEEE, 1321--1326."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2614106.2614182"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1201775.882368"},{"key":"e_1_2_1_26_1","volume-title":"Image and Geometry Processing for 3-D Cinematography","author":"Furukawa Yasutaka","unstructured":"Yasutaka Furukawa and Jean Ponce . 2010. Dense 3D motion capture from synchronized video streams . In Image and Geometry Processing for 3-D Cinematography . Springer , 193--211. Yasutaka Furukawa and Jean Ponce. 2010. Dense 3D motion capture from synchronized video streams. In Image and Geometry Processing for 3-D Cinematography. Springer, 193--211."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.168"},{"key":"e_1_2_1_28_1","volume-title":"Baraniuk","author":"Goldstein Tom","year":"2014","unstructured":"Tom Goldstein , Christoph Studer , and Richard G . Baraniuk . 2014 . A field guide to forward-backward splitting with a FASTA implementation. CoRR abs\/1411.3406 (2014). http:\/\/arxiv.org\/abs\/1411.3406 Tom Goldstein, Christoph Studer, and Richard G. Baraniuk. 2014. A field guide to forward-backward splitting with a FASTA implementation. CoRR abs\/1411.3406 (2014). http:\/\/arxiv.org\/abs\/1411.3406"},{"key":"e_1_2_1_29_1","volume-title":"VolumeDeform: Real-Time Volumetric Non-rigid Reconstruction","author":"Innmann Matthias","unstructured":"Matthias Innmann , Michael Zollh\u00f6fer , Matthias Nie\u00dfner , Christian Theobalt , and Marc Stamminger . 2016. VolumeDeform: Real-Time Volumetric Non-rigid Reconstruction . Springer International Publishing , Cham , 362--379. Matthias Innmann, Michael Zollh\u00f6fer, Matthias Nie\u00dfner, Christian Theobalt, and Marc Stamminger. 2016. VolumeDeform: Real-Time Volumetric Non-rigid Reconstruction. Springer International Publishing, Cham, 362--379."},{"key":"e_1_2_1_30_1","unstructured":"Inria. 2009. 4D-Repository. Retrieved from http:\/\/4drepository.inrialpes.fr\/pages\/home.  Inria. 2009. 4D-Repository. Retrieved from http:\/\/4drepository.inrialpes.fr\/pages\/home."},{"key":"e_1_2_1_31_1","volume-title":"Proceedings of the 2013 IEEE International Conference on Image Processing. 660--664","author":"Ismaeil K. A.","unstructured":"K. A. Ismaeil , D. Aouada , B. Mirbach , and B. Ottersten . 2013. Dynamic super resolution of depth sequences with non-rigid motions . In Proceedings of the 2013 IEEE International Conference on Image Processing. 660--664 . K. A. Ismaeil, D. Aouada, B. Mirbach, and B. Ottersten. 2013. Dynamic super resolution of depth sequences with non-rigid motions. In Proceedings of the 2013 IEEE International Conference on Image Processing. 660--664."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2016.04.006"},{"key":"e_1_2_1_33_1","volume-title":"Proc. 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW\u201915)","author":"Ismaeil K. Al","unstructured":"K. Al Ismaeil , D. Aouada , T. Solignac , B. Mirbach , and B. Ottersten . 2015. Real-time non-rigid multi-frame depth video super-resolution . In Proc. 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW\u201915) . IEEE, 8--16. K. Al Ismaeil, D. Aouada, T. Solignac, B. Mirbach, and B. Ottersten. 2015. Real-time non-rigid multi-frame depth video super-resolution. In Proc. 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW\u201915). IEEE, 8--16."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2622698"},{"key":"e_1_2_1_35_1","series-title":"Series D","volume-title":"A new approach to linear filtering and prediction problems. Trans. ASME--J. Basic Eng. 82","author":"Kalman Rudolph Emil","year":"1960","unstructured":"Rudolph Emil Kalman . 1960. A new approach to linear filtering and prediction problems. Trans. ASME--J. Basic Eng. 82 , Series D ( 1960 ), 35--45. Rudolph Emil Kalman. 1960. A new approach to linear filtering and prediction problems. Trans. ASME--J. Basic Eng. 82, Series D (1960), 35--45."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2014.2362059"},{"key":"e_1_2_1_37_1","volume-title":"Global correspondence optimization for non-rigid registration of depth scans. Computer Graphics Forum (SGP\u201908) 27, 5 (July","author":"Li Hao","year":"2008","unstructured":"Hao Li , Robert W. Sumner , and Mark Pauly . 2008. Global correspondence optimization for non-rigid registration of depth scans. Computer Graphics Forum (SGP\u201908) 27, 5 (July 2008 ), 1421--1430. Hao Li, Robert W. Sumner, and Mark Pauly. 2008. Global correspondence optimization for non-rigid registration of depth scans. Computer Graphics Forum (SGP\u201908) 27, 5 (July 2008), 1421--1430."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2508363.2508407"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1186\/1687-6180-2012-222"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCVW.2013.48"},{"key":"e_1_2_1_41_1","volume-title":"https:\/\/dev.windows.com\/en-us\/kinect","year":"2014","unstructured":"Microsoft. 2014. Kinect. ( 2014 ). https:\/\/dev.windows.com\/en-us\/kinect . Microsoft. 2014. Kinect. (2014). https:\/\/dev.windows.com\/en-us\/kinect."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2011.2179329"},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of the Symposium on Geometry Processing. 173--182","author":"Mitra Niloy J.","year":"2007","unstructured":"Niloy J. Mitra , Simon Flory , Maks Ovsjanikov , Natasha Gelfand , Leonidas Guibas , and Helmut Pottmann . 2007 . Dynamic geometry registration . In Proceedings of the Symposium on Geometry Processing. 173--182 . Niloy J. Mitra, Simon Flory, Maks Ovsjanikov, Natasha Gelfand, Leonidas Guibas, and Helmut Pottmann. 2007. Dynamic geometry registration. In Proceedings of the Symposium on Geometry Processing. 173--182."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2010.46"},{"key":"e_1_2_1_45_1","volume-title":"Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201915)","author":"Newcombe Richard A.","unstructured":"Richard A. Newcombe , Dieter Fox , and Steven M. Seitz . 2015. Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time . In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201915) . Richard A. Newcombe, Dieter Fox, and Steven M. Seitz. 2015. Dynamicfusion: Reconstruction and tracking of non-rigid scenes in real-time. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201915)."},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISMAR.2011.6092378"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV.2013.6474999"},{"key":"e_1_2_1_48_1","volume-title":"Proc. 2012 IEEE Conference on Computer Vision and Pattern Recognition. 1576--1583","author":"Olsson C.","unstructured":"C. Olsson and Y. Boykov . 2012. Curvature-based regularization for surface approximation . In Proc. 2012 IEEE Conference on Computer Vision and Pattern Recognition. 1576--1583 . C. Olsson and Y. Boykov. 2012. Curvature-based regularization for surface approximation. In Proc. 2012 IEEE Conference on Computer Vision and Pattern Recognition. 1576--1583."},{"key":"e_1_2_1_49_1","unstructured":"PMDTechnologies. 2012. Camboard Nano. http:\/\/www.pmdtec.com.  PMDTechnologies. 2012. Camboard Nano. http:\/\/www.pmdtec.com."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.5244\/C.26.112"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206804"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1409060.1409063"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1276377.1276478"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.5555\/1731309.1731332"},{"key":"e_1_2_1_55_1","volume-title":"Computer Vision: Algorithms and Applications","author":"Szeliski Richard","year":"2010","unstructured":"Richard Szeliski . 2010 . Computer Vision: Algorithms and Applications ( 1 st ed.). Springer-Verlag New York, Inc. , New York . Richard Szeliski. 2010. Computer Vision: Algorithms and Applications (1st ed.). Springer-Verlag New York, Inc., New York.","edition":"1"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2007.1006"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.56"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/1360612.1360696"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618520"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/1516522.1516526"},{"key":"e_1_2_1_61_1","volume-title":"Proceedings of the 5th Eurographics Symposium on Geometry Processing (SGP\u201907)","author":"Wand Michael","year":"2007","unstructured":"Michael Wand , Philipp Jenke , Qixing Huang , Martin Bokeloh , Leonidas Guibas , and Andreas Schilling . 2007 . Reconstruction of deforming geometry from time-varying point clouds . In Proceedings of the 5th Eurographics Symposium on Geometry Processing (SGP\u201907) . Eurographics Association, Aire-la-Ville, Switzerland, 49--58. Michael Wand, Philipp Jenke, Qixing Huang, Martin Bokeloh, Leonidas Guibas, and Andreas Schilling. 2007. Reconstruction of deforming geometry from time-varying point clouds. In Proceedings of the 5th Eurographics Symposium on Geometry Processing (SGP\u201907). Eurographics Association, Aire-la-Ville, Switzerland, 49--58."},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.5220\/0005234800050012"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126465"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.252"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33709-3_59"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.111"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2013.26"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.92"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2009.01633.x"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/2601097.2601165"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3177756","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3177756","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T19:05:51Z","timestamp":1750273551000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3177756"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,3,26]]},"references-count":70,"journal-issue":{"issue":"1s","published-print":{"date-parts":[[2018,3,31]]}},"alternative-id":["10.1145\/3177756"],"URL":"https:\/\/doi.org\/10.1145\/3177756","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,3,26]]},"assertion":[{"value":"2017-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-12-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-03-26","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}