{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:11:25Z","timestamp":1760231485491,"version":"build-2065373602"},"reference-count":46,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2022,9,23]],"date-time":"2022-09-23T00:00:00Z","timestamp":1663891200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100010418","name":"Institute for Information and Communications Technology Promotion (IITP)","doi-asserted-by":"publisher","award":["2017-0-01849","2021-0-00320","2021R1F1A1046513"],"award-info":[{"award-number":["2017-0-01849","2021-0-00320","2021R1F1A1046513"]}],"id":[{"id":"10.13039\/501100010418","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea (NRF)","doi-asserted-by":"publisher","award":["2017-0-01849","2021-0-00320","2021R1F1A1046513"],"award-info":[{"award-number":["2017-0-01849","2021-0-00320","2021R1F1A1046513"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Providing real-time interaction in an immersive environment has drawn considerable attention in the virtual training fields. Physics-based simulations are suitable for such environments; however, they require the definition and adjustment of coefficients that determine material properties, making the methods more complex and time-consuming. In this paper, we introduce a novel approach to simulating the soft-body deformation of an observed object. Using an off-the-shelf RGB-D sensor, the proposed approach tracks an object\u2019s movement and simulates its deformation in an iterative manner. Polygonal models with different resolutions are used to improve the simulation speed and visual quality. During the simulation process, a low-resolution model is used for surface deformation using neighboring feature points detected from the sensor, and a volumetric model is added for internal force estimation. To visualize the observed model in detail, the deformed and low-resolution model is mapped to a high-resolution model using mean value coordinate interpolation. To handle topological deformations, such as cutting or tearing, a part intersected by a cutting tool is recognized by the sensor and responds to external forces. As shown in the experimental results, our approach generates convincing deformations of observed objects in real time.<\/jats:p>","DOI":"10.3390\/s22197225","type":"journal-article","created":{"date-parts":[[2022,9,26]],"date-time":"2022-09-26T03:34:17Z","timestamp":1664163257000},"page":"7225","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Physics-Based Simulation of Soft-Body Deformation Using RGB-D Data"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2362-7669","authenticated-orcid":false,"given":"Daeun","family":"Kang","sequence":"first","affiliation":[{"name":"Department of Computer Science, Hanyang University, Seoul 04763, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jaeseok","family":"Moon","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Hanyang University, Seoul 04763, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Saeyoung","family":"Yang","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Hanyang University, Seoul 04763, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9253-2156","authenticated-orcid":false,"given":"Taesoo","family":"Kwon","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Hanyang University, Seoul 04763, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4209-4266","authenticated-orcid":false,"given":"Yejin","family":"Kim","sequence":"additional","affiliation":[{"name":"School of Games, Hongik University, Sejong 30016, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,9,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Jung, T., and Dieck, M.C.T. (2018). Augmented Reality and Virtual Reality, Springer.","DOI":"10.1007\/978-3-319-64027-3"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1080\/17434440.2021.1860750","article-title":"A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: An extension to different kinds of surgery","volume":"18","author":"Lungu","year":"2021","journal-title":"Expert Rev. Med. Devices"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Baek, S., Gil, Y.H., and Kim, Y. (2021). VR-Based Job Training System Using Tangible Interactions. Sensors, 21.","DOI":"10.3390\/s21206794"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Collins, T., and Bartoli, A. (2015, January 19\u201323). Realtime Shape-from-Template: System and Applications. Proceedings of the ISMAR, Yucatan, Mexico.","DOI":"10.1109\/ISMAR.2015.35"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Tien Ngo, D., Park, S., Jorstad, A., Crivellaro, A., Yoo, C.D., and Fua, P. (2015, January 7\u201313). Dense image registration and deformable surface reconstruction in presence of occlusions and minimal texture. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.262"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yu, R., Russell, C., Campbell, N.D., and Agapito, L. (2015, January 7\u201313). Direct, dense, and deformable: Template-based non-rigid 3d reconstruction from rgb video. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.111"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"931","DOI":"10.1109\/TPAMI.2010.158","article-title":"Linear local models for monocular reconstruction of deformable surfaces","volume":"33","author":"Salzmann","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2099","DOI":"10.1109\/TPAMI.2015.2392759","article-title":"Shape-from-template","volume":"37","author":"Bartoli","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/TPAMI.2015.2435739","article-title":"Template-based monocular 3D shape recovery using laplacian meshes","volume":"38","author":"Ngo","year":"2015","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"833","DOI":"10.1109\/TPAMI.2016.2562622","article-title":"A stable analytical framework for isometric shape-from-template by surface integration","volume":"39","author":"Chhatkuli","year":"2016","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Torresani, L., Kolmogorov, V., and Rother, C. (2008, January 12\u201318). Feature correspondence via graph matching: Models and global optimization. Proceedings of the European Conference on Computer Vision, Marseille, France.","DOI":"10.1007\/978-3-540-88688-4_44"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhou, F., and De la Torre, F. (2013, January 23\u201328). Deformable graph matching. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.376"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Collins, T., Mesejo, P., and Bartoli, A. (2014, January 6\u201312). An analysis of errors in graph-based keypoint matching and proposed solutions. Proceedings of the European Conference on Computer Vision, Zurich, Switzerland.","DOI":"10.1007\/978-3-319-10584-0_10"},{"key":"ref_14","unstructured":"Wang, T., Ling, H., Lang, C., Feng, S., and Hou, X. (November, January 27). Deformable surface tracking by graph matching. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/MMUL.2012.24","article-title":"Microsoft kinect sensor and its effect","volume":"19","author":"Zhang","year":"2012","journal-title":"IEEE Multimed."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Keselman, L., Iselin Woodfill, J., Grunnet-Jepsen, A., and Bhowmik, A. (2017, January 21\u201326). Intel realsense stereoscopic depth cameras. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.167"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1110","DOI":"10.1109\/TMM.2013.2246148","article-title":"Robust part-based hand gesture recognition using kinect sensor","volume":"15","author":"Ren","year":"2013","journal-title":"IEEE Trans. Multimed."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Patil, J.V., and Bailke, P. (2016, January 26\u201327). Real time facial expression recognition using RealSense camera and ANN. Proceedings of the 2016 International Conference on Inventive Computation Technologies (ICICT), Coimbatore, India.","DOI":"10.1109\/INVENTIVE.2016.7824820"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"387","DOI":"10.1111\/cgf.13134","article-title":"Performance-Based Biped Control using a Consumer Depth Camera","volume":"Volume 36","author":"Lee","year":"2017","journal-title":"Computer Graphics Forum"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1007\/s10916-018-0905-x","article-title":"Utilising the intel realsense camera for measuring health outcomes in clinical research","volume":"42","author":"Siena","year":"2018","journal-title":"J. Med Syst."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Terzopoulos, D., Platt, J., Barr, A., and Fleischer, K. (1987, January 27\u201331). Elastically deformable models. Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques, Anaheim, CA, USA.","DOI":"10.1145\/37401.37427"},{"key":"ref_22","unstructured":"Bathe, K.J. (2006). Finite Element Procedures, Klaus-Jurgen Bathe."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.gmod.2009.02.002","article-title":"Flexible simulation of deformable models using discontinuous Galerkin FEM","volume":"71","author":"Kaufmann","year":"2009","journal-title":"Graph. Model."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Irving, G., Teran, J., and Fedkiw, R. (2004, January 27\u201329). Invertible finite elements for robust simulation of large deformation. Proceedings of the 2004 ACM SIGGRAPH\/Eurographics Symposium on Computer Animation, Grenoble, France.","DOI":"10.1145\/1028523.1028541"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"13-es","DOI":"10.1145\/1276377.1276394","article-title":"Volume conserving finite element simulations of deformable models","volume":"26","author":"Irving","year":"2007","journal-title":"ACM Trans. Graph. TOG"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"16-es","DOI":"10.1145\/1276377.1276397","article-title":"A finite element method for animating large viscoplastic flow","volume":"26","author":"Bargteil","year":"2007","journal-title":"ACM Trans. Graph. TOG"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2070781.2024224","article-title":"Fast simulation of skeleton-driven deformable body characters","volume":"30","author":"Kim","year":"2011","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"M\u00fcller, M., Dorsey, J., McMillan, L., Jagnow, R., and Cutler, B. (2002, January 21\u201322). Stable real-time deformations. Proceedings of the 2002 ACM SIGGRAPH\/Eurographics Symposium on Computer Animation, San Antonio, TX, USA.","DOI":"10.1145\/545261.545269"},{"key":"ref_29","unstructured":"Nesme, M., Payan, Y., and Faure, F. (2\u2013September, January 29). Efficient, physically plausible finite elements. Proceedings of the EG Short Presentations. The Eurographics Association, Dublin, Ireland."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1145\/588272.588278","article-title":"Multiresolution green\u2019s function methods for interactive simulation of large-scale elastostatic objects","volume":"22","author":"James","year":"2003","journal-title":"ACM Trans. Graph. TOG"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"982","DOI":"10.1145\/1073204.1073300","article-title":"Real-time subspace integration for St. Venant-Kirchhoff deformable models","volume":"24","author":"James","year":"2005","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kim, T., and James, D.L. (2009, January 16\u201319). Skipping steps in deformable simulation with online model reduction. Proceedings of the ACM SIGGRAPH Asia 2009 Papers, Yokohama, Japan.","DOI":"10.1145\/1661412.1618469"},{"key":"ref_33","unstructured":"Haouchine, N., Dequidt, J., Kerrien, E., Berger, M.O., and Cotin, S. (2012, January 6\u20137). Physics-based augmented reality for 3d deformable object. Proceedings of the Eurographics Workshop on Virtual Reality Interaction and Physical Simulation, Darmstadt, Germany."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Leizea, I., Alvarez, H., Aguinaga, I., and Borro, D. (2014, January 10\u201312). Real-time deformation, registration and tracking of solids based on physical simulation. Proceedings of the 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Munich, Germany.","DOI":"10.1109\/ISMAR.2014.6948423"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Petit, A., Lippiello, V., and Siciliano, B. (October, January 28). Real-time tracking of 3D elastic objects with an RGB-D sensor. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7353928"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Sengupta, A., Krupa, A., and Marchand, E. (2019, January 6\u20139). Tracking of non-rigid objects using rgb-d camera. Proceedings of the 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), Bari, Italy.","DOI":"10.1109\/SMC.2019.8914543"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6\u201313). ORB: An efficient alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1109\/34.121791","article-title":"A Method for Registration of 3-D Shapes","volume":"14","author":"Besl","year":"1992","journal-title":"Trans. Pattern Anal. Mach. Intell."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"181","DOI":"10.4218\/etrij.17.2816.0045","article-title":"Motion Capture of the Human Body Using Multiple Depth Sensors","volume":"39","author":"Kim","year":"2017","journal-title":"ETRI J."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Martin, S., Thomaszewski, B., Grinspun, E., and Gross, M. (2011, January 7\u201311). Example-based elastic materials. Proceedings of the ACM SIGGRAPH 2011 Papers, Vancouver, BC, Canada.","DOI":"10.1145\/1964921.1964967"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2601097.2601116","article-title":"Projective dynamics: Fusing constraint projections for fast simulation","volume":"33","author":"Bouaziz","year":"2014","journal-title":"ACM Trans. Graph. TOG"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3272127.3275016","article-title":"Dexterous manipulation and control with volumetric muscles","volume":"37","author":"Lee","year":"2018","journal-title":"ACM Trans. Graph. TOG"},{"key":"ref_43","unstructured":"Ju, T., Schaefer, S., and Warren, J. (August, January 31). Mean value coordinates for closed triangular meshes. Proceedings of the ACM Siggraph, Los Angeles, CA, USA."},{"key":"ref_44","unstructured":"Mu\u00f1oz-Salinas, R., and Garrido-Jurado, S. (2022, March 01). ArUco Library. Available online: https:\/\/docs.opencv.org."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1080\/10867651.1997.10487468","article-title":"Fast, minimum storage ray-triangle intersection","volume":"2","author":"Trumbore","year":"1997","journal-title":"J. Graph. Tools"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Nienhuys, H.W., and van der Stappen, A.F. (2004). A Delaunay approach to interactive cutting in triangulated surfaces. Algorithmic Foundations of Robotics V, Springer.","DOI":"10.1007\/978-3-540-45058-0_8"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7225\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:38:23Z","timestamp":1760143103000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/19\/7225"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,23]]},"references-count":46,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["s22197225"],"URL":"https:\/\/doi.org\/10.3390\/s22197225","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2022,9,23]]}}}