{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,4]],"date-time":"2026-08-04T16:05:10Z","timestamp":1785859510648,"version":"3.56.0"},"reference-count":43,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2017,3,17]],"date-time":"2017-03-17T00:00:00Z","timestamp":1489708800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotics"],"abstract":"<jats:p>Performing tasks with a robot hand often requires a complete knowledge of the manipulated object, including its properties (shape, rigidity, surface texture) and its location in the environment, in order to ensure safe and efficient manipulation. While well-established procedures exist for the manipulation of rigid objects, as well as several approaches for the manipulation of linear or planar deformable objects such as ropes or fabric, research addressing the characterization of deformable objects occupying a volume remains relatively limited. The paper proposes an approach for tracking the deformation of non-rigid objects under robot hand manipulation using RGB-D data. The purpose is to automatically classify deformable objects as rigid, elastic, plastic, or elasto-plastic, based on the material they are made of, and to support recognition of the category of such objects through a robotic probing process in order to enhance manipulation capabilities. The proposed approach combines advantageously classical color and depth image processing techniques and proposes a novel combination of the fast level set method with a log-polar mapping of the visual data to robustly detect and track the contour of a deformable object in a RGB-D data stream. Dynamic time warping is employed to characterize the object properties independently from the varying length of the tracked contour as the object deforms. The proposed solution achieves a classification rate over all categories of material of up to 98.3%. When integrated in the control loop of a robot hand, it can contribute to ensure stable grasp, and safe manipulation capability that will preserve the physical integrity of the object.<\/jats:p>","DOI":"10.3390\/robotics6010005","type":"journal-article","created":{"date-parts":[[2017,3,17]],"date-time":"2017-03-17T11:22:56Z","timestamp":1489749776000},"page":"5","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["Visual Tracking of Deformation and Classification of Non-Rigid Objects with Robot Hand Probing"],"prefix":"10.3390","volume":"6","author":[{"given":"Fei","family":"Hui","sequence":"first","affiliation":[{"name":"School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON, K1N 6N5, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3103-9752","authenticated-orcid":false,"given":"Pierre","family":"Payeur","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON, K1N 6N5, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ana-Maria","family":"Cretu","sequence":"additional","affiliation":[{"name":"School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, ON, K1N 6N5, Canada"},{"name":"Department of Computer Science and Engineering, Universit\u00e9 du Qu\u00e9bec en Outaouais, Gatineau, QC, J8X 3X7, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2017,3,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"434","DOI":"10.1108\/01439910210440255","article-title":"Industrial applications of automatic manipulation of flexible materials","volume":"29","author":"Saadat","year":"2012","journal-title":"Int. J. Ind. Robots"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Henrich, D., and Worn, H. (2000). Automated Swing System and Unfolding Fabric, Springer. Robot Manipulation of Deformable Objects.","DOI":"10.1007\/978-1-4471-0749-1"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Houachine, N., Dequidt, J., Berger, M.O., and Cotin, S. (2014, January 10\u201312). Single view augmentation of 3D elastic objects. Proceedings of the IEEE International Symposium on Mixed and Augmented Reality, Munich, Germany.","DOI":"10.1109\/ISMAR.2014.6948432"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1311","DOI":"10.1177\/0278364911403178","article-title":"Manipulator and object tracking for in-hand 3D object modeling","volume":"30","author":"Krainin","year":"2011","journal-title":"Int. J. Robot. Res."},{"key":"ref_5","unstructured":"Petit, A., Lippiello, V., and Siciliano, B. (October, January 28). Real-time tracking of 3D elastic objects with an RGB-D sensor. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Hamburg, Germany."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zaidi, L., Bouzgarrou, B., Sabourin, L., and Menzouar, Y. (2015, January 27\u201331). Interaction modeling in the grasping and manipulation of 3D deformable objects. Proceedings of the IEEE International Conference on Advanced Robotics, Istanbul, Turkey.","DOI":"10.1109\/ICAR.2015.7251503"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Verl, A., Albu-Sch\u00e4ffer, A., Brock, O., and Raatz, A. (2015). Soft Robotics, Springer.","DOI":"10.1007\/978-3-662-44506-8"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Jimenez, A., and Al Hadithi, B.M. (2010). Robot Manipulators, Trends and Development, In-Tech.","DOI":"10.5772\/218"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1109\/TSMCB.2011.2176115","article-title":"Soft object deformation monitoring and learning for model-based robotic hand manipulation","volume":"42","author":"Cretu","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. Part B"},{"key":"ref_10","unstructured":"Mateo, C.M., Gil, P., Mira, D., and Torres, F. (2015, January 21\u201323). Analysis of shapes to measure surfaces. Proceedings of the IEEE Conf. Informatics in Control, Automation and Robotics, Colmar, Alsace, France."},{"key":"ref_11","unstructured":"Elbrechter, C., Haschke, R., and Ritter, H. (December, January 29). Folding paper with anthropomorphic robot hands using real-time physics-based modeling. Proceedings of the IEEE International Conference on Humanoid Robots, Osaka, Japan."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Frank, B., Schmedding, R., Stachniss, C., Teschner, M., and Burgard, W. (2010, January 18\u201322). Learning the elasticity parameters of deformable objects with a manipulation robot. Proceedings of the IEEE International Conference on Intelligent Robots and Systems, Taipei, Taiwan.","DOI":"10.1109\/IROS.2010.5653949"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Schulman, J., Lee, A., Ho, J., and Abbeel, P. (2013, January 6\u201310). Tracking deformable objects with point clouds. Proceedings of the IEEE International Conference on Robotics and Automation, Karlsruhe, Germany.","DOI":"10.1109\/ICRA.2013.6630714"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1446","DOI":"10.1177\/0278364914536939","article-title":"Identification and active exploration of deformable object boundary constraints through robotic manipulation","volume":"22","author":"Boonvisut","year":"2014","journal-title":"Int. J. Robot. Res."},{"key":"ref_15","first-page":"156","article-title":"Real-time non-rigid reconstruction using an RGB-D camera","volume":"33","author":"Izadi","year":"2014","journal-title":"ACM Trans. Graph."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1109\/TSMCC.2003.819701","article-title":"Automatic visual recognition of deformable objects for grasping and manipulation","volume":"34","author":"Foresti","year":"2004","journal-title":"IEEE Trans. Syst. Man Cybern. Appl. Rev."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1145\/358669.358692","article-title":"Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography","volume":"24","author":"Fischler","year":"1981","journal-title":"Commun. ACM"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1007\/BF01759061","article-title":"Refinements to nearest-neighbor searching in k-dimensional trees","volume":"6","author":"Sproull","year":"1991","journal-title":"Algorithmica"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"58","DOI":"10.1016\/j.imavis.2007.11.009","article-title":"Log-polar mapping template design: From task-level requirements to geometry parameters","volume":"26","author":"Traver","year":"2008","journal-title":"Image Vis. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"645","DOI":"10.1109\/TIP.2008.920737","article-title":"A real-time algorithm for the approximation of level-set-based curve evolution","volume":"17","author":"Shi","year":"2008","journal-title":"IEEE Trans. Image Process."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Romero, J., Kragic, D., Kyrki, V., and Argyros, A. (2008, January 19\u201323). Dynamic time warping for binocular hand tracking and Reconstruction. Proceedings of the IEEE International Conference on Robotics and Automation, Pasadena, CA, USA.","DOI":"10.1109\/ROBOT.2008.4543555"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Mateo, C.M., Gil, P., and Torres, F. (2016). 3D visual data-driven spatiotemporal deformations for non-rigid object grasping using robot hands. Sensors, 16.","DOI":"10.3390\/s16050640"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"3361","DOI":"10.1007\/s11042-014-1995-1","article-title":"Estimating material properties of deformable objects by considering global object behavior in video streams","volume":"74","author":"Choi","year":"2015","journal-title":"Multimedia Tools Appl."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1142\/S021984360800139X","article-title":"Birth of the object: Detection of objectness and extraction of object shape through object-action complexes","volume":"5","author":"Kraft","year":"2008","journal-title":"Int. J. Humanoid Robot."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1109\/TRO.2016.2533639","article-title":"Automatic 3-D manipulation of soft object by robotic arms with an adaptive deformation model","volume":"32","author":"Yip","year":"2016","journal-title":"IEEE Trans. Robot."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hur, J., Lim, H., and Ahn, S.C. (2014, January 8\u201310). 3D deformable spatial pyramid for dense 3D motion flow of deformable object. Proceedings of the ISVC2014, Las Vegas, NV, USA.","DOI":"10.1007\/978-3-319-14249-4_12"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1142\/S0219843609001784","article-title":"Active segmentation","volume":"6","author":"Mishra","year":"2009","journal-title":"Int. J. Humanoid Robot."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Macknojia, R., Chavez-Aragon, A., Payeur, P., and Lagani\u00e8re, R. (2013, January 15\u201317). Calibration of a network of Kinect sensors for robotic inspection over a large workspace. Proceedings of the IEEE Workshop on Robot Vision, Tampa, FL, USA.","DOI":"10.1109\/WORV.2013.6521936"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Rusu, R.B. (2009). Semantic 3D Object Maps for Everyday Manipulation in Human Living Environments. [Ph.D. Thesis, Technische Universit\u00e4t M\u00fcnchen].","DOI":"10.1007\/s13218-010-0059-6"},{"key":"ref_30","unstructured":"The Point Cloud Library (PCL). Available online: http:\/\/pointclouds.org\/."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Chessa, M., and Solari, F. (2015). Local Feature Extraction in Log-Polar Images, Springer International Publishing.","DOI":"10.1007\/978-3-319-23231-7_37"},{"key":"ref_32","unstructured":"Chen, Y., Rui, Y., and Huang, T.S. (2001, January 8\u201314). JPDAF based HMM for real-time contour tracking. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Kauai, HI, USA."},{"key":"ref_33","unstructured":"Xu, C., Pham, C.D., and Prince, J.I. (2000). Handbook of Medical Imaging, Volume 2. Medical Image Processing and Analysis, SPIE Press. Chapter 3: Image Segmentation Using Deformable Models."},{"key":"ref_34","unstructured":"Sonka, M., Hlavac, V., and Boyle, R. (2008). Image Processing, Analysis, and Machine Vision, Brooke\/Cole Publishing Company."},{"key":"ref_35","unstructured":"Wilson, D., and Laxminarayan, S. (2005). Handbook of Biomedical Image Analysis, Springer. Volume 3: Registration Models; Chapter 3: State of the Art of Level Set Methods in Segmentation and Registration of Medical Imaging Modalities."},{"key":"ref_36","unstructured":"Mishra, A., Aloimonos, Y., and Fah, C.L. (October, January 27). Active segmentation with fixation. Proceedings of the IEEE International Conference on Computer Vision, Kyoto, Japan."},{"key":"ref_37","unstructured":"Thida, M., Chan, K.L., and Eng, H.-L. (2006, January 11\u201313). An improved real-time contour tracking. Proceedings of the Pacific-Rim Symposium on Image and Video Technology, Hsinchu, Taiwan."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1109\/83.902291","article-title":"Active contours without edges","volume":"10","author":"Chan","year":"2001","journal-title":"IEEE Trans. Image Process."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Muller, M. (2007). Information Retrieval for Music and Motion, Springer.","DOI":"10.1007\/978-3-540-74048-3"},{"key":"ref_40","unstructured":"Adwan, S. (2011). Reviews, Refinements and New Ideas in Face Recognition, InTech. Chapter 12: Robust Face Detection through Eyes Localization Using Dynamic Time Warping Algorithm."},{"key":"ref_41","unstructured":"Smith, L.I. A Tutorial on Principal Components Analysis: Introduction. Available online: http:\/\/www.cs.otago.ac.nz\/cosc453\/student tutorials\/principal components.pdf."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Gonzalez-Sosa, E., Vera-Rodriguez, R., Fierrez, J., and Ortega-Garcia, J. (2014, January 24\u201328). Comparison of body shape descriptors for biometric recognition using MMW images. Proceedings of the 22nd International Conference on Pattern Recognition, Stockholm, Sweden.","DOI":"10.1109\/ICPR.2014.31"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Bibby, C., and Reid, I. (2010, January 23\u201326). Real-time tracking of multiple occluding objects using level sets. Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Istanbul, Turkey.","DOI":"10.1109\/CVPR.2010.5539818"}],"container-title":["Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2218-6581\/6\/1\/5\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:30:43Z","timestamp":1760207443000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2218-6581\/6\/1\/5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,3,17]]},"references-count":43,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2017,3]]}},"alternative-id":["robotics6010005"],"URL":"https:\/\/doi.org\/10.3390\/robotics6010005","relation":{},"ISSN":["2218-6581"],"issn-type":[{"value":"2218-6581","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,3,17]]}}}