{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:36:42Z","timestamp":1760240202779,"version":"build-2065373602"},"reference-count":35,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2019,4,6]],"date-time":"2019-04-06T00:00:00Z","timestamp":1554508800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Advances in Robotics are leading to a new generation of assistant robots working in ordinary, domestic settings. This evolution raises new challenges in the tasks to be accomplished by the robots. This is the case for object manipulation where the detect-approach-grasp loop requires a robust recovery stage, especially when the held object slides. Several proprioceptive sensors have been developed in the last decades, such as tactile sensors or contact switches, that can be used for that purpose; nevertheless, their implementation may considerably restrict the gripper\u2019s flexibility and functionality, increasing their cost and complexity. Alternatively, vision can be used since it is an undoubtedly rich source of information, and in particular, depth vision sensors. We present an approach based on depth cameras to robustly evaluate the manipulation success, continuously reporting about any object loss and, consequently, allowing it to robustly recover from this situation. For that, a Lab-colour segmentation allows the robot to identify potential robot manipulators in the image. Then, the depth information is used to detect any edge resulting from two-object contact. The combination of those techniques allows the robot to accurately detect the presence or absence of contact points between the robot manipulator and a held object. An experimental evaluation in realistic indoor environments supports our approach.<\/jats:p>","DOI":"10.3390\/s19071648","type":"journal-article","created":{"date-parts":[[2019,4,8]],"date-time":"2019-04-08T11:54:52Z","timestamp":1554724492000},"page":"1648","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Vision for Robust Robot Manipulation"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4495-6912","authenticated-orcid":false,"given":"Ester","family":"Martinez-Martin","sequence":"first","affiliation":[{"name":"RoViT, University of Alicante, 03690 San Vicente del Raspeig (Alicante), Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6227-3758","authenticated-orcid":false,"given":"Angel","family":"del Pobil","sequence":"additional","affiliation":[{"name":"RobInLab, Jaume I University, 12071 Castello de la Plana, Spain"},{"name":"Interaction Science Dept., Sungkyunkwan University, Jongno-Gu, Seoul 110-745, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,4,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Costa, A., Martinez-Martin, E., Cazorla, M., and Julian, V. (2018). PHAROS-physical assistant robot system. Sensors, 18.","DOI":"10.3390\/s18082633"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.patrec.2017.05.027","article-title":"A robotic platform for customized and interactive rehabilitation of persons with disabilities","volume":"99","author":"Cazorla","year":"2017","journal-title":"Pattern Recogn. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Duckett, T., Pearson, S., Blackmore, S., Grieve, B., Chen, W.H., Cielniak, G., Cleaversmith, J., Dai, J., Davis, S., and Fox, C. (arXiv, 2018). Agricultural robotics: The future of robotic agriculture, arXiv.","DOI":"10.31256\/WP2018.2"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Robinette, P., Li, W., Allen, R., Howard, A.M., and Wagner, A.R. (2016, January 7\u201310). Overtrust of robots in emergency evacuation scenarios. Proceedings of the 2016 11th ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Christchurch, New Zealand.","DOI":"10.1109\/HRI.2016.7451740"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"694","DOI":"10.1109\/THMS.2016.2571269","article-title":"Human mobility modeling for robot-assisted evacuation in complex indoor environments","volume":"46","author":"Tang","year":"2016","journal-title":"IEEE Trans. Hum. Mach. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Azenkot, S., Feng, C., and Cakmak, M. (2016, January 7\u201310). Enabling building service robots to guide blind people a participatory design approach. Proceedings of the 2016 11th ACM\/IEEE International Conference on Human-Robot Interaction (HRI), Christchurch, New Zealand.","DOI":"10.1109\/HRI.2016.7451727"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1080\/01969722.2018.1558013","article-title":"A panoramic survey on grasping research trends and topics","volume":"50","author":"Alonso","year":"2019","journal-title":"Cybern. Syst."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"eaau4984","DOI":"10.1126\/scirobotics.aau4984","article-title":"Learning ambidextrous robot grasping policies","volume":"4","author":"Mahler","year":"2019","journal-title":"Sci. Robot."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Morrison, D., Corke, P., and Leitner, J. (arXiv, 2018). Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach, arXiv.","DOI":"10.15607\/RSS.2018.XIV.021"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Laskey, M., Lee, J., Chuck, C., Gealy, D., Hsieh, W., Pokorny, F.T., Dragan, A.D., and Goldberg, K. (2016, January 21\u201324). Robot grasping in clutter: Using a hierarchy of supervisors for learning from demonstrations. Proceedings of the 2016 IEEE International Conference on Automation Science and Engineering (CASE), Fort Worth, TX, USA.","DOI":"10.1109\/COASE.2016.7743488"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Nogueira, J., Martinez-Cantin, R., Bernardino, A., and Jamone, L. (2016, January 9\u201314). Unscented Bayesian optimization for safe robot grasping. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759310"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1163\/156855394X00356","article-title":"Tactile sensing and control of robotic manipulation","volume":"8","author":"Howe","year":"1993","journal-title":"Adv. Robot."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Prats, M., del Pobil, A.P., and Sanz, P.J. (2013). Robot Physical Interaction through the Combination of Vision, Tactile and Force Feedback, Springer.","DOI":"10.1007\/978-3-642-33241-8"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1016\/j.robot.2015.07.015","article-title":"Tactile sensing in dexterous robot hands\u2014Review","volume":"74","author":"Kappassov","year":"2015","journal-title":"Robot. Auton. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Chen, T., and Ciocarlie, M. (arXiv, 2018). Proprioception-based grasping for unknown objects using a series-elastic-actuated gripper, arXiv.","DOI":"10.1109\/IROS.2018.8593787"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Homberg, B.S., Katzschmann, R.K., Dogar, M.R., and Rus, D. (2018). Robust proprioceptive grasping with a soft robot hand. Autonomous Robots, Springer.","DOI":"10.1007\/s10514-018-9754-1"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Eppner, C., H\u00f6fer, S., Jonschkowski, R., Mart\u00edn-Mart\u00edn, R., Sieverling, A., Wall, V., and Brock, O. (2017, January 19\u201325). Lessons from the Amazon Picking Challenge: Four Aspects of Building Robotic Systems. Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, Melbourne, Australia.","DOI":"10.24963\/ijcai.2017\/676"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1109\/TASE.2016.2600527","article-title":"Analysis and observations from the first amazon picking challenge","volume":"15","author":"Correll","year":"2018","journal-title":"IEEE Trans. Autom. Sci. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Hernandez, C., Bharatheesha, M., Ko, W., Gaiser, H., Tan, J., van Deurzen, K., de Vries, M., Mil, B.V., van Egmond, J., and Burger, R. (2017). Team Delft\u2019s robot winner of the amazon picking challenge 2016. RoboCup 2016: Robot World Cup XX, Springer International Publishing.","DOI":"10.1007\/978-3-319-68792-6_51"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Del Pobil, A.P., Kassawat, M., Duran, A.J., Arias, M., Nechyporenko, N., Mallick, A., Cervera, E., Subedi, D., Vasilev, I., and Cardin, D. (2017, January 16\u201318). UJI RobInLab\u2019s approach to the amazon robotics challenge 2017. Proceedings of the 2017 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI), Daegu, Korea.","DOI":"10.1109\/MFI.2017.8170448"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Nicodemou, V.C., Oikonomidis, I., and Argyros, A. (2019). Single-shot 3D hand pose estimation using radial basis function networks trained on synthetic data. Pattern Analysis and Applications, Springer.","DOI":"10.1007\/s10044-019-00801-7"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2883","DOI":"10.1109\/TPAMI.2017.2759736","article-title":"Hand-object contact force estimation from markerless visual tracking","volume":"40","author":"Pham","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Yuan, S., Garcia-Hernando, G., Stenger, B., Moon, G., Chang, J.Y., Lee, K.M., Molchanov, P., Kautz, J., Honari, S., and Ge, L. (2018, January 18\u201322). Depth-based 3D hand pose estimation: From current achievements to future goals. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition,, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00279"},{"key":"ref_24","first-page":"1","article-title":"Deep learning for computer vision: A brief review","volume":"2018","author":"Voulodimos","year":"2018","journal-title":"Comput. Intell. Neurosci."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.neucom.2015.09.116","article-title":"Deep learning for visual understanding: A review","volume":"187","author":"Guo","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_26","unstructured":"Bengio, Y., Courville, A., and Vincent, P. (arXiv, 2012). Unsupervised feature learning and deep learning: A review and new perspectives, arXiv."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","article-title":"Distinctive image features from scale-invariant keypoints","volume":"60","author":"Lowe","year":"2004","journal-title":"Int. J. Comput. Vis."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Lowe, D. (1999, January 20\u201327). Object recognition from local scale-invariant features. Proceedings of the Seventh IEEE International Conference on Computer Vision, Kerkyra, Greece.","DOI":"10.1109\/ICCV.1999.790410"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Alahi, A., Ortiz, R., and Vandergheynst, P. (2012, January 16\u201321). FREAK: Fast retina keypoint. Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA.","DOI":"10.1109\/CVPR.2012.6247715"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Rublee, E., Rabaud, V., Konolige, K., and Bradski, G. (2011, January 6\u201313). ORB: An efficient alternative to SIFT or SURF. Proceedings of the 2011 International Conference on Computer Vision, Barcelona, Spain.","DOI":"10.1109\/ICCV.2011.6126544"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Bay, H., Tuytelaars, T., and Gool, L.V. (2006). SURF: Speeded up robust features. Computer Vision\u2014ECCV 2006, Springer.","DOI":"10.1007\/11744023_32"},{"key":"ref_32","unstructured":"Martinez-Martin, E., and del Pobil, A.P. (November, January 30). Visual object recognition for robot tasks in real-life scenarios. Proceedings of the 10th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI), Jeju, Korea."},{"key":"ref_33","unstructured":"(2018, October 22). Rethink Robotics\u2014Baxter Robot. Available online: https:\/\/www.rethinkrobotics.com\/baxter\/."},{"key":"ref_34","unstructured":"(2018, October 22). Softbank Robotics\u2014Pepper. Available online: https:\/\/www.softbankrobotics.com\/emea\/en\/pepper."},{"key":"ref_35","unstructured":"(2018, October 22). HOBBIT\u2014The Mutual Care Robot. Available online: http:\/\/hobbit.acin.tuwien.ac.at\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/7\/1648\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:43:26Z","timestamp":1760186606000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/7\/1648"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,4,6]]},"references-count":35,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2019,4]]}},"alternative-id":["s19071648"],"URL":"https:\/\/doi.org\/10.3390\/s19071648","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2019,4,6]]}}}