{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,7]],"date-time":"2026-02-07T11:55:28Z","timestamp":1770465328359,"version":"3.49.0"},"reference-count":37,"publisher":"Cambridge University Press (CUP)","issue":"5","license":[{"start":{"date-parts":[[2024,3,5]],"date-time":"2024-03-05T00:00:00Z","timestamp":1709596800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Robotica"],"published-print":{"date-parts":[[2024,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Robots with multi-sensors always have a problem of weak pairing among different modals of the collected information produced by multi-sensors, which leads to a bad perception performance during robot interaction. To solve this problem, this paper proposes a Force Vision Sight (FVSight) sensor, which utilizes a distributed flexible tactile sensing array integrated with a vision unit. This innovative approach aims to enhance the overall perceptual capabilities for object recognition. The core idea is using one perceptual layer to trigger both tactile images and force-tactile arrays. It allows the two heterogeneous tactile modal information to be consistent in the temporal and spatial dimensions, thus solving the problem of weak pairing between visual and tactile data. Two experiments are specially designed, namely object classification and slip detection. A dataset containing 27 objects with deep presses and shallow presses is collected for classification, and then 20 slip experiments on three objects are conducted. The determination of slip and stationary state is accurately obtained by covariance operation on the tactile data. The experimental results show the reliability of generated multimodal data and the effectiveness of our proposed FVSight sensor.<\/jats:p>","DOI":"10.1017\/s0263574724000286","type":"journal-article","created":{"date-parts":[[2024,3,5]],"date-time":"2024-03-05T07:30:25Z","timestamp":1709623825000},"page":"1420-1435","source":"Crossref","is-referenced-by-count":6,"title":["A novel tactile sensor with multimodal vision and tactile units for multifunctional robot interaction"],"prefix":"10.1017","volume":"42","author":[{"given":"Pengwen","family":"Xiong","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuxuan","family":"Huang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3180-9086","authenticated-orcid":false,"given":"Yifan","family":"Yin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aiguo","family":"Song","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2024,3,5]]},"reference":[{"key":"S0263574724000286_ref12","doi-asserted-by":"crossref","unstructured":"[12] Gomes, D. F. , Lin, Z. and Luo, S. , \u201cGelTip: A Finger-shaped Optical Tactile Sensor for Robotic Manipulation,\u201d 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) (2020) pp. 9903\u20139909.","DOI":"10.1109\/IROS45743.2020.9340881"},{"key":"S0263574724000286_ref23","first-page":"969","article-title":"Object classification and grasp planning using visual and tactile sensing","volume":"46","author":"Sun","year":"2016","journal-title":"IEEE Trans. Syst."},{"key":"S0263574724000286_ref22","doi-asserted-by":"crossref","first-page":"1025","DOI":"10.1017\/S026357472200131X","article-title":"A flexible wearable e-skin sensing system for robotic teleoperation","volume":"41","author":"Zhong","year":"2023","journal-title":"Robotica"},{"key":"S0263574724000286_ref36","doi-asserted-by":"crossref","unstructured":"[36] Li, J. , Dong, S. and Adelson, E. , \u201cSlip Detection with Combined Tactile and Visual Information,\u201d 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane (2018) pp. 7772\u20137777.","DOI":"10.1109\/ICRA.2018.8460495"},{"key":"S0263574724000286_ref11","doi-asserted-by":"crossref","unstructured":"[11] Wang, S. , She, Y. , Romero, B. and Adelson, E. , \u201cGelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Finger,\u201d 2021 IEEE International Conference on Robotics and Automation (ICRA) (2021) pp. 6468\u20136475.","DOI":"10.1109\/ICRA48506.2021.9560783"},{"key":"S0263574724000286_ref27","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.ins.2014.08.066","article-title":"Kernel sparse representation for time series classification","volume":"292","author":"Chen","year":"2015","journal-title":"Inform. Sci."},{"key":"S0263574724000286_ref4","doi-asserted-by":"crossref","first-page":"107063","DOI":"10.1016\/j.nanoen.2022.107063","article-title":"A triboelectric-inductive hybrid tactile sensor for highly accurate object recognition","volume":"96","author":"Li","year":"2022","journal-title":"Nano Energy"},{"key":"S0263574724000286_ref9","doi-asserted-by":"crossref","unstructured":"[9] Yuan, W. , Li, R. , Srinivasan, M. A. and Adelson, E. H. , \u201cMeasurement of Shear and Slip with a GelSight Tactile Sensor,\u201d IEEE International Conference on Robotics and Automation (ICRA) (2015) pp. 304\u2013311.","DOI":"10.1109\/ICRA.2015.7139016"},{"key":"S0263574724000286_ref15","doi-asserted-by":"crossref","unstructured":"[15] Luo, S. , Yuan, W. , Adelson, E. , Cohn, A. G. and Fuentes, R. , \u201cViTac: Feature Sharing Between Vision and Tactile Sensing for Cloth Texture Recognition,\u201d 2018 IEEE International Conference on Robotics and Automation (ICRA) (2018) pp. 2722\u20132727.","DOI":"10.1109\/ICRA.2018.8460494"},{"key":"S0263574724000286_ref18","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1080\/01691864.2019.1632222","article-title":"Recent progress in tactile sensing and sensors for robotic manipulation: Can we turn tactile sensing into vision?","volume":"33","author":"Yamaguchi","year":"2019","journal-title":"Adv. Robotics"},{"key":"S0263574724000286_ref3","doi-asserted-by":"crossref","first-page":"2694","DOI":"10.3390\/s19122694","article-title":"An improved capacitive sensor for detecting the micro-clearance of spherical joints","volume":"19","author":"Wang","year":"2019","journal-title":"Sensors"},{"key":"S0263574724000286_ref6","doi-asserted-by":"crossref","first-page":"4300","DOI":"10.1109\/JSEN.2020.3030631","article-title":"F-TOUCH sensor: Concurrent geometry perception and multi-axis force measurement","volume":"21","author":"Li","year":"2021","journal-title":"IEEE Sens. J."},{"key":"S0263574724000286_ref2","article-title":"Enhancing robotic tactile exploration with multi-receptive graph convolutional networks","author":"Liao","journal-title":"IEEE Trans. Ind. Electron."},{"key":"S0263574724000286_ref29","doi-asserted-by":"crossref","DOI":"10.1007\/s11432-021-3512-6","article-title":"Robotic haptic adjective perception based on coupled sparse coding","volume":"66","author":"Xiong","year":"2023","journal-title":"Sci. China Inform. Sci."},{"key":"S0263574724000286_ref35","doi-asserted-by":"crossref","first-page":"523","DOI":"10.3390\/s19030523","article-title":"Learning spatio temporal tactile features with a ConvLSTM for the direction of slip detection","volume":"19","author":"Zapata-Impata","year":"2019","journal-title":"Sensors"},{"key":"S0263574724000286_ref37","doi-asserted-by":"crossref","first-page":"2214","DOI":"10.1109\/TMECH.2016.2551557","article-title":"A slip detection and correction strategy for precision robot grasping","volume":"21","author":"Stachowsky","year":"2016","journal-title":"IEEE\/ASME Trans. Mechatron."},{"key":"S0263574724000286_ref14","doi-asserted-by":"crossref","unstructured":"[14] Taylor, I. H. , Dong, S. and Rodriguez, A. , \u201cGelSlim 3.0: High-Resolution Measurement of Shape, Force and Slip in a Compact Tactile-Sensing Finger,\u201d 2022 International Conference on Robotics and Automation (ICRA) (2022) pp. 10781\u201310787.","DOI":"10.1109\/ICRA46639.2022.9811832"},{"key":"S0263574724000286_ref10","doi-asserted-by":"crossref","unstructured":"[10] Yuan, W. , Zhu, C. , Owens, A. , Srinivasan, M. A. and Adelson, E. H. , \u201cShape-independent Hardness Estimation using Deep Learning and a GelSight Tactile Sensor,\u201d 2017 IEEE International Conference on Robotics and Automation (ICRA) (2017) pp. 951\u2013958.","DOI":"10.1109\/ICRA.2017.7989116"},{"key":"S0263574724000286_ref5","doi-asserted-by":"crossref","first-page":"7391","DOI":"10.1109\/JSEN.2019.2916320","article-title":"Wearable system based on flexible FBG for respiratory and cardiac monitoring","volume":"19","author":"Presti","year":"2019","journal-title":"IEEE Sens. J."},{"key":"S0263574724000286_ref32","doi-asserted-by":"crossref","first-page":"156","DOI":"10.1016\/j.triboint.2012.12.018","article-title":"Friction-induced vibration of a lubricated mechanical system","volume":"61","author":"Sinou","year":"2013","journal-title":"Tribol. Int."},{"key":"S0263574724000286_ref25","first-page":"1","volume-title":"IEEE SENSORS","author":"Sankar","year":"2019"},{"key":"S0263574724000286_ref1","doi-asserted-by":"crossref","first-page":"21410","DOI":"10.1109\/JSEN.2022.3210210","article-title":"Hardware technology of vision-based tactile sensor: A review","volume":"22","author":"Zhang","year":"2022","journal-title":"IEEE Sens. J."},{"key":"S0263574724000286_ref20","doi-asserted-by":"crossref","first-page":"926","DOI":"10.1089\/soro.2020.0213","article-title":"Sim-to-real for high-resolution optical tactile sensing: From images to three-dimensional contact force distributions","volume":"9","author":"Sferrazza","year":"2022","journal-title":"Soft Robot."},{"key":"S0263574724000286_ref16","doi-asserted-by":"crossref","unstructured":"[16] Wei, F. , Zhao, J. , Shan, C. and Yuan, Z. , \u201cAlignment and Multi-Scale Fusion for Visual-Tactile Object Recognition,\u201d 2022 International Joint Conference on Neural Networks (IJCNN) (2022) pp. 1\u20138.","DOI":"10.1109\/IJCNN55064.2022.9892134"},{"key":"S0263574724000286_ref8","doi-asserted-by":"crossref","first-page":"2762","DOI":"10.3390\/s17122762","article-title":"GelSight: High-resolution robot tactile sensors for estimating geometry and force","volume":"17","author":"Yuan","year":"2017","journal-title":"Sensors"},{"key":"S0263574724000286_ref19","doi-asserted-by":"crossref","first-page":"111512","DOI":"10.1016\/j.sna.2019.07.036","article-title":"Flexible tactile sensor array for distributed tactile sensing and slip detection in robotic hand grasping","volume":"297","author":"Wang","year":"2019","journal-title":"Sens. Actuat. A: Phys."},{"key":"S0263574724000286_ref21","doi-asserted-by":"crossref","first-page":"3427","DOI":"10.1109\/LRA.2022.3146565","article-title":"Using collocated vision and tactile sensors for visual servoing and localization","volume":"7","author":"Chaudhury","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"S0263574724000286_ref33","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1109\/TRO.2020.3031245","article-title":"Slip detection for grasp stabilization with a multifingered tactile robot hand","volume":"37","author":"James","year":"2021","journal-title":"IEEE Trans. Robot."},{"key":"S0263574724000286_ref34","doi-asserted-by":"crossref","first-page":"3340","DOI":"10.1109\/LRA.2018.2852797","article-title":"Slip detection with a biomimetic tactile sensor","volume":"3","author":"James","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"S0263574724000286_ref17","doi-asserted-by":"crossref","unstructured":"[17] Xiong, P. and Yin, Y. , \u201cFVSight: A Novel Multimodal Tactile Sensor for Robotic Object Perception,\u201d 2022 IEEE International Conference on Networking, Sensing and Control (ICNSC) (2022) pp. 1\u20136.","DOI":"10.1109\/ICNSC55942.2022.10004135"},{"key":"S0263574724000286_ref31","doi-asserted-by":"crossref","first-page":"2259","DOI":"10.1109\/TII.2022.3195171","article-title":"Deeply supervised subspace learning for cross-modal material perception of known and unknown objects","volume":"19","author":"Xiong","year":"2023","journal-title":"IEEE Trans. Ind. Inform."},{"key":"S0263574724000286_ref7","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1017\/S0263574720000429","article-title":"Soft robotic finger embedded with visual sensor for bending perception","volume":"39","author":"Zhang","year":"2021","journal-title":"Robotica"},{"key":"S0263574724000286_ref24","doi-asserted-by":"crossref","unstructured":"[24] Yuan, W. , Wang, S. , Dong, S. and Adelson, E. , \u201cConnecting Look and Feel: Associating the Visual and Tactile Properties of Physical Materials,\u201d 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017) pp. 4494\u20134502.","DOI":"10.1109\/CVPR.2017.478"},{"key":"S0263574724000286_ref26","doi-asserted-by":"crossref","unstructured":"[26] Younes, H. , Ibrahim, A. , Rizk, M. and Valle, M. , \u201cData Oriented Approximate K-Nearest Neighbor Classifier for Touch Modality Recognition,\u201d 2019 15th Conference on Ph.D Research in Microelectronics and Electronics (PRIME) (2019) pp. 241\u2013244.","DOI":"10.1109\/PRIME.2019.8787753"},{"key":"S0263574724000286_ref13","doi-asserted-by":"crossref","unstructured":"[13] Donlon, E. , Dong, S. , Liu, M. , Li, J. , Adelson, E. and Rodriguez, A. , \u201cGelSlim: A High-Resolution, Compact, Robust, and Calibrated Tactile-sensing Finger,\u201d 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) (2018) pp. 1927\u20131934.","DOI":"10.1109\/IROS.2018.8593661"},{"key":"S0263574724000286_ref28","doi-asserted-by":"crossref","first-page":"656","DOI":"10.1109\/TIM.2016.2514779","article-title":"Object recognition using tactile measurements: Kernel sparse coding methods","volume":"65","author":"Liu","year":"2016","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"S0263574724000286_ref30","doi-asserted-by":"crossref","first-page":"6119","DOI":"10.1109\/TSMC.2023.3279023","article-title":"Robotic object perception based on multi-spectral few-shot coupled learning","volume":"53","author":"Xiong","year":"2023","journal-title":"IEEE Trans. Syst. Man. Cybern. Syst."}],"container-title":["Robotica"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0263574724000286","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,4,30]],"date-time":"2024-04-30T09:05:21Z","timestamp":1714467921000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0263574724000286\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,5]]},"references-count":37,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,5]]}},"alternative-id":["S0263574724000286"],"URL":"https:\/\/doi.org\/10.1017\/s0263574724000286","relation":{},"ISSN":["0263-5747","1469-8668"],"issn-type":[{"value":"0263-5747","type":"print"},{"value":"1469-8668","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,3,5]]}}}