{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T20:43:06Z","timestamp":1781815386994,"version":"3.54.5"},"reference-count":27,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2022,8,28]],"date-time":"2022-08-28T00:00:00Z","timestamp":1661644800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2019YFB1311005"],"award-info":[{"award-number":["2019YFB1311005"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["52175510"],"award-info":[{"award-number":["52175510"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["52188102"],"award-info":[{"award-number":["52188102"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2019QNRC001"],"award-info":[{"award-number":["2019QNRC001"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2019YFB1311005"],"award-info":[{"award-number":["2019YFB1311005"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["52175510"],"award-info":[{"award-number":["52175510"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["52188102"],"award-info":[{"award-number":["52188102"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2019QNRC001"],"award-info":[{"award-number":["2019QNRC001"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Young Elite Scientists Sponsorship Program by CAST","award":["2019YFB1311005"],"award-info":[{"award-number":["2019YFB1311005"]}]},{"name":"Young Elite Scientists Sponsorship Program by CAST","award":["52175510"],"award-info":[{"award-number":["52175510"]}]},{"name":"Young Elite Scientists Sponsorship Program by CAST","award":["52188102"],"award-info":[{"award-number":["52188102"]}]},{"name":"Young Elite Scientists Sponsorship Program by CAST","award":["2019QNRC001"],"award-info":[{"award-number":["2019QNRC001"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Tactile sensing plays an important role for robots\u2019 perception, but the existing tactile technologies have multiple limitations. Visual-tactile sensor (VTS) is a newly developed tactile detector; it perceives the contacting surface shape, or even more refined texture, by way of the contact deformation image captured by a camera. A conventional visual perception is usually formulated as a data processing. It suffers issues of cumbersome training set and complicated calibration procedures. A novel model-based depth perceptual scheme is proposed where a mapping from the image intensity to the contact geometry is mathematically formulated with an associated tailored fast solver. The hardware calibration requires single image only, leading to an outstanding algorithmic robustness. The non-uniformity of the illumination condition is embodied by the stereo model, resulting in a robust depth perception precision. Compression tests on a prototype VTS showed the method\u2019s capability in high-quality geometry reconstruction. Both contacting shape and texture were captured at a root-mean-square error down to a sub-millimeter level. The feasibility of the proposed in a pose estimation application is further experimentally validated. The associated tests yielded estimation errors that were all less than 3\u00b0 in terms of spatial orientation and all less than 1mm in terms of translation.<\/jats:p>","DOI":"10.3390\/s22176470","type":"journal-article","created":{"date-parts":[[2022,8,30]],"date-time":"2022-08-30T01:37:55Z","timestamp":1661823475000},"page":"6470","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Model-Based 3D Contact Geometry Perception for Visual Tactile Sensor"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5179-0274","authenticated-orcid":false,"given":"Jingjing","family":"Ji","sequence":"first","affiliation":[{"name":"State Key Lab of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuting","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Lab of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Huan","family":"Ma","sequence":"additional","affiliation":[{"name":"State Key Lab of Digital Manufacturing Equipment and Technology, Huazhong University of Science and Technology, Wuhan 430074, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,8,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Shimonomura, K. (2019). Tactile image sensors employing camera: A review. Sensors, 19.","DOI":"10.3390\/s19183933"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"7628","DOI":"10.1109\/JSEN.2020.2979662","article-title":"Visuotactile sensors with emphasis on GelSight sensor: A review","volume":"20","author":"Abad","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1007\/s10846-021-01431-0","article-title":"On the design and development of vision-based tactile sensors","volume":"102","author":"Shah","year":"2021","journal-title":"J. Intell. Rob. Syst. Theor. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Li, R., and Adelson, E.H. (2013, January 23\u201328). Sensing and recognizing surface textures using a GelSight sensor. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.164"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"501309","DOI":"10.1109\/TIM.2022.3165254","article-title":"Tactile-based fabric defect detection using convolutional neural network with attention mechanism","volume":"71","author":"Fang","year":"2022","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Yuan, W., Zhu, C., Owens, A., Srinivasan, M.A., and Adelson, E.H. (June, January 29). Shape-independent hardness estimation using deep learning and a GelSight tactile sensor. Proceedings of the 2017 International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989116"},{"key":"ref_7","unstructured":"Zhang, Y., Kan, Z., Tse, Y.A., Yang, Y., and Wang, M.Y. (2018). FingerVision tactile sensor design and slip detection using convolutional LSTM network. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Bauza, M., Bronars, A., and Rodriguez, A. (2022). Tac2Pose: Tactile object pose estimation from the first touch. arXiv.","DOI":"10.1177\/02783649231196925"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Johnson, M.K., and Adelson, E.H. (2009, January 20\u201325). Retrographic sensing for the measurement of surface texture and shape. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Miami, FL, USA.","DOI":"10.1109\/CVPRW.2009.5206534"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Yuan, W., Dong, S., and Adelson, E.H. (2017). GelSight: High-resolution robot tactile sensors for estimating geometry and force. Sensors, 17.","DOI":"10.3390\/s17122762"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Taylor, I., Dong, S., and Rodriguez, A. (2022, January 23\u201327). GelSlim 3.0: High-resolution measurement of shape, force and slip in a compact tactile-sensing finger. Proceedings of the 2022 IEEE International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9811832"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Li, J., Dong, S., and Adelson, E.H. (2018, January 12\u201315). End-to-end pixelwise surface normal estimation with convolutional neural networks and shape reconstruction using GelSight sensor. Proceedings of the 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ROBIO.2018.8665351"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wang, S., She, Y., Romero, B., and Adelson, E.H. (June, January 30). GelSight wedge: Measuring high-resolution 3d contact geometry with a compact robot finger. Proceedings of the 2021 IEEE International Conference on Robotics and Automation (ICRA), Xi\u2019an, China.","DOI":"10.1109\/ICRA48506.2021.9560783"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Sodhi, P., Kaess, M., Mukadam, M., and Anderson, S. (2022, January 23\u201327). Patchgraph: In-hand tactile tracking with learned surface normals. Proceedings of the 2022 IEEE International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9811953"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3838","DOI":"10.1109\/LRA.2020.2977257","article-title":"Digit: A novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation","volume":"5","author":"Lambeta","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"3323","DOI":"10.1109\/LRA.2020.2974649","article-title":"A vision-based soft somatosensory system for distributed pressure and temperature sensing","volume":"5","author":"Yu","year":"2020","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2386","DOI":"10.1109\/LRA.2019.2893434","article-title":"Sensing the frictional state of a robotic skin via subtractive color mixing","volume":"4","author":"Lin","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2147","DOI":"10.1109\/LRA.2021.3061306","article-title":"High-resolution 3-dimensional contact deformation tracking for FingerVision sensor with dense random color pattern","volume":"6","author":"Du","year":"2021","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"16492","DOI":"10.1109\/JSEN.2021.3076815","article-title":"An enhanced FingerVision for contact spatial surface sensing","volume":"21","author":"Yang","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"173438","DOI":"10.1109\/ACCESS.2019.2956882","article-title":"Ground truth force distribution for learning-based tactile sensing: A finite element approach","volume":"7","author":"Sferrazza","year":"2019","journal-title":"IEEE Access"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Ambrus, R., Guizilini, V., Kuppuswamy, N., Beasulieu, A., Gaidon, A., and Alspach, A. (2021, January 12\u201316). Monocular depth estimation for soft visuotactile sensors. Proceedings of the 2021 IEEE 4th International Conference on Soft Robotics (RoboSoft), New Haven, CT, USA.","DOI":"10.1109\/RoboSoft51838.2021.9479234"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Do, W.K., and Kennedy, M. (2022, January 23\u201327). DenseTact: Optical tactile sensor for dense shape reconstruction. Proceedings of the 2022 IEEE International Conference on Robotics and Automation (ICRA), Philadelphia, PA, USA.","DOI":"10.1109\/ICRA46639.2022.9811966"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Dong, S., Yuan, W., and Adelson, E.H. (2017, January 24\u201328). Improved GelSight tactile sensor for measuring geometry and slip. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8202149"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"116605","DOI":"10.1016\/j.image.2021.116605","article-title":"Near-field photometric stereo using a ring-light imaging device","volume":"102","author":"Fan","year":"2022","journal-title":"Signal Process Image Commun."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"3930","DOI":"10.1109\/LRA.2022.3146945","article-title":"TACTO: A fast, flexible and open-source simulator for high-resolution vision-based tactile sensors","volume":"7","author":"Wang","year":"2022","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Rusu, R.B., Blodow, N., and Beetz, M. (2009, January 12\u201317). Fast point feature histograms (FPFH) for 3D registration. Proceedings of the 2009 IEEE International Conference on Robotics and Automation (ICRA), Kobe, Japan.","DOI":"10.1109\/ROBOT.2009.5152473"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1109\/34.121791","article-title":"A method for registration of 3-D shapes","volume":"14","author":"Besl","year":"1992","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/17\/6470\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:18:59Z","timestamp":1760141939000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/17\/6470"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,28]]},"references-count":27,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2022,9]]}},"alternative-id":["s22176470"],"URL":"https:\/\/doi.org\/10.3390\/s22176470","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,28]]}}}