{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T16:57:10Z","timestamp":1791046630145,"version":"4.1.0"},"reference-count":44,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,2,28]],"date-time":"2023-02-28T00:00:00Z","timestamp":1677542400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2018AAA0103003"],"award-info":[{"award-number":["2018AAA0103003"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["62273342"],"award-info":[{"award-number":["62273342"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Natural Science Foundation of China","award":["2018AAA0103003"],"award-info":[{"award-number":["2018AAA0103003"]}]},{"name":"National Natural Science Foundation of China","award":["62273342"],"award-info":[{"award-number":["62273342"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>GelStereo sensing technology is capable of performing three-dimensional (3D) contact shape measurement under various contact structures such as bionic curved surfaces, which has promising advantages in the field of visuotactile sensing. However, due to multi-medium ray refraction in the imaging system, robust and high-precision tactile 3D reconstruction remains a challenging problem for GelStereo-type sensors with different structures. In this paper, we first propose a universal Refractive Stereo Ray Tracing (RSRT) model for GelStereo-type sensing systems to realize 3D reconstruction of the contact surface. Moreover, a relative geometry-based optimization method is presented to calibrate multiple parameters of the proposed RSRT model, such as the refractive indices and structural dimensions. Furthermore, extensive quantitative calibration experiments are performed on four different GelStereo sensing platforms; the experimental results show that the proposed calibration pipeline can achieve less than 0.35 mm in Euclidean distance error, based on which we believe that the proposed refractive calibration method can be further applied in more complex GelStereo-type and other similar visuotactile sensing systems. Such high-precision visuotactile sensors can facilitate the study of robotic dexterous manipulation.<\/jats:p>","DOI":"10.3390\/s23052675","type":"journal-article","created":{"date-parts":[[2023,3,1]],"date-time":"2023-03-01T02:02:56Z","timestamp":1677636176000},"page":"2675","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":14,"title":["High-Precision 3D Reconstruction Study with Emphasis on Refractive Calibration of GelStereo-Type Sensors"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2605-9097","authenticated-orcid":false,"given":"Chaofan","family":"Zhang","sequence":"first","affiliation":[{"name":"State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4750-3011","authenticated-orcid":false,"given":"Shaowei","family":"Cui","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1390-9219","authenticated-orcid":false,"given":"Shuo","family":"Wang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China"},{"name":"Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai 200031, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2664-0121","authenticated-orcid":false,"given":"Jingyi","family":"Hu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"School of Future Technology, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-1421-9770","authenticated-orcid":false,"given":"Yipeng","family":"Huangfu","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3864-1386","authenticated-orcid":false,"given":"Boyue","family":"Zhang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China"},{"name":"School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing 100049, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"91","DOI":"10.5014\/ajot.51.2.91","article-title":"Tactile defensiveness and stereotyped behaviors","volume":"51","author":"Baranek","year":"1997","journal-title":"Am. J. Occup. Ther."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1002\/rcs.3","article-title":"Human tactile perception as a standard for artificial tactile sensing\u2014A review","volume":"1","author":"Dargahi","year":"2004","journal-title":"Int. J. Med. Robot. Comput. Assist. Surg."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/j.cub.2003.12.054","article-title":"Early vision impairs tactile perception in the blind","volume":"14","author":"Spence","year":"2004","journal-title":"Curr. Biol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1619","DOI":"10.1109\/TRO.2020.3003230","article-title":"A review of tactile information: Perception and action through touch","volume":"36","author":"Li","year":"2020","journal-title":"IEEE Trans. Robot."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.scib.2019.10.021","article-title":"Recent progress in tactile sensors and their applications in intelligent systems","volume":"65","author":"Liu","year":"2020","journal-title":"Sci. Bull."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"7628","DOI":"10.1109\/JSEN.2020.2979662","article-title":"Visuotactile sensors with emphasis on gelsight sensor: A review","volume":"20","author":"Abad","year":"2020","journal-title":"IEEE Sens. J."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Chi, C., Sun, X., Xue, N., Li, T., and Liu, C. (2018). Recent progress in technologies for tactile sensors. Sensors, 18.","DOI":"10.3390\/s18040948"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"eaau6914","DOI":"10.1126\/scirobotics.aau6914","article-title":"A hierarchically patterned, bioinspired e-skin able to detect the direction of applied pressure for robotics","volume":"3","author":"Boutry","year":"2018","journal-title":"Sci. Robot."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4024","DOI":"10.1038\/s41467-019-12030-x","article-title":"An ultrahigh resolution pressure sensor based on percolative metal nanoparticle arrays","volume":"10","author":"Chen","year":"2019","journal-title":"Nat. Commun."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"e1500661","DOI":"10.1126\/sciadv.1500661","article-title":"Fingertip skin\u2013inspired microstructured ferroelectric skins discriminate static\/dynamic pressure and temperature stimuli","volume":"1","author":"Park","year":"2015","journal-title":"Sci. Adv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Yuan, W., Dong, S., and Adelson, E.H. (2017). Gelsight: High-resolution robot tactile sensors for estimating geometry and force. Sensors, 17.","DOI":"10.3390\/s17122762"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Yuan, W., Li, R., Srinivasan, M.A., and Adelson, E.H. (2015, January 26\u201330). Measurement of shear and slip with a GelSight tactile sensor. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139016"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Dong, S., Yuan, W., and Adelson, E.H. (2017, January 24\u201328). Improved gelsight tactile sensor for measuring geometry and slip. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8202149"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhang, C., Cui, S., Cai, Y., Hu, J., Wang, R., and Wang, S. (2022, January 23\u201327). Learning-based Six-axis Force\/Torque Estimation Using GelStereo Fingertip Visuotactile Sensing. Proceedings of the 2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Kyoto, Japan.","DOI":"10.1109\/IROS47612.2022.9981100"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Ma, D., Donlon, E., Dong, S., and Rodriguez, A. (2019, January 20\u201324). Dense tactile force estimation using GelSlim and inverse FEM. Proceedings of the 2019 International Conference on Robotics and Automation (ICRA), Montreal, QC, Canada.","DOI":"10.1109\/ICRA.2019.8794113"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Johnson, M.K., and Adelson, E.H. (2009, January 20\u201325). Retrographic sensing for the measurement of surface texture and shape. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA.","DOI":"10.1109\/CVPR.2009.5206534"},{"key":"ref_17","first-page":"1","article-title":"Self-Supervised Contact Geometry Learning by GelStereo Visuotactile Sensing","volume":"71","author":"Cui","year":"2021","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Li, R., Platt, R., Yuan, W., ten Pas, A., Roscup, N., Srinivasan, M.A., and Adelson, E. (2014, January 14\u201318). Localization and manipulation of small parts using gelsight tactile sensing. Proceedings of the 2014 IEEE\/RSJ International Conference on Intelligent Robots and Systems, Chicago, IL, USA.","DOI":"10.1109\/IROS.2014.6943123"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1385","DOI":"10.1177\/02783649211027233","article-title":"Cable manipulation with a tactile-reactive gripper","volume":"40","author":"She","year":"2021","journal-title":"Int. J. Robot. Res."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Wang, C., Wang, S., Romero, B., Veiga, F., and Adelson, E. (2020, January 25\u201329). Swingbot: Learning physical features from in-hand tactile exploration for dynamic swing-up manipulation. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341006"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"6015","DOI":"10.1109\/TIE.2021.3090697","article-title":"In-hand object localization using a novel high-resolution visuotactile sensor","volume":"69","author":"Cui","year":"2021","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_22","unstructured":"Pan, C., Lepert, M., Yuan, S., Antonova, R., and Bohg, J. (2022). Task-Driven In-Hand Manipulation of Unknown Objects with Tactile Sensing. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1038\/s42256-021-00439-3","article-title":"A soft thumb-sized vision-based sensor with accurate all-round force perception","volume":"4","author":"Sun","year":"2022","journal-title":"Nat. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Do, W.K., Jurewicz, B., and Kennedy III, M. (2022). DenseTact 2.0: Optical Tactile Sensor for Shape and Force Reconstruction. arXiv.","DOI":"10.1109\/ICRA46639.2022.9811966"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Gomes, D.F., Lin, Z., and Luo, S. (2020, January 25\u201329). GelTip: A finger-shaped optical tactile sensor for robotic manipulation. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9340881"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Yamaguchi, A., and Atkeson, C.G. (2017, January 15\u201317). Implementing tactile behaviors using fingervision. Proceedings of the 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids), Birmingham, UK.","DOI":"10.1109\/HUMANOIDS.2017.8246881"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1940002","DOI":"10.1142\/S0219843619400024","article-title":"Tactile behaviors with the vision-based tactile sensor FingerVision","volume":"16","author":"Yamaguchi","year":"2019","journal-title":"Int. J. Humanoid Robot."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1089\/soro.2017.0052","article-title":"The tactip family: Soft optical tactile sensors with 3D-printed biomimetic morphologies","volume":"5","author":"Pestell","year":"2018","journal-title":"Soft Robot."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"21131","DOI":"10.1109\/JSEN.2021.3100645","article-title":"Soft biomimetic optical tactile sensing with the TacTip: A review","volume":"21","author":"Lepora","year":"2021","journal-title":"IEEE Sens. J."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Donlon, E., Dong, S., Liu, M., Li, J., Adelson, E., and Rodriguez, A. (2018, January 1\u20135). Gelslim: A high-resolution, compact, robust, and calibrated tactile-sensing finger. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8593661"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Taylor, I., Dong, S., and Rodriguez, A. (2021). Gelslim3.0: High-resolution measurement of shape, force and slip in a compact tactile-sensing finger. arXiv.","DOI":"10.1109\/ICRA46639.2022.9811832"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Romero, B., Veiga, F., and Adelson, E. (August, January 31). Soft, round, high resolution tactile fingertip sensors for dexterous robotic manipulation. Proceedings of the 2020 IEEE International Conference on Robotics and Automation (ICRA), Paris, France.","DOI":"10.1109\/ICRA40945.2020.9196909"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Do, W.K., and Kennedy III, M. (2022). DenseTact: Optical Tactile Sensor for Dense Shape Reconstruction. arXiv.","DOI":"10.1109\/ICRA46639.2022.9811966"},{"key":"ref_34","unstructured":"Alspach, A., Hashimoto, K., Kuppuswamy, N., and Tedrake, R. (2019). Proceedings of the 2019 2nd IEEE International Conference on Soft Robotics (RoboSoft), IEEE."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Kuppuswamy, N., Alspach, A., Uttamchandani, A., Creasey, S., Ikeda, T., and Tedrake, R. (2020, January 25\u201329). Soft-bubble grippers for robust and perceptive manipulation. Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9341534"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"17727","DOI":"10.1109\/JSEN.2022.3195129","article-title":"Effects of refraction model on binocular visuotactile sensing of 3-D deformation","volume":"22","author":"Ma","year":"2022","journal-title":"IEEE Sensors J."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Hu, J., Cui, S., Wang, S., Zhang, C., Wang, R., Chen, L., and Li, Y. (2023). GelStereo Palm: A Novel Curved Visuotactile Sensor for 3D Geometry Sensing. IEEE Trans. Ind. Inform.","DOI":"10.1109\/TII.2023.3241685"},{"key":"ref_38","unstructured":"Glassner, A.S. (1989). An Introduction to Ray Tracing, Morgan Kaufmann."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1330","DOI":"10.1109\/34.888718","article-title":"A flexible new technique for camera calibration","volume":"22","author":"Zhang","year":"2000","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"30810","DOI":"10.3390\/s151229831","article-title":"Calibration techniques for accurate measurements by underwater camera systems","volume":"15","author":"Shortis","year":"2015","journal-title":"Sensors"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Meline, A., Triboulet, J., and Jouvencel, B. (2010, January 20-23). A camcorder for 3D underwater reconstruction of archeological objects. Proceedings of the OCEANS 2010 MTS\/IEEE SEATTLE, Seattle, WA, USA.","DOI":"10.1109\/OCEANS.2010.5664572"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"2280","DOI":"10.1016\/j.patcog.2014.01.005","article-title":"Automatic generation and detection of highly reliable fiducial markers under occlusion","volume":"47","year":"2014","journal-title":"Pattern Recognit."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1007\/BF01427149","article-title":"Iterative point matching for registration of free-form curves and surfaces","volume":"13","author":"Zhang","year":"1994","journal-title":"Int. J. Comput. Vis."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.advengsoft.2016.01.003","article-title":"Study on the reconstruction method of stereo vision in glass flume","volume":"94","author":"Du","year":"2016","journal-title":"Adv. Eng. Softw."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2675\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:44:44Z","timestamp":1760121884000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2675"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,28]]},"references-count":44,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23052675"],"URL":"https:\/\/doi.org\/10.3390\/s23052675","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,28]]}}}