{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T18:40:50Z","timestamp":1780512050134,"version":"3.54.1"},"reference-count":45,"publisher":"SAGE Publications","issue":"13","license":[{"start":{"date-parts":[[2025,5,24]],"date-time":"2025-05-24T00:00:00Z","timestamp":1748044800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2025,11]]},"abstract":"<jats:p>GelSight family of vision-based tactile sensors has proven to be effective for multiple robot perception and manipulation tasks. These sensors are based on an internal optical system and an embedded camera to capture the deformation of the soft sensor surface, inferring the high-resolution geometry of the objects in contact. However, customizing the sensors for different robot hands requires a tedious trial-and-error process to redesign the optical system. In this paper, we formulate the GelSight sensor design process as a systematic and objective-driven design problem and perform the design optimization with a physically accurate optical simulation. The method is based on modularizing and parameterizing the sensor\u2019s optical components and designing four generalizable objective functions to evaluate the sensor. We implement the method with an interactive and easy-to-use toolbox called OptiSense Studio. With the toolbox, non-sensor experts can quickly optimize their sensor design in both forward and inverse ways following our predefined modules and steps. We demonstrate our system with four different GelSight sensors by quickly optimizing their initial design in simulation and transferring it to the real sensors.<\/jats:p>","DOI":"10.1177\/02783649251339680","type":"journal-article","created":{"date-parts":[[2025,5,24]],"date-time":"2025-05-24T10:14:41Z","timestamp":1748081681000},"page":"2195-2214","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":4,"title":["A modularized design approach for GelSight family of vision-based tactile sensors"],"prefix":"10.1177","volume":"44","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1594-5264","authenticated-orcid":false,"given":"Arpit","family":"Agarwal","sequence":"first","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-9470-5813","authenticated-orcid":false,"given":"Mohammad Amin","family":"Mirzaee","sequence":"additional","affiliation":[{"name":"University of Illinois Urbana-Champaign"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8732-8783","authenticated-orcid":false,"given":"Xiping","family":"Sun","sequence":"additional","affiliation":[{"name":"University of Illinois Urbana-Champaign"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8014-356X","authenticated-orcid":false,"given":"Wenzhen","family":"Yuan","sequence":"additional","affiliation":[{"name":"University of Illinois Urbana-Champaign"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2025,5,24]]},"reference":[{"key":"e_1_3_4_2_1","doi-asserted-by":"publisher","DOI":"10.1002\/wics.101"},{"key":"e_1_3_4_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9561122"},{"key":"e_1_3_4_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/aisy.202300042"},{"key":"e_1_3_4_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2018.2852779"},{"key":"e_1_3_4_6_1","doi-asserted-by":"publisher","DOI":"10.3390\/s23052661"},{"key":"e_1_3_4_7_1","volume-title":"Blender Foundation","author":"Community BO","year":"2018","unstructured":"Community BO (2018) Blender - a 3D modelling and rendering package. In: Blender Foundation. Amsterdam: Stichting Blender Foundation. https:\/\/www.blender.org\/"},{"key":"e_1_3_4_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9811966"},{"key":"e_1_3_4_9_1","article-title":"Densetact-mini: an optical tactile sensor for grasping multi-scale objects from flat surfaces","author":"Do WK","year":"2023","unstructured":"Do WK, Dhawan AK, Kitzmann M, et al. (2023) Densetact-mini: an optical tactile sensor for grasping multi-scale objects from flat surfaces. arXiv preprint arXiv:2309.08860.","journal-title":"arXiv preprint arXiv:2309.08860"},{"key":"e_1_3_4_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2017.8202149"},{"key":"e_1_3_4_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2018.8593661"},{"key":"e_1_3_4_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275059"},{"key":"e_1_3_4_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1618452.1618487"},{"key":"e_1_3_4_14_1","article-title":"The cma evolution strategy: a tutorial","author":"Hansen N","year":"2016","unstructured":"Hansen N (2016) The cma evolution strategy: a tutorial. arXiv preprint arXiv:1604.00772.","journal-title":"arXiv preprint arXiv:1604.00772"},{"key":"e_1_3_4_15_1","unstructured":"Jakob W (2010) Mitsuba renderer. https:\/\/www.mitsuba-renderer.org."},{"key":"e_1_3_4_16_1","unstructured":"Jakob W (2024) BSDFs - Mitsuba 3 \u2014 mitsuba.readthedocs. https:\/\/mitsuba.readthedocs.io\/en\/latest\/src\/generated\/plugins_bsdfs.html"},{"key":"e_1_3_4_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528223.3530099"},{"key":"e_1_3_4_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206534"},{"key":"e_1_3_4_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2977257"},{"key":"e_1_3_4_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2014.6943123"},{"key":"e_1_3_4_21_1","volume-title":"9dtact: a compact vision-based tactile sensor for accurate 3d shape reconstruction and generalizable 6d force estimation","author":"Lin C","year":"2023","unstructured":"Lin C, Zhang H, Xu J, et al. (2023a) 9dtact: a compact vision-based tactile sensor for accurate 3d shape reconstruction and generalizable 6d force estimation. Piscataway: IEEE Robotics and Automation Letters."},{"key":"e_1_3_4_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2023.3302191"},{"key":"e_1_3_4_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/RoboSoft54090.2022.9762175"},{"key":"e_1_3_4_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/RoboSoft55895.2023.10122078"},{"key":"e_1_3_4_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/RoboSoft55895.2023.10122053"},{"key":"e_1_3_4_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2025.3527306"},{"key":"e_1_3_4_27_1","doi-asserted-by":"publisher","DOI":"10.1117\/1.JEI.26.1.013014"},{"key":"e_1_3_4_28_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-71151-1_10"},{"key":"e_1_3_4_29_1","volume-title":"Physically Based Rendering: From Theory to Implementation","author":"Pharr M","year":"2023","unstructured":"Pharr M, Jakob W, Humphreys G (2023) Physically Based Rendering: From Theory to Implementation. Cambridge: MIT Press."},{"key":"e_1_3_4_30_1","first-page":"2549","volume-title":"Conference on Robot Learning","author":"Qi H","year":"2023","unstructured":"Qi H, Yi B, Suresh S, et al. (2023) General in-hand object rotation with vision and touch. In: Conference on Robot Learning. Westminster: PMLR, 2549\u20132564."},{"key":"e_1_3_4_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9196909"},{"key":"e_1_3_4_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9197369"},{"key":"e_1_3_4_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3142412"},{"key":"e_1_3_4_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS47612.2022.9981863"},{"key":"e_1_3_4_35_1","volume-title":"Boundary Representation Modelling Techniques","author":"Stroud I","year":"2006","unstructured":"Stroud I (2006) Boundary Representation Modelling Techniques. Berlin: Springer Science & Business Media."},{"key":"e_1_3_4_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9812040"},{"key":"e_1_3_4_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9811832"},{"key":"e_1_3_4_38_1","article-title":"Gelsight360: an omnidirectional camera-based tactile sensor for dexterous robotic manipulation","author":"Tippur MH","year":"2023","unstructured":"Tippur MH, Adelson EH (2023) Gelsight360: an omnidirectional camera-based tactile sensor for dexterous robotic manipulation. arXiv preprint arXiv:2304.04268.","journal-title":"arXiv preprint arXiv:2304.04268"},{"key":"e_1_3_4_39_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-019-0686-2"},{"key":"e_1_3_4_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9560783"},{"key":"e_1_3_4_41_1","doi-asserted-by":"publisher","DOI":"10.1089\/soro.2017.0052"},{"key":"e_1_3_4_42_1","article-title":"An end-to-end differentiable framework for contact-aware robot design","author":"Xu J","year":"2021","unstructured":"Xu J, Chen T, Zlokapa L, et al. (2021a) An end-to-end differentiable framework for contact-aware robot design. arXiv preprint arXiv:2107.07501.","journal-title":"arXiv preprint arXiv:2107.07501"},{"key":"e_1_3_4_43_1","article-title":"An end-to-end differentiable framework for contact-aware robot design","author":"Xu J","year":"2021","unstructured":"Xu J, Chen T, Zlokapa L, et al. (2021b) An end-to-end differentiable framework for contact-aware robot design. arXiv preprint arXiv:2107.07501.","journal-title":"arXiv preprint arXiv:2107.07501"},{"key":"e_1_3_4_44_1","doi-asserted-by":"publisher","DOI":"10.3390\/s17122762"},{"key":"e_1_3_4_45_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.sna.2024.115090"},{"key":"e_1_3_4_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS55552.2023.10341646"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649251339680","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/02783649251339680","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649251339680","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:17:27Z","timestamp":1777457847000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/02783649251339680"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,24]]},"references-count":45,"journal-issue":{"issue":"13","published-print":{"date-parts":[[2025,11]]}},"alternative-id":["10.1177\/02783649251339680"],"URL":"https:\/\/doi.org\/10.1177\/02783649251339680","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,24]]}}}