{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,11]],"date-time":"2026-06-11T01:35:31Z","timestamp":1781141731871,"version":"3.54.1"},"reference-count":45,"publisher":"SAGE Publications","issue":"7","license":[{"start":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T00:00:00Z","timestamp":1759968000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"National Science Foundation","award":["2024646"],"award-info":[{"award-number":["2024646"]}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2026,6]]},"abstract":"<jats:p>Soft pneumatic robot manipulators are popular in industrial and human-interactive applications due to their compliance and flexibility. However, deploying them in real-world scenarios requires advanced sensing for tactile feedback and proprioception. Our work presents a novel vision-based approach for sensorizing soft robots. We demonstrate our approach on PneuGelSight, a pioneering pneumatic manipulator featuring high-resolution proprioception and tactile sensing via an embedded camera. To optimize the sensor\u2019s performance, we introduce a comprehensive pipeline that accurately simulates its optical and dynamic properties, facilitating a zero-shot knowledge transition from simulation to real-world applications. PneuGelSight and our sim-to-real pipeline provide a novel, easily implementable, and robust sensing methodology for soft robots, paving the way for the development of more advanced soft robots with enhanced sensory capabilities.<\/jats:p>","DOI":"10.1177\/02783649251378153","type":"journal-article","created":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:45:14Z","timestamp":1760035514000},"page":"1005-1020","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":4,"title":["PneuGelSight: Soft robotic vision-based proprioception and tactile sensing"],"prefix":"10.1177","volume":"45","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-1999-2806","authenticated-orcid":false,"given":"Ruohan","family":"Zhang","sequence":"first","affiliation":[{"name":"Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Urbana, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9663-7740","authenticated-orcid":false,"given":"Uksang","family":"Yoo","sequence":"additional","affiliation":[{"name":"Robotics Institute, Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yichen","family":"Li","sequence":"additional","affiliation":[{"name":"Robotics Institute, Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1594-5264","authenticated-orcid":false,"given":"Arpit","family":"Argawal","sequence":"additional","affiliation":[{"name":"Robotics Institute, Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8014-356X","authenticated-orcid":false,"given":"Wenzhen","family":"Yuan","sequence":"additional","affiliation":[{"name":"Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign, Urbana, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2025,10,9]]},"reference":[{"key":"e_1_3_5_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9561122"},{"key":"e_1_3_5_3_1","doi-asserted-by":"publisher","DOI":"10.1177\/02783649251339680"},{"key":"e_1_3_5_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/adma.201504015"},{"key":"e_1_3_5_5_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abq4821"},{"key":"e_1_3_5_6_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.adk4533"},{"key":"e_1_3_5_7_1","first-page":"1","volume-title":"Development of a Tactile Sensor Based on Biologically Inspired Edge Encoding","author":"Chorley C","year":"2009","unstructured":"Chorley C, Melhuish C, Pipe T, et al. (2009) Development of a Tactile Sensor Based on Biologically Inspired Edge Encoding, 1\u20136. IEEE."},{"key":"e_1_3_5_8_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.adi5908"},{"key":"e_1_3_5_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2015.7353689"},{"key":"e_1_3_5_10_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aan3028"},{"key":"e_1_3_5_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_5_12_1","first-page":"2553","volume-title":"Shape estimation in natural illumination CVPR 2011","author":"Johnson MK","year":"2011","unstructured":"Johnson MK, Adelson EH (2011) Shape estimation in natural illumination CVPR 2011, 2553\u20132560. IEEE."},{"key":"e_1_3_5_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2010324.1964941"},{"key":"e_1_3_5_14_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.aac5082"},{"key":"e_1_3_5_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA57147.2024.10610823"},{"key":"e_1_3_5_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2025.3527306"},{"key":"e_1_3_5_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/IECON.1990.149155"},{"key":"e_1_3_5_18_1","article-title":"Pointnet: deep learning on point sets for 3d classification and segmentation","author":"Qi CR","year":"2017","unstructured":"Qi CR, Su H, Mo K, et al. (2017) Pointnet: deep learning on point sets for 3d classification and segmentation. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).","journal-title":"2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"e_1_3_5_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48891.2023.10160893"},{"key":"e_1_3_5_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eml.2015.09.005"},{"key":"e_1_3_5_21_1","article-title":"U-net: convolutional networks for biomedical image segmentation","author":"Ronneberger O","year":"2015","unstructured":"Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. 2015 Medical Image Computing and Computer-Assisted Intervention (MICCAI).","journal-title":"2015 Medical Image Computing and Computer-Assisted Intervention (MICCAI)"},{"key":"e_1_3_5_22_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.ade2184"},{"key":"e_1_3_5_23_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.adj9769"},{"key":"e_1_3_5_24_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14543"},{"issue":"4","key":"e_1_3_5_25_1","first-page":"1877","article-title":"Sensing and reconstruction of 3-d deformation on pneumatic soft robots","volume":"26","author":"Scharff RBN","year":"2021","unstructured":"Scharff RBN, Fang G, Tian Y, et al. (2021) Sensing and reconstruction of 3-d deformation on pneumatic soft robots. IEEE 26(4): 1877\u20131885.","journal-title":"IEEE"},{"key":"e_1_3_5_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9197369"},{"key":"e_1_3_5_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48891.2023.10161321"},{"key":"e_1_3_5_28_1","doi-asserted-by":"publisher","DOI":"10.1364\/JOSA.68.000297"},{"key":"e_1_3_5_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9811832"},{"key":"e_1_3_5_30_1","doi-asserted-by":"publisher","DOI":"10.1177\/0278364920910465"},{"key":"e_1_3_5_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/RoboSoft55895.2023.10122097"},{"key":"e_1_3_5_32_1","doi-asserted-by":"publisher","DOI":"10.1002\/adma.201706383"},{"key":"e_1_3_5_33_1","unstructured":"Uppalapati NK Walt B Havens AJ et al. (2020) A berry picking robot with a hybrid soft-rigid arm: design and task space control. In Robotics: Science and Systems (RSS) Corvallis OR USA July 12\u201316 2020."},{"key":"e_1_3_5_34_1","first-page":"3","article-title":"Properties of cutaneous mechanoreceptors in the human hand related to touch sensation","volume":"3","author":"Vallbo A","year":"1984","unstructured":"Vallbo A, Johansson R (1984) Properties of cutaneous mechanoreceptors in the human hand related to touch sensation. Human Neurobiology 3: 3\u201314.","journal-title":"Human Neurobiology"},{"key":"e_1_3_5_35_1","doi-asserted-by":"publisher","DOI":"10.1177\/02783649231168954"},{"key":"e_1_3_5_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2975709"},{"key":"e_1_3_5_37_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2020.103427"},{"key":"e_1_3_5_38_1","article-title":"Cushsense: soft, stretchable, and comfortable tactile-sensing skin for physical human-robot interaction","author":"Xu B","year":"2024","unstructured":"Xu B, Zhong L, Zhang G, et al. (2024) Cushsense: soft, stretchable, and comfortable tactile-sensing skin for physical human-robot interaction. 2024 IEEE International Conference on Robotics and Automation (ICRA).","journal-title":"2024 IEEE International Conference on Robotics and Automation (ICRA)"},{"key":"e_1_3_5_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/HUMANOIDS.2017.8246881"},{"key":"e_1_3_5_40_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abc8801"},{"key":"e_1_3_5_41_1","doi-asserted-by":"crossref","unstructured":"Yoo U Zhao H Altamirano A et al. (2023) Toward zero-shot sim-to-real transfer learning for pneumatic soft robot 3d proprioceptive sensing. 2023 International Conference on Robotics and Automation (ICRA).","DOI":"10.1109\/ICRA48891.2023.10160384"},{"key":"e_1_3_5_42_1","doi-asserted-by":"publisher","DOI":"10.3390\/s17122762"},{"key":"e_1_3_5_43_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-32126-1"},{"key":"e_1_3_5_44_1","article-title":"GelSight svelte: a human finger-shaped single-camera tactile robot finger with large sensing coverage and proprioceptive sensing","author":"Zhao J","year":"2023","unstructured":"Zhao J, Adelson EH (2023) GelSight svelte: a human finger-shaped single-camera tactile robot finger with large sensing coverage and proprioceptive sensing. 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS).","journal-title":"2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)"},{"key":"e_1_3_5_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487590"},{"key":"e_1_3_5_46_1","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.adi2377"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649251378153","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/02783649251378153","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649251378153","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,5]],"date-time":"2026-06-05T06:01:07Z","timestamp":1780639267000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/02783649251378153"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,9]]},"references-count":45,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2026,6]]}},"alternative-id":["10.1177\/02783649251378153"],"URL":"https:\/\/doi.org\/10.1177\/02783649251378153","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,9]]}}}