{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T06:19:12Z","timestamp":1782886752476,"version":"3.54.5"},"reference-count":81,"publisher":"Springer Science and Business Media LLC","issue":"8070","license":[{"start":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T00:00:00Z","timestamp":1750809600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T00:00:00Z","timestamp":1750809600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Nature"],"published-print":{"date-parts":[[2025,7,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Mirroring the complex structures and diverse functions of natural organisms is a long-standing challenge in robotics<jats:sup>1\u20134<\/jats:sup>. Modern fabrication techniques have greatly expanded the feasible hardware<jats:sup>5\u20138<\/jats:sup>, but using these systems requires control software to translate the desired motions into actuator commands. Conventional robots can easily be modelled as rigid links connected by joints, but it remains an open challenge to model and control biologically inspired robots that are often soft or made of several materials, lack sensing capabilities and may change their material properties with use<jats:sup>9\u201312<\/jats:sup>. Here, we introduce a method that uses deep neural networks to map a video stream of a robot to its visuomotor Jacobian field (the sensitivity of all 3D points to the robot\u2019s actuators). Our method enables the control of robots from only a single camera, makes no assumptions about the robots\u2019 materials, actuation or sensing, and is trained without expert intervention by observing the execution of random commands. We demonstrate our method on a diverse set of robot manipulators that vary in actuation, materials, fabrication and cost. Our approach achieves accurate closed-loop control and recovers the causal dynamic structure of each robot. Because it enables robot control using a generic camera as the only sensor, we anticipate that our work will broaden the design space of robotic systems and serve as a starting point for lowering the barrier to robotic automation.<\/jats:p>","DOI":"10.1038\/s41586-025-09170-0","type":"journal-article","created":{"date-parts":[[2025,6,25]],"date-time":"2025-06-25T15:03:00Z","timestamp":1750863780000},"page":"89-95","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Controlling diverse robots by inferring Jacobian fields with deep networks"],"prefix":"10.1038","volume":"643","author":[{"ORCID":"https:\/\/orcid.org\/0009-0000-1857-1520","authenticated-orcid":false,"given":"Sizhe Lester","family":"Li","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6664-9417","authenticated-orcid":false,"given":"Annan","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Boyuan","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hanna","family":"Matusik","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9912-4729","authenticated-orcid":false,"given":"Chao","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5473-3566","authenticated-orcid":false,"given":"Daniela","family":"Rus","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0107-5704","authenticated-orcid":false,"given":"Vincent","family":"Sitzmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,6,25]]},"reference":[{"key":"9170_CR1","doi-asserted-by":"publisher","first-page":"eaar7650","DOI":"10.1126\/scirobotics.aar7650","volume":"3","author":"G-Z Yang","year":"2018","unstructured":"Yang, G.-Z. et al. The grand challenges of Science Robotics. Sci. Robot. 3, eaar7650 (2018).","journal-title":"Sci. Robot."},{"key":"9170_CR2","doi-asserted-by":"publisher","first-page":"287","DOI":"10.1016\/j.tibtech.2013.03.002","volume":"31","author":"S Kim","year":"2013","unstructured":"Kim, S., Laschi, C. & Trimmer, B. Soft robotics: a bioinspired evolution in robotics. Trends Biotechnol. 31, 287\u2013294 (2013).","journal-title":"Trends Biotechnol."},{"key":"9170_CR3","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1080\/11762320802557865","volume":"5","author":"D Trivedi","year":"2008","unstructured":"Trivedi, D., Rahn, C. D., Kier, W. M. & Walker, I. D. Soft robotics: biological inspiration, state of the art, and future research. Appl. Bionics Biomech. 5, 99\u2013117 (2008).","journal-title":"Appl. Bionics Biomech."},{"key":"9170_CR4","doi-asserted-by":"publisher","first-page":"1088","DOI":"10.1126\/science.1145803","volume":"318","author":"R Pfeifer","year":"2007","unstructured":"Pfeifer, R., Lungarella, M. & Iida, F. Self-organization, embodiment, and biologically inspired robotics. Science 318, 1088\u20131093 (2007).","journal-title":"Science"},{"key":"9170_CR5","doi-asserted-by":"publisher","first-page":"1800477","DOI":"10.1002\/admt.201800477","volume":"4","author":"C Majidi","year":"2019","unstructured":"Majidi, C. Soft-matter engineering for soft robotics. Adv. Mater. Technol. 4, 1800477 (2019).","journal-title":"Adv. Mater. Technol."},{"key":"9170_CR6","doi-asserted-by":"publisher","first-page":"522","DOI":"10.1038\/s41586-023-06684-3","volume":"623","author":"TJK Buchner","year":"2023","unstructured":"Buchner, T. J. K. et al. Vision-controlled jetting for composite systems and robots. Nature 623, 522\u2013530 (2023).","journal-title":"Nature"},{"key":"9170_CR7","doi-asserted-by":"publisher","first-page":"1261689","DOI":"10.1126\/science.1261689","volume":"347","author":"MA McEvoy","year":"2015","unstructured":"McEvoy, M. A. & Correll, N. Materials that couple sensing, actuation, computation, and communication. Science 347, 1261689 (2015).","journal-title":"Science"},{"key":"9170_CR8","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1146\/annurev-matsci-080921-102916","volume":"53","author":"AL Evenchik","year":"2023","unstructured":"Evenchik, A. L., Kane, A. Q., Oh, E. & Truby, R. L. Electrically controllable materials for soft, bioinspired machines. Annu. Rev. Mater. Res. 53, 225\u2013251 (2023).","journal-title":"Annu. Rev. Mater. Res."},{"key":"9170_CR9","doi-asserted-by":"publisher","first-page":"467","DOI":"10.1038\/nature14543","volume":"521","author":"D Rus","year":"2015","unstructured":"Rus, D. & Tolley, M. T. Design, fabrication and control of soft robots. Nature 521, 467\u2013475 (2015).","journal-title":"Nature"},{"key":"9170_CR10","doi-asserted-by":"publisher","first-page":"1728","DOI":"10.1109\/TRO.2022.3231360","volume":"39","author":"C Armanini","year":"2023","unstructured":"Armanini, C., Boyer, F., Mathew, A. T., Duriez, C. & Renda, F. Soft robots modeling: a structured overview. IEEE Trans. Robot. 39, 1728\u20131748 (2023).","journal-title":"IEEE Trans. Robot."},{"key":"9170_CR11","doi-asserted-by":"publisher","first-page":"2100165","DOI":"10.1002\/aisy.202100165","volume":"4","author":"J Wang","year":"2022","unstructured":"Wang, J. & Chortos, A. Control strategies for soft robot systems. Adv. Intell. Syst. 4, 2100165 (2022).","journal-title":"Adv. Intell. Syst."},{"key":"9170_CR12","doi-asserted-by":"publisher","first-page":"30","DOI":"10.1109\/MCS.2023.3253419","volume":"43","author":"C Della Santina","year":"2023","unstructured":"Della Santina, C., Duriez, C. & Rus, D. Model-based control of soft robots: a survey of the state of the art and open challenges. IEEE Control Syst. Mag. 43, 30\u201365 (2023).","journal-title":"IEEE Control Syst. Mag."},{"key":"9170_CR13","doi-asserted-by":"publisher","first-page":"eaah3690","DOI":"10.1126\/scirobotics.aah3690","volume":"1","author":"C Laschi","year":"2016","unstructured":"Laschi, C., Mazzolai, B. & Cianchetti, M. Soft robotics: technologies and systems pushing the boundaries of robot abilities. Sci. Robot. 1, eaah3690 (2016).","journal-title":"Sci. Robot."},{"key":"9170_CR14","doi-asserted-by":"crossref","unstructured":"Pratt, G. A. & Williamson, M. M. Series elastic actuators. In Proc. 1995 IEEE\/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots Vol. 1, 399\u2013406 (IEEE, 1995).","DOI":"10.1109\/IROS.1995.525827"},{"key":"9170_CR15","doi-asserted-by":"publisher","first-page":"eaaw1924","DOI":"10.1126\/scirobotics.aaw1924","volume":"4","author":"M Althoff","year":"2019","unstructured":"Althoff, M., Giusti, A., Liu, S. B. & Pereira, A. Effortless creation of safe robots from modules through self-programming and self-verification. Sci. Robot. 4, eaaw1924 (2019).","journal-title":"Sci. Robot."},{"key":"9170_CR16","doi-asserted-by":"publisher","first-page":"13132","DOI":"10.1073\/pnas.1713450114","volume":"114","author":"S Li","year":"2017","unstructured":"Li, S., Vogt, D. M., Rus, D. & Wood, R. J. Fluid-driven origami-inspired artificial muscles. Proc. Natl Acad. Sci. USA 114, 13132\u201313137 (2017).","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"9170_CR17","doi-asserted-by":"publisher","unstructured":"Liu, C., Moncada, A., Matusik, H., Erus, D. I. & Rus, D. A modular bio-inspired robotic hand with high sensitivity. In Proc. 2023 IEEE International Conference on Soft Robotics (RoboSoft) https:\/\/doi.org\/10.1109\/RoboSoft55895.2023.10121946 (IEEE, 2023).","DOI":"10.1109\/RoboSoft55895.2023.10121946"},{"key":"9170_CR18","doi-asserted-by":"crossref","unstructured":"Renda, F., Cacucciolo, V., Dias, J. & Seneviratne, L. Discrete Cosserat approach for soft robot dynamics: a new piece-wise constant strain model with torsion and shears. In Proc. 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) 5495\u20135502 (IEEE, 2016).","DOI":"10.1109\/IROS.2016.7759808"},{"key":"9170_CR19","doi-asserted-by":"publisher","first-page":"1900171","DOI":"10.1002\/aisy.201900171","volume":"2","author":"K Chin","year":"2020","unstructured":"Chin, K., Hellebrekers, T. & Majidi, C. Machine learning for soft robotic sensing and control. Adv. Intell. Syst. 2, 1900171 (2020).","journal-title":"Adv. Intell. Syst."},{"key":"9170_CR20","doi-asserted-by":"publisher","first-page":"e0246102","DOI":"10.1371\/journal.pone.0246102","volume":"16","author":"D Kim","year":"2021","unstructured":"Kim, D. et al. Review of machine learning methods in soft robotics. PLoS One\u00a016, e0246102 (2021).","journal-title":"PLoS One"},{"key":"9170_CR21","doi-asserted-by":"publisher","first-page":"726774","DOI":"10.3389\/frobt.2021.726774","volume":"8","author":"T George Thuruthel","year":"2021","unstructured":"George Thuruthel, T., Falotico, E., Beccai, L. & Iida, F. Machine learning techniques for soft robots. Front. Robot. AI 8, 726774 (2021).","journal-title":"Front. Robot. AI"},{"key":"9170_CR22","doi-asserted-by":"publisher","first-page":"3299","DOI":"10.1109\/LRA.2020.2976320","volume":"5","author":"RL Truby","year":"2020","unstructured":"Truby, R. L., Della Santina, C. & Rus, D. Distributed proprioception of 3d configuration in soft, sensorized robots via deep learning. IEEE Robot. Autom. Lett. 5, 3299\u20133306 (2020).","journal-title":"IEEE Robot. Autom. Lett."},{"key":"9170_CR23","doi-asserted-by":"publisher","first-page":"eaav1488","DOI":"10.1126\/scirobotics.aav1488","volume":"4","author":"TG Thuruthel","year":"2019","unstructured":"Thuruthel, T. G., Shih, B., Laschi, C. & Tolley, M. T. Soft robot perception using embedded soft sensors and recurrent neural networks. Sci. Robot. 4, eaav1488 (2019).","journal-title":"Sci. Robot."},{"key":"9170_CR24","doi-asserted-by":"crossref","unstructured":"Lagneau, R., Krupa, A. & Marchal, M. Active deformation through visual servoing of soft objects. In Proc. 2020 IEEE International Conference on Robotics and Automation 8978\u20138984 (IEEE, 2020).","DOI":"10.1109\/ICRA40945.2020.9197506"},{"key":"9170_CR25","doi-asserted-by":"publisher","first-page":"880","DOI":"10.1109\/TRO.2014.2309194","volume":"30","author":"MC Yip","year":"2014","unstructured":"Yip, M. C. & Camarillo, D. B. Model-less feedback control of continuum manipulators in constrained environments. IEEE Trans. Robot. 30, 880\u2013889 (2014).","journal-title":"IEEE Trans. Robot."},{"key":"9170_CR26","doi-asserted-by":"publisher","first-page":"11298","DOI":"10.1109\/LRA.2022.3194690","volume":"7","author":"A AlBeladi","year":"2022","unstructured":"AlBeladi, A., Ripperger, E., Hutchinson, S. & Krishnan, G. Hybrid eye-in-hand\/eye-to-hand image based visual servoing for soft continuum arms. IEEE Robot. Autom. Lett. 7, 11298\u201311305 (2022).","journal-title":"IEEE Robot. Autom. Lett."},{"key":"9170_CR27","doi-asserted-by":"crossref","unstructured":"Bern, J.M., Schnider, Y., Banzet, P., Kumar, N. & Coros, S. Soft robot control with a learned differentiable model. In Proc. 2020 3rd IEEE International Conference on Soft Robotics (RoboSoft) 417\u2013423 (IEEE, 2020).","DOI":"10.1109\/RoboSoft48309.2020.9116011"},{"key":"9170_CR28","doi-asserted-by":"publisher","first-page":"eabn1944","DOI":"10.1126\/scirobotics.abn1944","volume":"7","author":"B Chen","year":"2022","unstructured":"Chen, B., Kwiatkowski, R., Vondrick, C. & Lipson, H. Fully body visual self-modeling of robot morphologies. Sci. Robot. 7, eabn1944 (2022).","journal-title":"Sci. Robot."},{"key":"9170_CR29","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1007\/s10514-021-10005-w","volume":"46","author":"DP Losey","year":"2022","unstructured":"Losey, D. P. et al. Learning latent actions to control assistive robots. Auton. Robot. 46, 115\u2013147 (2022).","journal-title":"Auton. Robot."},{"key":"9170_CR30","doi-asserted-by":"crossref","unstructured":"Matusik, H., Liu, C. & Rus, D. Directly 3D printed, pneumatically actuated multi-material robotic hand. In Proc. 2024 IEEE International Conference on Robotics and Automation (ICRA) 2986\u20132992 (IEEE, 2024).","DOI":"10.1109\/ICRA57147.2024.10610016"},{"key":"9170_CR31","doi-asserted-by":"publisher","first-page":"632","DOI":"10.1126\/science.aar4586","volume":"360","author":"JI Lipton","year":"2018","unstructured":"Lipton, J. I. et al. Handedness in shearing auxetics creates rigid and compliant structures. Science 360, 632\u2013635 (2018).","journal-title":"Science"},{"key":"9170_CR32","unstructured":"Lapeyre, M. Poppy: Open-source, 3D Printed and Fully-Modular Robotic Platform for Science, Art and Education. PhD thesis, Univ. Bordeaux (2015)."},{"key":"9170_CR33","doi-asserted-by":"crossref","unstructured":"Mildenhall, B. et al. NeRF: representing scenes as neural radiance fields for view synthesis. In Proc. European Conference on Computer Vision (ECCV) (eds Vedaldi, A. et al.) 405\u2013421 (Springer, 2020).","DOI":"10.1007\/978-3-030-58452-8_24"},{"key":"9170_CR34","unstructured":"Sitzmann, V., Martel, J., Bergman, A., Lindell, D. & Wetzstein, G. Implicit neural representations with periodic activation functions. In Proc. 34th Conference on Neural Information Processing Systems (eds Larochelle, H. et al.) 7462\u20137473 (NeurIPS, 2020)."},{"key":"9170_CR35","doi-asserted-by":"crossref","unstructured":"Xie, Y. et al. Neural fields in visual computing and beyond. Comput. Graph. Forum 41, 641\u2013676 (2022).","DOI":"10.1111\/cgf.14505"},{"key":"9170_CR36","unstructured":"Craig, J. J. Introduction to Robotics: Mechanics and Control 2nd edn (Addison-Wesley, 1989)."},{"key":"9170_CR37","doi-asserted-by":"crossref","unstructured":"Siciliano, B., Sciavicco, L., Villani, L. & Oriolo, G. Robotics: Modelling, Planning and Control 1st edn (Springer, 2008).","DOI":"10.1007\/978-1-84628-642-1"},{"key":"9170_CR38","doi-asserted-by":"publisher","unstructured":"Yu, A., Ye, V., Tancik, M. & Kanazawa, A. pixelNeRF: neural radiance fields from one or few images. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) https:\/\/doi.org\/10.1109\/CVPR46437.2021.00455 (IEEE, 2021).","DOI":"10.1109\/CVPR46437.2021.00455"},{"key":"9170_CR39","doi-asserted-by":"publisher","unstructured":"Zhou, X., Koltun, V. & Kr\u00e4henb\u00fchl, P. Tracking objects as points. In Proc. European Conference on Computer Vision (ECCV) https:\/\/doi.org\/10.1007\/978-3-030-58548-8_28 (2020).","DOI":"10.1007\/978-3-030-58548-8_28"},{"key":"9170_CR40","doi-asserted-by":"crossref","unstructured":"Tewari, A. et al. Advances in neural rendering. Comput. Graph. Forum 41, 703\u2013735 (2022).","DOI":"10.1111\/cgf.14507"},{"key":"9170_CR41","unstructured":"Golemo, F., Taiga, A. A., Oudeyer, P.-Y. & Courville, A. Sim-to-real transfer with neural-augmented robot simulation. Proc. Mach. Learn. Res. 87, 817\u2013828 (2018)."},{"key":"9170_CR42","doi-asserted-by":"crossref","unstructured":"Higuera, C., Dong, S., Boots, B. & Mukadam, M. Neural contact fields: tracking extrinsic contact with tactile sensing. In Proc. 2023 IEEE International Conference on Robotics and Automation (ICRA) 12576\u201312582 (IEEE, 2023).","DOI":"10.1109\/ICRA48891.2023.10160526"},{"key":"9170_CR43","unstructured":"Zhong, S., Albini, A., Jones, O. P., Maiolino, P., Posner, I. Touching a NeRF: leveraging neural radiance fields for tactile sensory data generation. Proc. Mach. Learn. Res. 205, 1618\u20131628 (2023)."},{"key":"9170_CR44","doi-asserted-by":"crossref","unstructured":"Dou, Y., Yang, F., Liu, Y., Loquercio, A. & Owens, A. Tactile-augmented radiance fields. In Proc. IEEE\/CVF Conference on Computer Vision and Pattern Recognition 26519\u201326529 (IEEE, 2024).","DOI":"10.1109\/CVPR52733.2024.02505"},{"key":"9170_CR45","doi-asserted-by":"crossref","unstructured":"Pang, T., Terry Suh, H. J., Yang, L. & Tedrake, R. Global planning for contact-rich manipulation via local smoothing of quasi-dynamic contact models. In Proc. IEEE Transactions on Robotics 4691\u20134711 (IEEE, 2023).","DOI":"10.1109\/TRO.2023.3300230"},{"key":"9170_CR46","unstructured":"Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A. & Battaglia, P. Learning mesh-based simulation with graph networks. In Proc. International Conference on Learning Representations (ICLR, 2021)."},{"key":"9170_CR47","unstructured":"Tedrake, R. & the Drake Development Team. Drake: Model-based Design and Verification for Robotics https:\/\/drake.mit.edu (2019)."},{"key":"9170_CR48","doi-asserted-by":"crossref","unstructured":"St\u00f6lzle, M., Rus, D. & Della Santina, C. An experimental study of model-based control for planar handed shearing auxetics robots. In Proc. Experimental Robotics: The 18th International Symposium (eds Ang, M. H. Jr & Khatib, O.) 153\u2013167 (2023).","DOI":"10.1007\/978-3-031-63596-0_14"},{"key":"9170_CR49","doi-asserted-by":"crossref","unstructured":"Olson, E. Apriltag: a robust and flexible visual fiducial system. In Proc. 2011 IEEE International Conference on Robotics and Automation 3400\u20133407 (IEEE, 2011).","DOI":"10.1109\/ICRA.2011.5979561"},{"key":"9170_CR50","doi-asserted-by":"crossref","unstructured":"Teed, Z. & Deng, J. RAFT: recurrent all-pairs field transforms for optical flow. In Proc. European Conference on Computer Vision (ECCV) 402\u2013419 (Springer, 2020).","DOI":"10.1007\/978-3-030-58536-5_24"},{"key":"9170_CR51","unstructured":"Tewari, A. et al. Diffusion with forward models: solving stochastic inverse problems without direct supervision. In Proc. 37th Conference on Neural Information Processing Systems (eds Oh, A. et al.) 12349\u201312362 (NeurIPS, 2023)."},{"key":"9170_CR52","doi-asserted-by":"publisher","unstructured":"Caron, M. et al. Emerging properties in self-supervised vision transformers. In Proc. 2021 IEEE\/CVF International Conference on Computer Vision (ICCV) https:\/\/doi.org\/10.1109\/ICCV48922.2021.00951 (IEEE, 2021).","DOI":"10.1109\/ICCV48922.2021.00951"},{"key":"9170_CR53","doi-asserted-by":"crossref","unstructured":"Peng, S. et al. Convolutional occupancy networks. In Proc. Computer Vision \u2013 ECCV 2020: 16th European Conference (eds Vidaldi, A. et al.) 523\u2013540 (Springer, 2020).","DOI":"10.1007\/978-3-030-58580-8_31"},{"key":"9170_CR54","doi-asserted-by":"crossref","unstructured":"Mescheder, L., Oechsle, M., Niemeyer, M., Nowozin, S. & Geiger, A. Occupancy networks: learning 3d reconstruction in function space. In Proc. IEEE\/CVF Conference on Computer Vision and Pattern Recognition 4455\u20134465 (IEEE, 2019).","DOI":"10.1109\/CVPR.2019.00459"},{"key":"9170_CR55","unstructured":"Feydy, J. et al. Interpolating between optimal transport and MMD using sinkhorn divergences. In Proc. 22nd International Conference on Artificial Intelligence and Statistics 2681\u20132690 (PMLR, 2019)."},{"key":"9170_CR56","unstructured":"Cuturi, M. Sinkhorn distances: lightspeed computation of optimal transportation distances. In Proc. 27th Conference on Neural Information Processing Systems 2292\u20132300 (NeurIPS, 2013)."},{"key":"9170_CR57","doi-asserted-by":"crossref","unstructured":"Bowen, R.S., Tucker, R., Zabih, R. & Snavely, N. Dimensions of motion: monocular prediction through flow subspaces. In Proc. 2022 International Conference on 3D Vision (3DV) 454\u2013464 (IEEE, 2022).","DOI":"10.1109\/3DV57658.2022.00057"},{"key":"9170_CR58","doi-asserted-by":"crossref","unstructured":"Doersch, C. et al. TAPIR: Tracking any point with per-frame initialization and temporal refinement. In Proc. 2023 IEEE\/CVF International Conference on Computer Vision 10061\u201310072 (IEEE, 2023).","DOI":"10.1109\/ICCV51070.2023.00923"},{"key":"9170_CR59","doi-asserted-by":"crossref","unstructured":"Kirillov, A. et al. Segment anything. In Proc. 2023 IEEE\/CVF International Conference on Computer Vision 4015\u20134026 (IEEE, 2023).","DOI":"10.1109\/ICCV51070.2023.00371"},{"key":"9170_CR60","doi-asserted-by":"crossref","unstructured":"Lynch, K. M. Modern Robotics (Cambridge Univ. Press, 2017).","DOI":"10.1017\/9781316661239"},{"key":"9170_CR61","unstructured":"Bonet, J. & Wood, R. D. Nonlinear Continuum Mechanics for Finite Element Analysis (Cambridge Univ. Press, 1997)."},{"key":"9170_CR62","doi-asserted-by":"publisher","unstructured":"Chi, C. et al. Diffusion policy: visuomotor policy learning via action diffusion. Int. J. Rob. Res. https:\/\/doi.org\/10.1177\/02783649241273668 (2023).","DOI":"10.1177\/02783649241273668"},{"key":"9170_CR63","unstructured":"Krzanowski, W. J. Principles of Multivariate Analysis: A User\u2019s Perspective (Oxford Univ. Press, 1988)."},{"key":"9170_CR64","doi-asserted-by":"crossref","unstructured":"Tobin, J. et al. Domain randomization for transferring deep neural networks from simulation to the real world. In Proc. 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) 23\u201330 (IEEE, 2017).","DOI":"10.1109\/IROS.2017.8202133"},{"key":"9170_CR65","doi-asserted-by":"crossref","unstructured":"Peng, X.B., Andrychowicz, M., Zaremba, W. & Abbeel, P. Sim-to-real transfer of robotic control with dynamics randomization. In Proc. 2018 IEEE International Conference on Robotics and Automation (ICRA) 3803\u20133810 (IEEE, 2018).","DOI":"10.1109\/ICRA.2018.8460528"},{"key":"9170_CR66","unstructured":"Florence, P. R., Manuelli, L. & Tedrake, R. Dense object nets: learning dense visual object descriptors by and for robotic manipulation. In Proc. 2nd Conference on Robot Learning 373\u2013385 (PMLR, 2018)."},{"key":"9170_CR67","doi-asserted-by":"crossref","unstructured":"Lin, T.-Y. et al. Microsoft COCO: common objects in context. In Proc. Computer Vision\u2013ECCV 2014 (eds Fleet, D. et al.) 740\u2013755 (Springer, 2014).","DOI":"10.1007\/978-3-319-10602-1_48"},{"key":"9170_CR68","doi-asserted-by":"crossref","unstructured":"Ilievski, F., Mazzeo, A. D., Shepherd, R. F., Chen, X. & Whitesides, G. M. Soft robotics for chemists. Angew. Chem. Int. Ed. Engl. 50, 1890\u20131895 (2011).","DOI":"10.1002\/anie.201006464"},{"key":"9170_CR69","doi-asserted-by":"publisher","first-page":"3838","DOI":"10.1109\/LRA.2020.2977257","volume":"5","author":"M Lambeta","year":"2020","unstructured":"Lambeta, M. et al. DIGIT: a novel design for a low-cost compact high-resolution tactile sensor with application to in-hand manipulation. IEEE Robot. Autom. Lett. 5, 3838\u20133845 (2020).","journal-title":"IEEE Robot. Autom. Lett."},{"key":"9170_CR70","doi-asserted-by":"crossref","unstructured":"Shaw, K., Agarwal, A. & Pathak, D. LEAP Hand: low-cost, efficient, and anthropomorphic hand for robot learning. In Proc. Robotics: Science and Systems (RSS, 2023).","DOI":"10.15607\/RSS.2023.XIX.089"},{"key":"9170_CR71","doi-asserted-by":"crossref","unstructured":"Funabashi, S. et al. Stable in-grasp manipulation with a low-cost robot hand by using 3-axis tactile sensors with a CNN. In Proc. 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) 9166\u20139173 (IEEE, 2020).","DOI":"10.1109\/IROS45743.2020.9341362"},{"key":"9170_CR72","doi-asserted-by":"crossref","unstructured":"Zhu, H., Gupta, A., Rajeswaran, A., Levine, S. & Kumar, V. Dexterous manipulation with deep reinforcement learning: efficient, general, and low-cost. In Proc. 2019 International Conference on Robotics and Automation (ICRA) 3651\u20133657 (IEEE, 2019).","DOI":"10.1109\/ICRA.2019.8794102"},{"key":"9170_CR73","doi-asserted-by":"publisher","first-page":"795","DOI":"10.1109\/LRA.2021.3052422","volume":"6","author":"RL Truby","year":"2021","unstructured":"Truby, R. L., Chin, L. & Rus, D. A recipe for electrically-driven soft robots via 3D printed handed shearing auxetics. IEEE Robot. Autom. Lett. 6, 795\u2013802 (2021).","journal-title":"IEEE Robot. Autom. Lett."},{"key":"9170_CR74","doi-asserted-by":"crossref","unstructured":"Zhang, A., Wang, T.-H., Truby, R.L., Chin, L. & Rus, D. Machine learning best practices for soft robot proprioception. In Proc. 2023 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) 2564\u20132571 (IEEE, 2023).","DOI":"10.1109\/IROS55552.2023.10342379"},{"key":"9170_CR75","doi-asserted-by":"publisher","first-page":"eabq4385","DOI":"10.1126\/sciadv.abq4385","volume":"8","author":"RL Truby","year":"2022","unstructured":"Truby, R. L., Chin, L., Zhang, A. & Rus, D. Fluidic innervation sensorizes structures from a single build material. Sci. Adv. 8, eabq4385 (2022).","journal-title":"Sci. Adv."},{"key":"9170_CR76","doi-asserted-by":"publisher","first-page":"11509","DOI":"10.1109\/LRA.2022.3201604","volume":"7","author":"A Zhang","year":"2022","unstructured":"Zhang, A., Truby, R. L., Chin, L., Li, S. & Rus, D. Vision-based sensing for electrically-driven soft actuators. IEEE Robot. Autom. Lett. 7, 11509\u201311516 (2022).","journal-title":"IEEE Robot. Autom. Lett."},{"key":"9170_CR77","unstructured":"Ergo Jr: a low cost robot for education. Poppy Project https:\/\/www.poppy-project.org\/en\/robots\/poppy-ergo-jr\/ (2016)."},{"key":"9170_CR78","unstructured":"Assembly guide for the Ergo Jr. Poppy Project https:\/\/docs.poppy-project.org\/en\/assembly-guides\/ergo-jr\/ (2022)."},{"key":"9170_CR79","doi-asserted-by":"crossref","unstructured":"Salvato, E., Fenu, G., Medvet, E. & Pellegrino, F. A. Characterization of modeling errors affecting performances of a robotics deep reinforcement learning controller in a sim-to-real transfer. In Proc. 2021 44th International Convention on Information, Communication and Electronic Technology (MIPRO) 1154\u20131159 (IEEE, 2021).","DOI":"10.23919\/MIPRO52101.2021.9596864"},{"key":"9170_CR80","unstructured":"Chevalier-Boisvert, M., Alain, G., Golemo, F. & Nowrouzezahrai, D. Robo-PlaNet: learning to poke in a day. Preprint at https:\/\/arxiv.org\/abs\/1911.03594 (2019)."},{"key":"9170_CR81","unstructured":"Xu, Z., He, Z., Wu, J. & Song, S. Learning 3d dynamic scene representations for robot manipulation. In Proc. 4th Conference on Robot Learning (CoRL, 2020)."}],"container-title":["Nature"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41586-025-09170-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41586-025-09170-0","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41586-025-09170-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,2]],"date-time":"2025-07-02T13:49:55Z","timestamp":1751464195000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41586-025-09170-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,25]]},"references-count":81,"journal-issue":{"issue":"8070","published-print":{"date-parts":[[2025,7,3]]}},"alternative-id":["9170"],"URL":"https:\/\/doi.org\/10.1038\/s41586-025-09170-0","relation":{},"ISSN":["0028-0836","1476-4687"],"issn-type":[{"value":"0028-0836","type":"print"},{"value":"1476-4687","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,6,25]]},"assertion":[{"value":"10 July 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"16 May 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 June 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}