{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,12]],"date-time":"2026-10-12T15:53:25Z","timestamp":1791820405810,"version":"4.3.4"},"reference-count":54,"publisher":"Springer Science and Business Media LLC","issue":"8","license":[{"start":{"date-parts":[[2023,7,4]],"date-time":"2023-07-04T00:00:00Z","timestamp":1688428800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,7,4]],"date-time":"2023-07-04T00:00:00Z","timestamp":1688428800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Northeastern University USA"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Auton Robot"],"published-print":{"date-parts":[[2023,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Real-world grasp detection is challenging due to the stochasticity in grasp dynamics and the noise in hardware. Ideally, the system would adapt to the real world by training directly on physical systems. However, this is generally difficult due to the large amount of training data required by most grasp learning models. In this paper, we note that the planar grasp function is\n                    <jats:inline-formula>\n                      <jats:alternatives>\n                        <jats:tex-math>$$\\textrm{SE}(2)$$<\/jats:tex-math>\n                        <mml:math xmlns:mml=\"http:\/\/www.w3.org\/1998\/Math\/MathML\">\n                          <mml:mrow>\n                            <mml:mtext>SE<\/mml:mtext>\n                            <mml:mo>(<\/mml:mo>\n                            <mml:mn>2<\/mml:mn>\n                            <mml:mo>)<\/mml:mo>\n                          <\/mml:mrow>\n                        <\/mml:math>\n                      <\/jats:alternatives>\n                    <\/jats:inline-formula>\n                    -equivariant and demonstrate that this structure can be used to constrain the neural network used during learning. This creates an inductive bias that can significantly improve the sample efficiency of grasp learning and enable end-to-end training from scratch on a physical robot with as few as 600 grasp attempts. We call this method Symmetric Grasp learning (SymGrasp) and show that it can learn to grasp \u201cfrom scratch\u201d in less that 1.5 h of physical robot time. This paper represents an expanded and revised version of the conference paper Zhu et\u00a0al. (2022).\n                  <\/jats:p>","DOI":"10.1007\/s10514-023-10112-w","type":"journal-article","created":{"date-parts":[[2023,7,4]],"date-time":"2023-07-04T18:01:45Z","timestamp":1688493705000},"page":"1175-1193","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["On robot grasp learning using equivariant models"],"prefix":"10.1007","volume":"47","author":[{"given":"Xupeng","family":"Zhu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dian","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Guanang","family":"Su","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ondrej","family":"Biza","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robin","family":"Walters","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Robert","family":"Platt","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2023,7,4]]},"reference":[{"key":"10112_CR1","doi-asserted-by":"crossref","unstructured":"Berscheid, L., Friedrich, C., & Kr\u00f6ger, T. (2021). Robot learning of 6 dof grasping using model-based adaptive primitives. arXiv preprint arXiv:2103.12810","DOI":"10.1109\/ICRA48506.2021.9560901"},{"key":"10112_CR2","unstructured":"Breyer, M., Chung, J.\u00a0J., Ott, L., Siegwart, R., & Nieto, J. (2021). Volumetric grasping network: Real-time 6 dof grasp detection in clutter. arXiv preprint arXiv:2101.01132"},{"issue":"3","key":"10112_CR3","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1177\/0278364917700714","volume":"36","author":"B Calli","year":"2017","unstructured":"Calli, B., Singh, A., Bruce, J., Walsman, A., Konolige, K., Srinivasa, S., Abbeel, P., & Dollar, A. M. (2017). Yale-cmu-berkeley dataset for robotic manipulation research. The International Journal of Robotics Research, 36(3), 261\u2013268. https:\/\/doi.org\/10.1177\/0278364917700714","journal-title":"The International Journal of Robotics Research"},{"key":"10112_CR4","unstructured":"Cohen, T., Geiger, M., & Weiler, M. (2018). A general theory of equivariant cnns on homogeneous spaces. arXiv preprint arXiv:1811.02017"},{"key":"10112_CR5","unstructured":"Cohen, T., & Welling, M. (2016a). Group equivariant convolutional networks. In International conference on machine learning (pp. 2990\u20132999). PMLR."},{"key":"10112_CR6","unstructured":"Cohen, T.\u00a0S, & Welling, M. (2016b). Steerable cnns. arXiv preprint arXiv:1612.08498"},{"key":"10112_CR7","unstructured":"Coumans, E., & Bai, Y. (2016). Pybullet, a python module for physics simulation for games, robotics and machine learning. GitHub repository."},{"key":"10112_CR8","unstructured":"Danielczuk, M., Balakrishna, A., Brown, D.\u00a0S, Devgon, S., & Goldberg, K. (2020). Exploratory grasping: Asymptotically optimal algorithms for grasping challenging polyhedral objects. arXiv preprint arXiv:2011.05632"},{"key":"10112_CR9","doi-asserted-by":"crossref","unstructured":"Depierre, A., Dellandr\u00e9a, E., & Chen, L. (2018). Jacquard: A large scale dataset for robotic grasp detection. CoRR, arXiv:1803.11469","DOI":"10.1109\/IROS.2018.8593950"},{"key":"10112_CR10","doi-asserted-by":"crossref","unstructured":"Fang, H.-S., Wang, C., Gou, M., & Lu, C. (2020). Graspnet-1billion: A large-scale benchmark for general object grasping. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 11444\u201311453).","DOI":"10.1109\/CVPR42600.2020.01146"},{"key":"10112_CR11","doi-asserted-by":"crossref","unstructured":"Gao, J., Sun, C., Zhao, H., Shen, Y., Anguelov, D., Li, C., & Schmid, C. (2020). Vectornet: Encoding hd maps and agent dynamics from vectorized representation. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 11525\u201311533).","DOI":"10.1109\/CVPR42600.2020.01154"},{"key":"10112_CR12","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., & Sun, J. (2015). Deep residual learning for image recognition.","DOI":"10.1109\/CVPR.2016.90"},{"key":"10112_CR13","doi-asserted-by":"crossref","unstructured":"Huang, H., Wang, D., Zhu, X., Walters, R., & Platt, R. (2022). Edge grasp network: A graph-based se (3)-invariant approach to grasp detection. arXiv preprint arXiv:2211.00191","DOI":"10.1109\/ICRA48891.2023.10160728"},{"key":"10112_CR14","unstructured":"Ichnowski, J., Avigal, Y., Kerr, J., & Goldberg, K. (2021). Dex-nerf: Using a neural radiance field to grasp transparent objects. arXiv preprint arXiv:2110.14217"},{"key":"10112_CR15","doi-asserted-by":"crossref","unstructured":"James, S., Wohlhart, P., Kalakrishnan, M., Kalashnikov, D., Irpan, A., Ibarz, J., Levine, S., Hadsell, R., & Bousmalis, K. (2019). Sim-to-real via sim-to-sim: Data-efficient robotic grasping via randomized-to-canonical adaptation networks. In Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition (pp. 12627\u201312637).","DOI":"10.1109\/CVPR.2019.01291"},{"key":"10112_CR16","doi-asserted-by":"crossref","unstructured":"Jiang, Z., Zhu, Y., Svetlik, M., Fang, K., & Zhu, Y. (2021). Synergies between affordance and geometry: 6-dof grasp detection via implicit representations. arXiv preprint arXiv:2104.01542","DOI":"10.15607\/RSS.2021.XVII.024"},{"key":"10112_CR17","unstructured":"Kalashnikov, D., Irpan, A., Pastor, P., Ibarz, J., Herzog, A., Jang, E., Quillen, D., Holly, E., Kalakrishnan, M., Vanhoucke, V., & Levine, S. (2018). Qt-opt: Scalable deep reinforcement learning for vision-based robotic manipulation. CoRR, arXiv:1806.10293"},{"key":"10112_CR18","unstructured":"Kerr, J., Fu, L., Huang, H., Avigal, Y., Tancik, M., Ichnowski, J., Kanazawa, A., & Goldberg, K. Evo-nerf: Evolving nerf for sequential robot grasping of transparent objects. In 6th annual conference on robot learning."},{"key":"10112_CR19","unstructured":"Kingma, D. P., & Ba, J. (2015). Adam: A method for stochastic optimization. CoRR, arXiv:1412.6980"},{"key":"10112_CR20","first-page":"6417","volume":"34","author":"M Kofinas","year":"2021","unstructured":"Kofinas, M., Nagaraja, N., & Gavves, E. (2021). Roto-translated local coordinate frames for interacting dynamical systems. Advances in Neural Information Processing Systems, 34, 6417\u20136429.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"10112_CR21","unstructured":"Kostrikov, I., Yarats, D., & Fergus, R. (2020). Image augmentation is all you need: Regularizing deep reinforcement learning from pixels. CoRR, arXiv:2004.13649"},{"key":"10112_CR22","doi-asserted-by":"crossref","unstructured":"Kumra, S., Joshi, S., & Sahin, F. (2019). Antipodal robotic grasping using generative residual convolutional neural network. CoRR, arXiv:1909.04810","DOI":"10.1109\/IROS45743.2020.9340777"},{"key":"10112_CR23","unstructured":"Laskin, M., Lee, K., Stooke, A., Pinto, L., Abbeel, P., & Srinivas, A. (2020a). Reinforcement learning with augmented data. CoRR, arXiv:2004.14990"},{"key":"10112_CR24","unstructured":"Laskin, M., Srinivas, A., & Abbeel, P. (2020b). Curl: Contrastive unsupervised representations for reinforcement learning. In International conference on machine learning (pp. 5639\u20135650). PMLR."},{"issue":"4\u20135","key":"10112_CR25","doi-asserted-by":"publisher","first-page":"421","DOI":"10.1177\/0278364917710318","volume":"37","author":"S Levine","year":"2018","unstructured":"Levine, S., Pastor, P., Krizhevsky, A., Ibarz, J., & Quillen, D. (2018). Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection. The International Journal of Robotics Research, 37(4\u20135), 421\u2013436.","journal-title":"The International Journal of Robotics Research"},{"key":"10112_CR26","doi-asserted-by":"crossref","unstructured":"Mahler, J., Liang, J., Niyaz, S., Laskey, M., Doan, R., Liu, X., Ojea, J.\u00a0A., & Goldberg, K. (2017). Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics. arXiv preprint arXiv:1703.09312","DOI":"10.15607\/RSS.2017.XIII.058"},{"issue":"1","key":"10112_CR27","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1145\/3503250","volume":"65","author":"B Mildenhall","year":"2021","unstructured":"Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2021). Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1), 99\u2013106.","journal-title":"Communications of the ACM"},{"key":"10112_CR28","unstructured":"Mondal, A.\u00a0K., Nair, P., & Siddiqi, K. (2020). Group equivariant deep reinforcement learning. arXiv preprint arXiv:2007.03437"},{"key":"10112_CR29","doi-asserted-by":"crossref","unstructured":"Morrison, D., Corke, P., & Leitner, J. (2018). Closing the loop for robotic grasping: A real-time, generative grasp synthesis approach. arXiv preprint arXiv:1804.05172","DOI":"10.15607\/RSS.2018.XIV.021"},{"key":"10112_CR30","doi-asserted-by":"crossref","unstructured":"Mousavian, A., Eppner, C., & Fox, D. (2019). 6-dof graspnet: Variational grasp generation for object manipulation. In Proceedings of the IEEE\/CVF international conference on computer vision (ICCV), October.","DOI":"10.1109\/ICCV.2019.00299"},{"key":"10112_CR31","first-page":"8024","volume-title":"Advances in neural information processing systems 32","author":"A Paszke","year":"2019","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., Desmaison, A., Kopf, A., Yang, E., DeVito, Z., Raison, M., Tejani, A., Chilamkurthy, S., Steiner, B., Fang, L., & Chintala, S. (2019). Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d\u2019Alch\u00e9-Buc, E. Fox, & R. Garnett (Eds.), Advances in neural information processing systems 32 (pp. 8024\u20138035). Curran Associates Inc."},{"key":"10112_CR32","doi-asserted-by":"crossref","unstructured":"Pinto, L., & Gupta, A. (2015). Supersizing self-supervision: Learning to grasp from 50k tries and 700 robot hours. CoRR, arXiv:1509.06825","DOI":"10.1109\/ICRA.2016.7487517"},{"key":"10112_CR33","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1146\/annurev-control-062122-025215","volume":"6","author":"R Platt","year":"2023","unstructured":"Platt, R. (2023). Grasp learning: Models, methods, and performance. Annual Review of Control, Robotics, and Autonomous Systems, 6, 363\u2013389.","journal-title":"Annual Review of Control, Robotics, and Autonomous Systems"},{"key":"10112_CR34","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., & Brox, T. (2015). U-net: Convolutional networks for biomedical image segmentation. CoRR, arXiv:1505.04597","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"10112_CR35","doi-asserted-by":"crossref","unstructured":"Sajjan, S., Moore, M., Pan, M., Nagaraja, G., Lee, J., Zeng, A., & Song, S. (2020). Clear grasp: 3d shape estimation of transparent objects for manipulation. In 2020 IEEE international conference on robotics and automation (ICRA) (pp. 3634\u20133642). IEEE.","DOI":"10.1109\/ICRA40945.2020.9197518"},{"issue":"2","key":"10112_CR36","doi-asserted-by":"publisher","first-page":"1357","DOI":"10.1109\/LRA.2019.2895878","volume":"4","author":"V Satish","year":"2019","unstructured":"Satish, V., Mahler, J., & Goldberg, K. (2019). On-policy dataset synthesis for learning robot grasping policies using fully convolutional deep networks. IEEE Robotics and Automation Letters, 4(2), 1357\u20131364. https:\/\/doi.org\/10.1109\/LRA.2019.2895878","journal-title":"IEEE Robotics and Automation Letters"},{"key":"10112_CR37","unstructured":"Sharma, S., Suresh, A., Ramesh, R., & Ravindran, B. (2017). Learning to factor policies and action-value functions: Factored action space representations for deep reinforcement learning. arXiv preprint arXiv:1705.07269"},{"key":"10112_CR38","doi-asserted-by":"publisher","first-page":"4978","DOI":"10.1109\/LRA.2020.3004787","volume":"5","author":"S Song","year":"2020","unstructured":"Song, S., Zeng, A., Lee, J., & Funkhouser, T. (2020). Grasping in the wild: Learning 6dof closed-loop grasping from low-cost demonstrations. IEEE Robotics and Automation Letters, 5, 4978\u20134985.","journal-title":"IEEE Robotics and Automation Letters"},{"key":"10112_CR39","doi-asserted-by":"crossref","unstructured":"Sundermeyer, M., Mousavian, A., Triebel, R., & Fox, D. (2021). Contact-graspnet: Efficient 6-dof grasp generation in cluttered scenes. arXiv preprint arXiv:2103.14127","DOI":"10.1109\/ICRA48506.2021.9561877"},{"key":"10112_CR40","doi-asserted-by":"crossref","unstructured":"ten Pas, A., Gualtieri, M., Saenko, K., & Platt, R. Jr. (2017a). Grasp pose detection in point clouds. CoRR, arXiv:1706.09911","DOI":"10.1177\/0278364917735594"},{"issue":"13\u201314","key":"10112_CR41","doi-asserted-by":"crossref","first-page":"1455","DOI":"10.1177\/0278364917735594","volume":"36","author":"A ten Pas","year":"2017","unstructured":"ten Pas, A., Gualtieri, M., Saenko, K., & Platt, R. (2017b). Grasp pose detection in point clouds. The International Journal of Robotics Research, 36(13\u201314), 1455\u20131473.","journal-title":"The International Journal of Robotics Research"},{"key":"10112_CR42","first-page":"4199","volume":"33","author":"E van der Pol","year":"2020","unstructured":"van der Pol, E., Worrall, D., van Hoof, H., Oliehoek, F., & Welling, M. (2020). Mdp homomorphic networks: Group symmetries in reinforcement learning. Advances in Neural Information Processing Systems, 33, 4199\u20134210.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"10112_CR43","unstructured":"van\u00a0den Oord, A., Li, Y., & Vinyals, O. (2018). Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748"},{"key":"10112_CR44","unstructured":"Walters, R., Li, J., & Yu, R. (2020). Trajectory prediction using equivariant continuous convolution. arXiv preprint arXiv:2010.11344"},{"key":"10112_CR45","unstructured":"Wang, D., Kohler, C., & Platt, R. Jr. (2020a). Policy learning in SE(3) action spaces. CoRR, arXiv:2010.02798"},{"key":"10112_CR46","unstructured":"Wang, D., Walters, R., & Platt, R. (2022). SO(2)-equivariant reinforcement learning. In Int\u2019l conference on representation learning. https:\/\/openreview.net\/forum?id=7F9cOhdvfk"},{"key":"10112_CR47","unstructured":"Wang, R., Walters, R., Yu, R. (2020b). Incorporating symmetry into deep dynamics models for improved generalization. arXiv preprint arXiv:2002.03061"},{"key":"10112_CR48","unstructured":"Wang, D., Walters, R., Zhu, X., & Platt, R. (2021). Equivariant $$q$$ learning in spatial action spaces. In 5th annual conference on robot learning. https:\/\/openreview.net\/forum?id=IScz42A3iCI"},{"key":"10112_CR49","unstructured":"Weiler, M., & Cesa, G. (2019). General e(2)-equivariant steerable cnns. CoRR, arXiv:1911.08251"},{"issue":"3","key":"10112_CR50","doi-asserted-by":"publisher","first-page":"3791","DOI":"10.1109\/LRA.2020.2974686","volume":"5","author":"T Weng","year":"2020","unstructured":"Weng, T., Pallankize, A., Tang, Y., Kroemer, O., & Held, D. (2020). Multi-modal transfer learning for grasping transparent and specular objects. IEEE Robotics and Automation Letters, 5(3), 3791\u20133798.","journal-title":"IEEE Robotics and Automation Letters"},{"key":"10112_CR51","doi-asserted-by":"crossref","unstructured":"Zeng, A., Song, S., Welker, S., Lee, J., Rodriguez, A., & Funkhouser, T. A. (2018). Learning synergies between pushing and grasping with self-supervised deep reinforcement learning. CoRR, arXiv:1803.09956","DOI":"10.1109\/IROS.2018.8593986"},{"key":"10112_CR52","unstructured":"Zhan, A., Zhao, P., Pinto, L., Abbeel, P., & Laskin, M. (2020). A framework for efficient robotic manipulation. CoRR, arXiv:2012.07975"},{"key":"10112_CR53","doi-asserted-by":"crossref","unstructured":"Zhou, X., Lan, X., Zhang, H., Tian, Z., Zhang, Y., & Zheng, N. (2018). Fully convolutional grasp detection network with oriented anchor box. In 2018 IEEE\/RSJ international conference on intelligent robots and systems (IROS) (pp. 7223\u20137230). IEEE.","DOI":"10.1109\/IROS.2018.8594116"},{"key":"10112_CR54","unstructured":"Zhu, X, Wang., Dian, B., Ondrej, S., Guanang, W., Robin, P., Robert. (2022). Proceedings of Robotics: Science and Systems (RSS)."}],"container-title":["Autonomous Robots"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10514-023-10112-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10514-023-10112-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10514-023-10112-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,28]],"date-time":"2023-11-28T13:06:00Z","timestamp":1701176760000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10514-023-10112-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,7,4]]},"references-count":54,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2023,12]]}},"alternative-id":["10112"],"URL":"https:\/\/doi.org\/10.1007\/s10514-023-10112-w","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-2561658\/v1","asserted-by":"object"}]},"ISSN":["0929-5593","1573-7527"],"issn-type":[{"value":"0929-5593","type":"print"},{"value":"1573-7527","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,7,4]]},"assertion":[{"value":"7 February 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 May 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 July 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}},{"value":"The authors declare that they have no relevant financial or non-financial interests to disclose","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}