{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T23:19:35Z","timestamp":1771024775325,"version":"3.50.1"},"reference-count":32,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T00:00:00Z","timestamp":1616025600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61973065, 52075531"],"award-info":[{"award-number":["61973065, 52075531"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012226","name":"Fundamental Research Funds for the Central Universities","doi-asserted-by":"publisher","award":["N182612002"],"award-info":[{"award-number":["N182612002"]}],"id":[{"id":"10.13039\/501100012226","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Robot grasping is an important direction in intelligent robots. However, how to help robots grasp specific objects in multi-object scenes is still a challenging problem. In recent years, due to the powerful feature extraction capabilities of convolutional neural networks (CNN), various algorithms based on convolutional neural networks have been proposed to solve the problem of grasp detection. Different from anchor-based grasp detection algorithms, in this paper, we propose a keypoint-based scheme to solve this problem. We model an object or a grasp as a single point\u2014the center point of its bounding box. The detector uses keypoint estimation to find the center point and regress to all other object attributes such as size, direction, etc. Experimental results demonstrate that the accuracy of this method is 74.3% in the multi-object grasp dataset VMRD, and the performance on the single-object scene Cornell dataset is competitive with the current state-of-the-art grasp detection algorithm. Robot experiments demonstrate that this method can help robots grasp the target in single-object and multi-object scenes with overall success rates of 94% and 87%, respectively.<\/jats:p>","DOI":"10.3390\/s21062132","type":"journal-article","created":{"date-parts":[[2021,3,18]],"date-time":"2021-03-18T22:19:36Z","timestamp":1616105976000},"page":"2132","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Keypoint-Based Robotic Grasp Detection Scheme in Multi-Object Scenes"],"prefix":"10.3390","volume":"21","author":[{"given":"Tong","family":"Li","sequence":"first","affiliation":[{"name":"College of Information Science and Engineering, Northeastern University, Shenyang 110819, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8296-8039","authenticated-orcid":false,"given":"Fei","family":"Wang","sequence":"additional","affiliation":[{"name":"Faculty of Robot Science and Engineering, Northeastern University, Shenyang 110169, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Changlei","family":"Ru","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Northeastern University, Shenyang 110819, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yong","family":"Jiang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinghong","family":"Li","sequence":"additional","affiliation":[{"name":"College of Information Science and Engineering, Northeastern University, Shenyang 110819, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,18]]},"reference":[{"key":"ref_1","unstructured":"Jiang, Y., Moseson, S., and Saxena, A. (2011, January 9\u201313). Efficient grasping from RGBD images: Learning using a new rectangle representation. Proceedings of the 2011 IEEE International Conference on Robotics and Automation, Shanghai, China."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"705","DOI":"10.1177\/0278364914549607","article-title":"Deep Learning for Detecting Robotic Grasps","volume":"34","author":"Lenz","year":"2013","journal-title":"Int. J. Robot. Res."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Redmon, J., and Angelova, A. (2015, January 26\u201330). Real-time grasp detection using convolutional neural networks. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139361"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kumra, S., and Kanan, C. (2017, January 24\u201328). Robotic grasp detection using deep convolutional neural networks. Proceedings of the 2017 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, Canada.","DOI":"10.1109\/IROS.2017.8202237"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Guo, D., Sun, F., Liu, H., Kong, T., Fang, B., and Xi, N. (June, January 29). A hybrid deep architecture for robotic grasp detection. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore.","DOI":"10.1109\/ICRA.2017.7989191"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Zhou, X., Lan, X., Zhang, H., Tian, Z., Zhang, Y., and Zheng, N. (2018, January 1\u20135). Fully Convolutional Grasp Detection Network with Oriented Anchor Box. Proceedings of the 2018 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Madrid, Spain.","DOI":"10.1109\/IROS.2018.8594116"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1177\/0278364917710318","article-title":"Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection","volume":"37","author":"Levine","year":"2016","journal-title":"Int. J. Robot. Res."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Gualtieri, M., Pas, A.T., Saenko, K., and Platt, R. (2016, January 9\u201314). High precision grasp pose detection in dense clutter. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759114"},{"key":"ref_9","unstructured":"Jeffrey, M., and Ken, G. (2017, January 13\u201315). Learning deep policies for robot bin picking by simulating robust grasping sequences. Proceedings of the 1st Annual Conference on Robot Learning, PMLR, Mountain View, CA, USA."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Mahler, J., Matl, M., Liu, X., Li, A., Gealy, D., and Goldberg, K. (2018, January 21\u201325). Dex-net 3.0: Computing robust robot suction grasp targets in point clouds using a new analytic model and deep learning. Proceedings of the IEEE International Conference on Robotics and Automation, Exhibition Center, Australia.","DOI":"10.1109\/ICRA.2018.8460887"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3355","DOI":"10.1109\/LRA.2018.2852777","article-title":"Real-World Multiobject, Multigrasp Detection","volume":"3","author":"Chu","year":"2018","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zeng, A., Song, S., Yu, K.T., Donlon, E., Hogan, F.R., and Bauza, M. (2018, January 21\u201325). Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching. Proceedings of the 2018 IEEE International Conference on Robotics and Automation (ICRA), Brisbane, Australia.","DOI":"10.1109\/ICRA.2018.8461044"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhang, H., Lan, X., Bai, S., Zhou, X., Tian, Z., and Zheng, N. (2019, January 4\u20138). ROI-based Robotic Grasp Detection for Object Overlapping Scenes. Proceedings of the 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Macao, China.","DOI":"10.1109\/IROS40897.2019.8967869"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Zhang, H., Lan, X., Zhou, X., Tian, Z., Zhang, Y., and Zheng, N. (2018, January 6\u20139). Visual manipulation relationship network for autonomous robotics. Proceedings of the 2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids), Beijing, China.","DOI":"10.1109\/HUMANOIDS.2018.8625071"},{"key":"ref_15","first-page":"110","article-title":"Graspit! a versatile simulator for robotic grasping","volume":"11","author":"AMiller","year":"2004","journal-title":"Robot. Autom. Mag. IEEE"},{"key":"ref_16","unstructured":"Pelossof, R., Miller, A., Allen, P., and Jebara, T. (May, January 26). An SVM learning approach to robotic grasping. Proceedings of the IEEE International Conference on Robotics and Automation, New Orleans, LA, USA."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1109\/TRO.2013.2289018","article-title":"Data-driven grasp synthesis\u2014A survey","volume":"30","author":"Bohg","year":"2014","journal-title":"IEEE Trans. Robot."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1177\/0278364907087172","article-title":"Robotic grasping of novel objects using vision","volume":"27","author":"Saxena","year":"2008","journal-title":"Int. J. Robot. Res."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Le, Q.V., Kamm, D., Kara, A.F., and Ng, A.Y. (2010, January 3\u20138). Learning to grasp objects with multiple contact points. Proceedings of the Robotics and Automation(ICRA) 2010 IEEE International Conference, Anchorage, Alaska.","DOI":"10.1109\/ROBOT.2010.5509508"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Depierre, A., Dellandr\u00e9a, E., and Chen, L. (2020). Optimizing correlated graspability score and grasp regression for better grasp prediction. arXiv.","DOI":"10.1109\/ICRA48506.2021.9561198"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Li, B., Cao, H., Qu, Z., Hu, Y., Wang, Z., and Liang, Z. (2020). Event-based Robotic Grasping Detection with Neuromorphic Vision Sensor and Event-Stream Dataset. arXiv.","DOI":"10.3389\/fnbot.2020.00051"},{"key":"ref_23","unstructured":"Guo, D., Kong, T., Sun, F., and Liu, H. (2016, January 16\u201321). Object discovery and grasp detection with a shared convolutional neural network. Proceedings of the 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, Sweden."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Vohra, M., Prakash, R., and Behera, L. (2020). Real-time Grasp Pose Estimation for Novel Objects in Densely Cluttered Environment. arXiv.","DOI":"10.1109\/RO-MAN46459.2019.8956438"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Law, H., and Deng, J. (2018, January 8\u201314). Cornernet: Detecting objects as paired keypoints. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01264-9_45"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Newell, A., Yang, K., and Deng, J. (2016). Stacked hourglass networks for human pose estimation. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46484-8_29"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Yu, F., Wang, D., Shelhamer, E., and Darrell, T. (2018, January 18\u201322). Deep Layer Aggregation. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00255"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Cao, Z., Simon, T., Wei, S.E., and Sheikh, Y. (2017, January 21\u201326). Real-time multi-person 2d pose estimation using part affinity fields. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.143"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Papandreou, G., Zhu, T., Kanazawa, N., Toshev, A., Tompson, J., Bregler, C., and Murphy, K. (2017, January 21\u201326). Towards accurate multi-person pose estimation in the wild. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.395"},{"key":"ref_30","unstructured":"Yosinski, J., Clune, J., Bengio, Y., and Lipson, H. (2014). How transferable are features in deep neural networks?. arXiv."},{"key":"ref_31","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015). Faster rcnn: Towards real-time object detection with region proposal networks. arXiv."},{"key":"ref_32","first-page":"2999","article-title":"Focal Loss for Dense Object Detection","volume":"99","author":"Lin","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2132\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:37:34Z","timestamp":1760161054000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/6\/2132"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,18]]},"references-count":32,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["s21062132"],"URL":"https:\/\/doi.org\/10.3390\/s21062132","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,18]]}}}