{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T09:58:48Z","timestamp":1777715928536,"version":"3.51.4"},"reference-count":94,"publisher":"SAGE Publications","issue":"4","license":[{"start":{"date-parts":[[2022,2,28]],"date-time":"2022-02-28T00:00:00Z","timestamp":1646006400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2022,4]]},"abstract":"<jats:p>\n                    Contemporary grasp detection approaches employ deep learning to achieve robustness to sensor and object model uncertainty. The two dominant approaches design either grasp-quality scoring or anchor-based grasp recognition networks. This paper presents a different approach to grasp detection by treating it as keypoint detection in image-space. The deep network detects each grasp candidate as a pair of keypoints, convertible to the grasp representation g = { x, y, w, \u03b8}\n                    <jats:sup>T<\/jats:sup>\n                    , rather than a triplet or quartet of corner points. Decreasing the detection difficulty by grouping keypoints into pairs boosts performance. To promote capturing dependencies between keypoints, a non-local module is incorporated into the network design. A final filtering strategy based on discrete and continuous orientation prediction removes false correspondences and further improves grasp detection performance. GKNet, the approach presented here, achieves a good balance between accuracy and speed on the Cornell and the abridged Jacquard datasets (96.9% and 98.39% at 41.67 and 23.26 fps). Follow-up experiments on a manipulator evaluate GKNet using four types of grasping experiments reflecting different nuisance sources: static grasping, dynamic grasping, grasping at varied camera angles, and bin picking. GKNet outperforms reference baselines in static and dynamic grasping experiments while showing robustness to varied camera viewpoints and moderate clutter. The results confirm the hypothesis that grasp keypoints are an effective output representation for deep grasp networks that provide robustness to expected nuisance factors.\n                  <\/jats:p>","DOI":"10.1177\/02783649211069569","type":"journal-article","created":{"date-parts":[[2022,3,1]],"date-time":"2022-03-01T01:59:46Z","timestamp":1646099986000},"page":"361-389","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":46,"title":["GKNet: Grasp keypoint network for grasp candidates detection"],"prefix":"10.1177","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3974-1629","authenticated-orcid":false,"given":"Ruinian","family":"Xu","sequence":"first","affiliation":[{"name":"Intelligent Vision and Automation Laboratory (IVALab), School of Electrical and Computer Engineering, Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, GA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3290-8094","authenticated-orcid":false,"given":"Fu-Jen","family":"Chu","sequence":"additional","affiliation":[{"name":"Intelligent Vision and Automation Laboratory (IVALab), School of Electrical and Computer Engineering, Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, GA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6888-7002","authenticated-orcid":false,"given":"Patricio A","family":"Vela","sequence":"additional","affiliation":[{"name":"Intelligent Vision and Automation Laboratory (IVALab), School of Electrical and Computer Engineering, Institute for Robotics and Intelligent Machines, Georgia Institute of Technology, GA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2022,2,28]]},"reference":[{"key":"bibr1-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2016.2638453"},{"key":"bibr2-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33018085"},{"key":"bibr3-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2000.844081"},{"key":"bibr4-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2009.10.003"},{"key":"bibr5-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2018.2852779"},{"key":"bibr6-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICAR.2015.7251504"},{"key":"bibr7-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2015.2448951"},{"key":"bibr8-02783649211069569","unstructured":"Chang AX, Funkhouser T, Guibas L, et al. (2015) ShapeNet: an information-rich 3d model repository. arXiv preprint arXiv:1512.03012."},{"key":"bibr9-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2018.2852777"},{"key":"bibr10-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2016.2645124"},{"key":"bibr11-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(01)00158-0"},{"key":"bibr12-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"bibr13-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2018.8593950"},{"key":"bibr14-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6907124"},{"key":"bibr15-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00667"},{"key":"bibr16-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2007.364205"},{"key":"bibr17-02783649211069569","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2016.XII.036"},{"key":"bibr18-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2012.6386137"},{"key":"bibr19-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2013.6630636"},{"issue":"7","key":"bibr20-02783649211069569","first-page":"560","volume":"34","author":"Fujita M","year":"2020","journal-title":"Advanced Robotics"},{"key":"bibr21-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2016.7759114"},{"key":"bibr22-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989191"},{"key":"bibr23-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"bibr24-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2006.282366"},{"key":"bibr25-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2008.4650722"},{"key":"bibr26-02783649211069569","unstructured":"Jeng KY, Liu YC, Liu ZY, et al. (2020) Gdn: A coarse-to-fine (c2f) representation for end-to-end 6-dof grasp detection. arXiv preprint arXiv:2010.10695."},{"key":"bibr27-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5980145"},{"key":"bibr28-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2016.7759657"},{"key":"bibr29-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.1996.506534"},{"key":"bibr30-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139793"},{"key":"bibr31-02783649211069569","unstructured":"Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: International conference on learning representations, San Diego, 2015."},{"key":"bibr32-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1177\/0278364915594244"},{"key":"bibr33-02783649211069569","first-page":"2002","volume":"15","author":"Kragic D","year":"2002","journal-title":"Computational Vision and Active Perception Laboratory, Fiskartorpsv"},{"key":"bibr34-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2017.8202237"},{"key":"bibr35-02783649211069569","unstructured":"Lab RL (2013) Cornell grasping dataset. http:\/\/pr.cs.cornell.edu\/grasping\/rect_data\/data.php (Accessed 09-01-2017)."},{"key":"bibr36-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01264-9_45"},{"key":"bibr37-02783649211069569","doi-asserted-by":"crossref","unstructured":"Le QV, Kamm D, Kara AF, et al. (2010) Learning to grasp objects with multiple contact points. In: IEEE international conference on robotics and automation, Anchorage, AK, USA, 3\u20137 May 2010, pp. 5062\u20135069.","DOI":"10.1109\/ROBOT.2010.5509508"},{"key":"bibr38-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989545"},{"key":"bibr39-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1177\/0278364914549607"},{"issue":"1","key":"bibr40-02783649211069569","first-page":"1334","volume":"17","author":"Levine S","year":"2016","journal-title":"The Journal of Machine Learning Research"},{"key":"bibr41-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-50115-4_16"},{"key":"bibr42-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8794435"},{"key":"bibr43-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9197256"},{"key":"bibr44-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9197413"},{"key":"bibr45-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-28619-4_35"},{"key":"bibr46-02783649211069569","unstructured":"Mahler J, Goldberg K (2017) Learning deep policies for robot bin picking by simulating robust grasping sequences. In: Conference on robot learning, Proceedings of Machine Learning Research (PMLR), pp. 515\u2013524."},{"key":"bibr47-02783649211069569","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2017.XIII.058"},{"key":"bibr48-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8460887"},{"key":"bibr49-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aau4984"},{"key":"bibr50-02783649211069569","doi-asserted-by":"crossref","unstructured":"Mahler J, Pokorny FT, Hou B, et al. (2016) Dex-Net 1.0: A cloud-based network of 3d objects for robust grasp planning using a multi-armed bandit model with correlated rewards. In: IEEE international conference on robotics and automation, Stockholm, Sweden, 16\u201321 May 2016, pp. 1957\u20131964.","DOI":"10.1109\/ICRA.2016.7487342"},{"key":"bibr51-02783649211069569","first-page":"027836491985906","volume":"39","author":"Morrison D","year":"2019","journal-title":"The International Journal of Robotics Research"},{"key":"bibr52-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00299"},{"key":"bibr53-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9197318"},{"key":"bibr54-02783649211069569","unstructured":"Newell A, Huang Z, Deng J (2017) Associative embedding: end-to-end learning for joint detection and grouping. In: Advances in neural information processing systems, Long Beach, CA, USA, 4-9 December 2017, pp. 2277\u20132287."},{"key":"bibr55-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46484-8_29"},{"key":"bibr56-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9196740"},{"key":"bibr57-02783649211069569","unstructured":"Paszke A, Gross S, Chintala S, et al. (2017) Automatic differentiation in PyTorch. In:Neural information processing systems: autodiff workshop, Long Beach, CA, USA, 4-9 December 2017."},{"key":"bibr58-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2004.1308797"},{"key":"bibr59-02783649211069569","volume-title":"Learning visual features to predict hand orientations","author":"Piater JH","year":"2002"},{"key":"bibr60-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487517"},{"key":"bibr61-02783649211069569","first-page":"53","volume-title":"Conference on robot learning","author":"Qin Y","year":"2020"},{"key":"bibr62-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2010.5650493"},{"key":"bibr63-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139361"},{"key":"bibr64-02783649211069569","first-page":"91","volume-title":"Advances in Neural Information Processing Systems","author":"Ren S","year":"2015"},{"key":"bibr65-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2011.07.016"},{"key":"bibr66-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2019.2895878"},{"key":"bibr67-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1177\/0278364907087172"},{"key":"bibr68-02783649211069569","unstructured":"Saxena A, Wong LL, Ng AY (2008b) Learning grasp strategies with partial shape information. In: Proceedings of the AAAI conference on artificial intelligence, Chicago, IL, USA, 13-17 July 2008, pp. 1491\u20131494."},{"key":"bibr69-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICMSR.2019.8835468"},{"key":"bibr70-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1177\/027836499601500302"},{"key":"bibr71-02783649211069569","volume-title":"International conference on learning representations","author":"Simonyan K","year":"2015"},{"key":"bibr72-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS40897.2019.8967989"},{"key":"bibr73-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9561877"},{"key":"bibr74-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1177\/0278364917735594"},{"key":"bibr75-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-51532-8_19"},{"key":"bibr76-02783649211069569","unstructured":"Tremblay J, To T, Sundaralingam B, et al. (2018) Deep object pose estimation for semantic robotic grasping of household objects. In: Conference on robot learning, Zurich, Switzerland, 29-31 October 2018, pp. 306\u2013316."},{"key":"bibr77-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2015.7354004"},{"key":"bibr78-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2017.2651945"},{"key":"bibr79-02783649211069569","unstructured":"Viereck U, Pas At, Saenko K, et al. (2017) Learning a visuomotor controller for real world robotic grasping using simulated depth images. In: Conference on robot learning, Mountain View, CA, USA, 13-15 November 2017, pp. 291\u2013300."},{"key":"bibr80-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00813"},{"key":"bibr81-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1177\/1687814016668077"},{"key":"bibr82-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-64107-2_50"},{"key":"bibr83-02783649211069569","unstructured":"Wu C, Chen J, Cao Q, et al. (2020) Grasp proposal networks: an end-to-end solution for visual learning of robotic grasps. arXiv preprint arXiv:2009.12606."},{"key":"bibr84-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3062560"},{"key":"bibr85-02783649211069569","unstructured":"Xu R, Chu FJ, Vela PA (2020a) The abridged Jacquard dataset. https:\/\/smartech.gatech.edu\/handle\/1853\/64897."},{"key":"bibr86-02783649211069569","unstructured":"Xu R, Chu FJ, Vela PA (2020b) IVALAb: Grasp keypoint network git repository. https:\/\/github.com\/ivalab\/GraspKpNet."},{"key":"bibr87-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9197331"},{"key":"bibr88-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00255"},{"key":"bibr89-02783649211069569","doi-asserted-by":"publisher","DOI":"10.5220\/0006470701540161"},{"key":"bibr90-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2018.8593986"},{"key":"bibr91-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2018.8461044"},{"key":"bibr92-02783649211069569","doi-asserted-by":"crossref","unstructured":"Zhao B, Zhang H, Lan X, et al. (2020) Regnet: region-based grasp network for single-shot grasp detection in point clouds. arXiv preprint arXiv:2002.12647.","DOI":"10.1109\/ICRA48506.2021.9561920"},{"key":"bibr93-02783649211069569","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2018.8594116"},{"key":"bibr94-02783649211069569","volume-title":"Robotics: Science and Systems Workshop on Revisiting Contact-Turning a Problem into a Solution","author":"Zhou Y","year":"2017"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649211069569","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/02783649211069569","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649211069569","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:16:54Z","timestamp":1777457814000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/02783649211069569"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,28]]},"references-count":94,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2022,4]]}},"alternative-id":["10.1177\/02783649211069569"],"URL":"https:\/\/doi.org\/10.1177\/02783649211069569","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,2,28]]}}}