{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,4]],"date-time":"2026-06-04T15:38:17Z","timestamp":1780587497188,"version":"3.54.1"},"reference-count":33,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2024,7,9]],"date-time":"2024-07-09T00:00:00Z","timestamp":1720483200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,7,9]],"date-time":"2024-07-09T00:00:00Z","timestamp":1720483200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["52102455"],"award-info":[{"award-number":["52102455"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Process Lett"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In order to better solve the visual detection problem of manipulator grasping non-cooperative targets, we propose a method of grasp pose detection based on pixel point and feature fusion. By using the improved U2net network as the backbone for feature extraction and feature fusion of the input image, and the grasp prediction layer detects the grasp pose on each pixel. In order to adapt the U2net to grasp pose detection and improve its detection performance, we improve detection speed and control sampling depth by simplifying its network structure, while retaining some shallow features in feature fusion to enhance its feature extraction capability. We introduce depthwise separable convolution in the grasp prediction layer, further fusing the features extracted from the backbone to obtain predictive feature maps with stronger feature expressiveness. FocalLoss is selected as the loss function to solve the problem of unbalanced positive and negative samples in network training. We use the Cornell dataset for training and testing, perform pixel-level labeling on the image, and replace the labels that are not conducive to the actual grasping. This adaptation helps the dataset better suit the network training and testing while meeting the real-world grasping requirements of the manipulator. The evaluation results on image-wise and object-wise are 95.65% and 91.20% respectively, and the detection speed is 0.007\u00a0s\/frame. We also used the method for actual manipulator grasping experiments. The results show that our method has improved accuracy and speed compared with previous methods, and has strong generalization ability and portability.<\/jats:p>","DOI":"10.1007\/s11063-024-11662-5","type":"journal-article","created":{"date-parts":[[2024,7,9]],"date-time":"2024-07-09T15:22:41Z","timestamp":1720538561000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Detection Method of Manipulator Grasp Pose Based on RGB-D Image"],"prefix":"10.1007","volume":"56","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9183-3547","authenticated-orcid":false,"given":"Cheng","family":"Huang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhen","family":"Pang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiazhong","family":"Xu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,7,9]]},"reference":[{"issue":"6","key":"11662_CR1","doi-asserted-by":"publisher","first-page":"2718","DOI":"10.1109\/TMECH.2019.2945135","volume":"24","author":"R Li","year":"2019","unstructured":"Li R, Qiao H (2019) A survey of methods and strategies for high-precision robotic grasping and assembly tasks\u2014some new trends. IEEE\/ASME Trans Mechatron 24(6):2718\u20132732. https:\/\/doi.org\/10.1109\/TMECH.2019.2945135","journal-title":"IEEE\/ASME Trans Mechatron"},{"key":"11662_CR2","doi-asserted-by":"publisher","unstructured":"Solowjow E, Ugalde I, Shahapurkar Y et al (2020) Industrial robot grasping with deep learning using a programmable logic controller (plc). In: 2020 IEEE 16th international conference on automation science and engineering (CASE), pp 97\u2013103. https:\/\/doi.org\/10.1109\/CASE48305.2020.9216902","DOI":"10.1109\/CASE48305.2020.9216902"},{"issue":"1","key":"11662_CR3","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/computers10010001","volume":"10","author":"RR Devaraja","year":"2020","unstructured":"Devaraja RR, Maskeli\u016bnas R, Dama\u0161evi\u010dius R (2020) Design and evaluation of anthropomorphic robotic hand for object grasping and shape recognition. Computers 10(1):1. https:\/\/doi.org\/10.3390\/computers10010001","journal-title":"Computers"},{"issue":"5","key":"11662_CR4","doi-asserted-by":"publisher","first-page":"1343","DOI":"10.1007\/s12555-019-0186-2","volume":"18","author":"L Chen","year":"2020","unstructured":"Chen L, Huang P, Li Y, Meng Z (2020) Detecting graspable rectangles of objects in robotic grasping. Int J Control Autom Syst 18(5):1343\u20131352. https:\/\/doi.org\/10.1007\/s12555-019-0186-2","journal-title":"Int J Control Autom Syst"},{"issue":"4","key":"11662_CR5","doi-asserted-by":"publisher","first-page":"12235","DOI":"10.1109\/LRA.2022.3214785","volume":"7","author":"Y Shi","year":"2022","unstructured":"Shi Y, Tang Z, Cai X, Zhang H, Hu D, Xu X (2022) Symmetry grasp: symmetry-aware antipodal grasp detection from single-view RGB-D images. IEEE Robot Autom Lett 7(4):12235\u201312242. https:\/\/doi.org\/10.1109\/LRA.2022.3214785","journal-title":"IEEE Robot Autom Lett"},{"key":"11662_CR6","doi-asserted-by":"publisher","unstructured":"Weisz J, Allen P K (2012) Pose error robust grasping from contact wrench space metrics. In: 2012 IEEE international conference on robotics and automation, pp 557\u2013562. https:\/\/doi.org\/10.1109\/ICRA.2012.6224697","DOI":"10.1109\/ICRA.2012.6224697"},{"key":"11662_CR7","first-page":"78","volume-title":"Physics-based grasp planning through clutter","author":"MR Dogar","year":"2012","unstructured":"Dogar MR, Hsiao K, Ciocarlie M, Srinivasa S (2012) Physics-based grasp planning through clutter. MIT Press, Cambridge, pp 78\u201385"},{"key":"11662_CR8","doi-asserted-by":"crossref","unstructured":"Jiang Y, Moseson S, Saxena A (2011) Efficient grasping from rgbd images: Learning using a new rectangle representation. In: 2011 IEEE international conference on robotics and automation, pp 3304\u20133311","DOI":"10.1109\/ICRA.2011.5980145"},{"key":"11662_CR9","doi-asserted-by":"publisher","unstructured":"Fischinger D, Vincze M, Jiang Y (2013) Learning grasps for unknown objects in cluttered scenes. In: 2013 IEEE international conference on robotics and automation, pp 609\u2013616. https:\/\/doi.org\/10.1109\/ICRA.2013.6630636","DOI":"10.1109\/ICRA.2013.6630636"},{"issue":"9","key":"11662_CR10","doi-asserted-by":"publisher","first-page":"1167","DOI":"10.1177\/0278364915577105","volume":"34","author":"D Fischinger","year":"2015","unstructured":"Fischinger D, Weiss A, Vincze M (2015) Learning grasps with topographic features. Int J Robot Res 34(9):1167\u20131194. https:\/\/doi.org\/10.1177\/0278364915577105","journal-title":"Int J Robot Res"},{"key":"11662_CR11","doi-asserted-by":"publisher","unstructured":"Sun C, Yu Y, Liu H, Gu J (2015) Robotic grasp detection using extreme learning machine. In: 2015 IEEE international conference on robotics and biomimetics (ROBIO), pp 1115\u20131120. https:\/\/doi.org\/10.1109\/ROBIO.2015.7418921","DOI":"10.1109\/ROBIO.2015.7418921"},{"key":"11662_CR12","doi-asserted-by":"publisher","first-page":"139489","DOI":"10.1109\/ACCESS.2021.3118541","volume":"9","author":"V Sharma","year":"2021","unstructured":"Sharma V, Gupta M, Kumar A, Mishra D (2021) Video processing using deep learning techniques: a systematic literature review. IEEE Access 9:139489\u2013139507. https:\/\/doi.org\/10.1109\/ACCESS.2021.3118541","journal-title":"IEEE Access"},{"issue":"4","key":"11662_CR13","doi-asserted-by":"publisher","first-page":"3355","DOI":"10.1109\/LRA.2018.2852777","volume":"3","author":"FJ Chu","year":"2018","unstructured":"Chu FJ, Xu R, Vela PA (2018) Real-world multiobject, multigrasp detection. IEEE Robot Autom Lett 3(4):3355\u20133362. https:\/\/doi.org\/10.1109\/LRA.2018.2852777","journal-title":"IEEE Robot Autom Lett"},{"key":"11662_CR14","doi-asserted-by":"publisher","first-page":"159322","DOI":"10.1109\/ACCESS.2019.2950535","volume":"7","author":"Y Xu","year":"2019","unstructured":"Xu Y, Wang L, Yang A, Chen L (2019) GraspCNN: Real-time grasp detection using a new oriented diameter circle representation. IEEE Access 7:159322\u2013159331. https:\/\/doi.org\/10.1109\/ACCESS.2019.2950535","journal-title":"IEEE Access"},{"issue":"4\u20135","key":"11662_CR15","doi-asserted-by":"publisher","first-page":"705","DOI":"10.1177\/0278364914549607","volume":"34","author":"I Lenz","year":"2015","unstructured":"Lenz I, Lee H, Saxena A (2015) Deep learning for detecting robotic grasps. Int J Robot Res 34(4\u20135):705\u2013724. https:\/\/doi.org\/10.1177\/0278364914549607","journal-title":"Int J Robot Res"},{"key":"11662_CR16","unstructured":"Chu F J, Vela P A (2018) Deep grasp: detection and localization of grasps with deep neural networks. arXiv preprint at arXiv:1802.00520, 1"},{"key":"11662_CR17","doi-asserted-by":"publisher","unstructured":"Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. Adv Neural Inf Process Syst 28. https:\/\/doi.org\/10.1109\/TPAMI.2016.2577031","DOI":"10.1109\/TPAMI.2016.2577031"},{"key":"11662_CR18","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770\u2013778","DOI":"10.1109\/CVPR.2016.90"},{"issue":"6","key":"11662_CR19","first-page":"794","volume":"40","author":"J Xia","year":"2018","unstructured":"Xia J, Qian K, Ma XD, Liu H (2018) Fast detection of robot plane grab position based on concatenated convolution neural network. Robot 40(6):794\u2013802","journal-title":"Robot"},{"issue":"6","key":"11662_CR20","doi-asserted-by":"publisher","first-page":"2922","DOI":"10.1109\/TMECH.2020.3048441","volume":"26","author":"L Chen","year":"2021","unstructured":"Chen L, Huang P, Li Y, Meng Z (2021) Edge-dependent efficient grasp rectangle search in robotic grasp detection. IEEE\/ASME Trans Mechatron 26(6):2922\u20132931. https:\/\/doi.org\/10.1109\/TMECH.2020.3048441","journal-title":"IEEE\/ASME Trans Mechatron"},{"key":"11662_CR21","doi-asserted-by":"publisher","unstructured":"Redmon J, Angelova A (2015) Real-time grasp detection using convolutional neural networks. In: 2015 IEEE international conference on robotics and automation (ICRA), pp 1316\u20131322. https:\/\/doi.org\/10.1109\/ICRA.2015.7139361","DOI":"10.1109\/ICRA.2015.7139361"},{"key":"11662_CR22","doi-asserted-by":"publisher","unstructured":"Watson J, Hughes J, Iida F (2017) Real-world, real-time robotic grasping with convolutional neural networks. In: Towards autonomous robotic systems: 18th annual conference, TAROS 2017, Guildford, UK, Proceedings 18. Springer International Publishing, pp. 617\u2013626. https:\/\/doi.org\/10.1007\/978-3-319-64107-2_50","DOI":"10.1007\/978-3-319-64107-2_50"},{"key":"11662_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1155\/2021\/5512728","volume":"2021","author":"L Zhang","year":"2021","unstructured":"Zhang L, Wu D (2021) A single target grasp detection network based on convolutional neural network. Comput Intell Neurosci 2021:1\u201312. https:\/\/doi.org\/10.1155\/2021\/5512728","journal-title":"Comput Intell Neurosci"},{"issue":"2\u20133","key":"11662_CR24","doi-asserted-by":"publisher","first-page":"183","DOI":"10.1177\/0278364919859066","volume":"39","author":"D Morrison","year":"2020","unstructured":"Morrison D, Corke P, Leitner J (2020) Learning robust, real-time, reactive robotic grasping. Int J Robot Res 39(2\u20133):183\u2013201. https:\/\/doi.org\/10.1177\/0278364919859066","journal-title":"Int J Robot Res"},{"key":"11662_CR25","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TIM.2022.3165825","volume":"71","author":"H Cheng","year":"2022","unstructured":"Cheng H, Wang Y, Meng MQH (2022) A robot grasping system with single-stage anchor-free deep grasp detector. IEEE Trans Instrum Meas 71:1\u201312. https:\/\/doi.org\/10.1109\/TIM.2022.3165825","journal-title":"IEEE Trans Instrum Meas"},{"key":"11662_CR26","doi-asserted-by":"publisher","unstructured":"Kumra S, Joshi S, Sahin F (2020) Antipodal robotic grasping using generative residual convolutional neural network. In: 2020 IEEE\/RSJ international conference on intelligent robots and systems (IROS), pp 9626\u20139633. https:\/\/doi.org\/10.1109\/IROS45743.2020.9340777","DOI":"10.1109\/IROS45743.2020.9340777"},{"issue":"05","key":"11662_CR27","doi-asserted-by":"publisher","first-page":"2350010","DOI":"10.1142\/S021984362350010X","volume":"20","author":"Z Wang","year":"2023","unstructured":"Wang Z, Leng L, Zhou X, Zhao Y (2023) Robotic grasp pose detection method based on multiscale features. Int J Humanoid Rob 20(05):2350010. https:\/\/doi.org\/10.1142\/S021984362350010X","journal-title":"Int J Humanoid Rob"},{"key":"11662_CR28","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2020.107404","volume":"106","author":"X Qin","year":"2020","unstructured":"Qin X, Zhang Z, Huang C, Dehghan M, Zaiane OR, Jagersand M (2020) U2-Net: going deeper with nested U-structure for salient object detection. Pattern Recogn 106:107404. https:\/\/doi.org\/10.1016\/j.patcog.2020.107404","journal-title":"Pattern Recogn"},{"issue":"5","key":"11662_CR29","doi-asserted-by":"publisher","first-page":"2710","DOI":"10.3390\/s23052710","volume":"23","author":"X Mou","year":"2023","unstructured":"Mou X, Lei S, Zhou X (2023) YOLO-FR: a YOLOv5 infrared small target detection algorithm based on feature reassembly sampling method. Sensors 23(5):2710. https:\/\/doi.org\/10.3390\/s23052710","journal-title":"Sensors"},{"issue":"17","key":"11662_CR30","doi-asserted-by":"publisher","first-page":"8643","DOI":"10.3390\/app12178643","volume":"12","author":"A Zafar","year":"2022","unstructured":"Zafar A, Aamir M, Mohd Nawi N et al (2022) A comparison of pooling methods for convolutional neural networks. Appl Sci 12(17):8643. https:\/\/doi.org\/10.3390\/app12178643","journal-title":"Appl Sci"},{"key":"11662_CR31","doi-asserted-by":"publisher","first-page":"3129875","DOI":"10.1109\/TIM.2021.3129875","volume":"71","author":"D Liu","year":"2022","unstructured":"Liu D, Tao X, Yuan L, Du Y, Cong M (2022) Robotic objects detection and grasping in clutter based on cascaded deep convolutional neural network. IEEE Trans Instrum Meas 71:3129875. https:\/\/doi.org\/10.1109\/TIM.2021.3129875","journal-title":"IEEE Trans Instrum Meas"},{"key":"11662_CR32","first-page":"4875","volume":"7","author":"U Asif","year":"2018","unstructured":"Asif U, Tang J, Harrer S (2018) GraspNet: an efficient convolutional neural network for real-time grasp detection for low-powered devices. IJCAI 7:4875\u20134882","journal-title":"IJCAI"},{"key":"11662_CR33","doi-asserted-by":"publisher","unstructured":"Wang S, Jiang X, Zhao J, Wang X, Zhou W, Liu Y (2019) Efficient fully convolution neural network for generating pixel wise robotic grasps with high resolution images. In: 2019 IEEE international conference on robotics and biomimetics (ROBIO), pp 474\u2013480. https:\/\/doi.org\/10.1109\/ROBIO49542.2019.8961711","DOI":"10.1109\/ROBIO49542.2019.8961711"}],"container-title":["Neural Processing Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11662-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11063-024-11662-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11662-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,8,31]],"date-time":"2024-08-31T16:10:56Z","timestamp":1725120656000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11063-024-11662-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,9]]},"references-count":33,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,8]]}},"alternative-id":["11662"],"URL":"https:\/\/doi.org\/10.1007\/s11063-024-11662-5","relation":{},"ISSN":["1573-773X"],"issn-type":[{"value":"1573-773X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,9]]},"assertion":[{"value":"31 May 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 July 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"211"}}