{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,12]],"date-time":"2026-02-12T17:02:21Z","timestamp":1770915741635,"version":"3.50.1"},"reference-count":46,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2024,7,25]],"date-time":"2024-07-25T00:00:00Z","timestamp":1721865600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,7,25]],"date-time":"2024-07-25T00:00:00Z","timestamp":1721865600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/fund","name":"Ministerstvo \u0160kolstv\u00ed, Ml\u00e1de\u017ee a T\u011blov\u00fdchovy","doi-asserted-by":"publisher","award":["CZ.02.1.01\/0.0\/0.0\/17_049\/0008394"],"award-info":[{"award-number":["CZ.02.1.01\/0.0\/0.0\/17_049\/0008394"]}],"id":[{"id":"10.13039\/fund","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Intell Robot Syst"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Picking up non-trivial objects from a bin with a robotic arm is a common task of modern industrial processes. Here, an efficient data-driven method of grasping point detection, based on an attention squeeze parallel U-shaped neural network (ASP U-Net) for the bin picking task, is proposed. The method directly provides all necessary information about the feasible grasping points of objects, which are randomly or regularly arranged in a bin with side walls. Moreover, the method is able to evaluate and select the optimal grasping point among the feasible ones for two types of end effectors, i.e., a vacuum cup and a parallel gripper. The key element of the utilized ASP U-Net neural network is the transformation of a single RGB-Depth image of the bin containing nontrivial objects into a schematic grey-scale frame, where the positions and poses of the grasping points are coded into gradient geometric shapes. The experiments carried out in this study include a comprehensive set of scenes with randomly scattered, ordered, and semi-ordered objects arranged in impeccable or deformed bins. The results indicate outstanding accuracy with more than acceptable computational requirements. Additionally, the scaling possibilities of the method can offer extremely lightweight implementations, applicable, for example, to battery-powered edge-computing devices with low RAM capacity.<\/jats:p>","DOI":"10.1007\/s10846-024-02153-9","type":"journal-article","created":{"date-parts":[[2024,7,26]],"date-time":"2024-07-26T12:03:18Z","timestamp":1721995398000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Memory Efficient Deep Learning-Based Grasping Point Detection of Nontrivial Objects for Robotic Bin Picking"],"prefix":"10.1007","volume":"110","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7359-0764","authenticated-orcid":false,"given":"Petr","family":"Dolezel","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2324-162X","authenticated-orcid":false,"given":"Dominik","family":"Stursa","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2813-7343","authenticated-orcid":false,"given":"Dusan","family":"Kopecky","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2024,7,25]]},"reference":[{"key":"2153_CR1","doi-asserted-by":"publisher","unstructured":"Goel, R.,Gupta, P.:Robotics and industry 4.0. Advances in Science, Technology and Innovation, 157\u2013169 (2020). https:\/\/doi.org\/10.1007\/978-3-030-14544-6_9","DOI":"10.1007\/978-3-030-14544-6_9"},{"key":"2153_CR2","unstructured":"Robotics, I.F.: Robot Density nearly Doubled globally. https:\/\/ifr.org\/ifr-press-releases\/news\/robot-density-nearly-doubled-globally. 2021-1-7 (2021)"},{"issue":"3","key":"2153_CR3","doi-asserted-by":"publisher","first-page":"1677","DOI":"10.1007\/s10462-020-09888-5","volume":"54","author":"G Du","year":"2021","unstructured":"Du, G., Wang, K., Lian, S., Zhao, K.: Vision-based robotic grasping from object localization, object pose estimation to grasp estimation for parallel grippers: a review. Artif. Intell. Rev. 54(3), 1677\u20131734 (2021). https:\/\/doi.org\/10.1007\/s10462-020-09888-5","journal-title":"Artif. Intell. Rev."},{"key":"2153_CR4","doi-asserted-by":"publisher","unstructured":"Kumra, S., Kanan, C.: Robotic Grasp Detection Using Deep Convolutional Neural Networks, 2017-September, pp. 769\u2013776 (2017). https:\/\/doi.org\/10.1109\/IROS.2017.8202237","DOI":"10.1109\/IROS.2017.8202237"},{"key":"2153_CR5","doi-asserted-by":"publisher","unstructured":"Slabanja, J., Meden, B., Peer, P., Jakli\u010d, A., Solina, F.: Segmentation and Reconstruction of 3D Models from a Point Cloud with Deep Neural Networks, pp. 118\u2013123 (2018). https:\/\/doi.org\/10.1109\/ICTC.2018.8539500","DOI":"10.1109\/ICTC.2018.8539500"},{"key":"2153_CR6","unstructured":"InPicker: Applications (2021). https:\/\/inpicker.com\/applications\/"},{"key":"2153_CR7","unstructured":"Robotics, M.C.: Structured vs Random Bin Picking (2021). https:\/\/motioncontrolsrobotics.com\/structured-vs-random-bin-picking\/"},{"issue":"7\u20138","key":"2153_CR8","doi-asserted-by":"publisher","first-page":"560","DOI":"10.1080\/01691864.2019.1698463","volume":"34","author":"M Fujita","year":"2020","unstructured":"Fujita, M., Domae, Y., Noda, A., Garcia Ricardez, G.A., Nagatani, T., Zeng, A., Song, S., Rodriguez, A., Causo, A., Chen, I.M., Ogasawara, T.: What are the important technologies for bin picking? technology analysis of robots in competitions based on a set of performance metrics. Adv. Robot. 34(7\u20138), 560\u2013574 (2020). https:\/\/doi.org\/10.1080\/01691864.2019.1698463","journal-title":"Adv. Robot."},{"key":"2153_CR9","doi-asserted-by":"publisher","first-page":"172231","DOI":"10.1109\/ACCESS.2019.2956508","volume":"7","author":"L Jiao","year":"2019","unstructured":"Jiao, L., Zhao, J.: A survey on the new generation of deep learning in image processing. IEEE Access 7, 172231\u2013172263 (2019). https:\/\/doi.org\/10.1109\/ACCESS.2019.2956508","journal-title":"IEEE Access"},{"key":"2153_CR10","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3086417","author":"P Dolezel","year":"2021","unstructured":"Dolezel, P., Stursa, D., Kopecky, D., Jecha, J.: Memory efficient grasping point detection of nontrivial objects. IEEE Access (2021). https:\/\/doi.org\/10.1109\/ACCESS.2021.3086417","journal-title":"IEEE Access"},{"key":"2153_CR11","doi-asserted-by":"publisher","unstructured":"Tu, J., Wang, P., Liu, F.: PP-RCNN: Point-Pillars Feature Set Abstraction for 3D Real-time Object Detection, 2021-July (2021). https:\/\/doi.org\/10.1109\/IJCNN52387.2021.9534098","DOI":"10.1109\/IJCNN52387.2021.9534098"},{"key":"2153_CR12","doi-asserted-by":"publisher","unstructured":"Bai, X., Luo, Z., Zhou, L., Fu, H., Quan, L., Tai, C.-L.: D3Feat: Joint Learning of Dense Detection and Description of 3D Local Features, pp. 6358\u20136366 (2020). https:\/\/doi.org\/10.1109\/CVPR42600.2020.00639","DOI":"10.1109\/CVPR42600.2020.00639"},{"key":"2153_CR13","doi-asserted-by":"publisher","unstructured":"Deng, H., Birdal, T., Ilic, S.: PPFNet: Global Context Aware Local Features for Robust 3D Point Matching, pp. 195\u2013205 (2018). https:\/\/doi.org\/10.1109\/CVPR.2018.00028","DOI":"10.1109\/CVPR.2018.00028"},{"key":"2153_CR14","doi-asserted-by":"publisher","unstructured":"Gualtieri, M., Pas, A.T., Saenko, K., Platt, R.: High Precision Grasp Pose Detection in Dense Clutter, 2016-November, pp. 598\u2013605 (2016). https:\/\/doi.org\/10.1109\/IROS.2016.7759114","DOI":"10.1109\/IROS.2016.7759114"},{"issue":"2","key":"2153_CR15","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1023\/B:VISI.0000029664.99615.94","volume":"60","author":"DG Lowe","year":"2004","unstructured":"Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vision 60(2), 91\u2013110 (2004). https:\/\/doi.org\/10.1023\/B:VISI.0000029664.99615.94","journal-title":"Int. J. Comput. Vision"},{"key":"2153_CR16","doi-asserted-by":"publisher","first-page":"190","DOI":"10.1016\/j.rcim.2016.09.001","volume":"44","author":"S Astanin","year":"2017","unstructured":"Astanin, S., Antonelli, D., Chiabert, P., Alletto, C.: Reflective workpiece detection and localization for flexible robotic cells. Robotics and Computer-Integrated Manufacturing 44, 190\u2013198 (2017). https:\/\/doi.org\/10.1016\/j.rcim.2016.09.001","journal-title":"Robotics and Computer-Integrated Manufacturing"},{"key":"2153_CR17","doi-asserted-by":"publisher","unstructured":"Park, K., Patten, T., Vincze, M.: Pix2pose: Pixel-wise Coordinate Regression of Objects for 6D Pose Estimation, 2019-October, pp. 7667\u20137676 (2019). https:\/\/doi.org\/10.1109\/ICCV.2019.00776","DOI":"10.1109\/ICCV.2019.00776"},{"key":"2153_CR18","doi-asserted-by":"publisher","first-page":"36","DOI":"10.1016\/j.cag.2015.05.021","volume":"50","author":"P Han","year":"2015","unstructured":"Han, P., Zhao, G.: Cad-based 3d objects recognition in monocular images for mobile augmented reality. Computers and Graphics (Pergamon) 50, 36\u201346 (2015). https:\/\/doi.org\/10.1016\/j.cag.2015.05.021","journal-title":"Computers and Graphics (Pergamon)"},{"key":"2153_CR19","doi-asserted-by":"publisher","unstructured":"Ulrich, M., Wiedemann, C., Steger, C.: CAD-based Recognition of 3D Objects in Monocular Images, pp. 1191\u20131198 (2009). https:\/\/doi.org\/10.1109\/ROBOT.2009.5152511","DOI":"10.1109\/ROBOT.2009.5152511"},{"key":"2153_CR20","doi-asserted-by":"publisher","unstructured":"Koz\u00e1k, V., Sushkov, R., Kulich, M., P\u0159eu\u010dil, L.: Data-driven object pose estimation in a practical bin-picking application. Sensors 21(18) (2021). https:\/\/doi.org\/10.3390\/s21186093","DOI":"10.3390\/s21186093"},{"key":"2153_CR21","unstructured":"Intel: Depth Camera D435. https:\/\/www.intelrealsense.com\/depth-camera-d435\/. 2021-12-20 (2021)"},{"key":"2153_CR22","unstructured":"Microsoft: Kinect for Windows (2021). https:\/\/developer.microsoft.com\/windows\/kinect\/"},{"issue":"3\u20134","key":"2153_CR23","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1007\/s10846-017-0501-1","volume":"87","author":"K-T Song","year":"2017","unstructured":"Song, K.-T., Wu, C.-H., Jiang, S.-Y.: Cad-based pose estimation design for random bin picking using a rgb-d camera. Journal of Intelligent and Robotic Systems: Theory and Applications 87(3\u20134), 455\u2013470 (2017). https:\/\/doi.org\/10.1007\/s10846-017-0501-1","journal-title":"Journal of Intelligent and Robotic Systems: Theory and Applications"},{"key":"2153_CR24","doi-asserted-by":"publisher","unstructured":"He, R., Rojas, J., Guan, Y.: A 3D Object Detection and Pose Estimation Pipeline Using RGB-D Images, 2018-January, pp. 1527\u20131532 (2018). https:\/\/doi.org\/10.1109\/ROBIO.2017.8324634","DOI":"10.1109\/ROBIO.2017.8324634"},{"key":"2153_CR25","doi-asserted-by":"publisher","unstructured":"Iriondo, A., Lazkano, E., Ansuategi, A.: Affordance-based grasping point detection using graph convolutional networks for industrial bin-picking applications. eSensors (Switzerland) 21(3), 1\u201326 (2021). https:\/\/doi.org\/10.3390\/s21030816","DOI":"10.3390\/s21030816"},{"key":"2153_CR26","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1016\/j.promfg.2020.10.002","volume":"51","author":"AS Olesen","year":"2020","unstructured":"Olesen, A.S., Gergaly, B.B., Ryberg, E.A., Thomsen, M.R., Chrysostomou, D.: A Collaborative Robot Cell for Random Bin-picking Based on Deep Learning Policies and a Multi-gripper Switching Strategy 51, 3\u201310 (2020). https:\/\/doi.org\/10.1016\/j.promfg.2020.10.002","journal-title":"A Collaborative Robot Cell for Random Bin-picking Based on Deep Learning Policies and a Multi-gripper Switching Strategy"},{"key":"2153_CR27","doi-asserted-by":"publisher","unstructured":"Zhuang, C., Wang, Z., Zhao, H., Ding, H.: Semantic part segmentation method based 3d object pose estimation with rgb-d images for bin-picking. Robotics and Computer-Integrated Manufacturing 68 (2021). https:\/\/doi.org\/10.1016\/j.rcim.2020.102086","DOI":"10.1016\/j.rcim.2020.102086"},{"key":"2153_CR28","doi-asserted-by":"publisher","unstructured":"Hu, Y., Hugonot, J., Fua, P., Salzmann, M.: Segmentation-driven 6D Object Pose Estimation, 2019-June, pp. 3380\u20133389 (2019). https:\/\/doi.org\/10.1109\/CVPR.2019.00350","DOI":"10.1109\/CVPR.2019.00350"},{"key":"2153_CR29","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.3047388","author":"S Peng","year":"2020","unstructured":"Peng, S., Zhou, X., Liu, Y., Lin, H., Huang, Q., Bao, H.: Pvnet: pixel-wise voting network for 6dof object pose estimation. IEEE Trans. Pattern Anal. Mach. Intell. (2020). https:\/\/doi.org\/10.1109\/TPAMI.2020.3047388","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"2153_CR30","doi-asserted-by":"publisher","unstructured":"Le, T.-T., Lin, C.-Y.: Bin-picking for planar objects based on a deep learning network: a case study of usb packs. Sensors (Switzerland) 19(16) (2019). https:\/\/doi.org\/10.3390\/s19163602","DOI":"10.3390\/s19163602"},{"key":"2153_CR31","doi-asserted-by":"publisher","unstructured":"Wang, H., Situ, H., Zhuang, C.: 6D Pose Estimation for Bin-Picking Based on Improved Mask R-CNN and DenseFusion, 2021-September (2021). https:\/\/doi.org\/10.1109\/ETFA45728.2021.9613149","DOI":"10.1109\/ETFA45728.2021.9613149"},{"key":"2153_CR32","unstructured":"Holz, D., Behnke, S.: Fast Edge-based Detection and Localization of Transport Boxes and Pallets in RGB-D Images for Mobile Robot Bin Picking, pp. 133\u2013140 (2016)"},{"key":"2153_CR33","unstructured":"Fanuc: SCARA Robots for increased productivity (2021). https:\/\/www.fanuc.eu\/uk\/en\/robots\/robot-filter-page\/scara-series"},{"key":"2153_CR34","unstructured":"Staubli: Our SCARA industrial robots (2021). https:\/\/www.staubli.com\/en\/robotics\/product-range\/industrial-robots\/4-axis-scara-robots\/"},{"key":"2153_CR35","unstructured":"ABB: IRB 910SC SCARA (2021). https:\/\/new.abb.com\/products\/robotics\/industrial-robots\/irb-910sc\/"},{"key":"2153_CR36","unstructured":"NVIDIA: Jetson Nano Developer Kit (2021). https:\/\/developer.nvidia.com\/embedded\/jetson-nano-developer-kit\/"},{"key":"2153_CR37","doi-asserted-by":"publisher","unstructured":"Suzen, A.A., Duman, B., Sen, B.: Benchmark analysis of jetson tx2, jetson nano and raspberry pi using deep-cnn. In: 2020 International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), pp. 1\u20135 (2020). https:\/\/doi.org\/10.1109\/HORA49412.2020.9152915","DOI":"10.1109\/HORA49412.2020.9152915"},{"key":"2153_CR38","doi-asserted-by":"publisher","first-page":"234","DOI":"10.1007\/978-3-319-24574-4_28","volume":"9351","author":"O Ronneberger","year":"2015","unstructured":"Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 9351, 234\u2013241 (2015). https:\/\/doi.org\/10.1007\/978-3-319-24574-4_28","journal-title":"Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)"},{"key":"2153_CR39","unstructured":"Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: SqueezeNet: AlexNet-level accuracy with 50$$\\times $$ fewer parameters and $$<$$0.5MB model size (2016)"},{"key":"2153_CR40","doi-asserted-by":"publisher","unstructured":"Nanfack, G., Elhassouny, A., Oulad Haj\u00a0Thami, R.: Squeeze-SegNet: A New Fast Deep Convolutional Neural Network for Semantic Segmentation, 10696 (2018). https:\/\/doi.org\/10.1117\/12.2309497","DOI":"10.1117\/12.2309497"},{"key":"2153_CR41","doi-asserted-by":"publisher","unstructured":"Beheshti, N., Johnsson, L.: Squeeze u-net: a memory and energy efficient image segmentation network. In: 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1495\u20131504 (2020). https:\/\/doi.org\/10.1109\/CVPRW50498.2020.00190","DOI":"10.1109\/CVPRW50498.2020.00190"},{"key":"2153_CR42","unstructured":"Oktay, O., Schlemper, J., Folgoc, L.L., Lee, M., Heinrich, M., Misawa, K., Mori, K., McDonagh, S., Hammerla, N.Y., Kainz, B., Glocker, B., Rueckert, D.: Attention U-Net: Learning Where to Look for the Pancreas (2018)"},{"key":"2153_CR43","doi-asserted-by":"publisher","unstructured":"An, G.: The effects of adding noise during backpropagation training on a generalization performance. eNeural Computation 8(3), 643\u2013674 (1996). https:\/\/doi.org\/10.1162\/neco.1996.8.3.643","DOI":"10.1162\/neco.1996.8.3.643"},{"key":"2153_CR44","doi-asserted-by":"publisher","unstructured":"Dolezel, P., Stursa, D., Kopecky, D.: Suitable ASP U-Net Training Algorithms for Grasping Point Detection of Nontrivial Objects, pp. 1586\u20131591 (2022). https:\/\/doi.org\/10.1109\/CoDIT55151.2022.9803900","DOI":"10.1109\/CoDIT55151.2022.9803900"},{"key":"2153_CR45","unstructured":"Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR abs\/1412.6980 (2014). arXiv:1412.6980"},{"key":"2153_CR46","doi-asserted-by":"publisher","unstructured":"Dogo, E.M., Afolabi, O.J., Nwulu, N.I., Twala, B., Aigbavboa, C.O.: A comparative analysis of gradient descent-based optimization algorithms on convolutional neural networks. In: 2018 International Conference on Computational Techniques, Electronics and Mechanical Systems (CTEMS), pp. 92\u201399 (2018). https:\/\/doi.org\/10.1109\/CTEMS.2018.8769211","DOI":"10.1109\/CTEMS.2018.8769211"}],"container-title":["Journal of Intelligent &amp; Robotic Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10846-024-02153-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10846-024-02153-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10846-024-02153-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,26]],"date-time":"2024-09-26T13:12:36Z","timestamp":1727356356000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10846-024-02153-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,25]]},"references-count":46,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2024,9]]}},"alternative-id":["2153"],"URL":"https:\/\/doi.org\/10.1007\/s10846-024-02153-9","relation":{},"ISSN":["1573-0409"],"issn-type":[{"value":"1573-0409","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,25]]},"assertion":[{"value":"25 July 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 July 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"25 July 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing Interests"}}],"article-number":"110"}}