{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,4]],"date-time":"2026-11-04T03:17:18Z","timestamp":1793762238991,"version":"4.6.1"},"reference-count":36,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T00:00:00Z","timestamp":1683849600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>Intelligent manipulation of robots in an unstructured environment is an important application field of artificial intelligence, which means that robots must have the ability of autonomous cognition and decision-making. A typical example of this type of environment is a cluttered scene where objects are stacked and close together. In clutter, the target(s) may be one or more, and efficiently completing the target(s) grasping task is challenging. In this study, an efficient push-grasping method based on reinforcement learning is proposed for multiple target objects in clutter. The key point of this method is to consider the states of all the targets so that the pushing action can expand the grasping space of all targets as much as possible to achieve the minimum total number of pushing and grasping actions and then improve the efficiency of the whole system. At this point, we adopted the mask fusion of multiple targets, clearly defined the concept of graspable probability, and provided the reward mechanism of multi-target push-grasping. Experiments were conducted in both the simulation and real systems. The experimental results indicated that, compared with other methods, the proposed method performed better for multiple target objects and a single target in clutter. It is worth noting that our policy was only trained under simulation, which was then transferred to the real system without retraining or fine-tuning.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1188468","type":"journal-article","created":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T01:47:52Z","timestamp":1683856072000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":20,"title":["Efficient push-grasping for multiple target objects in clutter environments"],"prefix":"10.3389","volume":"17","author":[{"given":"Liangdong","family":"Wu","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yurou","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhengwei","family":"Li","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zhiyong","family":"Liu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,5,12]]},"reference":[{"key":"B1","article-title":"\u201cHindsight experience replay,\u201d","author":"Andrychowicz","year":"2017","journal-title":"Advances in neural information processing systems"},{"key":"B2","doi-asserted-by":"publisher","first-page":"3008","DOI":"10.1109\/ICRA.2017.7989345","volume":"2017","author":"Bauza","year":"2017"},{"key":"B3","doi-asserted-by":"publisher","first-page":"289","DOI":"10.1109\/TRO.2013.2289018","article-title":"Data-driven grasp synthesis\u2014a survey","volume":"30","author":"Bohg","year":"2013","journal-title":"IEEE Transact. Robot."},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v28i1.9052","article-title":"\u201cEfficient optimization for autonomous robotic manipulation of natural objects,\u201d","author":"Boularias","year":"2014","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 28"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v29i1.9378","article-title":"\u201cLearning to manipulate unknown objects in clutter by reinforcement Twenty-Ninth,\u201d","author":"Boularias","year":"2015","journal-title":"AAAI Conference on Artificial Intelligence"},{"key":"B6","doi-asserted-by":"publisher","first-page":"2370","DOI":"10.1109\/LRA.2018.2810544","article-title":"Learning object grasping for soft robot hands","volume":"3","author":"Choi","year":"2018","journal-title":"IEEE Robot. Automat. Lett."},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2011.6094737","article-title":"\u201cPush planning for object placement on cluttered table surfaces,\u201d","author":"Cosgun","year":"2011","journal-title":"2011 IEEE\/RSJ International Conference on Intelligent Robots and Systems"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1109\/COASE.2018.8560406","article-title":"\u201cLinear push policies to increase grasp access for robot bin picking,\u201d","author":"Danielczuk","year":"2018","journal-title":"2018 IEEE 14th International Conference on Automation Science and Engineering (CASE)."},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848","article-title":"\u201cImagenet: A large-scale hierarchical image database,\u201d","author":"Deng","year":"2009","journal-title":"2009 IEEE conference on computer vision and pattern recognition"},{"key":"B10","doi-asserted-by":"publisher","DOI":"10.1109\/IROS40897.2019.8967899","article-title":"\u201cDeep reinforcement learning for robotic pushing and picking in cluttered environment,\u201d","author":"Deng","year":"2019","journal-title":"2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)."},{"key":"B11","doi-asserted-by":"publisher","first-page":"2123","DOI":"10.1109\/IROS.2010.5652970","volume":"2010","author":"Dogar","year":"2010"},{"key":"B12","doi-asserted-by":"publisher","first-page":"11444","DOI":"10.1109\/CVPR42600.2020.01146","article-title":"\u201cGraspnet-1billion: A large-scale benchmark for general object grasping,\u201d","author":"Fang","year":"2020","journal-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition"},{"key":"B13","doi-asserted-by":"publisher","first-page":"662","DOI":"10.1109\/LRA.2019.2892591","article-title":"Pre-grasp sliding manipulation of thin objects using soft, compliant, or underactuated hands","volume":"4","author":"Hang","year":"2019","journal-title":"IEEE Robot. Automation Lett."},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9561073","article-title":"\u201cDipn: Deep interaction prediction network with application to clutter removal,\u201d","author":"Huang","year":"","journal-title":"2021 IEEE International Conference on Robotics and Automation (ICRA)."},{"key":"B15","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1109\/LRA.2021.3123373","article-title":"Visual foresight trees for object retrieval from clutter with non-prehensile rearrangement","volume":"7","author":"Huang","year":"","journal-title":"IEEE Robot. Automat. Lett."},{"key":"B16","doi-asserted-by":"publisher","first-page":"4700","DOI":"10.1109\/CVPR.2017.243","article-title":"\u201cDensely connected convolutional networks,\u201d","author":"Huang","year":"2017","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B17","first-page":"448","article-title":"\u201cBatch normalization: Accelerating deep network training by reducing internal covariate shift,\u201d","author":"Ioffe","year":"2015","journal-title":"International conference on machine learning."},{"key":"B18","first-page":"651","article-title":"\u201cScalable deep reinforcement learning for vision-based robotic manipulation,\u201d","author":"Kalashnikov","year":"2018","journal-title":"Conference on Robot Learning"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793972","article-title":"\u201cRobust object grasping in clutter via singulation\u201d","author":"Kiatos","year":"2019","journal-title":"2019 International Conference on Robotics and Automation (ICRA)."},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1109\/IROS45743.2020.9341545","article-title":"\u201cVisuomotor mechanical search: Learning to retrieve target objects in clutter,\u201d","author":"Kurenkov","year":"2020","journal-title":"2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)."},{"key":"B21","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1109\/MRA.2020.2976322","article-title":"Multifingered grasp planning via inference in deep neural networks: outperforming sampling by learning differentiable models","volume":"27","author":"Lu","year":"2020","journal-title":"IEEE Robot. Automat. Lett"},{"key":"B22","first-page":"515","article-title":"\u201cLearning deep policies for robot bin picking by simulating robust grasping sequences,\u201d","author":"Mahler","year":"2017","journal-title":"Conference on robot learning"},{"key":"B23","unstructured":"Dex-net 2.0: Deep learning to plan robust grasps with synthetic point clouds and analytic grasp metrics\n            MahlerJ.\n            LiangJ.\n            NiyazS.\n            LaskeyM.\n            DoanR.\n            LiuX.\n          arXiv [Preprint].2017"},{"key":"B24","doi-asserted-by":"publisher","first-page":"712","DOI":"10.1109\/LRA.2017.2783445","article-title":"Randomized physics-based motion planning for grasping in cluttered and uncertain environments","volume":"3","author":"Moll","year":"2017","journal-title":"IEEE Robot. Automat. Lett"},{"key":"B25","first-page":"807","article-title":"\u201cRectified linear units improve restricted boltzmann machines,\u201d","author":"Nair","year":"2010","journal-title":"Proceedings of the 27th International Conference on Machine Learning (ICML-10)"},{"key":"B26","doi-asserted-by":"publisher","DOI":"10.1016\/j.bspc.2021.102444","article-title":"A novel autonomous learning framework to enhance sEMG-based hand gesture recognition using depth information","author":"Ovur","year":"2021","journal-title":"Biomed. Signal Proc. Control"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1108\/AA-11-2020-0178","article-title":"Adaptive sensor fusion labeling framework for hand pose recognition in robot teleoperation","author":"Qi","year":"2021","journal-title":"Assem. Autom."},{"key":"B28","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1016\/j.neucom.2021.11.044","article-title":"DCNN based human activity recognition framework with depth vision guiding","volume":"486","author":"Qi","year":"2022","journal-title":"Neurocomputing"},{"key":"B29","doi-asserted-by":"publisher","first-page":"886","DOI":"10.1177\/0278364912442972","article-title":"From caging to grasping","volume":"31","author":"Rodriguez","year":"2012","journal-title":"Int. J. Robotics Res"},{"key":"B30","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9196647","article-title":"\u201cSplit deep q-learning for robust object singulation,\u201d","author":"Sarantopoulos","year":"2020","journal-title":"2020 IEEE International Conference on Robotics and Automation (ICRA)."},{"key":"B31","unstructured":"Learning to slide unknown objects with differentiable physics simulations\n            SongC.\n            BoulariasA.\n          arXiv [Preprint]2020"},{"key":"B32","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1007\/978-3-319-51532-8_19","article-title":"\u201cUsing geometry to detect grasp poses in 3d point clouds,\u201d","volume":"2018","author":"Ten Pas","year":"2018","journal-title":"Robotics Research"},{"key":"B33","doi-asserted-by":"publisher","first-page":"6337","DOI":"10.1109\/LRA.2021.3092640","article-title":"Efficient learning of goal-oriented push-grasping synergy in clutter","volume":"6","author":"Xu","year":"2021","journal-title":"IEEE Robot. Automat. Lett"},{"key":"B34","doi-asserted-by":"publisher","first-page":"2232","DOI":"10.1109\/LRA.2020.2970622","article-title":"A deep learning approach to grasping the invisible","volume":"5","author":"Yang","year":"2020","journal-title":"IEEE Robot. Automat. Lett."},{"key":"B35","doi-asserted-by":"publisher","first-page":"4238","DOI":"10.1109\/IROS.2018.8593986","volume":"2018","author":"Zeng","year":"2018"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.3389\/fncom.2023.1110889","article-title":"A neural learning approach for simultaneous object detection and grasp detection in cluttered scenes","author":"Zhang","year":"2023","journal-title":"Front. Comput. Neurosci"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1188468\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,12]],"date-time":"2023-05-12T01:47:59Z","timestamp":1683856079000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1188468\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,12]]},"references-count":36,"alternative-id":["10.3389\/fnbot.2023.1188468"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1188468","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,12]]},"article-number":"1188468"}}