{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,16]],"date-time":"2026-06-16T17:21:51Z","timestamp":1781630511458,"version":"3.54.5"},"reference-count":33,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,9,14]],"date-time":"2023-09-14T00:00:00Z","timestamp":1694649600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Neurorobot."],"abstract":"<jats:p>In this paper, we propose a deep reinforcement learning-based framework that enables adaptive and continuous control of a robot to push unseen objects from random positions to the target position. Our approach takes into account contact information in the design of the reward function, resulting in improved success rates, generalization for unseen objects, and task efficiency compared to policies that do not consider contact information. Through reinforcement learning using only one object in simulation, we obtain a learned policy for manipulating a single object, which demonstrates good generalization when applied to the task of pushing unseen objects. Finally, we validate the effectiveness of our approach in real-world scenarios.<\/jats:p>","DOI":"10.3389\/fnbot.2023.1271607","type":"journal-article","created":{"date-parts":[[2023,9,14]],"date-time":"2023-09-14T13:20:58Z","timestamp":1694697658000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["Learning adaptive reaching and pushing skills using contact information"],"prefix":"10.3389","volume":"17","author":[{"given":"Shuaijun","family":"Wang","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lining","family":"Sun","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fusheng","family":"Zha","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wei","family":"Guo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pengfei","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,9,14]]},"reference":[{"key":"B1","unstructured":"\u201cA probabilistic data-driven model for planar pushing,\u201d30083015\n            BauzaM.\n            RodriguezA.\n          New York, NYIEEE2017 IEEE International Conference on Robotics and Automation (ICRA)2017"},{"key":"B2","unstructured":"\u201cA data-efficient approach to precise and controlled pushing,\u201d336345\n            BauzaM.\n            HoganF. R.\n            RodriguezA.\n          Conference on Robot Learning2018"},{"key":"B3","doi-asserted-by":"publisher","first-page":"2703","DOI":"10.1109\/TRO.2022.3153785","article-title":"Object rearrangement through planar pushing: a theoretical analysis and validation","volume":"38","author":"Chai","year":"2022","journal-title":"IEEE Transact. Robot"},{"key":"B4","doi-asserted-by":"publisher","first-page":"829437","DOI":"10.3389\/fnbot.2022.829437","article-title":"Reinforcement learning with vision-proprioception model for robot planar pushing","volume":"16","author":"Cong","year":"2022","journal-title":"Front. Neurorobot"},{"key":"B5","unstructured":"CoumansE.\n            BaiY.\n          Pybullet, a python module for physics simulation for games, robotics and machine learning. 2016"},{"key":"B6","doi-asserted-by":"crossref","first-page":"1116","DOI":"10.1109\/IROS47612.2022.9981873","article-title":"\u201cLearning goal-oriented non-prehensile pushing in cluttered scenes,\u201d","volume-title":"2022 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Dengler","year":"2022"},{"key":"B7","doi-asserted-by":"crossref","first-page":"2123","DOI":"10.1109\/IROS.2010.5652970","article-title":"\u201cPush-grasping with dexterous hands: mechanics and a method,\u201d","volume-title":"2010 IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Dogar","year":"2010"},{"key":"B8","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1007\/978-3-030-28619-4_32","article-title":"\u201cLearning to singulate objects using a push proposal network,\u201d","volume-title":"Robotics Research: The 18th International Symposium ISRR","author":"Eitel","year":"2020"},{"key":"B9","doi-asserted-by":"publisher","first-page":"102517","DOI":"10.1016\/j.rcim.2022.102517","article-title":"A review on reinforcement learning for contact-rich robotic manipulation tasks","volume":"81","author":"Elguea-Aguinaco","year":"2023","journal-title":"Robot. Comp. Integr. Manuf"},{"key":"B10","unstructured":"\u201cSoft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,\u201d18611870\n            HaarnojaT.\n            ZhouA.\n            AbbeelP.\n            LevineS.\n          International Conference on Machine Learning2018"},{"key":"B11","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1109\/ICRA.2018.8461175","article-title":"\u201cReactive planar manipulation with convex hybrid mpc,\u201d","volume-title":"2018 IEEE International Conference on Robotics and Automation (ICRA)","author":"Hogan","year":"2018"},{"key":"B12","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1177\/027836499601500603","article-title":"Practical force-motion models for sliding manipulation","volume":"15","author":"Howe","year":"1996","journal-title":"Int. J. Robot. Res"},{"key":"B13","unstructured":"\u201cSe (2)-equivariant pushing dynamics models for tabletop object manipulations,\u201d427436\n            KimS.\n            LimB.\n            LeeY.\n            ParkF. C.\n          Conference on Robot Learning2023"},{"key":"B14","unstructured":"\u201cGraph inverse reinforcement learning from diverse videos,\u201d5566\n            KumarS.\n            ZamoraJ.\n            HansenN.\n            JangirR.\n            WangX.\n          Conference on Robot Learning2023"},{"key":"B15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.15607\/RSS.2018.XIV.024","article-title":"Push-net: deep planar pushing for objects with unknown physical properties","volume":"14","author":"Li","year":"2018","journal-title":"Robot. Sci. Syst"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1201","DOI":"10.1109\/TRO.2021.3104471","article-title":"Goal-driven robotic pushing using tactile and proprioceptive feedback","volume":"38","author":"Lloyd","year":"2021","journal-title":"IEEE Transact. Robot"},{"key":"B17","first-page":"186","article-title":"\u201cEstimating the friction parameters of pushed objects,\u201d","volume-title":"Proceedings of 1993 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS'93), Vol. 1","author":"Lynch","year":"1993"},{"key":"B18","doi-asserted-by":"publisher","first-page":"533","DOI":"10.1177\/027836499601500602","article-title":"Stable pushing: mechanics, controllability, and planning","volume":"15","author":"Lynch","year":"1996","journal-title":"Int. J. Robot. Res"},{"key":"B19","doi-asserted-by":"crossref","first-page":"9537","DOI":"10.1109\/IROS45743.2020.9341112","article-title":"\u201cEstimating an object's inertial parameters by robotic pushing: A data-driven approach,\u201d","volume-title":"2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Mavrakis","year":"2020"},{"key":"B20","doi-asserted-by":"crossref","first-page":"3400","DOI":"10.1109\/ICRA.2011.5979561","article-title":"\u201cApriltag: a robust and flexible visual fiducial system,\u201d","volume-title":"2011 IEEE International Conference on Robotics and Automation","author":"Olson","year":"2011"},{"key":"B21","doi-asserted-by":"crossref","first-page":"10584","DOI":"10.1109\/ICRA40945.2020.9197295","article-title":"\u201cPredicting pushing action effects on spatial object relations by learning internal prediction models,\u201d","volume-title":"2020 IEEE International Conference on Robotics and Automation (ICRA)","author":"Paus","year":"2020"},{"key":"B22","first-page":"12348","article-title":"Stable-baselines3: Reliable reinforcement learning implementations","volume":"22","author":"Raffin","year":"2021","journal-title":"J. Mach. Learn. Res"},{"key":"B23","doi-asserted-by":"crossref","first-page":"4066","DOI":"10.1109\/SMC42975.2020.9282951","article-title":"\u201cLearning continuous control actions for robotic grasping with reinforcement learning,\u201d","volume-title":"2020 IEEE International Conference on Systems, Man, and Cybernetics (SMC)","author":"Shahid","year":"2020"},{"key":"B24","doi-asserted-by":"publisher","first-page":"945","DOI":"10.1007\/s10462-021-09997-9","article-title":"Reinforcement learning in robotic applications: a comprehensive survey","volume":"55","author":"Singh","year":"2022","journal-title":"Artif. Intell. Rev"},{"key":"B25","unstructured":"\u201cIdentifying mechanical models of unknown objects with differentiable physics simulations,\u201d749760\n            SongC.\n            BoulariasA.\n          Learning for Dynamics and Control2020"},{"key":"B26","doi-asserted-by":"publisher","first-page":"8","DOI":"10.3389\/frobt.2020.00008","article-title":"Let's push things forward: a survey on robot pushing","volume":"7","author":"St\u00fcber","year":"2020","journal-title":"Front. Robot. AI"},{"key":"B27","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1109\/70.34770","article-title":"A new technique for fully autonomous and efficient 3 d robotics hand\/eye calibration","volume":"5","author":"Tsai","year":"1989","journal-title":"IEEE Transact. Robot. Automat"},{"key":"B28","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1109\/IROS51168.2021.9636836","article-title":"\u201cCocoi: contact-aware online context inference for generalizable non-planar pushing,\u201d","volume-title":"2021 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)","author":"Xu","year":"2021"},{"key":"B29","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1109\/IROS.1991.174510","article-title":"\u201cIdentification of the center of friction from pushing an object by a mobile robot,\u201d","volume-title":"Proceedings IROS'91: IEEE\/RSJ International Workshop on Intelligent Robots and Systems' 91","author":"Yoshikawa","year":"1991"},{"key":"B30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1109\/TNNLS.2023.3244186","article-title":"A novel robotic pushing and grasping method based on vision transformer and convolution","author":"Yu","year":"2023","journal-title":"IEEE Transact. Neural Netw. Learn. Syst"},{"key":"B31","first-page":"4238","author":"Zeng","year":"2018"},{"key":"B32","doi-asserted-by":"publisher","first-page":"1477","DOI":"10.1177\/0278364919872532","article-title":"Pushing revisited: differential flatness, trajectory planning, and stabilization","volume":"38","author":"Zhou","year":"2019","journal-title":"Int. J. Robot. Res"},{"key":"B33","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/451","article-title":"Fast model identification via physics engines for data-efficient policy search","author":"Zhu","year":"2017","journal-title":"arXiv"}],"container-title":["Frontiers in Neurorobotics"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1271607\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,14]],"date-time":"2023-09-14T13:21:16Z","timestamp":1694697676000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fnbot.2023.1271607\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,14]]},"references-count":33,"alternative-id":["10.3389\/fnbot.2023.1271607"],"URL":"https:\/\/doi.org\/10.3389\/fnbot.2023.1271607","relation":{},"ISSN":["1662-5218"],"issn-type":[{"value":"1662-5218","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,14]]},"article-number":"1271607"}}