{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T04:24:39Z","timestamp":1783484679914,"version":"3.55.0"},"reference-count":66,"publisher":"SAGE Publications","issue":"6-7","license":[{"start":{"date-parts":[[2021,4,6]],"date-time":"2021-04-06T00:00:00Z","timestamp":1617667200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2021,6]]},"abstract":"<jats:p>The objective of this work is to augment the basic abilities of a robot by learning to use sensorimotor primitives to solve complex long-horizon manipulation problems. This requires flexible generative planning that can combine primitive abilities in novel combinations and, thus, generalize across a wide variety of problems. In order to plan with primitive actions, we must have models of the actions: under what circumstances will executing this primitive successfully achieve some particular effect in the world? We use, and develop novel improvements to, state-of-the-art methods for active learning and sampling. We use Gaussian process methods for learning the constraints on skill effectiveness from small numbers of expensive-to-collect training examples. In addition, we develop efficient adaptive sampling methods for generating a comprehensive and diverse sequence of continuous candidate control parameter values (such as pouring waypoints for a cup) during planning. These values become end-effector goals for traditional motion planners that then solve for a full robot motion that performs the skill. By using learning and planning methods in conjunction, we take advantage of the strengths of each and plan for a wide variety of complex dynamic manipulation tasks. We demonstrate our approach in an integrated system, combining traditional robotics primitives with our newly learned models using an efficient robot task and motion planner. We evaluate our approach both in simulation and in the real world through measuring the quality of the selected primitive actions. Finally, we apply our integrated system to a variety of long-horizon simulated and real-world manipulation problems.<\/jats:p>","DOI":"10.1177\/02783649211004615","type":"journal-article","created":{"date-parts":[[2021,4,7]],"date-time":"2021-04-07T02:35:06Z","timestamp":1617762906000},"page":"866-894","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":63,"title":["Learning compositional models of robot skills for task and motion planning"],"prefix":"10.1177","volume":"40","author":[{"given":"Zi","family":"Wang","sequence":"first","affiliation":[{"name":"MIT CSAIL, MA, USA"},{"name":"Present address: Google, Mountain View, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6474-1276","authenticated-orcid":false,"given":"Caelan Reed","family":"Garrett","sequence":"additional","affiliation":[{"name":"MIT CSAIL, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6054-7145","authenticated-orcid":false,"given":"Leslie Pack","family":"Kaelbling","sequence":"additional","affiliation":[{"name":"MIT CSAIL, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8657-2450","authenticated-orcid":false,"given":"Tom\u00e1s","family":"Lozano-P\u00e9rez","sequence":"additional","affiliation":[{"name":"MIT CSAIL, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2021,4,6]]},"reference":[{"key":"bibr1-02783649211004615","first-page":"265","author":"Abadi M","year":"2016","journal-title":"12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16)"},{"key":"bibr2-02783649211004615","author":"Affandi RH","year":"2014","journal-title":"International Conference on Machine Learning (ICML)"},{"key":"bibr3-02783649211004615","author":"Bogunovic I","year":"2016","journal-title":"Advances in Neural Information Processing Systems (NeurIPS)"},{"key":"bibr4-02783649211004615","author":"Brandi S","year":"2014","journal-title":"Humanoids"},{"key":"bibr5-02783649211004615","author":"Bryan B","year":"2006","journal-title":"Advances in Neural Information Processing Systems (NeurIPS)"},{"key":"bibr6-02783649211004615","author":"Calli B","year":"2015","journal-title":"IEEE International Conference on Advanced Robotics (ICAR)"},{"key":"bibr7-02783649211004615","unstructured":"Catto E (2011) Box2D, A 2D Physics Engine for Games. Available at: http:\/\/box2d.org (accessed 10 March 2021)."},{"key":"bibr8-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487165"},{"key":"bibr9-02783649211004615","unstructured":"Coumans E, Bai Y (2016\u20132019) PyBullet, a python module for physics simulation for games, robotics and machine learning. Available at: http:\/\/pybullet.org (accessed 10 March 2021)."},{"key":"bibr10-02783649211004615","volume-title":"Automated Construction of Robotic Manipulation Programs","author":"Diankov R","year":"2010"},{"key":"bibr11-02783649211004615","unstructured":"Diankov R, Kuffner J (2008) OpenRAVE: A Planning Architecture for Autonomous Robotics. Technical Report CMU-RI-TR-08-34, Robotics Institute, Carnegie Mellon University."},{"key":"bibr12-02783649211004615","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2016.XII.033"},{"key":"bibr13-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1145\/358669.358692"},{"key":"bibr14-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1006\/game.1999.0740"},{"key":"bibr15-02783649211004615","author":"Gal Y","year":"2016","journal-title":"International Conference on Machine Learning (ICML)"},{"key":"bibr16-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-control-091420-084139"},{"key":"bibr17-02783649211004615","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2017.XIII.039"},{"key":"bibr18-02783649211004615","author":"Garrett CR","year":"2020","journal-title":"International Conference on Automated Planning and Scheduling (ICAPS)"},{"key":"bibr19-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1177\/0278364918802962"},{"key":"bibr20-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9196681"},{"key":"bibr21-02783649211004615","volume-title":"The Quaternion Bingham Distribution, 3D Object Detection, and Dynamic Manipulation","author":"Glover J","year":"2014"},{"key":"bibr22-02783649211004615","author":"Gotovos A","year":"2013","journal-title":"International Conference on Artificial Intelligence (IJCAI)"},{"key":"bibr23-02783649211004615","author":"Hafiz Affandi R","year":"2013","journal-title":"Advances in Neural Information Processing Systems (NeurIPS)"},{"key":"bibr24-02783649211004615","author":"Hermans T","year":"2013","journal-title":"Humanoids"},{"key":"bibr25-02783649211004615","author":"Huang J","year":"2017","journal-title":"IEEE Conference on Computer Vision and Pattern Recognition (CVPR)"},{"key":"bibr26-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5980391"},{"key":"bibr27-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913484072"},{"key":"bibr28-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989109"},{"key":"bibr29-02783649211004615","author":"Kapoor A","year":"2007","journal-title":"International Conference on Computer Vision (ICCV)"},{"key":"bibr30-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989327"},{"key":"bibr31-02783649211004615","author":"Kim B","year":"2018","journal-title":"AAAI Conference on Artificial Intelligence"},{"key":"bibr32-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1613\/jair.5575"},{"key":"bibr33-02783649211004615","author":"Kroemer O","year":"2016","journal-title":"International Symposium on Experimental Robotics (ISER)"},{"key":"bibr34-02783649211004615","author":"Kroemer O","year":"2016","journal-title":"Humanoids"},{"key":"bibr35-02783649211004615","author":"Kuffner JJ","year":"2000","journal-title":"IEEE International Conference on Robotics and Automation (ICRA)"},{"key":"bibr36-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1561\/2200000044"},{"key":"bibr37-02783649211004615","author":"Lakshminarayanan B","year":"2017","journal-title":"Advances in Neural Information Processing Systems (NeurIPS)"},{"key":"bibr38-02783649211004615","author":"Lin TY","year":"2014","journal-title":"European Conference on Computer Vision (ECCV)"},{"key":"bibr39-02783649211004615","unstructured":"McDermott D, Ghallab M, Howe A, et al. (1998) PDDL: The Planning Domain Definition Language. Technical report, Yale Center for Computational Vision and Control."},{"key":"bibr40-02783649211004615","volume-title":"Bayesian Learning for Neural Networks (Leacture Notes in Statistics","volume":"118","author":"Neal RM","year":"2012"},{"key":"bibr41-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1177\/0278364911421039"},{"key":"bibr42-02783649211004615","author":"OpenAI Akkaya I","year":"2019","journal-title":"arXiv preprint arXiv:1910.07113"},{"key":"bibr43-02783649211004615","author":"Pan Z","year":"2016","journal-title":"International Conference on Automated Planning and Scheduling (ICAPS)"},{"key":"bibr44-02783649211004615","first-page":"2825","volume":"12","author":"Pedregosa F","year":"2011","journal-title":"Journal of Machine Learning Research"},{"key":"bibr45-02783649211004615","author":"Quigley M","year":"2009","journal-title":"IEEE International Conference on Robotics and Automation (ICRA) Workshop on Open-Source Software"},{"key":"bibr46-02783649211004615","volume-title":"Gaussian Processes for Machine Learning","author":"Rasmussen CE","year":"2006"},{"key":"bibr47-02783649211004615","first-page":"91","author":"Ren S","year":"2015","journal-title":"Advances in Neural Information Processing Systems (NeurIPS)"},{"key":"bibr48-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5980567"},{"key":"bibr49-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2017.7989307"},{"key":"bibr50-02783649211004615","author":"Schenck C","year":"2017","journal-title":"Conference on Robot Learning (CoRL)"},{"key":"bibr51-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-23461-8_9"},{"key":"bibr52-02783649211004615","unstructured":"Sheffield Machine Learning Software (2012) GPy: A Gaussian process framework in Python. Available at: http:\/\/github.com\/SheffieldML\/GPy (accessed 10 March 2021)."},{"key":"bibr53-02783649211004615","author":"Srinivas N","year":"2010","journal-title":"International Conference on Machine Learning (ICML)"},{"key":"bibr54-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6906922"},{"key":"bibr55-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2010.2044949"},{"key":"bibr56-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(99)00052-1"},{"key":"bibr57-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2011.07.004"},{"key":"bibr58-02783649211004615","author":"Vasudevan S","year":"2009","journal-title":"IEEE International Conference on Robotics and Automation (ICRA)"},{"key":"bibr59-02783649211004615","author":"Wang AS","year":"2019","journal-title":"IEEE International Conference on Robotics and Automation (ICRA)"},{"key":"bibr60-02783649211004615","author":"Wang Z","year":"2018","journal-title":"The IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)"},{"key":"bibr61-02783649211004615","author":"Wang Z","year":"2016","journal-title":"International Conference on Artificial Intelligence and Statistics (AISTATS)"},{"key":"bibr62-02783649211004615","author":"Wipf DP","year":"2008","journal-title":"Advances in Neural Information Processing Systems (NeurIPS)"},{"key":"bibr63-02783649211004615","author":"Xia V","year":"2019","journal-title":"International Conference on Learning Representations (ICLR)"},{"key":"bibr64-02783649211004615","author":"Yamaguchi A","year":"2016","journal-title":"Humanoids"},{"key":"bibr65-02783649211004615","doi-asserted-by":"publisher","DOI":"10.1109\/HUMANOIDS.2014.7041472"},{"key":"bibr66-02783649211004615","author":"Yao Z","year":"2005","journal-title":"The IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS)"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649211004615","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/02783649211004615","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/02783649211004615","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:16:48Z","timestamp":1777457808000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/02783649211004615"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,6]]},"references-count":66,"journal-issue":{"issue":"6-7","published-print":{"date-parts":[[2021,6]]}},"alternative-id":["10.1177\/02783649211004615"],"URL":"https:\/\/doi.org\/10.1177\/02783649211004615","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,6]]}}}