{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T05:30:34Z","timestamp":1784179834425,"version":"3.55.0"},"reference-count":63,"publisher":"SAGE Publications","issue":"10","license":[{"start":{"date-parts":[[2015,5,26]],"date-time":"2015-05-26T00:00:00Z","timestamp":1432598400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2015,9]]},"abstract":"<jats:p>We consider the problem of learning preferences over trajectories for mobile manipulators such as personal robots and assembly line robots. The preferences we learn are more intricate than simple geometric constraints on trajectories; they are rather governed by the surrounding context of various objects and human interactions in the environment. We propose a coactive online learning framework for teaching preferences in contextually rich environments. The key novelty of our approach lies in the type of feedback expected from the user: the human user does not need to demonstrate optimal trajectories as training data, but merely needs to iteratively provide trajectories that slightly improve over the trajectory currently proposed by the system. We argue that this coactive preference feedback can be more easily elicited than demonstrations of optimal trajectories. Nevertheless, theoretical regret bounds of our algorithm match the asymptotic rates of optimal trajectory algorithms. We implement our algorithm on two high-degree-of-freedom robots, PR2 and Baxter, and present three intuitive mechanisms for providing such incremental feedback. In our experimental evaluation we consider two context rich settings, household chores and grocery store checkout, and show that users are able to train the robot with just a few feedbacks (taking only a few minutes).<\/jats:p>","DOI":"10.1177\/0278364915581193","type":"journal-article","created":{"date-parts":[[2015,5,27]],"date-time":"2015-05-27T09:44:44Z","timestamp":1432719884000},"page":"1296-1313","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":64,"title":["Learning preferences for manipulation tasks from online coactive feedback"],"prefix":"10.1177","volume":"34","author":[{"given":"Ashesh","family":"Jain","sequence":"first","affiliation":[{"name":"Department of Computer Science, Cornell University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shikhar","family":"Sharma","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Cornell University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thorsten","family":"Joachims","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Cornell University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ashutosh","family":"Saxena","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Cornell University, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2015,5,26]]},"reference":[{"key":"bibr1-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1177\/0278364910371999"},{"key":"bibr2-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-012-0160-0"},{"key":"bibr3-0278364915581193","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2007.III.030"},{"key":"bibr4-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2008.10.024"},{"key":"bibr5-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2012.6224742"},{"key":"bibr6-0278364915581193","volume-title":"Proceedings of robotics: science and systems","author":"Berg JVD","year":"2010"},{"key":"bibr7-0278364915581193","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2011.VII.002"},{"key":"bibr8-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2002.1045668"},{"key":"bibr9-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2006.886952"},{"key":"bibr10-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2010.5509685"},{"key":"bibr11-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2006.282100"},{"key":"bibr12-0278364915581193","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2012.VIII.007"},{"key":"bibr13-0278364915581193","unstructured":"Diankov R (2010) Automated Construction of Robotic Manipulation Programs. PhD Thesis, CMU, RI."},{"key":"bibr14-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/HRI.2013.6483603"},{"key":"bibr15-0278364915581193","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2013.IX.024"},{"key":"bibr16-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2009.5152773"},{"key":"bibr17-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2006.282390"},{"key":"bibr18-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2011.943230"},{"key":"bibr19-0278364915581193","first-page":"281","volume-title":"Robotics Research (Springer Tracts in Advanced Robotics","volume":"66","author":"Green CJ","year":"2011"},{"key":"bibr20-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2010.2049527"},{"key":"bibr21-0278364915581193","volume-title":"Proceedings of the international symposium on robotics research","author":"Jain A","year":"2013"},{"key":"bibr22-0278364915581193","volume-title":"Advances in Neural Information Processing Systems","author":"Jain A","year":"2013"},{"key":"bibr23-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1177\/0278364912438781"},{"key":"bibr24-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1145\/1150402.1150429"},{"key":"bibr25-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-009-5108-8"},{"key":"bibr26-0278364915581193","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2010.VI.034"},{"key":"bibr27-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1177\/0278364911406761"},{"key":"bibr28-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-33765-9_15"},{"key":"bibr29-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5980287"},{"key":"bibr30-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-010-5223-6"},{"key":"bibr31-0278364915581193","volume-title":"Advances in Neural Information Processing Systems","author":"Koppula H","year":"2011"},{"key":"bibr32-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6907442"},{"key":"bibr33-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1177\/02783640122067453"},{"key":"bibr34-0278364915581193","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2013.IX.012"},{"key":"bibr35-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/TRA.2003.819732"},{"key":"bibr36-0278364915581193","volume-title":"Proceedings of the international conference on machine learning","author":"Levine S","year":"2012"},{"key":"bibr37-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5980048"},{"key":"bibr38-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511809071"},{"key":"bibr39-0278364915581193","unstructured":"Nikolaidis S, Shah J (2012) Human-robot teaming using shared mental models. In: HRI, Workshop on human\u2013agent\u2013robot teamwork."},{"key":"bibr40-0278364915581193","doi-asserted-by":"crossref","unstructured":"Nikolaidis S, Shah J (2013) Human-robot cross-training: Computational formulation, modeling and evaluation of a human team training strategy. In: IEEE\/ACM ICHRI.","DOI":"10.1109\/HRI.2013.6483499"},{"key":"bibr41-0278364915581193","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2012.VIII.043"},{"key":"bibr42-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-40991-2_9"},{"key":"bibr43-0278364915581193","doi-asserted-by":"crossref","unstructured":"Ratliff N (2009) Learning to Search: Structured Prediction Techniques for Imitation Learning. PhD Thesis, CMU, RI.","DOI":"10.1007\/s10514-009-9121-3"},{"key":"bibr44-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1145\/1143844.1143936"},{"key":"bibr45-0278364915581193","volume-title":"Proceedings of the international conference on artificial intelligence and statistics","author":"Ratliff N","year":"2007"},{"key":"bibr46-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-009-9121-3"},{"key":"bibr47-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2009.5152817"},{"key":"bibr48-0278364915581193","volume-title":"Proceedings of the international conference on machine learning","author":"Ross S","year":"2013"},{"key":"bibr49-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1177\/0278364907087172"},{"key":"bibr50-0278364915581193","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2013.IX.031"},{"key":"bibr51-0278364915581193","volume-title":"Proceedings of the international conference on machine learning","author":"Shivaswamy P","year":"2012"},{"key":"bibr52-0278364915581193","volume-title":"Designing The User Interface: Strategies for Effective Human\u2013Computer Interaction","author":"Shneiderman B","year":"2010"},{"key":"bibr53-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1177\/0278364910369715"},{"key":"bibr54-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2007.4399486"},{"key":"bibr55-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2007.904911"},{"key":"bibr56-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/ROMAN.2001.981926"},{"key":"bibr57-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2012.2205651"},{"key":"bibr58-0278364915581193","volume-title":"Proceedings of the international conference on robotics and automation","author":"Tamane K","year":"2013"},{"key":"bibr59-0278364915581193","volume-title":"Advances in Neural Information Processing Systems","author":"Vernaza P","year":"2012"},{"key":"bibr60-0278364915581193","volume-title":"Advances in Neural Information Processing Systems","author":"Wilson A","year":"2012"},{"key":"bibr61-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2011.5979553"},{"key":"bibr62-0278364915581193","unstructured":"Ziebart BD, Maas A, Bagnell JA, Dey AK (2008) Maximum entropy inverse reinforcement learning. In: AAAI."},{"key":"bibr63-0278364915581193","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913488805"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364915581193","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0278364915581193","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364915581193","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:18:37Z","timestamp":1777457917000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0278364915581193"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,5,26]]},"references-count":63,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2015,9]]}},"alternative-id":["10.1177\/0278364915581193"],"URL":"https:\/\/doi.org\/10.1177\/0278364915581193","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,5,26]]}}}