{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T09:53:07Z","timestamp":1777715587179,"version":"3.51.4"},"reference-count":39,"publisher":"SAGE Publications","issue":"5-7","license":[{"start":{"date-parts":[[2017,5,8]],"date-time":"2017-05-08T00:00:00Z","timestamp":1494201600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"CPS: Synergy: Collaborative Research: Mutually Stabilized Correction in Physical Demonstration","award":["NSF CNS 1329891"],"award-info":[{"award-number":["NSF CNS 1329891"]}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2017,6]]},"abstract":"<jats:p>We propose a generalizable natural language interface that allows users to provide corrective instructions to an assistive robotic manipulator in real-time. This work is motivated by the desire to improve collaboration between humans and robots in a home environment. Allowing human operators to modify properties of how their robotic counterpart achieves a goal on-the-fly increases the utility of the system by incorporating the strengths of the human partner (e.g. visual acuity and environmental knowledge). This work is applicable to users with and without disability. Our natural language interface is based on the distributed correspondence graph, a probabilistic graphical model that assigns semantic meaning to user utterances in the context of the robot\u2019s environment and current behavior. We then use the desired corrections to alter the behavior of the robotic manipulator by treating the modifications as constraints on the motion generation (planning) paradigm. In this paper, we highlight four dimensions along which a user may wish to correct the behavior of his or her assistive manipulator. We develop our language model using data collected from Amazon Mechanical Turk to capture a comprehensive sample of terminology that people use to describe desired corrections. We then develop an end-to-end system using open-source speech-to-text software and a Kinova Robotics MICO robotic arm. To demonstrate the efficacy of our approach, we run a pilot study with users unfamiliar with robotic systems and analyze points of failure and future directions.<\/jats:p>","DOI":"10.1177\/0278364917706418","type":"journal-article","created":{"date-parts":[[2017,5,9]],"date-time":"2017-05-09T01:35:01Z","timestamp":1494293701000},"page":"684-698","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":21,"title":["Real-time natural language corrections for assistive robotic manipulators"],"prefix":"10.1177","volume":"36","author":[{"given":"Alexander","family":"Broad","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Northwestern University, USA"},{"name":"Rehabilitation Institute of Chicago, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jacob","family":"Arkin","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Rochester, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nathan","family":"Ratliff","sequence":"additional","affiliation":[{"name":"Lula Robotics Inc., USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Thomas","family":"Howard","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Rochester, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Brenna","family":"Argall","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering and Computer Science, Northwestern University, USA"},{"name":"Rehabilitation Institute of Chicago, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2017,5,8]]},"reference":[{"key":"bibr1-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139396"},{"key":"bibr2-0278364917706418","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1231"},{"key":"bibr3-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2010.07.002"},{"key":"bibr4-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1007\/s12369-012-0156-9"},{"key":"bibr5-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/DEVLRN.2010.5578872"},{"key":"bibr6-0278364917706418","volume-title":"Proceedings of the SPIE 9837, unmanned systems technology XVIII","author":"Barber D","year":"2016"},{"key":"bibr7-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1023\/B:AURO.0000016864.12513.77"},{"key":"bibr8-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2016.7759412"},{"issue":"3","key":"bibr9-0278364917706418","first-page":"217","volume":"36","author":"Busnel M","year":"1999","journal-title":"Journal of Rehabilitation Research and Development"},{"key":"bibr10-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-23778-7_25"},{"key":"bibr11-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1016\/0167-2789(90)90087-6"},{"key":"bibr12-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139984"},{"key":"bibr13-0278364917706418","volume-title":"IEEE\/RSJ international conference on intelligent robots and system (IROS) workshop on rehabilitation and assistive robotics","author":"Howard TM","year":"2014"},{"key":"bibr14-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2014.6907841"},{"key":"bibr15-0278364917706418","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2015.XI.031"},{"key":"bibr16-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139281"},{"key":"bibr17-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/ICORR.2009.5209527"},{"key":"bibr18-0278364917706418","first-page":"849","volume-title":"Proceedings of advances in neural information processing systems","author":"Kober J","year":"2009"},{"key":"bibr19-0278364917706418","doi-asserted-by":"publisher","DOI":"10.3390\/robotics2030122"},{"key":"bibr20-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/ICORR.2007.4428505"},{"key":"bibr21-0278364917706418","first-page":"1475","volume-title":"Proceedings of the national conference on artificial intelligence (AAAI)","author":"MacMahon M","year":"2006"},{"key":"bibr22-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2010.5509439"},{"key":"bibr23-0278364917706418","volume-title":"Proceedings of the European conference on mobile robots","author":"Mandel C","year":"2007"},{"key":"bibr24-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-00065-7_28"},{"key":"bibr25-0278364917706418","volume-title":"Proceedings of the national conference on artificial intelligence (AAAI)","author":"Mei H","year":"2016"},{"key":"bibr26-0278364917706418","first-page":"3111","author":"Mikolov T","year":"2013","journal-title":"Proceedings of advances in neural information processing systems"},{"key":"bibr27-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1177\/0278364914554471"},{"key":"bibr28-0278364917706418","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2016.XII.037"},{"key":"bibr29-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139778"},{"key":"bibr30-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1145\/1322192.1322206"},{"key":"bibr31-0278364917706418","first-page":"1488","volume-title":"Proceedings of international joint conference on artificial intelligence (IJCAI)","volume":"9","author":"Shimizu N","year":"2009"},{"key":"bibr32-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2010.5650464"},{"key":"bibr33-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v32i4.2384"},{"key":"bibr34-0278364917706418","first-page":"1507","volume-title":"Proceedings of AAAI conference on artificial intelligence, North America","author":"Tellex S","year":"2011"},{"key":"bibr35-0278364917706418","volume-title":"Proceedings of robotics: Science and systems","author":"Tellex S","year":"2013"},{"key":"bibr36-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2015.7354021"},{"key":"bibr37-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/ICORR.2007.4428489"},{"key":"bibr38-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/ICORR.2005.1501160"},{"key":"bibr39-0278364917706418","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2016.7759238"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364917706418","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0278364917706418","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364917706418","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:15:35Z","timestamp":1777457735000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0278364917706418"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,5,8]]},"references-count":39,"journal-issue":{"issue":"5-7","published-print":{"date-parts":[[2017,6]]}},"alternative-id":["10.1177\/0278364917706418"],"URL":"https:\/\/doi.org\/10.1177\/0278364917706418","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,5,8]]}}}