{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T16:09:32Z","timestamp":1775146172332,"version":"3.50.1"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,6,14]],"date-time":"2024-06-14T00:00:00Z","timestamp":1718323200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Hum.-Robot Interact."],"published-print":{"date-parts":[[2024,6,30]]},"abstract":"<jats:p>\n            Assistive robot arms try to help their users perform everyday tasks. One way robots can provide this assistance is\n            <jats:italic>shared autonomy<\/jats:italic>\n            . Within shared autonomy, both the human and robot maintain control over the robot\u2019s motion: as the robot becomes confident it understands what the human wants, it intervenes to automate the task. But how does the robot know these tasks in the first place? State-of-the-art approaches to shared autonomy often rely on prior knowledge. For instance, the robot may need to know the human\u2019s potential goals beforehand. During long-term interaction these methods will inevitably break down\u2014sooner or later the human will attempt to perform a task that the robot does not expect. Accordingly, in this article we formulate an alternate approach to shared autonomy that learns assistance from scratch. Our insight is that operators\n            <jats:italic>repeat<\/jats:italic>\n            important tasks on a daily basis (e.g., opening the fridge, making coffee). Instead of relying on prior knowledge, we therefore take advantage of these repeated interactions to learn assistive policies. We introduce SARI, an algorithm that\n            <jats:italic>recognizes<\/jats:italic>\n            the human\u2019s task,\n            <jats:italic>replicates<\/jats:italic>\n            similar demonstrations, and\n            <jats:italic>returns<\/jats:italic>\n            control when unsure. We then combine learning with control to demonstrate that the error of our approach is uniformly ultimately bounded. We perform simulations to support this error bound, compare our approach to imitation learning baselines, and explore its capacity to assist for an increasing number of tasks. Finally, we conduct three user studies with industry-standard methods and shared autonomy baselines, including a pilot test with a disabled user. Our results indicate that learning shared autonomy across repeated interactions matches existing approaches for known tasks and outperforms baselines on new tasks. See videos of our user studies here:\n            <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/youtu.be\/3vE4omSvLvc\">https:\/\/youtu.be\/3vE4omSvLvc<\/jats:ext-link>\n            .\n          <\/jats:p>","DOI":"10.1145\/3651994","type":"journal-article","created":{"date-parts":[[2024,3,8]],"date-time":"2024-03-08T09:19:01Z","timestamp":1709889541000},"page":"1-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["SARI: Shared Autonomy across Repeated Interaction"],"prefix":"10.1145","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0711-2051","authenticated-orcid":false,"given":"Ananth","family":"Jonnavittula","sequence":"first","affiliation":[{"name":"Virginia Tech, Blacksburg, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3160-6879","authenticated-orcid":false,"given":"Shaunak A.","family":"Mehta","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8787-5293","authenticated-orcid":false,"given":"Dylan P.","family":"Losey","sequence":"additional","affiliation":[{"name":"Virginia Tech, Blacksburg, USA"}]}],"member":"320","published-online":{"date-parts":[[2024,6,14]]},"reference":[{"key":"e_1_3_3_2_2","unstructured":"Evan Ackerman. 2019.Jaco Is a Low-power Robot Arm That Hooks to Your Wheelchair. IEEESpectrum"},{"key":"e_1_3_3_3_2","unstructured":"2021.Jaco Assistive Robot User Guide. Retrieved February 9 2023 from https:\/\/assistive.kinovarobotics.com\/uploads\/EN-UG-007-Jaco-user-guide-R05.pdf"},{"key":"e_1_3_3_4_2","unstructured":"Henny Admoni and Siddhartha Srinivasa. 2016. Predicting user intent through eye gaze for shared autonomy. In Association for the Advancement of Artificial Intelligence (AAAI) Fall Symposium."},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-control-061417-041727"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3171221.3171287"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9812332"},{"key":"e_1_3_3_8_2","first-page":"181","volume-title":"ACM\/IEEE International Conference on Human-robot Interaction","author":"Bhattacharjee Tapomayukh","year":"2020","unstructured":"Tapomayukh Bhattacharjee, Ethan K. Gordon, Rosario Scalise, Maria E. Cabrera, Anat Caspi, Maya Cakmak, and Siddhartha S. Srinivasa. 2020. Is more autonomy always better? Exploring preferences of users with mobility impairments in robot-assisted feeding. In ACM\/IEEE International Conference on Human-robot Interaction. 181\u2013190."},{"key":"e_1_3_3_9_2","first-page":"915","volume-title":"Uncertainty in Artificial Intelligence","author":"Bragg Jonathan","year":"2020","unstructured":"Jonathan Bragg and Emma Brunskill. 2020. Fake it till you make it: Learning-compatible performance support. In Uncertainty in Artificial Intelligence. 915\u2013924."},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364920921935"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/HRI.2019.8673192"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1097\/PHM.0b013e3181cf569b"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICORR.2009.5209484"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2972852"},{"key":"e_1_3_3_15_2","unstructured":"Erwin Coumans and Yunfei Bai. 2016\u20132021. PyBullet a Python Module for Physics Simulation for Games Robotics and Machine Learning. http:\/\/pybullet.org"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913490324"},{"key":"e_1_3_3_17_2","first-page":"4560","article-title":"AvE: Assistance via empowerment","volume":"33","author":"Du Yuqing","year":"2020","unstructured":"Yuqing Du, Stas Tiomkin, Emre Kiciman, Daniel Polani, Pieter Abbeel, and Anca Dragan. 2020. AvE: Assistance via empowerment. Advances in Neural Information Processing Systems 33 (2020), 4560\u20134571.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA40945.2020.9197411"},{"key":"e_1_3_3_19_2","volume-title":"International Symposium on Robotics Research","author":"Feng Ryan","year":"2019","unstructured":"Ryan Feng, Youngsun Kim, Gilwoo Lee, Ethan K. Gordon, Matt Schmittle, Shivaum Kumar, Tapomayukh Bhattacharjee, and Siddhartha S. Srinivasa. 2019. Robot-assisted feeding: Generalizing skewering strategies across food items on a realistic plate. In International Symposium on Robotics Research."},{"key":"e_1_3_3_20_2","first-page":"49","volume-title":"International Conference on Machine Learning","author":"Finn Chelsea","year":"2016","unstructured":"Chelsea Finn, Sergey Levine, and Pieter Abbeel. 2016. Guided cost learning: Deep inverse optimal control via policy optimization. In International Conference on Machine Learning. 49\u201358."},{"key":"e_1_3_3_21_2","volume-title":"Robotics: Science and Systems","author":"Fontaine Matthew","year":"2020","unstructured":"Matthew Fontaine and Stefanos Nikolaidis. 2020. A quality diversity approach to automatically generating human-robot interaction scenarios in shared autonomy. In Robotics: Science and Systems."},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.3389\/fnbot.2021.647930"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10710-017-9314-z"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2016.2593928"},{"key":"e_1_3_3_25_2","first-page":"1861","volume-title":"International Conference on Machine Learning","author":"Haarnoja Tuomas","year":"2018","unstructured":"Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. 2018. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International Conference on Machine Learning. PMLR, 1861\u20131870."},{"key":"e_1_3_3_26_2","article-title":"Learning representations that enable generalization in assistive tasks","author":"He Jerry Zhi-Yang","year":"2022","unstructured":"Jerry Zhi-Yang He, Aditi Raghunathan, Daniel S. Brown, Zackory Erickson, and Anca D. Dragan. 2022. Learning representations that enable generalization in assistive tasks. arXiv preprint arXiv:2212.03175 (2022).","journal-title":"arXiv preprint arXiv:2212.03175"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.5555\/2906831.2906839"},{"key":"e_1_3_3_28_2","first-page":"598","volume-title":"Conference on Robot Learning","author":"Hoque Ryan","year":"2022","unstructured":"Ryan Hoque, Ashwin Balakrishna, Ellen Novoseller, Albert Wilcox, Daniel S. Brown, and Ken Goldberg. 2022. ThriftyDAgger: Budget-aware novelty and risk gating for interactive imitation learning. In Conference on Robot Learning. 598\u2013608."},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3359614"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICORR.2015.7281253"},{"key":"e_1_3_3_31_2","first-page":"682","volume-title":"Conference on Robot Learning","author":"Jauhri Snehal","year":"2021","unstructured":"Snehal Jauhri, Carlos Celemin, and Jens Kober. 2021. Interactive imitation learning in state-space. In Conference on Robot Learning. 682\u2013692."},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.1177\/0278364918776060"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2020.XVI.011"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9562048"},{"key":"e_1_3_3_35_2","first-page":"1851","volume-title":"IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Jonnavittula Ananth","year":"2021","unstructured":"Ananth Jonnavittula and Dylan P. Losey. 2021. Learning to share autonomy across repeated interaction. In IEEE\/RSJ International Conference on Intelligent Robots and Systems. 1851\u20131858."},{"key":"e_1_3_3_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9811674"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-015-9459-7"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793698"},{"key":"e_1_3_3_39_2","volume-title":"Nonlinear Systems","author":"Khalil Hassan K.","year":"2002","unstructured":"Hassan K. Khalil. 2002. Nonlinear Systems. Vol. 3. Prentice-Hall."},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-021-10005-w"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2017.2765335"},{"key":"e_1_3_3_42_2","first-page":"1113","volume-title":"Conference on Robot Learning","author":"Lynch Corey","year":"2020","unstructured":"Corey Lynch, Mohi Khansari, Ted Xiao, Vikash Kumar, Jonathan Tompson, Sergey Levine, and Pierre Sermanet. 2020. Learning latent plans from play. In Conference on Robot Learning. 1113\u20131132."},{"key":"e_1_3_3_43_2","article-title":"Human-in-the-loop imitation learning using remote teleoperation","author":"Mandlekar Ajay","year":"2020","unstructured":"Ajay Mandlekar, Danfei Xu, Roberto Mart\u00edn-Mart\u00edn, Yuke Zhu, Li Fei-Fei, and Silvio Savarese. 2020. Human-in-the-loop imitation learning using remote teleoperation. arXiv preprint arXiv:2012.06733 (2020).","journal-title":"arXiv preprint arXiv:2012.06733"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/RoboSoft55895.2023.10122030"},{"key":"e_1_3_3_45_2","article-title":"Unified learning from demonstrations, corrections, and preferences during physical human-robot interaction","author":"Mehta Shaunak A.","year":"2022","unstructured":"Shaunak A. Mehta and Dylan P. Losey. 2022. Unified learning from demonstrations, corrections, and preferences during physical human-robot interaction. arXiv preprint arXiv:2207.03395 (2022).","journal-title":"arXiv preprint arXiv:2207.03395"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA46639.2022.9812230"},{"key":"e_1_3_3_47_2","article-title":"Dropoutdagger: A Bayesian approach to safe imitation learning","author":"Menda Kunal","year":"2017","unstructured":"Kunal Menda, Katherine Driggs-Campbell, and Mykel J. Kochenderfer. 2017. Dropoutdagger: A Bayesian approach to safe imitation learning. arXiv preprint arXiv:1709.06166 (2017).","journal-title":"arXiv preprint arXiv:1709.06166"},{"key":"e_1_3_3_48_2","first-page":"5041","volume-title":"IEEE\/RSJ International Conference on Intelligent Robots and Systems","author":"Menda Kunal","year":"2019","unstructured":"Kunal Menda, Katherine Driggs-Campbell, and Mykel J. Kochenderfer. 2019. Ensembledagger: A Bayesian approach to safe imitation learning. In IEEE\/RSJ International Conference on Intelligent Robots and Systems. IEEE, 5041\u20135048."},{"key":"e_1_3_3_49_2","volume-title":"Robotics: Science and Systems","author":"Muelling Katharina","year":"2015","unstructured":"Katharina Muelling, Arun Venkatraman, Jean-Sebastien Valois, John Downey, Jeffrey Weiss, Shervin Javdani, Martial Hebert, Andrew B. Schwartz, Jennifer L. Collinger, and J. Andrew Bagnell. 2015. Autonomy infused teleoperation with application to BCI manipulation. In Robotics: Science and Systems."},{"key":"e_1_3_3_50_2","article-title":"Harmonic: A multimodal dataset of assistive human-robot collaboration","author":"Newman Benjamin A.","year":"2018","unstructured":"Benjamin A. Newman, Reuben M. Aronson, Siddartha S. Srinivasa, Kris Kitani, and Henny Admoni. 2018. Harmonic: A multimodal dataset of assistive human-robot collaboration. arXiv preprint arXiv:1807.11154 (2018).","journal-title":"arXiv preprint arXiv:1807.11154"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/2909824.3020252"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1561\/2300000053"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2019.103344"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3142732"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9560758"},{"key":"e_1_3_3_56_2","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2018.XIV.005"},{"key":"e_1_3_3_57_2","first-page":"627","volume-title":"International Conference on Artificial Intelligence and Statistics","author":"Ross St\u00e9phane","year":"2011","unstructured":"St\u00e9phane Ross, Geoffrey Gordon, and Drew Bagnell. 2011. A reduction of imitation learning and structured prediction to no-regret online learning. In International Conference on Artificial Intelligence and Statistics. 627\u2013635."},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2020.XVI.072"},{"key":"e_1_3_3_59_2","volume-title":"Probability and Statistics","author":"Schervish Mark J.","year":"2014","unstructured":"Mark J. Schervish and Morris H. DeGroot. 2014. Probability and Statistics. Pearson Education."},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2021.3100603"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-021-10006-9"},{"key":"e_1_3_3_62_2","volume-title":"Robot Modeling and Control","author":"Spong Mark W.","year":"2006","unstructured":"Mark W. Spong, Seth Hutchinson, and Mathukumalli Vidyasagar. 2006. Robot Modeling and Control. Vol. 3. Wiley, New York."},{"key":"e_1_3_3_63_2","volume-title":"Americans with Disabilities: 2014","author":"Taylor Danielle M.","year":"2018","unstructured":"Danielle M. Taylor. 2018. Americans with Disabilities: 2014. US Census Bureau."},{"key":"e_1_3_3_64_2","volume-title":"AAAI","author":"Zhang Jiakai","year":"2017","unstructured":"Jiakai Zhang and Kyunghyun Cho. 2017. Query-efficient imitation learning for end-to-end autonomous driving. In AAAI."},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA48506.2021.9561839"}],"container-title":["ACM Transactions on Human-Robot Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3651994","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3651994","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T17:49:13Z","timestamp":1750268953000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3651994"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,14]]},"references-count":64,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,6,30]]}},"alternative-id":["10.1145\/3651994"],"URL":"https:\/\/doi.org\/10.1145\/3651994","relation":{},"ISSN":["2573-9522"],"issn-type":[{"value":"2573-9522","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,14]]},"assertion":[{"value":"2023-02-10","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-15","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}