{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T21:06:42Z","timestamp":1740172002894,"version":"3.37.3"},"reference-count":29,"publisher":"American Institute of Aeronautics and Astronautics (AIAA)","issue":"7","license":[{"start":{"date-parts":[[2022,5,21]],"date-time":"2022-05-21T00:00:00Z","timestamp":1653091200000},"content-version":"am","delay-in-days":324,"URL":"https:\/\/www.aiaa.org\/userlicenses\/1.0\/#CompEndUserLicense"}],"funder":[{"DOI":"10.13039\/100000185","name":"Defense Advanced Research Projects Agency","doi-asserted-by":"publisher","award":["HR0011-18-2-0043"],"award-info":[{"award-number":["HR0011-18-2-0043"]}],"id":[{"id":"10.13039\/100000185","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000104","name":"National Aeronautics and Space Administration","doi-asserted-by":"publisher","award":["NNX15AQ14H"],"award-info":[{"award-number":["NNX15AQ14H"]}],"id":[{"id":"10.13039\/100000104","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["arc.aiaa.org"],"crossmark-restriction":true},"short-container-title":["Journal of Aerospace Information Systems"],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:p> This work addresses the iterated nonstationary assistant selection problem, in which over the course of repeated interactions on a mission, an autonomous robot experiencing a fault must select a single human from among a group of assistants to restore it to operation. The assistants in our problem have a level of performance that changes as a function of their experience solving the problem. Our approach uses reinforcement learning via a multi-arm bandit formulation to learn about the capabilities of each potential human assistant and decide which human to task. This study, which is built on our past work, evaluates the potential for a Gaussian-process-based machine learning method to effectively model the complex dynamics associated with human learning and forgetting. Application of our method in simulation shows that our method is capable of tracking performance of human-like dynamics for learning and forgetting. Using a novel selection policy called the proficiency window, it is shown that our technique can outperform baseline selection strategies while providing guarantees on human use. Our work offers an effective potential alternative to dedicated human supervisors, with application to any human\u2013robot system where a set of humans is responsible for overseeing autonomous robot operations. <\/jats:p>","DOI":"10.2514\/1.i010921","type":"journal-article","created":{"date-parts":[[2021,5,21]],"date-time":"2021-05-21T11:22:01Z","timestamp":1621596121000},"page":"429-441","update-policy":"https:\/\/doi.org\/10.2514\/aiaa_crossmarkpolicy","source":"Crossref","is-referenced-by-count":2,"title":["Human-Aware Reinforcement Learning for Fault Recovery Using Contextual Gaussian Processes"],"prefix":"10.2514","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4650-7950","authenticated-orcid":false,"given":"Steve","family":"McGuire","sequence":"first","affiliation":[{"name":"University of California Santa Cruz, Santa Cruz, California 95064"}]},{"given":"P.","family":"Michael Furlong","sequence":"additional","affiliation":[{"name":"University of Waterloo, Waterloo, Ontario N2L 3G1, Canada"}]},{"given":"Christoffer","family":"Heckman","sequence":"additional","affiliation":[{"name":"University of Colorado at Boulder, Boulder, Colorado 80309"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4380-137X","authenticated-orcid":false,"given":"Simon","family":"Julier","sequence":"additional","affiliation":[{"name":"University College London, London WC1E 6BT, England, United Kingdom"}]},{"given":"Nisar","family":"Ahmed","sequence":"additional","affiliation":[{"name":"University of Colorado at Boulder, Boulder, Colorado 80303"}]}],"member":"1387","reference":[{"key":"r1","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2006.1638015"},{"key":"r2","unstructured":"RabideauG.BenowitzE. \u201cPrototyping an Onboard Scheduler for the Mars 2020 Rover,\u201d 10th International Workshop on Planning and Scheduling for Space (IWPSS 2017), Jet Propulsion Lab., National Aeronautics and Space Administration, Pasadena, CA, June 2017, http:\/\/hdl.handle.net\/2014\/47716 [retrieved 15 June 2017]."},{"key":"r3","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2018.2801468"},{"key":"r4","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2019.2894381"},{"key":"r8","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-015-9460-1"},{"key":"r9","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2004.1310942"},{"key":"r10","doi-asserted-by":"publisher","DOI":"10.2514\/1.I010307"},{"key":"r11","doi-asserted-by":"publisher","DOI":"10.1016\/S0921-8890(02)00378-0"},{"key":"r12","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2010.126"},{"key":"r13","doi-asserted-by":"publisher","DOI":"10.5898\/JHRI.3.1.Johnson"},{"key":"r14","doi-asserted-by":"publisher","DOI":"10.1002\/nav.3800020109"},{"key":"r15","doi-asserted-by":"publisher","DOI":"10.1109\/TRA.2002.803462"},{"key":"r16","doi-asserted-by":"publisher","DOI":"10.1287\/inte.20.4.133"},{"key":"r17","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCB.2012.2210212"},{"key":"r18","doi-asserted-by":"publisher","DOI":"10.2514\/1.50671"},{"key":"r23","doi-asserted-by":"publisher","DOI":"10.1214\/13-AOS1119"},{"key":"r25","doi-asserted-by":"publisher","DOI":"10.1016\/j.cor.2016.12.026"},{"key":"r26","doi-asserted-by":"publisher","DOI":"10.1037\/10011-000"},{"key":"r27","doi-asserted-by":"publisher","DOI":"10.1002\/cne.920180503"},{"key":"r28","doi-asserted-by":"publisher","DOI":"10.1037\/0278-7393.11.3.414"},{"volume-title":"FAA-H-8083-9A, Handbook, Aviation Instructor\u2019s","year":"2008","key":"r29"},{"key":"r30","doi-asserted-by":"publisher","DOI":"10.1037\/h0029790"},{"key":"r31","volume-title":"Human Learning: From Learning Curves to Learning Organizations","volume":"29","author":"Dar-El E. M.","year":"2013","edition":"1"},{"key":"r32","doi-asserted-by":"publisher","DOI":"10.2514\/8.155"},{"issue":"9","key":"r33","first-page":"40","volume":"8","author":"Carlson J. G.","year":"1976","journal-title":"Industrial Engineering"},{"issue":"1","key":"r34","first-page":"23","volume":"12","author":"Garg A.","year":"1961","journal-title":"Journal of Industrial Engineering"},{"key":"r39","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/3206.001.0001"},{"key":"r42","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-05318-5_9"},{"key":"r44","unstructured":"McGuireS. \u201cAutonomous On-line Learning of Assistant Selection Policies for Fault Recovery,\u201d Ph.D. Thesis, Univ. of Colorado, Boulder, CO, 2019."}],"container-title":["Journal of Aerospace Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/arc.aiaa.org\/doi\/am-pdf\/10.2514\/1.I010921","content-type":"application\/pdf","content-version":"am","intended-application":"unspecified"},{"URL":"https:\/\/arc.aiaa.org\/doi\/pdf\/10.2514\/1.I010921","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/arc.aiaa.org\/doi\/pdf\/10.2514\/1.I010921","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,14]],"date-time":"2023-07-14T17:19:02Z","timestamp":1689355142000},"score":1,"resource":{"primary":{"URL":"https:\/\/arc.aiaa.org\/doi\/10.2514\/1.I010921"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7]]},"references-count":29,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["10.2514\/1.I010921"],"URL":"https:\/\/doi.org\/10.2514\/1.i010921","relation":{},"ISSN":["1940-3151","2327-3097"],"issn-type":[{"type":"print","value":"1940-3151"},{"type":"electronic","value":"2327-3097"}],"subject":[],"published":{"date-parts":[[2021,7]]},"assertion":[{"value":"2020-10-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-02-24","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-04-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-05-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}