{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T17:16:15Z","timestamp":1780334175059,"version":"3.54.1"},"reference-count":64,"publisher":"American Association for the Advancement of Science (AAAS)","issue":"83","content-domain":{"domain":["www.science.org"],"crossmark-restriction":true},"short-container-title":["Sci. Robot."],"published-print":{"date-parts":[[2023,10,25]]},"abstract":"<jats:p>One challenge to achieving widespread success of augmentative exoskeletons is accurately adjusting the controller to provide cooperative assistance with their wearer. Often, the controller parameters are \u201ctuned\u201d to optimize a physiological or biomechanical objective. However, these approaches are resource intensive, while typically only enabling optimization of a single objective. In reality, the exoskeleton user experience is likely derived from many factors, including comfort, fatigue, and stability, among others. This work introduces an approach to conveniently tune the four parameters of an exoskeleton controller to maximize user preference. Our overarching strategy is to leverage the wearer to internally balance the experiential factors of wearing the system. We used an evolutionary algorithm to recommend potential parameters, which were ranked by a neural network that was pretrained with previously collected user preference data. The controller parameters that had the highest preference ranking were provided to the exoskeleton, and the wearer responded with real-time feedback as a forced-choice comparison. Our approach was able to converge on controller parameters preferred by the wearer with an accuracy of 88% on average when compared with randomly generated parameters. User-preferred settings stabilized in 43 \u00b1 7 queries. This work demonstrates that user preference can be leveraged to tune a partial-assist ankle exoskeleton in real time using a simple, intuitive interface, highlighting the potential for translating lower-limb wearable technologies into our daily lives.<\/jats:p>","DOI":"10.1126\/scirobotics.adg3705","type":"journal-article","created":{"date-parts":[[2023,10,18]],"date-time":"2023-10-18T17:58:18Z","timestamp":1697651898000},"update-policy":"https:\/\/doi.org\/10.34133\/aaas_crossmark","source":"Crossref","is-referenced-by-count":30,"title":["User preference optimization for control of ankle exoskeletons using sample efficient active learning"],"prefix":"10.1126","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5655-9354","authenticated-orcid":true,"given":"Ung Hee","family":"Lee","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, University of Michigan, 2350 Hayward, Ann Arbor, MI 48109, USA."},{"name":"Department of Robotics, University of Michigan, 2505 Hayward, Ann Arbor, MI 48109, USA."},{"name":"X, the Moonshot Factory, 100 Mayfield Ave., Mountain View, CA 94043, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9411-9827","authenticated-orcid":true,"given":"Varun S.","family":"Shetty","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, University of Michigan, 2350 Hayward, Ann Arbor, MI 48109, USA."},{"name":"Department of Robotics, University of Michigan, 2505 Hayward, Ann Arbor, MI 48109, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7736-9023","authenticated-orcid":true,"given":"Patrick W.","family":"Franks","sequence":"additional","affiliation":[{"name":"X, the Moonshot Factory, 100 Mayfield Ave., Mountain View, CA 94043, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jie","family":"Tan","sequence":"additional","affiliation":[{"name":"Robotics at Google, 1600 Amphitheatre Parkway, Mountain View, CA 94043, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2240-1801","authenticated-orcid":true,"given":"Georgios","family":"Evangelopoulos","sequence":"additional","affiliation":[{"name":"X, the Moonshot Factory, 100 Mayfield Ave., Mountain View, CA 94043, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1972-328X","authenticated-orcid":true,"given":"Sehoon","family":"Ha","sequence":"additional","affiliation":[{"name":"Robotics at Google, 1600 Amphitheatre Parkway, Mountain View, CA 94043, USA."},{"name":"Georgia Institute of Technology, 85 Fifth Street NW, Atlanta, GA 30308, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3880-1527","authenticated-orcid":true,"given":"Elliott J.","family":"Rouse","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, University of Michigan, 2350 Hayward, Ann Arbor, MI 48109, USA."},{"name":"Department of Robotics, University of Michigan, 2505 Hayward, Ann Arbor, MI 48109, USA."},{"name":"X, the Moonshot Factory, 100 Mayfield Ave., Mountain View, CA 94043, USA."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"221","reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNSRE.2016.2521160"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41551-020-00619-3"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12984-020-00663-9"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1152\/japplphysiol.01133.2014"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","unstructured":"J. Ye Y.\u00a0Nakashima B.\u00a0Zhang Y.\u00a0Kobayashi M.\u00a0G.\u00a0Fujie Functional electrical stimulation based on a pelvis support robot for gait rehabilitation of hemiplegic patients after stroke in 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (IEEE 2014) pp.\u00a03098\u20133101.","DOI":"10.1109\/EMBC.2014.6944278"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1097\/01241398-199705000-00022"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.3109\/03093649309164360"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.2106\/00004623-197658010-00007"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.gaitpost.2008.10.062"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2019.2955740"},{"key":"e_1_3_2_12_2","doi-asserted-by":"crossref","unstructured":"K. Seo J.\u00a0Lee Y.\u00a0Lee T.\u00a0Ha Y.\u00a0Shim Fully autonomous hip exoskeleton saves metabolic cost of walking in 2016 IEEE International Conference on Robotics and Automation (ICRA) (IEEE 2016) pp.\u00a04628\u20134635.","DOI":"10.1109\/ICRA.2016.7487663"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCS.2018.2866605"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1186\/1743-0003-12-1"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2008.915453"},{"key":"e_1_3_2_16_2","unstructured":"J.\u00a0R.\u00a0Koller D.\u00a0H.\u00a0Gates D.\u00a0P.\u00a0Ferris C.\u00a0D.\u00a0Remy \u2018Body-in-the-loop\u2019 optimization of assistive robotic devices: A validation study in Robotics: Science and Systems (RSS 2016) pp.\u00a01\u201310."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aal5054"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.aar5438"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNSRE.2021.3074154"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abj3487"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2019.2928514"},{"key":"e_1_3_2_22_2","doi-asserted-by":"crossref","unstructured":"K. Stewart C.\u00a0Diduch J.\u00a0Sensinger Assistive exoskeleton control with user-tuned multi-objective optimization in 2019 IEEE 16th International Conference on Rehabilitation Robotics (ICORR) (IEEE 2019) pp.\u00a0554\u2013558.","DOI":"10.1109\/ICORR.2019.8779386"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3144790"},{"key":"e_1_3_2_24_2","doi-asserted-by":"crossref","unstructured":"M. Tucker E.\u00a0Novoseller C.\u00a0Kann Y.\u00a0Sui Y.\u00a0Yue J.\u00a0W.\u00a0Burdick A.\u00a0D.\u00a0Ames Preferencebased learning for exoskeleton gait optimization in 2020 IEEE International Conference on Robotics and Automation (ICRA) (IEEE 2020) pp.\u00a02351\u20132357.","DOI":"10.1109\/ICRA40945.2020.9196661"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNSRE.2020.3033711"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cobme.2023.100487"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12984-021-00916-1"},{"key":"e_1_3_2_28_2","unstructured":"B. Settles \u201cActive learning literature survey \u201d (Tech. Rep. TR1648 University of Wisconsin-Madison 2009)."},{"key":"e_1_3_2_29_2","first-page":"1","article-title":"A survey of preference-based reinforcement learning methods","volume":"18","author":"Wirth C.","year":"2017","unstructured":"C.\u00a0Wirth, R.\u00a0Akrour, G.\u00a0Neumann, J.\u00a0F\u00fcrnkranz, A survey of preference-based reinforcement learning methods. J. Mach. Learn. Res. 18, 1\u201346 (2017).","journal-title":"J. Mach. Learn. Res."},{"key":"e_1_3_2_30_2","article-title":"Deep reinforcement learning from human preferences","volume":"30","author":"Christiano P. F.","year":"2017","unstructured":"P. F.\u00a0Christiano, J.\u00a0Leike, T.\u00a0Brown, M.\u00a0Martic, S.\u00a0Legg, D.\u00a0Amodei, Deep reinforcement learning from human preferences. Adv. Neural Inf. Process. Syst. 30, 10.48550\/arXiv.1706.03741, (2017).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_31_2","doi-asserted-by":"crossref","unstructured":"D. Sadigh A.\u00a0D.\u00a0Dragan S.\u00a0Sastry S.\u00a0A.\u00a0Seshia Active Preference-based Learning of Reward Functions (UC Berkeley 2017).","DOI":"10.15607\/RSS.2017.XIII.053"},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","unstructured":"E. B\u0131y\u0131k N.\u00a0Huynh M.\u00a0J.\u00a0Kochenderfer D.\u00a0Sadigh Active preference-based gaussian process regression for reward learning. arXiv:2005.02575 [cs.RO] (2020).","DOI":"10.15607\/RSS.2020.XVI.041"},{"key":"e_1_3_2_33_2","doi-asserted-by":"crossref","unstructured":"M. Tucker N.\u00a0Csomay-Shanklin W.-L. Ma A.\u00a0D.\u00a0Ames Preference-based learning for user-guided hzd gait generation on bipedal walking robots in 2021 IEEE International Conference on Robotics and Automation (ICRA) (IEEE 2021) pp.\u00a02804\u20132810.","DOI":"10.1109\/ICRA48506.2021.9561515"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.robot.2017.04.013"},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","unstructured":"M. Tucker M.\u00a0Cheng E.\u00a0Novoseller R.\u00a0Cheng Y.\u00a0Yue J.\u00a0W.\u00a0Burdick A.\u00a0D.\u00a0Ames Human preference-based learning for high-dimensional optimization of exoskeleton walking gaits in 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE 2020) pp.\u00a03423\u20133430.","DOI":"10.1109\/IROS45743.2020.9341416"},{"key":"e_1_3_2_36_2","doi-asserted-by":"crossref","unstructured":"K. Li M.\u00a0Tucker E.\u00a0B\u0131y\u0131k E.\u00a0Novoseller J.\u00a0W.\u00a0Burdick Y.\u00a0Sui D.\u00a0Sadigh Y.\u00a0Yue A.\u00a0D.\u00a0Ames Roial: Region of interest active learning for characterizing exoskeleton gait preference landscapes in 2021 IEEE International Conference on Robotics and Automation (ICRA) (IEEE 2021) pp.\u00a03212\u20133218.","DOI":"10.1109\/ICRA48506.2021.9560840"},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","unstructured":"C. Burges T.\u00a0Shaked E.\u00a0Renshaw A.\u00a0Lazier M.\u00a0Deeds N.\u00a0Hamilton G.\u00a0Hullender Learning to rank using gradient descent in Proceedings of the 22nd International Conference on Machine Learning (ICML 2005) pp.\u00a089\u201396.","DOI":"10.1145\/1102351.1102363"},{"key":"e_1_3_2_38_2","unstructured":"N. Hansen The CMA evolution strategy: A tutorial. arXiv:1604.00772 [cs.LG] (4 April 2016)."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1162\/evco.2007.15.1.1"},{"key":"e_1_3_2_40_2","doi-asserted-by":"crossref","unstructured":"D.\u00a0V.\u00a0Arnold N.\u00a0Hansen A (1+ 1)-cma-es for constrained optimisation in Proceedings of the 14th Annual Conference on Genetic and Evolutionary Computation (ACM 2012) pp.\u00a0297\u2013304.","DOI":"10.1145\/2330163.2330207"},{"key":"e_1_3_2_41_2","doi-asserted-by":"crossref","unstructured":"G. Lv H.\u00a0Xing J.\u00a0Lin R.\u00a0D.\u00a0Gregg C.\u00a0G.\u00a0Atkeson A task-invariant learning framework of lower-limb exoskeletons for assisting human locomotion in 2020 American Control Conference (ACC) (IEEE 2020) pp.\u00a0569\u2013576.","DOI":"10.23919\/ACC45564.2020.9147915"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1126\/scirobotics.abf1078"},{"key":"e_1_3_2_43_2","doi-asserted-by":"crossref","unstructured":"S.\u00a0S.\u00a0Stevens Psychophysics: Introduction to its perceptual neural and social prospects (Routledge 2017).","DOI":"10.4324\/9781315127675"},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","unstructured":"X. Peng Y.\u00a0Acosta-Sojo M.\u00a0I.\u00a0Wu L.\u00a0Stirling Perception of powered ankle exoskeleton actuation timing during walking: A pilot study in 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) (IEEE 2021) pp.\u00a04654\u20134657.","DOI":"10.1109\/EMBC46164.2021.9629925"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1611835114"},{"key":"e_1_3_2_46_2","unstructured":"C. Finn P.\u00a0Abbeel S.\u00a0Levine Model-agnostic meta-learning for fast adaptation of deep networks in International Conference on Machine Learning (PMLR 2017) pp.\u00a01126\u20131135."},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2020.2974685"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1162\/106365601750190398"},{"key":"e_1_3_2_49_2","doi-asserted-by":"crossref","unstructured":"C. Basu Q.\u00a0Yang D.\u00a0Hungerman M.\u00a0Sinahal A.\u00a0D.\u00a0Draqan Do you want your autonomous car to drive like you? in 2017 12th ACM\/IEEE International Conference on Human-Robot Interaction (HRI) (IEEE 2017) pp.\u00a0417\u2013425.","DOI":"10.1145\/2909824.3020250"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1145\/2094072.2094078"},{"key":"e_1_3_2_51_2","unstructured":"E. Baum F.\u00a0Wilczek Supervised learning of probability distributions by neural networks in Neural Information Processing Systems (NIPS 1987)."},{"key":"e_1_3_2_52_2","first-page":"324","article-title":"Rank analysis of incomplete block designs: I.\u00a0The method of paired comparisons","volume":"39","author":"Bradley R. A.","year":"1952","unstructured":"R. A.\u00a0Bradley, M. E.\u00a0Terry, Rank analysis of incomplete block designs: I.\u00a0The method of paired comparisons. Biometrika 39, 324\u2013345 (1952).","journal-title":"Biometrika"},{"key":"e_1_3_2_53_2","unstructured":"D.\u00a0P.\u00a0Kingma J.\u00a0Ba Adam: A method for stochastic optimization in 3rd International Conference on Learning Representations ICLR 2015 Conference Track Proceedings (ICLR 2015) pp.\u00a01\u201315."},{"key":"e_1_3_2_54_2","unstructured":"E. Bakshy L.\u00a0Dworkin B.\u00a0Karrer K.\u00a0Kashin B.\u00a0Letham A.\u00a0Murthy S.\u00a0Singh AE: A domain-agnostic platform for adaptive experimentation in Neural Information Processing Systems Workshops: Systems for ML (NeurIPS 2018)."},{"key":"e_1_3_2_55_2","doi-asserted-by":"crossref","unstructured":"R. Akrour M.\u00a0Schoenauer M.\u00a0Sebag Preference-based policy learning in Joint European Conference on Machine Learning and Knowledge Discovery in Databases (Springer 2011) pp.\u00a012\u201327.","DOI":"10.1007\/978-3-642-23780-5_11"},{"key":"e_1_3_2_56_2","doi-asserted-by":"crossref","unstructured":"T. Joachims Optimizing search engines using clickthrough data in Proceedings of the Eighth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (ACM 2002) pp.\u00a0133\u2013142.","DOI":"10.1145\/775047.775067"},{"key":"e_1_3_2_57_2","doi-asserted-by":"crossref","unstructured":"M.-F. Tsai T.-Y. Liu T.\u00a0Qin H.-H. Chen W.-Y. Ma Frank: A ranking method with fidelity loss in Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (ACM 2007) pp.\u00a0383\u2013390.","DOI":"10.1145\/1277741.1277808"},{"key":"e_1_3_2_58_2","doi-asserted-by":"crossref","unstructured":"S. Parthasarathy R.\u00a0Lotfian C.\u00a0Busso Ranking emotional attributes with deep neural networks in 2017 IEEE International Conference on Acoustics Speech and Signal Processing (ICASSP) (IEEE 2017) pp.\u00a04995\u20134999.","DOI":"10.1109\/ICASSP.2017.7953107"},{"key":"e_1_3_2_59_2","unstructured":"J. Wakunda A.\u00a0Zell A new selection scheme for steady-state evolution strategies in Proceedings of the 2nd Annual Conference on Genetic and Evolutionary Computation (ACM 2000) pp.\u00a0794\u2013801."},{"key":"e_1_3_2_60_2","doi-asserted-by":"crossref","unstructured":"J.-F. Duval H.\u00a0M.\u00a0Herr Flexsea: Flexible scalable electronics architecture for wearable robotic applications in 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob) (IEEE 2016) pp.\u00a01236\u20131241.","DOI":"10.1109\/BIOROB.2016.7523800"},{"key":"e_1_3_2_61_2","doi-asserted-by":"crossref","unstructured":"U.\u00a0H.\u00a0Lee C.-W. Pan E.\u00a0J.\u00a0Rouse Empirical characterization of a high-performance exterior-rotor type brushless dc motor and drive in 2019 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE 2019) pp.\u00a08018\u20138025.","DOI":"10.1109\/IROS40897.2019.8967626"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/LRA.2022.3147565"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-022-07736-w"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbiomech.2006.12.006"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.gaitpost.2013.01.029"}],"container-title":["Science Robotics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.science.org\/doi\/pdf\/10.1126\/scirobotics.adg3705","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,31]],"date-time":"2024-10-31T02:28:29Z","timestamp":1730341709000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.science.org\/doi\/10.1126\/scirobotics.adg3705"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,25]]},"references-count":64,"journal-issue":{"issue":"83","published-print":{"date-parts":[[2023,10,25]]}},"alternative-id":["10.1126\/scirobotics.adg3705"],"URL":"https:\/\/doi.org\/10.1126\/scirobotics.adg3705","relation":{},"ISSN":["2470-9476"],"issn-type":[{"value":"2470-9476","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,25]]},"assertion":[{"value":"2022-12-19","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-20","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"eadg3705"}}