{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T20:14:03Z","timestamp":1783455243429,"version":"3.55.0"},"reference-count":33,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T00:00:00Z","timestamp":1756425600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"},{"start":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T00:00:00Z","timestamp":1756425600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS\u20102310759"],"award-info":[{"award-number":["IIS\u20102310759"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS2416761"],"award-info":[{"award-number":["IIS2416761"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R21EB035378"],"award-info":[{"award-number":["R21EB035378"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["AI Magazine"],"published-print":{"date-parts":[[2025,9]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Ensuring that AI systems do what we, as humans, actually want them to do is one of the biggest open research challenges in AI alignment and safety. My research seeks to directly address this challenge by enabling AI systems to interact with humans to learn aligned and robust behaviors. The way robots and other AI systems behave is often the result of optimizing a reward function. However, manually designing good reward functions is highly challenging and error\u2010prone, even for domain experts. Although reward functions are often difficult to manually specify, human feedback in the form of demonstrations or preferences is often much easier to obtain but can be difficult to interpret due to ambiguity and noise. Thus, it is critical that AI systems take into account epistemic uncertainty over the human's true intent. As part of the AAAI New Faculty Highlight Program, I will give an overview of my research progress along the following fundamental research areas: (1) efficiently quantifying uncertainty over human intent, (2) directly optimizing behavior to be robust to uncertainty over human intent, and (3) actively querying for additional human input to reduce uncertainty over human\u00a0intent.<\/jats:p>","DOI":"10.1002\/aaai.70024","type":"journal-article","created":{"date-parts":[[2025,8,29]],"date-time":"2025-08-29T10:20:41Z","timestamp":1756462841000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Toward robust, interactive, and human\u2010aligned AI systems"],"prefix":"10.1002","volume":"46","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9570-1832","authenticated-orcid":false,"given":"Daniel S.","family":"Brown","sequence":"first","affiliation":[{"name":"Kahlert School of Computing University of Utah Salt Lake City Utah USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2025,8,29]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"crossref","unstructured":"Abbeel Pieter andAndrew Y.Ng.2004. \u201cApprenticeship Learning via Inverse Reinforcement Learning.\u201d InProceedings of the Twenty\u2010First International Conference on Machine Learning (ICML) 1\u20138.Banff Alberta Canada:ACM.","DOI":"10.1145\/1015330.1015430"},{"key":"e_1_2_8_3_1","unstructured":"Amodei Dario ChrisOlah JacobSteinhardt PaulChristiano JohnSchulman andDanMan\u00e9.2016. \u201cConcrete Problems in ai Safety.\u201darXiv preprint arXiv:1606.06565."},{"key":"e_1_2_8_4_1","doi-asserted-by":"crossref","unstructured":"Belsare Atharv ZohreKarimi ConnorMattson andDanielS Brown.2025. \u201cToward Zero\u2010Shot User Intent Recognition in Shared Autonomy.\u201d InInternational Conference on Human\u2010Robot Interaction (HRI).","DOI":"10.1109\/HRI61500.2025.10973985"},{"key":"e_1_2_8_5_1","unstructured":"Brown Daniel S. andScottNiekum.2017. \u201cToward Probabilistic Safety Bounds for Robot Learning From Demonstration.\u201d InAAAI Fall Symposium on AI for HRI AAAI."},{"key":"e_1_2_8_6_1","doi-asserted-by":"crossref","unstructured":"Brown Daniel S. andScottNiekum.2018. \u201cEfficient Probabilistic Performance Bounds for Inverse Reinforcement Learning.\u201d InProceedings of the AAAI Conference on Artificial Intelligence 5234\u20105241 New Orleans Louisiana:AAAI.","DOI":"10.1609\/aaai.v32i1.11755"},{"key":"e_1_2_8_7_1","unstructured":"Brown Daniel S. WonjoonGoo PrabhatNagarajan andScottNiekum.2019a. \u201cExtrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning From Observations.\u201d InInternational Conference on Machine Learning (ICML) 1109\u20131118.Long Beach California:PMLR."},{"key":"e_1_2_8_8_1","unstructured":"Brown Daniel S. WonjoonGoo andScottNiekum.2019b. \u201cBetter\u2010Than\u2010Demonstrator Imitation Learning via Automatically\u2010Ranked Demonstrations.\u201d InProceedings of the 3rd Conference on Robot Learning (CoRL) 330\u2013349.Zurich Switzerland:PMLR."},{"key":"e_1_2_8_9_1","unstructured":"Brown Daniel S. RussellColeman RaviSrinivasan andScottNiekum.2020a. \u201cSafe Imitation Learning via Fast Bayesian Reward Inference From Preferences.\u201d InInternational Conference on Machine Learning (ICML) 2142\u20132151.Vienna Austria:PMLR."},{"key":"e_1_2_8_10_1","first-page":"2962","volume-title":"Neural Information Processing Systems (NeurIPS)","author":"Brown Daniel S.","year":"2020"},{"key":"e_1_2_8_11_1","unstructured":"Brown Daniel S. JordanSchneider Anca D.Dragan andScottNiekum.2021. \u201cValue Alignment Verification.\u201d InProceedings of the 38th International Conference on Machine Learning (ICML) 1460\u20131470.PMLR."},{"key":"e_1_2_8_12_1","doi-asserted-by":"crossref","unstructured":"Chi Cheng SiyuanFeng YilunDu ZhenjiaXu EricCousineau BenjaminBurchfiel andShuranSong.2023. \u201cDiffusion Policy: Visuomotor Policy Learning via Action Diffusion.\u201d InProceedings of Robotics: Science and Systems (RSS) RSS Proceedings.","DOI":"10.15607\/RSS.2023.XIX.026"},{"key":"e_1_2_8_13_1","volume-title":"The Alignment Problem: How Can Machines Learn Human Values?","author":"Christian Brian","year":"2021"},{"key":"e_1_2_8_14_1","unstructured":"Florence Pete CoreyLynch AndyZeng OscarRamirez AyzaanWahid LauraDowns AdrianWong JohnnyLee IgorMordatch andJonathanTompson.2021. \u201cImplicit Behavioral Cloning.\u201d InConference on Robot Learning (CoRL) 1015\u20131030.London UK:PMLR."},{"key":"e_1_2_8_15_1","unstructured":"Zhi\u2010Yang He Jerry Daniel S.Brown ZackoryErickson andAncaDragan.2023. \u201cQuantifying Assistive Robustness via the Natural\u2010Adversarial Frontier.\u201d InConference on Robot Learning (CoRL) 1865\u20131886.PMLR."},{"key":"e_1_2_8_16_1","unstructured":"Hoque Ryan AshwinBalakrishna EllenNovoseller AlbertWilcox Daniel S.Brown andKenGoldberg.2021. \u201cThriftydagger: Budget\u2010Aware Novelty and Risk Gating for Interactive Imitation Learning.\u201d In5th Annual Conference on Robot Learning (CoRL) 1090\u20131101.London UK:PMLR."},{"key":"e_1_2_8_17_1","unstructured":"Javed Zaynah Daniel S.Brown SatvikSharma JerryZhu AshwinBalakrishna MarekPetrik Anca D.Dragan andKenGoldberg.2021. \u201cPolicy Gradient Bayesian Robust Optimization.\u201d InInternational Conference on Machine Learning (ICML) 4870\u20134880.PMLR."},{"key":"e_1_2_8_18_1","unstructured":"Kalra Akansha andDaniel S.Brown.2024. \u201cCan Differentiable Decision Trees Enable Interpretable Reward Learning From Human Feedback?.\u201d InReinforcement Learning Conference (RLC)."},{"key":"e_1_2_8_19_1","doi-asserted-by":"crossref","unstructured":"Karimi Zohre Shing\u2010HeiHo BaoThach AlanKuntz andDaniel S.Brown.2024. \u201cReward Learning From Suboptimal Demonstrations With Applications in Surgical Electrocautery.\u201d In2024 International Symposium on Medical Robotics (ISMR) 1\u20137.IEEE.","DOI":"10.1109\/ISMR63436.2024.10585785"},{"key":"e_1_2_8_20_1","unstructured":"Lee Seungjae YibinWang HarithejaEtukuru H. JinKim NurMuhammad MahiShafiullah andLerrelPinto.2024. \u201cBehavior Generation With Latent Actions.\u201d InProceedings of the 41st International Conference on Machine Learning (ICML)."},{"key":"e_1_2_8_21_1","unstructured":"Levine Sergey AviralKumar GeorgeTucker andJustinFu.2020. \u201cOffline Reinforcement Learning: Tutorial Review and Perspectives on Open Problems.\u201darXiv preprint arXiv:2005.01643."},{"key":"e_1_2_8_22_1","doi-asserted-by":"crossref","unstructured":"Liu Yi GauravDatta EllenNovoseller andDaniel S.Brown.2023. \u201cEfficient Preference\u2010Based Reinforcement Learning Using Learned Dynamics Models.\u201d In2023 IEEE International Conference on Robotics and Automation (ICRA) 2921\u20132928.IEEE.","DOI":"10.1109\/ICRA48891.2023.10161081"},{"key":"e_1_2_8_23_1","doi-asserted-by":"crossref","unstructured":"Mattson Connor andDanielS Brown.2023. \u201cLeveraging Human Feedback to Evolve and Discover Novel Emergent Behaviors in Robot Swarms.\u201d InProceedings of the Genetic and Evolutionary Computation Conference (GECCO) 56\u201364.Lisbon Portugal: ACM.","DOI":"10.1145\/3583131.3590443"},{"key":"e_1_2_8_24_1","doi-asserted-by":"crossref","unstructured":"Papadimitriou Dimitris andDaniel S.Brown.2024. \u201cBayesian Constraint Inference From User Demonstrations Based on Margin\u2010Respecting Preference Models.\u201d In2024 IEEE International Conference on Robotics and Automation (ICRA) 15039\u201315046.IEEE.","DOI":"10.1109\/ICRA57147.2024.10611095"},{"key":"e_1_2_8_25_1","unstructured":"Papadimitriou Dimitris UsmanAnwar andDaniel S.Brown.2022. \u201cBayesian Methods for Constraint Inference in Reinforcement Learning.\u201dTransactions on Machine Learning Research (TMLR)."},{"key":"e_1_2_8_26_1","unstructured":"Patil Basavasagar AkanshaKalra GuanhongTao andDaniel S.Brown.2025. \u201cHow vulnerable is my policy? Adversarial attacks on modern behavior cloning policies.\u201darXiv preprint arXiv:2502.03698."},{"key":"e_1_2_8_27_1","unstructured":"Ramachandran Deepak andEyalAmir.2007. \u201cBayesian Inverse Reinforcement Learning.\u201d InProceedings of the 20th International Joint Conference on Artificial Intelligence vol.7 2586\u20132591.Hyderabad India:IJCAI Organization."},{"key":"e_1_2_8_28_1","doi-asserted-by":"publisher","DOI":"10.1162\/daed_a_01899"},{"key":"e_1_2_8_29_1","unstructured":"Shin Daniel AncaDragan andDaniel S.Brown.2023. \u201cBenchmarks and Algorithms for Offline Preference\u2010Based Reward Learning.\u201dTransactions on Machine Learning Research (TMLR)."},{"key":"e_1_2_8_30_1","first-page":"1449","volume-title":"Advances in Neural Information Processing Systems (NeurIPS)","author":"Syed Umar","year":"2007"},{"key":"e_1_2_8_31_1","unstructured":"Thompson Jordan HaohanZhang andDanielS Brown.2023. \u201cTowards a Gaze\u2010Driven Assistive Neck Exoskeleton via Virtual Reality Data Collection.\u201d InHRI Workshop on Virtual Augmented and Mixed\u2010Reality for Human\u2010Robot Interactions."},{"key":"e_1_2_8_32_1","unstructured":"Thompson Jordan RonaldKoe AnthonyLe GabriellaGoodman Daniel S.Brown andAlanKuntz.2025. \u201cEarly Failure Detection in Autonomous Surgical Soft\u2010Tissue Manipulation via Uncertainty Quantification.\u201dRSS Workshop on Out\u2010of\u2010Distribution Generalization in Robotics."},{"key":"e_1_2_8_33_1","unstructured":"Tien Jeremy JerryZhi\u2010Yang He ZackoryErickson AncaDragan andDanielS Brown.2023. \u201cCausal Confusion and Reward Misidentification in Preference\u2010Based Reward Learning.\u201d InThe Eleventh International Conference on Learning Representations (ICLR)."},{"key":"e_1_2_8_34_1","doi-asserted-by":"crossref","unstructured":"Trinh Tu HaoyuChen andDaniel S.Brown.2024. \u201cAutonomous Assessment of Demonstration Sufficiency via Bayesian Inverse Reinforcement Learning.\u201d InProceedings of the 2024 ACM\/IEEE International Conference on Human\u2010Robot Interaction (HRI) 725\u2013733.Arlington VA:ACM\/IEEE.","DOI":"10.1145\/3610977.3634984"}],"container-title":["AI Magazine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aaai.70024","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1002\/aaai.70024","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aaai.70024","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/aaai.70024","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,7]],"date-time":"2026-07-07T19:23:48Z","timestamp":1783452228000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1002\/aaai.70024"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,29]]},"references-count":33,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2025,9]]}},"alternative-id":["10.1002\/aaai.70024"],"URL":"https:\/\/doi.org\/10.1002\/aaai.70024","archive":["Portico"],"relation":{},"ISSN":["0738-4602","2371-9621"],"issn-type":[{"value":"0738-4602","type":"print"},{"value":"2371-9621","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,29]]},"assertion":[{"value":"2025-05-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-29","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-08-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70024"}}