{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:19:09Z","timestamp":1750220349250,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":62,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,6,28]],"date-time":"2021-06-28T00:00:00Z","timestamp":1624838400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF (National Science Foundation)","doi-asserted-by":"publisher","award":["1907542"],"award-info":[{"award-number":["1907542"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,6,28]]},"DOI":"10.1145\/3461778.3462029","type":"proceedings-article","created":{"date-parts":[[2021,6,28]],"date-time":"2021-06-28T20:26:48Z","timestamp":1624912008000},"page":"1638-1653","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Hammers for Robots: Designing Tools for Reinforcement Learning Agents"],"prefix":"10.1145","author":[{"given":"Matthew V","family":"Law","sequence":"first","affiliation":[{"name":"Information Science Cornell University, United States"}]},{"given":"Zhilong","family":"Li","sequence":"additional","affiliation":[{"name":"Computer Science Cornell University, United States"}]},{"given":"Amit","family":"Rajesh","sequence":"additional","affiliation":[{"name":"Computer Science Cornell University, United States"}]},{"given":"Nikhil","family":"Dhawan","sequence":"additional","affiliation":[{"name":"Computer Science Cornell University, United States"}]},{"given":"Amritansh","family":"Kwatra","sequence":"additional","affiliation":[{"name":"Computer Science Cornell University, United States"}]},{"given":"Guy","family":"Hoffman","sequence":"additional","affiliation":[{"name":"Cornell University, United States"}]}],"member":"320","published-online":{"date-parts":[[2021,6,28]]},"reference":[{"key":"e_1_3_2_1_1_1","first-page":"445","article-title":"User-centered design","volume":"37","author":"Abras Chadia","year":"2004","unstructured":"Chadia Abras , Diane Maloney-Krichmar , Jenny Preece , 2004 . User-centered design . Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications 37 , 4 (2004), 445 \u2013 456 . Chadia Abras, Diane Maloney-Krichmar, Jenny Preece, 2004. User-centered design. Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications 37, 4 (2004), 445\u2013456.","journal-title":"Bainbridge, W. Encyclopedia of Human-Computer Interaction. Thousand Oaks: Sage Publications"},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of the 34th International Conference on Machine Learning -","volume":"185","author":"Anschel Oron","year":"2017","unstructured":"Oron Anschel , Nir Baram , and Nahum Shimkin . 2017 . Averaged-dqn: Variance reduction and stabilization for deep reinforcement learning . In Proceedings of the 34th International Conference on Machine Learning - Volume 70(ICML\u201917). JMLR.org, Sydney, NSW, Australia, 176\u2013 185 . Oron Anschel, Nir Baram, and Nahum Shimkin. 2017. Averaged-dqn: Variance reduction and stabilization for deep reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning - Volume 70(ICML\u201917). JMLR.org, Sydney, NSW, Australia, 176\u2013185."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3357236.3395525"},{"key":"e_1_3_2_1_4_1","unstructured":"Akanksha Atrey Kaleigh Clary and David\u00a0D. Jensen. 2019. Exploratory not explanatory: counterfactual analysis of saliency maps for Deep RL. CoRR abs\/1912.05743(2019). arxiv:1912.05743http:\/\/arxiv.org\/abs\/1912.05743  Akanksha Atrey Kaleigh Clary and David\u00a0D. Jensen. 2019. Exploratory not explanatory: counterfactual analysis of saliency maps for Deep RL. CoRR abs\/1912.05743(2019). arxiv:1912.05743http:\/\/arxiv.org\/abs\/1912.05743"},{"volume-title":"Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland UK) (CHI \u201919)","author":"L.","key":"e_1_3_2_1_5_1","unstructured":"Cynthia\u00a0 L. Bennett and Daniela\u00a0K. Rosner. 2019. The promise of empathy: Design, disability, and knowing the \u201dother \u201d. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland UK) (CHI \u201919) . Association for Computing Machinery, New York, NY, USA, 1\u201313. https:\/\/doi.org\/10.1145\/3290605.3300528 10.1145\/3290605.3300528 Cynthia\u00a0L. Bennett and Daniela\u00a0K. Rosner. 2019. The promise of empathy: Design, disability, and knowing the \u201dother\u201d. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasgow, Scotland UK) (CHI \u201919). Association for Computing Machinery, New York, NY, USA, 1\u201313. https:\/\/doi.org\/10.1145\/3290605.3300528"},{"key":"e_1_3_2_1_6_1","unstructured":"Greg Brockman Vicki Cheung Ludwig Pettersson Jonas Schneider John Schulman Jie Tang and Wojciech Zaremba. 2016. OpenAI Gym. CoRR abs\/1606.01540(2016). arxiv:1606.01540http:\/\/arxiv.org\/abs\/1606.01540  Greg Brockman Vicki Cheung Ludwig Pettersson Jonas Schneider John Schulman Jie Tang and Wojciech Zaremba. 2016. OpenAI Gym. CoRR abs\/1606.01540(2016). arxiv:1606.01540http:\/\/arxiv.org\/abs\/1606.01540"},{"volume-title":"The psychology of human-computer interaction","author":"Card K","key":"e_1_3_2_1_7_1","unstructured":"Stuart\u00a0 K Card , Thomas\u00a0 P. Moran , and Alan Newell . 1983. The psychology of human-computer interaction . Crc Press New York , NY , USA. Stuart\u00a0K Card, Thomas\u00a0P. Moran, and Alan Newell. 1983. The psychology of human-computer interaction. Crc Press New York, NY, USA."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"crossref","unstructured":"John\u00a0M Carroll and Judith\u00a0Reitman Olson. 1988. Mental models in human-computer interaction. Handbook of human-computer interaction(1988) 45\u201365.  John\u00a0M Carroll and Judith\u00a0Reitman Olson. 1988. Mental models in human-computer interaction. Handbook of human-computer interaction(1988) 45\u201365.","DOI":"10.1016\/B978-0-444-70536-5.50007-5"},{"key":"e_1_3_2_1_9_1","unstructured":"Geoffrey Cideron Mathieu Seurin Florian Strub and Olivier Pietquin. 2019. Self-educated language agent with hindsight experience replay for instruction following. CoRR abs\/1910.09451(2019). arxiv:1910.09451http:\/\/arxiv.org\/abs\/1910.09451  Geoffrey Cideron Mathieu Seurin Florian Strub and Olivier Pietquin. 2019. Self-educated language agent with hindsight experience replay for instruction following. CoRR abs\/1910.09451(2019). arxiv:1910.09451http:\/\/arxiv.org\/abs\/1910.09451"},{"volume-title":"About face: the essentials of interaction design","author":"Cooper Alan","key":"e_1_3_2_1_10_1","unstructured":"Alan Cooper , Robert Reimann , David Cronin , and Christopher Noessel . 2014. About face: the essentials of interaction design . John Wiley & Sons . Alan Cooper, Robert Reimann, David Cronin, and Christopher Noessel. 2014. About face: the essentials of interaction design. John Wiley & Sons."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.558708"},{"key":"e_1_3_2_1_12_1","first-page":"331","article-title":"Almost human: Anthropomorphism increases trust resilience in cognitive agents.Journal of Experimental Psychology","volume":"22","author":"De\u00a0Visser J","year":"2016","unstructured":"Ewart\u00a0 J De\u00a0Visser , Samuel\u00a0 S Monfort , Ryan McKendrick , Melissa\u00a0 AB Smith , Patrick\u00a0 E McKnight , Frank Krueger , and Raja Parasuraman . 2016 . Almost human: Anthropomorphism increases trust resilience in cognitive agents.Journal of Experimental Psychology : Applied 22 , 3 (2016), 331 . Ewart\u00a0J De\u00a0Visser, Samuel\u00a0S Monfort, Ryan McKendrick, Melissa\u00a0AB Smith, Patrick\u00a0E McKnight, Frank Krueger, and Raja Parasuraman. 2016. Almost human: Anthropomorphism increases trust resilience in cognitive agents.Journal of Experimental Psychology: Applied 22, 3 (2016), 331.","journal-title":"Applied"},{"key":"e_1_3_2_1_13_1","unstructured":"Shuby Deshpande Benjamin Eysenbach and Jeff Schneider. 2020. Interactive visualization for debugging rl. arxiv:2008.07331\u00a0[cs.LG]  Shuby Deshpande Benjamin Eysenbach and Jeff Schneider. 2020. Interactive visualization for debugging rl. arxiv:2008.07331\u00a0[cs.LG]"},{"key":"e_1_3_2_1_14_1","volume-title":"2014 AAAI Spring Symposium Series.","author":"Dewey Daniel","year":"2014","unstructured":"Daniel Dewey . 2014 . Reinforcement learning and the reward engineering principle . In 2014 AAAI Spring Symposium Series. Daniel Dewey. 2014. Reinforcement learning and the reward engineering principle. In 2014 AAAI Spring Symposium Series."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2011.07.006"},{"volume-title":"Experimental Design Research: Approaches, Perspectives, Applications","author":"Egan Paul","key":"e_1_3_2_1_16_1","unstructured":"Paul Egan and Jonathan Cagan . 2016. Human and computational approaches for design problem-solving . In Experimental Design Research: Approaches, Perspectives, Applications . Springer International Publishing , Cham , 187\u2013205. https:\/\/doi.org\/10.1007\/978-3-319-33781-4_11 10.1007\/978-3-319-33781-4_11 Paul Egan and Jonathan Cagan. 2016. Human and computational approaches for design problem-solving. In Experimental Design Research: Approaches, Perspectives, Applications. Springer International Publishing, Cham, 187\u2013205. https:\/\/doi.org\/10.1007\/978-3-319-33781-4_11"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302316"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1013115.1013152"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/242485.242493"},{"key":"e_1_3_2_1_20_1","volume-title":"What is human centred design?The Design Journal 17, 4","author":"Giacomin Joseph","year":"2014","unstructured":"Joseph Giacomin . 2014. What is human centred design?The Design Journal 17, 4 ( 2014 ), 606\u2013623. Joseph Giacomin. 2014. What is human centred design?The Design Journal 17, 4 (2014), 606\u2013623."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3166.3170"},{"key":"e_1_3_2_1_22_1","volume-title":"Proceedings of the 35th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol.\u00a080)","author":"Greydanus Samuel","year":"2018","unstructured":"Samuel Greydanus , Anurag Koul , Jonathan Dodge , and Alan Fern . 2018 . Visualizing and Understanding Atari Agents . In Proceedings of the 35th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol.\u00a080) , Jennifer Dy and Andreas Krause (Eds.). PMLR, Stockholmsm\u00e4ssan, Stockholm Sweden, 1792\u2013 1801. http:\/\/proceedings.mlr.press\/v80\/greydanus18a.html Samuel Greydanus, Anurag Koul, Jonathan Dodge, and Alan Fern. 2018. Visualizing and Understanding Atari Agents. In Proceedings of the 35th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol.\u00a080), Jennifer Dy and Andreas Krause (Eds.). PMLR, Stockholmsm\u00e4ssan, Stockholm Sweden, 1792\u20131801. http:\/\/proceedings.mlr.press\/v80\/greydanus18a.html"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1162\/artl_a_00301"},{"key":"e_1_3_2_1_24_1","unstructured":"David Ha and J\u00fcrgen Schmidhuber. 2018. World Models. CoRR abs\/1803.10122(2018). arxiv:1803.10122http:\/\/arxiv.org\/abs\/1803.10122  David Ha and J\u00fcrgen Schmidhuber. 2018. World Models. CoRR abs\/1803.10122(2018). arxiv:1803.10122http:\/\/arxiv.org\/abs\/1803.10122"},{"key":"e_1_3_2_1_25_1","unstructured":"Dylan Hadfield-Menell Anca\u00a0D. Dragan Pieter Abbeel and Stuart\u00a0J. Russell. 2016. Cooperative inverse reinforcement learning. CoRR abs\/1606.03137(2016). arxiv:1606.03137http:\/\/arxiv.org\/abs\/1606.03137  Dylan Hadfield-Menell Anca\u00a0D. Dragan Pieter Abbeel and Stuart\u00a0J. Russell. 2016. Cooperative inverse reinforcement learning. CoRR abs\/1606.03137(2016). arxiv:1606.03137http:\/\/arxiv.org\/abs\/1606.03137"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2020.106685"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.destud.2019.10.007"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/642611.642616"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3278721.3278776"},{"key":"e_1_3_2_1_30_1","volume-title":"Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems. 1615\u20131616","author":"Jacobs Elmer","year":"2014","unstructured":"Elmer Jacobs , Joost Broekens , and Catholijn Jonker . 2014 . Joy, distress, hope, and fear in reinforcement learning . In Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems. 1615\u20131616 . Elmer Jacobs, Joost Broekens, and Catholijn Jonker. 2014. Joy, distress, hope, and fear in reinforcement learning. In Proceedings of the 2014 International Conference on Autonomous Agents and Multi-agent Systems. 1615\u20131616."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300641"},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3322276.3322379"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/HRI.2016.7451807"},{"key":"e_1_3_2_1_34_1","volume-title":"Interface as mimesis. User centered system design: New perspectives on human-computer interaction","author":"Laurel Brenda","year":"1986","unstructured":"Brenda Laurel . 1986. Interface as mimesis. User centered system design: New perspectives on human-computer interaction ( 1986 ), 67\u201385. Brenda Laurel. 1986. Interface as mimesis. User centered system design: New perspectives on human-computer interaction (1986), 67\u201385."},{"key":"e_1_3_2_1_35_1","volume-title":"Proceedings of ACM Conference on Foundations of Digital Games. ACM.","author":"Liapis Antonios","year":"2013","unstructured":"Antonios Liapis , Georgios\u00a0 N Yannakakis , and Julian Togelius . 2013 . Sentient sketchbook: computer-assisted game level authoring . In Proceedings of ACM Conference on Foundations of Digital Games. ACM. Antonios Liapis, Georgios\u00a0N Yannakakis, and Julian Togelius. 2013. Sentient sketchbook: computer-assisted game level authoring. In Proceedings of ACM Conference on Foundations of Digital Games. ACM."},{"key":"e_1_3_2_1_36_1","volume-title":"Proceedings of the Conference on Robot Learning(Proceedings of Machine Learning Research, Vol.\u00a0100)","author":"Luck Kevin\u00a0Sebastian","year":"2020","unstructured":"Kevin\u00a0Sebastian Luck , Heni\u00a0Ben Amor , and Roberto Calandra . 2020 . Data-efficient co-adaptation of morphology and behaviour with deep reinforcement learning . In Proceedings of the Conference on Robot Learning(Proceedings of Machine Learning Research, Vol.\u00a0100) , Leslie\u00a0Pack Kaelbling, Danica Kragic, and Komei Sugiura (Eds.). PMLR, 854\u2013869. http:\/\/proceedings.mlr.press\/v100\/luck20a.html Kevin\u00a0Sebastian Luck, Heni\u00a0Ben Amor, and Roberto Calandra. 2020. Data-efficient co-adaptation of morphology and behaviour with deep reinforcement learning. In Proceedings of the Conference on Robot Learning(Proceedings of Machine Learning Research, Vol.\u00a0100), Leslie\u00a0Pack Kaelbling, Danica Kragic, and Komei Sugiura (Eds.). PMLR, 854\u2013869. http:\/\/proceedings.mlr.press\/v100\/luck20a.html"},{"key":"e_1_3_2_1_37_1","volume-title":"Proceedings of the Annual Meeting of the Cognitive Science Society, Vol.\u00a030","author":"Marinier P","year":"2008","unstructured":"Robert\u00a0 P Marinier and John\u00a0 E Laird . 2008 . Emotion-driven reinforcement learning . In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol.\u00a030 . Cognitive Science Society, Nashville, TN. Robert\u00a0P Marinier and John\u00a0E Laird. 2008. Emotion-driven reinforcement learning. In Proceedings of the Annual Meeting of the Cognitive Science Society, Vol.\u00a030. Cognitive Science Society, Nashville, TN."},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3432193"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447318.2015.1065696"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1038\/nature14236"},{"key":"e_1_3_2_1_41_1","unstructured":"Alex Mott Daniel Zoran Mike Chrzanowski Daan Wierstra and Danilo\u00a0J. Rezende. 2019. Towards interpretable reinforcement learning using attention augmented agents. CoRR abs\/1906.02500(2019). arxiv:1906.02500http:\/\/arxiv.org\/abs\/1906.02500  Alex Mott Daniel Zoran Mike Chrzanowski Daan Wierstra and Danilo\u00a0J. Rezende. 2019. Towards interpretable reinforcement learning using attention augmented agents. CoRR abs\/1906.02500(2019). arxiv:1906.02500http:\/\/arxiv.org\/abs\/1906.02500"},{"key":"e_1_3_2_1_42_1","unstructured":"Don Norman. 2013. The design of everyday things: Revised and expanded edition. Basic books.  Don Norman. 2013. The design of everyday things: Revised and expanded edition. Basic books."},{"key":"e_1_3_2_1_43_1","volume-title":"Some observations on mental models. Mental models 7, 112","author":"Norman A","year":"1983","unstructured":"Donald\u00a0 A Norman . 1983. Some observations on mental models. Mental models 7, 112 ( 1983 ), 7\u201314. Donald\u00a0A Norman. 1983. Some observations on mental models. Mental models 7, 112 (1983), 7\u201314."},{"key":"e_1_3_2_1_44_1","unstructured":"Donald\u00a0A Norman. 1991. Cognitive artifacts. Designing interaction: Psychology at the human-computer interface 1 1(1991) 17\u201338.  Donald\u00a0A Norman. 1991. Cognitive artifacts. Designing interaction: Psychology at the human-computer interface 1 1(1991) 17\u201338."},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"crossref","unstructured":"Donald\u00a0A Norman and Steven\u00a0W Draper (Eds.). 1986. User centered system design: New perspectives on human-computer interaction. (1986).  Donald\u00a0A Norman and Steven\u00a0W Draper (Eds.). 1986. User centered system design: New perspectives on human-computer interaction. (1986).","DOI":"10.1201\/b15703"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11023-019-09502-w"},{"key":"e_1_3_2_1_47_1","unstructured":"Luc Prieur. 2017. Python notebook for the Box2D Race car reinforce learning problem.https:\/\/gist.github.com\/lmclupr\/b35c89b2f8f81b443166e88b787b03ab.  Luc Prieur. 2017. Python notebook for the Box2D Race car reinforce learning problem.https:\/\/gist.github.com\/lmclupr\/b35c89b2f8f81b443166e88b787b03ab."},{"volume-title":"Motion, Interaction and Games","author":"Reda Daniele","key":"e_1_3_2_1_48_1","unstructured":"Daniele Reda , Tianxin Tao , and Michiel van\u00a0de Panne . 2020. Learning to locomote: understanding how environment design matters for deep reinforcement learning . In Motion, Interaction and Games . Association for Computing Machinery , New York, NY, USA , Article 16, 10\u00a0pages. https:\/\/doi.org\/10.1145\/3424636.3426907 10.1145\/3424636.3426907 Daniele Reda, Tianxin Tao, and Michiel van\u00a0de Panne. 2020. Learning to locomote: understanding how environment design matters for deep reinforcement learning. In Motion, Interaction and Games. Association for Computing Machinery, New York, NY, USA, Article 16, 10\u00a0pages. https:\/\/doi.org\/10.1145\/3424636.3426907"},{"key":"e_1_3_2_1_49_1","unstructured":"Matthias Rosynski Frank Kirchner and Matias Valdenegro-Toro. 2020. Are gradient-based saliency maps useful in deep reinforcement learning?arxiv:2012.01281\u00a0[cs.LG]  Matthias Rosynski Frank Kirchner and Matias Valdenegro-Toro. 2020. Are gradient-based saliency maps useful in deep reinforcement learning?arxiv:2012.01281\u00a0[cs.LG]"},{"key":"e_1_3_2_1_50_1","unstructured":"Christian Rupprecht Cyril Ibrahim and Christopher\u00a0J. Pal. 2019. Finding and visualizing weaknesses of deep reinforcement learning agents. CoRR abs\/1904.01318(2019). arxiv:1904.01318http:\/\/arxiv.org\/abs\/1904.01318  Christian Rupprecht Cyril Ibrahim and Christopher\u00a0J. Pal. 2019. Finding and visualizing weaknesses of deep reinforcement learning agents. CoRR abs\/1904.01318(2019). arxiv:1904.01318http:\/\/arxiv.org\/abs\/1904.01318"},{"key":"e_1_3_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2019.8793537"},{"key":"e_1_3_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1016\/0950-7051(92)90020-G"},{"key":"e_1_3_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2020.103367"},{"volume-title":"Affective Computing and Intelligent Interaction, Sidney D\u2019Mello, Arthur Graesser, Bj\u00f6rn Schuller, and Jean-Claude Martin (Eds.)","author":"Sequeira Pedro","key":"e_1_3_2_1_54_1","unstructured":"Pedro Sequeira , Francisco\u00a0 S. Melo , and Ana Paiva . 2011. Emotion-based intrinsic motivation for reinforcement learning agents . In Affective Computing and Intelligent Interaction, Sidney D\u2019Mello, Arthur Graesser, Bj\u00f6rn Schuller, and Jean-Claude Martin (Eds.) . Springer Berlin Heidelberg , Berlin, Heidelberg , 326\u2013336. Pedro Sequeira, Francisco\u00a0S. Melo, and Ana Paiva. 2011. Emotion-based intrinsic motivation for reinforcement learning agents. In Affective Computing and Intelligent Interaction, Sidney D\u2019Mello, Arthur Graesser, Bj\u00f6rn Schuller, and Jean-Claude Martin (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 326\u2013336."},{"key":"e_1_3_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1006\/imms.1993.1028"},{"key":"e_1_3_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/503376.503460"},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1177\/001872088903100601"},{"key":"e_1_3_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.293155"},{"key":"e_1_3_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357156"},{"key":"e_1_3_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.2200\/S00229ED1V01Y201003HCI009"},{"key":"e_1_3_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376301"},{"key":"e_1_3_2_1_62_1","unstructured":"Ruohan Zhang Bo Liu Yifeng Zhu Sihang Guo Mary Hayhoe Dana Ballard and Peter Stone. 2020. Human versus machine attention in deep reinforcement learning tasks. arxiv:2010.15942\u00a0[cs.LG]  Ruohan Zhang Bo Liu Yifeng Zhu Sihang Guo Mary Hayhoe Dana Ballard and Peter Stone. 2020. Human versus machine attention in deep reinforcement learning tasks. arxiv:2010.15942\u00a0[cs.LG]"}],"event":{"name":"DIS '21: Designing Interactive Systems Conference 2021","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"],"location":"Virtual Event USA","acronym":"DIS '21"},"container-title":["Designing Interactive Systems Conference 2021"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461778.3462029","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3461778.3462029","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3461778.3462029","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:07Z","timestamp":1750191427000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3461778.3462029"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,28]]},"references-count":62,"alternative-id":["10.1145\/3461778.3462029","10.1145\/3461778"],"URL":"https:\/\/doi.org\/10.1145\/3461778.3462029","relation":{},"subject":[],"published":{"date-parts":[[2021,6,28]]},"assertion":[{"value":"2021-06-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}