{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T06:17:46Z","timestamp":1782281866801,"version":"3.54.5"},"publisher-location":"New York, NY, USA","reference-count":48,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,3,17]],"date-time":"2020-03-17T00:00:00Z","timestamp":1584403200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,3,17]]},"DOI":"10.1145\/3377325.3377498","type":"proceedings-article","created":{"date-parts":[[2020,3,4]],"date-time":"2020-03-04T23:14:49Z","timestamp":1583363689000},"page":"454-464","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":234,"title":["Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems"],"prefix":"10.1145","author":[{"given":"Zana","family":"Bu\u00e7inca","sequence":"first","affiliation":[{"name":"Harvard University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Phoebe","family":"Lin","sequence":"additional","affiliation":[{"name":"Harvard University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Krzysztof Z.","family":"Gajos","sequence":"additional","affiliation":[{"name":"Harvard University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elena L.","family":"Glassman","sequence":"additional","affiliation":[{"name":"Harvard University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,3,17]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300233"},{"key":"e_1_3_2_1_2_1","volume-title":"Proceedings of the 25th International Conference on Intelligent User Interfaces (IUI '20)","author":"Arnold Kenneth C.","unstructured":"Kenneth C. Arnold , Krysta Chaunce , and Krzysztof Z. Gajos . 2020. Predictive Text Encourages Predictable Writing . In Proceedings of the 25th International Conference on Intelligent User Interfaces (IUI '20) . ACM, New York, NY, USA. Kenneth C. Arnold, Krysta Chaunce, and Krzysztof Z. Gajos. 2020. Predictive Text Encourages Predictable Writing. In Proceedings of the 25th International Conference on Intelligent User Interfaces (IUI '20). ACM, New York, NY, USA."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v7i1.5285"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1177\/0272989X14547740"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300234"},{"key":"e_1_3_2_1_6_1","unstructured":"Jonathan Chang Sean Gerrish Chong Wang Jordan L Boyd-Graber and David M Blei. 2009. Reading tea leaves: How humans interpret topic models. In Advances in neural information processing systems. 288--296.  Jonathan Chang Sean Gerrish Chong Wang Jordan L Boyd-Graber and David M Blei. 2009. Reading tea leaves: How humans interpret topic models. In Advances in neural information processing systems. 288--296."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1067\/mem.2003.22"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1821936116"},{"key":"e_1_3_2_1_9_1","volume-title":"Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608","author":"Doshi-Velez Finale","year":"2017","unstructured":"Finale Doshi-Velez and Been Kim . 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 ( 2017 ). Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017)."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1016\/S1071-5819(03)00038-7"},{"key":"e_1_3_2_1_11_1","unstructured":"K Anders Ericsson and Herbert A Simon. 1984. Protocol analysis: Verbal reports as data. the MIT Press.  K Anders Ericsson and Herbert A Simon. 1984. Protocol analysis: Verbal reports as data. the MIT Press."},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1086\/209500"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/W18-6511"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/DSAA.2018.00018"},{"key":"e_1_3_2_1_15_1","unstructured":"George Anthony Gorry and Michael S Scott Morton. 1971. A framework for management information systems. (1971).  George Anthony Gorry and Michael S Scott Morton. 1971. A framework for management information systems. (1971)."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287563"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359152"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025171.3025198"},{"key":"e_1_3_2_1_19_1","volume-title":"Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608","author":"Hoffman Robert R","year":"2018","unstructured":"Robert R Hoffman , Shane T Mueller , Gary Klein , and Jordan Litman . 2018. Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608 ( 2018 ). Robert R Hoffman, Shane T Mueller, Gary Klein, and Jordan Litman. 2018. Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:1812.04608 (2018)."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.7326\/0003-4819-120-2-199401150-00007"},{"key":"e_1_3_2_1_21_1","unstructured":"Ece Kamar. 2016. Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence.. In IJCAI. 4070--4073.  Ece Kamar. 2016. Directions in Hybrid Intelligence: Complementing AI Systems with Human Intelligence.. In IJCAI. 4070--4073."},{"key":"e_1_3_2_1_22_1","volume-title":"Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 467--474","author":"Kamar Ece","year":"2012","unstructured":"Ece Kamar , Severin Hacker , and Eric Horvitz . 2012 . Combining human and machine intelligence in large-scale crowdsourcing . In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 467--474 . Ece Kamar, Severin Hacker, and Eric Horvitz. 2012. Combining human and machine intelligence in large-scale crowdsourcing. In Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems-Volume 1. International Foundation for Autonomous Agents and Multiagent Systems, 467--474."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1093\/qje\/qjx032"},{"key":"e_1_3_2_1_24_1","volume-title":"Mental labour. Nature human behaviour 2, 12","author":"Kool Wouter","year":"2018","unstructured":"Wouter Kool and Matthew Botvinick . 2018. Mental labour. Nature human behaviour 2, 12 ( 2018 ), 899--908. Wouter Kool and Matthew Botvinick. 2018. Mental labour. Nature human behaviour 2, 12 (2018), 899--908."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1037\/a0020198"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2678025.2701399"},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1609\/hcomp.v7i1.5280"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287590"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939874"},{"key":"e_1_3_2_1_30_1","volume-title":"Manipulating User Trust via Misleading Black Box Explanations. arXiv preprint arXiv:1911.06473","author":"Lakkaraju Himabindu","year":"2019","unstructured":"Himabindu Lakkaraju and Osbert Bastani . 2019. \" How do I fool you?\" : Manipulating User Trust via Misleading Black Box Explanations. arXiv preprint arXiv:1911.06473 ( 2019 ). Himabindu Lakkaraju and Osbert Bastani. 2019. \"How do I fool you?\": Manipulating User Trust via Misleading Black Box Explanations. arXiv preprint arXiv:1911.06473 (2019)."},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1136\/bmjqs-2015-004417"},{"key":"e_1_3_2_1_32_1","volume-title":"Trust in automation: Designing for appropriate reliance. Human factors 46, 1","author":"Lee John D","year":"2004","unstructured":"John D Lee and Katrina A See . 2004. Trust in automation: Designing for appropriate reliance. Human factors 46, 1 ( 2004 ), 50--80. John D Lee and Katrina A See. 2004. Trust in automation: Designing for appropriate reliance. Human factors 46, 1 (2004), 50--80."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0020-7373(87)80013-5"},{"key":"e_1_3_2_1_34_1","volume-title":"Jennifer Wortman Vaughan, and Hanna Wallach","author":"Poursabzi-Sangdeh Forough","year":"2018","unstructured":"Forough Poursabzi-Sangdeh , Daniel G Goldstein , Jake M Hofman , Jennifer Wortman Vaughan, and Hanna Wallach . 2018 . Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018)."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.74"},{"key":"e_1_3_2_1_37_1","volume-title":"Heuristics made easy: An effort-reduction framework. Psychological bulletin 134, 2","author":"Shah Anuj K","year":"2008","unstructured":"Anuj K Shah and Daniel M Oppenheimer . 2008. Heuristics made easy: An effort-reduction framework. Psychological bulletin 134, 2 ( 2008 ), 207. Anuj K Shah and Daniel M Oppenheimer. 2008. Heuristics made easy: An effort-reduction framework. Psychological bulletin 134, 2 (2008), 207."},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.2310\/8000.2013.130860"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1177\/1098214005283748"},{"key":"e_1_3_2_1_40_1","volume-title":"Judgment under uncertainty: Heuristics and biases. science 185, 4157","author":"Tversky Amos","year":"1974","unstructured":"Amos Tversky and Daniel Kahneman . 1974. Judgment under uncertainty: Heuristics and biases. science 185, 4157 ( 1974 ), 1124--1131. Amos Tversky and Daniel Kahneman. 1974. Judgment under uncertainty: Heuristics and biases. science 185, 4157 (1974), 1124--1131."},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300831"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.4018\/jcini.2007010101"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSMCC.2006.871126"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3308532.3329441"},{"key":"e_1_3_2_1_45_1","unstructured":"Robert Andrew Wilson and Frank C Keil. 2001. The MIT encyclopedia of the cognitive sciences.  Robert Andrew Wilson and Frank C Keil. 2001. The MIT encyclopedia of the cognitive sciences."},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300509"},{"key":"e_1_3_2_1_47_1","volume-title":"Applied Intelligent Systems","author":"Zeleznikow John","unstructured":"John Zeleznikow . 2004. Building intelligent legal decision support systems: Past practice and future challenges . In Applied Intelligent Systems . Springer , 201--254. John Zeleznikow. 2004. Building intelligent legal decision support systems: Past practice and future challenges. In Applied Intelligent Systems. Springer, 201--254."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"crossref","unstructured":"Bolei Zhou Yiyou Sun David Bau and Antonio Torralba. 2018. Interpretable Basis Decomposition for Visual Explanation. In ECCV. 119--134.  Bolei Zhou Yiyou Sun David Bau and Antonio Torralba. 2018. Interpretable Basis Decomposition for Visual Explanation. In ECCV. 119--134.","DOI":"10.1007\/978-3-030-01237-3_8"}],"event":{"name":"IUI '20: 25th International Conference on Intelligent User Interfaces","location":"Cagliari Italy","acronym":"IUI '20","sponsor":["SIGAI ACM Special Interest Group on Artificial Intelligence","SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["Proceedings of the 25th International Conference on Intelligent User Interfaces"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3377325.3377498","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3377325.3377498","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:23:42Z","timestamp":1750202622000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3377325.3377498"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,17]]},"references-count":48,"alternative-id":["10.1145\/3377325.3377498","10.1145\/3377325"],"URL":"https:\/\/doi.org\/10.1145\/3377325.3377498","relation":{},"subject":[],"published":{"date-parts":[[2020,3,17]]},"assertion":[{"value":"2020-03-17","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}