{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,3]],"date-time":"2026-10-03T02:43:31Z","timestamp":1790995411199,"version":"4.1.0"},"reference-count":100,"publisher":"Association for Computing Machinery (ACM)","issue":"CSCW3","license":[{"start":{"date-parts":[[2021,1,5]],"date-time":"2021-01-05T00:00:00Z","timestamp":1609804800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Hum.-Comput. Interact."],"published-print":{"date-parts":[[2021,1,5]]},"abstract":"<jats:p>The wide adoption of Machine Learning (ML) technologies has created a growing demand for people who can train ML models. Some advocated the term \"machine teacher'' to refer to the role of people who inject domain knowledge into ML models. This \"teaching'' perspective emphasizes supporting the productivity and mental wellbeing of machine teachers through efficient learning algorithms and thoughtful design of human-AI interfaces. One promising learning paradigm is Active Learning (AL), by which the model intelligently selects instances to query a machine teacher for labels, so that the labeling workload could be largely reduced. However, in current AL settings, the human-AI interface remains minimal and opaque. A dearth of empirical studies further hinders us from developing teacher-friendly interfaces for AL algorithms. In this work, we begin considering AI explanations as a core element of the human-AI interface for teaching machines. When a human student learns, it is a common pattern to present one's own reasoning and solicit feedback from the teacher. When a ML model learns and still makes mistakes, the teacher ought to be able to understand the reasoning underlying its mistakes. When the model matures, the teacher should be able to recognize its progress in order to trust and feel confident about their teaching outcome. Toward this vision, we propose a novel paradigm of explainable active learning (XAL), by introducing techniques from the surging field of explainable AI (XAI) into an AL setting. We conducted an empirical study comparing the model learning outcomes, feedback content and experience with XAL, to that of traditional AL and coactive learning (providing the model's prediction without explanation). Our study shows benefits of AI explanation as interfaces for machine teaching--supporting trust calibration and enabling rich forms of teaching feedback, and potential drawbacks--anchoring effect with the model judgment and additional cognitive workload. Our study also reveals important individual factors that mediate a machine teacher's reception to AI explanations, including task knowledge, AI experience and Need for Cognition. By reflecting on the results, we suggest future directions and design implications for XAL, and more broadly, machine teaching through AI explanations.<\/jats:p>","DOI":"10.1145\/3432934","type":"journal-article","created":{"date-parts":[[2021,1,5]],"date-time":"2021-01-05T18:43:37Z","timestamp":1609872217000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":106,"title":["Explainable Active Learning (XAL)"],"prefix":"10.1145","volume":"4","author":[{"given":"Bhavya","family":"Ghai","sequence":"first","affiliation":[{"name":"Stony Brook University, Stony Brook, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Q. Vera","family":"Liao","sequence":"additional","affiliation":[{"name":"IBM Research AI, Yorktown Heights, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yunfeng","family":"Zhang","sequence":"additional","affiliation":[{"name":"IBM Research AI, Yorktown Heights, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rachel","family":"Bellamy","sequence":"additional","affiliation":[{"name":"IBM Research AI, Yorktown Heights, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Klaus","family":"Mueller","sequence":"additional","affiliation":[{"name":"Stony Brook University, Stony Brook, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,1,5]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2019 (accessed July 2019). US Inflation Calculator. https:\/\/www.usinflationcalculator.com\/.  2019 (accessed July 2019). US Inflation Calculator. https:\/\/www.usinflationcalculator.com\/."},{"key":"e_1_2_1_2_1","volume-title":"Proceedings of the 16th international conference on World Wide Web. 11--20","author":"Brusilovsky Peter","year":"2007","unstructured":"Jae-wook Ahn, Peter Brusilovsky , Jonathan Grady , Daqing He , and Sue Yeon Syn . 2007 . Open user profiles for adaptive news systems: help or harm? . In Proceedings of the 16th international conference on World Wide Web. 11--20 . Jae-wook Ahn, Peter Brusilovsky, Jonathan Grady, Daqing He, and Sue Yeon Syn. 2007. Open user profiles for adaptive news systems: help or harm?. In Proceedings of the 16th international conference on World Wide Web. 11--20."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v35i4.2513"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702509"},{"key":"e_1_2_1_5_1","volume-title":"Optimal Motion Control of a Ground Vehicle. Master's thesis","author":"Anisi David A.","unstructured":"David A. Anisi . 2003. Optimal Motion Control of a Ground Vehicle. Master's thesis . Royal Institute of Technology (KTH) , Stockholm, Sweden . David A. Anisi. 2003. Optimal Motion Control of a Ground Vehicle. Master's thesis. Royal Institute of Technology (KTH), Stockholm, Sweden."},{"key":"e_1_2_1_6_1","volume-title":"Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C Hoffman, Stephanie Houde, Q Vera Liao, Ronny Luss, Aleksandra Mojsilovi\u0107, et almbox.","author":"Arya Vijay","year":"2019","unstructured":"Vijay Arya , Rachel KE Bellamy , Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C Hoffman, Stephanie Houde, Q Vera Liao, Ronny Luss, Aleksandra Mojsilovi\u0107, et almbox. 2019 . One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques . arXiv preprint arXiv:1909.03012 (2019). Vijay Arya, Rachel KE Bellamy, Pin-Yu Chen, Amit Dhurandhar, Michael Hind, Samuel C Hoffman, Stephanie Houde, Q Vera Liao, Ronny Luss, Aleksandra Mojsilovi\u0107, et almbox. 2019. One Explanation Does Not Fit All: A Toolkit and Taxonomy of AI Explainability Techniques. arXiv preprint arXiv:1909.03012 (2019)."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-72927-3_5"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSC.2018.00059"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1214\/11-AOAS495"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/VAST.2015.7347637"},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the 25th International Conference on Intelligent User Interfaces. 454--464","author":"Zana Bucc","year":"2020","unstructured":"Zana Bucc inca, Phoebe Lin , Krzysztof Z Gajos , and Elena L Glassman . 2020 . Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems . In Proceedings of the 25th International Conference on Intelligent User Interfaces. 454--464 . Zana Bucc inca, Phoebe Lin, Krzysztof Z Gajos, and Elena L Glassman. 2020. Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems. In Proceedings of the 25th International Conference on Intelligent User Interfaces. 454--464."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.42.1.116"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.45.4.805"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAMD.2010.2051030"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2157689.2157693"},{"key":"e_1_2_1_16_1","first-page":"832","article-title":"Machine Learning Interpretability","volume":"8","author":"Carvalho Diogo V","year":"2019","unstructured":"Diogo V Carvalho , Eduardo M Pereira , and Jaime S Cardoso . 2019 . Machine Learning Interpretability : A Survey on Methods and Metrics. Electronics , Vol. 8 , 8 (2019), 832 . Diogo V Carvalho, Eduardo M Pereira, and Jaime S Cardoso. 2019. Machine Learning Interpretability: A Survey on Methods and Metrics. Electronics, Vol. 8, 8 (2019), 832.","journal-title":"A Survey on Methods and Metrics. Electronics"},{"key":"e_1_2_1_17_1","volume-title":"The Emerging Landscape of Explainable AI Planning and Decision Making. arXiv preprint arXiv:2002.11697","author":"Chakraborti Tathagata","year":"2020","unstructured":"Tathagata Chakraborti , Sarath Sreedharan , and Subbarao Kambhampati . 2020. The Emerging Landscape of Explainable AI Planning and Decision Making. arXiv preprint arXiv:2002.11697 ( 2020 ). Tathagata Chakraborti, Sarath Sreedharan, and Subbarao Kambhampati. 2020. The Emerging Landscape of Explainable AI Planning and Decision Making. arXiv preprint arXiv:2002.11697 (2020)."},{"key":"e_1_2_1_18_1","volume-title":"2010 5th ACM\/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 317--324","author":"Chao Crystal","year":"2010","unstructured":"Crystal Chao , Maya Cakmak , and Andrea L Thomaz . 2010 . Transparent active learning for robots . In 2010 5th ACM\/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 317--324 . Crystal Chao, Maya Cakmak, and Andrea L Thomaz. 2010. Transparent active learning for robots. In 2010 5th ACM\/IEEE International Conference on Human-Robot Interaction (HRI). IEEE, 317--324."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300789"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702145"},{"key":"e_1_2_1_21_1","volume-title":"Utopian: User-driven topic modeling based on interactive nonnegative matrix factorization","author":"Choo Jaegul","year":"2013","unstructured":"Jaegul Choo , Changhyun Lee , Chandan K Reddy , and Haesun Park . 2013 . Utopian: User-driven topic modeling based on interactive nonnegative matrix factorization . IEEE transactions on visualization and computer graphics, Vol. 19 , 12 (2013), 1992--2001. Jaegul Choo, Changhyun Lee, Chandan K Reddy, and Haesun Park. 2013. Utopian: User-driven topic modeling based on interactive nonnegative matrix factorization. IEEE transactions on visualization and computer graphics, Vol. 19, 12 (2013), 1992--2001."},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the KDD 2014 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Chou Sophie","year":"2014","unstructured":"Sophie Chou , William Li , and Ramesh Sridharan . 2014 . Democratizing data science . In Proceedings of the KDD 2014 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , New York, NY, USA. Citeseer, 24--27. Sophie Chou, William Li, and Ramesh Sridharan. 2014. Democratizing data science. In Proceedings of the KDD 2014 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, NY, USA. Citeseer, 24--27."},{"key":"e_1_2_1_23_1","volume-title":"Improving generalization with active learning. Machine learning","author":"Cohn David","year":"1994","unstructured":"David Cohn , Les Atlas , and Richard Ladner . 1994. Improving generalization with active learning. Machine learning , Vol. 15 , 2 ( 1994 ), 201--221. David Cohn, Les Atlas, and Richard Ladner. 1994. Improving generalization with active learning. Machine learning, Vol. 15, 2 (1994), 201--221."},{"key":"e_1_2_1_24_1","volume-title":"The Sage dictionary of statistics: a practical resource for students in the social sciences","author":"Cramer Duncan","unstructured":"Duncan Cramer and Dennis Laurence Howitt . 2004. The Sage dictionary of statistics: a practical resource for students in the social sciences . Sage . Duncan Cramer and Dennis Laurence Howitt. 2004. The Sage dictionary of statistics: a practical resource for students in the social sciences. Sage."},{"key":"e_1_2_1_25_1","first-page":"746","article-title":"Reducing labeling effort for structured prediction tasks","volume":"5","author":"Culotta Aron","year":"2005","unstructured":"Aron Culotta and Andrew McCallum . 2005 . Reducing labeling effort for structured prediction tasks . In AAAI , Vol. 5. 746 -- 751 . Aron Culotta and Andrew McCallum. 2005. Reducing labeling effort for structured prediction tasks. In AAAI, Vol. 5. 746--751.","journal-title":"AAAI"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390156.1390183"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302310"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1458082.1458165"},{"key":"e_1_2_1_29_1","volume-title":"Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608","author":"Doshi-Velez Finale","year":"2017","unstructured":"Finale Doshi-Velez and Been Kim . 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 ( 2017 ). Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017)."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.3115\/1699510.1699522"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/604045.604056"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357061"},{"key":"e_1_2_1_33_1","volume-title":"Selective sampling using the query by committee algorithm. Machine learning","author":"Freund Yoav","year":"1997","unstructured":"Yoav Freund , H Sebastian Seung , Eli Shamir , and Naftali Tishby . 1997. Selective sampling using the query by committee algorithm. Machine learning , Vol. 28 , 2--3 ( 1997 ), 133--168. Yoav Freund, H Sebastian Seung, Eli Shamir, and Naftali Tishby. 1997. Selective sampling using the query by committee algorithm. Machine learning, Vol. 28, 2--3 (1997), 133--168."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2559636.2559845"},{"key":"e_1_2_1_35_1","volume-title":"A survey of methods for explaining black box models. ACM computing surveys (CSUR)","author":"Guidotti Riccardo","year":"2019","unstructured":"Riccardo Guidotti , Anna Monreale , Salvatore Ruggieri , Franco Turini , Fosca Giannotti , and Dino Pedreschi . 2019. A survey of methods for explaining black box models. ACM computing surveys (CSUR) , Vol. 51 , 5 ( 2019 ), 93. Riccardo Guidotti, Anna Monreale, Salvatore Ruggieri, Franco Turini, Fosca Giannotti, and Dino Pedreschi. 2019. A survey of methods for explaining black box models. ACM computing surveys (CSUR), Vol. 51, 5 (2019), 93."},{"key":"e_1_2_1_36_1","volume-title":"Explainable artificial intelligence (xai)","author":"Gunning David","year":"2017","unstructured":"David Gunning . 2017. Explainable artificial intelligence (xai) . Defense Advanced Research Projects Agency (DARPA) , nd Web, Vol. 2 ( 2017 ). David Gunning. 2017. Explainable artificial intelligence (xai). Defense Advanced Research Projects Agency (DARPA), nd Web, Vol. 2 (2017)."},{"key":"e_1_2_1_37_1","volume-title":"Protodash: Fast interpretable prototype selection. arXiv preprint arXiv:1707.01212","author":"Gurumoorthy Karthik S","year":"2017","unstructured":"Karthik S Gurumoorthy , Amit Dhurandhar , and Guillermo Cecchi . 2017 . Protodash: Fast interpretable prototype selection. arXiv preprint arXiv:1707.01212 (2017). Karthik S Gurumoorthy, Amit Dhurandhar, and Guillermo Cecchi. 2017. Protodash: Fast interpretable prototype selection. arXiv preprint arXiv:1707.01212 (2017)."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025540"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.63.2.308"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300809"},{"key":"e_1_2_1_41_1","unstructured":"Sheng-Jun Huang Rong Jin and Zhi-Hua Zhou. 2010. Active learning by querying informative and representative examples. In Advances in neural information processing systems. 892--900.  Sheng-Jun Huang Rong Jin and Zhi-Hua Zhou. 2010. Active learning by querying informative and representative examples. In Advances in neural information processing systems. 892--900."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753529"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300641"},{"key":"e_1_2_1_44_1","unstructured":"Ronny Kohavi and Barry Becker. 1996. Adult Income dataset (UCI Machine Learning Repository). https:\/\/archive.ics.uci.edu\/ml\/datasets\/Adult  Ronny Kohavi and Barry Becker. 1996. Adult Income dataset (UCI Machine Learning Repository). https:\/\/archive.ics.uci.edu\/ml\/datasets\/Adult"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.31234\/osf.io\/nfc45"},{"key":"e_1_2_1_46_1","volume-title":"INFUSE: interactive feature selection for predictive modeling of high dimensional data","author":"Krause Josua","year":"2014","unstructured":"Josua Krause , Adam Perer , and Enrico Bertini . 2014. INFUSE: interactive feature selection for predictive modeling of high dimensional data . IEEE transactions on visualization and computer graphics, Vol. 20 , 12 ( 2014 ), 1614--1623. Josua Krause, Adam Perer, and Enrico Bertini. 2014. INFUSE: interactive feature selection for predictive modeling of high dimensional data. IEEE transactions on visualization and computer graphics, Vol. 20, 12 (2014), 1614--1623."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2858036.2858529"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2678025.2701399"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2013.6645235"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2030365.2030367"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287560.3287590"},{"key":"e_1_2_1_52_1","volume-title":"SIGIR?94","author":"Lewis David D","unstructured":"David D Lewis and William A Gale . 1994 a. A sequential algorithm for training text classifiers . In SIGIR?94 . Springer , 3--12. David D Lewis and William A Gale. 1994 a. A sequential algorithm for training text classifiers. In SIGIR?94. Springer, 3--12."},{"key":"e_1_2_1_53_1","volume-title":"SIGIR?94","author":"Lewis David D","unstructured":"David D Lewis and William A Gale . 1994 b. A sequential algorithm for training text classifiers . In SIGIR?94 . Springer , 3--12. David D Lewis and William A Gale. 1994 b. A sequential algorithm for training text classifiers. In SIGIR?94. Springer, 3--12."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1080\/10447319509526110"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376590"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/1864349.1864353"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1519023"},{"key":"e_1_2_1_58_1","volume-title":"IUI Workshops.","author":"Lim Brian Y","year":"2019","unstructured":"Brian Y Lim , Qian Yang , Ashraf M Abdul , and Danding Wang . 2019 . Why these Explanations? Selecting Intelligibility Types for Explanation Goals .. In IUI Workshops. Brian Y Lim, Qian Yang, Ashraf M Abdul, and Danding Wang. 2019. Why these Explanations? Selecting Intelligibility Types for Explanation Goals.. In IUI Workshops."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236386.3241340"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-74958-5_63"},{"key":"e_1_2_1_61_1","volume-title":"The Oxford Handbook of Thinking and Reasoning","author":"Lombrozo Tania","unstructured":"Tania Lombrozo . 2012. Explanation and Abductive Inference . In The Oxford Handbook of Thinking and Reasoning . Oxford University Press . Tania Lombrozo. 2012. Explanation and Abductive Inference. In The Oxford Handbook of Thinking and Reasoning. Oxford University Press."},{"key":"e_1_2_1_62_1","unstructured":"Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems. 4765--4774.  Scott M Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Advances in Neural Information Processing Systems. 4765--4774."},{"key":"e_1_2_1_63_1","volume-title":"Developing and validating trust measures for e-commerce: An integrative typology. Information systems research","author":"McKnight D Harrison","year":"2002","unstructured":"D Harrison McKnight , Vivek Choudhury , and Charles Kacmar . 2002. Developing and validating trust measures for e-commerce: An integrative typology. Information systems research , Vol. 13 , 3 ( 2002 ), 334--359. D Harrison McKnight, Vivek Choudhury, and Charles Kacmar. 2002. Developing and validating trust measures for e-commerce: An integrative typology. Information systems research, Vol. 13, 3 (2002), 334--359."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.2307\/259290"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1098-237X(199704)81:2<173::AID-SCE4>3.0.CO;2-C"},{"key":"e_1_2_1_66_1","volume-title":"A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems. arXiv","author":"Mohseni Sina","year":"2018","unstructured":"Sina Mohseni , Niloofar Zarei , and Eric D Ragan . 2018. A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems. arXiv ( 2018 ), arXiv--1811. Sina Mohseni, Niloofar Zarei, and Eric D Ragan. 2018. A Multidisciplinary Survey and Framework for Design and Evaluation of Explainable AI Systems. arXiv (2018), arXiv--1811."},{"key":"e_1_2_1_67_1","volume-title":"How do humans understand explanations from machine learning systems? an evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1802.00682","author":"Narayanan Menaka","year":"2018","unstructured":"Menaka Narayanan , Emily Chen , Jeffrey He , Been Kim , Sam Gershman , and Finale Doshi-Velez . 2018. How do humans understand explanations from machine learning systems? an evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1802.00682 ( 2018 ). Menaka Narayanan, Emily Chen, Jeffrey He, Been Kim, Sam Gershman, and Finale Doshi-Velez. 2018. How do humans understand explanations from machine learning systems? an evaluation of the human-interpretability of explanation. arXiv preprint arXiv:1802.00682 (2018)."},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2018.01.004"},{"key":"e_1_2_1_69_1","volume-title":"Jennifer Wortman Vaughan, and Hanna Wallach","author":"Poursabzi-Sangdeh Forough","year":"2018","unstructured":"Forough Poursabzi-Sangdeh , Daniel G Goldstein , Jake M Hofman , Jennifer Wortman Vaughan, and Hanna Wallach . 2018 . Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018). Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2018. Manipulating and measuring model interpretability. arXiv preprint arXiv:1802.07810 (2018)."},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.5555\/1248547.1248608"},{"key":"e_1_2_1_71_1","volume-title":"Squares: Supporting interactive performance analysis for multiclass classifiers","author":"Ren Donghao","year":"2016","unstructured":"Donghao Ren , Saleema Amershi , Bongshin Lee , Jina Suh , and Jason D Williams . 2016 . Squares: Supporting interactive performance analysis for multiclass classifiers . IEEE transactions on visualization and computer graphics, Vol. 23 , 1 (2016), 61--70. Donghao Ren, Saleema Amershi, Bongshin Lee, Jina Suh, and Jason D Williams. 2016. Squares: Supporting interactive performance analysis for multiclass classifiers. IEEE transactions on visualization and computer graphics, Vol. 23, 1 (2016), 61--70."},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.11491"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-019-09408-y"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/1719970.1720006"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.1145\/1957656.1957752"},{"key":"e_1_2_1_78_1","volume-title":"Proceedings of the conference on empirical methods in natural language processing. Association for Computational Linguistics, 1467--1478","author":"Settles Burr","year":"2011","unstructured":"Burr Settles . 2011 . Closing the loop: Fast, interactive semi-supervised annotation with queries on features and instances . In Proceedings of the conference on empirical methods in natural language processing. Association for Computational Linguistics, 1467--1478 . Burr Settles. 2011. Closing the loop: Fast, interactive semi-supervised annotation with queries on features and instances. In Proceedings of the conference on empirical methods in natural language processing. Association for Computational Linguistics, 1467--1478."},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.5555\/1613715.1613855"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.1145\/130385.130417"},{"key":"e_1_2_1_81_1","doi-asserted-by":"publisher","DOI":"10.5555\/2831071.2831072"},{"key":"e_1_2_1_82_1","volume-title":"Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, et almbox.","author":"Simard Patrice Y","year":"2017","unstructured":"Patrice Y Simard , Saleema Amershi , David M Chickering , Alicia Edelman Pelton , Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, et almbox. 2017 . Machine teaching: A new paradigm for building machine learning systems. arXiv preprint arXiv:1707.06742 (2017). Patrice Y Simard, Saleema Amershi, David M Chickering, Alicia Edelman Pelton, Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, et almbox. 2017. Machine teaching: A new paradigm for building machine learning systems. arXiv preprint arXiv:1707.06742 (2017)."},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1145\/3172944.3172965"},{"key":"e_1_2_1_84_1","unstructured":"Alison Smith-Renner Ron Fan Melissa Birchfield Tongshuang Wu Jordan Boyd-Graber Dan Weld and Leah Findlater. [n.d.]. No Explainability without Accountability: An Empirical Study of Explanations and Feedback in Interactive ML. ([n. d.]).  Alison Smith-Renner Ron Fan Melissa Birchfield Tongshuang Wu Jordan Boyd-Graber Dan Weld and Leah Findlater. [n.d.]. No Explainability without Accountability: An Empirical Study of Explanations and Feedback in Interactive ML. ([n. d.])."},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372870"},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.knosys.2018.07.010"},{"key":"e_1_2_1_87_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302322"},{"key":"e_1_2_1_88_1","volume-title":"Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) .","author":"Stumpf Simone","year":"2016","unstructured":"Simone Stumpf , Adrian Bussone , and Dympna O?sullivan. 2016 . Explanations considered harmful? user interactions with machine learning systems . In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) . Simone Stumpf, Adrian Bussone, and Dympna O?sullivan. 2016. Explanations considered harmful? user interactions with machine learning systems. In Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems (CHI) ."},{"key":"e_1_2_1_89_1","doi-asserted-by":"publisher","DOI":"10.1145\/1216295.1216316"},{"key":"e_1_2_1_90_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2009.03.004"},{"key":"e_1_2_1_91_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518895"},{"key":"e_1_2_1_92_1","volume-title":"arXiv preprint arXiv:1805.08578","author":"Teso Stefano","year":"2018","unstructured":"Stefano Teso and Kristian Kersting . 2018. \" Why Should I Trust Interactive Learners?\" Explaining Interactive Queries of Classifiers to Users . arXiv preprint arXiv:1805.08578 ( 2018 ). Stefano Teso and Kristian Kersting. 2018. \"Why Should I Trust Interactive Learners?\" Explaining Interactive Queries of Classifiers to Users. arXiv preprint arXiv:1805.08578 (2018)."},{"key":"e_1_2_1_93_1","volume-title":"Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. arXiv preprint arXiv:1806.07552","author":"Tomsett Richard","year":"2018","unstructured":"Richard Tomsett , Dave Braines , Dan Harborne , Alun Preece , and Supriyo Chakraborty . 2018. Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. arXiv preprint arXiv:1806.07552 ( 2018 ). Richard Tomsett, Dave Braines, Dan Harborne, Alun Preece, and Supriyo Chakraborty. 2018. Interpretable to whom? A role-based model for analyzing interpretable machine learning systems. arXiv preprint arXiv:1806.07552 (2018)."},{"key":"e_1_2_1_94_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300831"},{"key":"e_1_2_1_95_1","volume-title":"Theory of mind for learning and teaching: The nature and role of explanation. Cognitive development","author":"Wellman Henry M","year":"2004","unstructured":"Henry M Wellman and Kristin H Lagattuta . 2004. Theory of mind for learning and teaching: The nature and role of explanation. Cognitive development , Vol. 19 , 4 ( 2004 ), 479--497. Henry M Wellman and Kristin H Lagattuta. 2004. Theory of mind for learning and teaching: The nature and role of explanation. Cognitive development, Vol. 19, 4 (2004), 479--497."},{"key":"e_1_2_1_96_1","volume-title":"The What-If Tool: Interactive probing of machine learning models","author":"Wexler James","year":"2019","unstructured":"James Wexler , Mahima Pushkarna , Tolga Bolukbasi , Martin Wattenberg , Fernanda Vi\u00e9gas , and Jimbo Wilson . 2019. The What-If Tool: Interactive probing of machine learning models . IEEE transactions on visualization and computer graphics, Vol. 26 , 1 ( 2019 ), 56--65. James Wexler, Mahima Pushkarna, Tolga Bolukbasi, Martin Wattenberg, Fernanda Vi\u00e9gas, and Jimbo Wilson. 2019. The What-If Tool: Interactive probing of machine learning models. IEEE transactions on visualization and computer graphics, Vol. 26, 1 (2019), 56--65."},{"key":"e_1_2_1_97_1","doi-asserted-by":"publisher","DOI":"10.1145\/3319616"},{"key":"e_1_2_1_98_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2018.06.004"},{"key":"e_1_2_1_99_1","volume-title":"Manifold: A model-agnostic framework for interpretation and diagnosis of machine learning models","author":"Zhang Jiawei","year":"2018","unstructured":"Jiawei Zhang , Yang Wang , Piero Molino , Lezhi Li , and David S Ebert . 2018 . Manifold: A model-agnostic framework for interpretation and diagnosis of machine learning models . IEEE transactions on visualization and computer graphics, Vol. 25 , 1 (2018), 364--373. Jiawei Zhang, Yang Wang, Piero Molino, Lezhi Li, and David S Ebert. 2018. Manifold: A model-agnostic framework for interpretation and diagnosis of machine learning models. IEEE transactions on visualization and computer graphics, Vol. 25, 1 (2018), 364--373."},{"key":"e_1_2_1_100_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372852"},{"key":"e_1_2_1_101_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2002.1184002"}],"container-title":["Proceedings of the ACM on Human-Computer Interaction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3432934","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3432934","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:47:12Z","timestamp":1750193232000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3432934"}},"subtitle":["Toward AI Explanations as Interfaces for Machine Teachers"],"short-title":[],"issued":{"date-parts":[[2021,1,5]]},"references-count":100,"journal-issue":{"issue":"CSCW3","published-print":{"date-parts":[[2021,1,5]]}},"alternative-id":["10.1145\/3432934"],"URL":"https:\/\/doi.org\/10.1145\/3432934","relation":{},"ISSN":["2573-0142"],"issn-type":[{"value":"2573-0142","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,5]]},"assertion":[{"value":"2021-01-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}