{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,19]],"date-time":"2026-01-19T10:09:51Z","timestamp":1768817391635,"version":"3.49.0"},"reference-count":91,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,12,8]],"date-time":"2023-12-08T00:00:00Z","timestamp":1701993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"National Science Foundation Graduate Research Fellowship","award":["DGE-1762114, ONR N00014-18-1-2193, NSF RAPID 2040196"],"award-info":[{"award-number":["DGE-1762114, ONR N00014-18-1-2193, NSF RAPID 2040196"]}]},{"name":"WRF\/Cable Professorship, and the Allen Institute for Artificial Intelligence"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2023,12,31]]},"abstract":"<jats:p>\n            Research in human-centered AI has shown the benefits of systems that can explain their predictions. Methods that allow AI to take advice from humans in response to explanations are similarly useful. While both capabilities are well developed for\n            <jats:italic>transparent<\/jats:italic>\n            learning models (e.g., linear models and GA\n            <jats:sup>2<\/jats:sup>\n            Ms) and recent techniques (e.g., LIME and SHAP) can generate explanations for\n            <jats:italic>opaque<\/jats:italic>\n            models, little attention has been given to advice methods for opaque models. This article introduces LIMEADE, the first general framework that translates both positive and negative advice (expressed using high-level vocabulary such as that employed by post hoc explanations) into an update to an arbitrary, underlying opaque model. We demonstrate the generality of our approach with case studies on 70 real-world models across two broad domains: image classification and text recommendation. We show that our method improves accuracy compared to a rigorous baseline on the image classification domains. For the text modality, we apply our framework to a neural recommender system for scientific papers on a public website; our user study shows that our framework leads to significantly higher perceived user control, trust, and satisfaction.\n          <\/jats:p>","DOI":"10.1145\/3589345","type":"journal-article","created":{"date-parts":[[2023,3,28]],"date-time":"2023-03-28T11:57:15Z","timestamp":1680004635000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["LIMEADE: From AI Explanations to Advice Taking"],"prefix":"10.1145","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1677-6386","authenticated-orcid":false,"given":"Benjamin Charles Germain","family":"Lee","sequence":"first","affiliation":[{"name":"University of Washington &amp; Allen Institute for Artificial Intelligence, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4737-8444","authenticated-orcid":false,"given":"Doug","family":"Downey","sequence":"additional","affiliation":[{"name":"Northwestern University &amp; Allen Institute for Artificial Intelligence, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1804-2853","authenticated-orcid":false,"given":"Kyle","family":"Lo","sequence":"additional","affiliation":[{"name":"Allen Institute for Artificial Intelligence, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3255-0109","authenticated-orcid":false,"given":"Daniel S.","family":"Weld","sequence":"additional","affiliation":[{"name":"University of Washington &amp; Allen Institute for Artificial Intelligence, USA"}]}],"member":"320","published-online":{"date-parts":[[2023,12,8]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"11","volume-title":"Proceedings of the 16th International Conference on World Wide Web (WWW\u201907)","author":"Ahn Jae-wook","year":"2007","unstructured":"Jae-wook Ahn, Peter Brusilovsky, Jonathan Grady, Daqing He, and Sue Yeon Syn. 2007. Open user profiles for adaptive news systems: Help or harm? In Proceedings of the 16th International Conference on World Wide Web (WWW\u201907). ACM, New York, NY, 11\u201320. 10.1145\/1242572.1242575"},{"key":"e_1_3_2_3_2","first-page":"202","volume-title":"Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI\u201915)","author":"Ahn Jae-wook","year":"2015","unstructured":"Jae-wook Ahn, Peter Brusilovsky, and Shuguang Han. 2015. Personalized search: Reconsidering the value of open user models. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI\u201915). ACM, New York, NY, 202\u2013212. 10.1145\/2678025.2701410"},{"key":"e_1_3_2_4_2","doi-asserted-by":"crossref","first-page":"2927","DOI":"10.1109\/CVPR.2015.7298911","volume-title":"2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201915)","author":"Akata Zeynep","year":"2015","unstructured":"Zeynep Akata, Scott Reed, Daniel Walter, Honglak Lee, and Bernt Schiele. 2015. Evaluation of output embeddings for fine-grained image classification. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201915). 2927\u20132936. 10.1109\/CVPR.2015.7298911"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v35i4.2513"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1145\/2207676.2207680","volume-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI\u201912)","author":"Amershi Saleema","year":"2012","unstructured":"Saleema Amershi, James Fogarty, and Daniel Weld. 2012. Regroup: Interactive machine learning for on-demand group creation in social networks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI\u201912). ACM, New York, NY, 21\u201330. 10.1145\/2207676.2207680"},{"key":"e_1_3_2_7_2","volume-title":"Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI\u201919)","author":"Amershi Saleema","year":"2019","unstructured":"Saleema Amershi, Dan Weld, Mihaela Vorvoreanu, Adam Fourney, Besmira Nushi, Penny Collisson, Jina Suh, Shamsi Iqbal, Paul N. Bennett, Kori Inkpen, Jaime Teevan, Ruth Kikin-Gil, and Eric Horvitz. 2019. Guidelines for human-AI interaction. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI\u201919). Association for Computing Machinery, New York, NY, Article 3, 13 pages. 10.1145\/3290605.3300233"},{"key":"e_1_3_2_8_2","first-page":"84","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers)","author":"Ammar Waleed","year":"2018","unstructured":"Waleed Ammar, Dirk Groeneveld, Chandra Bhagavatula, Iz Beltagy, Miles Crawford, Doug Downey, Jason Dunkelberger, Ahmed Elgohary, Sergey Feldman, Vu Ha, Rodney Kinney, Sebastian Kohlmeier, Kyle Lo, Tyler Murray, Hsu-Han Ooi, Matthew Peters, Joanna Power, Sam Skjonsberg, Lucy Lu Wang, Chris Wilhelm, Zheng Yuan, Madeleine van Zuylen, and Oren Etzioni. 2018. Construction of the literature graph in semantic scholar. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 3 (Industry Papers). Association for Computational Linguistics, 84\u201391. 10.18653\/v1\/N18-3011"},{"key":"e_1_3_2_9_2","first-page":"49","volume-title":"Proceedings of the 2013 International Conference on Intelligent User Interfaces (IUI\u201913)","author":"Bakalov Fedor","year":"2013","unstructured":"Fedor Bakalov, Marie-Jean Meurs, Birgitta K\u00f6nig-Ries, Bahar Sateli, Ren\u00e9 Witte, Greg Butler, and Adrian Tsang. 2013. An approach to controlling user models and personalization effects in recommender systems. In Proceedings of the 2013 International Conference on Intelligent User Interfaces (IUI\u201913). ACM, New York, NY, 49\u201356. 10.1145\/2449396.2449405"},{"key":"e_1_3_2_10_2","first-page":"351","volume-title":"Computer Vision (ECCV\u201920)","author":"Bau David","year":"2020","unstructured":"David Bau, Steven Liu, Tongzhou Wang, Jun-Yan Zhu, and Antonio Torralba. 2020. Rewriting a deep generative model. In Computer Vision (ECCV\u201920), Andrea Vedaldi, Horst Bischof, Thomas Brox, and Jan-Michael Frahm (Eds.). Springer International Publishing, Cham, 351\u2013369."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00799-015-0156-0"},{"key":"e_1_3_2_12_2","first-page":"3615","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP\u201919)","author":"Beltagy Iz","year":"2019","unstructured":"Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciBERT: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP\u201919). Association for Computational Linguistics, 3615\u20133620. 10.18653\/v1\/D19-1371"},{"key":"e_1_3_2_13_2","first-page":"238","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Bhagavatula Chandra","year":"2018","unstructured":"Chandra Bhagavatula, Sergey Feldman, Russell Power, and Waleed Ammar. 2018. Content-based citation recommendation. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). Association for Computational Linguistics, 238\u2013251. 10.18653\/v1\/N18-1022"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1145\/2365952.2365964","volume-title":"Proceedings of the 6th ACM Conference on Recommender Systems (RecSys\u201912)","author":"Bostandjiev Svetlin","year":"2012","unstructured":"Svetlin Bostandjiev, John O\u2019Donovan, and Tobias H\u00f6llerer. 2012. TasteWeights: A visual interactive hybrid recommender system. In Proceedings of the 6th ACM Conference on Recommender Systems (RecSys\u201912). ACM, New York, NY, 35\u201342. 10.1145\/2365952.2365964"},{"key":"e_1_3_2_15_2","first-page":"107","volume-title":"Proceedings of the 2013 International Conference on Intelligent User Interfaces (IUI\u201913)","author":"Bostandjiev Svetlin","year":"2013","unstructured":"Svetlin Bostandjiev, John O\u2019Donovan, and Tobias H\u00f6llerer. 2013. LinkedVis: Exploring social and semantic career recommendations. In Proceedings of the 2013 International Conference on Intelligent User Interfaces (IUI\u201913). ACM Press, 107. 10.1145\/2449396.2449412"},{"key":"e_1_3_2_16_2","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/978-3-319-20618-9_9","volume-title":"Human Interface and the Management of Information. Information and Knowledge in Context","author":"Bruns Simon","year":"2015","unstructured":"Simon Bruns, Andr\u00e9 Calero Valdez, Christoph Greven, Martina Ziefle, and Ulrik Schroeder. 2015. What should I read next? A personalized visual publication recommender system. In Human Interface and the Management of Information. Information and Knowledge in Context, Sakae Yamamoto (Ed.). Springer International Publishing, Cham, 89\u2013100."},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1145\/3383313.3411539","volume-title":"Proceedings of the 14th ACM Conference on Recommender Systems (RecSys\u201920)","author":"Brusilovsky Peter","year":"2020","unstructured":"Peter Brusilovsky, Marco de Gemmis, Alexander Felfernig, Pasquale Lops, John O\u2019Donovan, Giovanni Semeraro, and Martijn C. Willemsen. 2020. Interfaces and human decision making for recommender systems. In Proceedings of the 14th ACM Conference on Recommender Systems (RecSys\u201920). Association for Computing Machinery, New York, NY, 613\u2013618. 10.1145\/3383313.3411539"},{"key":"e_1_3_2_18_2","doi-asserted-by":"crossref","first-page":"1721","DOI":"10.1145\/2783258.2788613","volume-title":"Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD\u201915)","author":"Caruana Rich","year":"2015","unstructured":"Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD\u201915). Association for Computing Machinery, New York, NY, 1721\u20131730. 10.1145\/2783258.2788613"},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","first-page":"2270","DOI":"10.18653\/v1\/2020.acl-main.207","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Cohan Arman","year":"2020","unstructured":"Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel Weld. 2020. SPECTER: Document-level representation learning using citation-informed transformers. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 2270\u20132282. 10.18653\/v1\/2020.acl-main.207"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11257-008-9051-3"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2013.08.003"},{"key":"e_1_3_2_22_2","series-title":"Proceedings of the 36th International Conference on Machine Learning","first-page":"1547","volume":"97","author":"Dasgupta Sanjoy","year":"2019","unstructured":"Sanjoy Dasgupta, Daniel Hsu, Stefanos Poulis, and Xiaojin Zhu. 2019. Teaching a black-box learner. In Proceedings of the 36th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, Long Beach, CA, 1547\u20131555. http:\/\/proceedings.mlr.press\/v97\/dasgupta19a.html."},{"key":"e_1_3_2_23_2","doi-asserted-by":"crossref","first-page":"248","DOI":"10.1109\/CVPR.2009.5206848","volume-title":"2009 IEEE Conference on Computer Vision and Pattern Recognition","author":"Deng Jia","year":"2009","unstructured":"Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition. 248\u2013255. 10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_24_2","article-title":"Towards a rigorous science of interpretable machine learning","author":"Doshi-Velez Finale","year":"2017","unstructured":"Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv:1702.08608 [cs, stat] (March2017). http:\/\/arxiv.org\/abs\/1702.08608. arXiv: 1702.08608.","journal-title":"arXiv:1702.08608 [cs, stat]"},{"key":"e_1_3_2_25_2","first-page":"595","volume-title":"Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR\u201908)","author":"Druck Gregory","year":"2008","unstructured":"Gregory Druck, Gideon Mann, and Andrew McCallum. 2008. Learning from labeled features using generalized expectation criteria. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR\u201908). Association for Computing Machinery, New York, NY, 595\u2013602. 10.1145\/1390334.1390436"},{"key":"e_1_3_2_26_2","first-page":"159","volume-title":"2010 ACM Conference on Recommender Systems (RecSys\u201910)","author":"Ekstrand Michael","year":"2010","unstructured":"Michael Ekstrand, Praveen Kannan, James Stemper, John Butler, Joseph Konstan, and John Riedl. 2010. Automatically building research reading lists. In 2010 ACM Conference on Recommender Systems (RecSys\u201910), ACM, 159\u2013166. 10.1145\/1864708.1864740"},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1007\/978-3-540-24775-3_5","volume-title":"Advances in Knowledge Discovery and Data Mining","author":"Godbole Shantanu","year":"2004","unstructured":"Shantanu Godbole and Sunita Sarawagi. 2004. Discriminative methods for multi-labeled classification. In Advances in Knowledge Discovery and Data Mining, Honghua Dai, Ramakrishnan Srikant, and Chengqi Zhang (Eds.). Springer, Berlin, 22\u201330."},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-8659.2009.01679.x"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236009"},{"key":"e_1_3_2_30_2","first-page":"3","volume-title":"Proceedings of the 9th ACM Conference on Recommender Systems (RecSys\u201915)","author":"Harper F. Maxwell","year":"2015","unstructured":"F. Maxwell Harper, Funing Xu, Harmanpreet Kaur, Kyle Condiff, Shuo Chang, and Loren Terveen. 2015. Putting users in control of their recommendations. In Proceedings of the 9th ACM Conference on Recommender Systems (RecSys\u201915). ACM Press, 3\u201310. 10.1145\/2792838.2800179"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2016.02.013"},{"key":"e_1_3_2_32_2","first-page":"770","volume-title":"2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201916)","author":"He Kaiming","year":"2016","unstructured":"Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR\u201916). 770\u2013778. 10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_33_2","first-page":"421","volume-title":"Proceedings of the 19th International Conference on World Wide Web (WWW\u201910)","author":"He Qi","year":"2010","unstructured":"Qi He, Jian Pei, Daniel Kifer, Prasenjit Mitra, and Lee Giles. 2010. Context-aware citation recommendation. In Proceedings of the 19th International Conference on World Wide Web (WWW\u201910). ACM, New York, NY, 421\u2013430. 10.1145\/1772690.1772734"},{"key":"e_1_3_2_34_2","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1145\/358916.358995","volume-title":"Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (CSCW\u201900)","author":"Herlocker Jonathan L.","year":"2000","unstructured":"Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (CSCW\u201900). ACM Press, 241\u2013250. 10.1145\/358916.358995"},{"issue":"2","key":"e_1_3_2_35_2","first-page":"65","article-title":"A simple sequentially rejective multiple test procedure","volume":"6","author":"Holm Sture","year":"1979","unstructured":"Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6, 2 (1979), 65\u201370. http:\/\/www.jstor.org\/stable\/4615733.","journal-title":"Scandinavian Journal of Statistics"},{"key":"e_1_3_2_36_2","first-page":"13","volume-title":"Proceedings of the 12th ACM Conference on Recommender Systems (RecSys\u201918","author":"Jin Yucheng","year":"2018","unstructured":"Yucheng Jin, Nava Tintarev, and Katrien Verbert. 2018. Effects of personal characteristics on music recommender systems with different levels of controllability. In Proceedings of the 12th ACM Conference on Recommender Systems (RecSys\u201918). ACM Press, 13\u201321. 10.1145\/3240323.3240358"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i11.21488"},{"key":"e_1_3_2_38_2","doi-asserted-by":"crossref","first-page":"2893","DOI":"10.1145\/3308558.3313700","volume-title":"The World Wide Web Conference (WWW\u201919)","author":"Kanakia Anshul","year":"2019","unstructured":"Anshul Kanakia, Zhihong Shen, Darrin Eide, and Kuansan Wang. 2019. A scalable hybrid research paper recommender system for Microsoft academic. In The World Wide Web Conference (WWW\u201919). ACM, New York, NY, 2893\u20132899. 10.1145\/3308558.3313700"},{"key":"e_1_3_2_39_2","first-page":"247","volume-title":"Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI\u201915)","author":"Kangasr\u00e4\u00e4si\u00f6 Antti","year":"2015","unstructured":"Antti Kangasr\u00e4\u00e4si\u00f6, Dorota Glowacka, and Samuel Kaski. 2015. Improving controllability and predictability of interactive recommendation interfaces for exploratory search. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI\u201915). ACM, New York, NY, 247\u2013251. 10.1145\/2678025.2701371"},{"key":"e_1_3_2_40_2","unstructured":"Andrej Karpathy. 2015. Arxiv sanity preserver. http:\/\/www.arxiv-sanity.com\/."},{"key":"e_1_3_2_41_2","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1145\/2043932.2043960","volume-title":"Proceedings of the 5th ACM Conference on Recommender Systems (RecSys\u201911)","author":"Knijnenburg Bart P.","year":"2011","unstructured":"Bart P. Knijnenburg, Niels J. M. Reijmer, and Martijn C. Willemsen. 2011. Each to his own: How different users call for different interaction methods in recommender systems. In Proceedings of the 5th ACM Conference on Recommender Systems (RecSys\u201911). ACM, New York, NY, 141\u2013148. 10.1145\/2043932.2043960"},{"key":"e_1_3_2_42_2","article-title":"Concept bottleneck models","volume":"2007","author":"Koh Pang Wei","year":"2020","unstructured":"Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. 2020. Concept bottleneck models. ArXiv abs\/2007.04612 (2020).","journal-title":"ArXiv"},{"key":"e_1_3_2_43_2","first-page":"126","volume-title":"Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI\u201915)","author":"Kulesza Todd","year":"2015","unstructured":"Todd Kulesza, Margaret Burnett, Weng-Keen Wong, and Simone Stumpf. 2015. Principles of explanatory debugging to personalize interactive machine learning. In Proceedings of the 20th International Conference on Intelligent User Interfaces (IUI\u201915). ACM, New York, NY, 126\u2013137. 10.1145\/2678025.2701399"},{"key":"e_1_3_2_44_2","first-page":"1","volume-title":"Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems (CHI\u201912","author":"Kulesza Todd","year":"2012","unstructured":"Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012. Tell me more?: The effects of mental model soundness on personalizing an intelligent agent. In Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems (CHI\u201912). ACM Press, 1. 10.1145\/2207676.2207678"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2013.6645235"},{"key":"e_1_3_2_46_2","unstructured":"Tsung-Yi Lin Michael Maire Serge Belongie Lubomir Bourdev Ross Girshick James Hays Pietro Perona Deva Ramanan C. Lawrence Zitnick and Piotr Doll\u00e1r. 2014. Microsoft COCO: Common Objects in Context. http:\/\/arxiv.org\/abs\/1405.0312."},{"key":"e_1_3_2_47_2","first-page":"425","volume-title":"Proceedings of the 19th National Conference on Artifical Intelligence (AAAI\u201904)","author":"Liu Bing","year":"2004","unstructured":"Bing Liu, Xiaoli Li, Wee Sun Lee, and Philip S. Yu. 2004. Text classification by labeling words. In Proceedings of the 19th National Conference on Artifical Intelligence (AAAI\u201904). AAAI Press, 425\u2013430."},{"key":"e_1_3_2_48_2","first-page":"6274","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Liu Frederick","year":"2019","unstructured":"Frederick Liu and Besim Avci. 2019. Incorporating priors with feature attribution on text classification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 6274\u20136283. 10.18653\/v1\/P19-1631"},{"key":"e_1_3_2_49_2","first-page":"3","volume-title":"Proceedings of the 1st Workshop on Engineering Computer-Human Interaction in Recommender Systems (EnCHIReS\u201916)","author":"Loepp Benedikt","year":"2016","unstructured":"Benedikt Loepp, Catalin-Mihai Barbu, and J\u00fcrgen Ziegler. 2016. Interactive recommending: Framework, state of research and future challenges. In Proceedings of the 1st Workshop on Engineering Computer-Human Interaction in Recommender Systems (EnCHIReS\u201916). 3\u201313. http:\/\/ceur-ws.org\/Vol-1705\/02-paper.pdf."},{"key":"e_1_3_2_50_2","volume-title":"In Proceedings of the ACL Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics.","author":"Loper Edward","year":"2002","unstructured":"Edward Loper and Steven Bird. 2002. NLTK: The natural language toolkit. In In Proceedings of the ACL Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics..Association for Computational Linguistics."},{"key":"e_1_3_2_51_2","first-page":"150","volume-title":"Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD\u201912)","author":"Lou Yin","year":"2012","unstructured":"Yin Lou, Rich Caruana, and Johannes Gehrke. 2012. Intelligible models for classification and regression. In Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD\u201912). Association for Computing Machinery, New York, NY, 150\u2013158. 10.1145\/2339530.2339556"},{"key":"e_1_3_2_52_2","first-page":"623","volume-title":"Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD\u201913)","author":"Lou Yin","year":"2013","unstructured":"Yin Lou, Rich Caruana, Johannes Gehrke, and Giles Hooker. 2013. Accurate intelligible models with pairwise interactions. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD\u201913). Association for Computing Machinery, New York, NY, 623\u2013631. 10.1145\/2487575.2487579"},{"key":"e_1_3_2_53_2","first-page":"4768","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS\u201917)","author":"Lundberg Scott M.","year":"2017","unstructured":"Scott M. Lundberg and Su-In Lee. 2017. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS\u201917). Curran Associates Inc., Red Hook, NY, 4768\u20134777."},{"key":"e_1_3_2_54_2","first-page":"403","volume-title":"Programs with Common Sense","author":"McCarthy John","year":"1968","unstructured":"John McCarthy. 1968. Programs with Common Sense. 403\u2013418."},{"key":"e_1_3_2_55_2","first-page":"17153","volume-title":"Advances in Neural Information Processing Systems","author":"Mu Jesse","year":"2020","unstructured":"Jesse Mu and Jacob Andreas. 2020. Compositional explanations of neurons. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M. F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 17153\u201317163. https:\/\/proceedings.neurips.cc\/paper\/2020\/file\/c74956ffb38ba48ed6ce977af6727275-Paper.pdf."},{"key":"e_1_3_2_56_2","doi-asserted-by":"crossref","first-page":"1085","DOI":"10.1145\/1357054.1357222","volume-title":"Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI\u201908)","author":"O\u2019Donovan John","year":"2008","unstructured":"John O\u2019Donovan, Barry Smyth, Brynjar Gretarsson, Svetlin Bostandjiev, and Tobias H\u00f6llerer. 2008. PeerChooser: Visual interactive recommendation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI\u201908). ACM, New York, NY, 1085\u20131088. 10.1145\/1357054.1357222"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2015.01.007"},{"key":"e_1_3_2_58_2","first-page":"2825","article-title":"Scikit-learn: Machine learning in python","volume":"12","author":"Pedregosa F.","year":"2011","unstructured":"F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine learning in python. Journal of Machine Learning Research 12 (2011), 2825\u20132830.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_59_2","first-page":"93","volume-title":"Proceedings of the 11th International Conference on Intelligent User Interfaces (IUI\u201906)","author":"Pu Pearl","year":"2006","unstructured":"Pearl Pu and Li Chen. 2006. Trust building with explanation interfaces. In Proceedings of the 11th International Conference on Intelligent User Interfaces (IUI\u201906). ACM, New York, NY, 93\u2013100. 10.1145\/1111449.1111475"},{"key":"e_1_3_2_60_2","first-page":"157","volume-title":"Proceedings of the 5th ACM Conference on Recommender Systems (RecSys\u201911)","author":"Pu Pearl","year":"2011","unstructured":"Pearl Pu, Li Chen, and Rong Hu. 2011. A user-centric evaluation framework for recommender systems. In Proceedings of the 5th ACM Conference on Recommender Systems (RecSys\u201911). ACM, New York, NY, 157\u2013164. 10.1145\/2043932.2043962"},{"key":"e_1_3_2_61_2","volume-title":"R: A Language and Environment for Statistical Computing","author":"Team R Core","year":"2018","unstructured":"R Core Team. 2018. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria. https:\/\/www.R-project.org\/."},{"key":"e_1_3_2_62_2","volume-title":"Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA\u201922)","author":"Radensky Marissa","year":"2022","unstructured":"Marissa Radensky, Doug Downey, Kyle Lo, Zoran Popovic, and Daniel S. Weld. 2022. Exploring the role of local and global explanations in recommender systems. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA\u201922). Association for Computing Machinery, New York, NY, Article 290, 7 pages. 10.1145\/3491101.3519795"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/1277741.1277758"},{"key":"e_1_3_2_64_2","doi-asserted-by":"crossref","first-page":"1135","DOI":"10.1145\/2939672.2939778","volume-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD\u201916)","author":"Ribeiro Marco Tulio","year":"2016","unstructured":"Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. \u201cWhy should i trust you?\u201d: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD\u201916). ACM, New York, NY, 1135\u20131144. 10.1145\/2939672.2939778"},{"key":"e_1_3_2_65_2","unstructured":"Laura Rieger Chandan Singh W. James Murdoch and Bin Yu. 2019. Interpretations are useful: penalizing explanations to align neural networks with prior knowledge. arxiv:1909.13584 [cs.LG]"},{"key":"e_1_3_2_66_2","first-page":"259","volume-title":"Proceedings of the 15th International Conference on Intelligent User Interfaces (IUI\u201910)","author":"Rosenthal Stephanie L.","year":"2010","unstructured":"Stephanie L. Rosenthal and Anind K. Dey. 2010. Towards maximizing the accuracy of human-labeled sensor data. In Proceedings of the 15th International Conference on Intelligent User Interfaces (IUI\u201910). Association for Computing Machinery, New York, NY, 259\u2013268. 10.1145\/1719970.1720006"},{"key":"e_1_3_2_67_2","first-page":"2662","volume-title":"Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI\u201917)","author":"Ross Andrew Slavin","year":"2017","unstructured":"Andrew Slavin Ross, Michael C. Hughes, and Finale Doshi-Velez. 2017. Right for the right reasons: Training differentiable models by constraining their explanations. In Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI\u201917). AAAI Press, 2662\u20132670."},{"key":"e_1_3_2_68_2","unstructured":"James Schaffer Tobias H\u00f6llerer and John O\u2019Donovan. 2015. Hypothetical Recommendation: A Study of Interactive Profile Manipulation Behavior for Recommender Systems. https:\/\/www.aaai.org\/ocs\/index.php\/FLAIRS\/FLAIRS15\/paper\/view\/10444."},{"key":"e_1_3_2_69_2","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1109\/TSA.2004.840937","article-title":"Boosting with prior knowledge for call classification","volume":"13","author":"Schapire Robert E.","year":"2005","unstructured":"Robert E. Schapire, Marie Rochery, Mazin G. Rahim, and Narendra Kumar Gupta. 2005. Boosting with prior knowledge for call classification. IEEE Transactions on Speech and Audio Processing 13 (2005), 174\u2013181.","journal-title":"IEEE Transactions on Speech and Audio Processing"},{"key":"e_1_3_2_70_2","doi-asserted-by":"crossref","unstructured":"Patrick Schramowski Wolfgang Stammer Stefano Teso Anna Brugger Xiaoting Shao Hans-Georg Luigs Anne-Katrin Mahlein and Kristian Kersting. 2020. Right for the Wrong Scientific Reasons: Revising Deep Networks by Interacting with their Explanations. arxiv:2001.05371 [cs.LG]","DOI":"10.1038\/s42256-020-0212-3"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.2200\/S00429ED1V01Y201207AIM018"},{"key":"e_1_3_2_72_2","article-title":"Machine teaching: A new paradigm for building machine learning systems","author":"Simard Patrice Y.","year":"2017","unstructured":"Patrice Y. Simard, Saleema Amershi, David M. Chickering, Alicia Edelman Pelton, Soroush Ghorashi, Christopher Meek, Gonzalo Ramos, Jina Suh, Johan Verwey, Mo Wang, and John Wernsing. 2017. Machine teaching: A new paradigm for building machine learning systems. arXiv:1707.06742 [cs, stat] (July2017). http:\/\/arxiv.org\/abs\/1707.06742. arXiv: 1707.06742.","journal-title":"arXiv:1707.06742 [cs, stat]"},{"key":"e_1_3_2_73_2","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1145\/2740908.2742839","volume-title":"Proceedings of the 24th International Conference on World Wide Web (WWW\u201915 Companion)","author":"Sinha Arnab","year":"2015","unstructured":"Arnab Sinha, Zhihong Shen, Yang Song, Hao Ma, Darrin Eide, Bo-June (Paul) Hsu, and Kuansan Wang. 2015. An overview of Microsoft academic service (MAS) and applications. In Proceedings of the 24th International Conference on World Wide Web (WWW\u201915 Companion). ACM, New York, NY, 243\u2013246. 10.1145\/2740908.2742839"},{"key":"e_1_3_2_74_2","first-page":"1","volume-title":"Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI\u201920)","author":"Smith-Renner Alison","year":"2020","unstructured":"Alison Smith-Renner, Ron Fan, Melissa Birchfield, Tongshuang Wu, Jordan Boyd-Graber, Daniel S. Weld, and Leah Findlater. 2020. No explainability without accountability: An empirical study of explanations and feedback in interactive ML. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI\u201920). Association for Computing Machinery, New York, NY, 1\u201313. 10.1145\/3313831.3376624"},{"key":"e_1_3_2_75_2","first-page":"132","volume-title":"A Statistical Interpretation of Term Specificity and Its Application in Retrieval","author":"Jones Karen Sparck","year":"1988","unstructured":"Karen Sparck Jones. 1988. A Statistical Interpretation of Term Specificity and Its Application in Retrieval. Taylor Graham Publishing, GBR, 132\u2013142."},{"key":"e_1_3_2_76_2","volume-title":"Reinforcement Learning: An Introduction (2nd ed.)","author":"Sutton Richard S.","year":"2018","unstructured":"Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction (2nd ed.). MIT Press, Cambridge, MA."},{"key":"e_1_3_2_77_2","doi-asserted-by":"crossref","unstructured":"Stefano Teso \u00d6znur Alkan Wolfang Stammer and Elizabeth Daly. 2022. Leveraging Explanations in Interactive Machine Learning: An Overview. 10.48550\/ARXIV.2207.14526","DOI":"10.3389\/frai.2023.1066049"},{"key":"e_1_3_2_78_2","doi-asserted-by":"crossref","first-page":"801","DOI":"10.1109\/ICDEW.2007.4401070","volume-title":"2007 IEEE 23rd International Conference on Data Engineering Workshop","author":"Tintarev Nava","year":"2007","unstructured":"Nava Tintarev and Judith Masthoff. 2007. A survey of explanations in recommender systems. In 2007 IEEE 23rd International Conference on Data Engineering Workshop. 801\u2013810. 10.1109\/ICDEW.2007.4401070"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-85820-3_15"},{"key":"e_1_3_2_80_2","first-page":"239","volume-title":"23rd International Conference on Intelligent User Interfaces (IUI\u201918)","author":"Tsai Chun-Hua","year":"2018","unstructured":"Chun-Hua Tsai and Peter Brusilovsky. 2018. Beyond the ranked list: User-driven exploration and diversification of social recommendation. In 23rd International Conference on Intelligent User Interfaces (IUI\u201918). ACM, New York, NY, 239\u2013250. 10.1145\/3172944.3172959"},{"key":"e_1_3_2_81_2","first-page":"391","volume-title":"Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI\u201919)","author":"Tsai Chun-Hua","year":"2019","unstructured":"Chun-Hua Tsai and Peter Brusilovsky. 2019. Explaining recommendations in an interactive hybrid social recommender. In Proceedings of the 24th International Conference on Intelligent User Interfaces (IUI\u201919). ACM, New York, NY, 391\u2013396. 10.1145\/3301275.3302318"},{"key":"e_1_3_2_82_2","doi-asserted-by":"crossref","first-page":"591","DOI":"10.1007\/s11257-020-09281-5","article-title":"The effects of controllability and explainability in a social recommender system","volume":"31","author":"Tsai Chun-Hua","year":"2021","unstructured":"Chun-Hua Tsai and Peter Brusilovsky. 2021. The effects of controllability and explainability in a social recommender system. User Modeling and User-adapted Interaction 31 (2021), 591\u2013627.","journal-title":"User Modeling and User-adapted Interaction"},{"key":"e_1_3_2_83_2","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc.https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2017\/file\/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf."},{"key":"e_1_3_2_84_2","first-page":"351","volume-title":"Proceedings of the 2013 International Conference on Intelligent User Interfaces (IUI\u201913)","author":"Verbert Katrien","year":"2013","unstructured":"Katrien Verbert, Denis Parra, Peter Brusilovsky, and Erik Duval. 2013. Visualizing recommendations to support exploration, transparency and controllability. In Proceedings of the 2013 International Conference on Intelligent User Interfaces (IUI\u201913). ACM, New York, NY, 351\u2013362. 10.1145\/2449396.2449442"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/2362394.2362395"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1023\/B:USER.0000028984.13876.9b"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00279"},{"key":"e_1_3_2_88_2","article-title":"GAM changer: Editing generalized additive models with interactive visualization","volume":"2112","author":"Wang Zijie Jay","year":"2021","unstructured":"Zijie Jay Wang, Alex Kale, Harsha Nori, Peter Stella, Mark E. Nunnally, Duen Horng Chau, Mihaela Vorvoreanu, Jennifer Wortman Vaughan, and Rich Caruana. 2021. GAM changer: Editing generalized additive models with interactive visualization. ArXiv abs\/2112.03245 (2021).","journal-title":"ArXiv"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1145\/3282486"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1145\/3319616"},{"key":"e_1_3_2_91_2","first-page":"326","volume-title":"Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD\u201904)","author":"Wu Xiaoyun","year":"2004","unstructured":"Xiaoyun Wu and Rohini Srihari. 2004. Incorporating prior knowledge with weighted margin support vector machines. In Proceedings of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD\u201904). Association for Computing Machinery, New York, NY, 326\u2013333. 10.1145\/1014052.1014089"},{"key":"e_1_3_2_92_2","article-title":"Explainable recommendation: A survey and new perspectives","author":"Zhang Yongfeng","year":"2018","unstructured":"Yongfeng Zhang and Xu Chen. 2018. Explainable recommendation: A survey and new perspectives. arXiv:1804.11192 [cs] (April2018). http:\/\/arxiv.org\/abs\/1804.11192. arXiv: 1804.11192.","journal-title":"arXiv:1804.11192 [cs]"}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3589345","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3589345","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:16Z","timestamp":1750291396000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3589345"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,8]]},"references-count":91,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12,31]]}},"alternative-id":["10.1145\/3589345"],"URL":"https:\/\/doi.org\/10.1145\/3589345","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,8]]},"assertion":[{"value":"2022-02-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-03-13","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}