{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T12:20:01Z","timestamp":1776082801739,"version":"3.50.1"},"reference-count":45,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2011,10,1]],"date-time":"2011-10-01T00:00:00Z","timestamp":1317427200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000145","name":"Division of Information and Intelligent Systems","doi-asserted-by":"publisher","award":["IIS-0803487IIS-0917366"],"award-info":[{"award-number":["IIS-0803487IIS-0917366"]}],"id":[{"id":"10.13039\/100000145","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2011,10]]},"abstract":"<jats:p>\n            Machine learning techniques are increasingly used in\n            <jats:italic>intelligent assistants<\/jats:italic>\n            , that is, software targeted at and continuously adapting to assist end users with email, shopping, and other tasks. Examples include desktop SPAM filters, recommender systems, and handwriting recognition. Fixing such intelligent assistants when they learn incorrect behavior, however, has received only limited attention. To directly support end-user \u201cdebugging\u201d of assistant behaviors learned via statistical machine learning, we present a Why-oriented approach which allows users to ask questions about how the assistant made its predictions, provides answers to these \u201cwhy\u201d questions, and allows users to interactively\n            <jats:italic>change<\/jats:italic>\n            these answers to debug the assistant's current and future predictions. To understand the strengths and weaknesses of this approach, we then conducted an exploratory study to investigate barriers that participants could encounter when debugging an intelligent assistant using our approach, and the information those participants requested to overcome these barriers. To help ensure the inclusiveness of our approach, we also explored how gender differences played a role in understanding barriers and information needs. We then used these results to consider opportunities for Why-oriented approaches to address user barriers and information needs.\n          <\/jats:p>","DOI":"10.1145\/2030365.2030367","type":"journal-article","created":{"date-parts":[[2012,6,1]],"date-time":"2012-06-01T15:51:28Z","timestamp":1338565888000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":54,"title":["Why-oriented end-user debugging of naive Bayes text classification"],"prefix":"10.1145","volume":"1","author":[{"given":"Todd","family":"Kulesza","sequence":"first","affiliation":[{"name":"Oregon State University"}]},{"given":"Simone","family":"Stumpf","sequence":"additional","affiliation":[{"name":"City University London"}]},{"given":"Weng-Keen","family":"Wong","sequence":"additional","affiliation":[{"name":"Oregon State University"}]},{"given":"Margaret M.","family":"Burnett","sequence":"additional","affiliation":[{"name":"Oregon State University"}]},{"given":"Stephen","family":"Perona","sequence":"additional","affiliation":[{"name":"Oregon State University"}]},{"given":"Amy J.","family":"Ko","sequence":"additional","affiliation":[{"name":"University of Washington"}]},{"given":"Ian","family":"Oberst","sequence":"additional","affiliation":[{"name":"Oregon State University"}]}],"member":"320","published-online":{"date-parts":[[2011,10,31]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-295X.84.2.191"},{"key":"e_1_2_1_2_1","unstructured":"Becker B. Kohavi R. and Sommerfield D. 2001. Visualizing the simple Bayesian classifier. In Information Visualization in Data Mining and Knowledge Discovery U. Fayyad et al. Eds. 237--249.   Becker B. Kohavi R. and Sommerfield D. 2001. Visualizing the simple Bayesian classifier. In Information Visualization in Data Mining and Knowledge Discovery U. Fayyad et al. Eds. 237--249."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1054972.1055094"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the 25 International Conference of Software Engineering. 93--103","author":"Burnett M.","unstructured":"Burnett , M. , Cook , C. , Pendse , O. , Rothermel , G. , Summet , J. , and Wallace , C . 2003. End-user software engineering with assertions in the spreadsheet paradigm . In Proceedings of the 25 International Conference of Software Engineering. 93--103 . Burnett, M., Cook, C., Pendse, O., Rothermel, G., Summet, J., and Wallace, C. 2003. End-user software engineering with assertions in the spreadsheet paradigm. In Proceedings of the 25 International Conference of Software Engineering. 93--103."},{"key":"e_1_2_1_6_1","unstructured":"Carroll J. and Rosson M. 1987. Paradox of the active user. In Interfacing Thought: Cognitive Aspects of Human-Computer Interaction J. Carroll Ed. MIT Press Cambridge MA 80--111.   Carroll J. and Rosson M. 1987. Paradox of the active user. In Interfacing Thought: Cognitive Aspects of Human-Computer Interaction J. Carroll Ed. MIT Press Cambridge MA 80--111."},{"key":"e_1_2_1_7_1","volume-title":"Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence. 67--75","author":"Chan H.","unstructured":"Chan , H. and Darwiche , A . 2004. Sensitivity analysis in Bayesian networks: from single to multiple parameters . In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence. 67--75 . Chan, H. and Darwiche, A. 2004. Sensitivity analysis in Bayesian networks: from single to multiple parameters. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence. 67--75."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.5555\/1622810.1622818"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/1378773.1378794"},{"key":"e_1_2_1_10_1","volume-title":"Proceedings of the AAAI Spring Symposium on Machine Learning.","author":"Cohen W.","year":"1996","unstructured":"Cohen , W. 1996 . Learning rules that classify e-mail . In Proceedings of the AAAI Spring Symposium on Machine Learning. Cohen, W. 1996. Learning rules that classify e-mail. In Proceedings of the AAAI Spring Symposium on Machine Learning."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1287\/isre.6.2.118"},{"key":"e_1_2_1_12_1","volume-title":"Proceedings of the 6th Workshop for Empirical Studies of Programmers. 59--76","author":"Davies S. P.","year":"1996","unstructured":"Davies , S. P. 1996 . Display-based problem solving strategies in computer programming . In Proceedings of the 6th Workshop for Empirical Studies of Programmers. 59--76 . Davies, S. P. 1996. Display-based problem solving strategies in computer programming. In Proceedings of the 6th Workshop for Empirical Studies of Programmers. 59--76."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1378773.1378804"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2008.4639077"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/944919.944968"},{"key":"e_1_2_1_16_1","unstructured":"Hastie T. Tibshirani R. and Friedman J. H. 2003. The Elements of Statistical Learning. Springer Berlin.  Hastie T. Tibshirani R. and Friedman J. H. 2003. The Elements of Statistical Learning. Springer Berlin."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1753326.1753529"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1133265.1133293"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2004.47"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1080\/08839519308949993"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/1502650.1502678"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1017\/S026988890200019X"},{"key":"e_1_2_1_24_1","unstructured":"Lieberman H. (ed.) 2001. Your Wish is My Command: Programming By Example. Morgan Kaufmann.  Lieberman H. (ed.) 2001. Your Wish is My Command: Programming By Example. Morgan Kaufmann."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1519023"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1620545.1620576"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240767"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/604045.604067"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/321075.321084"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/302979.303127"},{"key":"e_1_2_1_31_1","volume-title":"Gender differences in information processing: A selectivity interpretation. Cognitive and Affective Responses to Advertising","author":"Meyers-Levy J.","unstructured":"Meyers-Levy , J. 1989. Gender differences in information processing: A selectivity interpretation. Cognitive and Affective Responses to Advertising , P. Cafferata and A. Tybout Eds., Lexington Books . Meyers-Levy, J. 1989. Gender differences in information processing: A selectivity interpretation. Cognitive and Affective Responses to Advertising, P. Cafferata and A. Tybout Eds., Lexington Books."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/1124772.1124832"},{"key":"e_1_2_1_33_1","first-page":"841","article-title":"On discriminative vs generative classifiers: A comparison of logistic regression and na\u00efve Bayes","volume":"14","author":"Ng A. Y.","year":"2002","unstructured":"Ng , A. Y. and Jordan , M. I. 2002 . On discriminative vs generative classifiers: A comparison of logistic regression and na\u00efve Bayes . Advances Neural Inf. Process. Syst. 14 , 841 -- 848 . Ng, A. Y. and Jordan, M. I. 2002. On discriminative vs generative classifiers: A comparison of logistic regression and na\u00efve Bayes. Advances Neural Inf. Process. Syst. 14, 841--848.","journal-title":"Advances Neural Inf. Process. Syst."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357160"},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the AAAI Spring Symposium. 1822--1829","author":"Poulin B.","unstructured":"Poulin , B. , Eisner , R. , Szafron , D. , Lu , P. , Greiner , R. , Wishart , D. S. , Fyshe , A. , Pearcy , B. , Macdonell , C. , and Anvik , J . 2006. Visual explanation of evidence in additive classifiers . In Proceedings of the AAAI Spring Symposium. 1822--1829 . Poulin, B., Eisner, R., Szafron, D., Lu, P., Greiner, R., Wishart, D. S., Fyshe, A., Pearcy, B., Macdonell, C., and Anvik, J. 2006. Visual explanation of evidence in additive classifiers. In Proceedings of the AAAI Spring Symposium. 1822--1829."},{"key":"e_1_2_1_36_1","volume-title":"Artificial Intelligence: A Modern Approach","author":"Russell S. J.","year":"2003","unstructured":"Russell , S. J. and Norvig , P . 2003 . Artificial Intelligence: A Modern Approach . Prentice Hall , Upper Saddle River, NJ. Russell, S. J. and Norvig, P. 2003. Artificial Intelligence: A Modern Approach. Prentice Hall, Upper Saddle River, NJ."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/1216295.1216316"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1378773.1378781"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijhcs.2009.03.004"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1109\/VLHCC.2007.21"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/1357054.1357153"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518895"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/1240624.1240630"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/248448.248457"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/212430.212443"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/964442.964450"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1943403.1943423"}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2030365.2030367","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2030365.2030367","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T09:48:48Z","timestamp":1750240128000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2030365.2030367"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,10]]},"references-count":45,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2011,10]]}},"alternative-id":["10.1145\/2030365.2030367"],"URL":"https:\/\/doi.org\/10.1145\/2030365.2030367","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,10]]},"assertion":[{"value":"2010-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2011-03-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2011-10-31","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}