{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T15:19:29Z","timestamp":1776093569854,"version":"3.50.1"},"reference-count":90,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,12,8]],"date-time":"2023-12-08T00:00:00Z","timestamp":1701993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"German Federal Ministry for Economic Affairs and Climate Action in the project FabOS","award":["01MK20010N"],"award-info":[{"award-number":["01MK20010N"]}]},{"name":"German Federal Ministry of Education and Research","award":["01|S8067A"],"award-info":[{"award-number":["01|S8067A"]}]},{"name":"Baden-Wuerttemberg Ministry for Economic Affairs, Labour and Tourism"},{"name":"Lernende Systeme und Kognitive Robotik","award":["036-170017"],"award-info":[{"award-number":["036-170017"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Interact. Intell. Syst."],"published-print":{"date-parts":[[2023,12,31]]},"abstract":"<jats:p>\n            In the last 10 years, various automated machine learning (AutoML) systems have been proposed to build end-to-end machine learning (ML) pipelines with minimal human interaction. Even though such automatically synthesized ML pipelines are able to achieve competitive performance, recent studies have shown that users do not trust models constructed by AutoML due to missing transparency of AutoML systems and missing explanations for the constructed ML pipelines. In a requirements analysis study with 36 domain experts, data scientists, and AutoML researchers from different professions with vastly different expertise in ML, we collect detailed informational needs for AutoML. We propose\n            <jats:sc>XAutoML<\/jats:sc>\n            , an interactive visual analytics tool for explaining arbitrary AutoML optimization procedures and ML pipelines constructed by AutoML.\n            <jats:sc>XAutoML<\/jats:sc>\n            combines interactive visualizations with established techniques from explainable artificial intelligence (XAI) to make the complete AutoML procedure transparent and explainable. By integrating\n            <jats:sc>XAutoML<\/jats:sc>\n            with\n            <jats:sc>JupyterLab<\/jats:sc>\n            , experienced users can extend the visual analytics with ad-hoc visualizations based on information extracted from\n            <jats:sc>XAutoML<\/jats:sc>\n            . We validate our approach in a user study with the same diverse user group from the requirements analysis. All participants were able to extract useful information from\n            <jats:sc>XAutoML<\/jats:sc>\n            , leading to a significantly increased understanding of ML pipelines produced by AutoML and the AutoML optimization itself.\n          <\/jats:p>","DOI":"10.1145\/3625240","type":"journal-article","created":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T16:07:27Z","timestamp":1695917247000},"page":"1-39","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":17,"title":["XAutoML: A Visual Analytics Tool for Understanding and Validating Automated Machine Learning"],"prefix":"10.1145","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8705-9862","authenticated-orcid":false,"given":"Marc-Andr\u00e9","family":"Z\u00f6ller","sequence":"first","affiliation":[{"name":"USU Software AG, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1959-4545","authenticated-orcid":false,"given":"Waldemar","family":"Titov","sequence":"additional","affiliation":[{"name":"Institute of Ubiquitous Mobility Systems, Karlsruhe University of Applied Sciences, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3339-3720","authenticated-orcid":false,"given":"Thomas","family":"Schlegel","sequence":"additional","affiliation":[{"name":"Institute of Ubiquitous Mobility Systems, Karlsruhe University of Applied Sciences, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8250-2092","authenticated-orcid":false,"given":"Marco F.","family":"Huber","sequence":"additional","affiliation":[{"name":"Institute of Industrial Manufacturing and Management IFF, University of Stuttgart, and Department Cyber Cognitive Intelligence (CCI), Fraunhofer IPA, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,12,8]]},"reference":[{"key":"e_1_3_3_2_2","first-page":"2623","volume-title":"Proceedings of the International Conference on Knowledge Discovery and Data Mining","author":"Akiba Takuya","year":"2019","unstructured":"Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. Optuna: A next-generation hyperparameter optimization framework. In Proceedings of the International Conference on Knowledge Discovery and Data Mining. ACM, 2623\u20132631."},{"key":"e_1_3_3_3_2","unstructured":"International Conference on Machine Learning"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1080\/10447310802205776"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2017.2743990"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.5555\/2986459.2986743"},{"key":"e_1_3_3_7_2","volume-title":"International Conference on Learning and Intelligent Optimization","unstructured":"Andr\u00e9 Biedenkapp, Joshua Marben, Marius Lindauer, and Frank Hutter. 2018. CAVE: configuration assessment, visualization and evaluation. In International Conference on Learning and Intelligent Optimization, R. Battiti, M. Brunato, I. Kotsireas, and P. M. Pardalos (Eds.). Springer International Publishing, 115--130."},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2011.185"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1010933404324"},{"issue":"194","key":"e_1_3_3_10_2","first-page":"4","article-title":"SUS: A \u2018Quick and dirty\u2019 usability scale","volume":"189","author":"Brooke John","year":"1996","unstructured":"John Brooke. 1996. SUS: A \u2018Quick and dirty\u2019 usability scale. Usability Evaluation in Industry 189, 194 (1996), 4\u20137.","journal-title":"Usability Evaluation in Industry"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.1.12228"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.13681"},{"key":"e_1_3_3_13_2","doi-asserted-by":"crossref","first-page":"402","DOI":"10.1145\/3205455.3205586","volume-title":"Proceedings of the Genetic and Evolutionary Computation Conference","author":"Chen Boyuan","year":"2018","unstructured":"Boyuan Chen, Harvey Wu, Warren Mo, Ishanu Chattopadhyay, and Hod Lipson. 2018. Autostacker: A compositional evolutionary learning system. In Proceedings of the Genetic and Evolutionary Computation Conference. ACM, 402\u2013409."},{"key":"e_1_3_3_14_2","unstructured":"Alibaba Clouder. 2018. Shortening machine learning development cycle with AutoML. Retrieved 2019-03-01 from https:\/\/www.alibabacloud.com\/blog\/shortening-machine-learning-development-cycle-with-automl_594232"},{"key":"e_1_3_3_15_2","volume-title":"Statistical Power Analysis for the Behavioral Sciences","author":"Cohen Jacob","year":"1988","unstructured":"Jacob Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences. Lawrence Earlbaum Associates. 20\u201326 pages."},{"key":"e_1_3_3_16_2","first-page":"1","volume-title":"Proceedings of the Conference on Human Factors in Computing Systems","author":"Crisan Anamaria","year":"2021","unstructured":"Anamaria Crisan and Brittany Fiore-Gartland. 2021. Fits and starts: Enterprise use of AutoML and the role of humans in the loop. In Proceedings of the Conference on Human Factors in Computing Systems. ACM, 1\u201315."},{"key":"e_1_3_3_17_2","first-page":"1","volume-title":"Proceedings of the International Workshop on Data Management for End-to-End Machine Learning","author":"Das Piali","year":"2020","unstructured":"Piali Das, Valerio Perrone, Nikita Ivkin, Tanya Bansal, Zohar Karnin, Huibin Shen, Iaroslav Shcherbatyi, Yotam Elor, Wilton Wu, Aida Zolic, Thibaut Lienart, Alex Tang, Amr Ahmed, Jean Baptiste Faddoul, Rodolphe Jenatton, Fela Winkelmolen, Philip Gautier, Leo Dirac, Andre Perunicic, Miroslav Miladinovic, Giovanni Zappella, Cedric Archambeau, Matthias Seeger, Bhaskar Dutt, and Laurence Rouesnel. 2020. Amazon SageMaker autopilot: A white box AutoML solution at scale. In Proceedings of the International Workshop on Data Management for End-to-End Machine Learning. ACM, 1\u20137."},{"key":"e_1_3_3_18_2","unstructured":"Dataiku. 2021. Dataiku. Retrieved 16\/10\/2023 from https:\/\/www.dataiku.com"},{"key":"e_1_3_3_19_2","unstructured":"DataRobot. 2021. AI Cloud. Retrieved 30\/12\/2021 from https:\/\/www.datarobot.com\/platform\/"},{"key":"e_1_3_3_20_2","volume-title":"Workshop on Comprehensibility and Explanation in AI and ML","unstructured":"Derek Doran, Sarah Schulz, and Tarek R. Besold. 2017. What Does Explainable AI Really Mean? A new conceptualization of perspectives. In Workshop on Comprehensibility and Explanation in AI and ML, T. R. Besold and O. Kutz (Eds.). EUR-WS.org, 1--8."},{"key":"e_1_3_3_21_2","first-page":"1","volume-title":"Proceedings of the International Conference on Machine Learning AutoML Workshop","author":"Drori Iddo","year":"2018","unstructured":"Iddo Drori, Yamuna Krishnamurthy, Remi Rampin, Raoni de Paula Lourenco, Jorge Piazentin Ono, Kyunghyun Cho, Claudio Silva, and Juliana Freire. 2018. AlphaD3M: Machine learning pipeline synthesis. In Proceedings of the International Conference on Machine Learning AutoML Workshop. 1\u20138."},{"key":"e_1_3_3_22_2","first-page":"297","volume-title":"Proceedings of the International Conference on Intelligent User Interfaces","author":"Drozdal Jaimie","year":"2020","unstructured":"Jaimie Drozdal, Justin Weisz, Dakuo Wang, Gaurav Dass, Bingsheng Yao, Changruo Zhao, Michael Muller, Lin Ju, and Hui Su. 2020. Trust in AutoML: Exploring information needs for establishing trust in automated machine learning systems. In Proceedings of the International Conference on Intelligent User Interfaces. ACM, 297\u2013307."},{"key":"e_1_3_3_23_2","unstructured":"International Conference on Machine Learning AutoML Workshop"},{"key":"e_1_3_3_24_2","volume-title":"International Conference on Machine Learning","unstructured":"Stefan Falkner, Aaron Klein, and Frank Hutter. 2018. BOHB: Robust and Efficient Hyperparameter Optimization at Scale. In International Conference on Machine Learning, J. Dy and A. Krause (Eds.). PMLR, 1437--1446."},{"key":"e_1_3_3_25_2","unstructured":"Fedesoriano. 2021. Stroke Prediction Dataset. Retrieved 29\/12\/2021 from https:\/\/www.kaggle.com\/fedesoriano\/stroke-prediction-dataset"},{"key":"e_1_3_3_26_2","volume-title":"International Conference on Neural Information Processing Systems","unstructured":"Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Tobias Springenber, Manuel Blum, and Frank Hutter. 2015. Efficient and Robust Automated Machine Learning. In International Conference on Neural Information Processing Systems, C. Cortes, D. D. Lee, M. Sugiyama, and R. Garnett (Eds.). MIT Press, 2755--2763."},{"key":"e_1_3_3_27_2","volume-title":"Machine Learning and Knowledge Extraction","unstructured":"Alex A. Freitas. 2019. Automated machine learning for studying the trade-off between predictive accuracy and interpretability. In Machine Learning and Knowledge Extraction, A. Holzinger, P. Kieseberg, A M. Tjoa, and E. Weippl (Eds.). Springer International Publishing, 48--66."},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1013203451"},{"key":"e_1_3_3_29_2","volume-title":"International Conference on Intelligent User Interfaces","unstructured":"Yolanda Gil, James Honaker, Vito D. Orazio, Daniel Garijo, and Neda Jahanshad. 2019. Towards human-guided machine learning. In International Conference on Intelligent User Interfaces. Association for Computing Machinery, 614--624."},{"key":"e_1_3_3_30_2","volume-title":"Computer Graphics Forum","unstructured":"Michael Gleicher, Aditya Barve, Xinyi Yu, and Florian Heimerl. 2020. Boxer: Interactive comparison of classifier results. In Computer Graphics Forum, H. Hauser and P. Alliez (Eds.). Wiley, 181--193."},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1080\/10618600.2014.907095"},{"key":"e_1_3_3_32_2","first-page":"1487","volume-title":"Proceedings of the International Conference on Knowledge Discovery and Data Mining","author":"Golovin Daniel","year":"2017","unstructured":"Daniel Golovin, Benjamin Solnik, Subhodeep Moitra, Greg Kochanski, John Karro, and D. Sculley. 2017. Google vizier: A service for black-box optimization. In Proceedings of the International Conference on Knowledge Discovery and Data Mining. ACM, 1487\u20131495."},{"key":"e_1_3_3_33_2","unstructured":"Google LLC. 2022. Google Search Trends. (2022). Retrieved from https:\/\/trends.google.com\/trends\/explore?date=today5-y&q=automl machinelearning"},{"key":"e_1_3_3_34_2","first-page":"355","volume-title":"Proceedings of the Conference on Human Information Interaction and Retrieval","author":"Grappiolo Corrado","year":"2019","unstructured":"Corrado Grappiolo, Emile Van Gerwen, Jack Verhoosel, and Lou Somers. 2019. The semantic snake charmer search engine: A tool to facilitate data science in high-tech industry domains. In Proceedings of the Conference on Human Information Interaction and Retrieval. ACM, 355\u2013359."},{"key":"e_1_3_3_35_2","unstructured":"H2O.ai. 2019. H2O AutoML. Retrieved 08\/11\/2019 from http:\/\/docs.h2o.ai\/h2o\/latest-stable\/h2o-docs\/automl.html"},{"key":"e_1_3_3_36_2","first-page":"1","volume-title":"Proceedings of the Conference on Advanced Visual Interfaces","author":"Heyen Frank","year":"2020","unstructured":"Frank Heyen, Tanja Munz, Michael Neumann, Daniel Ortega, Ngoc Thang Vu, Daniel Weiskopf, and Michael Sedlmair. 2020. ClaVis: An interactive visual comparison system for classifiers. In Proceedings of the Conference on Advanced Visual Interfaces. ACM, 1\u20139."},{"key":"e_1_3_3_37_2","first-page":"65","article-title":"A simple sequentially rejective multiple test procedure","volume":"6","author":"Holm Sture","year":"1979","unstructured":"Sture Holm. 1979. A simple sequentially rejective multiple test procedure. Source: Scandinavian Journal of Statistics 6, 2 (1979), 65\u201370.","journal-title":"Source: Scandinavian Journal of Statistics"},{"key":"e_1_3_3_38_2","volume-title":"International Conference on Learning and Intelligent Optimization","unstructured":"Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2011. Sequential Model-Based Optimization for General Algorithm Configuration. In International Conference on Learning and Intelligent Optimization, C. A. C. Coello (Ed.). Springer Berlin, Heidelberg, 507--523."},{"key":"e_1_3_3_39_2","volume-title":"International Conference on Machine Learning","unstructured":"Frank Hutter, Holger H. Hoos, and Kevin Leyton-Brown. 2014. An Efficient Approach for Assessing Hyperparameter Importance. In International Conference on Machine Learning, E. P. Xing and T. Jebara (Eds.). Association for Computing Machinery, 754--762."},{"key":"e_1_3_3_40_2","unstructured":"Journal of Artificial Intelligence Research"},{"key":"e_1_3_3_41_2","volume-title":"IEEE Conference on Visualization","unstructured":"Alfred Inselberg and Bernard Dimsdale. 1990. Parallel Coordinates: A Tool for Visualizing Multi-Dimensional Geometry. In IEEE Conference on Visualization, A. Kaufman (Ed.). IEEE Computer Society Press, 361--378."},{"key":"e_1_3_3_42_2","volume-title":"IEEE International Conference on Data Mining","unstructured":"Gilad Katz, Eui Chul Richard Shin, and Dawn Song. 2017. ExploreKit: Automatic feature generation and selection. In IEEE International Conference on Data Mining, R. Gottumukkala, X. Ning, G. Dong, V. Raghavan, S. Aluru, G. Karypis, L. Miele, and X. Wu (Eds.). IEEE, 979--984."},{"key":"e_1_3_3_43_2","volume-title":"Positioning and Power in Academic Publishing: Players, Agents and Agendas","unstructured":"Thomas Kluyver, Benjamin Ragan-Kelley, Fernando P\u00e9rez, Brian Granger, Matthias Bussonnier, Jonathan Frederic, Kyle Kelley, Jessica Hamrick, Jason Grout, Sylvain Corlay, Paul Ivanov, Dami\u00e1n Avila, Safia Abdalla, and Carol Willing. 2016. Jupyter Notebooks --a publishing format for reproducible computational workflows. In Positioning and Power in Academic Publishing: Players, Agents and Agendas, B. Schmidt and F. Loizides (Eds.). IOS Press BV, 87--90."},{"key":"e_1_3_3_44_2","unstructured":"Journal of Machine Learning Research"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1002\/nav.3800020109"},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.1017\/S0140525X16001837"},{"key":"e_1_3_3_47_2","unstructured":"Hoang Thanh Lam Johann-Michael Thiebaut Mathieu Sinn Bei Chen Tiep Mai and Oznur Alkan. 2017. One button machine for automating feature engineering in relational databases. arXiv:1706.00327. Retrieved from https:\/\/arxiv.org\/abs\/1706.00327"},{"key":"e_1_3_3_48_2","unstructured":"Journal of Machine Learning Research"},{"key":"e_1_3_3_49_2","first-page":"1","volume-title":"Proceedings of the Conference on Human Factors in Computing Systems","author":"Liao Q. Vera","year":"2020","unstructured":"Q. Vera Liao, Daniel Gruen, and Sarah Miller. 2020. Questioning the AI: Informing design practices for explainable AI user experiences. In Proceedings of the Conference on Human Factors in Computing Systems. ACM, 1\u201315."},{"key":"e_1_3_3_50_2","unstructured":"Richard Liaw Eric Liang Robert Nishihara Philipp Moritz Joseph E. Gonzalez and Ion Stoica. 2018. Tune: A research platform for distributed model selection and training. arXiv preprint arXiv:1807.05118 (2018) 1--8."},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3236386.3241340"},{"key":"e_1_3_3_52_2","unstructured":"IEEE International Conference on Big Data"},{"key":"e_1_3_3_53_2","volume-title":"Python in Science Conference","unstructured":"Wes McKinney. 2010. Data structures for statistical computing in python. In Python in Science Conference, S. van der Walt and J. Millman (Eds.). SciPy, 56--61."},{"key":"e_1_3_3_54_2","unstructured":"Meta Platforms Inc. 2013. React - A JavaScript library for building user interfaces. Retrieved 16\/10\/2023 from https:\/\/reactjs.org\/"},{"key":"e_1_3_3_55_2","unstructured":"Microsoft. 2021. Azure Machine Learning. Retrieved 16\/10\/2023 from https:\/\/azure.microsoft.com\/en-us\/products\/machine-learning\/"},{"key":"e_1_3_3_56_2","unstructured":"Microsoft. 2021. Neural Network Intelligence. Retrieved 16\/10\/2023 from https:\/\/github.com\/microsoft\/nni"},{"key":"e_1_3_3_57_2","unstructured":"Machine Learning"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387166"},{"key":"e_1_3_3_59_2","unstructured":"Christoph Molnar. 2019. Interpretable Machine Learning: A Guide for Making Black Box Models Explainable. LeanPub."},{"key":"e_1_3_3_60_2","first-page":"1","volume-title":"Proceedings of the Conference on Human Factors in Computing Systems","author":"Muller Michael","year":"2019","unstructured":"Michael Muller, Ingrid Lange, Dakuo Wang, David Piorkowski, Jason Tsay, Q. Vera Liao, Casey Dugan, and Thomas Erickson. 2019. How data science workers work with data: Discovery, capture, curation, design, creation. In Proceedings of the Conference on Human Factors in Computing Systems. ACM, 1\u201314."},{"key":"e_1_3_3_61_2","first-page":"66","volume-title":"Proceedings of the International Conference on Machine Learning AutoML Workshop","author":"Olson Randal S.","year":"2016","unstructured":"Randal S. Olson and Jason H. Moore. 2016. TPOT: A tree-based pipeline optimization tool for automating machine learning. In Proceedings of the International Conference on Machine Learning AutoML Workshop. Springer International Publishing, 66\u201374."},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030361"},{"key":"e_1_3_3_63_2","volume-title":"SysML Conference","unstructured":"Heungseok Park, Jinwoong Kim, Minkyu Kim, Ji hoon Kim, Jaegul Choo, Jung woo Ha, and Nako Sung. 2019. VisualHyperTuner: Visual analytics for user-driven hyperparameter tuning of deep neural networks. In SysML Conference, I. Dhillon, D. Papailiopoulos, and V. Sze (Eds.). mlsys.org, 1--2."},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3030380"},{"key":"e_1_3_3_65_2","doi-asserted-by":"publisher","DOI":"10.1080\/14786440109462720"},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_3_3_67_2","unstructured":"Aleksandra P\u0142o\u0144ska and Piotr P\u0142o\u0144ski. 2021. MLJAR: State-of-the-art Automated Machine Learning Framework for Tabular Data. Version 0.10.3. Retrieved 16\/10\/2023 from https:\/\/github.com\/mljar\/mljar-supervised"},{"key":"e_1_3_3_68_2","first-page":"1","volume-title":"Proceedings of the Conference on Human Factors in Computing Systems","author":"Poursabzi-Sangdeh Forough","year":"2021","unstructured":"Forough Poursabzi-Sangdeh, Dan Goldstein, Jake Hofman, Jennifer Wortman Vaughan, and Hanna Wallach. 2021. Manipulating and measuring model interpretability. In Proceedings of the Conference on Human Factors in Computing Systems. ACM, 1\u201367."},{"key":"e_1_3_3_69_2","unstructured":"Project Jupyter. 2018. JupyterLab. Retrieved 16\/10\/2023 from https:\/\/jupyter.org\/"},{"key":"e_1_3_3_70_2","unstructured":"Recharts Group. 2015. Recharts - A Composable Charting Library Built on React Components. Retrieved 16\/10\/2023 from https:\/\/recharts.org\/"},{"key":"e_1_3_3_71_2","volume-title":"International Conference on Knowledge Discovery and Data Mining","unstructured":"Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. \u201cWhy Should I Trust You?\u201d Explaining the Predictions of Any Classifier. In International Conference on Knowledge Discovery and Data Mining, B. Krishnapuram and M. Shah (Eds.). Association for Computing Machinery, 1135--1144."},{"key":"e_1_3_3_72_2","unstructured":"AAAI Conference on Artificial Intelligence"},{"key":"e_1_3_3_73_2","unstructured":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics"},{"key":"e_1_3_3_74_2","first-page":"1","volume-title":"Proceedings of the Conference on Human Factors in Computing Systems","author":"Rule Adam","year":"2018","unstructured":"Adam Rule, Aur\u00e9lien Tabard, and James D. Hollan. 2018. Exploration and explanation in computational notebooks. In Proceedings of the Conference on Human Factors in Computing Systems. ACM, 1\u201312."},{"key":"e_1_3_3_75_2","first-page":"1","volume-title":"Proceedings of the Workshop on Human-In-the-Loop Data Analytics","author":"Santos Aecio","year":"2019","unstructured":"Aecio Santos, Sonia Castelo, Cristian Felix, Jorge Piazentin Ono, Bowen Yu, Sungsoo Hong, Claudio T. Silva, Enrico Bertini, and Juliana Freire. 2019. Visus: An interactive system for automatic machine learning model building and curation. In Proceedings of the Workshop on Human-In-the-Loop Data Analytics. ACM, 1\u20137."},{"key":"e_1_3_3_76_2","volume-title":"IEEE International Conference on Machine Learning and Applications","unstructured":"Nina Schaaf, Marco Huber, and Johannes Maucher. 2019. Enhancing decision tree based interpretation of deep neural networks through L1-orthogonal regularization. In IEEE International Conference on Machine Learning and Applications, M. A. Wani (Ed.). IEEE, 42--49."},{"key":"e_1_3_3_77_2","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1007\/978-981-10-0983-9_32","volume-title":"Knowledge Solutions: Tools, Methods, and Approaches to Drive Organizational Performance","author":"Serrat Olivier","year":"2017","unstructured":"Olivier Serrat. 2017. The five whys technique. In Knowledge Solutions: Tools, Methods, and Approaches to Drive Organizational Performance. Springer, Singapore, 307\u2013310."},{"issue":"1","key":"e_1_3_3_78_2","first-page":"1064","article-title":"explAIner: A visual analytics framework for interactive and explainable machine learning","volume":"26","author":"Spinner Thilo","year":"2019","unstructured":"Thilo Spinner, Udo Schlegel, Hanna Sch\u00e4fer, and Mennatallah El-Assady. 2019. explAIner: A visual analytics framework for interactive and explainable machine learning. IEEE Transactions on Visualization and Computer Graphics 26, 1 (2019), 1064\u20131074.","journal-title":"IEEE Transactions on Visualization and Computer Graphics"},{"key":"e_1_3_3_79_2","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/6.1.1"},{"key":"e_1_3_3_80_2","volume-title":"IEEE International Conference on Big Data","unstructured":"Thomas Swearingen, Will Drevo, Bennett Cyphers, Alfredo Cuesta-Infante, Arun Ross, and Kalyan Veeramachaneni. 2017. ATM: A distributed, collaborative, scalable system for automated machine learning. In IEEE International Conference on Big Data, J.-Y. Nie, Z. Obradovic, T. Suzumura, R. Ghosh, R. Nambiar, C. Wang, H. Zang, R. A. Baeza-Yates, X. Hu, J. Kepner, A. Cuzzocrea, J. Tang, and M. Toyoda (Eds.). IEEE, 151--162."},{"key":"e_1_3_3_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2011.37"},{"key":"e_1_3_3_82_2","volume-title":"Machine Learning and Systems","unstructured":"Chi Wang, Qingyun Wu, Markus Weimer, and Erkang Zhu. 2021. FLAML: A Fast and Lightweight AutoML Library. In Machine Learning and Systems, A. Smola, A. Dimakis, and I. Stoica (Eds.). mlsys.org, 434--447."},{"key":"e_1_3_3_83_2","first-page":"1","volume-title":"Proceedings of the Conference on Human Factors in Computing Systems","author":"Wang Dakuo","year":"2021","unstructured":"Dakuo Wang, Josh Andres, Justin Weisz, Erick Oduor, and Casey Dugan. 2021. AutoDS: Towards human-centered automation of data science. In Proceedings of the Conference on Human Factors in Computing Systems. ACM, 1\u201312."},{"key":"e_1_3_3_84_2","unstructured":"Dakuo Wang Q. Vera Liao Yunfeng Zhang Udayan Khurana Horst Samulowitz Soya Park Michael Muller and Lisa Amini. 2021. How much automation does a data scientist want? arXiv:2101.03970. Retrieved from https:\/\/arxiv.org\/abs\/2101.03970"},{"key":"e_1_3_3_85_2","first-page":"77","volume-title":"Proceedings of the International Conference on Intelligent User Interfaces","author":"Wang Dakuo","year":"2020","unstructured":"Dakuo Wang, Parikshit Ram, Daniel Karl I. Weidele, Sijia Liu, Michael Muller, Justin D. Weisz, Abel Valente, Arunima Chaudhary, Dustin Torres, Horst Samulowitz, and Lisa Amini. 2020. AutoAI: Automating the end-to-end AI lifecycle with humans-in-the-loop. In Proceedings of the International Conference on Intelligent User Interfaces. ACM, 77\u201378."},{"key":"e_1_3_3_86_2","first-page":"1","article-title":"Human-AI collaboration in data science: Exploring data scientists\u2019 perceptions of automated AI","volume":"3","author":"Wang Dakuo","year":"2019","unstructured":"Dakuo Wang, Justin D. Weisz, Michael Muller, Parikshit Ram, Werner Geyer, Casey Dugan, Yla Tausczik, Horst Samulowitz, and Alexander Gray. 2019. Human-AI collaboration in data science: Exploring data scientists\u2019 perceptions of automated AI. Human-Computer Interaction 3, CSCW (2019), 1\u201324.","journal-title":"Human-Computer Interaction"},{"key":"e_1_3_3_87_2","first-page":"1","volume-title":"Proceedings of the Conference on Human Factors in Computing Systems","author":"Wang Qianwen","year":"2019","unstructured":"Qianwen Wang, Yao Ming, Zhihua Jin, Qiaomu Shen, Dongyu Liu, Micah J. Smith, Kalyan Veeramachaneni, and Huamin Qu. 2019. ATMSeer: Increasing transparency and controllability in automated machine learning. In Proceedings of the Conference on Human Factors in Computing Systems. ACM, 1\u201312."},{"key":"e_1_3_3_88_2","volume-title":"IEEE Visualization Conference","unstructured":"Daniel Karl I. Weidele. 2019. Conditional Parallel Coordinates. In IEEE Visualization Conference, A. Endert, B. Fisher, and W. Willett (Eds.). IEEE, 221--225."},{"key":"e_1_3_3_89_2","volume-title":"International Conference on Intelligent User Interfaces","unstructured":"Daniel Karl I. Weidele, Justin D. Weisz, Erick Oduor, Michael Muller, Josh Andres, Alexander Gray, and Dakuo Wang. 2020. AutoAIViz: Opening the blackbox of automated artificial intelligence with conditional parallel coordinates. In International Conference on Intelligent User Interfaces, F. Patern\u00f2, N. Oliver, C. Conati, L. D. Spano, and N. Tintarev (Eds.). Association for Computing Machinery, 308--312."},{"key":"e_1_3_3_90_2","unstructured":"Journal of Artificial Intelligence Research"},{"key":"e_1_3_3_91_2","volume-title":"International Symposium on Intelligent Data Analysis","unstructured":"Marc-Andr\u00e9 Z\u00f6ller, Tien-Dung Nguyen, and Marco F. Huber. 2021. Incremental Search Space Construction for Machine Learning Pipeline Synthesis. In International Symposium on Intelligent Data Analysis, P. H. Abreu, P. P. Rodrigues, J. Gama, and A. Fern\u00e1ndez (Eds.). Springer Cham, 103--115."}],"container-title":["ACM Transactions on Interactive Intelligent Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3625240","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3625240","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:04Z","timestamp":1750287004000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3625240"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,8]]},"references-count":90,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12,31]]}},"alternative-id":["10.1145\/3625240"],"URL":"https:\/\/doi.org\/10.1145\/3625240","relation":{},"ISSN":["2160-6455","2160-6463"],"issn-type":[{"value":"2160-6455","type":"print"},{"value":"2160-6463","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,8]]},"assertion":[{"value":"2022-02-20","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-03","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}