{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,13]],"date-time":"2026-04-13T14:20:23Z","timestamp":1776090023951,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":48,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,5,6]],"date-time":"2021-05-06T00:00:00Z","timestamp":1620259200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["IIS-1940759, IIS-1940757"],"award-info":[{"award-number":["IIS-1940759, IIS-1940757"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,5,6]]},"DOI":"10.1145\/3411764.3445306","type":"proceedings-article","created":{"date-parts":[[2021,5,8]],"date-time":"2021-05-08T02:46:03Z","timestamp":1620441963000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":73,"title":["Whither AutoML? Understanding the Role of Automation in Machine Learning Workflows"],"prefix":"10.1145","author":[{"given":"Doris","family":"Xin","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering and Computer Sciences University of California, Berkeley, United States"}]},{"given":"Eva Yiwei","family":"Wu","sequence":"additional","affiliation":[{"name":"UC Berkeley, United States"}]},{"given":"Doris Jung-Lin","family":"Lee","sequence":"additional","affiliation":[{"name":"School of Information University of California, Berkeley, United States"}]},{"given":"Niloufar","family":"Salehi","sequence":"additional","affiliation":[{"name":"School of Information UC, Berkeley, United States"}]},{"given":"Aditya","family":"Parameswaran","sequence":"additional","affiliation":[{"name":"UC Berkeley, United States"}]}],"member":"320","published-online":{"date-parts":[[2021,5,7]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1207\/S15327051HCI1523_5"},{"key":"e_1_3_2_1_2_1","volume-title":"Facets: An Open Source Visualization Tool for Machine Learning Training Data. https:\/\/ai.googleblog.com\/2017\/07\/facets-open-source-visualization-tool.html","author":"Google","year":"2017","unstructured":"Google AI. 2017 . Facets: An Open Source Visualization Tool for Machine Learning Training Data. https:\/\/ai.googleblog.com\/2017\/07\/facets-open-source-visualization-tool.html Google AI. 2017. Facets: An Open Source Visualization Tool for Machine Learning Training Data. https:\/\/ai.googleblog.com\/2017\/07\/facets-open-source-visualization-tool.html"},{"key":"e_1_3_2_1_3_1","volume-title":"Website. Retrieved","year":"2020","unstructured":"AmazonSagemakerAutopilot. 2020 . Automate model development with Amazon SageMaker Autopilot . Website. Retrieved July 26, 2020 from https:\/\/docs.aws.amazon.com\/sagemaker\/latest\/dg\/autopilot-automate-model-development.html AmazonSagemakerAutopilot. 2020. Automate model development with Amazon SageMaker Autopilot. Website. Retrieved July 26, 2020 from https:\/\/docs.aws.amazon.com\/sagemaker\/latest\/dg\/autopilot-automate-model-development.html"},{"key":"e_1_3_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1609\/aimag.v35i4.2513"},{"key":"e_1_3_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2702123.2702509"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.14778\/3157794.3157797"},{"key":"e_1_3_2_1_8_1","volume-title":"Research Advances in Cloud Computing","author":"Baldini Ioana","unstructured":"Ioana Baldini , Paul Castro , Kerry Chang , Perry Cheng , Stephen Fink , Vatche Ishakian , Nick Mitchell , Vinod Muthusamy , Rodric Rabbah , Aleksander Slominski , 2017. Serverless computing: Current trends and open problems . In Research Advances in Cloud Computing . Springer , none, 1\u201320. Ioana Baldini, Paul Castro, Kerry Chang, Perry Cheng, Stephen Fink, Vatche Ishakian, Nick Mitchell, Vinod Muthusamy, Rodric Rabbah, Aleksander Slominski, 2017. Serverless computing: Current trends and open problems. In Research Advances in Cloud Computing. Springer, none, 1\u201320."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3329486.3329492"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951715622512"},{"key":"e_1_3_2_1_11_1","unstructured":"Corinna Cortes Xavi Gonzalvo Vitaly Kuznetsov Mehryar Mohri and Scott Yang. 2017. AdaNet: Adaptive Structural Learning of Artificial Neural Networks. arxiv:cs.LG\/1607.01097  Corinna Cortes Xavi Gonzalvo Vitaly Kuznetsov Mehryar Mohri and Scott Yang. 2017. AdaNet: Adaptive Structural Learning of Artificial Neural Networks. arxiv:cs.LG\/1607.01097"},{"key":"e_1_3_2_1_12_1","volume-title":"Website. Retrieved","year":"2020","unstructured":"Datarobot. 2020 . DataRobot Automated Machine Learning . Website. Retrieved July 18, 2020 from https:\/\/www.datarobot.com\/platform\/automated-machine-learning\/ Datarobot. 2020. DataRobot Automated Machine Learning. Website. Retrieved July 18, 2020 from https:\/\/www.datarobot.com\/platform\/automated-machine-learning\/"},{"key":"e_1_3_2_1_13_1","volume-title":"Website. Retrieved","year":"2020","unstructured":"dedoose. 2020 . Dedoose . Website. Retrieved September 15, 2020 from https:\/\/www.dedoose.com dedoose. 2020. Dedoose. Website. Retrieved September 15, 2020 from https:\/\/www.dedoose.com"},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377325.3377501"},{"key":"e_1_3_2_1_15_1","unstructured":"Sylvain Duranton J\u00f6rg Erlebach Camille Br\u00e9g\u00e9 Jane Danziger Andrea Gallego and Marc Pauly. 2020. What\u2019s Keeping Women Out of Data Science?https:\/\/www.bcg.com\/en-us\/publications\/2020\/what-keeps-women-out-data-science.  Sylvain Duranton J\u00f6rg Erlebach Camille Br\u00e9g\u00e9 Jane Danziger Andrea Gallego and Marc Pauly. 2020. What\u2019s Keeping Women Out of Data Science?https:\/\/www.bcg.com\/en-us\/publications\/2020\/what-keeps-women-out-data-science."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/604045.604056"},{"key":"e_1_3_2_1_17_1","volume-title":"Proceedings of the 28th International Conference on Neural Information Processing Systems -","volume":"2763","author":"Feurer Matthias","year":"2015","unstructured":"Matthias Feurer , Aaron Klein , Katharina Eggensperger , Jost\u00a0Tobias Springenberg , Manuel Blum , and Frank Hutter . 2015 . Efficient and Robust Automated Machine Learning . In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2(NIPS\u201915). MIT Press, Cambridge, MA, USA, 2755\u2013 2763 . Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost\u00a0Tobias Springenberg, Manuel Blum, and Frank Hutter. 2015. Efficient and Robust Automated Machine Learning. In Proceedings of the 28th International Conference on Neural Information Processing Systems - Volume 2(NIPS\u201915). MIT Press, Cambridge, MA, USA, 2755\u20132763."},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.14529\/jsfi170202"},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3301275.3302324"},{"key":"e_1_3_2_1_20_1","volume-title":"Website. Retrieved","author":"ML.","year":"2020","unstructured":"GoogleCloudAuto ML. 2020 . Google Cloud AutoML . Website. Retrieved July 18, 2020 from https:\/\/cloud.google.com\/automl GoogleCloudAutoML. 2020. Google Cloud AutoML. Website. Retrieved July 18, 2020 from https:\/\/cloud.google.com\/automl"},{"key":"e_1_3_2_1_21_1","volume-title":"Website. Retrieved","year":"2020","unstructured":"H2O.ai. 2020 . H2o.ai Automated Machine Learning . Website. Retrieved July 18, 2020 from https:\/\/docs.h2o.ai\/h2o\/latest-stable\/h2o-docs\/automl.html#automl-interface H2O.ai. 2020. H2o.ai Automated Machine Learning. Website. Retrieved July 18, 2020 from https:\/\/docs.h2o.ai\/h2o\/latest-stable\/h2o-docs\/automl.html#automl-interface"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3313831.3376177"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3392878"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330648"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2012.219"},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btz470"},{"key":"e_1_3_2_1_27_1","unstructured":"Angela Lee Doris Xin Doris Lee and Aditya Parameswaran. 2020. Demystifying a Dark Art: Understanding Real-World Machine Learning Model Development. arxiv:cs.LG\/2005.01520  Angela Lee Doris Xin Doris Lee and Aditya Parameswaran. 2020. Demystifying a Dark Art: Understanding Real-World Machine Learning Model Development. arxiv:cs.LG\/2005.01520"},{"key":"e_1_3_2_1_28_1","first-page":"59","article-title":"A Human-in-the-loop Perspective on AutoML: Milestones and the Road Ahead","volume":"42","author":"Jung\u00a0Lin Lee Doris","year":"2019","unstructured":"Doris Jung\u00a0Lin Lee , Stephen Macke , Doris Xin , Angela Lee , Silu Huang , and Aditya\u00a0 G. Parameswaran . 2019 . A Human-in-the-loop Perspective on AutoML: Milestones and the Road Ahead . IEEE Data Eng. Bull. 42 (2019), 59 \u2013 70 . Doris Jung\u00a0Lin Lee, Stephen Macke, Doris Xin, Angela Lee, Silu Huang, and Aditya\u00a0G. Parameswaran. 2019. A Human-in-the-loop Perspective on AutoML: Milestones and the Road Ahead. IEEE Data Eng. Bull. 42(2019), 59\u201370.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359174"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.25080\/Majora-92bf1922-00a"},{"key":"e_1_3_2_1_31_1","volume-title":"Website. Retrieved","author":"ML.","year":"2020","unstructured":"MicrosoftAzureAutomated ML. 2020 . Microsoft Azure Automated Machine Learning . Website. Retrieved September 15, 2020 from https:\/\/azure.microsoft.com\/en-us\/services\/machine-learning\/automatedml\/ MicrosoftAzureAutomatedML. 2020. Microsoft Azure Automated Machine Learning. Website. Retrieved September 15, 2020 from https:\/\/azure.microsoft.com\/en-us\/services\/machine-learning\/automatedml\/"},{"key":"e_1_3_2_1_32_1","unstructured":"Piero Molino Yaroslav Dudin and Sai\u00a0Sumanth Miryala. 2019. Ludwig: a type-based declarative deep learning toolbox. arxiv:cs.LG\/1909.07930  Piero Molino Yaroslav Dudin and Sai\u00a0Sumanth Miryala. 2019. Ludwig: a type-based declarative deep learning toolbox. arxiv:cs.LG\/1909.07930"},{"key":"e_1_3_2_1_33_1","unstructured":"Jorge\u00a0Piazentin Ono Sonia Castelo Roque Lopez Enrico Bertini Juliana Freire and Claudio Silva. 2020. PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines. arxiv:cs.HC\/2005.00160  Jorge\u00a0Piazentin Ono Sonia Castelo Roque Lopez Enrico Bertini Juliana Freire and Claudio Silva. 2020. PipelineProfiler: A Visual Analytics Tool for the Exploration of AutoML Pipelines. arxiv:cs.HC\/2005.00160"},{"key":"e_1_3_2_1_34_1","volume-title":"Conference on Innovative Data Systems Research (CIDR). CIDR","author":"Palkar Shoumik","year":"2017","unstructured":"Shoumik Palkar , James\u00a0 J Thomas , Anil Shanbhag , Deepak Narayanan , Holger Pirk , Malte Schwarzkopf , Saman Amarasinghe , Matei Zaharia , and Stanford InfoLab . 2017 . Weld: A common runtime for high performance data analytics . In Conference on Innovative Data Systems Research (CIDR). CIDR , Chaminade, California, 45. Shoumik Palkar, James\u00a0J Thomas, Anil Shanbhag, Deepak Narayanan, Holger Pirk, Malte Schwarzkopf, Saman Amarasinghe, Matei Zaharia, and Stanford InfoLab. 2017. Weld: A common runtime for high performance data analytics. In Conference on Innovative Data Systems Research (CIDR). CIDR, Chaminade, California, 45."},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/1866029.1866038"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1080\/07370024.2020.1734931"},{"key":"e_1_3_2_1_37_1","unstructured":"Yuji Roh Geon Heo and Steven\u00a0Euijong Whang. 2019. A Survey on Data Collection for Machine Learning: a Big Data \u2013 AI Integration Perspective. arxiv:cs.LG\/1811.03402  Yuji Roh Geon Heo and Steven\u00a0Euijong Whang. 2019. A Survey on Data Collection for Machine Learning: a Big Data \u2013 AI Integration Perspective. arxiv:cs.LG\/1811.03402"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518895"},{"key":"e_1_3_2_1_39_1","unstructured":"TransmogriFAI. 2020. TransmogrifAI. Website. Retrieved September 5 2020 from https:\/\/github.com\/salesforce\/TransmogrifAI  TransmogriFAI. 2020. TransmogrifAI. Website. Retrieved September 5 2020 from https:\/\/github.com\/salesforce\/TransmogrifAI"},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359338"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCSE.2011.37"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359313"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290605.3300911"},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"crossref","unstructured":"Daniel Karl\u00a0I. Weidele Justin\u00a0D. Weisz Eno Oduor Michael Muller Josh Andres Alexander Gray and Dakuo Wang. 2019. AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates. arxiv:cs.LG\/1912.06723  Daniel Karl\u00a0I. Weidele Justin\u00a0D. Weisz Eno Oduor Michael Muller Josh Andres Alexander Gray and Dakuo Wang. 2019. AutoAIViz: Opening the Blackbox of Automated Artificial Intelligence with Conditional Parallel Coordinates. arxiv:cs.LG\/1912.06723","DOI":"10.1145\/3377325.3377538"},{"key":"e_1_3_2_1_45_1","volume-title":"The Free Encyclopedia. https:\/\/en.wikipedia.org\/w\/index.php?title=One-hot&oldid=975049657 [Online","author":"Wikipedia Wikipedia","year":"2020","unstructured":"Wikipedia contributors. 2020. One-hot \u2014 Wikipedia , The Free Encyclopedia. https:\/\/en.wikipedia.org\/w\/index.php?title=One-hot&oldid=975049657 [Online ; accessed 17- September - 2020 ]. Wikipedia contributors. 2020. One-hot \u2014 Wikipedia, The Free Encyclopedia. https:\/\/en.wikipedia.org\/w\/index.php?title=One-hot&oldid=975049657 [Online; accessed 17-September-2020]."},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2017.2744878"},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.14778\/3297753.3297763"},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196709.3196729"}],"event":{"name":"CHI '21: CHI Conference on Human Factors in Computing Systems","location":"Yokohama Japan","acronym":"CHI '21","sponsor":["SIGCHI ACM Special Interest Group on Computer-Human Interaction"]},"container-title":["Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3411764.3445306","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3411764.3445306","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3411764.3445306","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T21:28:33Z","timestamp":1750195713000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3411764.3445306"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,6]]},"references-count":48,"alternative-id":["10.1145\/3411764.3445306","10.1145\/3411764"],"URL":"https:\/\/doi.org\/10.1145\/3411764.3445306","relation":{},"subject":[],"published":{"date-parts":[[2021,5,6]]},"assertion":[{"value":"2021-05-07","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}