{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,28]],"date-time":"2025-09-28T04:06:50Z","timestamp":1759032410913,"version":"3.41.0"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"OOPSLA","license":[{"start":{"date-parts":[[2019,10,10]],"date-time":"2019-10-10T00:00:00Z","timestamp":1570665600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000185","name":"DARPA","doi-asserted-by":"crossref","award":["FA8750-14-2-0242,FA8650-15-C-7564"],"award-info":[{"award-number":["FA8750-14-2-0242,FA8650-15-C-7564"]}],"id":[{"id":"10.13039\/100000185","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. ACM Program. Lang."],"published-print":{"date-parts":[[2019,10,10]]},"abstract":"<jats:p>We present AL, a novel automated machine learning system that learns to generate new supervised learning pipelines from an existing corpus of supervised learning programs. In contrast to existing automated machine learning tools, which typically implement a search over manually selected machine learning functions and classes, AL learns to identify the relevant classes in an API by analyzing dynamic program traces that use the target machine learning library. AL constructs a conditional probability model from these traces to estimate the likelihood of the generated supervised learning pipelines and uses this model to guide the search to generate pipelines for new datasets. Our evaluation shows that AL can produce successful pipelines for datasets that previous systems fail to process and produces pipelines with comparable predictive performance for datasets that previous systems process successfully.<\/jats:p>","DOI":"10.1145\/3360601","type":"journal-article","created":{"date-parts":[[2019,10,11]],"date-time":"2019-10-11T14:53:33Z","timestamp":1570805613000},"page":"1-28","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["AL: autogenerating supervised learning programs"],"prefix":"10.1145","volume":"3","author":[{"given":"Jos\u00e9 P.","family":"Cambronero","sequence":"first","affiliation":[{"name":"Massachusetts Institute of Technology, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Martin C.","family":"Rinard","sequence":"additional","affiliation":[{"name":"Massachusetts Institute of Technology, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,10,10]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2017. UCI: Solar Flare Data Set. https:\/\/archive.ics.uci.edu\/ml\/datasets\/Solar+Flare  2017. UCI: Solar Flare Data Set. https:\/\/archive.ics.uci.edu\/ml\/datasets\/Solar+Flare"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2006.03.004"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786849"},{"key":"e_1_2_1_4_1","unstructured":"Autosklearn. 2017. Github Repository Issue 292. https:\/\/github.com\/automl\/auto- sklearn\/issues\/292  Autosklearn. 2017. Github Repository Issue 292. https:\/\/github.com\/automl\/auto- sklearn\/issues\/292"},{"key":"e_1_2_1_5_1","volume-title":"Statistical inference for probabilistic functions of finite state Markov chains. The annals of mathematical statistics 37, 6","author":"Baum Leonard E","year":"1966","unstructured":"Leonard E Baum and Ted Petrie . 1966. Statistical inference for probabilistic functions of finite state Markov chains. The annals of mathematical statistics 37, 6 ( 1966 ), 1554\u20131563. Leonard E Baum and Ted Petrie. 1966. Statistical inference for probabilistic functions of finite state Markov chains. The annals of mathematical statistics 37, 6 (1966), 1554\u20131563."},{"key":"e_1_2_1_6_1","doi-asserted-by":"crossref","unstructured":"Allan Borodin Ran El-Yaniv and Vincent Gogan. 2004. Can we learn to beat the best stock. In Advances in Neural Information Processing Systems. 345\u2013352.  Allan Borodin Ran El-Yaniv and Vincent Gogan. 2004. Can we learn to beat the best stock. In Advances in Neural Information Processing Systems. 345\u2013352.","DOI":"10.1613\/jair.1336"},{"key":"e_1_2_1_7_1","volume-title":"ECML PKDD Workshop: Languages for Data Mining and Machine Learning. 108\u2013122","author":"Buitinck Lars","year":"2013","unstructured":"Lars Buitinck , Gilles Louppe , Mathieu Blondel , Fabian Pedregosa , Andreas Mueller , Olivier Grisel , Vlad Niculae , Peter Prettenhofer , Alexandre Gramfort , Jaques Grobler , Robert Layton , Jake VanderPlas , Arnaud Joly , Brian Holt , and Ga\u00ebl Varoquaux . 2013 . API design for machine learning software: experiences from the scikit-learn project . In ECML PKDD Workshop: Languages for Data Mining and Machine Learning. 108\u2013122 . Lars Buitinck, Gilles Louppe, Mathieu Blondel, Fabian Pedregosa, Andreas Mueller, Olivier Grisel, Vlad Niculae, Peter Prettenhofer, Alexandre Gramfort, Jaques Grobler, Robert Layton, Jake VanderPlas, Arnaud Joly, Brian Holt, and Ga\u00ebl Varoquaux. 2013. API design for machine learning software: experiences from the scikit-learn project. In ECML PKDD Workshop: Languages for Data Mining and Machine Learning. 108\u2013122."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_2_1_9_1","volume-title":"RECIPE: A Grammar-Based Framework for Automatically Evolving Classification Pipelines. In European Conference on Genetic Programming. Springer, 246\u2013261","author":"de S\u00e1 Alex GC","year":"2017","unstructured":"Alex GC de S\u00e1 , Walter Jos\u00e9 GS Pinto , Luiz Otavio VB Oliveira , and Gisele L Pappa . 2017 . RECIPE: A Grammar-Based Framework for Automatically Evolving Classification Pipelines. In European Conference on Genetic Programming. Springer, 246\u2013261 . Alex GC de S\u00e1, Walter Jos\u00e9 GS Pinto, Luiz Otavio VB Oliveira, and Gisele L Pappa. 2017. RECIPE: A Grammar-Based Framework for Automatically Evolving Classification Pipelines. In European Conference on Genetic Programming. Springer, 246\u2013261."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1207156109"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3062341.3062351"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3093333.3009851"},{"key":"e_1_2_1_13_1","unstructured":"Matthias Feurer Aaron Klein Katharina Eggensperger Jost Springenberg Manuel Blum and Frank Hutter. 2015. Efficient and robust automated machine learning. In Advances in Neural Information Processing Systems. 2962\u20132970.  Matthias Feurer Aaron Klein Katharina Eggensperger Jost Springenberg Manuel Blum and Frank Hutter. 2015. Efficient and robust automated machine learning. In Advances in Neural Information Processing Systems. 2962\u20132970."},{"key":"e_1_2_1_14_1","volume-title":"Introduction to the special issue on meta-learning. Machine learning 54, 3","author":"Giraud-Carrier Christophe","year":"2004","unstructured":"Christophe Giraud-Carrier , Ricardo Vilalta , and Pavel Brazdil . 2004. Introduction to the special issue on meta-learning. Machine learning 54, 3 ( 2004 ), 187\u2013193. Christophe Giraud-Carrier, Ricardo Vilalta, and Pavel Brazdil. 2004. Introduction to the special issue on meta-learning. Machine learning 54, 3 (2004), 187\u2013193."},{"key":"e_1_2_1_15_1","unstructured":"Google. 2017a. Kaggle Website. https:\/\/www.kaggle.com\/  Google. 2017a. Kaggle Website. https:\/\/www.kaggle.com\/"},{"key":"e_1_2_1_16_1","unstructured":"Google. 2017b. Kaggle Website. https:\/\/www.kaggle.com\/product- feedback\/39790  Google. 2017b. Kaggle Website. https:\/\/www.kaggle.com\/product- feedback\/39790"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2950334"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/2337223.2337322"},{"key":"e_1_2_1_19_1","first-page":"507","article-title":"Sequential Model-Based Optimization for General Algorithm Configuration","volume":"5","author":"Hutter Frank","year":"2011","unstructured":"Frank Hutter , Holger H Hoos , and Kevin Leyton-Brown . 2011 . Sequential Model-Based Optimization for General Algorithm Configuration . LION 5 (2011), 507 \u2013 523 . Frank Hutter, Holger H Hoos, and Kevin Leyton-Brown. 2011. Sequential Model-Based Optimization for General Algorithm Configuration. LION 5 (2011), 507\u2013523.","journal-title":"LION"},{"volume-title":"Titanic: Machine Learning from Disaster (Start here! Predict survival on the Titanic and get familiar with ML basics). https:\/\/www.kaggle.com\/c\/titanic","year":"2015","key":"e_1_2_1_20_1","unstructured":"Kaggle. 2015 . Titanic: Machine Learning from Disaster (Start here! Predict survival on the Titanic and get familiar with ML basics). https:\/\/www.kaggle.com\/c\/titanic Kaggle. 2015. Titanic: Machine Learning from Disaster (Start here! Predict survival on the Titanic and get familiar with ML basics). https:\/\/www.kaggle.com\/c\/titanic"},{"key":"e_1_2_1_21_1","unstructured":"Kaggle. 2017. Meta-Kaggle. https:\/\/www.kaggle.com\/kaggle\/meta- kaggle\/data  Kaggle. 2017. Meta-Kaggle. https:\/\/www.kaggle.com\/kaggle\/meta- kaggle\/data"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3025453.3025626"},{"key":"e_1_2_1_23_1","first-page":"1","article-title":"Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA","volume":"17","author":"Kotthoff Lars","year":"2016","unstructured":"Lars Kotthoff , Chris Thornton , Holger H Hoos , Frank Hutter , and Kevin Leyton-Brown . 2016 . Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA . Journal of Machine Learning Research 17 (2016), 1 \u2013 5 . Lars Kotthoff, Chris Thornton, Holger H Hoos, Frank Hutter, and Kevin Leyton-Brown. 2016. Auto-WEKA 2.0: Automatic model selection and hyperparameter optimization in WEKA. Journal of Machine Learning Research 17 (2016), 1\u20135.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_2_1_24_1","unstructured":"Ceshine Lee. 2017. Multiple Kaggle Kernels for Corporacion Favorita Grocery Sales Forecasting. https:\/\/www.kaggle.com\/ ceshine\/kernels?search=favorita&amp;userId=101136  Ceshine Lee. 2017. Multiple Kaggle Kernels for Corporacion Favorita Grocery Sales Forecasting. https:\/\/www.kaggle.com\/ ceshine\/kernels?search=favorita&amp;userId=101136"},{"key":"e_1_2_1_25_1","volume-title":"Is Key Hurdle to Insights.","author":"Lohr Steve","year":"2014","unstructured":"Steve Lohr . 2014. For Big-Data Scientists, \u2019Janitor Work \u2019 Is Key Hurdle to Insights. New York Times ( 2014 ). https: \/\/www.nytimes.com\/2014\/08\/18\/technology\/for- big- data- scientists- hurdle- to- insights- is- janitor- work.html Steve Lohr. 2014. For Big-Data Scientists, \u2019Janitor Work\u2019 Is Key Hurdle to Insights. New York Times (2014). https: \/\/www.nytimes.com\/2014\/08\/18\/technology\/for- big- data- scientists- hurdle- to- insights- is- janitor- work.html"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1065010.1065018"},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the 9th Python in Science Conference","volume":"445","author":"Wes","unstructured":"Wes McKinney et al. 2010. Data structures for statistical computing in python . In Proceedings of the 9th Python in Science Conference , Vol. 445 . Austin, TX, 51\u201356. Wes McKinney et al. 2010. Data structures for statistical computing in python. In Proceedings of the 9th Python in Science Conference, Vol. 445. Austin, TX, 51\u201356."},{"key":"e_1_2_1_28_1","volume-title":"Bayesian Sketch Learning for Program Synthesis. arXiv preprint arXiv:1703.05698","author":"Murali Vijayaraghavan","year":"2017","unstructured":"Vijayaraghavan Murali , Swarat Chaudhuri , and Chris Jermaine . 2017. Bayesian Sketch Learning for Program Synthesis. arXiv preprint arXiv:1703.05698 ( 2017 ). Vijayaraghavan Murali, Swarat Chaudhuri, and Chris Jermaine. 2017. Bayesian Sketch Learning for Program Synthesis. arXiv preprint arXiv:1703.05698 (2017)."},{"key":"e_1_2_1_29_1","volume-title":"Neural Sketch Learning for Conditional Program Generation. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=HkfXMzAb","author":"Murali Vijayaraghavan","year":"2018","unstructured":"Vijayaraghavan Murali , Letao Qi , Swarat Chaudhuri , and Chris Jermaine . 2018 . Neural Sketch Learning for Conditional Program Generation. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=HkfXMzAb Vijayaraghavan Murali, Letao Qi, Swarat Chaudhuri, and Chris Jermaine. 2018. Neural Sketch Learning for Conditional Program Generation. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=HkfXMzAb"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2908812.2908918"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1186\/s13040-017-0154-4"},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of AAAI-98 Workshop on Learning for Text Categorization. 95\u201398","author":"Pantel Patrick","year":"1998","unstructured":"Patrick Pantel , Dekang Lin , 1998 . Spamcop: A spam classification &amp; organization program . In Proceedings of AAAI-98 Workshop on Learning for Text Categorization. 95\u201398 . Patrick Pantel, Dekang Lin, et al. 1998. Spamcop: A spam classification &amp; organization program. In Proceedings of AAAI-98 Workshop on Learning for Text Categorization. 95\u201398."},{"key":"e_1_2_1_33_1","unstructured":"Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in PyTorch. (2017).  Adam Paszke Sam Gross Soumith Chintala Gregory Chanan Edward Yang Zachary DeVito Zeming Lin Alban Desmaison Luca Antiga and Adam Lerer. 2017. Automatic differentiation in PyTorch. (2017)."},{"key":"e_1_2_1_34_1","first-page":"46","article-title":"Pattern-matching-the gestalt approach","volume":"13","author":"Ratcliff John W","year":"1988","unstructured":"John W Ratcliff and David E Metzener . 1988 . Pattern-matching-the gestalt approach . Dr Dobbs Journal 13 , 7 (1988), 46 . John W Ratcliff and David E Metzener. 1988. Pattern-matching-the gestalt approach. Dr Dobbs Journal 13, 7 (1988), 46.","journal-title":"Dr Dobbs Journal"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2666356.2594321"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10044-012-0280-z"},{"key":"e_1_2_1_37_1","doi-asserted-by":"crossref","unstructured":"Roni Rosenfeld. 1996. A maximum entropy approach to adaptive statistical language modeling. (1996).  Roni Rosenfeld. 1996. A maximum entropy approach to adaptive statistical language modeling. (1996).","DOI":"10.1006\/csla.1996.0011"},{"key":"e_1_2_1_38_1","volume-title":"Term-weighting approaches in automatic text retrieval. Information processing &amp","author":"Salton Gerard","year":"1988","unstructured":"Gerard Salton and Christopher Buckley . 1988. Term-weighting approaches in automatic text retrieval. Information processing &amp ; management 24, 5 ( 1988 ), 513\u2013523. Gerard Salton and Christopher Buckley. 1988. Term-weighting approaches in automatic text retrieval. Information processing &amp; management 24, 5 (1988), 513\u2013523."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.25080\/Majora-92bf1922-011"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/3298239.3298463"},{"key":"e_1_2_1_41_1","volume-title":"2nd Summit on Advances in Programming Languages (SNAPL","author":"Singh Rishabh","year":"2017","unstructured":"Rishabh Singh and Pushmeet Kohli . 2017. AP: artificial programming . In 2nd Summit on Advances in Programming Languages (SNAPL 2017 ). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik . Rishabh Singh and Pushmeet Kohli. 2017. AP: artificial programming. In 2nd Summit on Advances in Programming Languages (SNAPL 2017). Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2017.109"},{"key":"e_1_2_1_43_1","volume-title":"A Grammar-Based Structural CNN Decoder for Code Generation. arXiv preprint arXiv:1811.06837","author":"Sun Zeyu","year":"2018","unstructured":"Zeyu Sun , Qihao Zhu , Lili Mou , Yingfei Xiong , Ge Li , and Lu Zhang . 2018. A Grammar-Based Structural CNN Decoder for Code Generation. arXiv preprint arXiv:1811.06837 ( 2018 ). Zeyu Sun, Qihao Zhu, Lili Mou, Yingfei Xiong, Ge Li, and Lu Zhang. 2018. A Grammar-Based Structural CNN Decoder for Code Generation. arXiv preprint arXiv:1811.06837 (2018)."},{"key":"e_1_2_1_44_1","volume-title":"Machine learning and its applications to biology. PLoS computational biology 3, 6","author":"Tarca Adi L","year":"2007","unstructured":"Adi L Tarca , Vincent J Carey , Xue-wen Chen, Roberto Romero , and Sorin Dr\u0103ghici . 2007. Machine learning and its applications to biology. PLoS computational biology 3, 6 ( 2007 ), e116. Adi L Tarca, Vincent J Carey, Xue-wen Chen, Roberto Romero, and Sorin Dr\u0103ghici. 2007. Machine learning and its applications to biology. PLoS computational biology 3, 6 (2007), e116."},{"key":"e_1_2_1_45_1","unstructured":"TPOT. 2018. Github Repository. https:\/\/github.com\/EpistasisLab\/tpot\/blob\/master\/tpot\/base.py#L1030- 1034  TPOT. 2018. Github Repository. https:\/\/github.com\/EpistasisLab\/tpot\/blob\/master\/tpot\/base.py#L1030- 1034"},{"key":"e_1_2_1_46_1","first-page":"2411","article-title":"Mulan: A java library for multi-label learning","author":"Tsoumakas Grigorios","year":"2011","unstructured":"Grigorios Tsoumakas , Eleftherios Spyromitros-Xioufis , Jozef Vilcek , and Ioannis Vlahavas . 2011 . Mulan: A java library for multi-label learning . Journal of Machine Learning Research 12 , Jul (2011), 2411 \u2013 2414 . Grigorios Tsoumakas, Eleftherios Spyromitros-Xioufis, Jozef Vilcek, and Ioannis Vlahavas. 2011. Mulan: A java library for multi-label learning. Journal of Machine Learning Research 12, Jul (2011), 2411\u20132414.","journal-title":"Journal of Machine Learning Research 12"},{"key":"e_1_2_1_47_1","unstructured":"Bojan Tunguz. 2018. Multiple Kaggle Kernels for Mercari Price Suggestion Challenge. https:\/\/www.kaggle.com\/tunguz\/ kernels?search=Mercari&amp;userId=417337  Bojan Tunguz. 2018. Multiple Kaggle Kernels for Mercari Price Suggestion Challenge. https:\/\/www.kaggle.com\/tunguz\/ kernels?search=Mercari&amp;userId=417337"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/2641190.2641198"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2970276.2970326"},{"key":"e_1_2_1_50_1","volume-title":"Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann.","author":"Witten Ian H","year":"2016","unstructured":"Ian H Witten , Eibe Frank , Mark A Hall , and Christopher J Pal . 2016 . Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann. Ian H Witten, Eibe Frank, Mark A Hall, and Christopher J Pal. 2016. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann."}],"container-title":["Proceedings of the ACM on Programming Languages"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3360601","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3360601","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3360601","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T23:22:59Z","timestamp":1750202579000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3360601"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,10]]},"references-count":50,"journal-issue":{"issue":"OOPSLA","published-print":{"date-parts":[[2019,10,10]]}},"alternative-id":["10.1145\/3360601"],"URL":"https:\/\/doi.org\/10.1145\/3360601","relation":{},"ISSN":["2475-1421"],"issn-type":[{"type":"electronic","value":"2475-1421"}],"subject":[],"published":{"date-parts":[[2019,10,10]]},"assertion":[{"value":"2019-10-10","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}