{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,24]],"date-time":"2026-10-24T04:14:31Z","timestamp":1792815271708,"version":"4.5.0"},"reference-count":365,"publisher":"Springer Science and Business Media LLC","issue":"5","license":[{"start":{"date-parts":[[2024,4,18]],"date-time":"2024-04-18T00:00:00Z","timestamp":1713398400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,18]],"date-time":"2024-04-18T00:00:00Z","timestamp":1713398400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"TAILOR, a project funded by EU Horizon 2020 research and innovation programme","award":["952215"],"award-info":[{"award-number":["952215"]}]},{"DOI":"10.13039\/501100003246","name":"Nederlandse Organisatie voor Wetenschappelijk Onderzoek","doi-asserted-by":"crossref","award":["OCENW.KLEIN.425"],"award-info":[{"award-number":["OCENW.KLEIN.425"]}],"id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003246","name":"Nederlandse Organisatie voor Wetenschappelijk Onderzoek","doi-asserted-by":"crossref","award":["628.011.002"],"award-info":[{"award-number":["628.011.002"]}],"id":[{"id":"10.13039\/501100003246","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Artif Intell Rev"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Automated machine learning (AutoML) is a young research area aiming at making high-performance machine learning techniques accessible to a broad set of users. This is achieved by identifying all design choices in creating a machine-learning model and addressing them automatically to generate performance-optimised models. In this article, we provide an extensive overview of the past and present, as well as future perspectives of AutoML. First, we introduce the concept of AutoML, formally define the problems it aims to solve and describe the three components underlying AutoML approaches: the search space, search strategy and performance evaluation. Next, we discuss hyperparameter optimisation (HPO) techniques commonly used in AutoML systems design, followed by providing an overview of the neural architecture search, a particular case of AutoML for automatically generating deep learning models. We further review and compare available AutoML systems. Finally, we provide a list of open challenges and future research directions. Overall, we offer a comprehensive overview for researchers and practitioners in the area of machine learning and provide a basis for further developments in AutoML.<\/jats:p>","DOI":"10.1007\/s10462-024-10726-1","type":"journal-article","created":{"date-parts":[[2024,4,18]],"date-time":"2024-04-18T01:01:56Z","timestamp":1713402116000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":158,"title":["Automated machine learning: past, present and future"],"prefix":"10.1007","volume":"57","author":[{"given":"Mitra","family":"Baratchi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Can","family":"Wang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steffen","family":"Limmer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jan N.","family":"van Rijn","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Holger","family":"Hoos","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Thomas","family":"B\u00e4ck","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus","family":"Olhofer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,4,18]]},"reference":[{"key":"10726_CR1","unstructured":"Abdelfattah MS, Mehrotra A, Dudziak L et\u00a0al (2021) Zero-cost proxies for lightweight NAS. In: Proceedings of the 9th international conference on learning representations, virtual event, Austria, 3\u20137 May 2021"},{"key":"10726_CR2","doi-asserted-by":"publisher","unstructured":"Ahmed K, Torresani L (2018) MaskConnect: connectivity learning by gradient descent. In: Proceedings of the 15th European conference on computer vision, Munich, Germany, 8\u201314 September 2018. pp 362\u2013378. https:\/\/doi.org\/10.1007\/978-3-030-01228-1_22","DOI":"10.1007\/978-3-030-01228-1_22"},{"key":"10726_CR3","doi-asserted-by":"crossref","unstructured":"Ahmed AA, Darwish SMS, El-Sherbiny MM (2019) A novel automatic CNN architecture design approach based on genetic algorithm. In: Proceedings of the international conference on advanced intelligent systems and informatics, Cairo, Egypt, 26\u201328 October 2019. pp 473\u2013482","DOI":"10.1007\/978-3-030-31129-2_43"},{"key":"10726_CR4","unstructured":"Akhauri Y, Abdelfattah MS (2023) Multi-predict: few shot predictors for efficient neural architecture search. In: Proceedings of the AutoML conference, Potsdam, Germany, 12\u201315 September 2023"},{"key":"10726_CR5","doi-asserted-by":"publisher","unstructured":"Akiba T, Sano S, Yanase T et\u00a0al (2019) Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM international conference on knowledge discovery & data mining, Anchorage, AK, USA, 4\u20138 August 2019. pp 2623\u20132631. https:\/\/doi.org\/10.1145\/3292500.3330701","DOI":"10.1145\/3292500.3330701"},{"key":"10726_CR6","unstructured":"Ali M (2020) PyCaret: an open source, low-code machine learning library in Python. PyCaret version 1.0"},{"issue":"2","key":"10726_CR7","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/j.asoc.2004.12.002","volume":"6","author":"S Ali","year":"2006","unstructured":"Ali S, Smith KA (2006) On learning algorithm selection for classification. Appl Soft Comput 6(2):119\u2013138","journal-title":"Appl Soft Comput"},{"issue":"1","key":"10726_CR8","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1109\/72.265960","volume":"5","author":"PJ Angeline","year":"1994","unstructured":"Angeline PJ, Saunders GM, Pollack JB (1994) An evolutionary algorithm that constructs recurrent neural networks. IEEE Trans Neural Netw 5(1):54\u201365. https:\/\/doi.org\/10.1109\/72.265960","journal-title":"IEEE Trans Neural Netw"},{"issue":"1","key":"10726_CR9","doi-asserted-by":"publisher","first-page":"252","DOI":"10.1016\/0041-5553(79)90085-5","volume":"19","author":"IA Antonov","year":"1979","unstructured":"Antonov IA, Saleev V (1979) An economic method of computing lp$$\\tau$$-sequences. USSR Comput Math Math Phys 19(1):252\u2013256","journal-title":"USSR Comput Math Math Phys"},{"key":"10726_CR10","doi-asserted-by":"publisher","unstructured":"Assun\u00e7\u00e3o F, Louren\u00e7o N, Ribeiro B et\u00a0al (2020) Evolution of scikit-learn pipelines with dynamic structured grammatical evolution. In: Proceedings of the 23rd international conference on the applications of evolutionary, Seville, Spain, 15\u201317 April 2020. pp 530\u2013545. https:\/\/doi.org\/10.1007\/978-3-030-43722-0_34","DOI":"10.1007\/978-3-030-43722-0_34"},{"issue":"1","key":"10726_CR11","first-page":"B1","volume":"97","author":"T B\u00e4ck","year":"1997","unstructured":"B\u00e4ck T, Fogel DB, Michalewicz Z (1997) Handbook of evolutionary computation. Release 97(1):B1","journal-title":"Release"},{"key":"10726_CR12","doi-asserted-by":"publisher","DOI":"10.1201\/9781482268713","volume-title":"Evolutionary computation 1: basic algorithms and operators","author":"T B\u00e4ck","year":"2018","unstructured":"B\u00e4ck T, Fogel DB, Michalewicz Z (2018) Evolutionary computation 1: basic algorithms and operators. CRC Press, Boca Raton"},{"key":"10726_CR13","doi-asserted-by":"publisher","unstructured":"Bai T, Luo J, Zhao J et\u00a0al (2021) Recent advances in adversarial training for adversarial robustness. In: Zhou Z (ed) Proceedings of the thirtieth international joint conference on artificial intelligence, virtual event\/Montreal, Canada, 19\u201327 August 2021. pp 4312\u20134321. https:\/\/doi.org\/10.24963\/ijcai.2021\/591","DOI":"10.24963\/ijcai.2021\/591"},{"key":"10726_CR14","unstructured":"Baker B, Gupta O, Naik N et\u00a0al (2017) Designing neural network architectures using reinforcement learning. In: Proceedings of the 5th international conference on learning representations, Toulon, France, 24\u201326 April 2017"},{"key":"10726_CR15","unstructured":"Baker B, Gupta O, Raskar R et\u00a0al (2018) Accelerating neural architecture search using performance prediction. In: Workshop track proceedings of the 6th international conference on learning representations, Vancouver, BC, Canada, 30 April\u20133 May 2018"},{"key":"10726_CR16","doi-asserted-by":"publisher","unstructured":"Balaji A, Allen A (2018) Benchmarking automatic machine learning frameworks. https:\/\/doi.org\/10.48550\/ARXIV.1808.06492","DOI":"10.48550\/ARXIV.1808.06492"},{"issue":"12","key":"10726_CR17","doi-asserted-by":"publisher","first-page":"5097","DOI":"10.1007\/s10115-023-01935-1","volume":"65","author":"R Barbudo","year":"2023","unstructured":"Barbudo R, Ventura S, Romero JR (2023) Eight years of AutoML: categorisation, review and trends. Knowl Inf Syst 65(12):5097\u20135149","journal-title":"Knowl Inf Syst"},{"key":"10726_CR18","unstructured":"Bender G, Kindermans P, Zoph B et\u00a0al (2018) Understanding and simplifying one-shot architecture search. In: Proceedings of the 35th international conference on machine learning, Stockholmsm\u00e4ssan, Stockholm, Sweden, 10\u201315 July 2018. pp 549\u2013558"},{"issue":"8","key":"10726_CR19","doi-asserted-by":"publisher","first-page":"1889","DOI":"10.1162\/089976600300015187","volume":"12","author":"Y Bengio","year":"2000","unstructured":"Bengio Y (2000) Gradient-based optimization of hyperparameters. Neural Comput 12(8):1889\u20131900","journal-title":"Neural Comput"},{"key":"10726_CR20","doi-asserted-by":"crossref","unstructured":"Bengio Y (2012) Practical recommendations for gradient-based training of deep architectures. In: Neural networks: tricks of the trade. Springer, pp 437\u2013478","DOI":"10.1007\/978-3-642-35289-8_26"},{"key":"10726_CR21","first-page":"281","volume":"13","author":"J Bergstra","year":"2012","unstructured":"Bergstra J, Bengio Y (2012) Random search for hyper-parameter optimization. J Mach Learn Res 13:281\u2013305","journal-title":"J Mach Learn Res"},{"key":"10726_CR22","unstructured":"Bergstra J, Bardenet R, Bengio Y et\u00a0al (2011) Algorithms for hyper-parameter optimization. In: Proceedings of the 24th international conference on neural information processing systems, Sierra Nevada, Spain, 16\u201317 December 2011. pp 2546\u20132554"},{"key":"10726_CR23","unstructured":"Bergstra J, Yamins D, Cox DD (2013) Making a science of model search: hyperparameter optimization in hundreds of dimensions for vision architectures. In: Proceedings of the 30th international conference on machine learning, Atlanta, GA, USA, 16\u201321 June 2013. pp 115\u2013123"},{"key":"10726_CR24","doi-asserted-by":"crossref","unstructured":"Biedenkapp A, Lindauer M, Eggensperger K et\u00a0al (2017) Efficient parameter importance analysis via ablation with surrogates. In: Proceedings of the thirty-first AAAI conference on artificial intelligence, San Francisco, CA, USA, 4\u20139 February 2017. pp 773\u2013779","DOI":"10.1609\/aaai.v31i1.10657"},{"key":"10726_CR25","unstructured":"Binder M, Pfisterer F, Bischl B (2020) Collecting empirical data about hyperparameters for data driven AutoML. In: Proceedings of the 7th international conference on machine learning workshop on automated machine learning, virtual event, 13\u201318 July 2020"},{"key":"10726_CR26","unstructured":"Bischl B, Casalicchio G, Feurer M et\u00a0al (2021) OpenML benchmarking suites. In: Proceedings of the neural information processing systems track on datasets and benchmarks 1, virtual event, December 2021"},{"key":"10726_CR27","doi-asserted-by":"crossref","unstructured":"Bisong E (2019) Google AutoML: cloud vision. In: Building machine learning and deep learning models on Google cloud platform. Springer, pp 581\u2013598","DOI":"10.1007\/978-1-4842-4470-8_42"},{"key":"10726_CR28","unstructured":"Bosman AW, Hoos HH, van Rijn JN (2023) A preliminary study of critical robustness distributions in neural network verification. In: Proceedings of the 6th workshop on formal methods for ML-enabled autonomous systems"},{"key":"10726_CR29","doi-asserted-by":"crossref","unstructured":"Botoeva E, Kouvaros P, Kronqvist J et\u00a0al (2020) Efficient verification of ReLU-based neural networks via dependency analysis. In: Proceedings of the 34th AAAI conference on artificial intelligence, New York, NY, USA, 7\u201312 February 2020. pp 3291\u20133299","DOI":"10.1609\/aaai.v34i04.5729"},{"issue":"1","key":"10726_CR30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1561\/2200000016","volume":"3","author":"S Boyd","year":"2011","unstructured":"Boyd S, Parikh N, Chu E et al (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found Trends Mach Learn 3(1):1\u2013122. https:\/\/doi.org\/10.1561\/2200000016","journal-title":"Found Trends Mach Learn"},{"key":"10726_CR31","doi-asserted-by":"publisher","unstructured":"Brazdil P, Soares C (2000) A comparison of ranking methods for classification algorithm selection. In: Proceedings of the 11th European conference on machine learning, Barcelona, Catalonia, Spain, 31 May\u20132 June. pp 63\u201374. https:\/\/doi.org\/10.1007\/3-540-45164-1_8","DOI":"10.1007\/3-540-45164-1_8"},{"key":"10726_CR32","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-67024-5","volume-title":"Metalearning: applications to automated machine learning and data mining","author":"P Brazdil","year":"2022","unstructured":"Brazdil P, van Rijn JN, Soares C et al (2022) Metalearning: applications to automated machine learning and data mining, 2nd edn. Springer, Cham","edition":"2"},{"issue":"1","key":"10726_CR33","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L (2001) Random forests. Mach Learn 45(1):5\u201332. https:\/\/doi.org\/10.1023\/A:1010933404324","journal-title":"Mach Learn"},{"key":"10726_CR34","unstructured":"Brock A, Lim T, Ritchie JM et\u00a0al (2018) SMASH: one-shot model architecture search through hypernetworks. In: Proceedings of the 6th international conference on learning representations, Vancouver, BC, Canada, 30 April\u20133 May 2018"},{"issue":"9","key":"10726_CR35","doi-asserted-by":"publisher","first-page":"1123","DOI":"10.3390\/e23091123","volume":"23","author":"B Brumen","year":"2021","unstructured":"Brumen B, Cernezel A, Bosnjak L (2021) Overview of machine learning process modelling. Entropy 23(9):1123","journal-title":"Entropy"},{"key":"10726_CR36","unstructured":"Buitinck L, Louppe G, Blondel M et\u00a0al (2013) API design for machine learning software: experiences from the scikit-learn project. In: Proceedings of the European conference on machine learning and principles and practice of knowledge discovery in databases workshop: languages for data mining and machine learning, Prague, Czech Republic, 23\u201327 September. pp 108\u2013122"},{"key":"10726_CR37","unstructured":"Cai H, Chen T, Zhang W et\u00a0al (2018a) Efficient architecture search by network transformation. In: Proceedings of the thirty-second AAAI conference on artificial intelligence, New Orleans, LA, USA, 2\u20137 February 2018. pp 2787\u20132794"},{"key":"10726_CR38","unstructured":"Cai H, Yang J, Zhang W et\u00a0al (2018b) Path-level network transformation for efficient architecture search. In: Proceedings of the 35th international conference on machine learning, Stockholmsm\u00e4ssan, Stockholm, Sweden, 10\u201315 July 2018. pp 677\u2013686"},{"key":"10726_CR39","unstructured":"Cai H, Zhu L, Han S (2019) ProxylessNAS: direct neural architecture search on target task and hardware. In: Proceedings of the 7th international conference on learning representations, New Orleans, LA, USA, 6\u20139 May 2019"},{"key":"10726_CR40","doi-asserted-by":"crossref","unstructured":"Cambronero JP, Cito J, Rinard MC (2020) AMS: generating AutoML search spaces from weak specifications. In: Proceedings of the 28th ACM joint European software engineering conference and symposium on the foundations of software engineering, virtual event, 8\u201313 November 2020. pp 763\u2013774","DOI":"10.1145\/3368089.3409700"},{"issue":"9","key":"10726_CR41","doi-asserted-by":"publisher","first-page":"3067","DOI":"10.1109\/TPAMI.2021.3062900","volume":"43","author":"B Celik","year":"2021","unstructured":"Celik B, Vanschoren J (2021) Adaptation strategies for automated machine learning on evolving data. IEEE Trans Pattern Anal Mach Intell 43(9):3067\u20133078. https:\/\/doi.org\/10.1109\/TPAMI.2021.3062900","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"issue":"6","key":"10726_CR42","doi-asserted-by":"publisher","first-page":"1897","DOI":"10.1007\/s10994-022-06262-0","volume":"112","author":"B Celik","year":"2023","unstructured":"Celik B, Singh P, Vanschoren J (2023) Online AutoML: an adaptive AutoML framework for online learning. Mach Learn 112(6):1897\u20131921","journal-title":"Mach Learn"},{"key":"10726_CR43","doi-asserted-by":"crossref","unstructured":"Chaslot G, Bakkes S, Szita I et\u00a0al (2008a) Monte-Carlo tree search: a new framework for game AI. In: Proceedings of the artificial intelligence and interactive digital entertainment. pp 216\u2013217","DOI":"10.1609\/aiide.v4i1.18700"},{"issue":"03","key":"10726_CR44","doi-asserted-by":"publisher","first-page":"343","DOI":"10.1142\/S1793005708001094","volume":"4","author":"GMJ Chaslot","year":"2008","unstructured":"Chaslot GMJ, Winands MH, Herik HJvd et al (2008b) Progressive strategies for Monte-Carlo tree search. New Math Nat Comput 4(03):343\u2013357","journal-title":"New Math Nat Comput"},{"key":"10726_CR45","doi-asserted-by":"publisher","unstructured":"Chen T, Guestrin C (2016) XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd international conference on knowledge discovery and data mining, San Francisco, CA, USA, 13\u201317 August 2016. pp 785\u2013794. https:\/\/doi.org\/10.1145\/2939672.2939785","DOI":"10.1145\/2939672.2939785"},{"key":"10726_CR46","unstructured":"Chen T, Goodfellow IJ, Shlens J (2016) Net2Net: accelerating learning via knowledge transfer. In: Proceedings of the 4th international conference on learning representations, San Juan, Puerto Rico, 2\u20134 May 2016"},{"key":"10726_CR47","doi-asserted-by":"publisher","unstructured":"Chen B, Wu H, Mo W et\u00a0al (2018) Autostacker: a compositional evolutionary learning system. In: Proceedings of the genetic and evolutionary computation conference, Kyoto, Japan, 15\u201319 July 2018. pp 402\u2013409. https:\/\/doi.org\/10.1145\/3205455.3205586","DOI":"10.1145\/3205455.3205586"},{"key":"10726_CR48","doi-asserted-by":"publisher","unstructured":"Chen X, Qiao B, Zhang W et\u00a0al (2019a) Neural feature search: a neural architecture for automated feature engineering. In: Proceedings of the 2019 IEEE international conference on data mining, Beijing, China, 8\u201311 November 2019. pp 71\u201380. https:\/\/doi.org\/10.1109\/ICDM.2019.00017","DOI":"10.1109\/ICDM.2019.00017"},{"key":"10726_CR49","doi-asserted-by":"crossref","unstructured":"Chen X, Xie L, Wu J et\u00a0al (2019b) Progressive differentiable architecture search: bridging the depth gap between search and evaluation. In: Proceedings of the IEEE international conference on computer vision, Seoul, Korea (South), 27 October\u20132 November 2019. pp 1294\u20131303","DOI":"10.1109\/ICCV.2019.00138"},{"key":"10726_CR50","doi-asserted-by":"crossref","unstructured":"Chen Y, Meng G, Zhang Q et\u00a0al (2019c) RENAS: reinforced evolutionary neural architecture search. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, 16\u201320 June 2019. pp 4787\u20134796","DOI":"10.1109\/CVPR.2019.00492"},{"key":"10726_CR51","doi-asserted-by":"crossref","unstructured":"Chen Z, Zhou Y, Huang Z (2019d) Auto-creation of effective neural network architecture by evolutionary algorithm and ResNet for image classification. In: Proceedings of the 2019 IEEE international conference on systems, man and cybernetics, Bari, Italy, 6\u20139 October 2019. pp 3895\u20133900","DOI":"10.1109\/SMC.2019.8914267"},{"key":"10726_CR52","doi-asserted-by":"crossref","unstructured":"Chen X, Wujek B (2020) AutoDAL: distributed active learning with automatic hyperparameter selection. In: Proceedings of the thirty-fourth AAAI conference on artificial intelligence, New York, NY, USA, 7\u201312 February 2020. pp 3537\u20133544","DOI":"10.1609\/aaai.v34i04.5759"},{"key":"10726_CR53","unstructured":"Chrabaszcz P, Loshchilov I, Hutter F (2017) A downsampled variant of ImageNet as an alternative to the CIFAR datasets. arXiv. http:\/\/arxiv.org\/abs\/1707.08819"},{"key":"10726_CR54","doi-asserted-by":"crossref","unstructured":"Chu X, Zhou T, Zhang B et\u00a0al (2020) Fair DARTS: eliminating unfair advantages in differentiable architecture search. In: Proceedings of the European conference on computer vision, Glasgow, UK, 23\u201328 August 2020. pp 465\u2013480","DOI":"10.1007\/978-3-030-58555-6_28"},{"key":"10726_CR55","unstructured":"Claesen M, Simm J, Popovic D et\u00a0al (2014) Easy hyperparameter search using Optunity. http:\/\/arxiv.org\/abs\/1412.1114"},{"key":"10726_CR56","doi-asserted-by":"crossref","unstructured":"Cui J, Chen P, Li R et\u00a0al (2019) Fast and practical neural architecture search. In: Proceedings of the IEEE international conference on computer vision, Seoul, Korea (South), 27 October\u20132 November 2019. pp 6509\u20136518","DOI":"10.1109\/ICCV.2019.00661"},{"key":"10726_CR57","doi-asserted-by":"crossref","unstructured":"Das P, Ivkin N, Bansal T et\u00a0al (2020) Amazon SageMaker Autopilot: a white box AutoML solution at scale. In: Proceedings of the fourth international workshop on data management for end-to-end machine learning, Portland, USA, 14 June 2020. pp 1\u20137","DOI":"10.1145\/3399579.3399870"},{"key":"10726_CR58","unstructured":"DataCanvas (2021) Hypernets. https:\/\/github.com\/DataCanvasIO\/Hypernets. Accessed 4 Nov 2021"},{"key":"10726_CR59","unstructured":"Dataiku (2023) Dataiku. https:\/\/www.dataiku.com\/. Accessed 20 Sept 2023"},{"key":"10726_CR60","unstructured":"DataRobot (2023) DataRobot AI platform. https:\/\/www.datarobot.com\/. Accessed 20 Sept 2023"},{"issue":"3","key":"10726_CR61","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1145\/3495256","volume":"65","author":"T De Bie","year":"2022","unstructured":"De Bie T, De Raedt L, Hern\u00e1ndez-Orallo J et al (2022) Automating data science. Commun ACM 65(3):76\u201387. https:\/\/doi.org\/10.1145\/3495256","journal-title":"Commun ACM"},{"issue":"9","key":"10726_CR62","doi-asserted-by":"publisher","first-page":"3211","DOI":"10.1007\/s10994-023-06336-7","volume":"112","author":"T de Menezes","year":"2023","unstructured":"de Menezes T, Filho S, Song H et al (2023) Classifier calibration: a survey on how to assess and improve predicted class probabilities. Mach Learn 112(9):3211\u20133260. https:\/\/doi.org\/10.1007\/s10994-023-06336-7","journal-title":"Mach Learn"},{"key":"10726_CR63","doi-asserted-by":"crossref","unstructured":"de\u00a0S\u00e1 AGC, Pinto WJGS, Oliveira LOVB et\u00a0al (2017) RECIPE: a grammar-based framework for automatically evolving classification pipelines. In: Proceedings of the genetic programming European conference, Amsterdam, The Netherlands, 19\u201321 April 2017. pp 246\u2013261","DOI":"10.1007\/978-3-319-55696-3_16"},{"key":"10726_CR64","doi-asserted-by":"publisher","DOI":"10.1007\/s10462-023-10569-2","author":"AM Del Valle","year":"2023","unstructured":"Del Valle AM, Mantovani RG, Cerri R (2023) A systematic literature review on AutoML for multi-target learning tasks. Artif Intell Rev. https:\/\/doi.org\/10.1007\/s10462-023-10569-2","journal-title":"Artif Intell Rev"},{"key":"10726_CR65","unstructured":"Deng B, Yan J, Lin D (2017) Peephole: predicting network performance before training. arXiv. http:\/\/arxiv.org\/abs\/1712.03351"},{"issue":"119","key":"10726_CR66","first-page":"4053","volume":"15","author":"T Desautels","year":"2014","unstructured":"Desautels T, Krause A, Burdick JW (2014) Parallelizing exploration-exploitation tradeoffs in Gaussian process bandit optimization. J Mach Learn Res 15(119):4053\u20134103","journal-title":"J Mach Learn Res"},{"key":"10726_CR67","unstructured":"Dimanov D, Balaguer-Ballester E, Singleton C et\u00a0al (2021) MONCAE: multi-objective neuroevolution of convolutional autoencoders. In: Proceedings of the 9th international conference on learning representations workshop on neural architecture search, virtual event, 3\u20137 May 2021"},{"key":"10726_CR68","doi-asserted-by":"crossref","unstructured":"Ding Y, Yao Q, Zhao H et\u00a0al (2021) DiffMG: differentiable meta graph search for heterogeneous graph neural networks. In: Proceedings of the 27th international conference on knowledge discovery & data mining, virtual event, Singapore, 14\u201318 August 2021. pp 279\u2013288","DOI":"10.1145\/3447548.3467447"},{"key":"10726_CR69","unstructured":"Domhan T, Springenberg JT, Hutter F (2015) Speeding up automatic hyperparameter optimization of deep neural networks by extrapolation of learning curves. In: Proceedings of the twenty-fourth international joint conference on artificial intelligence, Buenos Aires, Argentina, 25\u201331 July 2015. pp 3460\u20133468"},{"key":"10726_CR70","first-page":"760","volume":"32","author":"X Dong","year":"2019","unstructured":"Dong X, Yang Y (2019) Network pruning via transformable architecture search. Adv Neural Inf Process Syst 32:760\u2013771","journal-title":"Adv Neural Inf Process Syst"},{"key":"10726_CR71","unstructured":"Dong X, Yang Y (2020) NAS-Bench-201: extending the scope of reproducible neural architecture search. In: Proceedings of the 8th international conference on learning representations, Addis Ababa, Ethiopia, 26\u201330 April 2020"},{"key":"10726_CR72","first-page":"3634","volume":"44","author":"X Dong","year":"2021","unstructured":"Dong X, Liu L, Musial K et al (2021) NATS-Bench: benchmarking NAS algorithms for architecture topology and size. IEEE Trans Pattern Anal Mach Intell 44:3634\u20133646","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"10726_CR73","unstructured":"Dorogush AV, Ershov V, Gulin A (2018) CatBoost: gradient boosting with categorical features support. arXiv. http:\/\/arxiv.org\/abs\/1810.11363"},{"key":"10726_CR74","unstructured":"Eggensperger K, Feurer M, Hutter F et\u00a0al (2013) Towards an empirical foundation for assessing Bayesian optimization of hyperparameters. In: Proceedings of the conference on neural information processing systems workshop on Bayesian optimization in theory and practice, Lake Tahoe, NV, USA, 5\u20138 December 2013. p 3"},{"key":"10726_CR75","doi-asserted-by":"crossref","unstructured":"Eggensperger K, Hutter F, Hoos H et\u00a0al (2015) Efficient benchmarking of hyperparameter optimizers via surrogates. In: Proceedings of the twenty-ninth AAAI conference on artificial intelligence, Austin, Texas, USA, 25\u201330 January 2015. pp 1114\u20131120","DOI":"10.1609\/aaai.v29i1.9375"},{"key":"10726_CR76","unstructured":"Eggensperger K, M\u00fcller P, Mallik N et\u00a0al (2021) HPOBench: a collection of reproducible multi-fidelity benchmark problems for HPO. In: Proceedings of the neural information processing systems track on datasets and benchmarks 1, virtual event, December 2021"},{"key":"10726_CR77","doi-asserted-by":"publisher","unstructured":"Eldeeb H, Maher M, Matsuk O et\u00a0al (2022) AutoMLBench: a comprehensive experimental evaluation of automated machine learning frameworks. https:\/\/doi.org\/10.48550\/ARXIV.2204.08358","DOI":"10.48550\/ARXIV.2204.08358"},{"key":"10726_CR78","unstructured":"ElShawi R, Maher M, Sakr S (2019) Automated machine learning: state-of-the-art and open challenges. arXiv. http:\/\/arxiv.org\/abs\/1906.02287"},{"key":"10726_CR79","unstructured":"Elsken T, Metzen JH, Hutter F (2018) Simple and efficient architecture search for convolutional neural networks. In: Workshop track proceedings of the 6th international conference on learning representations, Vancouver, BC, Canada, 30 April\u20133 May 2018"},{"key":"10726_CR80","unstructured":"Elsken T, Metzen JH, Hutter F (2019a) Efficient multi-objective neural architecture search via Lamarckian evolution. In: Proceedings of the 7th international conference on learning representations, New Orleans, LA, USA, 6\u20139 May 2019"},{"issue":"1","key":"10726_CR81","first-page":"1997","volume":"20","author":"T Elsken","year":"2019","unstructured":"Elsken T, Metzen JH, Hutter F (2019b) Neural architecture search: a survey. J Mach Learn Res 20(1):1997\u20132017","journal-title":"J Mach Learn Res"},{"key":"10726_CR82","doi-asserted-by":"crossref","unstructured":"Elsken T, Staffler B, Metzen JH et\u00a0al (2020) Meta-learning of neural architectures for few-shot learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Seattle, WA, USA, 13\u201319 June 2020. pp 12365\u201312375","DOI":"10.1109\/CVPR42600.2020.01238"},{"key":"10726_CR83","unstructured":"Erickson N, Mueller J, Shirkov A et\u00a0al (2020) AutoGluon-Tabular: robust and accurate AutoML for structured data. In: Proceedings of the 7th international conference on machine learning workshop on automated machine learning, virtual event, 13\u201318 July 2020"},{"key":"10726_CR84","unstructured":"Erol K, Hendler JA, Nau DS (1994) UMCP: a sound and complete procedure for hierarchical task-network planning. In: Proceedings of the second international conference on artificial intelligence planning systems, University of Chicago, Chicago, IL, USA, 13\u201315 June 1994. pp 249\u2013254"},{"key":"10726_CR85","doi-asserted-by":"publisher","unstructured":"Escalante HJ (2021) Automated machine learning\u2014a brief review at the end of the early years. In: Automated design of machine learning and search algorithms. Natural computing series. Springer, pp 11\u201328. https:\/\/doi.org\/10.1007\/978-3-030-72069-8_2","DOI":"10.1007\/978-3-030-72069-8_2"},{"key":"10726_CR86","unstructured":"European\u00a0Commission\u00a0High\u00a0Level\u00a0Expert\u00a0Group\u00a0AI (2018) Ethics guidelines for trustworthy AI: high-level expert group on artificial intelligence. https:\/\/digital-strategy.ec.europa.eu\/en\/library\/ethics-guidelines-trustworthy-ai. 8 Apr 2019"},{"key":"10726_CR87","unstructured":"Evchenko M, Vanschoren J, Hoos HH et al (2021) Frugal machine learning. arXiv. http:\/\/arxiv.org\/abs\/2111.03731"},{"key":"10726_CR88","unstructured":"Falkner S, Klein A, Hutter F (2018) BOHB: robust and efficient hyperparameter optimization at scale. In: Proceedings of the 35th international conference on machine learning, Stockholmsm\u00e4ssan, Stockholm, Sweden, 10\u201315 July 2018. pp 1436\u20131445"},{"issue":"4","key":"10726_CR89","doi-asserted-by":"publisher","first-page":"431","DOI":"10.1007\/s10732-014-9275-9","volume":"22","author":"C Fawcett","year":"2016","unstructured":"Fawcett C, Hoos HH (2016) Analysing differences between algorithm configurations through ablation. J Heuristics 22(4):431\u2013458","journal-title":"J Heuristics"},{"key":"10726_CR90","unstructured":"Feffer M, Hirzel M, Hoffman SC et\u00a0al (2023) Searching for fairer machine learning ensembles. In: Proceedings of the AutoML conference, Potsdam, Germany, 12\u201315 September 2023"},{"key":"10726_CR91","doi-asserted-by":"crossref","unstructured":"Feurer M, Hutter F (2019) Hyperparameter optimization. In: Automated machine learning. Springer, pp 3\u201333","DOI":"10.1007\/978-3-030-05318-5_1"},{"key":"10726_CR92","unstructured":"Feurer M, Klein A, Eggensperger K et\u00a0al (2015) Efficient and robust automated machine learning. In: Advances in neural information processing systems 28: annual conference on neural information processing systems 2015, Montreal, QC, Canada, 7\u201312 December 2015. pp 2962\u20132970"},{"key":"10726_CR93","unstructured":"Feurer M, Eggensperger K, Falkner S et\u00a0al (2018) Practical automated machine learning for the AutoML challenge 2018. In: Proceedings of the international conference on machine learning workshop on automated machine learning, Stockholm, Sweden, 10\u201315 July 2018"},{"issue":"261","key":"10726_CR94","first-page":"1","volume":"23","author":"M Feurer","year":"2022","unstructured":"Feurer M, Eggensperger K, Falkner S et al (2022) Auto-Sklearn 2.0: hands-free AutoML via meta-learning. J Mach Learn Res 23(261):1\u201361","journal-title":"J Mach Learn Res"},{"key":"10726_CR95","unstructured":"Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: Proceedings of the international conference on machine learning, Sydney, NSW, Australia, 6\u201311 August 2017. pp 1126\u20131135"},{"key":"10726_CR96","unstructured":"Franceschi L, Donini M, Frasconi P et\u00a0al (2017) Forward and reverse gradient-based hyperparameter optimization. In: Proceedings of the international conference on machine learning, Sydney, NSW, Australia, 6\u201311 August 2017. pp 1165\u20131173"},{"key":"10726_CR97","doi-asserted-by":"publisher","unstructured":"Gao Y, Yang H, Zhang P et\u00a0al (2020) Graph neural architecture search. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence. pp 1403\u20131409. https:\/\/doi.org\/10.24963\/ijcai.2020\/195","DOI":"10.24963\/ijcai.2020\/195"},{"key":"10726_CR98","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1016\/j.neucom.2022.01.080","volume":"482","author":"F Gao","year":"2022","unstructured":"Gao F, Song B, Wang D et al (2022) MR-DARTS: restricted connectivity differentiable architecture search in multi-path search space. Neurocomputing 482:27\u201339","journal-title":"Neurocomputing"},{"key":"10726_CR99","doi-asserted-by":"publisher","DOI":"10.1017\/9781108348973","volume-title":"Bayesian optimization","author":"R Garnett","year":"2023","unstructured":"Garnett R (2023) Bayesian optimization. Cambridge University Press, Cambridge"},{"key":"10726_CR100","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1016\/j.neucom.2019.11.004","volume":"380","author":"EC Garrido-Merch\u00e1n","year":"2020","unstructured":"Garrido-Merch\u00e1n EC, Hern\u00e1ndez-Lobato D (2020) Dealing with categorical and integer-valued variables in Bayesian optimization with Gaussian processes. Neurocomputing 380:20\u201335. https:\/\/doi.org\/10.1016\/j.neucom.2019.11.004","journal-title":"Neurocomputing"},{"key":"10726_CR101","unstructured":"George J, Gao C, Liu R et\u00a0al (2020) A scalable and cloud-native hyperparameter tuning system. arXiv. http:\/\/arxiv.org\/abs\/2006.02085"},{"key":"10726_CR102","doi-asserted-by":"publisher","unstructured":"Gijsbers P, Vanschoren J (2020) GAMA: a general automated machine learning assistant. In: Proceedings of the European conference on machine learning and knowledge discovery in databases, Ghent, Belgium, 14\u201318 September 2020. pp 560\u2013564. https:\/\/doi.org\/10.1007\/978-3-030-67670-4_39","DOI":"10.1007\/978-3-030-67670-4_39"},{"key":"10726_CR103","unstructured":"Gijsbers P, Vanschoren J, Olson RS (2017) Layered TPOT: speeding up tree-based pipeline optimization. In: Proceedings of the international workshop on automatic selection, configuration and composition of machine learning algorithms co-located with the European conference on machine learning & principles and practice of knowledge discovery in databases, Skopje, Macedonia, 22 September 2017. pp 49\u201368"},{"key":"10726_CR104","unstructured":"Gijsbers P, LeDell E, Thomas J et\u00a0al (2019) An open source AutoML benchmark. In: Proceedings of the 6th international conference on machine learning workshop on automated machine learning, Long Beach, CA, USA, 9\u201315 June 2019"},{"key":"10726_CR105","doi-asserted-by":"publisher","unstructured":"Gijsbers P, Pfisterer F, van Rijn JN et\u00a0al (2021) Meta-learning for symbolic hyperparameter defaults. In: Proceedings of the genetic and evolutionary computation conference, companion volume, Lille, France, 10\u201314 July 2021. pp 151\u2013152. https:\/\/doi.org\/10.1145\/3449726.3459532","DOI":"10.1145\/3449726.3459532"},{"key":"10726_CR106","unstructured":"Gijsbers P, Bueno MLP, Coors S et\u00a0al (2022) AMLB: an AutoML benchmark. arXiv. http:\/\/arxiv.org\/abs\/2207.12560"},{"key":"10726_CR107","doi-asserted-by":"crossref","unstructured":"Ginsbourger D, Riche RL, Carraro L (2010) Kriging is well-suited to parallelize optimization. In: Proceedings of the computational intelligence in expensive optimization problems. pp 131\u2013162","DOI":"10.1007\/978-3-642-10701-6_6"},{"key":"10726_CR108","unstructured":"Ginsbourger D, Janusevskis J, Le Riche R (2011) Dealing with asynchronicity in parallel Gaussian Process based global optimization. Research report, Mines Saint-Etienne"},{"key":"10726_CR109","doi-asserted-by":"publisher","unstructured":"Goldberg DE, Deb K (1990) A comparative analysis of selection schemes used in genetic algorithms. In: Proceedings of the first workshop on foundations of genetic algorithms. Bloomington Campus, IN, USA, 15\u201318 July 1990. pp 69\u201393. https:\/\/doi.org\/10.1016\/b978-0-08-050684-5.50008-2","DOI":"10.1016\/b978-0-08-050684-5.50008-2"},{"key":"10726_CR110","unstructured":"Goodfellow IJ, Shlens J, Szegedy C (2015) Explaining and harnessing adversarial examples. In: Proceedings of the 3rd international conference on learning representations, San Diego, CA, USA, 7\u20139 May 2015. pp 1\u201311"},{"key":"10726_CR111","unstructured":"Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT Press. http:\/\/www.deeplearningbook.org"},{"key":"10726_CR112","doi-asserted-by":"publisher","unstructured":"Gopal B, Sridhar A, Zhang T et\u00a0al (2023) LISSNAS: locality-based iterative search space shrinkage for neural architecture search. In: Proceedings of the thirty-second international joint conference on artificial intelligence, Macao, SAR, China, 19\u201325 August 2023. pp 773\u2013781. https:\/\/doi.org\/10.24963\/ijcai.2023\/86","DOI":"10.24963\/ijcai.2023\/86"},{"key":"10726_CR113","unstructured":"Grazzi R, Franceschi L, Pontil M et\u00a0al (2020) On the iteration complexity of hypergradient computation. In: Proceedings of the 37th international conference on machine learning, virtual event, 13\u201318 July 2020. pp 3748\u20133758"},{"key":"10726_CR114","doi-asserted-by":"publisher","unstructured":"Gu Y, Wang L, Liu Y et\u00a0al (2021) DOTS: decoupling operation and topology in differentiable architecture search. In: IEEE conference on computer vision and pattern recognition, virtual event, 19\u201325 June 2021. pp 12311\u201312320. https:\/\/doi.org\/10.1109\/CVPR46437.2021.01213","DOI":"10.1109\/CVPR46437.2021.01213"},{"key":"10726_CR115","first-page":"61","volume":"11","author":"I Guyon","year":"2010","unstructured":"Guyon I, Saffari A, Dror G et al (2010) Model selection: beyond the Bayesian\/frequentist divide. J Mach Learn Res 11:61\u201387","journal-title":"J Mach Learn Res"},{"key":"10726_CR116","unstructured":"Guyon I, Chaabane I, Escalante HJ et\u00a0al (2016) A brief review of the ChaLearn AutoML challenge: any-time any-dataset learning without human intervention. In: Proceedings of the 2016 workshop on automatic machine learning, AutoML 2016, co-located with 33rd international conference on machine learning, New York City, NY, USA, 24 June 2016. pp 21\u201330"},{"key":"10726_CR117","unstructured":"H2O.ai (2017) H2O AutoML. http:\/\/docs.h2o.ai\/h2o\/latest-stable\/h2o-docs\/automl.html. Accessed 4 Nov 2022"},{"issue":"1","key":"10726_CR118","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1145\/1656274.1656278","volume":"11","author":"M Hall","year":"2009","unstructured":"Hall M, Frank E, Holmes G et al (2009) The WEKA data mining software: an update. SIGKDD Explor Newslett 11(1):10\u201318. https:\/\/doi.org\/10.1145\/1656274.1656278","journal-title":"SIGKDD Explor Newslett"},{"key":"10726_CR119","doi-asserted-by":"publisher","unstructured":"Han D, Kim J, Kim J (2017) Deep pyramidal residual networks. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21\u201326 July 2017. pp 6307\u20136315. https:\/\/doi.org\/10.1109\/CVPR.2017.668","DOI":"10.1109\/CVPR.2017.668"},{"issue":"1","key":"10726_CR120","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1162\/106365603321828970","volume":"11","author":"N Hansen","year":"2003","unstructured":"Hansen N, M\u00fcller SD, Koumoutsakos P (2003) Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evol Comput 11(1):1\u201318. https:\/\/doi.org\/10.1162\/106365603321828970","journal-title":"Evol Comput"},{"key":"10726_CR121","unstructured":"Hardt M, Price E, Srebro N (2016) Equality of opportunity in supervised learning. In: Lee DD, Sugiyama M, von Luxburg U et\u00a0al (eds) Proceedings of the advances in neural information processing systems 29: annual conference on neural information processing systems, Barcelona, Spain, 5\u201310 December 2016. pp 3315\u20133323"},{"key":"10726_CR122","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S et\u00a0al (2016) Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition, Las Vegas, NV, USA, 27\u201330 June 2016. pp 770\u2013778. https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"issue":"106","key":"10726_CR123","doi-asserted-by":"publisher","first-page":"622","DOI":"10.1016\/j.knosys.2020.106622","volume":"212","author":"X He","year":"2021","unstructured":"He X, Zhao K, Chu X (2021) AutoML: a survey of the state-of-the-art. Knowl Based Syst 212(106):622. https:\/\/doi.org\/10.1016\/j.knosys.2020.106622","journal-title":"Knowl Based Syst"},{"key":"10726_CR124","unstructured":"Head T, Kumar M, Nahrstaedt H et\u00a0al (2017) Sequential model-based optimization in Python. https:\/\/scikit-optimize.github.io\/stable"},{"key":"10726_CR125","doi-asserted-by":"publisher","first-page":"1809","DOI":"10.5555\/2503308.2343701","volume":"13","author":"P Hennig","year":"2012","unstructured":"Hennig P, Schuler CJ (2012) Entropy search for information-efficient global optimization. J Mach Learn Res 13:1809\u20131837. https:\/\/doi.org\/10.5555\/2503308.2343701","journal-title":"J Mach Learn Res"},{"key":"10726_CR126","doi-asserted-by":"crossref","unstructured":"Hochreiter S, Younger AS, Conwell PR (2001) Learning to learn using gradient descent. In: Proceedings of the international conference on artificial neural networks, Vienna, Austria, 21\u201325 August 2001. pp 87\u201394","DOI":"10.1007\/3-540-44668-0_13"},{"key":"10726_CR127","unstructured":"Hoffman MW, Shahriari B (2014) Modular mechanisms for Bayesian optimization. In: Proceedings of the conference on neural information processing systems workshop on Bayesian optimization, Montr\u00e9al, QC, Canada, 8\u201313 December 2014. pp 1\u20135"},{"key":"10726_CR128","unstructured":"Hollmann N, M\u00fcller S, Eggensperger K et\u00a0al (2023) TabPFN: a transformer that solves small tabular classification problems in a second. In: Proceedings of the eleventh international conference on learning representations, Kigali, Rwanda, 1\u20135 May 2023"},{"key":"10726_CR129","doi-asserted-by":"publisher","unstructured":"Hong W, Li G, Zhang W et\u00a0al (2020) DropNAS: grouped operation dropout for differentiable architecture search. In: Proceedings of the twenty-ninth international joint conference on artificial intelligence, IJCAI 2020. pp 2326\u20132332. https:\/\/doi.org\/10.24963\/ijcai.2020\/322","DOI":"10.24963\/ijcai.2020\/322"},{"issue":"2","key":"10726_CR130","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1145\/2076450.2076469","volume":"55","author":"HH Hoos","year":"2012","unstructured":"Hoos HH (2012) Programming by optimization. Commun ACM 55(2):70\u201380. https:\/\/doi.org\/10.1145\/2076450.2076469","journal-title":"Commun ACM"},{"issue":"09","key":"10726_CR131","doi-asserted-by":"publisher","first-page":"5149","DOI":"10.1109\/TPAMI.2021.3079209","volume":"44","author":"TM Hospedales","year":"2022","unstructured":"Hospedales TM, Antoniou A, Micaelli P et al (2022) Meta-learning in neural networks: a survey. IEEE Trans Pattern Anal Mach Intell 44(09):5149\u20135169. https:\/\/doi.org\/10.1109\/TPAMI.2021.3079209","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"10726_CR132","doi-asserted-by":"crossref","unstructured":"Hu YQ, Yu Y, Tu WW et\u00a0al (2019) Multi-fidelity automatic hyper-parameter tuning via transfer series expansion. In: Proceedings of the AAAI conference on artificial intelligence, Honolulu, HI, USA, 27 January\u20131 February 2019. pp 3846\u20133853","DOI":"10.1609\/aaai.v33i01.33013846"},{"key":"10726_CR133","doi-asserted-by":"crossref","unstructured":"Hu Y, Liang Y, Guo Z et\u00a0al (2020) Angle-based search space shrinking for neural architecture search. In: Proceedings of the 16th European conference on computer vision, Glasgow, UK, 23\u201328 August 2020. pp 119\u2013134","DOI":"10.1007\/978-3-030-58529-7_8"},{"key":"10726_CR134","doi-asserted-by":"publisher","unstructured":"Huang G, Liu Z, van\u00a0der Maaten L et\u00a0al (2017) Densely connected convolutional networks. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21\u201326 July 2017. pp 2261\u20132269. https:\/\/doi.org\/10.1109\/CVPR.2017.243","DOI":"10.1109\/CVPR.2017.243"},{"issue":"6","key":"10726_CR135","doi-asserted-by":"publisher","first-page":"4483","DOI":"10.1007\/s10462-021-10004-4","volume":"54","author":"M Huisman","year":"2021","unstructured":"Huisman M, van Rijn JN, Plaat A (2021) A survey of deep meta-learning. Artif Intell Rev 54(6):4483\u20134541","journal-title":"Artif Intell Rev"},{"key":"10726_CR136","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-023-06394-x","author":"M Huisman","year":"2023","unstructured":"Huisman M, Moerland TM, Plaat A et al (2023) Are LSTMs good few-shot learners? Mach Learn. https:\/\/doi.org\/10.1007\/s10994-023-06394-x","journal-title":"Mach Learn"},{"key":"10726_CR137","unstructured":"Hundt A, Jain V, Hager GD (2019) sharpDARTS: faster and more accurate differentiable architecture search. arXiv. http:\/\/arxiv.org\/abs\/1903.09900"},{"key":"10726_CR138","doi-asserted-by":"publisher","unstructured":"Hutter F, Hoos HH, Leyton-Brown K (2011) Sequential model-based optimization for general algorithm configuration. In: Proceedings of the 5th international conference on learning and intelligent optimization, Lille, France, 12\u201315 January 2015. pp 507\u2013523. https:\/\/doi.org\/10.1007\/978-3-642-25566-3_40","DOI":"10.1007\/978-3-642-25566-3_40"},{"key":"10726_CR139","unstructured":"Hutter F, Hoos HH, Leyton-Brown K (2014) An efficient approach for assessing hyperparameter importance. In: Proceedings of the international conference on machine learning, Beijing, China, 21\u201326 June 2014. pp 754\u2013762"},{"key":"10726_CR140","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-05318-5","volume-title":"Automated machine learning: methods, systems, challenges","author":"F Hutter","year":"2019","unstructured":"Hutter F, Kotthoff L, Vanschoren J (2019) Automated machine learning: methods, systems, challenges. Springer, Cham"},{"key":"10726_CR141","doi-asserted-by":"crossref","unstructured":"Irwin-Harris W, Sun Y, Xue B et\u00a0al (2019) A graph-based encoding for evolutionary convolutional neural network architecture design. In: Proceedings of the 2019 IEEE congress on evolutionary computation, Wellington, New Zealand, 10\u201313 June 2019. pp 546\u2013553","DOI":"10.1109\/CEC.2019.8790093"},{"key":"10726_CR142","doi-asserted-by":"publisher","unstructured":"Istrate R, Scheidegger F, Mariani G et\u00a0al (2019) TAPAS: train-less accuracy predictor for architecture search. In: Proceedings of the thirty-third AAAI conference on artificial intelligence, Honolulu, HI, USA, 27 January\u20131 February 2019. pp 3927\u20133934. https:\/\/doi.org\/10.1609\/aaai.v33i01.33013927","DOI":"10.1609\/aaai.v33i01.33013927"},{"key":"10726_CR143","doi-asserted-by":"publisher","first-page":"57","DOI":"10.1016\/j.imavis.2019.06.005","volume":"89","author":"Y Jaafra","year":"2019","unstructured":"Jaafra Y, Luc Laurent J, Deruyver A et al (2019) Reinforcement learning for neural architecture search: a review. Image Vis Comput 89:57\u201366. https:\/\/doi.org\/10.1016\/j.imavis.2019.06.005","journal-title":"Image Vis Comput"},{"key":"10726_CR144","unstructured":"Jamieson KG, Talwalkar A (2016) Non-stochastic best arm identification and hyperparameter optimization. In: Proceedings of the international conference on artificial intelligence and statistics, Cadiz, Spain, 9\u201311 May 2016. pp 240\u2013248"},{"key":"10726_CR145","unstructured":"Jang E, Gu S, Poole B (2017) Categorical reparameterization with Gumbel-Softmax. In: Proceedings of the 5th international conference on learning representations, Toulon, France, 24\u201326 April 2017"},{"key":"10726_CR146","doi-asserted-by":"publisher","first-page":"47","DOI":"10.3389\/fninf.2018.00047","volume":"12","author":"Z Jedrzejewski-Szmek","year":"2018","unstructured":"Jedrzejewski-Szmek Z, Abrahao KP, Jedrzejewska-Szmek J et al (2018) Parameter optimization using covariance matrix adaptation-evolutionary strategy (CMA-ES), an approach to investigate differences in channel properties between neuron subtypes. Front Neuroinform 12:47. https:\/\/doi.org\/10.3389\/fninf.2018.00047","journal-title":"Front Neuroinform"},{"key":"10726_CR147","doi-asserted-by":"publisher","unstructured":"Jin H, Song Q, Hu X (2019) Auto-Keras: an efficient neural architecture search system. In: Proceedings of the 25th international conference on knowledge discovery & data mining, Anchorage, AK, USA, 4\u20138 August 2019. pp 1946\u20131956. https:\/\/doi.org\/10.1145\/3292500.3330648","DOI":"10.1145\/3292500.3330648"},{"key":"10726_CR148","unstructured":"Jomaa HS, Grabocka J, Schmidt-Thieme L (2019) Hyp-RL: hyperparameter optimization by reinforcement learning. arXiv. http:\/\/arxiv.org\/abs\/1906.11527"},{"issue":"4","key":"10726_CR149","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1023\/A:1008306431147","volume":"13","author":"DR Jones","year":"1998","unstructured":"Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Glob Optim 13(4):455\u2013492. https:\/\/doi.org\/10.1023\/A:1008306431147","journal-title":"J Glob Optim"},{"key":"10726_CR150","unstructured":"Kandasamy K, Neiswanger W, Schneider J et\u00a0al (2018) Neural architecture search with Bayesian optimisation and optimal transport. In: Proceedings of the advances in neural information processing systems 31: annual conference on neural information processing systems, Montr\u00e9al, Canada, 3\u20138 December 2018. pp 2020\u20132029"},{"key":"10726_CR151","unstructured":"Karl F, Pielok T, Moosbauer J et\u00a0al (2022) Multi-objective hyperparameter optimization\u2014an overview. arXiv. http:\/\/arxiv.org\/abs\/2206.07438"},{"issue":"8","key":"10726_CR152","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3470918","volume":"54","author":"SK Karmaker","year":"2021","unstructured":"Karmaker SK, Hassan MM, Smith MJ et al (2021) AutoML to date and beyond: challenges and opportunities. ACM Comput Surv (CSUR) 54(8):1\u201336","journal-title":"ACM Comput Surv (CSUR)"},{"key":"10726_CR153","unstructured":"Ke G, Meng Q, Finley T et\u00a0al (2017) LightGBM: a highly efficient gradient boosting decision tree. In: Proceedings of annual conference on neural information processing systems, Long Beach, CA, USA, 4\u20139 December 2017. pp 3146\u20133154"},{"key":"10726_CR154","doi-asserted-by":"crossref","unstructured":"Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of international conference on neural networks, Perth, WA, Australia, 27 November\u20131 December 1995. pp 1942\u20131948","DOI":"10.1109\/ICNN.1995.488968"},{"key":"10726_CR155","unstructured":"Klein A, Hutter F (2019) Tabular benchmarks for joint architecture and hyperparameter optimization. arXiv. http:\/\/arxiv.org\/abs\/1905.04970"},{"key":"10726_CR156","unstructured":"Klein A, Falkner S, Bartels S et\u00a0al (2017a) Fast Bayesian optimization of machine learning hyperparameters on large datasets. In: Proceedings of the 20th international conference on artificial intelligence and statistics, Fort Lauderdale, FL, USA, 20\u201322 April 2017. pp 528\u2013536"},{"key":"10726_CR157","unstructured":"Klein A, Falkner S, Mansur N et\u00a0al (2017b) RoBO: a flexible and robust Bayesian optimization framework in Python. In: Proceedings of the conference on neural information processing systems Bayesian optimization workshop, Long Beach, CA, USA, 4\u20139 December 2017"},{"key":"10726_CR158","unstructured":"Klein A, Falkner S, Springenberg JT et\u00a0al (2017c) Learning curve prediction with Bayesian neural networks. In: Proceedings of the 5th international conference on learning representations, Toulon, France, 24\u201326 April 2017"},{"key":"10726_CR159","doi-asserted-by":"publisher","first-page":"45736","DOI":"10.1109\/ACCESS.2022.3169897","volume":"10","author":"N Klyuchnikov","year":"2022","unstructured":"Klyuchnikov N, Trofimov I, Artemova E et al (2022) NAS-Bench-NLP: neural architecture search benchmark for natural language processing. IEEE Access 10:45736\u201345747. https:\/\/doi.org\/10.1109\/ACCESS.2022.3169897","journal-title":"IEEE Access"},{"key":"10726_CR160","doi-asserted-by":"publisher","unstructured":"Kocsis L, Szepesv\u00e1ri C (2006) Bandit based Monte-Carlo planning. In: Proceedings of the machine learning: 17th European conference on machine learning, Berlin, Germany, 18\u201322 September 2006, proceedings. pp 282\u2013293. https:\/\/doi.org\/10.1007\/11871842_29","DOI":"10.1007\/11871842_29"},{"key":"10726_CR161","doi-asserted-by":"crossref","unstructured":"Komer B, Bergstra J, Eliasmith C (2014) HyperOPT-Sklearn: automatic hyperparameter configuration for scikit-learn. In: Proceedings of the international conference on machine learning workshop on automated machine learning, Beijing, China, 21\u201326 June 2014. p 50","DOI":"10.25080\/Majora-14bd3278-006"},{"key":"10726_CR162","unstructured":"K\u00f6nig M, Hoos HH, van Rijn JN (2020) Towards algorithm-agnostic uncertainty estimation: predicting classification error in an automated machine learning setting. In: Proceedings of the 7th international conference on machine learning workshop on automated machine learning, virtual event, 13\u201318 July 2020"},{"issue":"9","key":"10726_CR163","doi-asserted-by":"publisher","first-page":"4565","DOI":"10.1007\/s10994-022-06212-w","volume":"111","author":"M K\u00f6nig","year":"2022","unstructured":"K\u00f6nig M, Hoos HH, van Rijn JN (2022) Speeding up neural network robustness verification via algorithm configuration and an optimised mixed integer linear programming solver portfolio. Mach Learn 111(9):4565\u20134584","journal-title":"Mach Learn"},{"key":"10726_CR164","unstructured":"K\u00f6nig M, Bosman AW, Hoos HH et\u00a0al (2023) Critically assessing the state of the art in CPU-based local robustness verification. In: Proceedings of the workshop on artificial intelligence safety, Washington, DC, USA, 7\u201314 February 2023"},{"issue":"25","key":"10726_CR165","first-page":"1","volume":"18","author":"L Kotthoff","year":"2017","unstructured":"Kotthoff L, Thornton C, Hoos HH et al (2017) Auto-WEKA 2.0: automatic model selection and hyperparameter optimization in WEKA. J Mach Learn Res 18(25):1\u20135","journal-title":"J Mach Learn Res"},{"issue":"2","key":"10726_CR166","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1007\/BF00175355","volume":"4","author":"JR Koza","year":"1994","unstructured":"Koza JR (1994) Genetic programming as a means for programming computers by natural selection. Stat Comput 4(2):87\u2013112. https:\/\/doi.org\/10.1007\/BF00175355","journal-title":"Stat Comput"},{"issue":"12","key":"10726_CR167","doi-asserted-by":"publisher","first-page":"2150","DOI":"10.14778\/3229863.3240493","volume":"11","author":"T Kraska","year":"2018","unstructured":"Kraska T (2018) Northstar: an interactive data science system. Proc VLDB Endow 11(12):2150\u20132164. https:\/\/doi.org\/10.14778\/3229863.3240493","journal-title":"Proc VLDB Endow"},{"key":"10726_CR168","unstructured":"Krizhevsky A, Nair V, Hinton G (2010) CIFAR-10 (Canadian Institute for Advanced Research). Master\u2019s Thesis, Department of Computer Science, University of Toronto. p\u00a01"},{"key":"10726_CR169","unstructured":"Lacoste A, Marchand M, Laviolette F et\u00a0al (2014) Agnostic Bayesian learning of ensembles. In: Proceedings of the 31th international conference on machine learning, Beijing, China, 21\u201326 June 2014. pp 611\u2013619"},{"key":"10726_CR170","doi-asserted-by":"crossref","unstructured":"Larochelle H, Erhan D, Courville A et\u00a0al (2007) An empirical evaluation of deep architectures on problems with many factors of variation. In: Proceedings of the 24th international conference on machine learning, Corvallis, Oregon, USA, 20\u201324 June 2007. pp 473\u2013480","DOI":"10.1145\/1273496.1273556"},{"key":"10726_CR171","unstructured":"LasagneContributors (2022) Lasagne. https:\/\/github.com\/Lasagne\/Lasagne. Accessed 4 Nov 2022"},{"key":"10726_CR172","unstructured":"LeDell E, Poirier S (2020) H2O AutoML: scalable automatic machine learning. In: Proceedings of the 7th international conference on machine learning workshop on automated machine learning, virtual event, 13\u201318 July 2020"},{"key":"10726_CR173","doi-asserted-by":"publisher","unstructured":"Leite R, Brazdil P (2005) Predicting relative performance of classifiers from samples. In: Machine learning, proceedings of the twenty-second international conference, Bonn, Germany, 7\u201311 August 2005. pp 497\u2013503. https:\/\/doi.org\/10.1145\/1102351.1102414","DOI":"10.1145\/1102351.1102414"},{"key":"10726_CR174","unstructured":"Leite R, Brazdil P (2010) Active testing strategy to predict the best classification algorithm via sampling and metalearning. In: Proceedings of the 19th European conference on artificial intelligence, Lisbon, Portugal, 16\u201320 August 2010. pp 309\u2013314"},{"key":"10726_CR175","unstructured":"Li L, Talwalkar A (2019) Random search and reproducibility for neural architecture search. In: Proceedings of the thirty-fifth conference on uncertainty in artificial intelligence, Tel Aviv, Israel, 22\u201325 July 2019. p 129"},{"issue":"185","key":"10726_CR176","first-page":"1","volume":"18","author":"L Li","year":"2017","unstructured":"Li L, Jamieson KG, DeSalvo G et al (2017) Hyperband: a novel bandit-based approach to hyperparameter optimization. J Mach Learn Res 18(185):1\u201352","journal-title":"J Mach Learn Res"},{"key":"10726_CR177","doi-asserted-by":"crossref","unstructured":"Li YF, Wang H, Wei T et\u00a0al (2019) Towards automated semi-supervised learning. In: Proceedings of the AAAI conference on artificial intelligence, virtual event, 2\u20139 February 2021. pp 4237\u20134244","DOI":"10.1609\/aaai.v33i01.33014237"},{"key":"10726_CR178","unstructured":"Li L, Jamieson KG, Rostamizadeh A et\u00a0al (2020a) A system for massively parallel hyperparameter tuning. In: Proceedings of the machine learning and systems, Austin, TX, USA, 2\u20134 March 2020"},{"key":"10726_CR179","doi-asserted-by":"crossref","unstructured":"Li T, Zhang J, Bao K et\u00a0al (2020b) AutoST: efficient neural architecture search for spatio-temporal prediction. In: Proceedings of the 26th international conference on knowledge discovery & data mining, virtual event, 23\u201327 August 2020. pp 794\u2013802","DOI":"10.1145\/3394486.3403122"},{"key":"10726_CR180","doi-asserted-by":"publisher","unstructured":"Li Z, Deng J, Zhang G et\u00a0al (2020c) GP-NAS: Gaussian process based neural architecture search. In: 2020 IEEE conference on computer vision and pattern recognition, Seattle, WA, USA, 13\u201319 June 2020. pp 11930\u201311939. https:\/\/doi.org\/10.1109\/CVPR42600.2020.01195","DOI":"10.1109\/CVPR42600.2020.01195"},{"key":"10726_CR181","doi-asserted-by":"crossref","unstructured":"Li Y, Wen Z, Wang Y et\u00a0al (2021) One-shot graph neural architecture search with dynamic search space. In: Thirty-fifth AAAI conference on artificial intelligence, virtual event, 2\u20139 February 2021. pp 8510\u20138517","DOI":"10.1609\/aaai.v35i10.17033"},{"key":"10726_CR182","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.5805","author":"H Li","year":"2022","unstructured":"Li H, Liang Q, Chen M et al (2022) Pruning SMAC search space based on key hyperparameters. Concurr Comput Pract Exp. https:\/\/doi.org\/10.1002\/cpe.5805","journal-title":"Concurr Comput Pract Exp"},{"key":"10726_CR183","unstructured":"Liang H, Zhang S, Sun J et al (2019) DARTS+: improved differentiable architecture search with early stopping. arXiv. http:\/\/arxiv.org\/abs\/1909.06035"},{"key":"10726_CR184","unstructured":"Liaw R, Liang E, Nishihara R et\u00a0al (2018) Tune: a research platform for distributed model selection and training. arXiv. http:\/\/arxiv.org\/abs\/1807.05118"},{"issue":"243","key":"10726_CR185","first-page":"1","volume":"21","author":"M Lindauer","year":"2020","unstructured":"Lindauer M, Hutter F (2020) Best practices for scientific research on neural architecture search. J Mach Learn Res 21(243):1\u201318","journal-title":"J Mach Learn Res"},{"issue":"54","key":"10726_CR186","first-page":"1","volume":"23","author":"M Lindauer","year":"2022","unstructured":"Lindauer M, Eggensperger K, Feurer M et al (2022) SMAC3: a versatile Bayesian optimization package for hyperparameter optimization. J Mach Learn Res 23(54):1\u20139","journal-title":"J Mach Learn Res"},{"key":"10726_CR187","unstructured":"LinuxFoundation (2022) PyTorch. https:\/\/pytorch.org. Accessed 4 Nov 2022"},{"key":"10726_CR188","doi-asserted-by":"publisher","unstructured":"Liu C, Zoph B, Neumann M et\u00a0al (2018a) Progressive neural architecture search. In: Proceedings of the 15th European conference on computer vision, Munich, Germany, 8\u201314 September 2018. pp 19\u201335. https:\/\/doi.org\/10.1007\/978-3-030-01246-5_2","DOI":"10.1007\/978-3-030-01246-5_2"},{"key":"10726_CR189","unstructured":"Liu H, Simonyan K, Vinyals O et\u00a0al (2018b) Hierarchical representations for efficient architecture search. In: Proceedings of the 6th international conference on learning representations, Vancouver, BC, Canada, 30 April\u20133 May 2018"},{"key":"10726_CR190","doi-asserted-by":"publisher","unstructured":"Liu C, Chen LC, Schroff F et\u00a0al (2019a) Auto-DeepLab: hierarchical neural architecture search for semantic image segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, 16\u201320 June 2019. pp 82\u201392. https:\/\/doi.org\/10.1109\/CVPR.2019.00017","DOI":"10.1109\/CVPR.2019.00017"},{"key":"10726_CR191","unstructured":"Liu H, Simonyan K, Yang Y (2019b) DARTS: differentiable architecture search. In: Proceedings of the 7th international conference on learning representations, New Orleans, LA, USA, 6\u20139 May 2019"},{"key":"10726_CR192","doi-asserted-by":"crossref","unstructured":"Liu S, Ram P, Vijaykeerthy D et\u00a0al (2020) An ADMM based framework for AutoML pipeline configuration. In: Proceedings of the thirty-fourth AAAI conference on artificial intelligence, New York, NY, USA, 7\u201312 February 2020. pp 4892\u20134899","DOI":"10.1609\/aaai.v34i04.5926"},{"key":"10726_CR193","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2021.3100554","author":"Y Liu","year":"2021","unstructured":"Liu Y, Sun Y, Xue B et al (2021) A survey on evolutionary neural architecture search. IEEE Trans Neural Netw Learn Syst. https:\/\/doi.org\/10.1109\/TNNLS.2021.3100554","journal-title":"IEEE Trans Neural Netw Learn Syst"},{"key":"10726_CR194","doi-asserted-by":"publisher","unstructured":"Long FX, van Stein B, Frenzel M et\u00a0al (2022) Learning the characteristics of engineering optimization problems with applications in automotive crash. In: Fieldsend JE, Wagner M (eds) GECCO \u201922: genetic and evolutionary computation conference, Boston, MA, USA, 9\u201313 July 2022. pp 1227\u20131236. https:\/\/doi.org\/10.1145\/3512290.3528712","DOI":"10.1145\/3512290.3528712"},{"key":"10726_CR195","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.orp.2016.09.002","volume":"3","author":"M L\u00f3pez-Ib\u00e1\u00f1ez","year":"2016","unstructured":"L\u00f3pez-Ib\u00e1\u00f1ez M, Dubois-Lacoste J, C\u00e1ceres LP et al (2016) The irace package: iterated racing for automatic algorithm configuration. Oper Res Perspect 3:43\u201358. https:\/\/doi.org\/10.1016\/j.orp.2016.09.002","journal-title":"Oper Res Perspect"},{"key":"10726_CR196","doi-asserted-by":"crossref","unstructured":"Lorenzo PR, Nalepa J (2018) Memetic evolution of deep neural networks. In: Proceedings of the genetic and evolutionary computation conference, Kyoto, Japan, 15\u201319 July 2018. pp 505\u2013512","DOI":"10.1145\/3205455.3205631"},{"key":"10726_CR197","unstructured":"Lorraine J, Vicol P, Duvenaud D (2020) Optimizing millions of hyperparameters by implicit differentiation. In: Proceedings of the international conference on artificial intelligence and statistics, virtual event, 26\u201328 August 2020. pp 1540\u20131552"},{"key":"10726_CR198","unstructured":"Loshchilov I, Hutter F (2016) CMA-ES for hyperparameter optimization of deep neural networks. arXiv. http:\/\/arxiv.org\/abs\/1604.07269"},{"key":"10726_CR199","doi-asserted-by":"publisher","unstructured":"Loshchilov I, Schoenauer M, Sebag M (2012) Self-adaptive surrogate-assisted covariance matrix adaptation evolution strategy. In: Proceedings of the genetic and evolutionary computation conference, Philadelphia, PA, USA, 7\u201311 July 2012. pp 321\u2013328. https:\/\/doi.org\/10.1145\/2330163.2330210","DOI":"10.1145\/2330163.2330210"},{"key":"10726_CR200","doi-asserted-by":"publisher","unstructured":"Loshchilov I, Schoenauer M, Sebag M (2013) Bi-population CMA-ES algorithms with surrogate models and line searches. In: Proceedings of the genetic and evolutionary computation conference, Amsterdam, The Netherlands, 6\u201310 July 2013, companion material proceedings. pp 1177\u20131184. https:\/\/doi.org\/10.1145\/2464576.2482696","DOI":"10.1145\/2464576.2482696"},{"key":"10726_CR201","doi-asserted-by":"crossref","unstructured":"Louren\u00e7o N, Assun\u00e7\u00e3o F, Pereira FB et\u00a0al (2018) Structured grammatical evolution: a dynamic approach. In: Handbook of grammatical evolution. Springer, pp 137\u2013161","DOI":"10.1007\/978-3-319-78717-6_6"},{"key":"10726_CR202","doi-asserted-by":"crossref","unstructured":"Lu Z, Whalen I, Boddeti V et\u00a0al (2019) NSGA-Net: neural architecture search using multi-objective genetic algorithm. In: Proceedings of the genetic and evolutionary computation conference, Prague, Czech Republic, 13\u201317 July 2019. pp 419\u2013427","DOI":"10.1145\/3321707.3321729"},{"key":"10726_CR203","doi-asserted-by":"publisher","unstructured":"Lu Z, Whalen I, Dhebar YD et\u00a0al (2020) NSGA-Net: neural architecture search using multi-objective genetic algorithm (extended abstract). In: Proceedings of the twenty-ninth international joint conference on artificial intelligence, virtual event, 7\u20138 January 2021. pp 4750\u20134754. https:\/\/doi.org\/10.24963\/ijcai.2020\/659","DOI":"10.24963\/ijcai.2020\/659"},{"key":"10726_CR204","unstructured":"Lu S, Li J, Tan J et\u00a0al (2021) TNASP: a transformer-based NAS predictor with a self-evolution framework. In: Proceedings of the advances in neural information processing systems 34: annual conference on neural information processing systems, virtual event, 6\u201314 December 2021. pp 15125\u201315137"},{"key":"10726_CR205","unstructured":"Luo R, Tian F, Qin T et\u00a0al (2018) Neural architecture optimization. In: Proceedings of the advances in neural information processing systems 31: annual conference on neural information processing systems, Montr\u00e9al, Canada, 3\u20138 December 2018. pp 7827\u20137838"},{"key":"10726_CR206","unstructured":"Maclaurin D, Duvenaud D, Adams R (2015) Gradient-based hyperparameter optimization through reversible learning. In: Proceedings of the international conference on machine learning, Lille, France, 6\u201311 July 2015. pp 2113\u20132122"},{"key":"10726_CR207","unstructured":"Mahsereci M, Balles L, Lassner C et al (2017) Early stopping without a validation set. arXiv. http:\/\/arxiv.org\/abs\/1703.09580"},{"key":"10726_CR208","doi-asserted-by":"crossref","unstructured":"Marcus M, Kim G, Marcinkiewicz MA et\u00a0al (1994) The Penn treebank: annotating predicate argument structure. In: Human language technology: proceedings of a workshop held at Plainsboro, New Jersey, 8\u201311 March 1994","DOI":"10.3115\/1075812.1075835"},{"issue":"1","key":"10726_CR209","first-page":"3735","volume":"15","author":"R Martinez-Cantin","year":"2014","unstructured":"Martinez-Cantin R (2014) BayesOPT: a Bayesian optimization library for nonlinear optimization, experimental design and bandits. J Mach Learn Res 15(1):3735\u20133739","journal-title":"J Mach Learn Res"},{"issue":"3\u20134","key":"10726_CR210","doi-asserted-by":"publisher","first-page":"365","DOI":"10.1007\/s10710-010-9109-y","volume":"11","author":"RI McKay","year":"2010","unstructured":"McKay RI, Hoai NX, Whigham PA et al (2010) Grammar-based genetic programming: a survey. Genet Program Evol Mach 11(3\u20134):365\u2013396","journal-title":"Genet Program Evol Mach"},{"key":"10726_CR211","unstructured":"Mellor J, Turner J, Storkey A et\u00a0al (2021) Neural architecture search without training. In: Proceedings of the international conference on machine learning, virtual event, 18\u201324 July 2021. pp 7588\u20137598"},{"key":"10726_CR212","unstructured":"Mendoza H, Klein A, Feurer M et\u00a0al (2016) Towards automatically-tuned neural networks. In: Proceedings of the workshop on automatic machine learning, New York, NY, USA, 24 June 2016. pp 58\u201365"},{"key":"10726_CR213","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2022.3179131","author":"MH Meng","year":"2022","unstructured":"Meng MH, Bai G, Teo SG et al (2022) Adversarial robustness of deep neural networks: a survey from a formal verification perspective. IEEE Trans Depend Secur Comput. https:\/\/doi.org\/10.1109\/TDSC.2022.3179131","journal-title":"IEEE Trans Depend Secur Comput"},{"key":"10726_CR214","unstructured":"Microsoft (2021) Microsoft neural network intelligence. https:\/\/github.com\/microsoft\/nni. Accessed 4 Nov 2021"},{"key":"10726_CR215","doi-asserted-by":"crossref","unstructured":"Miikkulainen R, Liang J, Meyerson E et\u00a0al (2019) Evolving deep neural networks. In: Artificial intelligence in the age of neural networks and brain computing. Elsevier, pp 293\u2013312","DOI":"10.1016\/B978-0-12-815480-9.00015-3"},{"issue":"2","key":"10726_CR216","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1023\/A:1017988514716","volume":"49","author":"JDR Mill\u00e1n","year":"2002","unstructured":"Mill\u00e1n JDR, Posenato D, Dedieu E (2002) Continuous-action q-learning. Mach Learn 49(2):247\u2013265. https:\/\/doi.org\/10.1023\/A:1017988514716","journal-title":"Mach Learn"},{"key":"10726_CR217","unstructured":"Mohan A, Benjamins C, Wienecke K et\u00a0al (2023) AutoRL hyperparameter landscapes. In: Proceedings of the AutoML conference, Potsdam, Germany, 12\u201315 September 2023"},{"key":"10726_CR218","unstructured":"Mohr F, van Rijn JN (2022) Learning curves for decision making in supervised machine learning\u2014a survey. arXiv. http:\/\/arxiv.org\/abs\/2201.12150"},{"issue":"8","key":"10726_CR219","doi-asserted-by":"publisher","first-page":"9669","DOI":"10.1109\/TPAMI.2023.3251957","volume":"45","author":"F Mohr","year":"2023","unstructured":"Mohr F, van Rijn JN (2023) Fast and informative model selection using learning curve cross-validation. IEEE Trans Pattern Anal Mach Intell 45(8):9669\u20139680","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"10726_CR220","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-022-06200-0","author":"F Mohr","year":"2022","unstructured":"Mohr F, Wever M (2022) Naive automated machine learning. Mach Learn. https:\/\/doi.org\/10.1007\/s10994-022-06200-0","journal-title":"Mach Learn"},{"issue":"8","key":"10726_CR221","doi-asserted-by":"publisher","first-page":"1495","DOI":"10.1007\/s10994-018-5735-z","volume":"107","author":"F Mohr","year":"2018","unstructured":"Mohr F, Wever M, H\u00fcllermeier E (2018) ML-Plan: automated machine learning via hierarchical planning. Mach Learn 107(8):1495\u20131515. https:\/\/doi.org\/10.1007\/s10994-018-5735-z","journal-title":"Mach Learn"},{"key":"10726_CR222","doi-asserted-by":"crossref","unstructured":"Mohr F, Viering TJ, Loog M et\u00a0al (2022) LCDB 1.0: an extensive learning curves database for classification tasks. In: Proceedings of the joint European conference on machine learning and knowledge discovery in databases, Grenoble, France, 19\u201323 September 2022","DOI":"10.1007\/978-3-031-26419-1_1"},{"key":"10726_CR223","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-023-06389-8","author":"C Moussa","year":"2023","unstructured":"Moussa C, Patel YJ, Dunjko V et al (2023) Hyperparameter importance and optimization of quantum neural networks across small datasets. Mach Learn. https:\/\/doi.org\/10.1007\/s10994-023-06389-8","journal-title":"Mach Learn"},{"key":"10726_CR224","unstructured":"Mu\u00f1oz JP, Lyalyushkin N, Lacewell CW et\u00a0al (2022) Automated super-network generation for scalable neural architecture search. In: Proceedings of the international conference on automated machine learning, Baltimore, MD, USA, 25\u201327 July 2022, Johns Hopkins University. pp 5\/1\u201315"},{"key":"10726_CR225","unstructured":"Negrinho R, Gordon GJ (2017) DeepArchitect: automatically designing and training deep architectures. arXiv. http:\/\/arxiv.org\/abs\/1704.08792"},{"key":"10726_CR226","unstructured":"Nguyen V, Le T, Yamada M et\u00a0al (2021) Optimal transport kernels for sequential and parallel neural architecture search. In: Proceedings of the international conference on machine learning, virtual event, 18\u201324 July 2021. pp 8084\u20138095"},{"issue":"4","key":"10726_CR227","doi-asserted-by":"publisher","first-page":"609","DOI":"10.1109\/TAI.2021.3121663","volume":"3","author":"N Nguyen","year":"2022","unstructured":"Nguyen N, Chang JM (2022) CSNAS: contrastive self-supervised learning neural architecture search via sequential model-based optimization. IEEE Trans Artif Intell 3(4):609\u2013624. https:\/\/doi.org\/10.1109\/TAI.2021.3121663","journal-title":"IEEE Trans Artif Intell"},{"key":"10726_CR228","unstructured":"Nichol A, Achiam J, Schulman J (2018) On first-order meta-learning algorithms. arXiv. http:\/\/arxiv.org\/abs\/1803.02999"},{"key":"10726_CR229","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1016\/j.future.2021.08.022","volume":"127","author":"NO Nikitin","year":"2022","unstructured":"Nikitin NO, Vychuzhanin P, Sarafanov M et al (2022) Automated evolutionary approach for the design of composite machine learning pipelines. Future Gener Comput Syst 127:109\u2013125. https:\/\/doi.org\/10.1016\/j.future.2021.08.022","journal-title":"Future Gener Comput Syst"},{"key":"10726_CR230","unstructured":"Nogueira F (2014) Bayesian optimization: open source constrained global optimization tool for Python. https:\/\/github.com\/fmfn\/BayesianOptimization. Accessed 4 Nov 2022"},{"key":"10726_CR231","doi-asserted-by":"publisher","unstructured":"Olson RS, Bartley N, Urbanowicz RJ et\u00a0al (2016a) Evaluation of a tree-based pipeline optimization tool for automating data science. In: Proceedings of the genetic and evolutionary computation conference, Denver, CO, USA, 20\u201324 July 2016. pp 485\u2013492. https:\/\/doi.org\/10.1145\/2908812.2908918","DOI":"10.1145\/2908812.2908918"},{"key":"10726_CR232","doi-asserted-by":"publisher","unstructured":"Olson RS, Urbanowicz RJ, Andrews PC et\u00a0al (2016b) Automating biomedical data science through tree-based pipeline optimization. In: Proceedings of the applications of evolutionary computation: 19th European conference, Porto, Portugal, 30 March\u20131 April 2016. pp 123\u2013137. https:\/\/doi.org\/10.1007\/978-3-319-31204-0_9","DOI":"10.1007\/978-3-319-31204-0_9"},{"issue":"6","key":"10726_CR233","doi-asserted-by":"publisher","first-page":"2602","DOI":"10.1007\/s10618-021-00781-5","volume":"35","author":"G Ottervanger","year":"2021","unstructured":"Ottervanger G, Baratchi M, Hoos HH (2021) MultiETSC: automated machine learning for early time series classification. Data Min Knowl Disc 35(6):2602\u20132654","journal-title":"Data Min Knowl Disc"},{"key":"10726_CR234","unstructured":"\u00d6zt\u00fcrk E, Ferreira F, Jomaa H et\u00a0al (2022) Zero-shot AutoML with pretrained models. In: Proceedings of the international conference on machine learning, Baltimore, MD, USA, 17\u201323 July 2022. pp 17138\u201317155"},{"key":"10726_CR235","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1613\/jair.1.13596","volume":"74","author":"J Parker-Holder","year":"2022","unstructured":"Parker-Holder J, Rajan R, Song X et al (2022) Automated reinforcement learning (AutoRL): a survey and open problems. J Artif Intell Res 74:517\u2013568","journal-title":"J Artif Intell Res"},{"key":"10726_CR236","doi-asserted-by":"publisher","unstructured":"Parmentier L, Nicol O, Jourdan L et\u00a0al (2019) TPOT-SH: a faster optimization algorithm to solve the AutoML problem on large datasets. In: Proceedings of the 31st IEEE international conference on tools with artificial intelligence, Portland, OR, USA, 4\u20136 November 2019. pp 471\u2013478. https:\/\/doi.org\/10.1109\/ICTAI.2019.00072","DOI":"10.1109\/ICTAI.2019.00072"},{"key":"10726_CR237","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825\u20132830","journal-title":"J Mach Learn Res"},{"key":"10726_CR238","unstructured":"Perrone V, Shen H, Seeger MW et\u00a0al (2019) Learning search spaces for Bayesian optimization: another view of hyperparameter transfer learning. In: Proceedings of the advances in neural information processing systems 32: annual conference on neural information processing systems, Vancouver, BC, Canada, 8\u201314 December 2019. pp 12751\u201312761"},{"key":"10726_CR239","doi-asserted-by":"crossref","unstructured":"Perrone V, Donini M, Zafar MB et\u00a0al (2021) Fair Bayesian optimization. In: Proceedings of the 2021 AAAI\/ACM conference on AI, ethics, and society, virtual event, 19\u201321 May 2021. pp 854\u2013863","DOI":"10.1145\/3461702.3462629"},{"issue":"5\u20136","key":"10726_CR240","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1561\/2200000073","volume":"11","author":"G Peyr\u00e9","year":"2019","unstructured":"Peyr\u00e9 G, Cuturi M (2019) Computational optimal transport. Found Trends Mach Learn 11(5\u20136):355\u2013607. https:\/\/doi.org\/10.1561\/2200000073","journal-title":"Found Trends Mach Learn"},{"key":"10726_CR241","doi-asserted-by":"publisher","unstructured":"Pfisterer F, van Rijn JN, Probst P et\u00a0al (2021) Learning multiple defaults for machine learning algorithms. In: Genetic and evolutionary computation conference, companion volume, Lille, France, 10\u201314 July 2021. pp 241\u2013242. https:\/\/doi.org\/10.1145\/3449726.3459523","DOI":"10.1145\/3449726.3459523"},{"key":"10726_CR242","unstructured":"Pham H, Guan M, Zoph B et\u00a0al (2018) Efficient neural architecture search via parameters sharing. In: Proceedings of the 35th international conference on machine learning, Stockholmsm\u00e4ssan, Stockholm, Sweden, 10\u201315 July 2018. pp 4095\u20134104"},{"key":"10726_CR243","doi-asserted-by":"crossref","unstructured":"Pinto F, Soares C, Mendes-Moreira J (2016) Towards automatic generation of metafeatures. In: Advances in knowledge discovery and data mining\u201420th Pacific-Asia conference, PAKDD 2016, Auckland, New Zealand. pp 215\u2013226","DOI":"10.1007\/978-3-319-31753-3_18"},{"key":"10726_CR244","first-page":"1934","volume":"20","author":"P Probst","year":"2019","unstructured":"Probst P, Boulesteix A, Bischl B (2019) Tunability: importance of hyperparameters of machine learning algorithms. J Mach Learn Res 20:1934\u20131965","journal-title":"J Mach Learn Res"},{"key":"10726_CR245","doi-asserted-by":"crossref","unstructured":"Provost FJ, Jensen DD, Oates T (1999) Efficient progressive sampling. In: Proceedings of the fifth international conference on knowledge discovery and data mining, San Diego, CA, USA, 15\u201318 August 1999. pp 23\u201332","DOI":"10.1145\/312129.312188"},{"key":"10726_CR246","doi-asserted-by":"crossref","unstructured":"Pukelsheim F (2006) Optimal design of experiments, SIAM Classics edn. SIAM","DOI":"10.1137\/1.9780898719109"},{"key":"10726_CR247","unstructured":"Purucker L, Beel J (2023) Assembled-OpenML: creating efficient benchmarks for ensembles in AutoML with OpenML. arXiv"},{"key":"10726_CR248","doi-asserted-by":"publisher","DOI":"10.1145\/3558774","author":"Y Pushak","year":"2022","unstructured":"Pushak Y, Hoos H (2022) AutoML loss landscapes. ACM Trans Evol Learn Optim. https:\/\/doi.org\/10.1145\/3558774","journal-title":"ACM Trans Evol Learn Optim"},{"key":"10726_CR249","unstructured":"Rajeswaran A, Finn C, Kakade SM et\u00a0al (2019) Meta-learning with implicit gradients. In: Proceedings of the advances in neural information processing systems 32: annual conference on neural information processing systems 2019, NeurIPS 2019, 8\u201314 December 2019, Vancouver, BC, Canada. pp 113\u2013124"},{"key":"10726_CR250","doi-asserted-by":"publisher","unstructured":"Rakotoarison H, Schoenauer M, Sebag M (2019) Automated machine learning with Monte-Carlo tree search. In: Proceedings of the twenty-eighth international joint conference on artificial intelligence, Macao, China, 10\u201316 August 2019. pp 3296\u20133303. https:\/\/doi.org\/10.24963\/ijcai.2019\/457","DOI":"10.24963\/ijcai.2019\/457"},{"issue":"21","key":"10726_CR251","first-page":"1","volume":"17","author":"J Read","year":"2016","unstructured":"Read J, Reutemann P, Pfahringer B et al (2016) MEKA: a multi-label\/multi-target extension to Weka. J Mach Learn Res 17(21):1\u20135","journal-title":"J Mach Learn Res"},{"key":"10726_CR252","unstructured":"Real E, Moore S, Selle A et\u00a0al (2017) Large-scale evolution of image classifiers. In: Proceedings of the 34th international conference on machine learning, Sydney, NSW, Australia, 6\u201311 August 2017. pp 2902\u20132911"},{"key":"10726_CR253","doi-asserted-by":"publisher","unstructured":"Real E, Aggarwal A, Huang Y et\u00a0al (2019) Regularized evolution for image classifier architecture search. In: Proceedings of the thirty-third AAAI conference on artificial intelligence, Honolulu, HI, USA, 27 January\u20131 February 2019. pp 4780\u20134789. https:\/\/doi.org\/10.1609\/aaai.v33i01.33014780","DOI":"10.1609\/aaai.v33i01.33014780"},{"issue":"4","key":"10726_CR254","doi-asserted-by":"publisher","first-page":"76:1","DOI":"10.1145\/3447582","volume":"54","author":"P Ren","year":"2022","unstructured":"Ren P, Xiao Y, Chang X et al (2022) A comprehensive survey of neural architecture search: challenges and solutions. ACM Comput Surv 54(4):76:1-76:34. https:\/\/doi.org\/10.1145\/3447582","journal-title":"ACM Comput Surv"},{"key":"10726_CR255","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1016\/S0065-2458(08)60520-3","volume":"15","author":"JR Rice","year":"1976","unstructured":"Rice JR (1976) The algorithm selection problem. Adv Comput 15:65\u2013118","journal-title":"Adv Comput"},{"issue":"108","key":"10726_CR256","first-page":"101","volume":"240","author":"A Rivolli","year":"2022","unstructured":"Rivolli A, Garcia LP, Soares C et al (2022) Meta-features for meta-learning. Knowl Based Syst 240(108):101","journal-title":"Knowl Based Syst"},{"key":"10726_CR257","unstructured":"Ru BX, Wan X, Dong X et\u00a0al (2020a) Neural architecture search using Bayesian optimisation with Weisfeiler-Lehman kernel. arXiv. http:\/\/arxiv.org\/abs\/2006.07556"},{"key":"10726_CR258","first-page":"12057","volume":"33","author":"R Ru","year":"2020","unstructured":"Ru R, Esperanca P, Carlucci FM (2020b) Neural architecture generator optimization. Adv Neural Inf Process Syst 33:12057\u201312069","journal-title":"Adv Neural Inf Process Syst"},{"key":"10726_CR259","doi-asserted-by":"crossref","unstructured":"Sabharwal A, Samulowitz H, Tesauro G (2016) Selecting near-optimal learners via incremental data allocation. In: Proceedings of the thirtieth AAAI conference on artificial intelligence, Phoenix, AZ, USA, 12\u201317 February 2016. pp 2007\u20132015","DOI":"10.1609\/aaai.v30i1.10316"},{"issue":"1","key":"10726_CR260","doi-asserted-by":"publisher","first-page":"52","DOI":"10.1016\/j.jiixd.2023.10.002","volume":"2","author":"I Salehin","year":"2024","unstructured":"Salehin I, Islam MS, Saha P et al (2024) AutoML: a systematic review on automated machine learning with neural architecture search. J Inf Intell 2(1):52\u201381. https:\/\/doi.org\/10.1016\/j.jiixd.2023.10.002","journal-title":"J Inf Intell"},{"key":"10726_CR261","doi-asserted-by":"crossref","unstructured":"Salinas NRP, Baratchi M, van Rijn JN et\u00a0al (2021) Automated machine learning for satellite data: integrating remote sensing pre-trained models into AutoML systems. In: Proceedings of the joint European conference on machine learning and knowledge discovery in databases, virtual event, 13\u201317 September 2021. pp 447\u2013462","DOI":"10.1007\/978-3-030-86517-7_28"},{"key":"10726_CR262","unstructured":"Salinas D, Seeger MW, Klein A et\u00a0al (2022) Syne Tune: a library for large scale hyperparameter tuning and reproducible research. In: International conference on automated machine learning, Johns Hopkins University, Baltimore, MD, USA, 25\u201327 July 2022. pp 16\/1\u201323"},{"key":"10726_CR263","first-page":"4053","volume":"29","author":"S Saxena","year":"2016","unstructured":"Saxena S, Verbeek J (2016) Convolutional neural fabrics. Adv Neural Inf Process Syst 29:4053\u20134061","journal-title":"Adv Neural Inf Process Syst"},{"key":"10726_CR264","doi-asserted-by":"publisher","unstructured":"Schneider L, Sch\u00e4permeier L, Prager RP et\u00a0al (2022) HPO $${}^\\times$$ ELA: investigating hyperparameter optimization landscapes by means of exploratory landscape analysis. In: Rudolph G, Kononova AV, Aguirre HE et\u00a0al (eds) Parallel problem solving from nature\u2014PPSN XVII\u201417th international conference, PPSN 2022, Dortmund, Germany, 10\u201314 September 2022, proceedings, part I. pp 575\u2013589. https:\/\/doi.org\/10.1007\/978-3-031-14714-2_40","DOI":"10.1007\/978-3-031-14714-2_40"},{"key":"10726_CR265","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2211.04148","author":"A Scriven","year":"2022","unstructured":"Scriven A, Kedziora DJ, Musial K et al (2022) The technological emergence of AutoML: a survey of performant software and applications in the context of industry. arXiv. https:\/\/doi.org\/10.48550\/arXiv.2211.04148","journal-title":"arXiv"},{"key":"10726_CR266","unstructured":"Seeger MW, Williams CK, Lawrence ND (2003) Fast forward selection to speed up sparse Gaussian process regression. In: Proceedings of the international workshop on artificial intelligence and statistics, Key West, FL, USA, 3\u20136 January 2003. pp 254\u2013261"},{"issue":"1","key":"10726_CR267","doi-asserted-by":"publisher","first-page":"148","DOI":"10.1109\/JPROC.2015.2494218","volume":"104","author":"B Shahriari","year":"2016","unstructured":"Shahriari B, Swersky K, Wang Z et al (2016) Taking the human out of the loop: a review of Bayesian optimization. Proc IEEE 104(1):148\u2013175. https:\/\/doi.org\/10.1109\/JPROC.2015.2494218","journal-title":"Proc IEEE"},{"key":"10726_CR268","doi-asserted-by":"publisher","unstructured":"Shang Z, Zgraggen E, Buratti B et\u00a0al (2019) Democratizing data science through interactive curation of ML pipelines. In: Proceedings of the 2019 international conference on management of data. pp 1171\u20131188. https:\/\/doi.org\/10.1145\/3299869.3319863","DOI":"10.1145\/3299869.3319863"},{"key":"10726_CR269","doi-asserted-by":"publisher","unstructured":"Sharma A, van Rijn JN, Hutter F et\u00a0al (2019) Hyperparameter importance for image classification by residual neural networks. In: Discovery science\u201422nd international conference, Split, Croatia, 28\u201330 October 2019, proceedings. pp 112\u2013126. https:\/\/doi.org\/10.1007\/978-3-030-33778-0_10","DOI":"10.1007\/978-3-030-33778-0_10"},{"key":"10726_CR270","unstructured":"Shchur O, Turkmen AC, Erickson N et\u00a0al (2023) AutoGluon\u2013TimeSeries: AutoML for probabilistic time series forecasting. In: Proceedings of the AutoML conference, Potsdam, Germany, 12\u201315 September 2023"},{"key":"10726_CR271","unstructured":"Shen Y, Song K, Tan X et al (2023) HuggingGPT: solving AI tasks with ChatGPT and its friends in HuggingFace. arXiv. http:\/\/arxiv.org\/abs\/2303.17580"},{"key":"10726_CR272","unstructured":"Shi H, Pi R, Xu H et\u00a0al (2020) Bridging the gap between sample-based and one-shot neural architecture search with BONAS. In: Proceedings of the advances in neural information processing systems 33: annual conference on neural information processing systems, virtual event, 6\u201312 December 2020"},{"key":"10726_CR273","unstructured":"Shin R, Packer C, Song D (2018) Differentiable neural network architecture search. In: Workshop track proceedings of the 6th international conference on learning representations, Vancouver, BC, Canada, 30 April\u20133 May 2018"},{"issue":"7676","key":"10726_CR274","doi-asserted-by":"publisher","first-page":"354","DOI":"10.1038\/nature24270","volume":"550","author":"D Silver","year":"2017","unstructured":"Silver D, Schrittwieser J, Simonyan K et al (2017) Mastering the game of go without human knowledge. Nature 550(7676):354\u2013359","journal-title":"Nature"},{"issue":"6419","key":"10726_CR275","doi-asserted-by":"publisher","first-page":"1140","DOI":"10.1126\/science.aar6404","volume":"362","author":"D Silver","year":"2018","unstructured":"Silver D, Hubert T, Schrittwieser J et al (2018) A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science 362(6419):1140\u20131144","journal-title":"Science"},{"key":"10726_CR276","volume-title":"Evolutionary optimization algorithms","author":"D Simon","year":"2013","unstructured":"Simon D (2013) Evolutionary optimization algorithms. Wiley Online Library, Hoboken"},{"key":"10726_CR277","unstructured":"Snoek J, Larochelle H, Adams RP (2012) Practical Bayesian optimization of machine learning algorithms. In: Proceedings of the 25th international conference on neural information processing systems, Lake Tahoe, NV, USA, 3\u20136 Dec 2012. pp 2951\u20132959"},{"key":"10726_CR278","unstructured":"Snoek J, Swersky K, Zemel RS et\u00a0al (2014) Input warping for Bayesian optimization of non-stationary functions. In: Proceedings of the 31th international conference on machine learning, Beijing, China, 21\u201326 June 2014. pp 1674\u20131682"},{"issue":"4","key":"10726_CR279","first-page":"407","volume":"1","author":"IM Sobol","year":"1993","unstructured":"Sobol IM (1993) Sensitivity estimates for nonlinear mathematical models. Math Model Comput Exp 1(4):407\u2013414","journal-title":"Math Model Comput Exp"},{"key":"10726_CR280","unstructured":"Springenberg JT, Klein A, Falkner S et\u00a0al (2016) Bayesian optimization with robust Bayesian neural networks. In: Proceedings of the advances in neural information processing systems 29: annual conference on neural information processing systems, Barcelona, Spain, 5\u201310 December 2016. pp 4134\u20134142"},{"key":"10726_CR281","unstructured":"Stanley KO, Miikkulainen R (2002a) Efficient reinforcement learning through evolving neural network topologies. In: Proceedings of the genetic and evolutionary computation conference, New York, NY, USA, 9\u201313 July 2002. p\u00a09"},{"issue":"2","key":"10726_CR282","doi-asserted-by":"publisher","first-page":"99","DOI":"10.1162\/106365602320169811","volume":"10","author":"KO Stanley","year":"2002","unstructured":"Stanley KO, Miikkulainen R (2002b) Evolving neural networks through augmenting topologies. Evol Comput 10(2):99\u2013127","journal-title":"Evol Comput"},{"issue":"2","key":"10726_CR283","doi-asserted-by":"publisher","first-page":"185","DOI":"10.1162\/artl.2009.15.2.15202","volume":"15","author":"KO Stanley","year":"2009","unstructured":"Stanley KO, D\u2019Ambrosio DB, Gauci J (2009) A hypercube-based encoding for evolving large-scale neural networks. Artif Life 15(2):185\u2013212","journal-title":"Artif Life"},{"key":"10726_CR284","doi-asserted-by":"crossref","unstructured":"Steinruecken C, Smith E, Janz D et\u00a0al (2019) The automatic statistician. In: Automated machine learning. Springer, pp 161\u2013173","DOI":"10.1007\/978-3-030-05318-5_9"},{"key":"10726_CR285","doi-asserted-by":"publisher","unstructured":"Suganuma M, Shirakawa S, Nagao T (2018) A genetic programming approach to designing convolutional neural network architectures. In: Proceedings of the twenty-seventh international joint conference on artificial intelligence, Stockholm, Sweden, 13\u201319 July 2018. pp 5369\u20135373. https:\/\/doi.org\/10.24963\/ijcai.2018\/755","DOI":"10.24963\/ijcai.2018\/755"},{"issue":"9","key":"10726_CR286","doi-asserted-by":"publisher","first-page":"3840","DOI":"10.1109\/TCYB.2020.2983860","volume":"50","author":"Y Sun","year":"2020","unstructured":"Sun Y, Xue B, Zhang M et al (2020) Automatically designing CNN architectures using the genetic algorithm for image classification. IEEE Trans Cybern 50(9):3840\u20133854","journal-title":"IEEE Trans Cybern"},{"key":"10726_CR287","volume-title":"Reinforcement learning: an introduction","author":"RS Sutton","year":"2018","unstructured":"Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press, Cambridge"},{"key":"10726_CR288","doi-asserted-by":"publisher","unstructured":"Swearingen T, Drevo W, Cyphers B et\u00a0al (2017) ATM: a distributed, collaborative, scalable system for automated machine learning. In: Proceedings of the 2017 IEEE international conference on Big Data, Boston, MA, USA, 11\u201314 December 2017. pp 151\u2013162. https:\/\/doi.org\/10.1109\/BigData.2017.8257923","DOI":"10.1109\/BigData.2017.8257923"},{"key":"10726_CR289","unstructured":"Swersky K, Snoek J, Adams RP (2013) Multi-task Bayesian optimization. In: Proceedings of the advances in neural information processing systems 26: 27th annual conference on neural information processing systems. Lake Tahoe, NV, United States, 5\u20138 December 2013. pp 2004\u20132012"},{"key":"10726_CR290","doi-asserted-by":"publisher","DOI":"10.48550\/arxiv.1409.4011","author":"K Swersky","year":"2014","unstructured":"Swersky K, Duvenaud D, Snoek J et al (2014a) Raiders of the lost architecture: kernels for Bayesian optimization in conditional parameter spaces. arXiv. https:\/\/doi.org\/10.48550\/arxiv.1409.4011","journal-title":"arXiv"},{"key":"10726_CR291","unstructured":"Swersky K, Snoek J, Adams RP (2014b) Freeze-thaw Bayesian optimization. CoRR. http:\/\/arxiv.org\/abs\/1406.3896"},{"key":"10726_CR292","unstructured":"Talos A (2019) Autonomio Talos. Talos"},{"key":"10726_CR293","doi-asserted-by":"crossref","unstructured":"Tan M, Chen B, Pang R et\u00a0al (2019) MnasNet: platform-aware neural architecture search for mobile. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, 16\u201320 June 2019","DOI":"10.1109\/CVPR.2019.00293"},{"issue":"2","key":"10726_CR294","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1140\/epjc\/s10052-021-08950-y","volume":"81","author":"L Tani","year":"2021","unstructured":"Tani L, Rand D, Veelken C et al (2021) Evolutionary algorithms for hyperparameter optimization in machine learning for application in high energy physics. Eur Phys J C 81(2):1\u20139","journal-title":"Eur Phys J C"},{"key":"10726_CR295","doi-asserted-by":"publisher","unstructured":"Templier P, Rachelson E, Wilson DG (2021) A geometric encoding for neural network evolution. In: Proceedings of the genetic and evolutionary computation conference. pp 919\u2013927. https:\/\/doi.org\/10.1145\/3449639.3459361","DOI":"10.1145\/3449639.3459361"},{"key":"10726_CR296","doi-asserted-by":"publisher","first-page":"94718","DOI":"10.1109\/ACCESS.2022.3202220","volume":"10","author":"J Tetteroo","year":"2022","unstructured":"Tetteroo J, Baratchi M, Hoos HH (2022) Automated machine learning for covid-19 forecasting. IEEE Access 10:94718\u201394737. https:\/\/doi.org\/10.1109\/ACCESS.2022.3202220","journal-title":"IEEE Access"},{"key":"10726_CR297","unstructured":"Thakur A, Krohn-Grimberghe A (2015) AutoCompete: a framework for machine learning competition. In: Proceedings of the 2th international conference on machine learning workshop on automated machine learning, Lille, France, 6\u201311 July 2015"},{"issue":"3\/4","key":"10726_CR298","doi-asserted-by":"publisher","first-page":"285","DOI":"10.2307\/2332286","volume":"25","author":"WR Thompson","year":"1933","unstructured":"Thompson WR (1933) On the likelihood that one unknown probability exceeds another in view of the evidence of two samples. Biometrika 25(3\/4):285\u2013294","journal-title":"Biometrika"},{"key":"10726_CR299","doi-asserted-by":"publisher","unstructured":"Thornton C, Hutter F, Hoos HH et\u00a0al (2013) Auto-WEKA: combined selection and hyperparameter optimization of classification algorithms. In: Proceedings of the 19th international conference on knowledge discovery and data mining, Chicago, IL, USA, 11\u201314 August 2013. pp 847\u2013855. https:\/\/doi.org\/10.1145\/2487575.2487629","DOI":"10.1145\/2487575.2487629"},{"key":"10726_CR300","doi-asserted-by":"publisher","unstructured":"Tornede A, Deng D, Eimer T et\u00a0al (2023a) AutoML in the age of large language models: current challenges, future opportunities and risks. CoRR abs\/2306.08107. https:\/\/doi.org\/10.48550\/ARXIV.2306.08107. http:\/\/arxiv.org\/abs\/2306.08107","DOI":"10.48550\/ARXIV.2306.08107"},{"key":"10726_CR301","doi-asserted-by":"publisher","first-page":"427","DOI":"10.1613\/jair.1.14340","volume":"77","author":"T Tornede","year":"2023","unstructured":"Tornede T, Tornede A, Hanselle J et al (2023b) Towards green automated machine learning: status quo and future directions. J Artif Intell Res 77:427\u2013457. https:\/\/doi.org\/10.1613\/jair.1.14340","journal-title":"J Artif Intell Res"},{"key":"10726_CR302","doi-asserted-by":"publisher","unstructured":"Truong A, Walters A, Goodsitt J et\u00a0al (2019) Towards automated machine learning: evaluation and comparison of AutoML approaches and tools. In: Proceedings of the 31st international conference on tools with artificial intelligence, Portland, OR, USA, 4\u20136 November 2019. https:\/\/doi.org\/10.1109\/ictai.2019.00209","DOI":"10.1109\/ictai.2019.00209"},{"key":"10726_CR303","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3322996","author":"E Tseng","year":"2019","unstructured":"Tseng E, Yu F, Yang Y et al (2019) Hyperparameter optimization in black-box image processing using differentiable proxies. ACM Trans Graph. https:\/\/doi.org\/10.1145\/3306346.3322996","journal-title":"ACM Trans Graph"},{"key":"10726_CR304","doi-asserted-by":"publisher","unstructured":"van Rijn JN, Hutter F (2018) Hyperparameter importance across datasets. In: Proceedings of the 24th international conference on knowledge discovery & data mining, London, UK, 19\u201323 August 2018. pp 2367\u20132376. https:\/\/doi.org\/10.1145\/3219819.3220058","DOI":"10.1145\/3219819.3220058"},{"key":"10726_CR305","doi-asserted-by":"publisher","unstructured":"van Rijn JN, Abdulrahman SM, Brazdil P et\u00a0al (2015) Fast algorithm selection using learning curves. In: Advances in intelligent data analysis XIV\u201414th international symposium, Saint Etienne, France, 22\u201324 October 2015, proceedings. pp 298\u2013309. https:\/\/doi.org\/10.1007\/978-3-319-24465-5_26","DOI":"10.1007\/978-3-319-24465-5_26"},{"key":"10726_CR306","unstructured":"Vanschoren J (2018) Chapter\u00a02\u2014meta-learning. In: Automated machine learning. Springer, pp 39\u201368"},{"issue":"2","key":"10726_CR307","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1145\/2641190.2641198","volume":"15","author":"J Vanschoren","year":"2013","unstructured":"Vanschoren J, van Rijn JN, Bischl B et al (2013) OpenML: networked science in machine learning. SIGKDD Explor 15(2):49\u201360. https:\/\/doi.org\/10.1145\/2641190.2641198","journal-title":"SIGKDD Explor"},{"key":"10726_CR308","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1186\/1471-2105-7-91","volume":"7","author":"S Varma","year":"2006","unstructured":"Varma S, Simon R (2006) Bias in error estimation when using cross-validation for model selection. BMC Bioinform 7:91. https:\/\/doi.org\/10.1186\/1471-2105-7-91","journal-title":"BMC Bioinform"},{"key":"10726_CR309","unstructured":"Wan X, Ru B, Esperan\u00e7a PM et\u00a0al (2022) On redundancy and diversity in cell-based neural architecture search. In: The tenth international conference on learning representations, virtual event, 25\u201329 April 2022"},{"key":"10726_CR310","doi-asserted-by":"crossref","unstructured":"Wang L, Zhao Y, Jinnai Y et\u00a0al (2020) Neural architecture search using deep neural networks and Monte Carlo tree search. In: The thirty-fourth AAAI conference on artificial intelligence, New York, NY, USA, 7\u201312 February 2020. pp 9983\u20139991","DOI":"10.1609\/aaai.v34i06.6554"},{"issue":"106","key":"10726_CR311","first-page":"602","volume":"223","author":"C Wang","year":"2021","unstructured":"Wang C, Wang H, Zhou C et al (2021a) Experiencethinking: constrained hyperparameter optimization based on knowledge and pruning. Knowl Based Syst 223(106):602","journal-title":"Knowl Based Syst"},{"key":"10726_CR312","unstructured":"Wang C, Wu Q, Weimer M et\u00a0al (2021b) FLAML: a fast and lightweight AutoML library. In: Proceedings of machine learning and systems 2021, virtual event, 5\u20139 April 2021"},{"key":"10726_CR313","unstructured":"Wang S, Zhang H, Xu K et\u00a0al (2021c) Beta-CROWN: efficient bound propagation with per-neuron split constraints for neural network robustness verification. In: Advances in neural information processing systems 34: annual conference on neural information processing systems 2021, virtual event, 6\u201314 December 2021. pp 29909\u201329921"},{"key":"10726_CR314","unstructured":"Wang C, Baratchi M, B\u00e4ck T et\u00a0al (2022a) Towards automated machine learning for time-series forecasting. Under review"},{"key":"10726_CR315","doi-asserted-by":"publisher","DOI":"10.3390\/engproc2022018017","author":"C Wang","year":"2022","unstructured":"Wang C, Baratchi M, B\u00e4ck T et al (2022b) Towards time-series feature engineering in automated machine learning for multi-step-ahead forecasting. Eng Proc. https:\/\/doi.org\/10.3390\/engproc2022018017","journal-title":"Eng Proc"},{"issue":"101","key":"10726_CR316","first-page":"822","volume":"104","author":"J Waring","year":"2020","unstructured":"Waring J, Lindvall C, Umeton R (2020) Automated machine learning: review of the state-of-the-art and opportunities for healthcare. Artif Intell Med 104(101):822","journal-title":"Artif Intell Med"},{"key":"10726_CR317","doi-asserted-by":"publisher","unstructured":"Watanabe S, Roux JL (2014) Black box optimization for automatic speech recognition. In: Proceedings of the IEEE international conference on acoustics, speech and signal processing, Florence, Italy, 4\u20139 May 2014. pp 3256\u20133260. https:\/\/doi.org\/10.1109\/ICASSP.2014.6854202","DOI":"10.1109\/ICASSP.2014.6854202"},{"key":"10726_CR318","unstructured":"Watkins CJCH (1989) Learning from delayed rewards. PhD Thesis, University of Cambridge"},{"key":"10726_CR319","unstructured":"Weerts H, Pfisterer F, Feurer M et\u00a0al (2023) Can fairness be automated? Guidelines and opportunities for fairness-aware AutoML. http:\/\/arxiv.org\/abs\/2303.08485"},{"key":"10726_CR320","unstructured":"Wei T, Wang C, Rui Y et\u00a0al (2016) Network morphism. In: Proceedings of the international conference on machine learning. pp 564\u2013572"},{"key":"10726_CR321","unstructured":"Wever M, Mohr F, H\u00fcllermeier E (2018) Automated multi-label classification based on ML-Plan. arXiv. http:\/\/arxiv.org\/abs\/1811.04060"},{"key":"10726_CR322","doi-asserted-by":"crossref","unstructured":"White C, Neiswanger W, Savani Y (2021a) BANANAS: Bayesian optimization with neural architectures for neural architecture search. In: Proceedings of the thirty-fifth AAAI conference on artificial intelligence, virtual event, 2\u20139 February 2021. pp 10293\u201310301","DOI":"10.1609\/aaai.v35i12.17233"},{"key":"10726_CR323","unstructured":"White C, Zela A, Ru R et\u00a0al (2021b) How powerful are performance predictors in neural architecture search? In: Advances in neural information processing systems 34: annual conference on neural information processing systems 2021, virtual event, 6\u201314 December 2021. pp 28454\u201328469"},{"issue":"3","key":"10726_CR324","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1007\/BF00992696","volume":"8","author":"RJ Williams","year":"1992","unstructured":"Williams RJ (1992) Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach Learn 8(3):229\u2013256","journal-title":"Mach Learn"},{"key":"10726_CR325","unstructured":"Wistuba M (2017) Finding competitive network architectures within a day using UCT. arXiv. http:\/\/arxiv.org\/abs\/1712.07420"},{"key":"10726_CR326","doi-asserted-by":"publisher","unstructured":"Wistuba M (2018) Deep learning architecture search by neuro-cell-based evolution with function-preserving mutations. In: Proceedings of the machine learning and knowledge discovery in databases\u2014European conference, Dublin, Ireland, 10\u201314 September 2018. pp 243\u2013258. https:\/\/doi.org\/10.1007\/978-3-030-10928-8_15","DOI":"10.1007\/978-3-030-10928-8_15"},{"key":"10726_CR327","doi-asserted-by":"crossref","unstructured":"Wistuba M, Schilling N, Schmidt-Thieme L (2015a) Hyperparameter search space pruning\u2014a new component for sequential model-based hyperparameter optimization. In: Proceedings of the European conference on machine learning and knowledge discovery in databases, Porto, Portugal, 7\u201311 September 2015. pp 104\u2013119","DOI":"10.1007\/978-3-319-23525-7_7"},{"key":"10726_CR328","doi-asserted-by":"crossref","unstructured":"Wistuba M, Schilling N, Schmidt-Thieme L (2015b) Sequential model-free hyperparameter tuning. In: Proceedings of the 2015 IEEE international conference on data mining, Atlantic City, NJ, USA, 14\u201317 November 2015. pp 1033\u20131038","DOI":"10.1109\/ICDM.2015.20"},{"key":"10726_CR329","unstructured":"Wistuba M, Rawat A, Pedapati T (2019) A survey on neural architecture search. arXiv. http:\/\/arxiv.org\/abs\/1905.01392"},{"key":"10726_CR330","unstructured":"Wong C, Houlsby N, Lu Y et\u00a0al (2018) Transfer learning with neural AutoML. In: Proceedings of the 32nd international conference on neural information processing systems, Montr\u00e9al, Canada, 3\u20138 December 2018. pp 8366\u20138375"},{"key":"10726_CR331","doi-asserted-by":"crossref","unstructured":"Wu B, Dai X, Zhang P et\u00a0al (2019) FBNet: hardware-aware efficient convnet design via differentiable neural architecture search. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Long Beach, CA, USA, 16\u201320 June 2019. pp 10734\u201310742","DOI":"10.1109\/CVPR.2019.01099"},{"key":"10726_CR332","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1016\/j.neucom.2020.06.064","volume":"409","author":"J Wu","year":"2020","unstructured":"Wu J, Chen S, Liu X (2020) Efficient hyperparameter optimization through model-based reinforcement learning. Neurocomputing 409:381\u2013393. https:\/\/doi.org\/10.1016\/j.neucom.2020.06.064","journal-title":"Neurocomputing"},{"key":"10726_CR333","doi-asserted-by":"crossref","unstructured":"Wu Q, Wang C, Huang S (2021) Frugal optimization for cost-related hyperparameters. In: Proceedings of the AAAI conference on artificial intelligence, virtual event, 2\u20139 February 2021. pp 10347\u201310354","DOI":"10.1609\/aaai.v35i12.17239"},{"key":"10726_CR334","doi-asserted-by":"crossref","unstructured":"Xie L, Yuille A (2017) Genetic CNN. In: Proceedings of the IEEE international conference on computer vision, Venice, Italy, 22\u201329 October 2017. pp 1379\u20131388","DOI":"10.1109\/ICCV.2017.154"},{"key":"10726_CR335","unstructured":"Xie S, Zheng H, Liu C et\u00a0al (2019) SNAS: stochastic neural architecture search. In: Proceedings of the 7th international conference on learning representations, New Orleans, LA, USA, 6\u20139 May 2019"},{"issue":"9","key":"10726_CR336","doi-asserted-by":"publisher","first-page":"183:1","DOI":"10.1145\/3473330","volume":"54","author":"L Xie","year":"2022","unstructured":"Xie L, Chen X, Bi K et al (2022a) Weight-sharing neural architecture search: a battle to shrink the optimization gap. ACM Comput Surv 54(9):183:1-183:37. https:\/\/doi.org\/10.1145\/3473330","journal-title":"ACM Comput Surv"},{"issue":"6","key":"10726_CR337","doi-asserted-by":"publisher","first-page":"1473","DOI":"10.1109\/TEVC.2022.3147526","volume":"26","author":"X Xie","year":"2022","unstructured":"Xie X, Liu Y, Sun Y et al (2022b) BenchENAS: a benchmarking platform for evolutionary neural architecture search. IEEE Trans Evol Comput 26(6):1473\u20131485. https:\/\/doi.org\/10.1109\/TEVC.2022.3147526","journal-title":"IEEE Trans Evol Comput"},{"key":"10726_CR338","unstructured":"Xu Y, Xie L, Zhang X et\u00a0al (2020) PC-DARTS: partial channel connections for memory-efficient architecture search. In: Proceedings of the 8th international conference on learning representations, Addis Ababa, Ethiopia, 26\u201330 April 2020"},{"key":"10726_CR339","doi-asserted-by":"publisher","unstructured":"Xue S, Wang R, Zhang B et\u00a0al (2021) IDARTS: interactive differentiable architecture search. In: Proceedings of the 2021 IEEE international conference on computer vision, Montreal, QC, Canada, 10\u201317 October 2021. pp 1143\u20131152. https:\/\/doi.org\/10.1109\/ICCV48922.2021.00120","DOI":"10.1109\/ICCV48922.2021.00120"},{"issue":"12","key":"10726_CR340","doi-asserted-by":"publisher","first-page":"3166","DOI":"10.14778\/3415478.3415542","volume":"13","author":"A Yakovlev","year":"2020","unstructured":"Yakovlev A, Moghadam HF, Moharrer A et al (2020) Oracle AutoML: a fast and predictive AutoML pipeline. Proc VLDB Endow 13(12):3166\u20133180","journal-title":"Proc VLDB Endow"},{"key":"10726_CR341","doi-asserted-by":"publisher","unstructured":"Yang C, Akimoto Y, Kim DW et\u00a0al (2019) OBOE: collaborative filtering for AutoML model selection. In: Proceedings of the 25th international conference on knowledge discovery & data mining, Anchorage, AK, USA, 4\u20138 August 2019. pp 1173\u2014-1183. https:\/\/doi.org\/10.1145\/3292500.3330909","DOI":"10.1145\/3292500.3330909"},{"key":"10726_CR342","doi-asserted-by":"publisher","unstructured":"Yang C, Fan J, Wu Z et\u00a0al (2020) AutoML pipeline selection: efficiently navigating the combinatorial space. In: Proceedings of the 26th international conference on knowledge discovery & data mining. pp 1446\u20131456. https:\/\/doi.org\/10.1145\/3394486.3403197","DOI":"10.1145\/3394486.3403197"},{"key":"10726_CR343","doi-asserted-by":"crossref","unstructured":"Yang J, Liu Y, Xu H (2023) HOTNAS: hierarchical optimal transport for neural architecture search. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, Vancouver, BC, Canada, 17\u201324 June 2023. pp 11990\u201312000","DOI":"10.1109\/CVPR52729.2023.01154"},{"issue":"9","key":"10726_CR344","doi-asserted-by":"publisher","first-page":"1423","DOI":"10.1109\/5.784219","volume":"87","author":"X Yao","year":"1999","unstructured":"Yao X (1999) Evolving artificial neural networks. Proc IEEE 87(9):1423\u20131447. https:\/\/doi.org\/10.1109\/5.784219","journal-title":"Proc IEEE"},{"key":"10726_CR345","unstructured":"Yao Q, Wang M, Escalante HJ et\u00a0al (2018) Taking human out of learning applications: a survey on automated machine learning. arXiv. http:\/\/arxiv.org\/abs\/1810.13306"},{"key":"10726_CR346","doi-asserted-by":"crossref","unstructured":"Ye P, Li B, Li Y et\u00a0al (2022) b-DARTS: beta-decay regularization for differentiable architecture search. In: Proceedings of the IEEE conference on computer vision and pattern recognition, New Orleans, LA, USA, 19\u201320 June 2022. pp 10874\u201310883","DOI":"10.1109\/CVPR52688.2022.01060"},{"key":"10726_CR347","unstructured":"Yelp (2014) Metric optimization engine. https:\/\/github.com\/Yelp\/MOE"},{"key":"10726_CR348","unstructured":"Ying C, Klein A, Christiansen E et\u00a0al (2019) Nas-bench-101: towards reproducible neural architecture search. In: Proceedings of the international conference on machine learning, Long Beach, CA, USA, 9\u201315 June 2019. pp 7105\u20137114"},{"key":"10726_CR349","unstructured":"Yu K, Sciuto C, Jaggi M et\u00a0al (2020) Evaluating the search phase of neural architecture search. In: Proceedings of the 8th international conference on learning representations, Addis Ababa, Ethiopia, 26\u201330 April 2020"},{"key":"10726_CR350","doi-asserted-by":"crossref","unstructured":"Zagoruyko S, Komodakis N (2016) Wide residual networks. In: Proceedings of the British machine vision conference 2016, York, UK, 19\u201322 September 2016","DOI":"10.5244\/C.30.87"},{"key":"10726_CR351","unstructured":"Zela A, Klein A, Falkner S et\u00a0al (2018) Towards automated deep learning: efficient joint neural architecture and hyperparameter search. arXiv. http:\/\/arxiv.org\/abs\/1807.06906"},{"key":"10726_CR352","unstructured":"Zela A, Siems J, Hutter F (2020) NAS-Bench-1Shot1: benchmarking and dissecting one-shot neural architecture search. In: Proceedings of the 8th international conference on learning representations, Addis Ababa, Ethiopia, 26\u201330 April 2020"},{"key":"10726_CR353","unstructured":"Zela A, Siems JN, Zimmer L et\u00a0al (2022) Surrogate NAS benchmarks: going beyond the limited search spaces of tabular NAS benchmarks. In: The tenth international conference on learning representations, virtual event, 25\u201329 April 2022"},{"key":"10726_CR354","doi-asserted-by":"publisher","unstructured":"Zhang X, Zhou X, Lin M et\u00a0al (2018) ShuffleNet: an extremely efficient convolutional neural network for mobile devices. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 18\u201322 June 2018. pp 6848\u20136856. https:\/\/doi.org\/10.1109\/CVPR.2018.00716","DOI":"10.1109\/CVPR.2018.00716"},{"key":"10726_CR355","unstructured":"Zhang C, Ren M, Urtasun R (2019a) Graph hypernetworks for neural architecture search. In: Proceedings of the 7th international conference on learning representations, New Orleans, LA, USA, 6\u20139 May 2019"},{"key":"10726_CR356","unstructured":"Zhang H, Yu Y, Jiao J et\u00a0al (2019b) Theoretically principled trade-off between robustness and accuracy. In: Chaudhuri K, Salakhutdinov R (eds) Proceedings of the 36th international conference on machine learning, ICML 2019, 9\u201315 June 2019, Long Beach, CA, USA. pp 7472\u20137482"},{"key":"10726_CR357","doi-asserted-by":"crossref","unstructured":"Zhang M, Li H, Pan S et\u00a0al (2020) Overcoming multi-model forgetting in one-shot NAS with diversity maximization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Seattle, WA, USA, 13\u201319 June 2020. pp 7809\u20137818","DOI":"10.1109\/CVPR42600.2020.00783"},{"issue":"5","key":"10726_CR358","doi-asserted-by":"publisher","first-page":"1072","DOI":"10.1109\/TEVC.2022.3140855","volume":"26","author":"H Zhang","year":"2022","unstructured":"Zhang H, Jin Y, Hao K (2022a) Evolutionary search for complete neural network architectures with partial weight sharing. IEEE Trans Evol Comput 26(5):1072\u20131086. https:\/\/doi.org\/10.1109\/TEVC.2022.3140855","journal-title":"IEEE Trans Evol Comput"},{"key":"10726_CR359","unstructured":"Zhang W, Lin Z, Shen Y et\u00a0al (2022b) Deep and flexible graph neural architecture search. In: Proceedings of the 39th international conference on machine learning, Baltimore, MD, USA, 17\u201323 July 2022. pp 26362\u201326374"},{"key":"10726_CR360","doi-asserted-by":"publisher","unstructured":"Zhong Z, Yan J, Wu W et\u00a0al (2018) Practical block-wise neural network architecture generation. In: Proceedings of the 2018 IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 18\u201322 June 2018. pp 2423\u20132432. https:\/\/doi.org\/10.1109\/CVPR.2018.00257","DOI":"10.1109\/CVPR.2018.00257"},{"key":"10726_CR361","doi-asserted-by":"crossref","unstructured":"Zhou Q, Zheng X, Cao L et\u00a0al (2021) EC-DARTS: inducing equalized and consistent optimization into DARTS. In: Proceedings of the IEEE international conference on computer vision, Montr\u00e9al, QC, Canada, 10\u201317 October 2021. pp 11986\u201311995","DOI":"10.1109\/ICCV48922.2021.01177"},{"issue":"9","key":"10726_CR362","doi-asserted-by":"publisher","first-page":"3079","DOI":"10.1109\/TPAMI.2021.3067763","volume":"43","author":"L Zimmer","year":"2021","unstructured":"Zimmer L, Lindauer M, Hutter F (2021) Auto-PyTorch tabular: multi-fidelity metalearning for efficient and robust AutoDL. IEEE Trans Pattern Anal Mach Intell 43(9):3079\u20133090","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"10726_CR363","doi-asserted-by":"publisher","first-page":"409","DOI":"10.1613\/jair.1.11854","volume":"70","author":"MA Z\u00f6ller","year":"2021","unstructured":"Z\u00f6ller MA, Huber MF (2021) Benchmark and survey of automated machine learning frameworks. J Artif Intell Res 70:409\u2013472","journal-title":"J Artif Intell Res"},{"key":"10726_CR364","unstructured":"Zoph B, Le QV (2017) Neural architecture search with reinforcement learning. In: Proceedings of the 5th international conference on learning representations, Toulon, France, 24\u201326 April 2017"},{"key":"10726_CR365","doi-asserted-by":"publisher","unstructured":"Zoph B, Vasudevan V, Shlens J et\u00a0al (2018) Learning transferable architectures for scalable image recognition. In: Proceedings of the 2018 IEEE conference on computer vision and pattern recognition, Salt Lake City, UT, USA, 18\u201322 June 2018. pp 8697\u20138710. https:\/\/doi.org\/10.1109\/CVPR.2018.00907","DOI":"10.1109\/CVPR.2018.00907"}],"container-title":["Artificial Intelligence Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-024-10726-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10462-024-10726-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10462-024-10726-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,17]],"date-time":"2024-05-17T02:16:28Z","timestamp":1715912188000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10462-024-10726-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,18]]},"references-count":365,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2024,5]]}},"alternative-id":["10726"],"URL":"https:\/\/doi.org\/10.1007\/s10462-024-10726-1","relation":{},"ISSN":["1573-7462"],"issn-type":[{"value":"1573-7462","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,18]]},"assertion":[{"value":"10 February 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"18 April 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no conflict of interest.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}],"article-number":"122"}}