{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,31]],"date-time":"2026-03-31T23:35:45Z","timestamp":1775000145032,"version":"3.50.1"},"reference-count":78,"publisher":"Association for Computing Machinery (ACM)","issue":"6","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2022,2]]},"abstract":"<jats:p>\n            The ever-growing demand and complexity of machine learning are putting pressure on hyper-parameter tuning systems:\n            <jats:italic>while the evaluation cost of models continues to increase, the scalability of state-of-the-arts starts to become a crucial bottleneck.<\/jats:italic>\n            In this paper, inspired by our experience when deploying hyper-parameter tuning in a real-world application in production and the limitations of existing systems, we propose Hyper-Tune, an efficient and robust distributed hyper-parameter tuning framework. Compared with existing systems, Hyper-Tune highlights multiple system optimizations, including (1) automatic resource allocation, (2) asynchronous scheduling, and (3) multi-fidelity optimizer. We conduct extensive evaluations on benchmark datasets and a large-scale real-world dataset in production. Empirically, with the aid of these optimizations, Hyper-Tune outperforms competitive hyper-parameter tuning systems on a wide range of scenarios, including XGBoost, CNN, RNN, and some architectural hyper-parameters for neural networks. Compared with the state-of-the-art BOHB and A-BOHB, Hyper-Tune achieves up to 11.2X and 5.1X speedups, respectively.\n          <\/jats:p>","DOI":"10.14778\/3514061.3514071","type":"journal-article","created":{"date-parts":[[2022,6,22]],"date-time":"2022-06-22T22:26:10Z","timestamp":1655936770000},"page":"1256-1265","source":"Crossref","is-referenced-by-count":21,"title":["Hyper-tune"],"prefix":"10.14778","volume":"15","author":[{"given":"Yang","family":"Li","sequence":"first","affiliation":[{"name":"Peking University and Kuaishou Technology"}]},{"given":"Yu","family":"Shen","sequence":"additional","affiliation":[{"name":"Peking University and Kuaishou Technology"}]},{"given":"Huaijun","family":"Jiang","sequence":"additional","affiliation":[{"name":"Peking University and Kuaishou Technology"}]},{"given":"Wentao","family":"Zhang","sequence":"additional","affiliation":[{"name":"Peking University"}]},{"given":"Jixiang","family":"Li","sequence":"additional","affiliation":[{"name":"Kuaishou Technology"}]},{"given":"Ji","family":"Liu","sequence":"additional","affiliation":[{"name":"Kuaishou Technology"}]},{"given":"Ce","family":"Zhang","sequence":"additional","affiliation":[{"name":"ETH Z\u00fcrich"}]},{"given":"Bin","family":"Cui","sequence":"additional","affiliation":[{"name":"Peking University"}]}],"member":"320","published-online":{"date-parts":[[2022,6,22]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Asynchronous Batch Bayesian Optimisation with Improved Local Penalisation. In International Conference on Machine Learning. PMLR, 253--262","author":"Alvi Ahsan","year":"2019","unstructured":"Ahsan Alvi , Binxin Ru , Jan-Peter Calliess , Stephen Roberts , and Michael A Osborne . 2019 . Asynchronous Batch Bayesian Optimisation with Improved Local Penalisation. In International Conference on Machine Learning. PMLR, 253--262 . Ahsan Alvi, Binxin Ru, Jan-Peter Calliess, Stephen Roberts, and Michael A Osborne. 2019. Asynchronous Batch Bayesian Optimisation with Improved Local Penalisation. In International Conference on Machine Learning. PMLR, 253--262."},{"key":"e_1_2_1_2_1","volume-title":"Differential Evolution for Neural Architecture Search. arXiv preprint arXiv:2012.06400","author":"Awad Noor","year":"2020","unstructured":"Noor Awad , Neeratyoy Mallik , and Frank Hutter . 2020. Differential Evolution for Neural Architecture Search. arXiv preprint arXiv:2012.06400 ( 2020 ). Noor Awad, Neeratyoy Mallik, and Frank Hutter. 2020. Differential Evolution for Neural Architecture Search. arXiv preprint arXiv:2012.06400 (2020)."},{"key":"e_1_2_1_3_1","unstructured":"Javad Azimi Alan Fern and Xiaoli Z Fern. 2010. Batch bayesian optimization via simulation matching. In Advances in Neural Information Processing Systems. 109--117.  Javad Azimi Alan Fern and Xiaoli Z Fern. 2010. Batch bayesian optimization via simulation matching. In Advances in Neural Information Processing Systems. 109--117."},{"key":"e_1_2_1_4_1","volume-title":"Practical neural network performance prediction for early stopping. arXiv preprint arXiv:1705.10823 2, 3","author":"Baker Bowen","year":"2017","unstructured":"Bowen Baker , Otkrist Gupta , Ramesh Raskar , and Nikhil Naik . 2017. Practical neural network performance prediction for early stopping. arXiv preprint arXiv:1705.10823 2, 3 ( 2017 ), 6. Bowen Baker, Otkrist Gupta, Ramesh Raskar, and Nikhil Naik. 2017. Practical neural network performance prediction for early stopping. arXiv preprint arXiv:1705.10823 2, 3 (2017), 6."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098021"},{"key":"e_1_2_1_6_1","first-page":"281","article-title":"Random search for hyper-parameter optimization","author":"Bergstra James","year":"2012","unstructured":"James Bergstra and Yoshua Bengio . 2012 . Random search for hyper-parameter optimization . Journal of Machine Learning Research 13 , Feb (2012), 281 -- 305 . James Bergstra and Yoshua Bengio. 2012. Random search for hyper-parameter optimization. Journal of Machine Learning Research 13, Feb (2012), 281--305.","journal-title":"Journal of Machine Learning Research 13"},{"key":"e_1_2_1_7_1","unstructured":"James S Bergstra R\u00e9mi Bardenet Yoshua Bengio and Bal\u00e1zs K\u00e9gl. 2011. Algorithms for hyper-parameter optimization. In Advances in neural information processing systems. 2546--2554.  James S Bergstra R\u00e9mi Bardenet Yoshua Bengio and Bal\u00e1zs K\u00e9gl. 2011. Algorithms for hyper-parameter optimization. In Advances in neural information processing systems. 2546--2554."},{"key":"e_1_2_1_8_1","volume-title":"SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle. In Conference on Innovative Data Systems Research.","author":"Boehm Matthias","year":"2020","unstructured":"Matthias Boehm , Iulian Antonov , Sebastian Baunsgaard , Mark Dokter , Robert Erich Ginthoer , Kevin Innerebner , Florijan Klezin , Stefanie Lindstaedt , Arnab Phani , Benjamin Rath , 2020 . SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle. In Conference on Innovative Data Systems Research. Matthias Boehm, Iulian Antonov, Sebastian Baunsgaard, Mark Dokter, Robert Erich Ginthoer, Kevin Innerebner, Florijan Klezin, Stefanie Lindstaedt, Arnab Phani, Benjamin Rath, et al. 2020. SystemDS: A Declarative Machine Learning System for the End-to-End Data Science Lifecycle. In Conference on Innovative Data Systems Research."},{"key":"e_1_2_1_9_1","volume-title":"Data Validation for Machine Learning. In 3rd Conference on Machine Learning and Systems (MLSys).","author":"Breck Eric","year":"2019","unstructured":"Eric Breck , Neoklis Polyzotis , Sudip Roy , Steven Euijong Whang , and Martin Zinkevich . 2019 . Data Validation for Machine Learning. In 3rd Conference on Machine Learning and Systems (MLSys). Eric Breck, Neoklis Polyzotis, Sudip Roy, Steven Euijong Whang, and Martin Zinkevich. 2019. Data Validation for Machine Learning. In 3rd Conference on Machine Learning and Systems (MLSys)."},{"key":"e_1_2_1_10_1","unstructured":"Tom B Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell etal 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020).  Tom B Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020)."},{"key":"e_1_2_1_11_1","unstructured":"Wenming CAO Canta ZHENG Zhiyue YAN and Weixin XIE. 2022. Geometric deep learning: progress applications and challenges. Information Sciences 65 126101 (2022) 1--126101.  Wenming CAO Canta ZHENG Zhiyue YAN and Weixin XIE. 2022. Geometric deep learning: progress applications and challenges. Information Sciences 65 126101 (2022) 1--126101."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_2_1_13_1","volume-title":"Bryan Kian Hsiang Low, and Patrick Jaillet","author":"Dai Zhongxiang","year":"2019","unstructured":"Zhongxiang Dai , Haibin Yu , Bryan Kian Hsiang Low, and Patrick Jaillet . 2019 . Bayesian Optimization Meets Bayesian Optimal Stopping . (2019), 1496--1506. Zhongxiang Dai, Haibin Yu, Bryan Kian Hsiang Low, and Patrick Jaillet. 2019. Bayesian Optimization Meets Bayesian Optimal Stopping. (2019), 1496--1506."},{"key":"e_1_2_1_14_1","first-page":"3460","article-title":"Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves","volume":"15","author":"Domhan Tobias","year":"2015","unstructured":"Tobias Domhan , Jost Tobias Springenberg , and Frank Hutter . 2015 . Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves .. In IJCAI , Vol. 15. 3460 -- 3468 . Tobias Domhan, Jost Tobias Springenberg, and Frank Hutter. 2015. Speeding Up Automatic Hyperparameter Optimization of Deep Neural Networks by Extrapolation of Learning Curves.. In IJCAI, Vol. 15. 3460--8.","journal-title":"IJCAI"},{"key":"e_1_2_1_15_1","volume-title":"International Conference on Learning Representations.","author":"Dong Xuanyi","year":"2019","unstructured":"Xuanyi Dong and Yi Yang . 2019 . NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search . In International Conference on Learning Representations. Xuanyi Dong and Yi Yang. 2019. NAS-Bench-201: Extending the Scope of Reproducible Neural Architecture Search. In International Conference on Learning Representations."},{"key":"e_1_2_1_16_1","volume-title":"International Conference on Machine Learning. PMLR, 1437--1446","author":"Falkner Stefan","year":"2018","unstructured":"Stefan Falkner , Aaron Klein , and Frank Hutter . 2018 . BOHB: Robust and efficient hyperparameter optimization at scale . In International Conference on Machine Learning. PMLR, 1437--1446 . Stefan Falkner, Aaron Klein, and Frank Hutter. 2018. BOHB: Robust and efficient hyperparameter optimization at scale. In International Conference on Machine Learning. PMLR, 1437--1446."},{"key":"e_1_2_1_17_1","unstructured":"Matthias Feurer Aaron Klein Katharina Eggensperger Jost Springenberg Manuel Blum and Frank Hutter. 2015. Efficient and robust automated machine learning. In Advances in neural information processing systems. 2962--2970.  Matthias Feurer Aaron Klein Katharina Eggensperger Jost Springenberg Manuel Blum and Frank Hutter. 2015. Efficient and robust automated machine learning. In Advances in neural information processing systems. 2962--2970."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2011.5767930"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098043"},{"key":"e_1_2_1_20_1","unstructured":"Javier Gonz\u00e1lez Zhenwen Dai Philipp Hennig and Neil Lawrence. 2016. Batch bayesian optimization via local penalization. In Artificial Intelligence and Statistics. 648--657.  Javier Gonz\u00e1lez Zhenwen Dai Philipp Hennig and Neil Lawrence. 2016. Batch bayesian optimization via local penalization. In Artificial Intelligence and Statistics. 648--657."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_22_1","volume-title":"Long short-term memory. Neural computation 9, 8","author":"Hochreiter Sepp","year":"1997","unstructured":"Sepp Hochreiter and J\u00fcrgen Schmidhuber . 1997. Long short-term memory. Neural computation 9, 8 ( 1997 ), 1735--1780. Sepp Hochreiter and J\u00fcrgen Schmidhuber. 1997. Long short-term memory. Neural computation 9, 8 (1997), 1735--1780."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33013846"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-25566-3_40"},{"key":"e_1_2_1_25_1","doi-asserted-by":"crossref","unstructured":"Frank Hutter Lars Kotthoff and Joaquin Vanschoren (Eds.). 2018. Automated Machine Learning: Methods Systems Challenges. Springer. In press available at http:\/\/automl.org\/book.  Frank Hutter Lars Kotthoff and Joaquin Vanschoren (Eds.). 2018. Automated Machine Learning: Methods Systems Challenges. Springer. In press available at http:\/\/automl.org\/book.","DOI":"10.1007\/978-3-030-05318-5"},{"key":"e_1_2_1_26_1","volume-title":"Automated machine learning: methods, systems, challenges","author":"Hutter Frank","unstructured":"Frank Hutter , Lars Kotthoff , and Joaquin Vanschoren . 2019. Automated machine learning: methods, systems, challenges . Springer Nature . Frank Hutter, Lars Kotthoff, and Joaquin Vanschoren. 2019. Automated machine learning: methods, systems, challenges. Springer Nature."},{"key":"e_1_2_1_27_1","unstructured":"Max Jaderberg Valentin Dalibard Simon Osindero Wojciech M Czarnecki Jeff Donahue Ali Razavi Oriol Vinyals Tim Green Iain Dunning Karen Simonyan etal 2017. Population based training of neural networks. arXiv preprint arXiv:1711.09846 (2017).  Max Jaderberg Valentin Dalibard Simon Osindero Wojciech M Czarnecki Jeff Donahue Ali Razavi Oriol Vinyals Tim Green Iain Dunning Karen Simonyan et al. 2017. Population based training of neural networks. arXiv preprint arXiv:1711.09846 (2017)."},{"key":"e_1_2_1_28_1","unstructured":"Kevin Jamieson and Ameet Talwalkar. 2016. Non-stochastic best arm identification and hyperparameter optimization. In Artificial Intelligence and Statistics. 240--248.  Kevin Jamieson and Ameet Talwalkar. 2016. Non-stochastic best arm identification and hyperparameter optimization. In Artificial Intelligence and Statistics. 240--248."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1008306431147"},{"key":"e_1_2_1_30_1","volume-title":"Multi-fidelity bayesian optimisation with continuous approximations. arXiv preprint arXiv:1703.06240","author":"Kandasamy Kirthevasan","year":"2017","unstructured":"Kirthevasan Kandasamy , Gautam Dasarathy , Jeff Schneider , and Barnabas Poczos . 2017. Multi-fidelity bayesian optimisation with continuous approximations. arXiv preprint arXiv:1703.06240 ( 2017 ). Kirthevasan Kandasamy, Gautam Dasarathy, Jeff Schneider, and Barnabas Poczos. 2017. Multi-fidelity bayesian optimisation with continuous approximations. arXiv preprint arXiv:1703.06240 (2017)."},{"key":"e_1_2_1_31_1","volume-title":"Asynchronous Parallel Bayesian Optimisation via Thompson Sampling. stat 1050","author":"Kandasamy Kirthevasan","year":"2017","unstructured":"Kirthevasan Kandasamy , Akshay Krishnamurthy , Jeff Schneider , and Barnab\u00e1s P\u00f3czos . 2017. Asynchronous Parallel Bayesian Optimisation via Thompson Sampling. stat 1050 ( 2017 ), 25. Kirthevasan Kandasamy, Akshay Krishnamurthy, Jeff Schneider, and Barnab\u00e1s P\u00f3czos. 2017. Asynchronous Parallel Bayesian Optimisation via Thompson Sampling. stat 1050 (2017), 25."},{"key":"e_1_2_1_32_1","volume-title":"International Conference on Artificial Intelligence and Statistics. 133--142","author":"Kandasamy Kirthevasan","year":"2018","unstructured":"Kirthevasan Kandasamy , Akshay Krishnamurthy , Jeff Schneider , and Barnab\u00e1s P\u00f3czos . 2018 . Parallelised Bayesian Optimisation via Thompson Sampling . In International Conference on Artificial Intelligence and Statistics. 133--142 . Kirthevasan Kandasamy, Akshay Krishnamurthy, Jeff Schneider, and Barnab\u00e1s P\u00f3czos. 2018. Parallelised Bayesian Optimisation via Thompson Sampling. In International Conference on Artificial Intelligence and Statistics. 133--142."},{"key":"e_1_2_1_33_1","volume-title":"Neural Architecture Search with Bayesian Optimisation and Optimal Transport. Advances in Neural Information Processing Systems 31","author":"Kandasamy Kirthevasan","year":"2018","unstructured":"Kirthevasan Kandasamy , Willie Neiswanger , Jeff Schneider , Barnabas Poczos , and Eric P Xing . 2018. Neural Architecture Search with Bayesian Optimisation and Optimal Transport. Advances in Neural Information Processing Systems 31 ( 2018 ). Kirthevasan Kandasamy, Willie Neiswanger, Jeff Schneider, Barnabas Poczos, and Eric P Xing. 2018. Neural Architecture Search with Bayesian Optimisation and Optimal Transport. Advances in Neural Information Processing Systems 31 (2018)."},{"key":"e_1_2_1_34_1","unstructured":"Aaron Klein Stefan Falkner Simon Bartels Philipp Hennig and Frank Hutter. 2017. Fast bayesian optimization of machine learning hyperparameters on large datasets. In Artificial Intelligence and Statistics. PMLR 528--536.  Aaron Klein Stefan Falkner Simon Bartels Philipp Hennig and Frank Hutter. 2017. Fast bayesian optimization of machine learning hyperparameters on large datasets. In Artificial Intelligence and Statistics. PMLR 528--536."},{"key":"e_1_2_1_35_1","volume-title":"Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. 528--536","author":"Klein Aaron","year":"2017","unstructured":"Aaron Klein , Stefan Falkner , Simon Bartels , Philipp Hennig , and Frank Hutter . 2017 . Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets . In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. 528--536 . Aaron Klein, Stefan Falkner, Simon Bartels, Philipp Hennig, and Frank Hutter. 2017. Fast Bayesian Optimization of Machine Learning Hyperparameters on Large Datasets. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics. 528--536."},{"key":"e_1_2_1_36_1","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Klein Aaron","year":"2017","unstructured":"Aaron Klein , Stefan Falkner , Jost Tobias Springenberg , and Frank Hutter . 2017 . Learning curve prediction with Bayesian neural networks . Proceedings of the International Conference on Learning Representations (2017). Aaron Klein, Stefan Falkner, Jost Tobias Springenberg, and Frank Hutter. 2017. Learning curve prediction with Bayesian neural networks. Proceedings of the International Conference on Learning Representations (2017)."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.14778\/3229863.3240493"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352129"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Li Lisha","year":"2018","unstructured":"Lisha Li , Kevin Jamieson , Giulia DeSalvo , Afshin Rostamizadeh , and Ameet Talwalkar . 2018 . Hyperband: A novel bandit-based approach to hyperparameter optimization . Proceedings of the International Conference on Learning Representations (2018), 1--48. Lisha Li, Kevin Jamieson, Giulia DeSalvo, Afshin Rostamizadeh, and Ameet Talwalkar. 2018. Hyperband: A novel bandit-based approach to hyperparameter optimization. Proceedings of the International Conference on Learning Representations (2018), 1--48."},{"key":"e_1_2_1_40_1","first-page":"230","article-title":"A System for Massively Parallel Hyperparameter Tuning","volume":"2","author":"Li Liam","year":"2020","unstructured":"Liam Li , Kevin Jamieson , Afshin Rostamizadeh , Ekaterina Gonina , Jonathan Ben-tzur, Moritz Hardt , Benjamin Recht , and Ameet Talwalkar . 2020 . A System for Massively Parallel Hyperparameter Tuning . Proceedings of Machine Learning and Systems 2 (2020), 230 -- 246 . Liam Li, Kevin Jamieson, Afshin Rostamizadeh, Ekaterina Gonina, Jonathan Ben-tzur, Moritz Hardt, Benjamin Recht, and Ameet Talwalkar. 2020. A System for Massively Parallel Hyperparameter Tuning. Proceedings of Machine Learning and Systems 2 (2020), 230--246.","journal-title":"Proceedings of Machine Learning and Systems"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i04.5910"},{"key":"e_1_2_1_42_1","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"35","author":"Li Yang","year":"2021","unstructured":"Yang Li , Yu Shen , Jiawei Jiang , Jinyang Gao , Ce Zhang , and Bin Cui . 2021 . MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurements . In Proceedings of the AAAI Conference on Artificial Intelligence , Vol. 35 . 8491--8500. Yang Li, Yu Shen, Jiawei Jiang, Jinyang Gao, Ce Zhang, and Bin Cui. 2021. MFES-HB: Efficient Hyperband with Multi-Fidelity Quality Measurements. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35. 8491--8500."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467061"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476249.3476270"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11432-020-3117-3"},{"key":"e_1_2_1_46_1","volume-title":"Tune: A Research Platform for Distributed Model Selection and Training. arXiv preprint arXiv:1807.05118","author":"Liaw Richard","year":"2018","unstructured":"Richard Liaw , Eric Liang , Robert Nishihara , Philipp Moritz , Joseph E Gonzalez , and Ion Stoica . 2018 . Tune: A Research Platform for Distributed Model Selection and Training. arXiv preprint arXiv:1807.05118 (2018). Richard Liaw, Eric Liang, Robert Nishihara, Philipp Moritz, Joseph E Gonzalez, and Ion Stoica. 2018. Tune: A Research Platform for Distributed Model Selection and Training. arXiv preprint arXiv:1807.05118 (2018)."},{"key":"e_1_2_1_47_1","volume-title":"DARTS: Differentiable Architecture Search. In International Conference on Learning Representations.","author":"Liu Hanxiao","year":"2018","unstructured":"Hanxiao Liu , Karen Simonyan , and Yiming Yang . 2018 . DARTS: Differentiable Architecture Search. In International Conference on Learning Representations. Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018. DARTS: Differentiable Architecture Search. In International Conference on Learning Representations."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3350546.3360740"},{"key":"e_1_2_1_49_1","unstructured":"Leonel Aguilar Melgar David Dao Shaoduo Gan Nezihe Merve G\u00fcrel Nora Hollenstein Jiawei Jiang Bojan Karlas Thomas Lemmin Tian Li Yang Li Xi Rao Johannes Rausch C\u00e9dric Renggli Luka Rimanic Maurice Weber Shuai Zhang Zhikuan Zhao Kevin Schawinski Wentao Wu and Ce Zhang. 2021. Ease.ML: A Lifecycle Management System for Machine Learning. In CIDR.  Leonel Aguilar Melgar David Dao Shaoduo Gan Nezihe Merve G\u00fcrel Nora Hollenstein Jiawei Jiang Bojan Karlas Thomas Lemmin Tian Li Yang Li Xi Rao Johannes Rausch C\u00e9dric Renggli Luka Rimanic Maurice Weber Shuai Zhang Zhikuan Zhao Kevin Schawinski Wentao Wu and Ce Zhang. 2021. Ease.ML: A Lifecycle Management System for Machine Learning. In CIDR."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3319874"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.14778\/3407790.3407816"},{"key":"e_1_2_1_52_1","volume-title":"TPOT: A tree-based pipeline optimization tool for automating machine learning. In Automated Machine Learning","author":"Olson Randal S","year":"2019","unstructured":"Randal S Olson and Jason H Moore . 2019 . TPOT: A tree-based pipeline optimization tool for automating machine learning. In Automated Machine Learning . Springer , 151--160. Randal S Olson and Jason H Moore. 2019. TPOT: A tree-based pipeline optimization tool for automating machine learning. In Automated Machine Learning. Springer, 151--160."},{"key":"e_1_2_1_53_1","unstructured":"Matthias Poloczek Jialei Wang and Peter Frazier. 2017. Multi-information source optimization. In Advances in Neural Information Processing Systems. 4288--4298.  Matthias Poloczek Jialei Wang and Peter Frazier. 2017. Multi-information source optimization. In Advances in Neural Information Processing Systems. 4288--4298."},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-019-00552-1"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33014780"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137628.3137631"},{"key":"e_1_2_1_57_1","volume-title":"Neural architecture search using bayesian optimisation with weisfeiler-lehman kernel. arXiv preprint arXiv:2006.07556 3","author":"Ru Binxin","year":"2020","unstructured":"Binxin Ru , Xingchen Wan , Xiaowen Dong , and Michael Osborne . 2020. Neural architecture search using bayesian optimisation with weisfeiler-lehman kernel. arXiv preprint arXiv:2006.07556 3 ( 2020 ). Binxin Ru, Xingchen Wan, Xiaowen Dong, and Michael Osborne. 2020. Neural architecture search using bayesian optimisation with weisfeiler-lehman kernel. arXiv preprint arXiv:2006.07556 3 (2020)."},{"key":"e_1_2_1_58_1","volume-title":"Noisy Blackbox Optimization with Multi-Fidelity Queries: A Tree Search Approach. arXiv: Machine Learning","author":"Sen Rajat","year":"2018","unstructured":"Rajat Sen , Kirthevasan Kandasamy , and Sanjay Shakkottai . 2018. Noisy Blackbox Optimization with Multi-Fidelity Queries: A Tree Search Approach. arXiv: Machine Learning ( 2018 ). Rajat Sen, Kirthevasan Kandasamy, and Sanjay Shakkottai. 2018. Noisy Blackbox Optimization with Multi-Fidelity Queries: A Tree Search Approach. arXiv: Machine Learning (2018)."},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3319863"},{"key":"e_1_2_1_60_1","volume-title":"ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies. arXiv preprint arXiv:2110.10423","author":"Shen Yu","year":"2021","unstructured":"Yu Shen , Yang Li , Jian Zheng , Wentao Zhang , Peng Yao , Jixiang Li , Sen Yang , Ji Liu , and Cui Bin . 2021. ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies. arXiv preprint arXiv:2110.10423 ( 2021 ). Yu Shen, Yang Li, Jian Zheng, Wentao Zhang, Peng Yao, Jixiang Li, Sen Yang, Ji Liu, and Cui Bin. 2021. ProxyBO: Accelerating Neural Architecture Search via Bayesian Optimization with Zero-cost Proxies. arXiv preprint arXiv:2110.10423 (2021)."},{"key":"e_1_2_1_61_1","volume-title":"NAS-Bench-301 and the case for surrogate benchmarks for neural architecture search. arXiv preprint arXiv:2008.09777","author":"Siems Julien","year":"2020","unstructured":"Julien Siems , Lucas Zimmer , Arber Zela , Jovita Lukasik , Margret Keuper , and Frank Hutter . 2020. NAS-Bench-301 and the case for surrogate benchmarks for neural architecture search. arXiv preprint arXiv:2008.09777 ( 2020 ). Julien Siems, Lucas Zimmer, Arber Zela, Jovita Lukasik, Margret Keuper, and Frank Hutter. 2020. NAS-Bench-301 and the case for surrogate benchmarks for neural architecture search. arXiv preprint arXiv:2008.09777 (2020)."},{"key":"e_1_2_1_62_1","unstructured":"Jasper Snoek Hugo Larochelle and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems.  Jasper Snoek Hugo Larochelle and Ryan P Adams. 2012. Practical bayesian optimization of machine learning algorithms. In Advances in neural information processing systems."},{"key":"e_1_2_1_63_1","volume-title":"Proceedings of the 27th International Conference on Machine Learning. Omnipress.","author":"Srinivas Niranjan","year":"2010","unstructured":"Niranjan Srinivas , Andreas Krause , Sham Kakade , and Matthias Seeger . 2010 . Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design . In Proceedings of the 27th International Conference on Machine Learning. Omnipress. Niranjan Srinivas, Andreas Krause, Sham Kakade, and Matthias Seeger. 2010. Gaussian Process Optimization in the Bandit Setting: No Regret and Experimental Design. In Proceedings of the 27th International Conference on Machine Learning. Omnipress."},{"key":"e_1_2_1_64_1","volume-title":"Seeger","author":"Tiao Louis C.","year":"2020","unstructured":"Louis C. Tiao , Aaron Klein , C. Archambeau , and Matthias W . Seeger . 2020 . Model-based Asynchronous Hyperparameter Optimization . arXiv preprint arXiv:2003.10865 (2020). Louis C. Tiao, Aaron Klein, C. Archambeau, and Matthias W. Seeger. 2020. Model-based Asynchronous Hyperparameter Optimization. arXiv preprint arXiv:2003.10865 (2020)."},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/2641190.2641198"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939502.2939516"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.14778\/3282495.3282499"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.5555\/2540128.2540383"},{"key":"e_1_2_1_69_1","volume-title":"Bananas: Bayesian optimization with neural architectures for neural architecture search. arXiv preprint arXiv:1910.11858","author":"White Colin","year":"2019","unstructured":"Colin White , Willie Neiswanger , and Yash Savani . 2019 . Bananas: Bayesian optimization with neural architectures for neural architecture search. arXiv preprint arXiv:1910.11858 (2019). Colin White, Willie Neiswanger, and Yash Savani. 2019. Bananas: Bayesian optimization with neural architectures for neural architecture search. arXiv preprint arXiv:1910.11858 (2019)."},{"key":"e_1_2_1_70_1","unstructured":"Jian Wu Saul Toscanopalmerin Peter I Frazier and Andrew Gordon Wilson. 2019. Practical multi-fidelity Bayesian optimization for hyperparameter tuning. (2019) 284.  Jian Wu Saul Toscanopalmerin Peter I Frazier and Andrew Gordon Wilson. 2019. Practical multi-fidelity Bayesian optimization for hyperparameter tuning. (2019) 284."},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389743"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389696"},{"key":"e_1_2_1_73_1","volume-title":"PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search. In International Conference on Learning Representations.","author":"Xu Yuhui","year":"2019","unstructured":"Yuhui Xu , Lingxi Xie , Xiaopeng Zhang , Xin Chen , Guo-Jun Qi , Qi Tian , and Hongkai Xiong . 2019 . PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search. In International Conference on Learning Representations. Yuhui Xu, Lingxi Xie, Xiaopeng Zhang, Xin Chen, Guo-Jun Qi, Qi Tian, and Hongkai Xiong. 2019. PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Search. In International Conference on Learning Representations."},{"key":"e_1_2_1_74_1","volume-title":"Taking human out of learning applications: A survey on automated machine learning. arXiv preprint arXiv:1810.13306","author":"Yao Quanming","year":"2018","unstructured":"Quanming Yao , Mengshuo Wang , Yuqiang Chen , Wenyuan Dai , Yu-Feng Li , Wei-Wei Tu , Qiang Yang , and Yang Yu. 2018. Taking human out of learning applications: A survey on automated machine learning. arXiv preprint arXiv:1810.13306 ( 2018 ). Quanming Yao, Mengshuo Wang, Yuqiang Chen, Wenyuan Dai, Yu-Feng Li, Wei-Wei Tu, Qiang Yang, and Yang Yu. 2018. Taking human out of learning applications: A survey on automated machine learning. arXiv preprint arXiv:1810.13306 (2018)."},{"key":"e_1_2_1_75_1","first-page":"39","article-title":"Accelerating the machine learning lifecycle with MLflow","volume":"41","author":"Zaharia Matei","year":"2018","unstructured":"Matei Zaharia , Andrew Chen , Aaron Davidson , Ali Ghodsi , Sue Ann Hong , Andy Konwinski , Siddharth Murching , Tomas Nykodym , Paul Ogilvie , Mani Parkhe , 2018 . Accelerating the machine learning lifecycle with MLflow . IEEE Data Eng. Bull. 41 , 4 (2018), 39 -- 45 . Matei Zaharia, Andrew Chen, Aaron Davidson, Ali Ghodsi, Sue Ann Hong, Andy Konwinski, Siddharth Murching, Tomas Nykodym, Paul Ogilvie, Mani Parkhe, et al. 2018. Accelerating the machine learning lifecycle with MLflow. IEEE Data Eng. Bull. 41, 4 (2018), 39--45.","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_2_1_76_1","volume-title":"Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation. arXiv preprint arXiv:2110.12654","author":"Zhang Xinyi","year":"2021","unstructured":"Xinyi Zhang , Zhuo Chang , Yang Li , Hong Wu , Jian Tan , Feifei Li , and Bin Cui . 2021. Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation. arXiv preprint arXiv:2110.12654 ( 2021 ). Xinyi Zhang, Zhuo Chang, Yang Li, Hong Wu, Jian Tan, Feifei Li, and Bin Cui. 2021. Facilitating Database Tuning with Hyper-Parameter Optimization: A Comprehensive Experimental Evaluation. arXiv preprint arXiv:2110.12654 (2021)."},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457291"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00907"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3514061.3514071","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T09:26:11Z","timestamp":1672219571000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3514061.3514071"}},"subtitle":["towards efficient hyper-parameter tuning at scale"],"short-title":[],"issued":{"date-parts":[[2022,2]]},"references-count":78,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2022,2]]}},"alternative-id":["10.14778\/3514061.3514071"],"URL":"https:\/\/doi.org\/10.14778\/3514061.3514071","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2022,2]]}}}