{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T16:49:58Z","timestamp":1790700598771,"version":"4.1.0"},"reference-count":24,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,8]]},"abstract":"<jats:p>This paper presents MLOS (ML Optimized Systems), a flexible framework that bridges the gap between benchmarking, experimentation, and optimization of software systems. It allows users to create one-click benchmarking and experimentation scenarios for multi-VM setups in the cloud with optional standard and custom metrics collection and data management of the results. MLOS provides a collection of pluggable optimizers (ML or otherwise) for efficiently exploring the configuration space and finding optimal values for parameters across the entire software stack, including VM, OS kernel, and userland applications. It has a convenient lightweight interface for data storage, access, and visualization for a user-friendly notebook experience. These features make it a useful platform for both systems developers and auto-tuning researchers. MLOS is an active open-source project and is being used within Azure Data. A video demonstrating MLOS is available at https:\/\/aka.ms\/MLOS\/VLDB-2024-demo-video.<\/jats:p>","DOI":"10.14778\/3685800.3685852","type":"journal-article","created":{"date-parts":[[2024,11,8]],"date-time":"2024-11-08T17:25:21Z","timestamp":1731086721000},"page":"4269-4272","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["MLOS in Action: Bridging the Gap Between Experimentation and Auto-Tuning in the Cloud"],"prefix":"10.14778","volume":"17","author":[{"given":"Brian","family":"Kroth","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sergiy","family":"Matusevych","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rana","family":"Alotaibi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yiwen","family":"Zhu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anja","family":"Gruenheid","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuanyuan","family":"Tian","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,11,8]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2024. SQLite. Retrieved Jan 22 2024 from https:\/\/www.sqlite.org"},{"key":"e_1_2_1_2_1","unstructured":"DBT Solutions AB. 2024. DBTune. https:\/\/www.dbtune.com"},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Van Aken et al. 2017. Automatic database management system tuning through large-scale machine learning. In SIGMOD. 1009--1024.","DOI":"10.1145\/3035918.3064029"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.14778\/3450980.3450992"},{"key":"e_1_2_1_5_1","volume-title":"Kernel Parameter tuning for Memcached. GitHub\/preprint","author":"Bonde Anand","year":"2022","unstructured":"Anand Bonde. 2022. Kernel Parameter tuning for Memcached. GitHub\/preprint (2022). https:\/\/github.com\/anandbonde\/anandbonde.github.io\/blob\/main\/docs\/kernel_parameter_tuning_for_memcached.pdf"},{"key":"e_1_2_1_6_1","unstructured":"Jes\u00fas Camacho-Rodr\u00edguez et al. 2023. LST-Bench: Benchmarking Log-Structured Tables in the Cloud. arXiv preprint arXiv:2305.01120 (2023)."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3399579.3399927"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732240.2732246"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the USENIX Annual Technical Conference (ATC). 1--14","author":"Dmitry","unstructured":"Dmitry Duplyakin et al. 2019. The Design and Operation of CloudLab. In Proceedings of the USENIX Annual Technical Conference (ATC). 1--14."},{"key":"e_1_2_1_10_1","volume-title":"Performance Roulette: How Cloud Weather Affects ML-Based System Optimization. In ML for Systems Workshop at NeurIPS.","author":"Johannes","unstructured":"Johannes Freischuetz et al. 2023. Performance Roulette: How Cloud Weather Affects ML-Based System Optimization. In ML for Systems Workshop at NeurIPS."},{"key":"e_1_2_1_11_1","volume-title":"Google Vizier: A Service for Black-Box Optimization (KDD '17). 1487--1495.","author":"Daniel Golovin","year":"2017","unstructured":"Daniel Golovin et al. 2017. Google Vizier: A Service for Black-Box Optimization (KDD '17). 1487--1495."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.3390\/s22155930"},{"key":"e_1_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Konstantinos Kanellis et al. 2022. LlamaTune: sample-efficient DBMS configuration tuning. arXiv preprint arXiv:2203.05128 (2022).","DOI":"10.14778\/3551793.3551844"},{"key":"e_1_2_1_14_1","unstructured":"Jiale Lao et al. 2023. GPTuner: A Manual-Reading Database Tuning System via GPT-Guided Bayesian Optimization. arXiv preprint arXiv:2311.03157 (2023)."},{"key":"e_1_2_1_15_1","first-page":"495","article-title":"Constrained Bayesian Optimization with Noisy Experiments","volume":"14","author":"Benjamin Letham","year":"2019","unstructured":"Benjamin Letham et al. 2019. Constrained Bayesian Optimization with Noisy Experiments. Bayesian Analysis 14, 2 (2019), 495--519.","journal-title":"Bayesian Analysis"},{"key":"e_1_2_1_16_1","volume-title":"Ax: Adaptive Experimentation Platform. https:\/\/ax.dev","author":"Platforms Meta","year":"2024","unstructured":"Meta Platforms, Inc. 2024. Ax: Adaptive Experimentation Platform. https:\/\/ax.dev"},{"key":"e_1_2_1_17_1","doi-asserted-by":"crossref","unstructured":"Van Otterlo et al. 2012. Reinforcement learning and Markov decision processes. In Reinforcement learning: State-of-the-art. 3--42.","DOI":"10.1007\/978-3-642-27645-3_1"},{"key":"e_1_2_1_18_1","unstructured":"OtterTune. 2024. OtterTune. Retrieved Jan 2 2024 from https:\/\/ottertune.com"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3520171"},{"key":"e_1_2_1_20_1","first-page":"1910","article-title":"A demonstration of the OtterTune automatic database management system tuning service","volume":"11","author":"Bohan Zhang","year":"2018","unstructured":"Bohan Zhang et al. 2018. A demonstration of the OtterTune automatic database management system tuning service. PVLDB 11, 12 (2018), 1910--1913.","journal-title":"PVLDB"},{"key":"e_1_2_1_21_1","volume-title":"Dike: A Benchmark Suite for Distributed Transactional Databases. In Companion of the 2023 International Conference on Management of Data (SIGMOD '23)","author":"Huidong","unstructured":"Huidong Zhang et al. 2023. Dike: A Benchmark Suite for Distributed Transactional Databases. In Companion of the 2023 International Conference on Management of Data (SIGMOD '23). 95--98."},{"key":"e_1_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Ji Zhang et al. 2019. An end-to-end automatic cloud database tuning system using deep reinforcement learning (SIGMOD '19). 415--432.","DOI":"10.1145\/3299869.3300085"},{"key":"e_1_2_1_23_1","volume-title":"Proceedings of the 2021 International Conference on Management of Data.","author":"Yiwen","unstructured":"Yiwen Zhu et al. 2021. KEA: Tuning an Exabyte-Scale Data Infrastructure. In Proceedings of the 2021 International Conference on Management of Data."},{"key":"e_1_2_1_24_1","volume-title":"Towards Building Autonomous Data Services on Azure. In Companion of the 2023 International Conference on Management of Data. 217--224","author":"Yiwen","unstructured":"Yiwen Zhu et al. 2023. Towards Building Autonomous Data Services on Azure. In Companion of the 2023 International Conference on Management of Data. 217--224."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3685800.3685852","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,31]],"date-time":"2024-12-31T05:28:01Z","timestamp":1735622881000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3685800.3685852"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8]]},"references-count":24,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["10.14778\/3685800.3685852"],"URL":"https:\/\/doi.org\/10.14778\/3685800.3685852","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,8]]},"assertion":[{"value":"2024-11-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}