{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,17]],"date-time":"2026-07-17T04:05:47Z","timestamp":1784261147571,"version":"3.55.0"},"reference-count":78,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:p>Database management systems (DBMSs) are notoriously difficult to deploy and administer. Self-driving DBMSs seek to remove these impediments by managing themselves automatically. Despite decades of DBMS auto-tuning research, a truly autonomous, self-driving DBMS is yet to come. But recent advancements in artificial intelligence and machine learning (ML) have moved this goal closer.<\/jats:p><jats:p>Given this, we present a system implementation treatise towards achieving a self-driving DBMS. We first provide an overview of the NoisePage self-driving DBMS that uses ML to predict the DBMS's behavior and optimize itself without human support or guidance. The system's architecture has three main ML-based components: (1) workload forecasting, (2) behavior modeling, and (3) action planning. We then describe the system design principles to facilitate holistic autonomous operations. Such prescripts reduce the complexity of the problem, thereby enabling a DBMS to converge to a better and more stable configuration more quickly.<\/jats:p>","DOI":"10.14778\/3476311.3476411","type":"journal-article","created":{"date-parts":[[2021,10,28]],"date-time":"2021-10-28T22:48:43Z","timestamp":1635461323000},"page":"3211-3221","source":"Crossref","is-referenced-by-count":28,"title":["Make your database system dream of electric sheep"],"prefix":"10.14778","volume":"14","author":[{"given":"Andrew","family":"Pavlo","sequence":"first","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew","family":"Butrovich","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Ma","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Prashanth","family":"Menon","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wan Shen","family":"Lim","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dana","family":"Van Aken","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William","family":"Zhang","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,10,28]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2021. NoisePage - Database Management System Project. https:\/\/noise.page. 2021. NoisePage - Database Management System Project. https:\/\/noise.page."},{"key":"e_1_2_1_2_1","unstructured":"2021. Self-Driving Database | Autonomous Database Oracle 19c. https:\/\/oracle.com\/database\/autonomous-database\/. 2021. Self-Driving Database | Autonomous Database Oracle 19c. https:\/\/oracle.com\/database\/autonomous-database\/."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1075405.1075406"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.5555\/645926.671701"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007568.1007609"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.14778\/3450980.3450992"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2610502"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2915231"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-22849-5_18"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2015.2429128"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2007477.1952691"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.14778\/3457390.3457404"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/564691.564747"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/1325851.1325856"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/645923.673646"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/645926.671696"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02161-9_1"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2012.02.060"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.5555\/1287369.1287454"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273496.1273521"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2445583.2445588"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3314035"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2903733"},{"key":"e_1_2_1_24_1","unstructured":"Rog\u00e9rio De Lemos Holger Giese Hausi A M\u00fcller Mary Shaw Jesper Andersson Marin Litoiu Bradley Schmerl Gabriel Tamura Norha M Villegas Thomas Vogel etal 2013. Software engineering for self-adaptive systems: A second research roadmap. In Software Engineering for Self-Adaptive Systems II. Springer 1--32. Rog\u00e9rio De Lemos Holger Giese Hausi A M\u00fcller Mary Shaw Jesper Andersson Marin Litoiu Bradley Schmerl Gabriel Tamura Norha M Villegas Thomas Vogel et al. 2013. Software engineering for self-adaptive systems: A second research roadmap. In Software Engineering for Self-Adaptive Systems II . Springer 1--32."},{"key":"e_1_2_1_25_1","unstructured":"Karl Dias Mark Ramacher Uri Shaft Venkateshwaran Venkataramani and Graham Wood. 2005. Automatic Performance Diagnosis and Tuning in Oracle. In CIDR. Karl Dias Mark Ramacher Uri Shaft Venkateshwaran Venkataramani and Graham Wood. 2005. Automatic Performance Diagnosis and Tuning in Oracle. In CIDR ."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687627.1687767"},{"key":"e_1_2_1_27_1","doi-asserted-by":"crossref","unstructured":"Gabriel Campero Durand Marcus Pinnecke Rufat Piriyev Mahmoud Mohsen David Broneske Gunter Saake Maya S. Sekeran Fabi\u00e1n Rodriguez and Laxmi Balami. 2018. GridFormation: Towards Self-Driven Online Data Partitioning Using Reinforcement Learning (aiDM'18). Article 1 7 pages. Gabriel Campero Durand Marcus Pinnecke Rufat Piriyev Mahmoud Mohsen David Broneske Gunter Saake Maya S. Sekeran Fabi\u00e1n Rodriguez and Laxmi Balami. 2018. GridFormation: Towards Self-Driven Online Data Partitioning Using Reinforcement Learning (aiDM'18) . Article 1 7 pages.","DOI":"10.1145\/3211954.3211956"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2004.175"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.5555\/645482.653582"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/509383.509385"},{"key":"e_1_2_1_31_1","unstructured":"Stratos Idreos Martin L. Kersten and Stefan Manegold. 2007. Database Cracking. In CIDR. 68--78. Stratos Idreos Martin L. Kersten and Stefan Manegold. 2007. Database Cracking. In CIDR . 68--78."},{"key":"e_1_2_1_32_1","volume-title":"Adaptive Cardinality Estimation. CoRR abs\/1711.08330","author":"Ivanov Oleg","year":"2017","unstructured":"Oleg Ivanov and Sergey Bartunov . 2017. Adaptive Cardinality Estimation. CoRR abs\/1711.08330 ( 2017 ). http:\/\/arxiv.org\/abs\/1711.08330 Oleg Ivanov and Sergey Bartunov. 2017. Adaptive Cardinality Estimation. CoRR abs\/1711.08330 (2017). http:\/\/arxiv.org\/abs\/1711.08330"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1062455.1062464"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2003.1160055"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3320218"},{"key":"e_1_2_1_36_1","unstructured":"Sushil Kumar. 2003. Oracle Database 10g: The Self-Managing Database. White Paper. Sushil Kumar. 2003. Oracle Database 10g: The Self-Managing Database. White Paper."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352129"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3296957.3173191"},{"key":"e_1_2_1_40_1","volume-title":"Proceedings of the International Workshop on Real-Time Databases, Issues and Applications (RTDB). 72--78","author":"Listgarten Sherry","year":"1996","unstructured":"Sherry Listgarten and Marie-Anne Neimat . 1996 . Modelling Costs for a MM-DBMS . In Proceedings of the International Workshop on Real-Time Databases, Issues and Applications (RTDB). 72--78 . Sherry Listgarten and Marie-Anne Neimat. 1996. Modelling Costs for a MM-DBMS. In Proceedings of the International Workshop on Real-Time Databases, Issues and Applications (RTDB). 72--78."},{"key":"e_1_2_1_41_1","volume-title":"Cardinality Estimation Using Neural Networks (CASCON '15)","author":"Liu Henry","year":"2015","unstructured":"Henry Liu , Mingbin Xu , Ziting Yu , Vincent Corvinelli , and Calisto Zuzarte . 2015 . Cardinality Estimation Using Neural Networks (CASCON '15) . 53--59. Henry Liu, Mingbin Xu, Ziting Yu, Vincent Corvinelli, and Calisto Zuzarte. 2015. Cardinality Estimation Using Neural Networks (CASCON '15). 53--59."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389768"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3196908"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457276"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3452838"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/3211954.3211957"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.14778\/3342263.3342646"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/MCS.2011.940571"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/2786805.2786853"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2467800"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.5555\/1097871.1098137"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989444"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209889.3209890"},{"key":"e_1_2_1_54_1","volume-title":"Dmitry S. Yershov, and Emilio Frazzoli.","author":"Paden Brian","year":"2016","unstructured":"Brian Paden , Michal C\u00e1p , Sze Zheng Yong , Dmitry S. Yershov, and Emilio Frazzoli. 2016 . A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles. IEEE Trans. on Intelligent Vehicles ( 2016), 33--55. Brian Paden, Michal C\u00e1p, Sze Zheng Yong, Dmitry S. Yershov, and Emilio Frazzoli. 2016. A Survey of Motion Planning and Control Techniques for Self-Driving Urban Vehicles. IEEE Trans. on Intelligent Vehicles (2016), 33--55."},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.5555\/998688.1007145"},{"key":"e_1_2_1_56_1","volume-title":"Self-Driving Database Management Systems. In Conference on Innovative Data Systems Research (CIDR '17)","author":"Pavlo Andrew","year":"2017","unstructured":"Andrew Pavlo , Gustavo Angulo , Joy Arulraj , Haibin Lin , Jiexi Lin , Lin Ma , Prashanth Menon , Todd C. Mowry , Matthew Perron , Ian Quah , Siddharth Santurkar , Anthony Tomasic , Skye Toor , Dana Van Aken , Ziqi Wang , Yingjun Wu , Ran Xian , and Tieying Zhang . 2017 . Self-Driving Database Management Systems. In Conference on Innovative Data Systems Research (CIDR '17) . Andrew Pavlo, Gustavo Angulo, Joy Arulraj, Haibin Lin, Jiexi Lin, Lin Ma, Prashanth Menon, Todd C. Mowry, Matthew Perron, Ian Quah, Siddharth Santurkar, Anthony Tomasic, Skye Toor, Dana Van Aken, Ziqi Wang, Yingjun Wu, Ran Xian, and Tieying Zhang. 2017. Self-Driving Database Management Systems. In Conference on Innovative Data Systems Research (CIDR '17)."},{"key":"e_1_2_1_57_1","volume-title":"Lisa Lee, and Ruslan Salakhutdinov.","author":"Pavlo Andrew","year":"2019","unstructured":"Andrew Pavlo , Matthew Butrovich , Ananya Joshi , Lin Ma , Prashanth Menon , Dana Van Aken , Lisa Lee, and Ruslan Salakhutdinov. 2019 . External vs. Internal : An Essay on Machine Learning Agents for Autonomous Database Management Systems. IEEE Data Engineering Bulletin ( 2019), 32--46. Andrew Pavlo, Matthew Butrovich, Ananya Joshi, Lin Ma, Prashanth Menon, Dana Van Aken, Lisa Lee, and Ruslan Salakhutdinov. 2019. External vs. Internal: An Essay on Machine Learning Agents for Autonomous Database Management Systems. IEEE Data Engineering Bulletin (2019), 32--46."},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/2213836.2213844"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.14778\/2078324.2078325"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/564691.564757"},{"key":"e_1_2_1_61_1","volume-title":"Data Engineering Workshops (ICDEW), 2010 IEEE 26th International Conference on. 63--68","author":"Rogers J.","unstructured":"J. Rogers , O. Papaemmanouil , and U. Cetintemel . 2010. A generic auto-provisioning framework for cloud databases . In Data Engineering Workshops (ICDEW), 2010 IEEE 26th International Conference on. 63--68 . J. Rogers, O. Papaemmanouil, and U. Cetintemel. 2010. A generic auto-provisioning framework for cloud databases. In Data Engineering Workshops (ICDEW), 2010 IEEE 26th International Conference on. 63--68."},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1109\/CLOUD.2011.42"},{"key":"e_1_2_1_63_1","volume-title":"Felix Martin Schuhknecht, and Jens Dittrich","author":"Sharma Ankur","year":"2018","unstructured":"Ankur Sharma , Felix Martin Schuhknecht, and Jens Dittrich . 2018 . The Case for Automatic Database Administration using Deep Reinforcement Learning. CoRR abs\/1801.05643 (2018). http:\/\/arxiv.org\/abs\/1801.05643 Ankur Sharma, Felix Martin Schuhknecht, and Jens Dittrich. 2018. The Case for Automatic Database Administration using Deep Reinforcement Learning. CoRR abs\/1801.05643 (2018). http:\/\/arxiv.org\/abs\/1801.05643"},{"key":"e_1_2_1_64_1","doi-asserted-by":"crossref","unstructured":"David Silver Aja Huang Chris J. Maddison etal 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529 7587 (2016) 484--489. David Silver Aja Huang Chris J. Maddison et al. 2016. Mastering the game of Go with deep neural networks and tree search. Nature 529 7587 (2016) 484--489.","DOI":"10.1038\/nature16961"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.5555\/645927.672349"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.5555\/1182635.1164220"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1145\/1012888.1005739"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.5555\/961322.961367"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.5555\/846219.847390"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3064029"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3452754"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/3003665.3003669"},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.5555\/1287369.1287373"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/1463788.1463792"},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/2465478.2465479"},{"key":"e_1_2_1_76_1","doi-asserted-by":"publisher","DOI":"10.14778\/3229863.3236222"},{"key":"e_1_2_1_77_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3300085"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.14778\/3397230.3397238"},{"key":"e_1_2_1_80_1","doi-asserted-by":"publisher","DOI":"10.5555\/1316689.1316783"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3476311.3476411","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,11]],"date-time":"2024-09-11T00:11:20Z","timestamp":1726013480000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3476311.3476411"}},"subtitle":["towards self-driving operation"],"short-title":[],"issued":{"date-parts":[[2021,7]]},"references-count":78,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["10.14778\/3476311.3476411"],"URL":"https:\/\/doi.org\/10.14778\/3476311.3476411","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2021,7]]}}}