{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T10:16:27Z","timestamp":1781518587416,"version":"3.54.1"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"13","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2021,9]]},"abstract":"<jats:p>\n            Modern data processing systems require optimization at massive scale, and using machine learning to optimize these systems (ML-for-systems) has shown promising results. Unfortunately, ML-for-systems is subject to over generalizations that do not capture the large variety of workload patterns, and tend to augment the performance of certain subsets in the workload while regressing performance for others. In this paper, we introduce a performance safeguard system, called\n            <jats:italic>PerfGuard<\/jats:italic>\n            , that designs pre-production experiments for deploying ML-for-systems. Instead of searching the entire space of query plans (a well-known, intractable problem), we focus on query plan deltas (a significantly smaller space). PerfGuard formalizes these differences, and correlates plan deltas to important feedback signals, like execution cost. We describe the deep learning architecture and the end-to-end pipeline in PerfGuard that could be used with general relational databases. We show that this architecture improves on baseline models, and that our pipeline identifies key query plan components as major contributors to plan disparity. Offline experimentation shows PerfGuard as a promising approach, with many opportunities for future improvement.\n          <\/jats:p>","DOI":"10.14778\/3484224.3484233","type":"journal-article","created":{"date-parts":[[2021,10,28]],"date-time":"2021-10-28T22:36:50Z","timestamp":1635460610000},"page":"3362-3375","source":"Crossref","is-referenced-by-count":6,"title":["PerfGuard"],"prefix":"10.14778","volume":"14","author":[{"given":"Remmelt","family":"Ammerlaan","sequence":"first","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gilbert","family":"Antonius","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marc","family":"Friedman","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"H M Sajjad","family":"Hossain","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alekh","family":"Jindal","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Peter","family":"Orenberg","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hiren","family":"Patel","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shi","family":"Qiao","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Vijay","family":"Ramani","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lucas","family":"Rosenblatt","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abhishek","family":"Roy","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Irene","family":"Shaffer","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Soundarajan","family":"Srinivasan","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus","family":"Weimer","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,10,28]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2012.64"},{"key":"e_1_2_1_2_1","unstructured":"Azure. 2021. Azure VM NC Series. https:\/\/docs.microsoft.com\/en-us\/azure\/virtual-machines\/nc-series.  Azure. 2021. Azure VM NC Series. https:\/\/docs.microsoft.com\/en-us\/azure\/virtual-machines\/nc-series."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3289600.3290967"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.14778\/1454159.1454166"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1017\/ATSIP.2020.13"},{"key":"e_1_2_1_6_1","volume-title":"TPC Benchmark DS Standard Specification Version 1.1. 0","author":"Transaction Processing Performance Council","year":"2012","unstructured":"Transaction Processing Performance Council . [n.d.]. TPC Benchmark DS Standard Specification Version 1.1. 0 ( 2012 ). URL : http:\/\/www.tpc.org\/tpcc () ([n. d.]). Transaction Processing Performance Council. [n.d.]. TPC Benchmark DS Standard Specification Version 1.1. 0 (2012). URL: http:\/\/www.tpc.org\/tpcc () ([n. d.])."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3324957"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389711"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.14778\/3407790.3407820"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.14778\/3329772.3329780"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2009.130"},{"key":"e_1_2_1_12_1","unstructured":"Shohedul Hasan Saravanan Thirumuruganathan Jees Augustine Nick Koudas and Gautam Das. 2019. Multi-attribute selectivity estimation using deep learning. arXiv:1903.09999  Shohedul Hasan Saravanan Thirumuruganathan Jees Augustine Nick Koudas and Gautam Das. 2019. Multi-attribute selectivity estimation using deep learning. arXiv:1903.09999"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389741"},{"key":"e_1_2_1_14_1","unstructured":"Rojeh Hayek and Oded Shmueli. 2020. Nn-based transformation of any SQL cardinality estimator for handling distinct and OR and NOT. arXiv:2004.07009  Rojeh Hayek and Oded Shmueli. 2020. Nn-based transformation of any SQL cardinality estimator for handling distinct and OR and NOT. arXiv:2004.07009"},{"key":"e_1_2_1_15_1","doi-asserted-by":"crossref","unstructured":"Benjamin Hilprecht Andreas Schmidt Moritz Kulessa Alejandro Molina Kristian Kersting and Carsten Binnig. 2019. DeepDB: learn from data not from queries! arXiv:1909.00607  Benjamin Hilprecht Andreas Schmidt Moritz Kulessa Alejandro Molina Kristian Kersting and Carsten Binnig. 2019. DeepDB: learn from data not from queries! arXiv:1909.00607","DOI":"10.14778\/3384345.3384349"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1162\/089976602760128018"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3357223.3362726"},{"key":"e_1_2_1_18_1","volume-title":"NeurIPS'2018 Workshop.","author":"Paolis Kaluza M Clara De","year":"2018","unstructured":"M Clara De Paolis Kaluza , Saeed Amizadeh , and Rose Yu . 2018 . A neural framework for learning DAG to DAG translation . In NeurIPS'2018 Workshop. M Clara De Paolis Kaluza, Saeed Amizadeh, and Rose Yu. 2018. A neural framework for learning DAG to DAG translation. In NeurIPS'2018 Workshop."},{"key":"e_1_2_1_19_1","volume-title":"Learned Cardinalities: Estimating Correlated Joins with Deep Learning. In CIDR","author":"Kipf Andreas","year":"2019","unstructured":"Andreas Kipf , Thomas Kipf , Bernhard Radke , Viktor Leis , Peter A. Boncz , and Alfons Kemper . 2019 . Learned Cardinalities: Estimating Correlated Joins with Deep Learning. In CIDR 2019, 9th Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 13--16, 2019, Online Proceedings . www.cidrdb.org. http:\/\/cidrdb.org\/cidr2019\/papers\/p101-kipf-cidr19.pdf Andreas Kipf, Thomas Kipf, Bernhard Radke, Viktor Leis, Peter A. Boncz, and Alfons Kemper. 2019. Learned Cardinalities: Estimating Correlated Joins with Deep Learning. In CIDR 2019, 9th Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 13--16, 2019, Online Proceedings. www.cidrdb.org. http:\/\/cidrdb.org\/cidr2019\/papers\/p101-kipf-cidr19.pdf"},{"key":"e_1_2_1_20_1","volume-title":"SOSD: A Benchmark for Learned Indexes. CoRR abs\/1911.13014","author":"Kipf Andreas","year":"2019","unstructured":"Andreas Kipf , Ryan Marcus , Alexander van Renen , Mihail Stoian , Alfons Kemper , Tim Kraska , and Thomas Neumann . 2019 . SOSD: A Benchmark for Learned Indexes. CoRR abs\/1911.13014 (2019). arXiv:1911.13014 http:\/\/arxiv.org\/abs\/1911.13014 Andreas Kipf, Ryan Marcus, Alexander van Renen, Mihail Stoian, Alfons Kemper, Tim Kraska, and Thomas Neumann. 2019. SOSD: A Benchmark for Learned Indexes. CoRR abs\/1911.13014 (2019). arXiv:1911.13014 http:\/\/arxiv.org\/abs\/1911.13014"},{"key":"e_1_2_1_21_1","volume-title":"Kipf and Max Welling","author":"Thomas","year":"2016","unstructured":"Thomas N. Kipf and Max Welling . 2016 . Semi-Supervised Classification with Graph Convolutional Networks. CoRR abs\/1609.02907 (2016). http:\/\/arxiv.org\/abs\/1609.02907 Thomas N. Kipf and Max Welling. 2016. Semi-Supervised Classification with Graph Convolutional Networks. CoRR abs\/1609.02907 (2016). http:\/\/arxiv.org\/abs\/1609.02907"},{"key":"e_1_2_1_22_1","volume-title":"Ani Kristo, Guillaume Leclerc, Samuel Madden, Hongzi Mao, and Vikram Nathan.","author":"Kraska Tim","year":"2019","unstructured":"Tim Kraska , Mohammad Alizadeh , Alex Beutel , Ed H. Chi , Ani Kristo, Guillaume Leclerc, Samuel Madden, Hongzi Mao, and Vikram Nathan. 2019 . SageDB: A Learned Database System. In CIDR 2019, 9th Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 13--16, 2019, Online Proceedings . www.cidrdb.org. http:\/\/cidrdb.org\/cidr2019\/papers\/p117-kraska-cidr19.pdf Tim Kraska, Mohammad Alizadeh, Alex Beutel, Ed H. Chi, Ani Kristo, Guillaume Leclerc, Samuel Madden, Hongzi Mao, and Vikram Nathan. 2019. SageDB: A Learned Database System. In CIDR 2019, 9th Biennial Conference on Innovative Data Systems Research, Asilomar, CA, USA, January 13--16, 2019, Online Proceedings. www.cidrdb.org. http:\/\/cidrdb.org\/cidr2019\/papers\/p117-kraska-cidr19.pdf"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3196909"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/s41109-019-0195-3"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.14778\/2350229.2350269"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3341302.3342080"},{"key":"e_1_2_1_27_1","volume-title":"Bao: Learning to Steer Query Optimizers. ArXiv abs\/2004.03814","author":"Marcus Ryan","year":"2020","unstructured":"Ryan Marcus , Parimarjan Negi , Hongzi Mao , Nesime Tatbul , M. Alizadeh , and T. Kraska . 2020 . Bao: Learning to Steer Query Optimizers. ArXiv abs\/2004.03814 (2020). Ryan Marcus, Parimarjan Negi, Hongzi Mao, Nesime Tatbul, M. Alizadeh, and T. Kraska. 2020. Bao: Learning to Steer Query Optimizers. ArXiv abs\/2004.03814 (2020)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.14778\/3342263.3342644"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3384706"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457568"},{"key":"e_1_2_1_31_1","volume-title":"Flow-Loss: Learning Cardinality Estimates That Matter. arXiv preprint arXiv:2101.04964","author":"Negi Parimarjan","year":"2021","unstructured":"Parimarjan Negi , Ryan Marcus , Andreas Kipf , Hongzi Mao , Nesime Tatbul , Tim Kraska , and Mohammad Alizadeh . 2021. Flow-Loss: Learning Cardinality Estimates That Matter. arXiv preprint arXiv:2101.04964 ( 2021 ). Parimarjan Negi, Ryan Marcus, Andreas Kipf, Hongzi Mao, Nesime Tatbul, Tim Kraska, and Mohammad Alizadeh. 2021. Flow-Loss: Learning Cardinality Estimates That Matter. arXiv preprint arXiv:2101.04964 (2021)."},{"key":"e_1_2_1_32_1","volume-title":"CoRR abs\/1410.3314","author":"Neumann Marion","year":"2014","unstructured":"Marion Neumann , Roman Garnett , Christian Bauckhage , and Kristian Kersting . 2014. Propagation Kernels . CoRR abs\/1410.3314 ( 2014 ). http:\/\/arxiv.org\/abs\/1410.3314 Marion Neumann, Roman Garnett, Christian Bauckhage, and Kristian Kersting. 2014. Propagation Kernels. CoRR abs\/1410.3314 (2014). http:\/\/arxiv.org\/abs\/1410.3314"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.5555\/3298483.3298589"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209889.3209890"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920902"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415554"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3380584"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.5555\/2999611.2999715"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3452790"},{"key":"e_1_2_1_40_1","volume-title":"Partitioned Learned Bloom Filter. CoRR abs\/2006.03176","author":"Vaidya Kapil","year":"2020","unstructured":"Kapil Vaidya , Eric Knorr , Tim Kraska , and Michael Mitzenmacher . 2020. Partitioned Learned Bloom Filter. CoRR abs\/2006.03176 ( 2020 ). arXiv:2006.03176 https:\/\/arxiv.org\/abs\/2006.03176 Kapil Vaidya, Eric Knorr, Tim Kraska, and Michael Mitzenmacher. 2020. Partitioned Learned Bloom Filter. CoRR abs\/2006.03176 (2020). arXiv:2006.03176 https:\/\/arxiv.org\/abs\/2006.03176"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.14778\/3461535.3461552"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.14778\/3291264.3291267"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3452830"},{"key":"e_1_2_1_44_1","volume-title":"Yu","author":"Wu Zonghan","year":"2019","unstructured":"Zonghan Wu , Shirui Pan , Fengwen Chen , Guodong Long , Chengqi Zhang , and Philip S . Yu . 2019 . A Comprehensive Survey on Graph Neural Networks. CoRR abs\/1901.00596 (2019). http:\/\/arxiv.org\/abs\/1901.00596 Zonghan Wu, Shirui Pan, Fengwen Chen, Guodong Long, Chengqi Zhang, and Philip S. Yu. 2019. A Comprehensive Survey on Graph Neural Networks. CoRR abs\/1901.00596 (2019). http:\/\/arxiv.org\/abs\/1901.00596"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.14778\/3421424.3421432"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.14778\/3368289.3368294"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-012-0280-z"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.14778\/3461535.3461539"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3484224.3484233","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T09:39:11Z","timestamp":1672220351000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3484224.3484233"}},"subtitle":["deploying ML-for-systems without performance regressions, almost!"],"short-title":[],"issued":{"date-parts":[[2021,9]]},"references-count":48,"journal-issue":{"issue":"13","published-print":{"date-parts":[[2021,9]]}},"alternative-id":["10.14778\/3484224.3484233"],"URL":"https:\/\/doi.org\/10.14778\/3484224.3484233","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2021,9]]}}}