{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T03:06:04Z","timestamp":1787022364939,"version":"build-2736575974"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"3","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2018,11]]},"abstract":"<jats:p>\n                    Query optimizers are notorious for inaccurate cost estimates, leading to poor performance. The root of the problem lies in inaccurate cardinality estimates, i.e., the size of intermediate (and final) results in a query plan. These estimates also determine the resources consumed in modern shared cloud infrastructures. In this paper, we present C\n                    <jats:sc>ARD<\/jats:sc>\n                    L\n                    <jats:sc>EARNER<\/jats:sc>\n                    , a machine learning based approach to learn cardinality models from previous job executions and use them to predict the cardinalities in future jobs. The key intuition in our approach is that shared cloud workloads are often recurring and overlapping in nature, and so we could learn cardinality models for overlapping subgraph templates. We discuss various learning approaches and show how learning a large number of smaller models results in high accuracy and explainability. We further present an exploration technique to avoid learning bias by considering alternate join orders and learning cardinality models over them. We describe the feedback loop to apply the learned models back to future job executions. Finally, we show a detailed evaluation of our models (up to 5 orders of magnitude less error), query plans (60% applicability), performance (up to 100% faster, 3x fewer resources), and exploration (optimal in few 10s of executions).\n                  <\/jats:p>","DOI":"10.14778\/3291264.3291267","type":"journal-article","created":{"date-parts":[[2019,2,4]],"date-time":"2019-02-04T08:13:43Z","timestamp":1549268023000},"page":"210-222","source":"Crossref","is-referenced-by-count":68,"title":["Towards a learning optimizer for shared clouds"],"prefix":"10.14778","volume":"12","author":[{"given":"Chenggang","family":"Wu","sequence":"first","affiliation":[{"name":"University of California, Berkeley"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alekh","family":"Jindal","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Saeed","family":"Amizadeh","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hiren","family":"Patel","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wangchao","family":"Le","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shi","family":"Qiao","sequence":"additional","affiliation":[{"name":"Microsoft"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sriram","family":"Rao","sequence":"additional","affiliation":[{"name":"Facebook"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2018,11]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"SIGMOD (to appear)","author":"Reinforcement Learning Predictive Indexing","year":"2018"},{"key":"e_1_2_1_2_1","first-page":"21","volume-title":"NSDI","author":"Agarwal S.","year":"2012"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2465351.2465355"},{"key":"e_1_2_1_4_1","unstructured":"Asimov System. https:\/\/mywindowshub.com\/microsoft-uses-real-time-telemetry-asimov-build-test-update-windows-9\/. Asimov System. https:\/\/mywindowshub.com\/microsoft-uses-real-time-telemetry-asimov-build-test-update-windows-9\/."},{"key":"e_1_2_1_5_1","unstructured":"Amazon Athena. https:\/\/aws.amazon.com\/athena\/. Amazon Athena. https:\/\/aws.amazon.com\/athena\/."},{"key":"e_1_2_1_6_1","unstructured":"Google BigQuery. https:\/\/cloud.google.com\/bigquery. Google BigQuery. https:\/\/cloud.google.com\/bigquery."},{"key":"e_1_2_1_7_1","volume-title":"OSDI","author":"Boutin E.","year":"2014"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536222.2536223"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.14778\/1454159.1454166"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.14778\/1453856.1453977"},{"key":"e_1_2_1_11_1","unstructured":"Azure Data Lake. https:\/\/azure.microsoft.com\/en-us\/solutions\/data-lake\/. Azure Data Lake. https:\/\/azure.microsoft.com\/en-us\/solutions\/data-lake\/."},{"key":"e_1_2_1_12_1","unstructured":"Adaptive Query Processing in DB2. https:\/\/www.ibm.com\/support\/knowledgecenter\/en\/ssw_ibm_i_73\/rzajq\/rzajqAQP.htm. Adaptive Query Processing in DB2. https:\/\/www.ibm.com\/support\/knowledgecenter\/en\/ssw_ibm_i_73\/rzajq\/rzajqAQP.htm."},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1037\/0033-2909.118.3.392"},{"key":"e_1_2_1_14_1","unstructured":"I. Goodfellow Y. Bengio A. Courville and Y. Bengio. Deep learning volume 1. MIT press Cambridge 2016. I. Goodfellow Y. Bengio A. Courville and Y. Bengio. Deep learning volume 1. MIT press Cambridge 2016."},{"key":"e_1_2_1_15_1","first-page":"19","volume-title":"IEEE Data Engineering Bulletin","volume":"18","author":"Graefe G.","year":"1995"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/645478.757691"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2304510.2304525"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/115790.115835"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/304182.304209"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.14778\/3192965.3192971"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3190656"},{"key":"e_1_2_1_22_1","first-page":"117","volume-title":"OSDI","author":"Jyothi S. A.","year":"2016"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2882940"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2610531"},{"key":"e_1_2_1_25_1","doi-asserted-by":"crossref","unstructured":"T. Kraska A. Beutel E. H. Chi J. Dean and N. Polyzotis. The case for learned index structures. arXiv preprint arXiv:1712.01208v2 2017. T. Kraska A. Beutel E. H. Chi J. Dean and N. Polyzotis. The case for learned index structures. arXiv preprint arXiv:1712.01208v2 2017.","DOI":"10.1145\/3183713.3196909"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.14778\/2850583.2850594"},{"key":"e_1_2_1_27_1","unstructured":"SIGMOD Blog. http:\/\/wp.sigmod.org\/?p=1075. SIGMOD Blog. http:\/\/wp.sigmod.org\/?p=1075."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007568.1007642"},{"key":"e_1_2_1_29_1","volume-title":"Inc.","author":"Martello S.","year":"1990"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1016\/0377-2217(84)90282-0"},{"key":"e_1_2_1_31_1","volume-title":"Irwin Chicago","author":"Neter J.","year":"1996"},{"key":"e_1_2_1_32_1","unstructured":"Optimizer Adaptive Features in Oracle. https:\/\/blogs.oracle.com\/optimizer\/optimizer-adaptive-features-in-oracle-database-12c-release-2. Optimizer Adaptive Features in Oracle. https:\/\/blogs.oracle.com\/optimizer\/optimizer-adaptive-features-in-oracle-database-12c-release-2."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2987550.2987566"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3056100"},{"key":"e_1_2_1_35_1","unstructured":"Selectivity Guesses. https:\/\/www.sqlskills.com\/blogs\/joe\/selectivity-guesses-in-absence-of-statistics\/. Selectivity Guesses. https:\/\/www.sqlskills.com\/blogs\/joe\/selectivity-guesses-in-absence-of-statistics\/."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/582095.582099"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3127479.3131613"},{"key":"e_1_2_1_38_1","unstructured":"Adaptive Query Processing in SQL Server. https:\/\/docs.microsoft.com\/en-us\/sql\/relational-databases\/performance\/adaptive-query-processing. Adaptive Query Processing in SQL Server. https:\/\/docs.microsoft.com\/en-us\/sql\/relational-databases\/performance\/adaptive-query-processing."},{"key":"e_1_2_1_39_1","first-page":"19","volume-title":"PVLDB","author":"Stillger M.","year":"2001"},{"key":"e_1_2_1_40_1","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1111\/j.2517-6161.1974.tb00994.x","article-title":"Cross-validatory choice and assessment of statistical predictions","volume":"36","author":"Stone M.","year":"1974","journal-title":"Roy. Stat. Soc."},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2588555.2610515"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687553.1687609"},{"key":"e_1_2_1_43_1","unstructured":"TPC-H Benchmark. http:\/\/www.tpc.org\/tpch. TPC-H Benchmark. http:\/\/www.tpc.org\/tpch."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2882927"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3064029"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.14778\/3007328.3007332"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-012-0280-z"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3291264.3291267","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,14]],"date-time":"2024-07-14T10:40:34Z","timestamp":1720953634000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3291264.3291267"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,11]]},"references-count":47,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2018,11]]}},"alternative-id":["10.14778\/3291264.3291267"],"URL":"https:\/\/doi.org\/10.14778\/3291264.3291267","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2018,11]]}}}