{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,19]],"date-time":"2026-05-19T07:13:25Z","timestamp":1779174805409,"version":"3.51.4"},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,12,8]],"date-time":"2023-12-08T00:00:00Z","timestamp":1701993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2023,12,8]]},"abstract":"<jats:p>Analytical query workloads are prone to rapid fluctuations in resource demands. These rapid, hard to predict resource demand changes make provisioning a challenge. Users must either over provision at excessive cost or suffer poor query latency when demand spikes. Prior work shows the viability of using cloud functions to match the supply of compute to the workload demand without provisioning resources ahead of time. For low query volumes, this approach is less costly at reasonable performance compared to provisioned systems, but as query volumes increase the cost overhead of cloud functions outweighs the benefit gained by rapid elasticity. In this work, we propose a novel strategy combining rapidly scalable but expensive resources with slow to start but inexpensive virtual machines to gain the benefit of elasticity without losing out on the cost savings of provisioned resources. We demonstrate a technique that minimizes cost over a wide range of workloads, environmental conditions, and compute costs while providing stable query performance. We implement these ideas in Cackle and demonstrate that it achieves similar performance and cost per query across a wide range of workloads, avoiding the cost and performance cliffs of alternative approaches.<\/jats:p>","DOI":"10.1145\/3626720","type":"journal-article","created":{"date-parts":[[2023,12,12]],"date-time":"2023-12-12T14:01:21Z","timestamp":1702389681000},"page":"1-25","source":"Crossref","is-referenced-by-count":5,"title":["Cackle: Analytical Workload Cost and Performance Stability With Elastic Pools"],"prefix":"10.1145","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2879-977X","authenticated-orcid":false,"given":"Matthew","family":"Perron","sequence":"first","affiliation":[{"name":"MIT CSAIL, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7675-6080","authenticated-orcid":false,"given":"Raul","family":"Castro Fernandez","sequence":"additional","affiliation":[{"name":"University of Chicago, Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-5037-5205","authenticated-orcid":false,"given":"David","family":"DeWitt","sequence":"additional","affiliation":[{"name":"MIT CSAIL, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6122-0590","authenticated-orcid":false,"given":"Michael","family":"Cafarella","sequence":"additional","affiliation":[{"name":"MIT CSAIL, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7470-3265","authenticated-orcid":false,"given":"Samuel","family":"Madden","sequence":"additional","affiliation":[{"name":"MIT CSAIL, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,12,12]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415545"},{"key":"e_1_2_1_2_1","unstructured":"Alibaba Cluster Trace Program - Cluster Trace v2018 2018. Alibaba Cluster Trace Program - Cluster Trace v2018. Github. Posted at https:\/\/github.com\/alibaba\/clusterdata\/blob\/master\/cluster-trace-v2018\/trace_2018.md.."},{"key":"e_1_2_1_3_1","unstructured":"amazon ec2 spot instances pricing 2023. amazon ec2 spot instances pricing. https:\/\/aws.amazon.com\/ec2\/spot\/pricing\/."},{"key":"e_1_2_1_4_1","unstructured":"amazon redshift serverless 2023. amazon redshift serverless. https:\/\/aws.amazon.com\/blogs\/aws\/introducing-amazon-redshift-serverless-run-analytics-at-any-scale-without-having-to-manage-infrastructure\/."},{"key":"e_1_2_1_5_1","unstructured":"Amazon S3 [n. d.]. Amazon S3. https:\/\/aws.amazon.com\/s3\/."},{"key":"e_1_2_1_6_1","unstructured":"Amazon Web Services 2023. Amazon Web Services. https:\/\/aws.amazon.com\/."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526045"},{"key":"e_1_2_1_8_1","volume-title":"The multiplicative weights update method: a meta-algorithm and applications. Theory of computing 8, 1","author":"Arora Sanjeev","year":"2012","unstructured":"Sanjeev Arora, Elad Hazan, and Satyen Kale. 2012. The multiplicative weights update method: a meta-algorithm and applications. Theory of computing 8, 1 (2012), 121--164."},{"key":"e_1_2_1_9_1","unstructured":"AWS Lambda 2023. AWS Lambda. https:\/\/aws.amazon.com\/lambda\/."},{"key":"e_1_2_1_10_1","unstructured":"Azure Blob Storage 2023. Azure Blob Storage. https:\/\/azure.microsoft.com\/en-us\/products\/storage\/blobs\/."},{"key":"e_1_2_1_11_1","unstructured":"Azure Synapse Analytics 2023. Azure Synapse Analytics. https:\/\/azure.microsoft.com\/en-us\/products\/synapse-analytics\/."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3589306"},{"key":"e_1_2_1_13_1","first-page":"82","article-title":"Integrated Querying of SQL database data and S3 data in Amazon Redshift","volume":"41","author":"Cai Mengchu","year":"2018","unstructured":"Mengchu Cai, Martin Grund, Anurag Gupta, Fabian Nagel, Ippokratis Pandis, Yannis Papakonstantinou, and Michalis Petropoulos. 2018. Integrated Querying of SQL database data and S3 data in Amazon Redshift. IEEE Data Eng. Bull. 41, 2 (2018), 82--90. http:\/\/sites.computer.org\/debull\/A18june\/p82.pdf","journal-title":"IEEE Data Eng. Bull."},{"key":"e_1_2_1_14_1","volume-title":"TPC-H benchmark specification. Published at http:\/\/www.tcp.org\/hspec. html 21","author":"Transaction Processing Performance Council","year":"2008","unstructured":"Transaction Processing Performance Council. 2008. TPC-H benchmark specification. Published at http:\/\/www.tcp.org\/hspec. html 21 (2008), 592--603."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2903741"},{"key":"e_1_2_1_16_1","unstructured":"databricks serverless 2023. databricks serverless. https:\/\/docs.databricks.com\/en\/serverless-compute\/index.html."},{"key":"e_1_2_1_17_1","unstructured":"databricks sql 2023. databricks sql. https:\/\/www.databricks.com\/product\/databricks-sql."},{"key":"e_1_2_1_18_1","unstructured":"Databricks SQL Documentation 2023. Queueing and autoscaling. Posted at https:\/\/docs.databricks.com\/sql\/admin\/create-sql-warehouse.html#queueing-and-autoscaling.."},{"key":"e_1_2_1_19_1","unstructured":"google cloud functions 2023. google cloud functions. https:\/\/cloud.google.com\/functions."},{"key":"e_1_2_1_20_1","unstructured":"Google Cloud Storage 2023. Google Cloud Storage. https:\/\/cloud.google.com\/storage."},{"key":"e_1_2_1_21_1","unstructured":"gRPC 2023. gRPC. https:\/\/grpc.io\/."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742795"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/3492321.3527539"},{"key":"e_1_2_1_24_1","volume-title":"13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18)","author":"Klimovic Ana","year":"2018","unstructured":"Ana Klimovic, Yawen Wang, Patrick Stuedi, Animesh Trivedi, Jonas Pfefferle, and Christos Kozyrakis. 2018. Pocket: Elastic ephemeral storage for serverless analytics. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). 427--444."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3542929.3563483"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389758"},{"key":"e_1_2_1_27_1","first-page":"1049","article-title":"The Making of TPC-DS","volume":"6","author":"Nambiar Raghunath Othayoth","year":"2006","unstructured":"Raghunath Othayoth Nambiar and Meikel Poess. 2006. The Making of TPC-DS.. In VLDB, Vol. 6. 1049--1058.","journal-title":"VLDB"},{"key":"e_1_2_1_28_1","unstructured":"orc specification 2023. orc specification. https:\/\/orc.apache.org\/specification\/."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476311.3476391"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3380609"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/3429357.3430522"},{"key":"e_1_2_1_32_1","volume-title":"Multi-cluster Warehouses","author":"Documentation Snowflake","year":"2023","unstructured":"Snowflake Documentation: Multi-cluster Warehouses 2023. Multi-cluster Warehouses. Snowflake Documentation. Posted at https:\/\/docs.snowflake.com\/en\/user-guide\/warehouses-multicluster#label-mcw-scaling-policies.."},{"key":"e_1_2_1_33_1","unstructured":"Tableau 2023. Tableau. https:\/\/www.tableau.com\/."},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3190650"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/2934664"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626720","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3626720","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,22]],"date-time":"2025-08-22T13:01:54Z","timestamp":1755867714000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626720"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,8]]},"references-count":35,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,12,8]]}},"alternative-id":["10.1145\/3626720"],"URL":"https:\/\/doi.org\/10.1145\/3626720","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,8]]}}}