{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T10:11:24Z","timestamp":1782814284451,"version":"3.54.5"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"3","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62532001"],"award-info":[{"award-number":["62532001"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2026,5,18]]},"abstract":"<jats:p>Modern cloud data warehouses store data in micro-partitions and rely on metadata (e.g., zonemaps) for efficient data pruning during query processing. Maintaining data clustering in a large-scale table is crucial for effective data pruning. Existing automatic clustering approaches lack the flexibility required in dynamic cloud environments with continuous data ingestion and evolving workloads. This paper advocates a clean separation between reclustering policy and clustering-key selection. We introduce the concept of boundary micro-partitions that sit on the boundary of query ranges. We then present WAIR, a workload-aware algorithm to identify and recluster only boundary micro-partitions most critical for pruning efficiency. WAIR achieves near-optimal (with respect to fully sorted table layouts) query performance but incurs significantly lower reclustering cost with a theoretical upper bound. We further implement the algorithm into a prototype reclustering service and evaluate on standard benchmarks (TPC-H, DSB) and a real-world workload. Results show that WAIR improves query performance and reduces the overall cost compared to existing solutions.<\/jats:p>","DOI":"10.1145\/3802127","type":"journal-article","created":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T18:19:16Z","timestamp":1779128356000},"page":"1-27","source":"Crossref","is-referenced-by-count":1,"title":["Workload-Aware Incremental Reclustering in Cloud Data Warehouses"],"prefix":"10.1145","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-0429-3011","authenticated-orcid":false,"given":"Yipeng","family":"Liu","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0095-0626","authenticated-orcid":false,"given":"Renfei","family":"Zhou","sequence":"additional","affiliation":[{"name":"Carnegie Mellon University, Pittsburgh, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5297-7557","authenticated-orcid":false,"given":"Jiaqi","family":"Yan","sequence":"additional","affiliation":[{"name":"Snowflake Inc., San Carlos, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-4821-1558","authenticated-orcid":false,"given":"Huanchen","family":"Zhang","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,5,18]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2025. Amazon EC2 Pricing. https:\/\/aws.amazon.com\/ec2\/pricing."},{"key":"e_1_2_1_2_1","unstructured":"2025. Amazon Redshift. https:\/\/aws.amazon.com\/redshift\/."},{"key":"e_1_2_1_3_1","unstructured":"2025. Amazon Redshift Vacuuming Tables. https:\/\/docs.aws.amazon.com\/redshift\/latest\/dg\/t_Reclaiming_storage_sp ace202.html."},{"key":"e_1_2_1_4_1","unstructured":"2025. Amazon S3 pricing. https:\/\/aws.amazon.com\/s3\/pricing\/."},{"key":"e_1_2_1_5_1","unstructured":"2025. Amazon Web Services. https:\/\/aws.amazon.com."},{"key":"e_1_2_1_6_1","unstructured":"2025. Apache Arrow. https:\/\/arrow.apache.org."},{"key":"e_1_2_1_7_1","unstructured":"2025. Apache Iceberg. https:\/\/iceberg.apache.org."},{"key":"e_1_2_1_8_1","unstructured":"2025. Apache Iceberg In-Place Table Evolution. https:\/\/iceberg.apache.org\/docs\/latest\/evolution."},{"key":"e_1_2_1_9_1","unstructured":"2025. Apache ORC. https:\/\/orc.apache.org."},{"key":"e_1_2_1_10_1","unstructured":"2025. Apache Parquet. https:\/\/parquet.apache.org."},{"key":"e_1_2_1_11_1","unstructured":"2025. Databricks. https:\/\/www.databricks.com."},{"key":"e_1_2_1_12_1","unstructured":"2025. Databricks Automatic Liquid Clustering. https:\/\/www.databricks.com\/blog\/announcing-automatic-liquidclustering."},{"key":"e_1_2_1_13_1","unstructured":"2025. Databricks Liquid Clustering. https:\/\/www.databricks.com\/blog\/announcing-general-availability-liquidclustering."},{"key":"e_1_2_1_14_1","unstructured":"2025. Delta Lake. https:\/\/delta.io."},{"key":"e_1_2_1_15_1","unstructured":"2025. Delta Lake Liquid Clustering. https:\/\/github.com\/delta-io\/delta\/issues\/1874."},{"key":"e_1_2_1_16_1","unstructured":"2025. Delta Lake Liquid Clustering Design Doc. https:\/\/docs.google.com\/document\/d\/1FWR3odjOw4v4-hjFy_hVaNd xHVs4WuK1asfB6M6XEMw."},{"key":"e_1_2_1_17_1","unstructured":"2025. Distribution Advisor in Azure Synapse SQL. https:\/\/learn.microsoft.com\/en-us\/azure\/synapse-analytics\/sql\/dist ribution-advisor."},{"key":"e_1_2_1_18_1","unstructured":"2025. Dremio. https:\/\/www.dremio.com."},{"key":"e_1_2_1_19_1","unstructured":"2025. Dremio Apache Iceberg Clustering. https:\/\/www.dremio.com\/blog\/dremios-apache-iceberg-clustering-technicalblog\/."},{"key":"e_1_2_1_20_1","unstructured":"2025. Redis. https:\/\/redis.io."},{"key":"e_1_2_1_21_1","unstructured":"2025. Snowflake. https:\/\/www.snowflake.com."},{"key":"e_1_2_1_22_1","unstructured":"2025. Snowflake Clustering Keys and Clustered Tables. https:\/\/docs.snowflake.com\/en\/user-guide\/tables-clusteringkeys."},{"key":"e_1_2_1_23_1","unstructured":"2025. Tsinghua Open Source Mirror. https:\/\/mirrors.tuna.tsinghua.edu.cn."},{"key":"e_1_2_1_24_1","unstructured":"2025. Tsinghua Open Source Mirror Logs. https:\/\/mirrors.tuna.tsinghua.edu.cn\/logs\/neomirrors."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415545"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526045"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.14778\/3685800.3685825"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626246.3653397"},{"key":"e_1_2_1_29_1","unstructured":"T.H. Cormen C.E. Leiserson R.L. Rivest and C. Stein. 2022. Introduction to Algorithms fourth edition. MIT Press."},{"key":"e_1_2_1_30_1","unstructured":"The Transaction Processing Council. 2024. TPC-H Benchmark (Revision 4.0.0). (2024)."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2903741"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.14778\/3484224.3484234"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3626246.3653379"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457270"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2024.3397508"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.14778\/3611479.3611486"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.5555\/3001460.3001507"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-8176-4842-8_4"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2463708"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/373626.373678"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415568"},{"key":"e_1_2_1_42_1","volume-title":"VLDB'98, Proceedings of 24th International Conference on Very Large Data Bases","author":"Moerkotte Guido","year":"1998","unstructured":"Guido Moerkotte. 1998. Small Materialized Aggregates: A Light Weight Index Structure for Data Warehousing. In VLDB'98, Proceedings of 24th International Conference on Very Large Data Bases, August 24-27, 1998, New York City, New York, USA, Ashish Gupta, Oded Shmueli, and Jennifer Widom (Eds.). Morgan Kaufmann, 476-487. http:\/\/www.vldb.org\/conf\/1998\/p476.pdf"},{"key":"e_1_2_1_43_1","unstructured":"Guy M Morton. 1966. A computer oriented geodetic data base and a new technique in file sequencing. (1966)."},{"key":"e_1_2_1_44_1","volume-title":"DuckDB - A Modern Modular and Extensible Database System. In 1st International Workshop on Composable Data Management Systems, CDMS@VLDB 2022","author":"Raasveldt Mark","year":"2022","unstructured":"Mark Raasveldt. 2022. DuckDB - A Modern Modular and Extensible Database System. In 1st International Workshop on Composable Data Management Systems, CDMS@VLDB 2022, Sydney, Australia, September 9, 2022, Satyanarayana R. Valluri and Mohamed Za\u00eft (Eds.). https:\/\/cdmsworkshop.github.io\/2022\/Proceedings\/Keynotes\/Abstract_MarkRaasve ldt.pdf"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3320212"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-29193-8_10"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1109\/BIGDATA62323.2024.10825970"},{"key":"e_1_2_1_48_1","volume-title":"Building An Elastic Query Engine on Disaggregated Storage. In 17th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2020","author":"Vuppalapati Midhul","year":"2020","unstructured":"Midhul Vuppalapati, Justin Miron, Rachit Agarwal, Dan Truong, Ashish Motivala, and Thierry Cruanes. 2020. Building An Elastic Query Engine on Disaggregated Storage. In 17th USENIX Symposium on Networked Systems Design and Implementation, NSDI 2020, Santa Clara, CA, USA, February 25-27, 2020, Ranjita Bhagwan and George Porter (Eds.). USENIX Association, 449-462. https:\/\/www.usenix.org\/conference\/nsdi20\/presentation\/vuppalapati"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3389770"},{"key":"e_1_2_1_50_1","volume-title":"11th Conference on Innovative Data Systems Research, CIDR 2021, Virtual Event, January 11-15, 2021, Online Proceedings. www.cidrdb.org. http:\/\/cidrdb.org\/cidr2021\/papers\/cidr2021_paper17","author":"Zaharia Matei","year":"2021","unstructured":"Matei Zaharia, Ali Ghodsi, Reynold Xin, and Michael Armbrust. 2021. Lakehouse: A New Generation of Open Platforms that Unify Data Warehousing and Advanced Analytics. In 11th Conference on Innovative Data Systems Research, CIDR 2021, Virtual Event, January 11-15, 2021, Online Proceedings. www.cidrdb.org. http:\/\/cidrdb.org\/cidr2021\/papers\/cidr2021_paper17.pdf"},{"key":"e_1_2_1_51_1","doi-asserted-by":"crossref","first-page":"491","DOI":"10.14778\/3570690.3570699","article-title":"Dalton: Learned Partitioning for Distributed Data Streams","volume":"16","author":"Zapridou Eleni","year":"2022","unstructured":"Eleni Zapridou, Ioannis Mytilinis, and Anastasia Ailamaki. 2022. Dalton: Learned Partitioning for Distributed Data Streams. Proc. VLDB Endow. 16, 3 (2022), 491-504.","journal-title":"Proc. VLDB Endow."},{"key":"e_1_2_1_52_1","volume-title":"Cost-Intelligent Data Analytics in the Cloud. In 14th Conference on Innovative Data Systems Research, CIDR 2024","author":"Zhang Huanchen","year":"2024","unstructured":"Huanchen Zhang, Yihao Liu, and Jiaqi Yan. 2024. Cost-Intelligent Data Analytics in the Cloud. In 14th Conference on Innovative Data Systems Research, CIDR 2024, Chaminade, HI, USA, January 14-17, 2024. www.cidrdb.org. https:\/\/www.cidrdb.org\/cidr2024\/papers\/p78-zhang.pdf"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137765.3137769"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3722212.3724447"}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3802127","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,18]],"date-time":"2026-05-18T18:30:36Z","timestamp":1779129036000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3802127"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,18]]},"references-count":54,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2026,5,18]]}},"alternative-id":["10.1145\/3802127"],"URL":"https:\/\/doi.org\/10.1145\/3802127","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,18]]}}}