{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,1]],"date-time":"2026-06-01T20:39:06Z","timestamp":1780346346825,"version":"3.54.1"},"reference-count":56,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:p>Hosting database services on cloud systems has become a common practice. This has led to the increasing volume of database workloads, which provides the opportunity for pattern analysis. Discovering workload patterns from a business logic perspective is conducive to better understanding the trends and characteristics of the database system. However, existing workload pattern discovery systems are not suitable for large-scale cloud databases which are commonly employed by the industry. This is because the workload patterns of large-scale cloud databases are generally far more complicated than those of ordinary databases.<\/jats:p>\n          <jats:p>In this paper, we propose Alibaba Workload Miner (AWM), a real-time system for discovering workload patterns in complicated large-scale workloads. AW M encodes and discovers the SQL query patterns logged from user requests and optimizes the querying processing based on the discovered patterns. First, Data Collection &amp; Preprocessing Module collects streaming query logs and encodes them into high-dimensional feature embeddings with rich semantic contexts and execution features. Next, Online Workload Mining Module separates encoded query by business groups and discovers the workload patterns for each group. Meanwhile, Offline Training Module collects labels and trains the classification model using the labels. Finally, Pattern-based Optimizing Module optimizes query processing in cloud databases by exploiting discovered patterns. Extensive experimental results on one synthetic dataset and two real-life datasets (extracted from Alibaba Cloud databases) show that AW M enhances the accuracy of pattern discovery by 66% and reduce the latency of online inference by 22%, compared with the state-of-the-arts.<\/jats:p>","DOI":"10.14778\/3611540.3611557","type":"journal-article","created":{"date-parts":[[2023,9,15]],"date-time":"2023-09-15T11:32:37Z","timestamp":1694777557000},"page":"3689-3701","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Real-Time Workload Pattern Analysis for Large-Scale Cloud Databases"],"prefix":"10.14778","volume":"16","author":[{"given":"Jiaqi","family":"Wang","sequence":"first","affiliation":[{"name":"Zhejiang University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tianyi","family":"Li","sequence":"additional","affiliation":[{"name":"Aalborg University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anni","family":"Wang","sequence":"additional","affiliation":[{"name":"Alibaba Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoze","family":"Liu","sequence":"additional","affiliation":[{"name":"Purdue University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lu","family":"Chen","sequence":"additional","affiliation":[{"name":"Zhejiang University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jie","family":"Chen","sequence":"additional","affiliation":[{"name":"Alibaba Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jianye","family":"Liu","sequence":"additional","affiliation":[{"name":"Alibaba Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junyang","family":"Wu","sequence":"additional","affiliation":[{"name":"Zhejiang University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feifei","family":"Li","sequence":"additional","affiliation":[{"name":"Alibaba Group"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yunjun","family":"Gao","sequence":"additional","affiliation":[{"name":"Zhejiang University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,8]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Alibaba Cloud. 2022. Alibaba Cloud Databases. https:\/\/www.alibabacloud.com\/product\/databases"},{"key":"e_1_2_1_2_1","unstructured":"Amazon EC. 2015. Amazon web services. http:\/\/aws.amazon.com\/es\/ec2\/"},{"key":"e_1_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Wei Cao Xiaojie Feng Boyuan Liang Tianyu Zhang Yusong Gao Yunyang Zhang and Feifei Li. 2021. LogStore: A Cloud-Native and Multi-Tenant Log Database. In SIGMOD. 2464--2476.","DOI":"10.1145\/3448016.3457565"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-015-0395-0"},{"key":"e_1_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Tianqi Chen and Carlos Guestrin. 2016. XGBoost: A Scalable Tree Boosting System. In KDD. 785--794.","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_2_1_6_1","doi-asserted-by":"crossref","unstructured":"Alexis Conneau Kartikay Khandelwal Naman Goyal Vishrav Chaudhary Guillaume Wenzek Francisco Guzm\u00e1n \u00c9douard Grave Myle Ott Luke Zettlemoyer and Veselin Stoyanov. 2020. Unsupervised Cross-lingual Representation Learning at Scale. In ACL. 8440--8451.","DOI":"10.18653\/v1\/2020.acl-main.747"},{"key":"e_1_2_1_7_1","volume-title":"Microsoft Azure: planning, deploying, and managing your data center in the cloud","author":"Copeland Marshall","unstructured":"Marshall Copeland, Julian Soh, Anthony Puca, Mike Manning, and David Gollob. 2015. Microsoft Azure: planning, deploying, and managing your data center in the cloud. Apress."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.14778\/3352063.3352120"},{"key":"e_1_2_1_9_1","doi-asserted-by":"crossref","unstructured":"Sudipto Das Miroslav Grbic Igor Ilic Isidora Jovandic Andrija Jovanovic Vivek R. Narasayya Miodrag Radulovic Maja Stikic Gaoxiang Xu and Surajit Chaudhuri. 2019. Automatically Indexing Millions of Databases in Microsoft Azure SQL Database. In SIGMOD. 666--679.","DOI":"10.1145\/3299869.3314035"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.14778\/3430915.3430931"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687627.1687767"},{"key":"e_1_2_1_12_1","volume-title":"PsiDB: A framework for batched query processing and optimization","author":"Eslami Mehrad","unstructured":"Mehrad Eslami, Yicheng Tu, Hadi Charkhgard, Zichen Xu, and Jiacheng Liu. 2019. PsiDB: A framework for batched query processing and optimization. In IEEE BigData. 6046--6048."},{"key":"e_1_2_1_13_1","doi-asserted-by":"crossref","unstructured":"Yunjun Gao Xiaoze Liu Junyang Wu Tianyi Li Pengfei Wang and Lu Chen. 2022. ClusterEA: Scalable Entity Alignment with Stochastic Training and Normalized Mini-batch Similarities. In KDD. 421--431.","DOI":"10.1145\/3534678.3539331"},{"key":"e_1_2_1_14_1","doi-asserted-by":"crossref","unstructured":"Congcong Ge Xiaoze Liu Lu Chen Baihua Zheng and Yunjun Gao. 2021. Make It Easy: An Effective End-to-End Entity Alignment Framework. In SIGIR. 777--786.","DOI":"10.1145\/3404835.3462870"},{"key":"e_1_2_1_15_1","first-page":"237","article-title":"LargeEA: Aligning Entities for Large-scale Knowledge Graphs","volume":"15","author":"Ge Congcong","year":"2022","unstructured":"Congcong Ge, Xiaoze Liu, Lu Chen, Baihua Zheng, and Yunjun Gao. 2022. LargeEA: Aligning Entities for Large-scale Knowledge Graphs. PVLDB 15, 2 (2022), 237--245.","journal-title":"PVLDB"},{"key":"e_1_2_1_16_1","volume-title":"CollaborEM: A Self-supervised Entity Matching Framework Using Multi-features Collaboration. TKDE","author":"Ge Congcong","year":"2021","unstructured":"Congcong Ge, Pengfei Wang, Lu Chen, Xiaoze Liu, Baihua Zheng, and Yunjun Gao. 2021. CollaborEM: A Self-supervised Entity Matching Framework Using Multi-features Collaboration. TKDE (2021), 1--14."},{"key":"e_1_2_1_17_1","volume-title":"Mining frequent patterns in data streams at multiple time granularities. Next generation data mining 212","author":"Giannella Chris","year":"2003","unstructured":"Chris Giannella, Jiawei Han, Jian Pei, Xifeng Yan, and Philip S Yu. 2003. Mining frequent patterns in data streams at multiple time granularities. Next generation data mining 212 (2003), 191--212."},{"key":"e_1_2_1_18_1","doi-asserted-by":"crossref","unstructured":"Georgios Giannikis Darko Makreshanski Gustavo Alonso and Donald Kossmann. 2013. Workload optimization using shareddb. In SIGMOD. 1045--1048.","DOI":"10.1145\/2463676.2463678"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732279.2732280"},{"key":"e_1_2_1_20_1","volume-title":"The minimum description length principle","author":"Gr\u00fcnwald Peter D","unstructured":"Peter D Gr\u00fcnwald. 2007. The minimum description length principle. MIT press."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.14778\/3402707.3402746"},{"key":"e_1_2_1_22_1","unstructured":"Srinivasan Iyer Ioannis Konstas Alvin Cheung and Luke Zettlemoyer. 2016. Summarizing source code using a neural attention model. In ACL. 2073--2083."},{"key":"e_1_2_1_23_1","volume-title":"Query2Vec: An Evaluation of NLP Techniques for Generalized Workload Analytics. PVLDB 11, 5","author":"Jain Shrainik","year":"2018","unstructured":"Shrainik Jain, Bill Howe, Jiaqi Yan, and Thierry Cruanes. 2018. Query2Vec: An Evaluation of NLP Techniques for Generalized Workload Analytics. PVLDB 11, 5 (2018)."},{"key":"e_1_2_1_24_1","volume-title":"Frequent pattern mining in data streams. Data streams: Models and algorithms","author":"Jin Ruoming","year":"2007","unstructured":"Ruoming Jin and Gagan Agrawal. 2007. Frequent pattern mining in data streams. Data streams: Models and algorithms (2007), 61--84."},{"key":"e_1_2_1_25_1","volume-title":"Pocket data: The need for TPC-MOBILE","author":"Kennedy Oliver","unstructured":"Oliver Kennedy, Jerry Ajay, Geoffrey Challen, and Lukasz Ziarek. 2015. Pocket data: The need for TPC-MOBILE. In TPCTC. Springer, 8--25."},{"key":"e_1_2_1_26_1","volume-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL. 4171--4186.","author":"Ming-Wei Chang Jacob Devlin","year":"2019","unstructured":"Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In NAACL. 4171--4186."},{"key":"e_1_2_1_27_1","volume-title":"Building your next big thing with google cloud platform: A guide for developers and enterprise architects","author":"Krishnan S. P. T.","unstructured":"S. P. T. Krishnan and Jose L Ugia Gonzalez. 2015. Building your next big thing with google cloud platform: A guide for developers and enterprise architects. Springer."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2018.2831214"},{"key":"e_1_2_1_29_1","volume-title":"International conference on machine learning. PMLR, 1188--1196","author":"Le Quoc","year":"2014","unstructured":"Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In International conference on machine learning. PMLR, 1188--1196."},{"key":"e_1_2_1_30_1","first-page":"3028","article-title":"openGauss","volume":"14","author":"Li Guoliang","year":"2021","unstructured":"Guoliang Li, Xuanhe Zhou, Ji Sun, Xiang Yu, Yue Han, Lianyuan Jin, Wenbo Li, Tianqing Wang, and Shifu Li. 2021. openGauss: An Autonomous Database System. PVLDB 14, 12 (2021), 3028--3041.","journal-title":"An Autonomous Database System. PVLDB"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.14778\/3450980.3450987"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.14778\/3384345.3384353"},{"key":"e_1_2_1_33_1","doi-asserted-by":"crossref","unstructured":"Xiaoze Liu Junyang Wu Tianyi Li Lu Chen and Yunjun Gao. 2023. Unsupervised Entity Alignment for Temporal Knowledge Graphs. In WWW. 2528--2538.","DOI":"10.1145\/3543507.3583381"},{"key":"e_1_2_1_34_1","doi-asserted-by":"crossref","unstructured":"Xiaoze Liu Zheng Yin Chao Zhao Congcong Ge Lu Chen Yunjun Gao Dimeng Li Ziting Wang Gaozhong Liang Jian Tan and Feifei Li. 2022. PinSQL: Pinpoint Root Cause SQLs to Resolve Performance Issues in Cloud Databases. In ICDE. 2549--2561.","DOI":"10.1109\/ICDE53745.2022.00236"},{"key":"e_1_2_1_35_1","volume-title":"Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)."},{"key":"e_1_2_1_36_1","volume-title":"A survey of visual transformers. TNNLS","author":"Liu Yang","year":"2023","unstructured":"Yang Liu, Yao Zhang, Yixin Wang, Feng Hou, Jin Yuan, Jiang Tian, Yang Zhang, Zhongchao Shi, Jianping Fan, and Zhiqiang He. 2023. A survey of visual transformers. TNNLS (2023), 1--21."},{"key":"e_1_2_1_37_1","volume-title":"Ahmed Hefny, Gustavo Mezerhane, Andrew Pavlo, and Geoffrey J. Gordon.","author":"Ma Lin","year":"2018","unstructured":"Lin Ma, Dana Van Aken, Ahmed Hefny, Gustavo Mezerhane, Andrew Pavlo, and Geoffrey J. Gordon. 2018. Query-based Workload Forecasting for Self-Driving Database Management Systems. In SIGMOD. 631--645."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14778\/3389133.3389136"},{"key":"e_1_2_1_39_1","volume-title":"Bao: Making Learned Query Optimization Practical. In SIGMOD. 1275--1288.","author":"Marcus Ryan","year":"2021","unstructured":"Ryan Marcus, Parimarjan Negi, Hongzi Mao, Nesime Tatbul, Mohammad Alizadeh, and Tim Kraska. 2021. Bao: Making Learned Query Optimization Practical. In SIGMOD. 1275--1288."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.14778\/2977797.2977804"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.14778\/3342263.3342644"},{"key":"e_1_2_1_42_1","volume-title":"Alekh Jindal, and Samuel Madden.","author":"Mozafari Barzan","year":"2013","unstructured":"Barzan Mozafari, Carlo Curino, Alekh Jindal, and Samuel Madden. 2013. Performance and resource modeling in highly-concurrent OLTP workloads. In SIGMOD. 301--312."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.14778\/3503585.3503600"},{"key":"e_1_2_1_44_1","first-page":"1","article-title":"Self-Driving Database Management Systems","volume":"4","author":"Pavlo Andrew","year":"2017","unstructured":"Andrew Pavlo, Gustavo Angulo, Joy Arulraj, Haibin Lin, Jiexi Lin, Lin Ma, Prashanth Menon, Todd C Mowry, Matthew Perron, Ian Quah, et al. 2017. Self-Driving Database Management Systems. In CIDR, Vol. 4. 1.","journal-title":"CIDR"},{"key":"e_1_2_1_45_1","volume-title":"Carlo Curino, and Raghu Ramakrishnan","author":"Psallidas Fotis","year":"2022","unstructured":"Fotis Psallidas, Ashvin Agrawal, Chandru Sugunan, Khaled Ibrahim, Konstantinos Karanasos, Jes\u00fas Camacho-Rodr\u00edguez, Avrilia Floratou, Carlo Curino, and Raghu Ramakrishnan. 2022. OneProvenance: Efficient Extraction of Dynamic Coarse-Grained Provenance from Database Logs. arXiv preprint arXiv:2210.14047 (2022)."},{"key":"e_1_2_1_46_1","doi-asserted-by":"crossref","unstructured":"Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In EMNLP. 3980--3990.","DOI":"10.18653\/v1\/D19-1410"},{"key":"e_1_2_1_47_1","volume-title":"RESTful web services. \" O'Reilly Media","author":"Richardson Leonard","unstructured":"Leonard Richardson and Sam Ruby. 2008. RESTful web services. \" O'Reilly Media, Inc.\"."},{"key":"e_1_2_1_48_1","doi-asserted-by":"crossref","unstructured":"Xiu Tang Sai Wu Mingli Song Shanshan Ying Feifei Li and Gang Chen. 2022. PreQR: Pre-training Representation for SQL Understanding. In SIGMOD. 204--216.","DOI":"10.1145\/3514221.3517878"},{"key":"e_1_2_1_49_1","doi-asserted-by":"crossref","unstructured":"Quoc Trung Tran Konstantinos Morfonios and Neoklis Polyzotis. 2015. Oracle Workload Intelligence. In SIGMOD. 1669--1681.","DOI":"10.1145\/2723372.2742791"},{"key":"e_1_2_1_50_1","doi-asserted-by":"crossref","unstructured":"Alexandre Verbitski Anurag Gupta Debanjan Saha Murali Brahmadesam Kamal Gupta Raman Mittal Sailesh Krishnamurthy Sandor Maurice Tengiz Kharatishvili and Xiaofeng Bao. 2017. Amazon aurora: Design considerations for high throughput cloud-native relational databases. In SIGMOD. 1041--1052.","DOI":"10.1145\/3035918.3056101"},{"key":"e_1_2_1_51_1","volume-title":"SEA: A Scalable Entity Alignment System. arXiv preprint arXiv:2304.07065","author":"Wu Junyang","year":"2023","unstructured":"Junyang Wu, Tianyi Li, Lu Chen, Yunjun Gao, and Ziheng Wei. 2023. SEA: A Scalable Entity Alignment System. arXiv preprint arXiv:2304.07065 (2023)."},{"key":"e_1_2_1_52_1","doi-asserted-by":"crossref","unstructured":"Dong Young Yoon Ning Niu and Barzan Mozafari. 2016. DBSherlock: A Performance Diagnostic Tool for Transactional Databases. In SIGMOD. 1599--1614.","DOI":"10.1145\/2882903.2915218"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.14778\/3485450.3485456"},{"key":"e_1_2_1_54_1","unstructured":"Rong Zhu Ziniu Wu Chengliang Chai Andreas Pfadler Bolin Ding Guoliang Li and Jingren Zhou. 2022. Learned Query Optimizer: At the Forefront of AI-Driven Databases. In EDBT. 1--4."},{"key":"e_1_2_1_55_1","volume-title":"Abhishek Modi, Manoj Kumar, Deli Zhang, Kartheek Muthyala, Nick Jurgens, et al.","author":"Zhu Yiwen","year":"2021","unstructured":"Yiwen Zhu, Subru Krishnan, Konstantinos Karanasos, Isha Tarte, Conor Power, Abhishek Modi, Manoj Kumar, Deli Zhang, Kartheek Muthyala, Nick Jurgens, et al. 2021. Kea: Tuning an exabyte-scale data infrastructure. In SIGMOD. 2667--2680."},{"key":"e_1_2_1_56_1","doi-asserted-by":"crossref","unstructured":"Zainab Zolaktaf Mostafa Milani and Rachel Pottinger. 2020. Facilitating SQL query composition and analysis. In SIGMOD. 209--224.","DOI":"10.1145\/3318464.3380602"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3611540.3611557","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,10]],"date-time":"2025-09-10T22:32:37Z","timestamp":1757543557000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3611540.3611557"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8]]},"references-count":56,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["10.14778\/3611540.3611557"],"URL":"https:\/\/doi.org\/10.14778\/3611540.3611557","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2023,8]]},"assertion":[{"value":"2023-08-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}