{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,11]],"date-time":"2025-09-11T22:51:52Z","timestamp":1757631112446,"version":"3.44.0"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:p>\n            The surge in data analytics has fostered burgeoning demand for\n            <jats:italic toggle=\"yes\">AnalyticDB<\/jats:italic>\n            on Alibaba Cloud, which has well served thousands of customers from various business sectors. The most notable feature is the diversity of the workloads it handles, including batch processing, real-time data analytics, and unstructured data analytics. To improve the overall performance for such diverse workloads, one of the major challenges is to optimize long-running complex queries without sacrificing the processing efficiency of short-running interactive queries. While existing methods attempt to utilize runtime dynamic statistics for adaptive query processing, they often focus on specific scenarios instead of providing a holistic solution.\n          <\/jats:p>\n          <jats:p>\n            To address this challenge, we propose a new framework called\n            <jats:italic toggle=\"yes\">Anser<\/jats:italic>\n            , which enhances the design of traditional distributed data warehouses by embedding a new information sharing mechanism. This allows for the efficient management of the production and consumption of various dynamic information across the system. Building on top of\n            <jats:italic toggle=\"yes\">Anser<\/jats:italic>\n            , we introduce a novel scheduling policy that optimizes both data and information exchanges within the physical plan, enabling the acceleration of complex analytical queries without sacrificing the performance of short-running interactive queries. We conduct comprehensive experiments over public and in-house workloads to demonstrate the effectiveness and efficiency of our proposed information sharing framework.\n          <\/jats:p>","DOI":"10.14778\/3611540.3611553","type":"journal-article","created":{"date-parts":[[2023,9,15]],"date-time":"2023-09-15T11:32:37Z","timestamp":1694777557000},"page":"3636-3648","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Anser: Adaptive Information Sharing Framework of AnalyticDB"],"prefix":"10.14778","volume":"16","author":[{"given":"Liang","family":"Lin","sequence":"first","affiliation":[{"name":"Alibaba Group, Hangzhou, China"}]},{"given":"Yuhan","family":"Li","sequence":"additional","affiliation":[{"name":"Alibaba Group, Hangzhou, China"}]},{"given":"Bin","family":"Wu","sequence":"additional","affiliation":[{"name":"Alibaba Group, Hangzhou, China"}]},{"given":"Huijun","family":"Mai","sequence":"additional","affiliation":[{"name":"Alibaba Group, Hangzhou, China"}]},{"given":"Renjie","family":"Lou","sequence":"additional","affiliation":[{"name":"Alibaba Group, Hangzhou, China"}]},{"given":"Jian","family":"Tan","sequence":"additional","affiliation":[{"name":"Alibaba Group, Hangzhou, China"}]},{"given":"Feifei","family":"Li","sequence":"additional","affiliation":[{"name":"Alibaba Group, Hangzhou, China"}]}],"member":"320","published-online":{"date-parts":[[2023,8]]},"reference":[{"unstructured":"[n. d.]. Apache Hive. https:\/\/hive.apache.org\/. Last accessed 2023-03-01.","key":"e_1_2_1_1_1"},{"unstructured":"[n. d.]. Apache Kafka. https:\/\/kafka.apache.org\/. Last accessed 2023-03-01.","key":"e_1_2_1_2_1"},{"unstructured":"[n. d.]. Elastic Compute Service. https:\/\/www.alibabacloud.com\/product\/ecs.Last accessed 2023-03-01.","key":"e_1_2_1_3_1"},{"unstructured":"[n. d.]. HDFS Architecture Guide. https:\/\/hadoop.apache.org\/docs\/r1.2.1\/hdfs_design.html. Last accessed 2023-03-01.","key":"e_1_2_1_4_1"},{"unstructured":"[n. d.]. Impala Runtime Filtering. https:\/\/impala.apache.org\/docs\/build\/html\/topics\/impala_runtime_filtering.html. Last accessed 2023-03-01.","key":"e_1_2_1_5_1"},{"unstructured":"[n. d.]. Object Storage Service (OSS). https:\/\/www.alibabacloud.com\/product\/object-storage-service?spm=a3c0i.23458820.2359477120.2.26a77d3fagA3sE. Last accessed 2023-03-01.","key":"e_1_2_1_6_1"},{"unstructured":"[n. d.]. Parameter Sensitive Plan optimization. https:\/\/learn.microsoft.com\/en-us\/sql\/relational-databases\/performance\/parameter-sensitivity-plan-optimization?view=sql-server-ver16. Last accessed 2023-03-01.","key":"e_1_2_1_7_1"},{"doi-asserted-by":"publisher","key":"e_1_2_1_8_1","DOI":"10.1145\/2465351.2465355"},{"doi-asserted-by":"publisher","key":"e_1_2_1_9_1","DOI":"10.1145\/2723372.2742797"},{"doi-asserted-by":"publisher","key":"e_1_2_1_10_1","DOI":"10.1145\/3514221.3526045"},{"doi-asserted-by":"publisher","key":"e_1_2_1_11_1","DOI":"10.1145\/342009.335420"},{"doi-asserted-by":"publisher","key":"e_1_2_1_12_1","DOI":"10.1145\/1066157.1066171"},{"doi-asserted-by":"publisher","key":"e_1_2_1_13_1","DOI":"10.14778\/2536222.2536235"},{"doi-asserted-by":"publisher","key":"e_1_2_1_14_1","DOI":"10.14778\/3352063.3352124"},{"key":"e_1_2_1_15_1","first-page":"12","article-title":"Including group-by in query optimization","volume":"94","author":"Chaudhuri Surajit","year":"1994","unstructured":"Surajit Chaudhuri and Kyuseok Shim. 1994. Including group-by in query optimization. In VLDB, Vol. 94. 12--15.","journal-title":"VLDB"},{"doi-asserted-by":"publisher","key":"e_1_2_1_16_1","DOI":"10.1007\/s007780050036"},{"doi-asserted-by":"publisher","key":"e_1_2_1_17_1","DOI":"10.1007\/s007780050036"},{"doi-asserted-by":"publisher","key":"e_1_2_1_18_1","DOI":"10.1145\/974121.974129"},{"doi-asserted-by":"crossref","unstructured":"Amol Deshpande Joseph M Hellerstein et al. 2004. Lifting the burden of history from adaptive query processing. In VLDB. Citeseer 948--959.","key":"e_1_2_1_19_1","DOI":"10.1016\/B978-012088469-8.50083-8"},{"doi-asserted-by":"crossref","unstructured":"Amol Deshpande Joseph M Hellerstein and Vijayshankar Raman. 2006. Adaptive query processing: why how when what next. (2006) 806--807.","key":"e_1_2_1_20_1","DOI":"10.1145\/1142473.1142603"},{"doi-asserted-by":"publisher","key":"e_1_2_1_21_1","DOI":"10.1145\/3448016.3457270"},{"key":"e_1_2_1_22_1","volume-title":"1989 ACM SIGMOD international conference on Management of data)","author":"DeWitt David J.","year":"1989","unstructured":"David J. DeWitt Donovan A. Schneider. 1989. A Performance Evaluation of Four Parallel Join Algorithms in a Shared-Nothing Multiprocessor Environment. 1989 ACM SIGMOD international conference on Management of data) (1989), 110--121."},{"doi-asserted-by":"publisher","key":"e_1_2_1_23_1","DOI":"10.1145\/1247480.1247598"},{"key":"e_1_2_1_24_1","first-page":"19","article-title":"The cascades framework for query optimization","volume":"18","author":"Graefe Goetz","year":"1995","unstructured":"Goetz Graefe. 1995. The cascades framework for query optimization. IEEE Data Eng. Bull. 18, 3 (1995), 19--29.","journal-title":"IEEE Data Eng. Bull."},{"doi-asserted-by":"publisher","key":"e_1_2_1_25_1","DOI":"10.1145\/67544.66960"},{"doi-asserted-by":"publisher","key":"e_1_2_1_26_1","DOI":"10.1145\/2723372.2742795"},{"key":"e_1_2_1_27_1","volume-title":"VLDB","volume":"95","author":"Gupta Ashish","year":"1995","unstructured":"Ashish Gupta, Venky Harinarayan, and Dallan Quass. 1995. Aggregate-query processing in data warehousing environments. In VLDB, Vol. 95. Citeseer, 358--369."},{"key":"e_1_2_1_28_1","volume-title":"1","author":"Hellerstein Joseph M","year":"2007","unstructured":"Joseph M Hellerstein, Peter J Haas, and Helen J Wang. 2007. 2007 Test-of-time Award \"Online Aggregation\". (2007), 1."},{"doi-asserted-by":"publisher","key":"e_1_2_1_29_1","DOI":"10.1007\/s007780050037"},{"key":"e_1_2_1_30_1","volume-title":"Sideways Information Passing for Push-Style Query Processing. 2008 IEEE 24th International Conference on Data Engineering","author":"Zachary","year":"2008","unstructured":"Zachary G. Ives and Nicholas E. Taylor. 2008. Sideways Information Passing for Push-Style Query Processing. 2008 IEEE 24th International Conference on Data Engineering (2008), 774--783."},{"doi-asserted-by":"publisher","key":"e_1_2_1_31_1","DOI":"10.1145\/356924.356928"},{"doi-asserted-by":"publisher","key":"e_1_2_1_32_1","DOI":"10.1145\/276304.276315"},{"doi-asserted-by":"publisher","key":"e_1_2_1_33_1","DOI":"10.1109\/ICDE.2002.994787"},{"doi-asserted-by":"publisher","key":"e_1_2_1_34_1","DOI":"10.14778\/1454159.1454178"},{"doi-asserted-by":"publisher","key":"e_1_2_1_35_1","DOI":"10.1007\/s41019-018-0074-4"},{"key":"e_1_2_1_36_1","first-page":"149","article-title":"R* Optimizer Validation and Performance Evaluation","volume":"149","author":"Mackert Lothar F","year":"1986","unstructured":"Lothar F Mackert and Guy M Lohman. 1986. R* Optimizer Validation and Performance Evaluation. Very Large Data Bases: Proceedings 149 (1986), 149.","journal-title":"Very Large Data Bases: Proceedings"},{"doi-asserted-by":"publisher","key":"e_1_2_1_37_1","DOI":"10.14778\/3503585.3503601"},{"key":"e_1_2_1_38_1","volume-title":"Accelerating Feature Engineering with Adaptive Partial Aggregation Tree. 2018 IEEE International Conference on Big Data (Big Data)","author":"Oyamada M.","year":"2018","unstructured":"M. Oyamada. 2018. Accelerating Feature Engineering with Adaptive Partial Aggregation Tree. 2018 IEEE International Conference on Big Data (Big Data) (2018), 5417--5419."},{"volume-title":"Exploiting functional dependence in query optimization","author":"Paulley Glenn Norman","unstructured":"Glenn Norman Paulley. 2001. Exploiting functional dependence in query optimization. University of Waterloo.","key":"e_1_2_1_39_1"},{"doi-asserted-by":"publisher","key":"e_1_2_1_40_1","DOI":"10.1109\/ICDE.2003.1260805"},{"doi-asserted-by":"publisher","key":"e_1_2_1_41_1","DOI":"10.1145\/233269.233360"},{"doi-asserted-by":"publisher","key":"e_1_2_1_42_1","DOI":"10.1109\/ICDE.2019.00196"},{"doi-asserted-by":"publisher","key":"e_1_2_1_43_1","DOI":"10.1145\/3318464.3380584"},{"key":"e_1_2_1_44_1","first-page":"19","article-title":"LEODB2's learning optimizer","volume":"1","author":"Stillger Michael","year":"2001","unstructured":"Michael Stillger, Guy M Lohman, Volker Markl, and Mokhtar Kandil. 2001. LEODB2's learning optimizer. In VLDB, Vol. 1. 19--28.","journal-title":"VLDB"},{"key":"e_1_2_1_45_1","volume-title":"The case for shared nothing. Database Engineering Bulletin)","author":"Stonebraker Michael","year":"1986","unstructured":"Michael Stonebraker. 1986. The case for shared nothing. Database Engineering Bulletin) (1986), 4--9."},{"doi-asserted-by":"publisher","key":"e_1_2_1_46_1","DOI":"10.14778\/3415478.3415541"},{"key":"e_1_2_1_47_1","volume-title":"Hash Adaptive Bloom Filter. 2021 IEEE 37th International Conference on Data Engineering (ICDE)","author":"Xie Rongbiao","year":"2021","unstructured":"Rongbiao Xie, Meng Li, Zheyu Miao, Rong Gu, He Huang, Haipeng Dai, and Guihai Chen. 2021. Hash Adaptive Bloom Filter. 2021 IEEE 37th International Conference on Data Engineering (ICDE) (2021), 636--647."},{"doi-asserted-by":"publisher","key":"e_1_2_1_48_1","DOI":"10.1109\/TSG.2014.2343997"},{"key":"e_1_2_1_49_1","volume-title":"9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12)","author":"Zaharia Matei","year":"2012","unstructured":"Matei Zaharia, Mosharaf Chowdhury, Tathagata Das, Ankur Dave, Justin Ma, Murphy McCauly, Michael J Franklin, Scott Shenker, and Ion Stoica. 2012. Resilient distributed datasets: A fault-tolerant abstraction for in-memory cluster computing. In 9th USENIX Symposium on Networked Systems Design and Implementation (NSDI 12). 15--28."},{"key":"e_1_2_1_50_1","volume-title":"Spark SQL Query Optimization Based on Runtime Statistics Collection. 2021 IEEE 6th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA)","author":"Zhao Yong","year":"2021","unstructured":"Yong Zhao and Rong Chen. 2021. Spark SQL Query Optimization Based on Runtime Statistics Collection. 2021 IEEE 6th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA) (2021), 250--255."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3611540.3611553","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,10]],"date-time":"2025-09-10T22:33:03Z","timestamp":1757543583000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3611540.3611553"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8]]},"references-count":50,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["10.14778\/3611540.3611553"],"URL":"https:\/\/doi.org\/10.14778\/3611540.3611553","relation":{},"ISSN":["2150-8097"],"issn-type":[{"type":"print","value":"2150-8097"}],"subject":[],"published":{"date-parts":[[2023,8]]},"assertion":[{"value":"2023-08-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}