{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,11]],"date-time":"2025-09-11T22:53:26Z","timestamp":1757631206171,"version":"3.44.0"},"reference-count":33,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2023,8]]},"abstract":"<jats:p>Query compilation and hardware acceleration are important technologies for optimizing the performance of data processing engines. There have been many works on the exploration and adoption of these techniques in recent years. However, a number of engines still refrain from adopting them because of some reasons. One of the common reasons claims that the intricacies of these techniques make engines too complex to maintain. Another major barrier is the lack of widely accepted architectures and libraries of these techniques, which leads to the adoption often starting from scratch with lots of effort. In this paper, we propose Intel Big Data Analytic Toolkit (BDTK), an open-source C++ acceleration toolkit library for analytical data processing engines. BDTK provides lightweight, easy-to-connect, reusable components with interoperable interfaces to support query compilation and hardware accelerators. The query compilation in BDTK leverages vectorized execution and data-centric code generation to achieve high performance. BDTK could be integrated into different engines and helps them to adapt query compilation and hardware accelerators to optimize performance bottlenecks with less engineering effort.<\/jats:p>","DOI":"10.14778\/3611540.3611558","type":"journal-article","created":{"date-parts":[[2023,9,15]],"date-time":"2023-09-15T11:32:37Z","timestamp":1694777557000},"page":"3702-3714","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Big Data Analytic Toolkit: A General-Purpose, Modular, and Heterogeneous Acceleration Toolkit for Data Analytical Engines"],"prefix":"10.14778","volume":"16","author":[{"given":"Jiang","family":"Li","sequence":"first","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qi","family":"Xie","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan","family":"Ma","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jian","family":"Ma","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kunshang","family":"Ji","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yizhong","family":"Zhang","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chaojun","family":"Zhang","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yixiu","family":"Chen","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gangsheng","family":"Wu","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Zhang","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kaidi","family":"Yang","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xinyi","family":"He","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qiuyang","family":"Shen","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanting","family":"Tao","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haiwei","family":"Zhao","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Penghui","family":"Jiao","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chengfei","family":"Zhu","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Qian","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cheng","family":"Xu","sequence":"additional","affiliation":[{"name":"Intel Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2023,8]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526045"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/320455.320457"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3526054"},{"key":"e_1_2_1_4_1","unstructured":"BlazingDB. 2023. Home - SQL-Bblaz Ing. https:\/\/blazingsql.com\/. (Accessed on 02\/23\/2023)."},{"key":"e_1_2_1_5_1","first-page":"225","article-title":"MonetDB\/X100: Hyper-Pipelining Query Execution","volume":"5","author":"Boncz Peter A","year":"2005","unstructured":"Peter A Boncz, Marcin Zukowski, and Niels Nes. 2005. MonetDB\/X100: Hyper-Pipelining Query Execution.. In Cidr, Vol. 5. 225--237.","journal-title":"Cidr"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.14778\/3554821.3554822"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/115372.115320"},{"key":"e_1_2_1_8_1","volume-title":"Gandiva: A LLVM-based Analytical Expression Compiler for Apache Arrow | Apache Arrow. https:\/\/arrow.apache.org\/blog\/2018\/12\/05\/gandiva-donation\/. (Accessed on 02\/26\/2023).","author":"Foundation Apache Software","year":"2018","unstructured":"Apache Software Foundation. 2018. Gandiva: A LLVM-based Analytical Expression Compiler for Apache Arrow | Apache Arrow. https:\/\/arrow.apache.org\/blog\/2018\/12\/05\/gandiva-donation\/. (Accessed on 02\/26\/2023)."},{"key":"e_1_2_1_9_1","unstructured":"Apache Software Foundation. 2022. Apache Arrow DataFusion --- Arrow Data-Fusion documentation. https:\/\/arrow.apache.org\/datafusion\/. (Accessed on 02\/23\/2023)."},{"key":"e_1_2_1_10_1","unstructured":"Apache Software Foundation. 2023. Apache Arrow | Apache Arrow. https:\/\/arrow.apache.org\/. (Accessed on 02\/23\/2023)."},{"key":"e_1_2_1_11_1","unstructured":"Apache Software Foundation. 2023. Arrow Columnar Format --- Apache Arrow v11.0.0. https:\/\/arrow.apache.org\/docs\/format\/Columnar.html. (Accessed on 02\/23\/2023)."},{"key":"e_1_2_1_12_1","unstructured":"LLVM Foundation. 2023. The LLVM Compiler Infrastructure Project. https:\/\/llvm.org\/. (Accessed on 02\/23\/2023)."},{"key":"e_1_2_1_13_1","unstructured":"Trino Software Foundation. 2023. Trino | Distributed SQL query engine for big data. https:\/\/trino.io\/. (Accessed on 02\/23\/2023)."},{"key":"e_1_2_1_14_1","unstructured":"Carnegie Mellon University Database Group. 2023. NoisePage - Self-Driving Database Management System. https:\/\/noise.page\/. (Accessed on 02\/23\/2023)."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.14778\/3551793.3551801"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2011.5767867"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.14778\/3275366.3284966"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-020-00643-4"},{"key":"e_1_2_1_19_1","unstructured":"Maksim Kita. 2022. JIT in ClickHouse. https:\/\/clickhouse.com\/blog\/clickhouse-just-in-time-compiler-jit. (Accessed on 02\/23\/2023)."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.14778\/3151113.3151114"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.14778\/2002938.2002940"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476311.3476410"},{"key":"e_1_2_1_23_1","volume-title":"Umbra: A Disk-Based System with In-Memory Performance.. In CIDR.","author":"Neumann Thomas","year":"2020","unstructured":"Thomas Neumann and Michael J Freitag. 2020. Umbra: A Disk-Based System with In-Memory Performance.. In CIDR."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.14778\/3554821.3554829"},{"key":"e_1_2_1_25_1","unstructured":"Pola-rs. 2023. Polars. https:\/\/www.pola.rs\/. (Accessed on 02\/23\/2023)."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2915244"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1995441.1995446"},{"key":"e_1_2_1_28_1","unstructured":"Substrait. 2023. Home - Substrait: Cross-Language Serialization for Relational Algebra. https:\/\/substrait.io\/. (Accessed on 02\/23\/2023)."},{"key":"e_1_2_1_29_1","unstructured":"Alex Suhan and Todd Mostak. 2015. MapD: Massive throughput database queries with LLVM on GPUs."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/DCC.2017.88"},{"volume-title":"High performance compilers for parallel computing","author":"Wolfe Michael Joseph","key":"e_1_2_1_31_1","unstructured":"Michael Joseph Wolfe. 1995. High performance compilers for parallel computing. Addison-Wesley Longman Publishing Co., Inc."},{"key":"e_1_2_1_32_1","first-page":"10","article-title":"Spark: Cluster computing with working sets","volume":"10","author":"Zaharia Matei","year":"2010","unstructured":"Matei Zaharia, Mosharaf Chowdhury, Michael J Franklin, Scott Shenker, Ion Stoica, et al. 2010. Spark: Cluster computing with working sets. HotCloud 10, 10--10 (2010), 95.","journal-title":"HotCloud"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.3390\/app10061915"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3611540.3611558","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,10]],"date-time":"2025-09-10T22:32:49Z","timestamp":1757543569000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3611540.3611558"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8]]},"references-count":33,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2023,8]]}},"alternative-id":["10.14778\/3611540.3611558"],"URL":"https:\/\/doi.org\/10.14778\/3611540.3611558","relation":{},"ISSN":["2150-8097"],"issn-type":[{"type":"print","value":"2150-8097"}],"subject":[],"published":{"date-parts":[[2023,8]]},"assertion":[{"value":"2023-08-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}