{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,29]],"date-time":"2025-09-29T12:40:17Z","timestamp":1759149617394,"version":"3.44.0"},"reference-count":65,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["LQN25F020009"],"award-info":[{"award-number":["LQN25F020009"]}]},{"name":"National Regional Innovation and Development Joint Fund","award":["U24A20254"],"award-info":[{"award-number":["U24A20254"]}]},{"name":"Key Research Program of Zhejiang Province","award":["2023C01037"],"award-info":[{"award-number":["2023C01037"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Database Syst."],"published-print":{"date-parts":[[2026,3,31]]},"abstract":"<jats:p>Generation-based testing techniques have proven effective in detecting logic bugs in DBMS, often stemming from the improper implementation of query optimizers. However, existing generation-based debugging tools predominantly rely on random testing, which tends to overlook critical error-prone areas such as multi-table joining and grouped aggregation. In this article, we propose TQS, a novel testing framework targeted at detecting logic bugs arising from SPJG (Select-Project-Join-Group By) query optimizations. Given a target DBMS, TQS achieves the goal with two key components: Data-guided Schema and Query Generation (DSG) and Knowledge-guided Query Space Exploration (KQE). DSG addresses the key challenge of multi-table query debugging: how to generate ground-truth (query, result) pairs for verification. DSG utilizes data derived from dimensionally aggregated data cubes, which store data of grouped metric columns. It maps data from data cubes to a wide table, applies database normalization techniques to the wide table to generate a testing schema and maintains a bitmap index for result tracking. To improve debug efficiency, DSG also artificially inserts some noises into the generated data. To avoid repetitive query space search, KQE guides the generation of error-prone cubes, and forms the problem as isomorphic graph set discovery and combines the graph embedding and weighted random walk for query generation. We evaluated TQS on four popular DBMSs: MySQL, MariaDB, TiDB, and PolarDB. Experimental results show that TQS is effective in finding logic bugs of SPJG query optimization in database management systems. It successfully detected 226 bugs within 24 hours, including 63 bugs in MySQL, 52 in MariaDB, 68 in TiDB, and 43 in PolarDB, respectively.<\/jats:p>","DOI":"10.1145\/3764583","type":"journal-article","created":{"date-parts":[[2025,8,28]],"date-time":"2025-08-28T11:32:26Z","timestamp":1756380746000},"page":"1-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Unveiling Logic Bugs in SPJG Query Optimizations within DBMS"],"prefix":"10.1145","volume":"51","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8611-0283","authenticated-orcid":false,"given":"Xiu","family":"Tang","sequence":"first","affiliation":[{"name":"Zhejiang University","place":["Hangzhou, China"]}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-3293-2227","authenticated-orcid":false,"given":"Shijie","family":"Yang","sequence":"additional","affiliation":[{"name":"Zhejiang University","place":["Hangzhou, China"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7903-1496","authenticated-orcid":false,"given":"Sai","family":"Wu","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Zhejiang University","place":["Hangzhou, China"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9964-2470","authenticated-orcid":false,"given":"Dongxiang","family":"Zhang","sequence":"additional","affiliation":[{"name":"Zhejiang University","place":["Hangzhou, China"]}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-2689-6020","authenticated-orcid":false,"given":"Wenchao","family":"Zhou","sequence":"additional","affiliation":[{"name":"Alibaba Group","place":["Hangzhou, China"]}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-0770-5775","authenticated-orcid":false,"given":"Feifei","family":"Li","sequence":"additional","affiliation":[{"name":"Alibaba Group","place":["Hangzhou, China"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7483-0045","authenticated-orcid":false,"given":"Gang","family":"Chen","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Zhejiang University","place":["Hangzhou, China"]}]}],"member":"320","published-online":{"date-parts":[[2025,9,29]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"197","volume-title":"Proceedings of the International Semantic Web Conference","author":"Alu\u00e7 G\u00fcnes","year":"2014","unstructured":"G\u00fcnes Alu\u00e7, Olaf Hartig, M. Tamer \u00d6zsu, and Khuzaima Daudjee. 2014. Diversified stress testing of RDF data management systems. In Proceedings of the International Semantic Web Conference. Springer, 197\u2013212."},{"key":"e_1_3_2_3_2","first-page":"197","volume-title":"Proceedings of the International Semantic Web Conference","author":"Alu\u00e7 G\u00fcne\u015f","year":"2014","unstructured":"G\u00fcne\u015f Alu\u00e7, Olaf Hartig, M. Tamer \u00d6zsu, and Khuzaima Daudjee. 2014. Diversified stress testing of RDF data management systems. In Proceedings of the International Semantic Web Conference. 197\u2013212."},{"issue":"8","key":"e_1_3_2_4_2","doi-asserted-by":"crossref","first-page":"906","DOI":"10.14778\/3204028.3204034","article-title":"HD-index: Pushing the scalability-accuracy boundary for approximate kNN search in high-dimensional spaces","volume":"11","author":"Arora Akhil","year":"2018","unstructured":"Akhil Arora, Sakshi Sinha, Piyush Kumar, and Arnab Bhattacharya. 2018. HD-index: Pushing the scalability-accuracy boundary for approximate kNN search in high-dimensional spaces. Proc. VLDB Endow. 11, 8 (2018), 906\u2013919.","journal-title":"Proc. VLDB Endow."},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Jinsheng Ba and Manuel Rigger. 2024. CERT: Finding performance issues in database systems through the lens of cardinality estimation. In ICSE. 133:1\u2013133:13.","DOI":"10.1145\/3597503.3639076"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","unstructured":"Jinsheng Ba and Manuel Rigger. 2024. Keep it simple: Testing databases via differential query plans. Proc. ACM Manag. Data 2 3 (2024) 188.","DOI":"10.1145\/3654991"},{"key":"e_1_3_2_7_2","first-page":"63","volume-title":"Proceedings of the IEEE Transactions on Knowledge and Data Engineering","author":"Bagan Guillaume","year":"2017","unstructured":"Guillaume Bagan, Angela Bonifati, Radu Ciucanu, George H. L. Fletcher, Aur\u00e9lien Lemay, and Nicky Advokaat. 2017. gMark: Schema-driven generation of graphs and queries. In Proceedings of the IEEE Transactions on Knowledge and Data Engineering. 63\u201364."},{"key":"e_1_3_2_8_2","first-page":"1243","volume-title":"Proceedings of the 33rd International Conference on Very Large Data Bases","author":"Bati Hardik","year":"2007","unstructured":"Hardik Bati, Leo Giakoumakis, Steve Herbert, and Aleksandras Surna. 2007. A genetic approach for random testing of database systems. In Proceedings of the 33rd International Conference on Very Large Data Bases. ACM, 1243\u20131251."},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1145\/1247480.1247520","volume-title":"Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data","author":"Binnig Carsten","year":"2007","unstructured":"Carsten Binnig, Donald Kossmann, Eric Lo, and M. Tamer \u00d6zsu. 2007. QAGen: Generating query-aware test databases. In Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data. ACM, 341\u2013352."},{"key":"e_1_3_2_10_2","first-page":"29","volume-title":"Proceedings of the Semantics in Databases","author":"Biskup Joachim","year":"1995","unstructured":"Joachim Biskup. 1995. Achievements of relational database schema design theory revisited. In Proceedings of the Semantics in Databases. Springer, 29\u201354."},{"key":"e_1_3_2_11_2","first-page":"143","volume-title":"Proceedings of the SIGMOD.","author":"Biskup Joachim","year":"1979","unstructured":"Joachim Biskup, Umeshwar Dayal, and Philip A. Bernstein. 1979. Synthesizing independent database schemas. In Proceedings of the SIGMOD.Philip A. Bernstein (Ed.), ACM, 143\u2013151."},{"issue":"2","key":"e_1_3_2_12_2","first-page":"1","article-title":"The berlin sparql benchmark","volume":"5","author":"Bizer Christian","year":"2009","unstructured":"Christian Bizer and Andreas Schultz. 2009. The berlin sparql benchmark. IJSWIS 5, 2 (2009), 1\u201324.","journal-title":"IJSWIS"},{"key":"e_1_3_2_13_2","first-page":"16","volume-title":"Proceedings of the Extended Semantic Web Conference","author":"Blum Daniel","year":"2011","unstructured":"Daniel Blum and Sara Cohen. 2011. Grr: Generating random RDF. In Proceedings of the Extended Semantic Web Conference. Springer, 16\u201330."},{"key":"e_1_3_2_14_2","unstructured":"Manuel Bodirsky. 2015. Graph homomorphisms and universal algebra course notes. TU Dresden (2015) 6\u201318."},{"key":"e_1_3_2_15_2","first-page":"1","volume-title":"Proceedings of the 7th International Workshop on Satisfiability Modulo Theories","author":"Brummayer Robert","year":"2009","unstructured":"Robert Brummayer and Armin Biere. 2009. Fuzzing and delta-debugging SMT solvers. In Proceedings of the 7th International Workshop on Satisfiability Modulo Theories. 1\u20135."},{"key":"e_1_3_2_16_2","first-page":"1097","volume-title":"Proceedings of the VLDB","author":"Bruno Nicolas","year":"2005","unstructured":"Nicolas Bruno and Surajit Chaudhuri. 2005. Flexible database generators. In Proceedings of the VLDB. ACM, 1097\u20131107."},{"issue":"12","key":"e_1_3_2_17_2","doi-asserted-by":"crossref","first-page":"1721","DOI":"10.1109\/TKDE.2006.190","article-title":"Generating queries with cardinality constraints for DBMS testing","volume":"18","author":"Bruno Nicolas","year":"2006","unstructured":"Nicolas Bruno, Surajit Chaudhuri, and Dilys Thomas. 2006. Generating queries with cardinality constraints for DBMS testing. IEEE Transactions on Knowledge and Data Engineering 18, 12 (2006), 1721\u20131725.","journal-title":"IEEE Transactions on Knowledge and Data Engineering"},{"key":"e_1_3_2_18_2","first-page":"2477","volume-title":"Proceedings of the SIGMOD","author":"Cao Wei","year":"2021","unstructured":"Wei Cao, Yingqiang Zhang, Xinjun Yang, Feifei Li, Sheng Wang, Qingda Hu, Xuntao Cheng, Zongzhi Chen, Zhenjun Liu, Jing Fang, Bo Wang, Yuhui Wang, Haiqing Sun, Ze Yang, Zhushi Cheng, Sen Chen, Jian Wu, Wei Hu, Jianwei Zhao, Yusong Gao, Songlu Cai, Yunyang Zhang, and Jiawang Tong. 2021. PolarDB serverless: A cloud native database for disaggregated data centers. In Proceedings of the SIGMOD. ACM, 2477\u20132489."},{"key":"e_1_3_2_19_2","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1007\/978-3-642-28891-3_12","volume-title":"Proceedings of the NASA Formal Methods Symposium","author":"Cuoq Pascal","year":"2012","unstructured":"Pascal Cuoq, Benjamin Monate, Anne Pacalet, Virgile Prevosto, John Regehr, Boris Yakobowski, and Xuejun Yang. 2012. Testing static analyzers with randomly generated programs. In Proceedings of the NASA Formal Methods Symposium. Springer, 120\u2013125."},{"key":"e_1_3_2_20_2","unstructured":"Angjela Davitkova Damjan Gjurovski and Sebastian Michel. 2022. LMKG: Learned models for cardinality estimation in knowledge graphs. In EDBT. 2:169\u20132:182."},{"key":"e_1_3_2_21_2","article-title":"DB-Engines Ranking","year":"2018","unstructured":"DB-Engines. 2018. DB-Engines Ranking. [EB\/OL]. (2018). Retrieved May 2024 from https:\/\/db-engines.com\/en\/ranking","journal-title":"[EB\/OL]"},{"issue":"3","key":"e_1_3_2_22_2","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1145\/44498.44499","article-title":"New methods and fast algorithms for database normalization","volume":"13","author":"Diederich Jim","year":"1988","unstructured":"Jim Diederich and Jack Milton. 1988. New methods and fast algorithms for database normalization. ACM Transactions on Database Systems 13, 3 (1988), 339\u2013365.","journal-title":"ACM Transactions on Database Systems"},{"issue":"5","key":"e_1_3_2_23_2","first-page":"730","article-title":"Efficient streaming subgraph isomorphism with graph neural networks","volume":"14","author":"Duong Chi Thang","year":"2021","unstructured":"Chi Thang Duong, Dung Hoang, Hongzhi Yin, Matthias Weidlich, Quoc Viet Hung Nguyen, and Karl Aberer. 2021. Efficient streaming subgraph isomorphism with graph neural networks. VLDB 14, 5 (2021), 730\u2013742.","journal-title":"VLDB"},{"key":"e_1_3_2_24_2","doi-asserted-by":"crossref","first-page":"619","DOI":"10.1145\/2723372.2742786","volume-title":"Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data","author":"Erling Orri","year":"2015","unstructured":"Orri Erling, Alex Averbuch, Josep Larriba-Pey, Hassan Chafi, Andrey Gubichev, Arnau Prat, Minh-Duc Pham, and Peter Boncz. 2015. The LDBC social network benchmark: Interactive workload. In Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data. 619\u2013630."},{"key":"e_1_3_2_25_2","volume-title":"Computers and Intractability: A Guide to the Theory of NP-Completeness","author":"Garey M. R.","year":"1979","unstructured":"M. R. Garey and David S. Johnson. 1979. Computers and Intractability: A Guide to the Theory of NP-Completeness. W. H. Freeman."},{"key":"e_1_3_2_26_2","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1145\/191839.191886","volume-title":"Proceedings of the 1994 ACM SIGMOD International Conference on Management of Data","author":"Gray Jim","year":"1994","unstructured":"Jim Gray, Prakash Sundaresan, Susanne Englert, Kenneth Baclawski, and Peter J. Weinberger. 1994. Quickly generating billion-record synthetic databases. In Proceedings of the 1994 ACM SIGMOD International Conference on Management of Data. ACM,243\u2013252."},{"key":"e_1_3_2_27_2","first-page":"1","volume-title":"Proceedings of the 5th International Workshop on Testing Database Systems","author":"Gu Zhongxian","year":"2012","unstructured":"Zhongxian Gu, Mohamed A. Soliman, and Florian M. Waas. 2012. Testing the accuracy of query optimizers. In Proceedings of the 5th International Workshop on Testing Database Systems. 1\u20136."},{"issue":"2","key":"e_1_3_2_28_2","first-page":"158","article-title":"LUBM: A benchmark for OWL knowledge base systems","volume":"3","author":"Guo Yuanbo","year":"2005","unstructured":"Yuanbo Guo, Zhengxiang Pan, and Jeff Heflin. 2005. LUBM: A benchmark for OWL knowledge base systems. Journal of Web Semantics 3, 2-3 (2005), 158\u2013182.","journal-title":"Journal of Web Semantics"},{"key":"e_1_3_2_29_2","first-page":"1243","volume-title":"Proceedings of the 32nd International Conference on Very Large Data Bases","author":"Houkj\u00e6r Kenneth","year":"2006","unstructured":"Kenneth Houkj\u00e6r, Kristian Torp, and Rico Wind. 2006. Simple and realistic data generation. In Proceedings of the 32nd International Conference on Very Large Data Bases. ACM, 1243\u20131246."},{"issue":"12","key":"e_1_3_2_30_2","first-page":"3072","article-title":"TiDB: A raft-based HTAP database","volume":"13","author":"Huang Dongxu","year":"2020","unstructured":"Dongxu Huang, Qi Liu, Qiu Cui, Zhuhe Fang, Xiaoyu Ma, Fei Xu, Li Shen, Liu Tang, Yuxing Zhou, Menglong Huang, et\u00a0al. 2020. TiDB: A raft-based HTAP database. VLDB 13, 12 (2020), 3072\u20133084.","journal-title":"VLDB"},{"issue":"2","key":"e_1_3_2_31_2","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1093\/comjnl\/42.2.100","article-title":"TANE: An efficient algorithm for discovering functional and approximate dependencies","volume":"42","author":"Huhtala Yka","year":"1999","unstructured":"Yka Huhtala, Juha K\u00e4rkk\u00e4inen, Pasi Porkka, and Hannu Toivonen. 1999. TANE: An efficient algorithm for discovering functional and approximate dependencies. The Computer Journal 42, 2 (1999), 100\u2013111.","journal-title":"The Computer Journal"},{"issue":"1","key":"e_1_3_2_32_2","first-page":"57","article-title":"APOLLO: Automatic detection and diagnosis of performance regressions in database systems","volume":"13","author":"Jung Jinho","year":"2019","unstructured":"Jinho Jung, Hong Hu, Joy Arulraj, Taesoo Kim, and Woon-Hak Kang. 2019. APOLLO: Automatic detection and diagnosis of performance regressions in database systems. VLDB 13, 1 (2019), 57\u201370.","journal-title":"VLDB"},{"key":"e_1_3_2_33_2","first-page":"590","volume-title":"Proceedings of the 2017 32nd IEEE\/ACM International Conference on Automated Software Engineering","author":"Kapus Timotej","year":"2017","unstructured":"Timotej Kapus and Cristian Cadar. 2017. Automatic testing of symbolic execution engines via program generation and differential testing. In Proceedings of the 2017 32nd IEEE\/ACM International Conference on Automated Software Engineering. IEEE Computer Society, 590\u2013600."},{"key":"e_1_3_2_34_2","first-page":"238","volume-title":"Proceedings of the 2008 23rd IEEE\/ACM International Conference on Automated Software Engineering","author":"Khalek Shadi Abdul","year":"2008","unstructured":"Shadi Abdul Khalek, Bassem Elkarablieh, Yai O. Laleye, and Sarfraz Khurshid. 2008. Query-aware test generation using a relational constraint solver. In Proceedings of the 2008 23rd IEEE\/ACM International Conference on Automated Software Engineering. IEEE Computer Society, 238\u2013247."},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1145\/1858996.1859063","volume-title":"Proceedings of the 25th IEEE\/ACM International Conference on Automated Software Engineering","author":"Khalek Shadi Abdul","year":"2010","unstructured":"Shadi Abdul Khalek and Sarfraz Khurshid. 2010. Automated SQL query generation for systematic testing of database engines. In Proceedings of the 25th IEEE\/ACM International Conference on Automated Software Engineering. ACM, 329\u2013332."},{"key":"e_1_3_2_36_2","first-page":"216","volume-title":"Proceedings of the ACM Sigplan Notices","author":"Le Vu","year":"2014","unstructured":"Vu Le, Mehrdad Afshari, and Zhendong Su. 2014. Compiler validation via equivalence modulo inputs. In Proceedings of the ACM Sigplan Notices. ACM, 216\u2013226."},{"issue":"2","key":"e_1_3_2_37_2","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1007\/s00778-009-0157-y","article-title":"A framework for testing DBMS features","volume":"19","author":"Lo Eric","year":"2010","unstructured":"Eric Lo, Carsten Binnig, Donald Kossmann, M. Tamer \u00d6zsu, and Wing-Kai Hon. 2010. A framework for testing DBMS features. VLDB J. 19, 2 (2010), 203\u2013230.","journal-title":"VLDB J."},{"key":"e_1_3_2_38_2","volume-title":"The Theory of Relational Databases","author":"Maier David","year":"1983","unstructured":"David Maier. 1983. The Theory of Relational Databases. Computer Science Press."},{"key":"e_1_3_2_39_2","article-title":"MariaDB hints","year":"2022","unstructured":"MariaDB. 2022. MariaDB hints. [EB\/OL]. (2022). Retrieved May 2024 from https:\/\/mariadb.com\/kb\/en\/optimizer-switch\/","journal-title":"[EB\/OL]"},{"key":"e_1_3_2_40_2","article-title":"Mariadb Homepage","year":"2022","unstructured":"Mariadb. 2022. Mariadb Homepage. [EB\/OL]. (2022). Retrieved from https:\/\/mariadb.org\/","journal-title":"[EB\/OL]"},{"issue":"1","key":"e_1_3_2_41_2","first-page":"100","article-title":"Differential testing for software","volume":"10","author":"McKeeman William M.","year":"1998","unstructured":"William M. McKeeman. 1998. Differential testing for software. Digital Technical Journal 10, 1 (1998), 100\u2013107.","journal-title":"Digital Technical Journal"},{"key":"e_1_3_2_42_2","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1145\/1376616.1376668","volume-title":"Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data","author":"Mishra Chaitanya","year":"2008","unstructured":"Chaitanya Mishra, Nick Koudas, and Calisto Zuzarte. 2008. Generating targeted queries for database testing. In Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. ACM, 499\u2013510."},{"key":"e_1_3_2_43_2","article-title":"MySQL hints","year":"2022","unstructured":"MySQL. 2022. MySQL hints. [EB\/OL]. (2022). Retrieved May 2024 from https:\/\/dev.mysql.com\/doc\/refman\/8.0\/en\/optimizer-hints.html","journal-title":"[EB\/OL]"},{"key":"e_1_3_2_44_2","article-title":"MySQL Homepage","year":"2022","unstructured":"MySQL. 2022. MySQL Homepage. [EB\/OL]. (2022). Retrieved May 2024 from https:\/\/www.mysql.com","journal-title":"[EB\/OL]"},{"key":"e_1_3_2_45_2","article-title":"Developer Survey Results","author":"Overflow Stack","year":"2021","unstructured":"Stack Overflow. 2021. Developer Survey Results. [EB\/OL]. (2021). Retrieved May 2024 from https:\/\/insights.stackoverflow.com\/survey\/2021","journal-title":"[EB\/OL]"},{"key":"e_1_3_2_46_2","doi-asserted-by":"crossref","first-page":"821","DOI":"10.1145\/2882903.2915203","volume-title":"Proceedings of the 2016 International Conference on Management of Data.","author":"Papenbrock Thorsten","year":"2016","unstructured":"Thorsten Papenbrock and Felix Naumann. 2016. A hybrid approach to functional dependency discovery. In Proceedings of the 2016 International Conference on Management of Data.Fatma \u00d6zcan, Georgia Koutrika, and Sam Madden (Eds.), ACM, 821\u2013833."},{"key":"e_1_3_2_47_2","first-page":"342","volume-title":"Proceedings of the EDBT","author":"Papenbrock Thorsten","year":"2017","unstructured":"Thorsten Papenbrock and Felix Naumann. 2017. Data-driven schema normalization. In Proceedings of the EDBT. OpenProceedings.org, 342\u2013353."},{"key":"e_1_3_2_48_2","first-page":"1045","volume-title":"Proceedings of the VLDB","author":"Poess Meikel","year":"2004","unstructured":"Meikel Poess and John M. Stephens. 2004. Generating thousand benchmark queries in seconds. In Proceedings of the VLDB. Morgan Kaufmann, 1045\u20131053."},{"issue":"4","key":"e_1_3_2_49_2","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1145\/2897356.2897362","article-title":"Performance monitoring in SAP HANA\u2019s continuous integration process","volume":"43","author":"Rehmann Kim-Thomas","year":"2016","unstructured":"Kim-Thomas Rehmann, Changyun Seo, Dongwon Hwang, Binh Than Truong, Alexander Boehm, and Dong Hun Lee. 2016. Performance monitoring in SAP HANA\u2019s continuous integration process. SIGMETRICS Perform. Evaluation Rev. 43, 4 (2016), 43\u201352.","journal-title":"SIGMETRICS Perform. Evaluation Rev."},{"key":"e_1_3_2_50_2","first-page":"1140","volume-title":"Proceedings of the ACM Joint Meeting on ESEC and FSE","author":"Rigger Manuel","year":"2020","unstructured":"Manuel Rigger and Zhendong Su. 2020. Detecting optimization bugs in database engines via non-optimizing reference engine construction. In Proceedings of the ACM Joint Meeting on ESEC and FSE. 1140\u20131152."},{"key":"e_1_3_2_51_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3428279","article-title":"Finding bugs in database systems via query partitioning","volume":"4","author":"Rigger Manuel","year":"2020","unstructured":"Manuel Rigger and Zhendong Su. 2020. Finding bugs in database systems via query partitioning. Proceedings of the ACM on Programming Languages 4, OOPSLA (2020), 1\u201330.","journal-title":"Proceedings of the ACM on Programming Languages"},{"key":"e_1_3_2_52_2","article-title":"SQLancer","author":"Rigger Manuel","year":"2020","unstructured":"Manuel Rigger and Zhendong Su. 2020. SQLancer. [EB\/OL]. (2020). Retrieved May 2024 from https:\/\/github.com\/sqlancer\/sqlancer","journal-title":"[EB\/OL]"},{"key":"e_1_3_2_53_2","first-page":"667","volume-title":"Proceedings of the OSDI 20","author":"Rigger Manuel","year":"2020","unstructured":"Manuel Rigger and Zhendong Su. 2020. Testing database engines via pivoted query synthesis. In Proceedings of the OSDI 20. 667\u2013682."},{"key":"e_1_3_2_54_2","first-page":"222","volume-title":"Proceedings of the ICDE","author":"Schmidt Michael","year":"2009","unstructured":"Michael Schmidt, Thomas Hornung, Georg Lausen, and Christoph Pinkel. 2009. SP\u2303 2Bench: A SPARQL performance benchmark. In Proceedings of the ICDE. IEEE, 222\u2013233."},{"key":"e_1_3_2_55_2","article-title":"SQLSmith","author":"Seltenreich Andreas","year":"2020","unstructured":"Andreas Seltenreich. 2020. SQLSmith. [EB\/OL]. (2020). Retrieved May 2024 from https:\/\/github.com\/anse1\/sqlsmith","journal-title":"[EB\/OL]"},{"key":"e_1_3_2_56_2","first-page":"618","volume-title":"Proceedings of the VLDB.","author":"Slutz Donald R.","year":"1998","unstructured":"Donald R. Slutz. 1998. Massive stochastic testing of SQL. In Proceedings of the VLDB.Ashish Gupta, Oded Shmueli, and Jennifer Widom (Eds.), Morgan Kaufmann, 618\u2013622."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3588909"},{"key":"e_1_3_2_58_2","article-title":"TiDB hints","year":"2022","unstructured":"TiDB. 2022. TiDB hints. [EB\/OL]. (2022). Retrieved May 2024 from https:\/\/docs.pingcap.com\/tidb\/v5.3\/optimizer-hints","journal-title":"[EB\/OL]"},{"key":"e_1_3_2_59_2","first-page":"1215","volume-title":"Proceedings of the SIGMOD","author":"Vartak Manasi","year":"2010","unstructured":"Manasi Vartak, Venkatesh Raghavan, and Elke A. Rundensteiner. 2010. QRelX: Generating meaningful queries that provide cardinality assurance. In Proceedings of the SIGMOD. ACM, 1215\u20131218."},{"issue":"11","key":"e_1_3_2_60_2","first-page":"1458","article-title":"Embedded functional dependencies and data-completeness tailored database design","volume":"12","author":"Wei Ziheng","year":"2019","unstructured":"Ziheng Wei and Sebastian Link. 2019. Embedded functional dependencies and data-completeness tailored database design. VLDB 12, 11 (2019), 1458\u20131470.","journal-title":"VLDB"},{"key":"e_1_3_2_61_2","first-page":"99","volume-title":"Proceedings of the SSDBM","author":"Wu Kesheng","year":"2002","unstructured":"Kesheng Wu, Ekow J. Otoo, and Arie Shoshani. 2002. Compressing bitmap indexes for faster search operations. In Proceedings of the SSDBM. IEEE Computer Society, 99\u2013108."},{"key":"e_1_3_2_62_2","first-page":"4:1\u20134:6","volume-title":"Proceedings of the SIGMOD","author":"Yan Jiaqi","year":"2018","unstructured":"Jiaqi Yan, Qiuye Jin, Shrainik Jain, Stratis D. Viglas, and Allison W. Lee. 2018. Snowtrail: Testing with production queries on a cloud database. In Proceedings of the SIGMOD. ACM, 4:1\u20134:6."},{"key":"e_1_3_2_63_2","first-page":"283","volume-title":"Proceedings of the SIGPLAN, PLDI","author":"Yang Xuejun","year":"2011","unstructured":"Xuejun Yang, Yang Chen, Eric Eide, and John Regehr. 2011. Finding and understanding bugs in C compilers. In Proceedings of the SIGPLAN, PLDI. ACM, 283\u2013294."},{"issue":"6","key":"e_1_3_2_64_2","first-page":"984","article-title":"Efficient bi-triangle counting for large bipartite networks","volume":"14","author":"Yang Yixing","year":"2021","unstructured":"Yixing Yang, Yixiang Fang, Maria E. Orlowska, Wenjie Zhang, and Xuemin Lin. 2021. Efficient bi-triangle counting for large bipartite networks. VLDB 14, 6 (2021), 984\u2013996.","journal-title":"VLDB"},{"key":"e_1_3_2_65_2","article-title":"Neural subgraph matching","volume":"2007","author":"Ying Rex","year":"2020","unstructured":"Rex Ying, Zhaoyu Lou, Jiaxuan You, Chengtao Wen, Arquimedes Canedo, and Jure Leskovec. 2020. Neural subgraph matching. CoRR abs\/2007.03092 (2020).","journal-title":"CoRR"},{"key":"e_1_3_2_66_2","first-page":"955","volume-title":"CCS","author":"Zhong Rui","year":"2020","unstructured":"Rui Zhong, Yongheng Chen, Hong Hu, Hangfan Zhang, Wenke Lee, and Dinghao Wu. 2020. SQUIRREL: Testing database management systems with language validity and coverage feedback. In CCS. ACM, 955\u2013970."}],"container-title":["ACM Transactions on Database Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3764583","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,29]],"date-time":"2025-09-29T12:21:46Z","timestamp":1759148506000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3764583"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,29]]},"references-count":65,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,3,31]]}},"alternative-id":["10.1145\/3764583"],"URL":"https:\/\/doi.org\/10.1145\/3764583","relation":{},"ISSN":["0362-5915","1557-4644"],"issn-type":[{"type":"print","value":"0362-5915"},{"type":"electronic","value":"1557-4644"}],"subject":[],"published":{"date-parts":[[2025,9,29]]},"assertion":[{"value":"2024-05-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-30","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}