{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T04:17:58Z","timestamp":1782879478384,"version":"3.54.5"},"reference-count":70,"publisher":"Association for Computing Machinery (ACM)","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2019,9,15]]},"abstract":"<jats:p>The practical art of constructing database management systems (DBMSs) involves a morass of trade-offs among query execution speed, query optimization speed, standards compliance, feature parity, modularity, portability, and other goals. It is no surprise that DBMSs, like all complex software systems, contain bugs that can adversely affect their performance. The performance of DBMSs is an important metric as it determines how quickly an application can take in new information and use it to make new decisions.<\/jats:p>\n          <jats:p>Both developers and users face challenges while dealing with performance regression bugs. First, developers usually find it challenging to manually design test cases to uncover performance regressions since DBMS components tend to have complex interactions. Second, users encountering performance regressions are often unable to report them, as the regression-triggering queries could be complex and database-dependent. Third, developers have to expend a lot of effort on localizing the root cause of the reported bugs, due to the system complexity and software development complexity.<\/jats:p>\n          <jats:p>Given these challenges, this paper presents the design of Apollo, a toolchain for automatically detecting, reporting, and diagnosing performance regressions in DBMSs. We demonstrate that Apollo automates the generation of regression-triggering queries, simplifies the bug reporting process for users, and enables developers to quickly pinpoint the root cause of performance regressions. By automating the detection and diagnosis of performance regressions, Apollo reduces the labor cost of developing efficient DBMSs.<\/jats:p>","DOI":"10.14778\/3357377.3357382","type":"journal-article","created":{"date-parts":[[2019,11,19]],"date-time":"2019-11-19T19:25:15Z","timestamp":1574191515000},"page":"57-70","source":"Crossref","is-referenced-by-count":65,"title":["APOLLO"],"prefix":"10.14778","volume":"13","author":[{"given":"Jinho","family":"Jung","sequence":"first","affiliation":[{"name":"Georgia Institute of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hong","family":"Hu","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joy","family":"Arulraj","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taesoo","family":"Kim","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Woonhak","family":"Kang","sequence":"additional","affiliation":[{"name":"Bay Inc."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2019,9]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"PostgreSQL. https:\/\/www.postgresql.org\/.  PostgreSQL. https:\/\/www.postgresql.org\/."},{"key":"e_1_2_1_2_1","unstructured":"PostgreSQL Bug Reporting Guidelines. https:\/\/www.postgresql.org\/list\/pgsql-bugs\/.  PostgreSQL Bug Reporting Guidelines. https:\/\/www.postgresql.org\/list\/pgsql-bugs\/."},{"key":"e_1_2_1_3_1","unstructured":"PostgreSQL Performance Regression Reports. https:\/\/www.postgresql.org\/search\/?m=1&q=performance+regression&l=8&d=-1&s=r.  PostgreSQL Performance Regression Reports. https:\/\/www.postgresql.org\/search\/?m=1&q=performance+regression&l=8&d=-1&s=r."},{"key":"e_1_2_1_4_1","unstructured":"PostgreSQL Roadmap. https:\/\/wiki.postgresql.org\/wiki\/Todo.  PostgreSQL Roadmap. https:\/\/wiki.postgresql.org\/wiki\/Todo."},{"key":"e_1_2_1_5_1","unstructured":"PostgreSQL Testing. https:\/\/www.postgresql.org\/developer\/testing\/.  PostgreSQL Testing. https:\/\/www.postgresql.org\/developer\/testing\/."},{"key":"e_1_2_1_6_1","unstructured":"SQLite. https:\/\/www.sqlite.org\/index.html.  SQLite. https:\/\/www.sqlite.org\/index.html."},{"key":"e_1_2_1_7_1","unstructured":"SQLite Bug Reporting Guidelines. https:\/\/www.chiark.greenend.org.uk\/~sgtatham\/bugs.html.  SQLite Bug Reporting Guidelines. https:\/\/www.chiark.greenend.org.uk\/~sgtatham\/bugs.html."},{"key":"e_1_2_1_8_1","unstructured":"SQLite Performance Regression Reports. https:\/\/www.sqlite.org\/src\/rptview?rn=1.  SQLite Performance Regression Reports. https:\/\/www.sqlite.org\/src\/rptview?rn=1."},{"key":"e_1_2_1_9_1","unstructured":"SQLite Roadmap. https:\/\/sqlite.org\/src4\/doc\/trunk\/www\/design.wiki.  SQLite Roadmap. https:\/\/sqlite.org\/src4\/doc\/trunk\/www\/design.wiki."},{"key":"e_1_2_1_10_1","unstructured":"SQLite Testing. https:\/\/sqlite.org\/testing.html.  SQLite Testing. https:\/\/sqlite.org\/testing.html."},{"key":"e_1_2_1_11_1","unstructured":"Database Language SQL. http:\/\/www.contrib.andrew.cmu.edu\/~shadow\/sql\/sql1992.txt 1992.  Database Language SQL. http:\/\/www.contrib.andrew.cmu.edu\/~shadow\/sql\/sql1992.txt 1992."},{"key":"e_1_2_1_12_1","unstructured":"How to Report Bugs Effectively. https:\/\/www.chiark.greenend.org.uk\/~sgtatham\/bugs.html 1999.  How to Report Bugs Effectively. https:\/\/www.chiark.greenend.org.uk\/~sgtatham\/bugs.html 1999."},{"key":"e_1_2_1_13_1","unstructured":"Git-scm. https:\/\/git-scm.com 2005.  Git-scm. https:\/\/git-scm.com 2005."},{"key":"e_1_2_1_14_1","unstructured":"Fossil-scm. https:\/\/fossil-scm.org 2006.  Fossil-scm. https:\/\/fossil-scm.org 2006."},{"key":"e_1_2_1_15_1","unstructured":"TPC-C Benchmark. http:\/\/www.tpc.org\/tpcc\/spec\/tpcc_current.pdf 2007.  TPC-C Benchmark. http:\/\/www.tpc.org\/tpcc\/spec\/tpcc_current.pdf 2007."},{"key":"e_1_2_1_16_1","unstructured":"Fighting Regressions with Git Bisect. http:\/\/www.linux-kongress.org\/2009\/abstracts.html#3_7_1 2008.  Fighting Regressions with Git Bisect. http:\/\/www.linux-kongress.org\/2009\/abstracts.html#3_7_1 2008."},{"key":"e_1_2_1_17_1","unstructured":"Fossil-bisect. https:\/\/www.fossil-scm.org\/index.html\/help\/bisect 2008.  Fossil-bisect. https:\/\/www.fossil-scm.org\/index.html\/help\/bisect 2008."},{"key":"e_1_2_1_18_1","unstructured":"Git-bisect. https:\/\/git-scm.com\/docs\/git-bisect 2008.  Git-bisect. https:\/\/git-scm.com\/docs\/git-bisect 2008."},{"key":"e_1_2_1_19_1","unstructured":"A Data-Driven Glimpse into the Burgeoning New Field. http:\/\/emc.com\/collateral\/about\/news\/emc-data-science-study-wp.pdf 2011.  A Data-Driven Glimpse into the Burgeoning New Field. http:\/\/emc.com\/collateral\/about\/news\/emc-data-science-study-wp.pdf 2011."},{"key":"e_1_2_1_20_1","unstructured":"RQG\n\n  \n  : Random Query Generator. https:\/\/launchpad.net\/randgen 2012.  RQG: Random Query Generator. https:\/\/launchpad.net\/randgen 2012."},{"key":"e_1_2_1_21_1","unstructured":"100x Faster Postgres Performance by Changing 1 Line. https:\/\/www.datadoghq.com\/blog\/100x-faster-postgres-performance-by-changing-1--line\/ 2013.  100x Faster Postgres Performance by Changing 1 Line. https:\/\/www.datadoghq.com\/blog\/100x-faster-postgres-performance-by-changing-1--line\/ 2013."},{"key":"e_1_2_1_22_1","unstructured":"AFL\n\n  \n  : American Fuzzy Lop 2015. http:\/\/lcamtuf.coredump.cx\/afl\/.  AFL: American Fuzzy Lop 2015. http:\/\/lcamtuf.coredump.cx\/afl\/."},{"key":"e_1_2_1_23_1","unstructured":"PQuery: Multithreaded SQL Tester \/ Reducer. https:\/\/github.com\/Percona-QA\/pquery 2015.  PQuery: Multithreaded SQL Tester \/ Reducer. https:\/\/github.com\/Percona-QA\/pquery 2015."},{"key":"e_1_2_1_24_1","unstructured":"Finding Bugs in SQLite the Easy Way 2016. https:\/\/lcamtuf.blogspot.com\/2015\/04\/finding-bugs-in-sqlite-easy-way.html.  Finding Bugs in SQLite the Easy Way 2016. https:\/\/lcamtuf.blogspot.com\/2015\/04\/finding-bugs-in-sqlite-easy-way.html."},{"key":"e_1_2_1_25_1","unstructured":"OSS-Fuzz: Continuous Fuzzing for Open Source Software 2016. https:\/\/github.com\/google\/oss-fuzz.  OSS-Fuzz: Continuous Fuzzing for Open Source Software 2016. https:\/\/github.com\/google\/oss-fuzz."},{"key":"e_1_2_1_26_1","unstructured":"SQLSmith. https:\/\/github.com\/anse1\/sqlsmith 2016.  SQLSmith. https:\/\/github.com\/anse1\/sqlsmith 2016."},{"key":"e_1_2_1_27_1","unstructured":"libFuzzer. http:\/\/llvm.org\/docs\/LibFuzzer.html 2018.  libFuzzer. http:\/\/llvm.org\/docs\/LibFuzzer.html 2018."},{"key":"e_1_2_1_28_1","unstructured":"PostgreSQL LIMIT and OFFSET. https:\/\/www.postgresql.org\/docs\/11\/queries-limit.html 2018.  PostgreSQL LIMIT and OFFSET. https:\/\/www.postgresql.org\/docs\/11\/queries-limit.html 2018."},{"key":"e_1_2_1_29_1","unstructured":"PostgreSQL Single-User Mode. https:\/\/www.postgresql.org\/docs\/11\/app-postgres.html 2018.  PostgreSQL Single-User Mode. https:\/\/www.postgresql.org\/docs\/11\/app-postgres.html 2018."},{"key":"e_1_2_1_30_1","unstructured":"PostgreSQL Table Partitioning. https:\/\/www.postgresql.org\/docs\/current\/ddl-partitioning.html 2019.  PostgreSQL Table Partitioning. https:\/\/www.postgresql.org\/docs\/current\/ddl-partitioning.html 2019."},{"key":"e_1_2_1_31_1","unstructured":"SQLite performance bug response. http:\/\/mailinglists.sqlite.org\/cgi-bin\/mailman\/private\/sqlite-users\/2019-April\/083864.html 2019.  SQLite performance bug response. http:\/\/mailinglists.sqlite.org\/cgi-bin\/mailman\/private\/sqlite-users\/2019-April\/083864.html 2019."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-74958-5_5"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/1273463.1273467"},{"key":"e_1_2_1_35_1","volume-title":"MIT","author":"Bruening D.","year":"2004"},{"key":"e_1_2_1_36_1","volume-title":"Linux Kongress","author":"De Melo A. C.","year":"2010"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1007\/11681878_14"},{"key":"e_1_2_1_38_1","unstructured":"M. J. Eager. Introduction to the DWARF Debugging Format 2012.  M. J. Eager. Introduction to the DWARF Debugging Format 2012."},{"key":"e_1_2_1_39_1","unstructured":"R. Elmasri and S. Navathe. Fundamentals of Database Systems. Addison-Wesley Publishing Company USA 6th edition 2010.  R. Elmasri and S. Navathe. Fundamentals of Database Systems. Addison-Wesley Publishing Company USA 6th edition 2010."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/2025113.2025179"},{"key":"e_1_2_1_41_1","volume-title":"DBTEST. ACM","author":"Garcia R.","year":"2009"},{"key":"e_1_2_1_42_1","doi-asserted-by":"crossref","unstructured":"G. Graefe. Query Evaluation Techniques for Large Databases. ACM Computing Surveys (CSUR) 25(2):73--169 1993.  G. Graefe. Query Evaluation Techniques for Large Databases. ACM Computing Surveys (CSUR) 25(2):73--169 1993.","DOI":"10.1145\/152610.152611"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1147\/sj.94.0242"},{"key":"e_1_2_1_44_1","first-page":"445","volume-title":"SECURITY","author":"Holler C.","year":"2012"},{"issue":"6","key":"e_1_2_1_45_1","first-page":"77","volume":"47","author":"Jin G.","year":"2012","journal-title":"Understanding and Detecting Real-world Performance Bugs. SIGPLAN Notices"},{"issue":"5","key":"e_1_2_1_46_1","first-page":"526","article-title":"Towards practical differential privacy for sql queries","volume":"11","author":"Johnson N.","year":"2018","journal-title":"PVLDB"},{"key":"e_1_2_1_47_1","volume-title":"ACM","author":"Kersten M. L.","year":"2018"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243804"},{"key":"e_1_2_1_49_1","first-page":"19","volume-title":"IFIP ICTCS","author":"L\u00e4mmel R.","year":"2006"},{"key":"e_1_2_1_50_1","volume-title":"Velocity and Variety","author":"Laney D.","year":"2001"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213874"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2304510.2304517"},{"key":"e_1_2_1_53_1","first-page":"141","volume-title":"Sigplan Notices","author":"Liblit B.","year":"2003"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/1064978.1065014"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/96267.96279"},{"key":"e_1_2_1_56_1","first-page":"39","volume-title":"Sigmod Record","author":"Pirahesh H.","year":"1992"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF01932308"},{"key":"e_1_2_1_58_1","unstructured":"R. Ramakrishnan and J. Gehrke. Database Management Systems. McGraw-Hill Inc. New York NY USA 3 edition 2003.  R. Ramakrishnan and J. Gehrke. Database Management Systems. McGraw-Hill Inc. New York NY USA 3 edition 2003."},{"key":"e_1_2_1_59_1","first-page":"335","volume-title":"SIGPLAN Notices","author":"Regehr J.","year":"2012"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/2897356.2897362"},{"key":"e_1_2_1_61_1","volume-title":"Intel Press","author":"Reinders J.","year":"2005"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/1460833.1460869"},{"key":"e_1_2_1_63_1","first-page":"618","volume-title":"VLDB","volume":"98","author":"Slutz D. R.","year":"1998"},{"key":"e_1_2_1_64_1","first-page":"561","volume-title":"SIGPLAN Notices","author":"Song L.","year":"2014"},{"issue":"1","key":"e_1_2_1_65_1","first-page":"44","article-title":"Intel \"Big Data","volume":"42","author":"Stonebraker M.","year":"2013","journal-title":"Science and Technology Center Vision and Execution Plan. SIGMOD Rec."},{"key":"e_1_2_1_66_1","first-page":"426","volume-title":"CHINACOM","author":"Wang J.","year":"2013"},{"key":"e_1_2_1_67_1","volume-title":"Proceedings of the 39th IEEE Symposium on Security and Privacy (Oakland)","author":"Xu W.","year":"2018"},{"key":"e_1_2_1_68_1","first-page":"51","volume-title":"IEEE Data Eng. Bull.","author":"Yagoub K.","year":"2008"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209950.3209958"},{"key":"e_1_2_1_70_1","volume-title":"Proceedings of the 27th USENIX Security Symposium (Security)","author":"Yun I.","year":"2018"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2011.01.002"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3357377.3357382","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:37:46Z","timestamp":1672223866000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3357377.3357382"}},"subtitle":["automatic detection and diagnosis of performance regressions in database systems"],"short-title":[],"issued":{"date-parts":[[2019,9]]},"references-count":70,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2019,9,15]]}},"alternative-id":["10.14778\/3357377.3357382"],"URL":"https:\/\/doi.org\/10.14778\/3357377.3357382","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2019,9]]}}}