{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T04:04:50Z","timestamp":1773720290206,"version":"3.50.1"},"reference-count":27,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2017,11,9]],"date-time":"2017-11-09T00:00:00Z","timestamp":1510185600000},"content-version":"vor","delay-in-days":365,"URL":"http:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100004318","name":"Microsoft Corporation","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100004318","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["IIS-0415101, IIS-0639106, CNS-0838948, IIS-1016205 and EIA-0080123"],"award-info":[{"award-number":["IIS-0415101, IIS-0639106, CNS-0838948, IIS-1016205 and EIA-0080123"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Database Syst."],"published-print":{"date-parts":[[2017,3,31]]},"abstract":"<jats:p>It is surprisingly hard to obtain accurate and precise measurements of the time spent executing a query because there are many sources of variance. To understand these sources, we review relevant per-process and overall measures obtainable from the Linux kernel and introduce a structural causal model relating these measures. A thorough correlational analysis provides strong support for this model. We attempted to determine why a particular measurement wasn\u2019t repeatable and then to devise ways to eliminate or reduce that variance. This enabled us to articulate a timing protocol that applies to proprietary DBMSes, that ensures the repeatability of a query, and that obtains a quite accurate query execution time while dropping very few outliers. This resulting query time measurement procedure, termed the Tucson Timing Protocol Version 2 (TTPv2), consists of the following steps: (i) perform sanity checks to ensure data validity; (ii) drop some query executions via clearly motivated predicates; (iii) drop some entire queries at a cardinality, again via clearly motivated predicates; (iv) for those that remain, compute a single measured time by a carefully justified formula over the underlying measures of the remaining query executions; and (v) perform post-analysis sanity checks. The result is a mature, general, robust, self-checking protocol that provides a more precise and more accurate timing of the query. The protocol is also applicable to other operating domains in which measurements of multiple processes each doing computation and I\/O is needed.<\/jats:p>","DOI":"10.1145\/2996454","type":"journal-article","created":{"date-parts":[[2016,11,9]],"date-time":"2016-11-09T08:36:49Z","timestamp":1478680609000},"page":"1-42","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["DBMS Metrology"],"prefix":"10.1145","volume":"42","author":[{"given":"Sabah","family":"Currim","sequence":"first","affiliation":[{"name":"University of Arizona"}]},{"given":"Richard T.","family":"Snodgrass","sequence":"additional","affiliation":[{"name":"University of Arizona"}]},{"given":"Young-Kyoon","family":"Suh","sequence":"additional","affiliation":[{"name":"University of Arizona"}]},{"given":"Rui","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Arizona"}]}],"member":"320","published-online":{"date-parts":[[2016,11,9]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Modern Processors: Where Does Time Go? Technical Report UW-CS-TR-1394. Computer Sciences","author":"Ailamaki A. G.","year":"1999","unstructured":"A. G. Ailamaki, D. J. DeWitt, M. D. Hill, and D. A. Wood. 1999. DBMSes on Modern Processors: Where Does Time Go? Technical Report UW-CS-TR-1394. Computer Sciences, University of Wisconsin."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2012.64"},{"key":"e_1_2_2_3_1","unstructured":"T. Bowden B. Bauer J. Nerin S. Feng and S. Seibold. 2009. Linux kernel documentation - The \/proc file system. Retrieved from https:\/\/www.kernel.org\/doc\/Documentation\/filesystems\/proc.txt."},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807128.1807152"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2465331"},{"key":"e_1_2_2_6_1","unstructured":"R. Faulkner and R. Gomes. 1991. The process file system and process model in UNIX system V. In USENIX Winter. USENIX 243--252."},{"key":"e_1_2_2_7_1","unstructured":"R. F. Forman J. M. Pechacek and W. H. Schwane. 2001. Apparatus and method for measuring transaction time in a computer system. IBM Patent (US 6178449). (January 2001)."},{"key":"e_1_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2009.130"},{"key":"e_1_2_2_9_1","first-page":"1","article-title":"Testing SQL server\u2019s query optimizer: Challenges, techniques and experiences","volume":"31","author":"Gikoumakis L.","year":"2008","unstructured":"L. Gikoumakis, and C. Galindo-Legaria. 2008. Testing SQL server\u2019s query optimizer: Challenges, techniques and experiences. IEEE Data Engineering Bulletin 31, 1 (Aug. 2008), 36--43.","journal-title":"IEEE Data Engineering Bulletin"},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/645914.671620"},{"key":"e_1_2_2_11_1","unstructured":"Intel Corp. 2015. Intel\u00ae Turbo Boost Technology 2.0. Retrieved from http:\/\/www.intel.com\/content\/www\/us\/en\/architecture-and-technology\/turbo-boost\/turbo-boost-technology.html."},{"key":"e_1_2_2_12_1","volume-title":"Introduction to Computer System Performance Evaluation","author":"Kant K.","unstructured":"K. Kant. 1992. Introduction to Computer System Performance Evaluation. McGraw-Hill."},{"key":"e_1_2_2_13_1","unstructured":"Linux. 2015. Kernel Asynchronous I\/O (AIO) Support for Linux. Retrieved from http:\/\/lse.sourceforge.net\/io\/aio.html."},{"key":"e_1_2_2_14_1","unstructured":"Oracle Corp. 2014. PreparedStatement in Java 7. Retrieved from http:\/\/docs.oracle.com\/javase\/7\/docs\/api\/java\/sql\/PreparedStatement.html."},{"key":"e_1_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4302-6197-1"},{"key":"e_1_2_2_16_1","volume-title":"Proceedings of the VLDB International Conference. Morgan Kaufmann Publishers Inc., 621--632","author":"Stillger M.","unstructured":"M. Stillger, G. M. Lohman, V. Markl, and M. Kaqndil. 2001. LEO - DB2\u2019s learning optimizer. In Proceedings of the VLDB International Conference. Morgan Kaufmann Publishers Inc., 621--632."},{"key":"e_1_2_2_17_1","unstructured":"Y.-K. Suh. 2015. Exploring Causal Factors of DBMS Thrashing. Ph.D. Dissertation. Department of Computer Science University of Arizona."},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.14778\/2733004.2733050"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","unstructured":"S. W. Thomas R. T. Snodgrass and R. Zhang. 2014. Benchmark frameworks and \u03c4bench. Software\u2014Practice and Experience 44 9 (April 2014) 1047--1075. 10.1002\/spe.2189","DOI":"10.1002\/spe.2189"},{"key":"e_1_2_2_20_1","unstructured":"TPC. 2010a. TPC Transaction Processing Performance Council\u2014TPC-H. Retrieved from http:\/\/www.tpc.org\/tpch\/."},{"key":"e_1_2_2_21_1","unstructured":"TPC. 2010b. Transaction Processing Performance Council\u2014TPC-C. Retrieved from http:\/\/www.tpc.org\/tpcc\/."},{"key":"e_1_2_2_22_1","volume-title":"accessed","author":"Developers Valgrind","year":"2010","unstructured":"Valgrind Developers. accessed October 27, 2010. Callgrind: A call-graph generating cache and branch prediction profiler. Retrieved from http:\/\/valgrind.org\/docs\/manual\/cl-manual.html."},{"key":"e_1_2_2_23_1","unstructured":"Working Group 2 of the Joint Committee for Guides in Metrology (JCGM\/WG 2). 2008. International Vocabulary of Metrology\u2014Basic and General Concepts and Associated Terms. (2008). Retrieved from http:\/\/jcgm.bipm.org\/vim\/en\/2.2.html."},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.5555\/977401.978145"},{"key":"e_1_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.129222"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.14778\/2095686.2095687"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2012.110"}],"container-title":["ACM Transactions on Database Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2996454","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2996454","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2996454","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,18]],"date-time":"2025-11-18T09:23:36Z","timestamp":1763457816000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2996454"}},"subtitle":["Measuring Query Time"],"short-title":[],"issued":{"date-parts":[[2016,11,9]]},"references-count":27,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2017,3,31]]}},"alternative-id":["10.1145\/2996454"],"URL":"https:\/\/doi.org\/10.1145\/2996454","relation":{},"ISSN":["0362-5915","1557-4644"],"issn-type":[{"value":"0362-5915","type":"print"},{"value":"1557-4644","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,11,9]]},"assertion":[{"value":"2015-08-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-09-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-11-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}