{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T18:00:54Z","timestamp":1772906454244,"version":"3.50.1"},"reference-count":62,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2018,8]]},"abstract":"<jats:p>F1 Query is a stand-alone, federated query processing platform that executes SQL queries against data stored in different file-based formats as well as different storage systems at Google (e.g., Bigtable, Spanner, Google Spreadsheets, etc.). F1 Query eliminates the need to maintain the traditional distinction between different types of data processing workloads by simultaneously supporting: (i) OLTP-style point queries that affect only a few records; (ii) low-latency OLAP querying of large amounts of data; and (iii) large ETL pipelines. F1 Query has also significantly reduced the need for developing hard-coded data processing pipelines by enabling declarative queries integrated with custom business logic. F1 Query satisfies key requirements that are highly desirable within Google: (i) it provides a unified view over data that is fragmented and distributed over multiple data sources; (ii) it leverages datacenter resources for performant query processing with high throughput and low latency; (iii) it provides high scalability for large data sizes by increasing computational parallelism; and (iv) it is extensible and uses innovative approaches to integrate complex business logic in declarative query processing. This paper presents the end-to-end design of F1 Query. Evolved out of F1, the distributed database originally built to manage Google's advertising data, F1 Query has been in production for multiple years at Google and serves the querying needs of a large number of users and systems.<\/jats:p>","DOI":"10.14778\/3229863.3229871","type":"journal-article","created":{"date-parts":[[2018,9,10]],"date-time":"2018-09-10T12:12:28Z","timestamp":1536581548000},"page":"1835-1848","source":"Crossref","is-referenced-by-count":27,"title":["F1 query"],"prefix":"10.14778","volume":"11","author":[{"given":"Bart","family":"Samwel","sequence":"first","affiliation":[{"name":"Google LLC"}]},{"given":"John","family":"Cieslewicz","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Ben","family":"Handy","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Jason","family":"Govig","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Petros","family":"Venetis","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Chanjun","family":"Yang","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Keith","family":"Peters","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Jeff","family":"Shute","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Daniel","family":"Tenedorio","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Himani","family":"Apte","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Felix","family":"Weigel","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"David","family":"Wilhite","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Jiacheng","family":"Yang","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Jun","family":"Xu","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Jiexing","family":"Li","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Zhan","family":"Yuan","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Craig","family":"Chasseur","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Qiang","family":"Zeng","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Ian","family":"Rae","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Anurag","family":"Biyani","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Andrew","family":"Harn","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Yang","family":"Xia","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Andrey","family":"Gubichev","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Amr","family":"El-Helw","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Orri","family":"Erling","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Zhepeng","family":"Yan","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Mohan","family":"Yang","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Yiqun","family":"Wei","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Thanh","family":"Do","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Colin","family":"Zheng","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Goetz","family":"Graefe","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Somayeh","family":"Sardashti","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Ahmed M.","family":"Aly","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Divy","family":"Agrawal","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Ashish","family":"Gupta","sequence":"additional","affiliation":[{"name":"Google LLC"}]},{"given":"Shiv","family":"Venkataraman","sequence":"additional","affiliation":[{"name":"Google LLC"}]}],"member":"320","published-online":{"date-parts":[[2018,8]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"AWS Lambda. https:\/\/aws.amazon.com\/lambda\/.  AWS Lambda. https:\/\/aws.amazon.com\/lambda\/."},{"key":"e_1_2_1_2_1","unstructured":"BSON (binary JSON). http:\/\/bsonspec.org.  BSON (binary JSON). http:\/\/bsonspec.org."},{"key":"e_1_2_1_3_1","unstructured":"Google BigQuery. https:\/\/cloud.google.com\/bigquery.  Google BigQuery. https:\/\/cloud.google.com\/bigquery."},{"key":"e_1_2_1_4_1","unstructured":"Google Cloud Dataflow. https:\/\/cloud.google.com\/dataflow.  Google Cloud Dataflow. https:\/\/cloud.google.com\/dataflow."},{"key":"e_1_2_1_5_1","unstructured":"Google Cloud Functions. https:\/\/cloud.google.com\/functions\/docs\/.  Google Cloud Functions. https:\/\/cloud.google.com\/functions\/docs\/."},{"key":"e_1_2_1_6_1","unstructured":"Inside Capacitor BigQuery's next-generation columnar storage format. https:\/\/cloud.google.com\/blog\/big-data\/2016\/04\/inside-capacitor-bigquerys-next-generation-columnar-storage-format.  Inside Capacitor BigQuery's next-generation columnar storage format. https:\/\/cloud.google.com\/blog\/big-data\/2016\/04\/inside-capacitor-bigquerys-next-generation-columnar-storage-format."},{"key":"e_1_2_1_7_1","unstructured":"Jinja. http:\/\/jinja.pocoo.org.  Jinja. http:\/\/jinja.pocoo.org."},{"key":"e_1_2_1_8_1","unstructured":"Oracle database cloud service. https:\/\/cloud.oracle.com\/database.  Oracle database cloud service. https:\/\/cloud.oracle.com\/database."},{"key":"e_1_2_1_9_1","unstructured":"Protocol Buffers. https:\/\/developers.google.com\/protocol-buffers.  Protocol Buffers. https:\/\/developers.google.com\/protocol-buffers."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824076"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742797"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3056103"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2011.5767935"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/319628.319650"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/319989.319991"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2015.7113294"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807167.1807280"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/872757.872877"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1806596.1806638"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/1365815.1365816"},{"key":"e_1_2_1_21_1","first-page":"1","volume-title":"WebDB","author":"Chasseur C.","year":"2013","unstructured":"C. Chasseur , Y. Li , and J. M. Patel . Enabling JSON document stores in relational systems . In WebDB , pages 1 -- 6 , 2013 . C. Chasseur, Y. Li, and J. M. Patel. Enabling JSON document stores in relational systems. In WebDB, pages 1--6, 2013."},{"issue":"12","key":"e_1_2_1_22_1","first-page":"1318","article-title":"Tenzing: A SQL implementation on the MapReduce framework","volume":"4","author":"Chattopadhyay B.","year":"2011","unstructured":"B. Chattopadhyay , L. Lin , W. Liu , S. Mittal , P. Aragonda , V. Lychagina , Y. Kwon , and M. Wong . Tenzing: A SQL implementation on the MapReduce framework . PVLDB , 4 ( 12 ): 1318 -- 1327 , 2011 . B. Chattopadhyay, L. Lin, W. Liu, S. Mittal, P. Aragonda, V. Lychagina, Y. Kwon, and M. Wong. Tenzing: A SQL implementation on the MapReduce framework. PVLDB, 4(12):1318--1327, 2011.","journal-title":"PVLDB"},{"key":"e_1_2_1_23_1","first-page":"261","volume-title":"OSDI","author":"Corbett J. C.","year":"2012","unstructured":"J. C. Corbett , J. Dean , M. Epstein , A. Fikes , C. Frost , J. J. Furman , S. Ghemawat , A. Gubarev , C. Heiser , P. Hochschild , W. C. Hsieh , S. Kanthak , E. Kogan , H. Li , A. Lloyd , S. Melnik , D. Mwaura , D. Nagle , S. Quinlan , R. Rao , L. Rolig , Y. Saito , M. Szymaniak , C. Taylor , R. Wang , and D. Woodford . Spanner: Google's globally-distributed database . In OSDI , pages 261 -- 264 , 2012 . J. C. Corbett, J. Dean, M. Epstein, A. Fikes, C. Frost, J. J. Furman, S. Ghemawat, A. Gubarev, C. Heiser, P. Hochschild, W. C. Hsieh, S. Kanthak, E. Kogan, H. Li, A. Lloyd, S. Melnik, D. Mwaura, D. Nagle, S. Quinlan, R. Rao, L. Rolig, Y. Saito, M. Szymaniak, C. Taylor, R. Wang, and D. Woodford. Spanner: Google's globally-distributed database. In OSDI, pages 261--264, 2012."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2903741"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2408776.2408794"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1629175.1629198"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/122058.122071"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/129888.129894"},{"key":"e_1_2_1_29_1","first-page":"1297","volume-title":"VLDB","author":"Du F.","year":"2004","unstructured":"F. Du , S. Amer-Yahia , and J. Freire . ShreX: Managing XML documents in relational databases . In VLDB , pages 1297 -- 1300 , 2004 . F. Du, S. Amer-Yahia, and J. Freire. ShreX: Managing XML documents in relational databases. In VLDB, pages 1297--1300, 2004."},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/509252.509292"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687553.1687567"},{"key":"e_1_2_1_32_1","first-page":"209","volume-title":"VLDB","author":"Fushimi S.","year":"1986","unstructured":"S. Fushimi , M. Kitsuregawa , and H. Tanaka . An overview of the system software of a parallel relational database machine GRACE . In VLDB , pages 209 -- 219 , 1986 . S. Fushimi, M. Kitsuregawa, and H. Tanaka. An overview of the system software of a parallel relational database machine GRACE. In VLDB, pages 209--219, 1986."},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/945445.945450"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/93605.98720"},{"issue":"3","key":"e_1_2_1_35_1","first-page":"19","article-title":"The cascades framework for query optimization","volume":"18","author":"Graefe G.","year":"1995","unstructured":"G. Graefe . The cascades framework for query optimization . IEEE Data Engineering Bulletin , 18 ( 3 ): 19 -- 29 , 1995 . G. Graefe. The cascades framework for query optimization. IEEE Data Engineering Bulletin, 18(3):19--29, 1995.","journal-title":"IEEE Data Engineering Bulletin"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/1132960.1132964"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2742795"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14778\/2732977.2732999"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.14778\/2350229.2350259"},{"key":"e_1_2_1_40_1","first-page":"33","volume-title":"HotCloud","author":"Hendrickson S.","year":"2016","unstructured":"S. Hendrickson , S. Sturdevant , T. Harter , V. Venkataramani , A. C. Arpaci-Dusseau , and R. H. Arpaci-Dusseau . Serverless computation with OpenLambda . In HotCloud , pages 33 -- 39 , 2016 . S. Hendrickson, S. Sturdevant, T. Harter, V. Venkataramani, A. C. Arpaci-Dusseau, and R. H. Arpaci-Dusseau. Serverless computation with OpenLambda. In HotCloud, pages 33--39, 2016."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.5555\/229593.229595"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/276305.276338"},{"key":"e_1_2_1_44_1","first-page":"494","volume-title":"VLDB","author":"Jaedicke M.","year":"1999","unstructured":"M. Jaedicke and B. Mitschang . User-defined table operators: Enhancing extensibility for ORDBMS . In VLDB , pages 494 -- 505 , 1999 . M. Jaedicke and B. Mitschang. User-defined table operators: Enhancing extensibility for ORDBMS. In VLDB, pages 494--505, 1999."},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.5555\/645841.670883"},{"key":"e_1_2_1_46_1","volume-title":"CIDR","author":"Kornacker M.","year":"2015","unstructured":"M. Kornacker , A. Behm , V. Bittorf , T. Bobrovytsky , C. Ching , A. Choi , J. Erickson , M. Grund , D. Hecht , M. Jacobs , I. Joshi , L. Kuff , D. Kumar , A. Leblang , N. Li , I. Pandis , H. Robinson , D. Rorke , S. Rus , J. Russell , D. Tsirogiannis , S. Wanderman-Milne , and M. Yoder . Impala: A modern, open-source SQL engine for Hadoop . In CIDR , 2015 . M. Kornacker, A. Behm, V. Bittorf, T. Bobrovytsky, C. Ching, A. Choi, J. Erickson, M. Grund, D. Hecht, M. Jacobs, I. Joshi, L. Kuff, D. Kumar, A. Leblang, N. Li, I. Pandis, H. Robinson, D. Rorke, S. Rus, J. Russell, D. Tsirogiannis, S. Wanderman-Milne, and M. Yoder. Impala: A modern, open-source SQL engine for Hadoop. In CIDR, 2015."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/371578.371598"},{"key":"e_1_2_1_48_1","first-page":"294","volume-title":"VLDB","author":"Linnemann V.","year":"1988","unstructured":"V. Linnemann , K. K\u00fcspert , P. Dadam , P. Pistor , R. Erbe , A. Kemper , N. S\u00fcdkamp , G. Walch , and M. Wallrath . Design and implementation of an extensible database management system supporting user defined data types and functions . In VLDB , pages 294 -- 305 , 1988 . V. Linnemann, K. K\u00fcspert, P. Dadam, P. Pistor, R. Erbe, A. Kemper, N. S\u00fcdkamp, G. Walch, and M. Wallrath. Design and implementation of an extensible database management system supporting user defined data types and functions. In VLDB, pages 294--305, 1988."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-82375-6_2"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2904444"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920886"},{"key":"e_1_2_1_52_1","first-page":"468","volume-title":"VLDB","author":"Nakayama M.","year":"1988","unstructured":"M. Nakayama , M. Kitsuregawa , and M. Takagi . Hash-partitioned join method using dynamic destaging strategy . In VLDB , pages 468 -- 478 , 1988 . M. Nakayama, M. Kitsuregawa, and M. Takagi. Hash-partitioned join method using dynamic destaging strategy. In VLDB, pages 468--478, 1988."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/1376616.1376726"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2015.7113378"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536222.2536232"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2975159"},{"issue":"1","key":"e_1_2_1_57_1","first-page":"4","article-title":"The case for shared nothing","volume":"9","author":"Stonebraker M.","year":"1986","unstructured":"M. Stonebraker . The case for shared nothing . IEEE Database Engineering Bulletin , 9 ( 1 ): 4 -- 9 , 1986 . M. Stonebraker. The case for shared nothing. IEEE Database Engineering Bulletin, 9(1):4--9, 1986.","journal-title":"IEEE Database Engineering Bulletin"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687553.1687609"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3056101"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.5555\/846219.847359"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1145\/3872.3874"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2934664"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2012.148"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3229863.3229871","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:15:56Z","timestamp":1672222556000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3229863.3229871"}},"subtitle":["declarative querying at scale"],"short-title":[],"issued":{"date-parts":[[2018,8]]},"references-count":62,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2018,8]]}},"alternative-id":["10.14778\/3229863.3229871"],"URL":"https:\/\/doi.org\/10.14778\/3229863.3229871","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2018,8]]}}}