{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,13]],"date-time":"2025-11-13T07:11:53Z","timestamp":1763017913798},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2018,8]]},"abstract":"<jats:p>\n            Stream processing applications reduce the latency of batch data pipelines and enable engineers to quickly identify production issues. Many times, a service can log data to distinct streams, even if they relate to the same real-world event (e.g., a search on Facebook's search bar). Furthermore, the logging of related events can appear on the server side with different delay, causing one stream to be significantly behind the other in terms of logged\n            <jats:italic>event times<\/jats:italic>\n            for a given log entry. To be able to stitch this information together with\n            <jats:italic>low latency<\/jats:italic>\n            , we need to be able to\n            <jats:italic>join<\/jats:italic>\n            two different streams where each stream may have its own characteristics regarding the degree in which its data is\n            <jats:italic>out-of-order<\/jats:italic>\n            . Doing so in a streaming fashion is challenging as a join operator consumes lots of memory, especially with significant data volumes. This paper describes an end-to-end streaming join service that addresses the challenges above through a streaming join operator that uses an adaptive stream synchronization algorithm that is able to handle the different distributions we observe in real-world streams regarding their event times. This synchronization scheme paces the parsing of new data and reduces overall operator memory footprint while still providing high accuracy. We have integrated this into a streaming SQL system and have successfully reduced the latency of several batch pipelines using this approach.\n          <\/jats:p>","DOI":"10.14778\/3229863.3229869","type":"journal-article","created":{"date-parts":[[2018,9,10]],"date-time":"2018-09-10T12:12:28Z","timestamp":1536581548000},"page":"1809-1821","source":"Crossref","is-referenced-by-count":18,"title":["Providing streaming joins as a service at Facebook"],"prefix":"10.14778","volume":"11","author":[{"given":"Gabriela","family":"Jacques-Silva","sequence":"first","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ran","family":"Lei","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luwei","family":"Cheng","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guoqiang Jerry","family":"Chen","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kuen","family":"Ching","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tanji","family":"Hu","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuan","family":"Mei","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kevin","family":"Wilfong","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rithin","family":"Shetty","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Serhat","family":"Yilmaz","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anirban","family":"Banerjee","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Benjamin","family":"Heintz","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shridar","family":"Iyer","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Anshul","family":"Jaiswal","sequence":"additional","affiliation":[{"name":"Facebook Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2018,8]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Uber Engineerings Open Source Streaming Analytics Platform. https:\/\/eng.uber.com\/athenax\/","author":"Introducing","year":"2017","unstructured":"Introducing AthenaX , Uber Engineerings Open Source Streaming Analytics Platform. https:\/\/eng.uber.com\/athenax\/ , 2017 . Introducing AthenaX, Uber Engineerings Open Source Streaming Analytics Platform. https:\/\/eng.uber.com\/athenax\/, 2017."},{"key":"e_1_2_1_2_1","volume-title":"https:\/\/docs.aws.amazon.com\/kinesisanalytics\/latest\/dev","author":"Data Analytics Amazon Kinesis","year":"2018","unstructured":"Amazon Kinesis Data Analytics . https:\/\/docs.aws.amazon.com\/kinesisanalytics\/latest\/dev , 2018 . Amazon Kinesis Data Analytics. https:\/\/docs.aws.amazon.com\/kinesisanalytics\/latest\/dev, 2018."},{"key":"e_1_2_1_3_1","volume-title":"https:\/\/calcite.apache.org\/","author":"Calcite Apache","year":"2018","unstructured":"Apache Calcite . https:\/\/calcite.apache.org\/ , 2018 . Apache Calcite. https:\/\/calcite.apache.org\/, 2018."},{"key":"e_1_2_1_4_1","volume-title":"https:\/\/databricks.com\/blog\/2018\/03\/13\/introducing-stream-stream-joins-in-apache-spark-2-3.html","author":"Stream-Stream Introducing","year":"2018","unstructured":"Introducing Stream-Stream Joins in Apache Spark 2.3. https:\/\/databricks.com\/blog\/2018\/03\/13\/introducing-stream-stream-joins-in-apache-spark-2-3.html , 2018 . Introducing Stream-Stream Joins in Apache Spark 2.3. https:\/\/databricks.com\/blog\/2018\/03\/13\/introducing-stream-stream-joins-in-apache-spark-2-3.html, 2018."},{"key":"e_1_2_1_5_1","volume-title":"https:\/\/prestodb.io\/","year":"2018","unstructured":"Presto. https:\/\/prestodb.io\/ , 2018 . Presto. https:\/\/prestodb.io\/, 2018."},{"key":"e_1_2_1_6_1","volume-title":"https:\/\/github.com\/facebook\/rocksdb\/","author":"DB.","year":"2018","unstructured":"Rocks DB. https:\/\/github.com\/facebook\/rocksdb\/ , 2018 . RocksDB. https:\/\/github.com\/facebook\/rocksdb\/, 2018."},{"key":"e_1_2_1_7_1","volume-title":"http:\/\/facebook.github.io\/zstd\/","year":"2018","unstructured":"Zstd. http:\/\/facebook.github.io\/zstd\/ , 2018 . Zstd. http:\/\/facebook.github.io\/zstd\/, 2018."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536222.2536231"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536222.2536229"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824076"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2465272"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3190664"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1016028.1016032"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137765.3137777"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2904441"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/872757.872765"},{"key":"e_1_2_1_17_1","volume-title":"OSDI","author":"Dean J.","year":"2004","unstructured":"J. Dean and S. Ghemawat . MapReduce: Simplified data processing on large clusters . In OSDI , 2004 . J. Dean and S. Ghemawat. MapReduce: Simplified data processing on large clusters. In OSDI, 2004."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1031171.1031189"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/1099554.1099587"},{"key":"e_1_2_1_20_1","volume-title":"VLDB","author":"Gedik B.","year":"2007","unstructured":"B. Gedik , P. S. Yu , and R. R. Bordawekar . Executing stream joins on the Cell processor . VLDB , 2007 . B. Gedik, P. S. Yu, and R. R. Bordawekar. Executing stream joins on the Cell processor. VLDB, 2007."},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/69.273032"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1147\/JRD.2013.2243535"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.14778\/3007263.3007272"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2003.1260804"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2485278.2485280"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2007.367851"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2746485"},{"key":"e_1_2_1_28_1","volume-title":"DockerCon","author":"Narayanan A.","year":"2014","unstructured":"A. Narayanan . Tupperware : Containerized deployment at Facebook . In DockerCon , 2014 . A. Narayanan. Tupperware: Containerized deployment at Facebook. In DockerCon, 2014."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137765.3137770"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2815400.2815401"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989389"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2010.5447738"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132750"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.5555\/382009.383658"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/SSDBM.2007.43"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3229863.3229869","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:11:44Z","timestamp":1672222304000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3229863.3229869"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8]]},"references-count":35,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2018,8]]}},"alternative-id":["10.14778\/3229863.3229869"],"URL":"https:\/\/doi.org\/10.14778\/3229863.3229869","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2018,8]]}}}