{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T02:20:41Z","timestamp":1773886841501,"version":"3.50.1"},"reference-count":30,"publisher":"Association for Computing Machinery (ACM)","issue":"9","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2019,5]]},"abstract":"<jats:p>We design and implement Megaphone, a data migration mechanism for stateful distributed dataflow engines with latency objectives. When compared to existing migration mechanisms, Megaphone has the following differentiating characteristics: (i) migrations can be subdivided to a configurable granularity to avoid latency spikes, and (ii) migrations can be prepared ahead of time to avoid runtime coordination. Megaphone is implemented as a library on an unmodified timely dataflow implementation, and provides an operator interface compatible with its existing APIs. We evaluate Megaphone on established benchmarks with varying amounts of state and observe that compared to na\u00efve approaches Megaphone reduces service latencies during reconfiguration by orders of magnitude without significantly increasing steady-state overhead.<\/jats:p>","DOI":"10.14778\/3329772.3329777","type":"journal-article","created":{"date-parts":[[2019,6,24]],"date-time":"2019-06-24T13:43:16Z","timestamp":1561383796000},"page":"1002-1015","source":"Crossref","is-referenced-by-count":47,"title":["Megaphone"],"prefix":"10.14778","volume":"12","author":[{"given":"Moritz","family":"Hoffmann","sequence":"first","affiliation":[{"name":"Vasiliki Kalavri"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Andrea","family":"Lattuada","sequence":"additional","affiliation":[{"name":"ETH Zurich"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Frank","family":"McSherry","sequence":"additional","affiliation":[{"name":"Timothy Roscoe"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,5]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Bringing Pokemon GO to life on Google Cloud. https:\/\/cloudplatform.googleblog.com\/2016\/09\/bringing-Pokemon-GO-to-life-on-Google-Cloud.html.  Bringing Pokemon GO to life on Google Cloud. https:\/\/cloudplatform.googleblog.com\/2016\/09\/bringing-Pokemon-GO-to-life-on-Google-Cloud.html."},{"key":"e_1_2_1_2_1","unstructured":"IBM Streams (accessed: November 2017). https:\/\/www.ibm.com\/ch-en\/marketplace\/stream-computing.  IBM Streams (accessed: November 2017). https:\/\/www.ibm.com\/ch-en\/marketplace\/stream-computing."},{"key":"e_1_2_1_3_1","unstructured":"New Tweets per second record and how! https:\/\/blog.twitter.com\/engineering\/en_us\/a\/2013\/new-tweets-per-second-record-and-how.html.  New Tweets per second record and how! https:\/\/blog.twitter.com\/engineering\/en_us\/a\/2013\/new-tweets-per-second-record-and-how.html."},{"key":"e_1_2_1_4_1","unstructured":"M.\n       \n      Abadi F.\n       \n      McSherry D. G.\n       \n      Murray and \n      \n      \n      T. L.\n       \n      Rodeheffer\n      \n  \n  . \n  Formal analysis of a distributed algorithm for tracking progress. In D. Beyer and M. Boreale editors Formal Techniques for Distributed Systems - Joint IFIP WG 6.1 International Conference FMOODS\/FORTE\n   \n  2013 Held as Part of the 8th International Federated Conference on Distributed Computing Techniques DisCoTec 2013 Florence Italy June 3--5 2013. Proceedings volume \n  7892\n   of \n  Lecture Notes in Computer Science pages \n  5\n  --\n  19\n  . \n  Springer 2013.  M. Abadi F. McSherry D. G. Murray and T. L. Rodeheffer. Formal analysis of a distributed algorithm for tracking progress. In D. Beyer and M. Boreale editors Formal Techniques for Distributed Systems - Joint IFIP WG 6.1 International Conference FMOODS\/FORTE 2013 Held as Part of the 8th International Federated Conference on Distributed Computing Techniques DisCoTec 2013 Florence Italy June 3--5 2013. Proceedings volume 7892 of Lecture Notes in Computer Science pages 5--19. Springer 2013."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536222.2536229"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.14778\/2824032.2824076"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183713.3190664"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2247596.2247647"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137765.3137777"},{"key":"e_1_2_1_10_1","unstructured":"P. Carbone A. Katsifodimos S. Ewen V. Markl S. Haridi and K. Tzoumas. Apache Flink: Stream and batch processing in a single engine. Data Engineering 38(4) 2015.  P. Carbone A. Katsifodimos S. Ewen V. Markl S. Haridi and K. Tzoumas. Apache Flink: Stream and batch processing in a single engine. Data Engineering 38(4) 2015."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.14778\/2002974.2002977"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/2723372.2723726"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989356"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3078597.3078613"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2463676.2465282"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.14778\/3137765.3137786"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2012.24"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/2611286.2611294"},{"key":"e_1_2_1_19_1","first-page":"1","volume-title":"Proceedings of the 5th ACM SIGMOD Workshop on Algorithms and Systems for MapReduce and Beyond","author":"Hoffmann M."},{"key":"e_1_2_1_20_1","first-page":"783","volume-title":"13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18)","author":"Kalavri V.","year":"2018"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.14778\/3231751.3231765"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2517349.2522738"},{"key":"e_1_2_1_23_1","unstructured":"NEXMark benchmark. http:\/\/datalab.cs.pdx.edu\/niagaraST\/NEXMark.  NEXMark benchmark. http:\/\/datalab.cs.pdx.edu\/niagaraST\/NEXMark."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/3173162.3173170"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2452376.2452384"},{"key":"e_1_2_1_26_1","first-page":"25","volume-title":"ICDE","author":"Shah M. A.","year":"2002"},{"key":"e_1_2_1_27_1","unstructured":"P. Tucker K. Tufte V. Papadimos and D. Maier. NEXMark---A Benchmark for Queries over Data Streams DRAFT. Technical report OGI School of Science & Engineering at OHSU 2002.  P. Tucker K. Tufte V. Papadimos and D. Maier. NEXMark---A Benchmark for Queries over Data Streams DRAFT. Technical report OGI School of Science & Engineering at OHSU 2002."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2015.7113328"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/2517349.2522737"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1007568.1007617"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3329772.3329777","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:35:55Z","timestamp":1672223755000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3329772.3329777"}},"subtitle":["latency-conscious state migration for distributed streaming dataflows"],"short-title":[],"issued":{"date-parts":[[2019,5]]},"references-count":30,"journal-issue":{"issue":"9","published-print":{"date-parts":[[2019,5]]}},"alternative-id":["10.14778\/3329772.3329777"],"URL":"https:\/\/doi.org\/10.14778\/3329772.3329777","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2019,5]]}}}