{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T01:17:47Z","timestamp":1778807867810,"version":"3.51.4"},"reference-count":27,"publisher":"Association for Computing Machinery (ACM)","issue":"10","license":[{"start":{"date-parts":[[2016,9,22]],"date-time":"2016-09-22T00:00:00Z","timestamp":1474502400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Commun. ACM"],"published-print":{"date-parts":[[2016,9,22]]},"abstract":"<jats:p>We describe the timely dataflow model for distributed computation and its implementation in the Naiad system. The model supports stateful iterative and incremental computations. It enables both low-latency stream processing and high-throughput batch processing, using a new approach to coordination that combines asynchronous and fine-grained synchronous execution. We describe two of the programming frameworks built on Naiad: GraphLINQ for parallel graph processing, and differential dataflow for nested iterative and incremental computations. We show that a general-purpose system can achieve performance that matches, and sometimes exceeds, that of specialized systems.<\/jats:p>","DOI":"10.1145\/2983551","type":"journal-article","created":{"date-parts":[[2016,9,22]],"date-time":"2016-09-22T17:55:52Z","timestamp":1474566952000},"page":"75-83","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":29,"title":["Incremental, iterative data processing with timely dataflow"],"prefix":"10.1145","volume":"59","author":[{"given":"Derek G.","family":"Murray","sequence":"first","affiliation":[{"name":"Google, Mountain View, CA,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Frank","family":"McSherry","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Michael","family":"Isard","sequence":"additional","affiliation":[{"name":"Google, Mountain View, CA,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rebecca","family":"Isaacs","sequence":"additional","affiliation":[{"name":"Google, Mountain View, CA,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Paul","family":"Barham","sequence":"additional","affiliation":[{"name":"Google, Mountain View, CA,"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Martin","family":"Abadi","sequence":"additional","affiliation":[{"name":"Google, Mountain View, CA,"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2016,9,22]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Proc. FORTE","author":"Abadi M.","year":"2015"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-17524-9_3"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536222.2536229"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.14778\/1687627.1687655"},{"key":"e_1_2_1_5_1","volume-title":"Proc. OSDI (Nov.","author":"Chang F.","year":"2006"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1327452.1327492"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/129888.129894"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.14778\/2350229.2350245"},{"key":"e_1_2_1_9_1","volume-title":"Proc. HotOS (May","author":"Gog I.","year":"2015"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/2741948.2741968"},{"key":"e_1_2_1_11_1","volume-title":"Proc. OSDI (Oct.","author":"Gonzalez J.E.","year":"2012"},{"key":"e_1_2_1_12_1","volume-title":"Proc. OSDI (Oct.","author":"Gonzalez J.E.","year":"2014"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1272996.1273005"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/PROC.1987.13876"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/2685048.2685095"},{"key":"e_1_2_1_16_1","volume-title":"Proc. HotOS (May","author":"McSherry F.","year":"2015"},{"key":"e_1_2_1_17_1","volume-title":"Proc. CIDR (Jan.","author":"McSherry F.","year":"2013"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.14778\/1920841.1920886"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/2517349.2522738"},{"key":"e_1_2_1_20_1","volume-title":"Proc. NSDI (Mar.","author":"Murray D.G.","year":"2011"},{"key":"e_1_2_1_21_1","volume-title":"Proc. OSDI (Oct.","author":"Peng D.","year":"2010"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2594291.2594305"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/151646.151647"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2003.1198390"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1629575.1629600"},{"key":"e_1_2_1_26_1","volume-title":"Proc. OSDI (Dec.","author":"Yu Y.","year":"2008"},{"key":"e_1_2_1_27_1","volume-title":"Proc. NSDI (Apr.","author":"Zaharia M.","year":"2012"}],"container-title":["Communications of the ACM"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2983551","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2983551","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:56:01Z","timestamp":1750222561000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2983551"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,9,22]]},"references-count":27,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2016,9,22]]}},"alternative-id":["10.1145\/2983551"],"URL":"https:\/\/doi.org\/10.1145\/2983551","relation":{},"ISSN":["0001-0782","1557-7317"],"issn-type":[{"value":"0001-0782","type":"print"},{"value":"1557-7317","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,9,22]]},"assertion":[{"value":"2016-09-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}