{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T15:55:34Z","timestamp":1776182134698,"version":"3.50.1"},"reference-count":7,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2015,8]]},"abstract":"<jats:p>Apache Kafka is a scalable publish-subscribe messaging system with its core architecture as a distributed commit log. It was originally built at LinkedIn as its centralized event pipelining platform for online data integration tasks. Over the past years developing and operating Kafka, we extend its log-structured architecture as a replicated logging backbone for much wider application scopes in the distributed environment. In this abstract, we will talk about our design and engineering experience to replicate Kafka logs for various distributed data-driven systems at LinkedIn, including source-of-truth data storage and stream processing.<\/jats:p>","DOI":"10.14778\/2824032.2824063","type":"journal-article","created":{"date-parts":[[2015,9,16]],"date-time":"2015-09-16T12:18:17Z","timestamp":1442405897000},"page":"1654-1655","source":"Crossref","is-referenced-by-count":125,"title":["Building a replicated logging system with Apache Kafka"],"prefix":"10.14778","volume":"8","author":[{"given":"Guozhang","family":"Wang","sequence":"first","affiliation":[{"name":"LinkedIn Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joel","family":"Koshy","sequence":"additional","affiliation":[{"name":"LinkedIn Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sriram","family":"Subramanian","sequence":"additional","affiliation":[{"name":"LinkedIn Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kartik","family":"Paramasivam","sequence":"additional","affiliation":[{"name":"LinkedIn Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mammad","family":"Zadeh","sequence":"additional","affiliation":[{"name":"LinkedIn Corporation"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Neha","family":"Narkhede","sequence":"additional","affiliation":[{"name":"Confluent Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jun","family":"Rao","sequence":"additional","affiliation":[{"name":"Confluent Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jay","family":"Kreps","sequence":"additional","affiliation":[{"name":"Confluent Inc."}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joe","family":"Stein","sequence":"additional","affiliation":[{"name":"Big Data Open Source Security LLC"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2015,8]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Apache Kafka. http:\/\/kafka.apache.org.  Apache Kafka. http:\/\/kafka.apache.org."},{"key":"e_1_2_1_2_1","unstructured":"Apache Samza. http:\/\/samza.apache.org.  Apache Samza. http:\/\/samza.apache.org."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2535930"},{"key":"e_1_2_1_4_1","volume-title":"USENIX ATC","author":"Hunt P.","year":"2010","unstructured":"P. Hunt , M. Konar , F. P. Junqueira , and B. Reed . Zookeeper: Wait-free coordination for internet-scale systems . In USENIX ATC , 2010 . P. Hunt, M. Konar, F. P. Junqueira, and B. Reed. Zookeeper: Wait-free coordination for internet-scale systems. In USENIX ATC, 2010."},{"key":"e_1_2_1_6_1","first-page":"305","volume-title":"USENIX ATC","author":"Ongaro D.","year":"2014","unstructured":"D. Ongaro and J. K. Ousterhout . In search of an understandable consensus algorithm . In USENIX ATC , pages 305 -- 319 , 2014 . D. Ongaro and J. K. Ousterhout. In search of an understandable consensus algorithm. In USENIX ATC, pages 305--319, 2014."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1713254.1713276"},{"key":"e_1_2_1_8_1","first-page":"1135","volume-title":"SIGMOD","author":"Qiao L.","year":"2013","unstructured":"L. Qiao On brewing fresh Espresso: Linkedin's distributed data serving platform . In SIGMOD , pages 1135 -- 1146 , 2013 . 10.1145\/2463676.2465298 L. Qiao et al. On brewing fresh Espresso: Linkedin's distributed data serving platform. In SIGMOD, pages 1135--1146, 2013. 10.1145\/2463676.2465298"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/2824032.2824063","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T10:03:08Z","timestamp":1672221788000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/2824032.2824063"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,8]]},"references-count":7,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2015,8]]}},"alternative-id":["10.14778\/2824032.2824063"],"URL":"https:\/\/doi.org\/10.14778\/2824032.2824063","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2015,8]]}}}