{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,28]],"date-time":"2026-01-28T21:48:43Z","timestamp":1769636923703,"version":"3.49.0"},"reference-count":56,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2018,5,31]],"date-time":"2018-05-31T00:00:00Z","timestamp":1527724800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NSF-sponsored Institute for Computational Sustainability at Cornell University, NSF, DARPA, ARPA-e, NYSERDA, Huawei, and Mellanox"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Syst."],"published-print":{"date-parts":[[2018,5,31]]},"abstract":"<jats:p>Cloud computing services often replicate data and may require ways to coordinate distributed actions. Here we present Derecho, a library for such tasks. The API provides interfaces for structuring applications into patterns of subgroups and shards, supports state machine replication within them, and includes mechanisms that assist in restart after failures. Running over 100Gbps RDMA, Derecho can send millions of events per second in each subgroup or shard and throughput peaks at 16GB\/s, substantially outperforming prior solutions. Configured to run purely on TCP, Derecho is still substantially faster than comparable widely used, highly-tuned, standard tools. The key insight is that on modern hardware (including non-RDMA networks), data-intensive protocols should be built from non-blocking data-flow components.<\/jats:p>","DOI":"10.1145\/3302258","type":"journal-article","created":{"date-parts":[[2019,4,4]],"date-time":"2019-04-04T18:38:37Z","timestamp":1554403117000},"page":"1-49","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":29,"title":["Derecho"],"prefix":"10.1145","volume":"36","author":[{"given":"Sagar","family":"Jha","sequence":"first","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan","family":"Behrens","sequence":"additional","affiliation":[{"name":"Cornell University, USA and MIT, Cambridge, MA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Theo","family":"Gkountouvas","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mae","family":"Milano","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weijia","family":"Song","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Edward","family":"Tremel","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Robbert Van","family":"Renesse","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sydney","family":"Zink","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kenneth P.","family":"Birman","sequence":"additional","affiliation":[{"name":"Cornell University, Ithaca, NY, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2019,4,2]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"{n. d.}. LibPaxos: Open-source Paxos. Retrieved from http:\/\/libpaxos.sourceforge.net\/."},{"key":"e_1_2_1_2_1","unstructured":"{n. d.}. RDMA-Paxos: Open-source Paxos. Retrieved from https:\/\/github.com\/wangchenghku\/RDMA-PAXOS."},{"key":"e_1_2_1_3_1","unstructured":"2011. Vsync Reliable Multicast Library. Retrieved from http:\/\/vsync.codeplex.com\/."},{"key":"e_1_2_1_4_1","unstructured":"2012. Gbcast Protocol. Retrieved from https:\/\/en.wikipedia.org\/wiki\/Gbcast."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/2535930"},{"key":"e_1_2_1_6_1","unstructured":"Bela Ban. 2002. JGroups Reliable Multicast Library. Retrieved from http:\/\/jgroups.org\/."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/1629575.1629579"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 2018 48th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN\u201918)","author":"Birman Jonathan","year":"2018","unstructured":"Behrens, Jonathan and Birman, Ken and Jha, Sagar and Tremel, Edward. 2018. RDMC: A reliable RDMA multicast for large objects. In Proceedings of the 2018 48th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN\u201918). IEEE Computer Society, Los Alamitos, CA, 1--12."},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201914)","author":"Belay Adam","year":"2014","unstructured":"Adam Belay, George Prekas, Ana Klimovic, Samuel Grossman, Christos Kozyrakis, and Edouard Bugnion. 2014. IX: A protected dataplane operating system for high throughput and low latency. In Proceedings of the 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201914). USENIX Association, Broomfield, CO, 49--65."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/2222848"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/323647.323636"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/2222848"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/41457.37515"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/7351.7478"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1835698.1835701"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043556.2043571"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1281100.1281103"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/226643.226647"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/214451.214456"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/989.357400"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2391229.2391230"},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201912)","author":"Corbett James C.","year":"2012","unstructured":"James C. Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, J. J. Furman, Sanjay Ghemawat, Andrey Gubarev, Christopher Heiser, Peter Hochschild, Wilson Hsieh, Sebastian Kanthak, Eugene Kogan, Hongyi Li, Alexander Lloyd, Sergey Melnik, David Mwaura, David Nagle, Sean Quinlan, Rajesh Rao, Lindsay Rolig, Yasushi Saito, Michal Szymaniak, Christopher Taylor, Ruth Wang, and Dale Woodford. 2012. Spanner: Google\u2019s globally-distributed database. In Proceedings of the 10th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201912). USENIX Association, Berkeley, CA, 251--264. http:\/\/dl.acm.org\/citation.cfm?id&equals;2387880.2387905"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/2815400.2815427"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2774993.2774999"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (NSDI\u201914)","author":"Dragojevi\u0107 Aleksandar","year":"2014","unstructured":"Aleksandar Dragojevi\u0107, Dushyanth Narayanan, Orion Hodson, and Miguel Castro. 2014. FaRM: Fast remote memory. In Proceedings of the 11th USENIX Conference on Networked Systems Design and Implementation (NSDI\u201914). USENIX Association, Berkeley, CA, 401--414. http:\/\/dl.acm.org\/citation.cfm?id&equals;2616448.2616486"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3149.214121"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/822082.823140"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2005.53"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/564585.564601"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/1813654.1813656"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2934872.2934908"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/79147.79161"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/2619239.2626299"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/1146381.1146408"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/279227.279229"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/383962.383969"},{"key":"e_1_2_1_37_1","volume-title":"Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201916)","author":"Li Jialin","year":"2016","unstructured":"Jialin Li, Ellis Michael, Naveen Kr Sharma, Adriana Szekeres, and Dan RK Ports. 2016. Just say NO to Paxos overhead: Replacing consensus with network ordering. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201916)."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/SRDS.2014.15"},{"key":"e_1_2_1_39_1","unstructured":"D. Mazieres. 2007. Paxos made practical. Technical report. Retrieved from http:\/\/www.scs.stanford.edu\/dm\/home\/papers."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/2535461.2535475"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.5555\/2643634.2643666"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/1965724.1965751"},{"key":"e_1_2_1_43_1","volume-title":"Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201914)","author":"Peter Simon","year":"2014","unstructured":"Simon Peter, Jialin Li, Irene Zhang, Dan R. K. Ports, Doug Woos, Arvind Krishnamurthy, Thomas Anderson, and Timothy Roscoe. 2014. Arrakis: The operating system is the control plane. In Proceedings of the 11th USENIX Conference on Operating Systems Design and Implementation (OSDI\u201914). USENIX Association, Berkeley, CA, 1--16."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2749246.2749267"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3078597.3078598"},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the 11th International Workshop on Distributed Algorithms (WDAG\u201997)","author":"Prisco Roberto De","unstructured":"Roberto De Prisco, Butler W. Lampson, and Nancy A. Lynch. 1997. Revisiting the paxos algorithm. In Proceedings of the 11th International Workshop on Distributed Algorithms (WDAG\u201997). Springer-Verlag, London, 111--125."},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/1394127.1394128"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/98163.98167"},{"key":"e_1_2_1_49_1","volume-title":"Google\u2019s Jeff Dean spotlights data center inner workings. C|Net Reviews (May","author":"Shankland Steve","year":"2008","unstructured":"Steve Shankland. 2008. Google\u2019s Jeff Dean spotlights data center inner workings. C|Net Reviews (May 2008). Retrieved from https:\/\/www.cnet.com\/news\/google-spotlights-data-center-inner-workings\/."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2987550.2987578"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1145\/2673577"},{"key":"e_1_2_1_52_1","volume-title":"Proceedings of the 6th Conference on Symposium on Operating Systems Design 8 Implementation","volume":"6","author":"van Renesse Robbert","unstructured":"Robbert van Renesse and Fred B. Schneider. 2004. Chain replication for supporting high throughput and availability. In Proceedings of the 6th Conference on Symposium on Operating Systems Design 8 Implementation, Volume 6 (OSDI\u201904). USENIX Association, Berkeley, CA, USA, 7--7."},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/224056.224061"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3127479.3128609"},{"key":"e_1_2_1_55_1","volume-title":"Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201917)","author":"Wei Michael","year":"2017","unstructured":"Michael Wei, Amy Tai, Christopher J. Rossbach, Ittai Abraham, Maithem Munshed, Medhavi Dhawan, Jim Stabile, Udi Wieder, Scott Fritchie, Steven Swanson, Michael J. Freedman, and Dahlia Malkhi. 2017. vCorfu: A cloud-scale object store on a shared log. In Proceedings of the 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201917). USENIX Association, 35--49."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2815400.2815419"}],"container-title":["ACM Transactions on Computer Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3302258","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3302258","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T01:01:48Z","timestamp":1750208508000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3302258"}},"subtitle":["Fast State Machine Replication for Cloud Services"],"short-title":[],"issued":{"date-parts":[[2018,5,31]]},"references-count":56,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2018,5,31]]}},"alternative-id":["10.1145\/3302258"],"URL":"https:\/\/doi.org\/10.1145\/3302258","relation":{},"ISSN":["0734-2071","1557-7333"],"issn-type":[{"value":"0734-2071","type":"print"},{"value":"1557-7333","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,5,31]]},"assertion":[{"value":"2017-09-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2018-12-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2019-04-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}