{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T17:05:56Z","timestamp":1781283956658,"version":"3.54.1"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2023,6,13]],"date-time":"2023-06-13T00:00:00Z","timestamp":1686614400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Manag. Data"],"published-print":{"date-parts":[[2023,6,13]]},"abstract":"<jats:p>As in the general industry, there is a trend in Meta's data centers to migrate data from locally attached SSDs to cloud storage. We extended RocksDB [26], a widely used open-source storage engine designed and built for local SSDs, to leverage disaggregated storage. RocksDB's design, such as its data and log files' access patterns, makes an append-only distributed file system a desirable underlying storage. At Meta, we built disaggregated RocksDB using Tectonic File System [35], which so far had mainly been used for our data warehouse and blob storage stacks. We identified that metadata overhead and tail latencies were Tectonic's major performance gaps and addressed them accordingly. We improved the reliability, performance and other requirements with both general and customized optimizations to the core engine in RocksDB. We also took the time to deeply understand the common challenges presented by applications running on RocksDB and implemented enhancements to address them. This architecture enabled RocksDB to adapt to a more distributed architecture for performance enhancements.<\/jats:p>","DOI":"10.1145\/3589772","type":"journal-article","created":{"date-parts":[[2023,6,20]],"date-time":"2023-06-20T20:26:45Z","timestamp":1687292805000},"page":"1-24","source":"Crossref","is-referenced-by-count":21,"title":["Disaggregating RocksDB: A Production Experience"],"prefix":"10.1145","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0576-2226","authenticated-orcid":false,"given":"Siying","family":"Dong","sequence":"first","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-4528-3857","authenticated-orcid":false,"given":"Shiva Shankar","family":"P","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8515-5233","authenticated-orcid":false,"given":"Satadru","family":"Pan","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-7734-6741","authenticated-orcid":false,"given":"Anand","family":"Ananthabhotla","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-3564-5185","authenticated-orcid":false,"given":"Dhanabal","family":"Ekambaram","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-8662-6067","authenticated-orcid":false,"given":"Abhinav","family":"Sharma","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-4459-9958","authenticated-orcid":false,"given":"Shobhit","family":"Dayal","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-4612-9010","authenticated-orcid":false,"given":"Nishant Vinaybhai","family":"Parikh","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-5463-2125","authenticated-orcid":false,"given":"Yanqin","family":"Jin","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-5200-8095","authenticated-orcid":false,"given":"Albert","family":"Kim","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-6800-0123","authenticated-orcid":false,"given":"Sushil","family":"Patil","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-0455-9564","authenticated-orcid":false,"given":"Jay","family":"Zhuang","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-4818-8396","authenticated-orcid":false,"given":"Sam","family":"Dunster","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-1200-5948","authenticated-orcid":false,"given":"Akanksha","family":"Mahajan","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5051-1624","authenticated-orcid":false,"given":"Anirudh","family":"Chelluri","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-4128-7361","authenticated-orcid":false,"given":"Chaitanya","family":"Datye","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-6831-0139","authenticated-orcid":false,"given":"Lucas Vasconcelos","family":"Santana","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-8458-2134","authenticated-orcid":false,"given":"Nitin","family":"Garg","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-0245-8877","authenticated-orcid":false,"given":"Omkar","family":"Gawde","sequence":"additional","affiliation":[{"name":"Meta, Menlo Park, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,6,20]]},"reference":[{"key":"e_1_2_2_1_1","unstructured":"[n. d.]. Amazon EBS. https:\/\/aws.amazon.com\/ebs\/."},{"key":"e_1_2_2_2_1","unstructured":"[n. d.]. Cachelib Repo. https:\/\/github.com\/facebook\/CacheLib."},{"key":"e_1_2_2_3_1","unstructured":"[n. d.]. Ceph File system. https:\/\/docs.ceph.com\/en\/pacific\/cephfs\/index.html."},{"key":"e_1_2_2_4_1","unstructured":"[n. d.]. Distributed locks with Redis. https:\/\/redis.io\/topics\/distlock."},{"key":"e_1_2_2_5_1","unstructured":"[n. d.]. GlusterFS. https:\/\/www.gluster.org\/."},{"key":"e_1_2_2_6_1","unstructured":"[n. d.]. Hbase. https:\/\/hbase.apache.org\/."},{"key":"e_1_2_2_7_1","unstructured":"[n. d.]. HDFS. https:\/\/hadoop.apache.org\/docs\/r1.2.1\/hdfs_design.html."},{"key":"e_1_2_2_8_1","unstructured":"[n. d.]. MySQL. https:\/\/www.mysql.com\/."},{"key":"e_1_2_2_9_1","unstructured":"[n. d.]. RDMA. http:\/\/www.rdmaconsortium.org\/."},{"key":"e_1_2_2_10_1","unstructured":"[n. d.]. RocksDB Benchmark Wiki Page. https:\/\/github.com\/facebook\/rocksdb\/wiki\/Performance-Benchmarks."},{"key":"e_1_2_2_11_1","unstructured":"2009. Rados. https:\/\/ceph.io\/en\/news\/blog\/2009\/the-rados-distributed-object-store\/."},{"key":"e_1_2_2_12_1","unstructured":"2015. Introduction to HDFS Erasure Coding in Apache Hadoop. https:\/\/blog.cloudera.com\/introduction-to-hdfs-erasure-coding-in-apache-hadoop\/."},{"key":"e_1_2_2_13_1","unstructured":"2020. RocksDB-Cloud remote compaction. https:\/\/rockset.com\/blog\/remote-compactions-in-rocksdb-cloud\/."},{"key":"e_1_2_2_14_1","unstructured":"2021. Cachelib. https:\/\/engineering.fb.com\/2021\/09\/02\/open-source\/cachelib\/."},{"key":"e_1_2_2_15_1","unstructured":"2021. How we built a general purpose key value store for Facebook with ZippyDB. https:\/\/engineering.fb.com\/2021\/08\/06\/core-data\/zippydb\/."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.14778\/2757807.2757810"},{"key":"e_1_2_2_17_1","volume-title":"14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20)","author":"Berg Benjamin","unstructured":"Benjamin Berg, Daniel S. Berger, Sara McAllister, Isaac Grosof, Sathya Gunasekar, Jimmy Lu, Michael Uhlar, Jim Carrig, Nathan Beckmann, Mor Harchol-Balter, and Gregory R. Ganger. 2020. The CacheLib Caching Engine: Design and Experiences at Scale. In 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20). USENIX Association, 753--768. https:\/\/www.usenix.org\/conference\/osdi20\/presentation\/berg"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3373376.3378504"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3190508.3190532"},{"key":"e_1_2_2_20_1","volume-title":"18th USENIX Conference on File and Storage Technologies (FAST 20)","author":"Cao Wei","year":"2020","unstructured":"Wei Cao, Yang Liu, Zhushi Cheng, Ning Zheng, Wei Li, Wenjie Wu, Linqiang Ouyang, Peng Wang, Yijing Wang, Ray Kuan, et al. 2020. {POLARDB} Meets Computational Storage: Efficiently Support Analytical Workloads in {Cloud-Native} Relational Database. In 18th USENIX Conference on File and Storage Technologies (FAST 20). 29--41."},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1365815.1365816"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/2491245"},{"key":"e_1_2_2_23_1","doi-asserted-by":"crossref","unstructured":"Jeffrey Dean. 2010. Evolution and future directions of large-scale storage and computation systems at Google. (2010).","DOI":"10.1145\/1807128.1807130"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/129888.129894"},{"key":"e_1_2_2_25_1","first-page":"3","article-title":"Optimizing Space Amplification in RocksDB","volume":"3","author":"Dong Siying","year":"2017","unstructured":"Siying Dong, Mark Callaghan, Leonidas Galanis, Dhruba Borthakur, Tony Savor, and Michael Strum. 2017. Optimizing Space Amplification in RocksDB.. In CIDR, Vol. 3. 3.","journal-title":"CIDR"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3483840"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/3026877.3026897"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/945445.945450"},{"key":"e_1_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3239563"},{"key":"e_1_2_2_30_1","volume-title":"USENIX winter","author":"Hitz Dave","unstructured":"Dave Hitz, James Lau, and Michael A Malcolm. 1994. File System Design for an NFS File Server Appliance.. In USENIX winter, Vol. 94. 10--5555."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2901318.2901337"},{"key":"e_1_2_2_32_1","volume-title":"14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17)","author":"Nanavati Mihir","year":"2017","unstructured":"Mihir Nanavati, Jake Wires, and Andrew Warfield. 2017. Decibel: Isolation and Sharing in Disaggregated {Rack-Scale} Storage. In 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). 17--33."},{"key":"e_1_2_2_33_1","volume-title":"10th USENIX Symposium on Operating Systems Design and Implementation (OSDI 12)","author":"Nightingale Edmund B","year":"2012","unstructured":"Edmund B Nightingale, Jeremy Elson, Jinliang Fan, Owen Hofmann, Jon Howell, and Yutaka Suzue. 2012. Flat datacenter storage. In 10th USENIX Symposium on Operating Systems Design and Implementation (OSDI 12). 1--15."},{"key":"e_1_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1007\/s002360050048"},{"key":"e_1_2_2_35_1","volume-title":"19th USENIX Conference on File and Storage Technologies (FAST 21)","author":"Pan Satadru","year":"2021","unstructured":"Satadru Pan, Theano Stavrinos, Yunqiao Zhang, Atul Sikaria, Pavel Zakharov, Abhinav Sharma, Shiva Shankar P, Mike Shuey, Richard Wareing, Monika Gangapuram, Guanglei Cao, Christian Preseau, Pratap Singh, Kestutis Patiejunas, JR Tipton, Ethan Katz-Bassett, and Wyatt Lloyd. 2021. Facebook's Tectonic Filesystem: Efficiency from Exascale. In 19th USENIX Conference on File and Storage Technologies (FAST 21). USENIX Association, 217--231. https:\/\/www.usenix.org\/conference\/fast21\/presentation\/pan"},{"key":"e_1_2_2_36_1","unstructured":"Rockset. 2018. RocksDBCloud. https:\/\/rockset.com\/blog\/rocksdb-cloud-enabling-the-next-generation-of-cloud-native-databases\/."},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2815400.2815408"},{"key":"e_1_2_2_38_1","volume-title":"Distributed OS for Hardware Resource Disaggregation. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18)","author":"Shan Yizhou","year":"2018","unstructured":"Yizhou Shan, Yutong Huang, Yilun Chen, and Yiying Zhang. 2018. LegoOS: A Disseminated, Distributed OS for Hardware Resource Disaggregation. In 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18). USENIX Association, Carlsbad, CA, 69--87. https:\/\/www.usenix.org\/conference\/osdi18\/presentation\/shan"},{"key":"e_1_2_2_39_1","first-page":"4","article-title":"The case for shared nothing","volume":"9","author":"Stonebraker Michael","year":"1986","unstructured":"Michael Stonebraker. 1986. The case for shared nothing. IEEE Database Eng. Bull. 9, 1 (1986), 4--9.","journal-title":"IEEE Database Eng. Bull."},{"key":"e_1_2_2_40_1","volume-title":"Live recovery of bit corruptions in datacenter storage systems. arXiv preprint arXiv:1805.02790","author":"Tai Amy","year":"2018","unstructured":"Amy Tai, Andrew Kryczka, Shobhit Kanaujia, Chris Petersen, Mikhail Antonov, Muhammad Waliji, Kyle Jamieson, Michael J Freedman, and Asaf Cidon. 2018. Live recovery of bit corruptions in datacenter storage systems. arXiv preprint arXiv:1805.02790 (2018)."},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3056101"},{"key":"e_1_2_2_42_1","volume-title":"17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20)","author":"Vuppalapati Midhul","year":"2020","unstructured":"Midhul Vuppalapati, Justin Miron, Rachit Agarwal, Dan Truong, Ashish Motivala, and Thierry Cruanes. 2020. Building an elastic query engine on disaggregated storage. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20). 449--462."}],"container-title":["Proceedings of the ACM on Management of Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3589772","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3589772","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:49:22Z","timestamp":1750182562000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3589772"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,13]]},"references-count":42,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,6,13]]}},"alternative-id":["10.1145\/3589772"],"URL":"https:\/\/doi.org\/10.1145\/3589772","relation":{},"ISSN":["2836-6573"],"issn-type":[{"value":"2836-6573","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,13]]}}}