{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,17]],"date-time":"2025-09-17T03:13:18Z","timestamp":1758078798226,"version":"3.44.0"},"reference-count":51,"publisher":"Association for Computing Machinery (ACM)","issue":"12","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2025,8]]},"abstract":"<jats:p>High-reliability distributed coordination services have become an indispensable part of modern large-scale distributed systems. Popular coordination services (e.g., ZooKeeper) adopt a single-writer design to provide a centralized service for managing system metadata, including various configuration information and data catalogs, and to provide distributed synchronization functions. With the continuous increase in metadata size and the scale of distributed systems, these coordination services gradually become performance bottlenecks due to their limitations in capacity, read and write performance, and scalability.<\/jats:p>\n          <jats:p>To bridge the gaps, we propose FDBKeeper, a novel solution that enables scalable coordination services on distributed ACID key-value database systems. Our motivation is that transactional key-value stores (i.e., FoundationDB) meet the demands of performance and scalability required by large-scale distributed systems over coordination service. To leverage these advantages, coordination services can be implemented as an upper layer on top of distributed ACID key-value databases. Our experimental results demonstrate that FDBKeeper significantly outperforms ZooKeeper across key metrics. Additionally, FDBKeeper reduces hardware resource costs on average by 33% in the production environment, resulting in substantial monetary cost savings. We have successfully replaced ZooKeeper with FDBKeeper in the production-grade ClickHouse cluster deployment.<\/jats:p>","DOI":"10.14778\/3750601.3750623","type":"journal-article","created":{"date-parts":[[2025,9,16]],"date-time":"2025-09-16T13:38:05Z","timestamp":1758029885000},"page":"5004-5016","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["FDBKeeper: Enabling Scalable Coordination Services for Metadata Management Using Distributed Key-Value Databases"],"prefix":"10.14778","volume":"18","author":[{"given":"Jun-Peng","family":"Zhu","sequence":"first","affiliation":[{"name":"East China Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lingfeng","family":"Zhang","sequence":"additional","affiliation":[{"name":"East China Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peng","family":"Cai","sequence":"additional","affiliation":[{"name":"East China Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xuan","family":"Zhou","sequence":"additional","affiliation":[{"name":"East China Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Peisen","family":"Zhao","sequence":"additional","affiliation":[{"name":"East China Normal University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xue","family":"Wang","sequence":"additional","affiliation":[{"name":"Moqi Inc, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Linpeng","family":"Tang","sequence":"additional","affiliation":[{"name":"Moqi Inc, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,9,16]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2021. KRaft. https:\/\/kafka.apache.org"},{"key":"e_1_2_1_2_1","unstructured":"2023. ClickHouse. https:\/\/github.com\/ClickHouse\/ClickHouse"},{"key":"e_1_2_1_3_1","volume-title":"Retrieved","year":"2023","unstructured":"2023. Grafana. Retrieved in March, 2023 from https:\/\/grafana.com\/."},{"key":"e_1_2_1_4_1","volume-title":"Retrieved","author":"Exporter Node","year":"2023","unstructured":"2023. Node Exporter. Retrieved in March, 2023 from https:\/\/github.com\/prometheus\/node_exporter."},{"key":"e_1_2_1_5_1","volume-title":"Retrieved","year":"2023","unstructured":"2023. Prometheus. Retrieved in March, 2023 from https:\/\/prometheus.io\/."},{"key":"e_1_2_1_6_1","volume-title":"Retrived","year":"2023","unstructured":"2023. RaftKeeper. Retrived in May, 2023 from https:\/\/github.com\/JDRaftKeeper\/RaftKeeper."},{"key":"e_1_2_1_7_1","volume-title":"Retrived","author":"Consistency DB","year":"2024","unstructured":"2024. FoundationDB Consistency. Retrived in Sep, 2024 from https:\/\/apple.github.io\/foundationdb\/consistency.html,."},{"key":"e_1_2_1_8_1","volume-title":"Retrieved","author":"Pushgateway Prometheus","year":"2024","unstructured":"2024. Prometheus Pushgateway. Retrieved in September, 2024 from https:\/\/github.com\/prometheus\/pushgateway."},{"key":"e_1_2_1_9_1","volume-title":"system-tables-zookeeper. Retrived","year":"2024","unstructured":"2024. system-tables-zookeeper. Retrived in May, 2024 from https:\/\/clickhouse.com\/docs\/en\/operations\/system-tables\/zookeeper."},{"key":"e_1_2_1_10_1","volume-title":"Retrieved in September","author":"DB","year":"2024","unstructured":"2024. YugabyteDB - the cloud native distributed SQL database for mission-criticalapplications. Retrieved in September, 2024 from https:\/\/www.yugabyte.com\/."},{"key":"e_1_2_1_11_1","volume-title":"Retrieved in September","year":"2024","unstructured":"2024. ZooKeeper: A Distributed Coordination Service for Distributed Applications. Retrieved in September, 2024 from https:\/\/zookeeper.apache.org\/doc\/current\/zookeeperOver.html."},{"key":"e_1_2_1_12_1","volume-title":"Retrived","year":"2024","unstructured":"2024. ZooKeeper-watch. Retrived in May, 2024 from https:\/\/zookeeper.apache.org\/doc\/r3.9.1\/zookeeperProgrammers.html#ch_zkWatches."},{"key":"e_1_2_1_13_1","volume-title":"Retrieved in May","year":"2025","unstructured":"2025. 4 letter words command of ZooKeeper. Retrieved in May, 2025 from https:\/\/zookeeper.apache.org\/doc\/r3.9.1\/zookeeperAdmin.html#sc_zkCommands."},{"key":"e_1_2_1_14_1","volume-title":"Retrieved","author":"Coordination Cluster","year":"2025","unstructured":"2025. Cluster Coordination. Retrieved in May, 2025 from https:\/\/github.com\/apple\/foundationdb\/wiki\/Cluster-Coordination."},{"key":"e_1_2_1_15_1","volume-title":"Retrieved","year":"2025","unstructured":"2025. Configuration. Retrieved in May, 2025 from https:\/\/apple.github.io\/foundationdb\/configuration.html."},{"key":"e_1_2_1_16_1","volume-title":"Retrieved in May","year":"2025","unstructured":"2025. FDB Recovery Internals. Retrieved in May, 2025 from https:\/\/github.com\/apple\/foundationdb\/blob\/main\/design\/recovery-internals.md."},{"key":"e_1_2_1_17_1","volume-title":"Known Limitations - FoundationDB. Retrived","year":"2024","unstructured":"2025. Known Limitations - FoundationDB. Retrived in January, 2024 from https:\/\/apple.github.io\/foundationdb\/known-limitations.html."},{"key":"e_1_2_1_18_1","volume-title":"Retrieved","author":"Status Machine-Readable","year":"2025","unstructured":"2025. Machine-Readable Status. Retrieved in May, 2025 from https:\/\/apple.github.io\/foundationdb\/mr-status.html."},{"key":"e_1_2_1_19_1","volume-title":"Retrieved in June","year":"2025","unstructured":"2025. Performance of FoundationDB. Retrieved in June, 2025 from https:\/\/apple.github.io\/foundationdb\/performance.html."},{"key":"e_1_2_1_20_1","volume-title":"Retrived in January","year":"2025","unstructured":"2025. Proof of Linearizability in FDBKeeper. Retrived in January, 2025 from https:\/\/github.com\/DASE-iDDS\/FDBKeeper\/blob\/main\/FDBKeeper_Theorem_Proof.pdf."},{"key":"e_1_2_1_21_1","volume-title":"Retrieved","author":"Cloud Serving Benchmark YCSB","year":"2025","unstructured":"2025. YCSB - Yahoo! Cloud Serving Benchmark. Retrieved in May, 2025 from https:\/\/github.com\/brianfrankcooper\/YCSB."},{"key":"e_1_2_1_22_1","volume-title":"Proceedings of the 27th ACM Symposium on Operating Systems Principles. 353\u2013369","author":"Aghayev Abutalib","year":"2019","unstructured":"Abutalib Aghayev, Sage Weil, Michael Kuchnik, Mark Nelson, Gregory R Ganger, and George Amvrosiadis. 2019. File systems unit as distributed storage backends: lessons from 10 years of Ceph evolution. In Proceedings of the 27th ACM Symposium on Operating Systems Principles. 353\u2013369."},{"key":"e_1_2_1_23_1","volume-title":"truetime and the cap theorem. Google Research","author":"Brewer Eric","year":"2017","unstructured":"Eric Brewer. 2017. Spanner, truetime and the cap theorem. Google Research (2017)."},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.5555\/1298455.1298487"},{"key":"e_1_2_1_25_1","unstructured":"ByteDance. 2023. ByConity. https:\/\/github.com\/ByConity\/ByConity"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.14778\/3685800.3685812"},{"key":"e_1_2_1_27_1","volume-title":"FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an Example. arXiv preprint arXiv:2203.14859","author":"Copik Marcin","year":"2022","unstructured":"Marcin Copik, Alexandru Calotoiu, Pengyu Zhou, Konstantin Taranov, and Torsten Hoefler. 2022. FaaSKeeper: Learning from Building Serverless Services with ZooKeeper as an Example. arXiv preprint arXiv:2203.14859 (2022)."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2882903.2903741"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2022.3170574"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476311.3476385"},{"key":"e_1_2_1_31_1","volume-title":"Retrived in May","year":"2023","unstructured":"etcd. 2023. Etcd. Retrived in May, 2023 from https:\/\/github.com\/etcd-io\/etcd."},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3423138"},{"key":"e_1_2_1_33_1","unstructured":"hashicorp. 2023. Consul. https:\/\/github.com\/hashicorp\/consul"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415535"},{"key":"e_1_2_1_35_1","volume-title":"2010 USENIX Annual Technical Conference (USENIX ATC 10)","author":"Hunt Patrick","year":"2010","unstructured":"Patrick Hunt, Mahadev Konar, Flavio P Junqueira, and Benjamin Reed. 2010. {ZooKeeper}: Wait-free coordination for internet-scale systems. In 2010 USENIX Annual Technical Conference (USENIX ATC 10)."},{"key":"e_1_2_1_36_1","volume-title":"2011IEEE\/IFIP 41st International Conference on Dependable Systems & Networks (DSN). IEEE, 245\u2013256","author":"Junqueira Flavio P","year":"2011","unstructured":"Flavio P Junqueira, Benjamin C Reed, and Marco Seraini. 2011. Zab: High-performance broadcast for primary-backup systems. In 2011IEEE\/IFIP 41st International Conference on Dependable Systems & Networks (DSN). IEEE, 245\u2013256."},{"key":"e_1_2_1_37_1","volume-title":"2016 USENIX Annual Technical Conference (USENIX ATC 16)","author":"Lev-Ari Kfir","year":"2016","unstructured":"Kfir Lev-Ari, Edward Bortnikov, Idit Keidar, and Alexander Shraer. 2016. Modular composition of coordination services. In 2016 USENIX Annual Technical Conference (USENIX ATC 16). 251\u2013264."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2018.2842183"},{"key":"e_1_2_1_39_1","volume-title":"Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 1\u201312","author":"Li Siyang","year":"2017","unstructured":"Siyang Li, Youyou Lu, Jiwu Shu, Yang Hu, and Tao Li. 2017. Locofs: A loosely-coupled metadata service for distributed file systems. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. 1\u201312."},{"key":"e_1_2_1_40_1","volume-title":"20th USENIX Conference on File and Storage Technologies (FAST 22)","author":"Lv Wenhao","year":"2022","unstructured":"Wenhao Lv, Youyou Lu, Yiming Zhang, Peile Duan, and Jiwu Shu. 2022. {InfiniFS}: An efficient metadata service for {Large-Scale} distributed filesystems. In 20th USENIX Conference on File and Storage Technologies (FAST 22). 313\u2013328."},{"key":"e_1_2_1_41_1","volume-title":"15th USENIX Conference on File and Storage Technologies (FAST 17)","author":"Niazi Salman","year":"2017","unstructured":"Salman Niazi, Mahmoud Ismail, Seif Haridi, Jim Dowling, Steffen Grohsschmiedt, and Mikael Ronstr\u00f6m. 2017. {HopsFS}: Scaling hierarchical file system metadata using {NewSQL} databases. In 15th USENIX Conference on File and Storage Technologies (FAST 17). 89\u2013104."},{"key":"e_1_2_1_42_1","unstructured":"Diego Ongaro and John Ousterhout. 2014. In search of an understandable consensus algorithm. In 2014 USENIX annual technical conference (USENIX ATC 14). 305\u2013319."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2014.25"},{"key":"e_1_2_1_44_1","unstructured":"Tom Schreiber and Derek Chia. [n.d.]. ClickHouse Keeper: A ZooKeeper Alternative Written in C++. https:\/\/clickhouse.com\/blog\/clickhouse-keeper-a-zookeeper-alternative-written-in-cpp"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2010.5496972"},{"key":"e_1_2_1_46_1","unstructured":"Avi Silberschatz Henry F. Korth and S. Sudarshan. 2020. Database System Concepts Seventh Edition. McGraw-Hill Book Company. https:\/\/www.db-book.com\/"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3318464.3386134"},{"key":"e_1_2_1_48_1","volume-title":"2017 IEEE International Conference on Cluster Computing (CLUSTER). IEEE, 359\u2013369","author":"Tang Houjun","year":"2017","unstructured":"Houjun Tang, Suren Byna, Bin Dong, Jialin Liu, and Quincey Koziol. 2017. Someta: Scalable object-centric metadata management for high performance computing. In 2017 IEEE International Conference on Cluster Computing (CLUSTER). IEEE, 359\u2013369."},{"key":"e_1_2_1_49_1","volume-title":"2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 1174\u20131183","author":"Wang Teng","year":"2017","unstructured":"Teng Wang, Adam Moody, Yue Zhu, Kathryn Mohror, Kento Sato, Tanzima Islam, and Weikuan Yu. 2017. Metakv: A key-value store for metadata management of distributed burst buffers. In 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, 1174\u20131183."},{"key":"e_1_2_1_50_1","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1109\/TPDS.2022.3232382","article-title":"PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory","volume":"34","author":"Zhang Yiwen","year":"2022","unstructured":"Yiwen Zhang, Jian Zhou, Xinhao Min, Song Ge, Jiguang Wan, Ting Yao, and Daohui Wang. 2022. PetaKV: Building Efficient Key-Value Store for File System Metadata on Persistent Memory. IEEE Transactions on Parallel and Distributed Systems 34, 3 (2022), 843\u2013855.","journal-title":"IEEE Transactions on Parallel and Distributed Systems"},{"key":"e_1_2_1_51_1","volume-title":"Proceedings of the 2021 International Conference on Management of Data. 2653\u20132666","author":"Zhou Jingyu","year":"2021","unstructured":"Jingyu Zhou, Meng Xu, Alexander Shraer, Bala Namasivayam, Alex Miller, Evan Tschannen, Steve Atherton, Andrew J Beamon, Rusty Sears, John Leach, et al. 2021. Foundationdb: A distributed unbundled transactional key value store. In Proceedings of the 2021 International Conference on Management of Data. 2653\u20132666."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3750601.3750623","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,16]],"date-time":"2025-09-16T13:41:36Z","timestamp":1758030096000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3750601.3750623"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8]]},"references-count":51,"journal-issue":{"issue":"12","published-print":{"date-parts":[[2025,8]]}},"alternative-id":["10.14778\/3750601.3750623"],"URL":"https:\/\/doi.org\/10.14778\/3750601.3750623","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2025,8]]},"assertion":[{"value":"2025-09-16","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}