{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,12]],"date-time":"2026-06-12T17:05:37Z","timestamp":1781283937784,"version":"3.54.1"},"reference-count":47,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2024,9,14]],"date-time":"2024-09-14T00:00:00Z","timestamp":1726272000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Archit. Code Optim."],"published-print":{"date-parts":[[2024,9,30]]},"abstract":"<jats:p>LSM-based key-value stores suffer from sub-optimal performance due to their slow and heavy background compactions. The compaction brings severe CPU and network overhead on high-speed disaggregated storage. This article further reveals that data-intensive compression in compaction consumes a significant portion of CPU power. Moreover, the multi-threaded compactions cause substantial CPU contention and network traffic during high-load periods. Based on the above observations, we propose fine-grained dynamical compaction offloading by leveraging the modern Data Processing Unit (DPU) to alleviate the CPU and network overhead. To achieve this, we first customized a file system to enable efficient data access for DPU. We then leverage the Arm cores on the DPU to meet the burst CPU and network requirements to reduce resource contention and data movement. We further employ dedicated hardware-based accelerators on the DPU to speed up the compression in compactions. We integrate our DPU-offloaded compaction with RocksDB and evaluate it with NVIDIA\u2019s latest Bluefield-2 DPU on a real system. The evaluation shows that the DPU is an effective solution to solve the CPU bottleneck and reduce data traffic of compaction. The results show that compaction performance is accelerated by 2.86 to 4.03 times, system write and read throughput is improved by up to 3.2 times and 1.4 times respectively, and host CPU contention and network traffic are effectively reduced compared to the fine-tuned CPU-only baseline.<\/jats:p>","DOI":"10.1145\/3656584","type":"journal-article","created":{"date-parts":[[2024,4,9]],"date-time":"2024-04-09T11:52:39Z","timestamp":1712663559000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["D\n            <sup>2<\/sup>\n            Comp: Efficient Offload of LSM-tree Compaction with Data Processing Units on Disaggregated Storage"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0157-6496","authenticated-orcid":false,"given":"Chen","family":"Ding","sequence":"first","affiliation":[{"name":"Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5295-4680","authenticated-orcid":false,"given":"Jian","family":"Zhou","sequence":"additional","affiliation":[{"name":"Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7757-4083","authenticated-orcid":false,"given":"Kai","family":"Lu","sequence":"additional","affiliation":[{"name":"Huazhong University of Science and Technology, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-6075-6431","authenticated-orcid":false,"given":"Sicen","family":"Li","sequence":"additional","affiliation":[{"name":"Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-2113-7014","authenticated-orcid":false,"given":"Yiqin","family":"Xiong","sequence":"additional","affiliation":[{"name":"Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4160-9475","authenticated-orcid":false,"given":"Jiguang","family":"Wan","sequence":"additional","affiliation":[{"name":"Wuhan National Laboratory for Optoelectronics, Huazhong University of Science and Technology, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-9086-2944","authenticated-orcid":false,"given":"Ling","family":"Zhan","sequence":"additional","affiliation":[{"name":"Wenhua College, Wuhan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,9,14]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"2023. NVM Express Over Fabrics Specification. https:\/\/nvmexpress.org\/developers\/nvme-of-specification\/"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341301.3359656"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.14778\/2757807.2757810"},{"key":"e_1_3_2_5_2","unstructured":"AMD. 2023. Alveo U200 FPGA. https:\/\/www.xilinx.com\/products\/boards-and-kits\/alveo\/u200.html#specifications"},{"key":"e_1_3_2_6_2","unstructured":"AMD. 2023. Vitis Data Compression Libraries. https:\/\/docs.amd.com\/r\/en-US\/Vitis_Libraries\/data_compression\/benchmark.html_1"},{"key":"e_1_3_2_7_2","first-page":"753","volume-title":"2019 USENIX Annual Technical Conference (USENIX ATC\u201919)","author":"Balmau Oana","year":"2019","unstructured":"Oana Balmau, Florin Dinu, Willy Zwaenepoel, Karan Gupta, Ravishankar Chandhiramoorthi, and Diego Didona. 2019. SILK: Preventing latency spikes in log-structured merge key-value stores. In 2019 USENIX Annual Technical Conference (USENIX ATC\u201919). 753\u2013766."},{"key":"e_1_3_2_8_2","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1145\/3373376.3378504","volume-title":"Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems","author":"Bindschaedler Laurent","year":"2020","unstructured":"Laurent Bindschaedler, Ashvin Goel, and Willy Zwaenepoel. 2020. Hailstorm: Disaggregated compute and storage for distributed LSM-based databases. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems. 301\u2013316."},{"key":"e_1_3_2_9_2","unstructured":"Broadcom. 2023. Stingray PS1100R SmartNIC. https:\/\/www.broadcom.com\/products\/storage\/ethernet-storage-adapters-ics\/ps1100r"},{"key":"e_1_3_2_10_2","first-page":"1","volume-title":"2021 IEEE Hot Chips 33 Symposium (HCS\u201921)","author":"Burres Brad","year":"2021","unstructured":"Brad Burres, Dan Daly, Mark Debbage, Eliel Louzoun, Christine Severns-Williams, Naru Sundar, Nadav Turbovich, Barry Wolford, and Yadong Li. 2021. Intel\u2019s hyperscale-ready infrastructure processing unit (IPU). In 2021 IEEE Hot Chips 33 Symposium (HCS\u201921). IEEE, 1\u201316."},{"key":"e_1_3_2_11_2","first-page":"1","volume-title":"2021 IEEE Hot Chips 33 Symposium (HCS\u201921)","author":"Burstein Idan","year":"2021","unstructured":"Idan Burstein. 2021. Nvidia data center processing unit (DPU) architecture. In 2021 IEEE Hot Chips 33 Symposium (HCS\u201921). IEEE, 1\u201320."},{"key":"e_1_3_2_12_2","first-page":"17","volume-title":"19th USENIX Conference on File and Storage Technologies (FAST\u201921)","author":"Chen Hao","year":"2021","unstructured":"Hao Chen, Chaoyi Ruan, Cheng Li, Xiaosong Ma, and Yinlong Xu. 2021. SpanDB: A fast, cost-effective LSM-tree based KV store on hybrid storage. In 19th USENIX Conference on File and Storage Technologies (FAST\u201921). 17\u201332."},{"key":"e_1_3_2_13_2","first-page":"53","volume-title":"2023 IEEE 34th International Conference on Application-specific Systems, Architectures and Processors (ASAP\u201923)","author":"Chen Jiahong","year":"2023","unstructured":"Jiahong Chen, Shengzhe Wang, Zhihao Zhang, Suzhen Wu, and Bo Mao. 2023. iKnowFirst: An efficient DPU-assisted compaction for LSM-tree-based key-value stores. In 2023 IEEE 34th International Conference on Application-specific Systems, Architectures and Processors (ASAP\u201923). IEEE, 53\u201360."},{"key":"e_1_3_2_14_2","first-page":"155","volume-title":"Proceedings of the 14th USENIX Conference on Operating Systems Design and Implementation","author":"Dai Yifan","year":"2020","unstructured":"Yifan Dai, Yien Xu, Aishwarya Ganesan, Ramnatthan Alagappan, Brian Kroth, Andrea C. Arpaci-Dusseau, and Remzi H. Arpaci-Dusseau. 2020. From WiscKey to Bourbon: A learned index for log-structured merge trees. In Proceedings of the 14th USENIX Conference on Operating Systems Design and Implementation. 155\u2013171."},{"key":"e_1_3_2_15_2","unstructured":"Vast Data. 2023. DPU Storage Server. https:\/\/vastdata.com\/press-releases\/vast-data-collaborates-with-nvidia-to-build-the-data-foundation-for-ai-superclusters"},{"key":"e_1_3_2_16_2","doi-asserted-by":"crossref","unstructured":"Peter Deutsch. 1996. RFC1951: DEFLATECompressed Data Format Specification Version 1.3. (1996).","DOI":"10.17487\/rfc1951"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2022.3188268"},{"key":"e_1_3_2_18_2","first-page":"33","volume-title":"19th USENIX Conference on File and Storage Technologies (FAST\u201921)","author":"Dong Siying","year":"2021","unstructured":"Siying Dong, Andrew Kryczka, Yanqin Jin, and Michael Stumm. 2021. Evolution of development priorities in key-value stores serving large-scale applications: The RocksDB experience. In 19th USENIX Conference on File and Storage Technologies (FAST\u201921). 33\u201349."},{"key":"e_1_3_2_19_2","unstructured":"Facebook. 2023. Remote Compaction. https:\/\/github.com\/facebook\/rocksdb\/wiki\/Remote-Compaction-(Experimental)."},{"key":"e_1_3_2_20_2","unstructured":"Daniel Firestone Andrew Putnam Sambhrama Mundkur Derek Chiou Alireza Dabagh Mike Andrewartha Hari Angepat Vivek Bhanu Adrian Caulfield Eric Chung Harish Kumar Chandrappa Somesh Chaturmohta Matt Humphrey Jack Lavier Norman Lam Fengfen Liu Kalin Ovtcharov Jitu Padhye Gautham Popuri Shachar Raindel Tejas Sapre Mark Shaw Gabriel Silva Madhan Sivakumar Nisheeth Srivastava Anshuman Verma Qasim Zuhair Deepak Bansal Doug Burger Kushagra Vaid David A. Maltz and Albert Greenberg. 2018. Azure accelerated networking: SmartNICs in the public cloud. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201918). USENIX Association Renton WA 51\u201366. https:\/\/www.usenix.org\/conference\/nsdi18\/presentation\/firestone"},{"key":"e_1_3_2_21_2","unstructured":"Google. 2023. LevelDB. https:\/\/github.com\/google\/leveldb"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3579370.3594769"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3603269.3604880"},{"key":"e_1_3_2_24_2","first-page":"163","volume-title":"2019 USENIX Annual Technical Conference (USENIX ATC\u201919)","author":"Hu Xiaokang","year":"2019","unstructured":"Xiaokang Hu, Fuzong Wang, Weigang Li, Jian Li, and Haibing Guan. 2019. QZFS: QAT accelerated compression in file system for application agnostic and cost efficient data storage. In 2019 USENIX Annual Technical Conference (USENIX ATC\u201919). 163\u2013176."},{"key":"e_1_3_2_25_2","doi-asserted-by":"crossref","unstructured":"Dongxu Huang Qi Liu Qiu Cui Zhuhe Fang Xiaoyu Ma Fei Xu Li Shen Liu Tang Yuxing Zhou Menglong Huang Wan Wei Cong Liu Jian Zhang Jianjun Li Xuelian Wu Lingyu Song Ruoxi Sun Shuaipeng Yu Lei Zhao Nicholas Cameron Liquan Pei and Xin Tang. 2020. TiDB: a Raft-based HTAP database. Proceedings of the VLDB Endowment 13 12 (2020) 3072\u20133084.","DOI":"10.14778\/3415478.3415535"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457297"},{"key":"e_1_3_2_27_2","unstructured":"Intel. 2023. Infrastructure Processing Unit.https:\/\/www.intel.com.au\/content\/www\/au\/en\/products\/network-io\/smartnic.html"},{"key":"e_1_3_2_28_2","first-page":"17","volume-title":"2021 IEEE Symposium on High-Performance Interconnects (HOTI\u201921)","author":"Jain Arpan","year":"2021","unstructured":"Arpan Jain, Nawras Alnaasan, Aamir Shafi, Hari Subramoni, and Dhabaleswar K. Panda. 2021. Accelerating CPU-based distributed DNN training on modern HPC clusters using BlueField-2 DPUs. In 2021 IEEE Symposium on High-Performance Interconnects (HOTI\u201921). IEEE, 17\u201324."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477132.3483565"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341301.3359628"},{"key":"e_1_3_2_31_2","unstructured":"Qiang Li Lulu Chen Xiaoliang Wang Shuo Huang Qiao Xiang Yuanyuan Dong Wenhui Yao Minfei Huang Puyuan Yang Shanyang Liu Zhaosheng Zhu Huayong Wang Haonan Qiu Derui Liu Shaozong Liu Yujie Zhou Yaohui Wu Zhiwu Wu Shang Gao Chao Han Zicheng Luo Yuchao Shao Gexiao Tian Zhongjie Wu Zheng Cao Jinbo Wu Jiwu Shu Jie Wu and Jiesheng Wu. 2023. Fisc: A Large-scale cloud-native-oriented file system. In 21st USENIX Conference on File and Storage Technologies (FAST\u201923). USENIX Association Santa Clara CA 231\u2013246. https:\/\/www.usenix.org\/conference\/fast23\/presentation\/li-qiang-fisc"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3589980"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3341302.3342079"},{"key":"e_1_3_2_34_2","unstructured":"Marvell. 2023. OCTEON 10 DPU.https:\/\/www.marvell.com\/products\/data-processing-units.html"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415546"},{"key":"e_1_3_2_36_2","first-page":"106","volume-title":"Proceedings of the 2021 ACM SIGCOMM 2021 Conference","author":"Min Jaehong","year":"2021","unstructured":"Jaehong Min, Ming Liu, Tapan Chugh, Chenxingyu Zhao, Andrew Wei, In Hwan Doh, and Arvind Krishnamurthy. 2021. Gimbal: Enabling multi-tenant storage disaggregation on SmartNIC JBOFs. In Proceedings of the 2021 ACM SIGCOMM 2021 Conference. 106\u2013122."},{"key":"e_1_3_2_37_2","unstructured":"Nvidia. 2023. Bluefield-2 DPU.https:\/\/www.nvidia.com\/en-us\/networking\/products\/data-processing-unit\/"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1007\/s002360050048"},{"key":"e_1_3_2_39_2","unstructured":"Facebook. 2023. RocksDB a persistent key-value store for fast storage enviroments. http:\/\/rocksdb.org\/"},{"key":"e_1_3_2_40_2","unstructured":"Premio. 2023. DPU Accelerated Server. https:\/\/premioinc.com\/collections\/dpu-server"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477132.3483555"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.17487\/RFC3530"},{"key":"e_1_3_2_43_2","first-page":"1","volume-title":"2022 IEEE International Conference on Cluster Computing (CLUSTER\u201922)","author":"Sun Shangyi","year":"2022","unstructured":"Shangyi Sun, Rui Zhang, Ming Yan, and Jie Wu. 2022. SKV: A SmartNIC-offloaded distributed key-value store. In 2022 IEEE International Conference on Cluster Computing (CLUSTER\u201922). IEEE, 1\u201311."},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","first-page":"2835","DOI":"10.1109\/ICDE55515.2023.00217","volume-title":"2023 IEEE 39th International Conference on Data Engineering (ICDE\u201923)","author":"Wang Ruihong","year":"2023","unstructured":"Ruihong Wang, Jianguo Wang, Prishita Kadam, M. Tamer \u00d6zsu, and Walid G. Aref. 2023. dLSM: An LSM-based index for memory disaggregation. In 2023 IEEE 39th International Conference on Data Engineering (ICDE\u201923). IEEE, 2835\u20132849."},{"key":"e_1_3_2_45_2","doi-asserted-by":"crossref","first-page":"819","DOI":"10.1109\/Cluster48925.2021.00070","volume-title":"2021 IEEE International Conference on Cluster Computing (CLUSTER\u201921)","author":"Wang Xiaoliang","year":"2021","unstructured":"Xiaoliang Wang, Jianchuan Li, Peiquan Jin, Kuankuan Guo, Yuanjin Lin, and Ming Zhao. 2021. Supporting elastic compaction of LSM-tree with a FaaS cluster. In 2021 IEEE International Conference on Cluster Computing (CLUSTER\u201921). IEEE, 819\u2013820."},{"key":"e_1_3_2_46_2","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1109\/CloudCom.2017.14","volume-title":"2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom\u201917)","author":"Yang Ziye","year":"2017","unstructured":"Ziye Yang, James R. Harris, Benjamin Walker, Daniel Verkamp, Changpeng Liu, Cunyin Chang, Gang Cao, Jonathan Stern, Vishal Verma, and Luse E. Paul. 2017. SPDK: A development kit to build high performance storage applications. In 2017 IEEE International Conference on Cloud Computing Technology and Science (CloudCom\u201917). IEEE, 154\u2013161."},{"key":"e_1_3_2_47_2","unstructured":"Teng Zhang Jianying Wang Xuntao Cheng Hao Xu Nanlong Yu Gui Huang Tieying Zhang Dengcheng He Feifei Li Wei Cao Zhongdong Huang and Jianling Sun. 2020. FPGA-accelerated compactions for LSM-based Key-value store. In 18th USENIX Conference on File and Storage Technologies (FAST\u201920). USENIX Association Santa Clara CA 225\u2013237. https:\/\/www.usenix.org\/conference\/fast20\/presentation\/zhang-teng"},{"key":"e_1_3_2_48_2","unstructured":"Weidong Zhang Erci Xu Qiuping Wang Xiaolu Zhang Yuesheng Gu Zhenwei Lu Tao Ouyang Guanqun Dai Wenwen Peng Zhe Xu Shuo Zhang Dong Wu Yilei Peng Tianyun Wang Haoran Zhang Jiasheng Wang Wenyuan Yan Yuanyuan Dong Wenhui Yao Zhongjie Wu Lingjun Zhu Chao Shi Yinhu Wang Rong Liu Junping Wu Jiaji Zhu and Jiesheng Wu. 2024. What\u2019s the Story in EBS Glory: Evolutions and Lessons in Building Cloud Block Store. In 22nd USENIX Conference on File and Storage Technologies (FAST\u201924). USENIX Association Santa Clara CA 277\u2013291. https:\/\/www.usenix.org\/conference\/fast24\/presentation\/zhang-weidong"}],"container-title":["ACM Transactions on Architecture and Code Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3656584","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3656584","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:57:31Z","timestamp":1750291051000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3656584"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,14]]},"references-count":47,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,9,30]]}},"alternative-id":["10.1145\/3656584"],"URL":"https:\/\/doi.org\/10.1145\/3656584","relation":{},"ISSN":["1544-3566","1544-3973"],"issn-type":[{"value":"1544-3566","type":"print"},{"value":"1544-3973","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,14]]},"assertion":[{"value":"2024-02-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-02","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-14","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}