{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T16:23:00Z","timestamp":1781022180609,"version":"3.54.1"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2022,4,28]],"date-time":"2022-04-28T00:00:00Z","timestamp":1651104000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Storage"],"published-print":{"date-parts":[[2022,5,31]]},"abstract":"<jats:p>Memory disaggregation is a promising technique in datacenters with the benefit of improving resource utilization, failure isolation, and elasticity. Hashing indexes have been widely used to provide fast lookup services in distributed memory systems. However, traditional hashing indexes become inefficient for disaggregated memory, since the computing power in the memory pool is too weak to execute complex index requests. To provide efficient indexing services in disaggregated memory scenarios, this article proposes RACE hashing, a one-sided RDMA-Conscious Extendible hashing index with lock-free remote concurrency control and efficient remote resizing. RACE hashing enables all index operations to be efficiently executed by using only one-sided RDMA verbs without involving any compute resource in the memory pool. To support remote concurrent access with high performance, RACE hashing leverages a lock-free remote concurrency control scheme to enable different clients to concurrently operate the same hashing index in the memory pool in a lock-free manner. To resize the hash table with low overheads, RACE hashing leverages an extendible remote resizing scheme to reduce extra RDMA accesses caused by extendible resizing and allow concurrent request execution during resizing. Extensive experimental results demonstrate that RACE hashing outperforms state-of-the-art distributed in-memory hashing indexes by 1.4\u201313.7\u00d7 in YCSB hybrid workloads.<\/jats:p>","DOI":"10.1145\/3511895","type":"journal-article","created":{"date-parts":[[2022,4,28]],"date-time":"2022-04-28T13:10:28Z","timestamp":1651151428000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["RACE: One-sided RDMA-conscious Extendible Hashing"],"prefix":"10.1145","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9982-5130","authenticated-orcid":false,"given":"Pengfei","family":"Zuo","sequence":"first","affiliation":[{"name":"Huawei Cloud, Shenzhen, Guangdong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8927-3325","authenticated-orcid":false,"given":"Qihui","family":"Zhou","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3581-2477","authenticated-orcid":false,"given":"Jiazhao","family":"Sun","sequence":"additional","affiliation":[{"name":"Huawei Cloud, Shenzhen, Guangdong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0885-5236","authenticated-orcid":false,"given":"Liu","family":"Yang","sequence":"additional","affiliation":[{"name":"Huawei Cloud, Shenzhen, Guangdong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5741-064X","authenticated-orcid":false,"given":"Shuangwu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Huawei Cloud, Shenzhen, Guangdong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7730-3796","authenticated-orcid":false,"given":"Yu","family":"Hua","sequence":"additional","affiliation":[{"name":"Huazhong University of Science and Technology, Wuhan, Hubei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6313-6288","authenticated-orcid":false,"given":"James","family":"Cheng","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0798-966X","authenticated-orcid":false,"given":"Rongfeng","family":"He","sequence":"additional","affiliation":[{"name":"Huawei, Shenzhen, Guangdong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1666-531X","authenticated-orcid":false,"given":"Huabing","family":"Yan","sequence":"additional","affiliation":[{"name":"Huawei, Chengdu, Sichuan, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,4,28]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"2022. Gen-Z Technology. Retrieved from https:\/\/genzconsortium.org\/."},{"key":"e_1_3_2_3_2","unstructured":"2022. Memcached\u2014A Distributed Memory Object Caching System. Retrieved from https:\/\/memcached.org\/."},{"key":"e_1_3_2_4_2","unstructured":"2022. Redis. Retrieved from https:\/\/redis.io\/."},{"key":"e_1_3_2_5_2","first-page":"775","volume-title":"Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201918)","author":"Aguilera Marcos K.","year":"2018","unstructured":"Marcos K. Aguilera, Nadav Amit, Irina Calciu, Xavier Deguillard, Jayneel Gandhi, Stanko Novakovic, Arun Ramanathan, Pratap Subrahmanyam, Lalith Suresh, Kiran Tati, Rajesh Venkatasubramanian, and Michael Wei. 2018. Remote regions: A simple abstraction for remote memory. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201918). 775\u2013787."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3317550.3321433"},{"key":"e_1_3_2_7_2","volume-title":"Proceedings of the USENIX Conference on File and Storage Technologies (FAST\u201914)","author":"Asanovi\u0107 Krste","year":"2014","unstructured":"Krste Asanovi\u0107 and David Patterson. 2014. FireBox: A hardware building block for 2020 warehouse-scale computers. In Proceedings of the USENIX Conference on File and Storage Technologies (FAST\u201914)."},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/2254756.2254766"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/HOTI.2015.22"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3152434.3152447"},{"key":"e_1_3_2_11_2","first-page":"799","volume-title":"Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201920)","author":"Chen Zhangyu","year":"2020","unstructured":"Zhangyu Chen, Yu Hua, Bo Ding, and Pengfei Zuo. 2020. Lock-free concurrent level hashing for persistent memory. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201920). 799\u2013812."},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/1807128.1807152"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/2785956.2787492"},{"key":"e_1_3_2_14_2","doi-asserted-by":"crossref","unstructured":"Ana L\u00facia De Moura and Roberto Ierusalimschy. 2009. Revisiting coroutines. ACM Trans. Program. Lang. Syst. 31 2 Article 6 (February 2009) 31 pages.","DOI":"10.1145\/1462166.1462167"},{"key":"e_1_3_2_15_2","first-page":"401","volume-title":"Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201914)","author":"Dragojevi\u0107 Aleksandar","year":"2014","unstructured":"Aleksandar Dragojevi\u0107, Dushyanth Narayanan, Miguel Castro, and Orion Hodson. 2014. FaRM: Fast remote memory. In Proceedings of the 11th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201914). 401\u2013414."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/320083.320092"},{"key":"e_1_3_2_17_2","first-page":"371","volume-title":"Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201913)","author":"Fan Bin","year":"2013","unstructured":"Bin Fan, David G. Andersen, and Michael Kaminsky. 2013. Memc3: Compact and concurrent memcache with dumber caching and smarter hashing. In Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201913). 371\u2013384."},{"key":"e_1_3_2_18_2","volume-title":"Proceedings of the 15th Workshop on Hot Topics in Operating Systems (HotOS\u201915)","author":"Faraboschi Paolo","year":"2015","unstructured":"Paolo Faraboschi, Kimberly Keeton, Tim Marsland, and Dejan Milojicic. 2015. Beyond processor-centric operating systems. In Proceedings of the 15th Workshop on Hot Topics in Operating Systems (HotOS\u201915)."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1145\/261124.261125"},{"key":"e_1_3_2_20_2","first-page":"249","volume-title":"Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201916)","author":"Gao Peter X.","year":"2016","unstructured":"Peter X. Gao, Akshay Narayan, Sagar Karandikar, Joao Carreira, Sangjin Han, Rachit Agarwal, Sylvia Ratnasamy, and Scott Shenker. 2016. Network requirements for resource disaggregation. In Proceedings of the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201916). 249\u2013264."},{"key":"e_1_3_2_21_2","article-title":"CoroBase: Coroutine-oriented main-memory database engine","volume":"2010","author":"He Yongjun","year":"2020","unstructured":"Yongjun He, Jiacheng Lu, and Tianzheng Wang. 2020. CoroBase: Coroutine-oriented main-memory database engine. arXiv: 2010.15981. Retrieved from https:\/\/arxiv.org\/abs\/2010.15981.","journal-title":"arXiv"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-87779-0_24"},{"key":"e_1_3_2_23_2","unstructured":"Hewlett Packard Corporation. 2015. The Machine: A New Kind of Computer. Retrieved from https:\/\/www.hpl.hp.com\/research\/systems-research\/themachine\/."},{"key":"e_1_3_2_24_2","unstructured":"Intel Corporation. 2022. Intel Rack Scale Design Architecture. Retrieved from https:\/\/www.intel.com\/content\/www\/us\/en\/architecture-and-technology\/rack-scale-design-overview.html."},{"key":"e_1_3_2_25_2","first-page":"437","volume-title":"Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201916)","author":"Kalia Anuj","year":"2016","unstructured":"Anuj Kalia, Michael Kaminsky, and David G. Andersen. 2016. Design guidelines for high performance RDMA systems. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201916). 437\u2013450."},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/2540708.2540748"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132756"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1145\/2749469.2750416"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/2592798.2592820"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3064015"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/1555754.1555789"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/HPCA.2012.6168955"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/2168836.2168855"},{"key":"e_1_3_2_34_2","first-page":"103","volume-title":"Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201915)","author":"Mitchell Christopher","year":"2013","unstructured":"Christopher Mitchell, Yifeng Geng, and Jinyang Li. 2013. Using one-sided RDMA reads to build a fast, CPU-efficient key-value store. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201915). 103\u2013114."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/71.963420"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.5555\/3323298.3323302"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDCS.2014.70"},{"key":"e_1_3_2_38_2","first-page":"385","volume-title":"Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201913)","author":"Nishtala Rajesh","year":"2013","unstructured":"Rajesh Nishtala, Hans Fugal, Steven Grimm, Marc Kwiatkowski, Herman Lee, Harry C. Li, Ryan McElroy, Mike Paleczny, Daniel Peek, Paul Saab, et\u00a0al. 2013. Scaling memcache at facebook. In Proceedings of the 10th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201913). 385\u2013398."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jalgor.2003.12.002"},{"key":"e_1_3_2_40_2","first-page":"315","volume-title":"Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201920)","author":"Ruan Zhenyuan","year":"2020","unstructured":"Zhenyuan Ruan, Malte Schwarzkopf, Marcos K. Aguilera, and Adam Belay. 2020. AIFM: High-performance, application-integrated far memory. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201920). 315\u2013332."},{"key":"e_1_3_2_41_2","first-page":"69","volume-title":"Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201918)","author":"Shan Yizhou","year":"2018","unstructured":"Yizhou Shan, Yutong Huang, Yilun Chen, and Yiying Zhang. 2018. LegoOS: A disseminated, distributed OS for hardware resource disaggregation. In Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201918). 69\u201387."},{"key":"e_1_3_2_42_2","first-page":"255","volume-title":"Proceedings of the 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201919)","author":"Shrivastav Vishal","year":"2019","unstructured":"Vishal Shrivastav, Asaf Valadarsky, Hitesh Ballani, Paolo Costa, Ki Suh Lee, Han Wang, Rachit Agarwal, and Hakim Weatherspoon. 2019. Shoal: A network architecture for disaggregated racks. In Proceedings of the 16th USENIX Symposium on Networked Systems Design and Implementation (NSDI\u201919). 255\u2013270."},{"key":"e_1_3_2_43_2","first-page":"33","volume-title":"Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201920)","author":"Tsai Shin-Yeh","year":"2020","unstructured":"Shin-Yeh Tsai, Yizhou Shan, and Yiying Zhang. 2020. Disaggregating persistent memory and controlling them remotely: An exploration of passive disaggregated key-value stores. In Proceedings of the USENIX Annual Technical Conference (USENIX ATC\u201920). 33\u201348."},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132762"},{"key":"e_1_3_2_45_2","first-page":"261","volume-title":"Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201920)","author":"Wang Chenxi","year":"2020","unstructured":"Chenxi Wang, Haoran Ma, Shi Liu, Yuanqi Li, Zhenyuan Ruan, Khanh Nguyen, Michael D Bond, Ravi Netravali, Miryung Kim, and Guoqing Harry Xu. 2020. Semeru: A memory-disaggregated managed runtime. In Proceedings of the 14th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201920). 261\u2013280."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2018.00049"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/2815400.2815419"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.14778\/2735508.2735511"},{"key":"e_1_3_2_49_2","first-page":"461","volume-title":"Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201918)","author":"Zuo Pengfei","year":"2018","unstructured":"Pengfei Zuo, Yu Hua, and Jie Wu. 2018. Write-optimized and high-performance hashing index scheme for persistent memory. In Proceedings of the 13th USENIX Symposium on Operating Systems Design and Implementation (OSDI\u201918). 461\u2013476."}],"container-title":["ACM Transactions on Storage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511895","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3511895","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:48:50Z","timestamp":1750182530000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3511895"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,28]]},"references-count":48,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2022,5,31]]}},"alternative-id":["10.1145\/3511895"],"URL":"https:\/\/doi.org\/10.1145\/3511895","relation":{},"ISSN":["1553-3077","1553-3093"],"issn-type":[{"value":"1553-3077","type":"print"},{"value":"1553-3093","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,4,28]]},"assertion":[{"value":"2021-11-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-04-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}