{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T17:21:28Z","timestamp":1767979288534,"version":"3.49.0"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"1","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62302175 and 62272185"],"award-info":[{"award-number":["62302175 and 62272185"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Research Grants Council of Hong Kong","award":["AoE\/P-404\/18"],"award-info":[{"award-number":["AoE\/P-404\/18"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Storage"],"published-print":{"date-parts":[[2026,2,28]]},"abstract":"<jats:p>Minimum-storage regenerating (MSR) codes are provably optimal erasure codes that minimize the repair bandwidth (i.e., the amount of traffic being transferred during a repair operation), while minimizing storage redundancy, in distributed storage systems. However, the practical repair performance of MSR codes still has significant room for improvements, as their mathematical structure makes repair operations difficult to parallelize. In this article, we present HyperParaRC, a parallel repair framework for MSR codes. HyperParaRC leverages the sub-packetization nature of MSR codes to parallelize the repair of sub-blocks and balance repair load (i.e., the amount of traffic sent or received by a node) across available nodes. We first demonstrate that there exists a trade-off between repair bandwidth and maximum repair load. We then propose an affinity-based heuristic for HyperParaRC, which approximately minimizes the maximum repair load by examining the bandwidth incurred during sub-block computations and significantly reduces the search time for large coding parameters compared with our earlier work, ParaRC. Based on our affinity-based heuristic, we further design a full-node recovery mechanism for HyperParaRC that combines both intra-stripe and inter-stripe parallel repair scheduling to repair multiple lost blocks in a failed node. We prototype HyperParaRC on Hadoop HDFS and evaluate it on Alibaba Cloud. Our evaluation results show that HyperParaRC reduces both single-block repair and full-node recovery times compared with state-of-the-art repair approaches.<\/jats:p>","DOI":"10.1145\/3773990","type":"journal-article","created":{"date-parts":[[2025,10,30]],"date-time":"2025-10-30T09:22:36Z","timestamp":1761816156000},"page":"1-38","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Harnessing Parallelism for Fast Data Repair in MSR-Coded Storage"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3582-0792","authenticated-orcid":false,"given":"Xiaolu","family":"Li","sequence":"first","affiliation":[{"name":"Huazhong University of Science and Technology","place":["Wuhan, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-2894-2943","authenticated-orcid":false,"given":"Han","family":"Yuan","sequence":"additional","affiliation":[{"name":"Huazhong University of Science and Technology","place":["Wuhan, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-0503-862X","authenticated-orcid":false,"given":"Xuan","family":"Liu","sequence":"additional","affiliation":[{"name":"Huazhong University of Science and Technology","place":["Wuhan, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-7677-7513","authenticated-orcid":false,"given":"Junlong","family":"Zhang","sequence":"additional","affiliation":[{"name":"Huazhong University of Science and Technology","place":["Wuhan, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4501-4364","authenticated-orcid":false,"given":"Patrick P. C.","family":"Lee","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, The Chinese University of Hong Kong","place":["Shatin, Hong Kong"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1265-7141","authenticated-orcid":false,"given":"Yuchong","family":"Hu","sequence":"additional","affiliation":[{"name":"Wuhan National Laboratory for Optoelectronics","place":["Wuhan, China"]},{"name":"School of computer, Huazhong University of Science and Technology","place":["Wuhan, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4674-6006","authenticated-orcid":false,"given":"Dan","family":"Feng","sequence":"additional","affiliation":[{"name":"Huazhong University of Science and Technology","place":["Wuhan, China"]}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2026,1,9]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"Accessed in Sept. 2025. Alibaba Cloud - Elastic Compute Service. Retrieved from https:\/\/www.alibabacloud.com\/product\/ecs-pricing-list\/en"},{"key":"e_1_3_1_3_2","unstructured":"Accessed in Sept. 2025. Alibaba Cloud - ESSDs. Retrieved from https:\/\/www.alibabacloud.com\/help\/en\/elastic-compute-service\/latest\/essds"},{"key":"e_1_3_1_4_2","unstructured":"Accessed in Sept. 2025. Apache Hadoop 3.3.4 - HDFS Architecture. Retrieved from https:\/\/hadoop.apache.org\/docs\/r3.3.4\/hadoop-project-dist\/hadoop-hdfs\/HdfsDesign.html"},{"key":"e_1_3_1_5_2","unstructured":"Accessed in Sept. 2025. Apache Hadoop 3.3.4 - HDFS Erasure Coding. Retrieved from https:\/\/hadoop.apache.org\/docs\/r3.3.4\/hadoop-project-dist\/hadoop-hdfs\/HDFSErasureCoding.html"},{"key":"e_1_3_1_6_2","unstructured":"Accessed in Sept. 2025. Backblaze Vaults: Zettabyte-Scale Cloud Storage Architecture. Retrieved from https:\/\/www.backblaze.com\/blog\/vault-cloud-storage-architecture\/"},{"key":"e_1_3_1_7_2","unstructured":"Accessed in Sept. 2025. Ceph - Erasure code. Retrieved from http:\/\/docs.ceph.com\/docs\/master\/rados\/operations\/erasure-code\/"},{"key":"e_1_3_1_8_2","unstructured":"Accessed in Sept. 2025. Intel Intelligent Storage Acceleration Library. Retrieved from https:\/\/github.com\/intel\/isa-l"},{"key":"e_1_3_1_9_2","unstructured":"Accessed in Sept. 2025. redis.io. Retrieved from https:\/\/redis.io\/"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3337821.3337831"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2010.2054295"},{"key":"e_1_3_1_12_2","volume-title":"Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation","author":"Ford Daniel","year":"2010","unstructured":"Daniel Ford, Fran\u00e7ois Labelle, Florentina I. Popovici, Murray Stokely, Van-Anh Truong, Luiz Barroso, Carrie Grimes, and Sean Quinlan. 2010. Availability in globally distributed storage systems. In Proceedings of the 8th USENIX Conference on Operating Systems Design and Implementation."},{"key":"e_1_3_1_13_2","doi-asserted-by":"publisher","DOI":"10.5555\/3724648.3724657"},{"key":"e_1_3_1_14_2","volume-title":"Proceedings of the 10th USENIX Conference on File and Storage Technologies","author":"Hu Yuchong","year":"2012","unstructured":"Yuchong Hu, Henry C. H. Chen, Patrick P. C. Lee, and Yang Tang. 2012. NCCloud: Applying network coding for the storage repair in a cloud-of-clouds. In Proceedings of the 10th USENIX Conference on File and Storage Technologies."},{"key":"e_1_3_1_15_2","volume-title":"Proceedings of the 19th USENIX Conference on File and Storage Technologies","author":"Hu Yuchong","year":"2021","unstructured":"Yuchong Hu, Liangfeng Cheng, Qiaori Yao, Patrick P. C. Lee, Weichun Wang, and Wei Chen. 2021. Exploiting combined locality for wide-stripe erasure coding in distributed storage. In Proceedings of the 19th USENIX Conference on File and Storage Technologies."},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.5555\/2342821.2342823"},{"key":"e_1_3_1_17_2","article-title":"The Wonder Shaper 1.4.1","author":"Hubert Bert","year":"2025","unstructured":"Bert Hubert, Jacco Geul, and Simon S\u00e9hier. Accessed in Sept. 2025. The Wonder Shaper 1.4.1. Retrieved from https:\/\/github.com\/magnific0\/wondershaper","journal-title":"https:\/\/github.com\/magnific0\/wondershaper"},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.5555\/3585938.3585939"},{"key":"e_1_3_1_19_2","volume-title":"Proceedings of the 10th USENIX Conference on File and Storage Technologies","author":"Khan Osama","year":"2012","unstructured":"Osama Khan, Randal C. Burns, James S. Plank, William Pierce, and Cheng Huang. 2012. Rethinking erasure codes for cloud file systems: Minimizing I\/O for recovery and degraded reads. In Proceedings of the 10th USENIX Conference on File and Storage Technologies."},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.5555\/3277355.3277439"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TBDATA.2017.2749255"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.5555\/3154690.3154744"},{"key":"e_1_3_1_23_2","volume-title":"Proceedings of the 21st USENIX Conference on File and Storage Technologies","author":"Li Xiaolu","year":"2023","unstructured":"Xiaolu Li, Keyun Cheng, Kaicheng Tang, Patrick P. C. Lee, Yuchong Hu, Dan Feng, Jie Li, and Ting-Yi Wu. 2023. ParaRC: Embracing sub-packetization for repair parallelization in msr-coded storage. In Proceedings of the 21st USENIX Conference on File and Storage Technologies."},{"key":"e_1_3_1_24_2","volume-title":"Proceedings of the 17th USENIX Conference on File and Storage Technologies","author":"Li Xiaolu","year":"2019","unstructured":"Xiaolu Li, Runhui Li, Patrick P. C. Lee, and Yuchong Hu. 2019. OpenEC: Toward unified and configurable erasure coding management in distributed storage systems. In Proceedings of the 17th USENIX Conference on File and Storage Technologies."},{"issue":"2","key":"e_1_3_1_25_2","first-page":"13:1\u201313:29","article-title":"Repair pipelining for erasure-coded storage: Algorithms and evaluation","volume":"17","author":"Li Xiaolu","year":"2021","unstructured":"Xiaolu Li, Zuoru Yang, Jinhong Li, Runhui Li, Patrick P. C. Lee, Qun Huang, and Yuchong Hu. 2021. Repair pipelining for erasure-coded storage: Algorithms and evaluation. ACM Transactions on Storage 17, 2, 13:1\u201313:29.","journal-title":"ACM Transactions on Storage"},{"key":"e_1_3_1_26_2","volume-title":"Proceedings of the 2021 USENIX Annual Technical Conference","author":"Lin Shiyao","year":"2021","unstructured":"Shiyao Lin, Guowen Gong, Zhirong Shen, Patrick P. C. Lee, and Jiwu Shu. 2021. Boosting full-node repair in erasure-coded storage. In Proceedings of the 2021 USENIX Annual Technical Conference."},{"key":"e_1_3_1_27_2","volume-title":"Proceedings of the Eleventh European Conference on Computer Systems","author":"Mitra Subrata","year":"2016","unstructured":"Subrata Mitra, Rajesh Panta, Moo-Ryong Ra, and Saurabh Bagchi. 2016. Partial-parallel-repair (PPR): A distributed technique for repairing erasure coded storage. In Proceedings of the Eleventh European Conference on Computer Systems."},{"key":"e_1_3_1_28_2","volume-title":"Proceedings of the 11th USENIX conference on Operating Systems Design and Implementation","author":"Muralidhar Subramanian","year":"2014","unstructured":"Subramanian Muralidhar, Wyatt Lloyd, Sabyasachi Roy, Cory Hill, Ernest Lin, Weiwen Liu, Satadru Pan, Shiva Shankar, Viswanath Sivakumar, Linpeng Tang, et\u00a0al. 2014. f4: Facebook\u2019s warm BLOB storage system. In Proceedings of the 11th USENIX conference on Operating Systems Design and Implementation."},{"key":"e_1_3_1_29_2","volume-title":"Proceedings of the 14th Usenix Conference on File and Storage Technologies","author":"Pamies-Juarez Lluis","year":"2016","unstructured":"Lluis Pamies-Juarez, Filip Blagojevic, Robert Mateescu, Cyril Guyot, Eyal En Gad, and Zvonimir Bandic. 2016. Opening the chrysalis: On the real repair performance of MSR codes. In Proceedings of the 14th Usenix Conference on File and Storage Technologies."},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1098\/rspl.1895.0041"},{"key":"e_1_3_1_31_2","volume-title":"Proceedings of 24th International Conference on Computing in High Energy and Nuclear Physics","author":"Peters Andreas-Joachim","year":"2019","unstructured":"Andreas-Joachim Peters, Michal Kamil Simon, and Elvin Alin Sindrilaru. 2019. Erasure coding for production in the EOS open storage system. In Proceedings of 24th International Conference on Computing in High Energy and Nuclear Physics."},{"key":"e_1_3_1_32_2","volume-title":"Proceedings of the 7th USENIX Conference on File and Storage Technologies","author":"Plank James S.","year":"2009","unstructured":"James S. Plank, Jianqiang Luo, Catherine D. Schuman, Lihao Xu, and Zooko Wilcox-O\u2019Hearn. 2009. A performance evaluation and examination of open-source erasure coding libraries for storage. In Proceedings of the 7th USENIX Conference on File and Storage Technologies."},{"key":"e_1_3_1_33_2","volume-title":"Proceedings of the 13th USENIX Conference on File and Storage Technologies","author":"Rashmi K. V.","year":"2015","unstructured":"K. V. Rashmi, Preetum Nakkiran, Jingyan Wang, Nihar B. Shah, and Kannan Ramchandran. 2015. Having your cake and eating it too: Jointly optimal erasure codes for I\/O, storage, and network-bandwidth. In Proceedings of the 13th USENIX Conference on File and Storage Technologies."},{"key":"e_1_3_1_34_2","volume-title":"Proceedings of the 5th USENIX Conference on Hot Topics in Storage and File Systems","author":"Rashmi K. V.","year":"2013","unstructured":"K. V. Rashmi, Nihar B. Shah, Dikang Gu, Hairong Kuang, Dhruba Borthakur, and Kannan Ramchandran. 2013. A solution to the network challenges of data recovery in erasure-coded distributed storage systems: A study on the Facebook warehouse cluster. In Proceedings of the 5th USENIX Conference on Hot Topics in Storage and File Systems."},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/2619239.2626325"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2017.2715043"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1137\/0108018"},{"issue":"5","key":"e_1_3_1_38_2","first-page":"325","article-title":"XORing elephants: Novel erasure codes for big data","volume":"6","author":"Sathiamoorthy Maheswaran","year":"2013","unstructured":"Maheswaran Sathiamoorthy, Megasthenis Asteris, Dimitris Papailiopoulos, Alexandros G. Dimakis, Ramkumar Vadali, Scott Chen, and Dhruba Borthakur. 2013. XORing elephants: Novel erasure codes for big data. Proceedings of the 39th International Conference on Very Large Data Bases 6, 5, 325\u2013336.","journal-title":"Proceedings of the 39th International Conference on Very Large Data Bases"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2011.2178588"},{"key":"e_1_3_1_40_2","volume-title":"Proceedings of the 2023 USENIX Annual Technical Conference","author":"Shan Yingdi","year":"2023","unstructured":"Yingdi Shan, Kang Chen, and Yongwei Wu. 2023. Explore data placement algorithm forbalanced recovery load distribution. In Proceedings of the 2023 USENIX Annual Technical Conference."},{"key":"e_1_3_1_41_2","volume-title":"Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles","author":"Shan Yingdi","year":"2021","unstructured":"Yingdi Shan, Kang Chen, Tuoyu Gong, Lidong Zhou, Tai Zhou, and Yongwei Wu. 2021. Geometric partitioning: Explore the boundary of optimal erasure code repair. In Proceedings of the ACM SIGOPS 28th Symposium on Operating Systems Principles."},{"key":"e_1_3_1_42_2","first-page":"556","volume-title":"Proceedings of the 49th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks","author":"Shen Zhirong","year":"2019","unstructured":"Zhirong Shen, Xiaolu Li, and Patrick P. C. Lee. 2019. Fast predictive repair in erasure-coded storage. In Proceedings of the 49th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks. 556\u2013567."},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2012.2227110"},{"key":"e_1_3_1_44_2","volume-title":"Proceedings of the 16th USENIX Conference on File and Storage Technologies","author":"Vajha Myna","year":"2018","unstructured":"Myna Vajha, Vinayak Ramkumar, Bhagyashree Puranik, Ganesh Kini, Elita Lobo, Birenjith Sasidharan, P. Vijay Kumar, Alexandar Barg, Min Ye, Srinivasan Narayanamurthy, et\u00a0al. 2018. Clay codes: Moulding MDS codes to yield an MSR code. In Proceedings of the 16th USENIX Conference on File and Storage Technologies."},{"key":"e_1_3_1_45_2","volume-title":"Proceedings of the 2019 USENIX Conference on Usenix Annual Technical Conference","author":"Wang Zhufan","year":"2019","unstructured":"Zhufan Wang, Guangyan Zhang, Yang Wang, Qinglin Yang, and Jiaji Zhu. 2019. Dayu: Fast and low-interference data recovery in very-large storage systems. In Proceedings of the 2019 USENIX Conference on Usenix Annual Technical Conference."},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.5555\/646334.687814"},{"issue":"10","key":"e_1_3_1_47_2","first-page":"2386","article-title":"SelectiveEC: Towards balanced recovery load on erasure-coded storage systems","volume":"33","author":"Xu Liangliang","year":"2021","unstructured":"Liangliang Xu, Min Lyu, Qiliang Li, Lingjiang Xie, Cheng Li, and Yinlong Xu. 2021. SelectiveEC: Towards balanced recovery load on erasure-coded storage systems. IEEE Transactions on Parallel and Distributed Systems 33, 10, 2386\u20132400.","journal-title":"IEEE Transactions on Parallel and Distributed Systems"},{"key":"e_1_3_1_48_2","volume-title":"Proceedings of the 42nd IEEE International Conference on Distributed Computing Systems","author":"Yao Qiaori","year":"2022","unstructured":"Qiaori Yao, Yuchong Hu, Xinyuan Tu, Patrick P. C. Lee Lee, Dan Feng, Xia Zhu, Xiaoyang Zhang, Zhen Yao, and Wenjia Wei. 2022. PivotRepair: Fast pipelined repair for erasure-coded hot storage. In Proceedings of the 42nd IEEE International Conference on Distributed Computing Systems."},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3472456.3472494"}],"container-title":["ACM Transactions on Storage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3773990","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T13:04:50Z","timestamp":1767963890000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3773990"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,9]]},"references-count":48,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,2,28]]}},"alternative-id":["10.1145\/3773990"],"URL":"https:\/\/doi.org\/10.1145\/3773990","relation":{},"ISSN":["1553-3077","1553-3093"],"issn-type":[{"value":"1553-3077","type":"print"},{"value":"1553-3093","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,9]]},"assertion":[{"value":"2024-12-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-13","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}