{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T06:55:38Z","timestamp":1772866538300,"version":"3.50.1"},"reference-count":53,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2021,5,28]],"date-time":"2021-05-28T00:00:00Z","timestamp":1622160000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Research Grants Council of Hong Kong","award":["GRF 14216316 and AoE\/P-404\/18"],"award-info":[{"award-number":["GRF 14216316 and AoE\/P-404\/18"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61872414, 61502191, and 61802365"],"award-info":[{"award-number":["61872414, 61502191, and 61802365"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Storage"],"published-print":{"date-parts":[[2021,5,30]]},"abstract":"<jats:p>\n            We propose\n            <jats:italic>repair pipelining<\/jats:italic>\n            , a technique that speeds up the repair performance in general erasure-coded storage. By carefully scheduling the repair of failed data in small-size units across storage nodes in a pipelined manner, repair pipelining reduces the single-block repair time to approximately the same as the normal read time for a single block in homogeneous environments. We further design different extensions of repair pipelining algorithms for heterogeneous environments and multi-block repair operations. We implement a repair pipelining prototype, called\n            <jats:sans-serif>ECPipe<\/jats:sans-serif>\n            , and integrate it as a middleware system into two versions of Hadoop Distributed File System (HDFS) (namely, HDFS-RAID and HDFS-3) as well as Quantcast File System. Experiments on a local testbed and Amazon EC2 show that repair pipelining significantly improves the performance of degraded reads and full-node recovery over existing repair techniques.\n          <\/jats:p>","DOI":"10.1145\/3436890","type":"journal-article","created":{"date-parts":[[2021,5,28]],"date-time":"2021-05-28T17:51:05Z","timestamp":1622224265000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":27,"title":["Repair Pipelining for Erasure-coded Storage: Algorithms and Evaluation"],"prefix":"10.1145","volume":"17","author":[{"given":"Xiaolu","family":"Li","sequence":"first","affiliation":[{"name":"The Chinese University of Hong Kong, Shatin, Hong Kong, China"}]},{"given":"Zuoru","family":"Yang","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Shatin, Hong Kong, China"}]},{"given":"Jinhong","family":"Li","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Shatin, Hong Kong, China"}]},{"given":"Runhui","family":"Li","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Shatin, Hong Kong, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4501-4364","authenticated-orcid":false,"given":"Patrick P. C.","family":"Lee","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Shatin, Hong Kong, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2387-6131","authenticated-orcid":false,"given":"Qun","family":"Huang","sequence":"additional","affiliation":[{"name":"Peking University, Haidian, Beijing, China"}]},{"given":"Yuchong","family":"Hu","sequence":"additional","affiliation":[{"name":"Huazhong University of Science and Technology, Wuhan, China"}]}],"member":"320","published-online":{"date-parts":[[2021,5,28]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Facebook. 2020. Facebook\u2019s Hadoop. Retrieved from https:\/\/github.com\/facebookarchive\/hadoop-20.  Facebook. 2020. Facebook\u2019s Hadoop. Retrieved from https:\/\/github.com\/facebookarchive\/hadoop-20."},{"key":"e_1_2_1_2_1","unstructured":"Hadoop. 2020. Hadoop 3.1.1 HDFS. Retrieved from https:\/\/hadoop.apache.org\/docs\/r3.1.1\/.  Hadoop. 2020. Hadoop 3.1.1 HDFS. Retrieved from https:\/\/hadoop.apache.org\/docs\/r3.1.1\/."},{"key":"e_1_2_1_3_1","unstructured":"Iperf. 2020. Iperf. Retrieved from https:\/\/iperf.fr\/.  Iperf. 2020. Iperf. Retrieved from https:\/\/iperf.fr\/."},{"key":"e_1_2_1_4_1","unstructured":"Redis. 2020. Redis. Retrieved from http:\/\/redis.io\/.  Redis. 2020. Redis. Retrieved from http:\/\/redis.io\/."},{"key":"e_1_2_1_5_1","unstructured":"Linux. 2020. tc. Retrieved from https:\/\/linux.die.net\/man\/8\/tc.  Linux. 2020. tc. Retrieved from https:\/\/linux.die.net\/man\/8\/tc."},{"key":"e_1_2_1_6_1","volume-title":"Proceedings of the International Conference on Principles of Distributed Systems (OPODIS\u201913)","author":"Aguilera Marcos K.","year":"2013","unstructured":"Marcos K. Aguilera . 2013 . Geo-distributed Storage in Data Centers . In Proceedings of the International Conference on Principles of Distributed Systems (OPODIS\u201913) . Marcos K. Aguilera. 2013. Geo-distributed Storage in Data Centers. In Proceedings of the International Conference on Principles of Distributed Systems (OPODIS\u201913)."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/2643634.2643636"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/2592798.2592816"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3337821.3337831"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/1251175.1251200"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043556.2043571"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/3154690.3154742"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/2534169.2486021"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.5555\/1267680.1267684"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.5555\/1251254.1251264"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2010.2054295"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.5555\/1924943.1924948"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/945445.945450"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/366622.366647"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/143371.143383"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2019.2902835"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3149349"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/2342821.2342823"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2014.2311808"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/2829988.2787488"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.5555\/2208461.2208481"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/1833515.1833884"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1109\/DSN.2014.47"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.5555\/3154690.3154744"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/3323298.3323330"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1145\/2901318.2901328"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.5555\/2685048.2685078"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.191995"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043556.2043560"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.14778\/2536222.2536234"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.5555\/2930583.2930590"},{"key":"e_1_2_1_37_1","first-page":"6","article-title":"Erasure codes for storage systems: A brief primer. ;login","volume":"38","author":"Plank James S.","year":"2013","unstructured":"James S. Plank . 2013 . Erasure codes for storage systems: A brief primer. ;login : USENIX Mag. 38 , 6 (Dec 2013), 44\u201350. James S. Plank. 2013. Erasure codes for storage systems: A brief primer. ;login: USENIX Mag. 38, 6 (Dec 2013), 44\u201350.","journal-title":"USENIX Mag."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.5555\/1525908.1525927"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2018.2806342"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.5555\/2750482.2750489"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.5555\/2534861.2534869"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/2740070.2626325"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1137\/0108018"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.5555\/1960475.1960489"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.14778\/2535573.2488339"},{"key":"e_1_2_1_46_1","volume-title":"Proceedings of the 49th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN\u201919)","author":"Shen Zhirong","unstructured":"Zhirong Shen , Xiaolu Li , and Patrick P. C. Lee . 2019. Fast predictive repair in erasure-coded storage . In Proceedings of the 49th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN\u201919) . 556\u2013567. Zhirong Shen, Xiaolu Li, and Patrick P. C. Lee. 2019. Fast predictive repair in erasure-coded storage. In Proceedings of the 49th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN\u201919). 556\u2013567."},{"key":"e_1_2_1_47_1","volume-title":"Proceedings of the 35th IEEE International Parallel & Distributed Processing Symposium (IPDPS\u201920)","author":"Shen Zhirong","year":"2020","unstructured":"Zhirong Shen , Jiwu Shu , Zhijie Huang , and Yingxun Fu . 2020 . ClusterSR: Cluster-aware scattered repair in erasure-coded storage . In Proceedings of the 35th IEEE International Parallel & Distributed Processing Symposium (IPDPS\u201920) . 42\u201351. Zhirong Shen, Jiwu Shu, Zhijie Huang, and Yingxun Fu. 2020. ClusterSR: Cluster-aware scattered repair in erasure-coded storage. In Proceedings of the 35th IEEE International Parallel & Distributed Processing Symposium (IPDPS\u201920). 42\u201351."},{"key":"e_1_2_1_48_1","volume-title":"Proceedings of the 46th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN\u201916)","author":"Shen Zhirong","unstructured":"Zhirong Shen , Jiwu Shu , and Patrick P. C. Lee . 2016. Reconsidering single failure recovery in clustered file systems . In Proceedings of the 46th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN\u201916) . 323\u2013334. Zhirong Shen, Jiwu Shu, and Patrick P. C. Lee. 2016. Reconsidering single failure recovery in clustered file systems. In Proceedings of the 46th Annual IEEE\/IFIP International Conference on Dependable Systems and Networks (DSN\u201916). 323\u2013334."},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2010.5496972"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2611354.2611370"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.5555\/3189759.3189773"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.5555\/646334.687814"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.5031"}],"container-title":["ACM Transactions on Storage"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3436890","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3436890","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:02:20Z","timestamp":1750197740000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3436890"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,28]]},"references-count":53,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2021,5,30]]}},"alternative-id":["10.1145\/3436890"],"URL":"https:\/\/doi.org\/10.1145\/3436890","relation":{},"ISSN":["1553-3077","1553-3093"],"issn-type":[{"value":"1553-3077","type":"print"},{"value":"1553-3093","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,28]]},"assertion":[{"value":"2019-07-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2020-11-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-05-28","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}