{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T23:54:17Z","timestamp":1772236457853,"version":"3.50.1"},"reference-count":29,"publisher":"Association for Computing Machinery (ACM)","issue":"13","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2014,8]]},"abstract":"<jats:p>We present Fatman, an enterprise-scale archival storage based on volunteer contribution resources from underutilized web servers, usually deployed on thousands of nodes with spare storage capacity. Fatman is specifically designed for enhancing the utilization of existing storage resources and cutting down the hardware purchase cost. Two major concerned issues of the system design are maximizing the resource utilization of volunteer nodes without violating Service Level Objectives (SLOs) and minimizing the cost without reducing the availability of archival system.<\/jats:p>\n          <jats:p>Fatman has been widely deployed on tens of thousands of server nodes across several datacenters, provided more than 100PB storage capacity and served dozens of internal mass-data applications. The system realizes an efficient storage quota consolidation by strong isolation and budget limitation, to maximally support resources contribution without any degradation on host-level SLOs. It firstly improves data reliability by applying disk failure prediction to minish failure recovery cost, named fault-aware data management, dramatically reduces the MTTR by 76.3% and decreases file crash ratio by 35% on real-life product workload.<\/jats:p>","DOI":"10.14778\/2733004.2733078","type":"journal-article","created":{"date-parts":[[2015,5,12]],"date-time":"2015-05-12T15:37:52Z","timestamp":1431445072000},"page":"1748-1753","source":"Crossref","is-referenced-by-count":13,"title":["Fatman"],"prefix":"10.14778","volume":"7","author":[{"given":"An","family":"Qin","sequence":"first","affiliation":[{"name":"Baidu, Inc"}]},{"given":"Dianming","family":"Hu","sequence":"additional","affiliation":[{"name":"Baidu, Inc"}]},{"given":"Jun","family":"Liu","sequence":"additional","affiliation":[{"name":"Baidu, Inc"}]},{"given":"Wenjun","family":"Yang","sequence":"additional","affiliation":[{"name":"Baidu, Inc"}]},{"given":"Dai","family":"Tan","sequence":"additional","affiliation":[{"name":"Baidu, Inc"}]}],"member":"320","published-online":{"date-parts":[[2014,8]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"File system archiving focus on EMC. http:\/\/www.emcemea.com\/materials\/FileSystemAssess\/ESG_FSA_Whitepaper.pdf.  File system archiving focus on EMC. http:\/\/www.emcemea.com\/materials\/FileSystemAssess\/ESG_FSA_Whitepaper.pdf."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/945445.945462"},{"key":"e_1_2_1_3_1","first-page":"331","volume-title":"VLDB'88","author":"Bitton D.","year":"1988","unstructured":"D. Bitton and J. Gray . Disk shadowing . In VLDB'88 , pages 331 -- 338 , 1988 . D. Bitton and J. Gray. Disk shadowing. In VLDB'88, pages 331--338, 1988."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/176979.176981"},{"key":"e_1_2_1_5_1","first-page":"215","volume-title":"FAST'07","author":"Cipar J.","year":"2007","unstructured":"J. Cipar , M. D. Corner , and E. D. Berger . TFS: A transparent file system for contributory storage . In FAST'07 , pages 215 -- 229 , 2007 . J. Cipar, M. D. Corner, and E. D. Berger. TFS: A transparent file system for contributory storage. In FAST'07, pages 215--229, 2007."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.5555\/2580126.2580611"},{"key":"e_1_2_1_7_1","volume-title":"f","author":"J.","year":"2006","unstructured":"J. f . Paris, J. f. Paris, T. J. E. Schwarz, T. J. E. Schwarz, D. D. E. Long, and D. D. E. Long. Evaluating the reliability of storage systems. Technical report, 2006 . J. f. Paris, J. f. Paris, T. J. E. Schwarz, T. J. E. Schwarz, D. D. E. Long, and D. D. E. Long. Evaluating the reliability of storage systems. Technical report, 2006."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/945445.945450"},{"key":"e_1_2_1_9_1","first-page":"202","volume-title":"ICML","author":"Hamerly G.","year":"2001","unstructured":"G. Hamerly and C. Elkan . Bayesian approaches to failure prediction for disk drives . In ICML , pages 202 -- 209 , 2001 . G. Hamerly and C. Elkan. Bayesian approaches to failure prediction for disk drives. In ICML, pages 202--209, 2001."},{"key":"e_1_2_1_10_1","volume-title":"The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines","author":"Hoelzle U.","year":"2009","unstructured":"U. Hoelzle and L. A. Barroso . The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines . Morgan and Claypool Publishers , 1 st edition, 2009 . U. Hoelzle and L. A. Barroso. The Datacenter as a Computer: An Introduction to the Design of Warehouse-Scale Machines. Morgan and Claypool Publishers, 1st edition, 2009.","edition":"1"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.5555\/2342821.2342823"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2002.802886"},{"key":"e_1_2_1_13_1","first-page":"111","volume-title":"FAST'08","author":"Jiang W.","year":"2008","unstructured":"W. Jiang , C. Hu , Y. Zhou , and A. Kanevsky . Are disks the dominant contributor for storage failures? a comprehensive study of storage subsystem failure characteristics . In FAST'08 , pages 111 -- 125 , 2008 . W. Jiang, C. Hu, Y. Zhou, and A. Kanevsky. Are disks the dominant contributor for storage failures? a comprehensive study of storage subsystem failure characteristics. In FAST'08, pages 111--125, 2008."},{"key":"e_1_2_1_14_1","volume-title":"FAST'12","author":"Khan O.","year":"2012","unstructured":"O. Khan , A. Burns , J. Plank , W. Pierce , and C. Huang . Rethinking erasure codes for cloud file systems: Minimizing i\/o for recovery and degraded reads . In FAST'12 , 2012 . O. Khan, A. Burns, J. Plank, W. Pierce, and C. Huang. Rethinking erasure codes for cloud file systems: Minimizing i\/o for recovery and degraded reads. In FAST'12, 2012."},{"key":"e_1_2_1_15_1","unstructured":"S. M. Larson C. D. Snow M. Shirts V. S. P and V. S. Pande. Folding@home and Genome@home: Using distributed computing to tackle previously intractable problems in computational biology.  S. M. Larson C. D. Snow M. Shirts V. S. P and V. S. Pande. Folding@home and Genome@home: Using distributed computing to tackle previously intractable problems in computational biology."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/989.990"},{"key":"e_1_2_1_17_1","first-page":"783","volume-title":"Journal of Machine Learning Research","author":"Murray J. F.","year":"2005","unstructured":"J. F. Murray , G. F. Hughes , and K. Kreutz-Delgado . Machine learning methods for predicting failures in hard drives: A multiple-instance application . Journal of Machine Learning Research , pages 783 -- 816 , 2005 . J. F. Murray, G. F. Hughes, and K. Kreutz-Delgado. Machine learning methods for predicting failures in hard drives: A multiple-instance application. Journal of Machine Learning Research, pages 783--816, 2005."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/264359.264360"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1137\/0108018"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/11558989_21"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.14778\/2535573.2488339"},{"key":"e_1_2_1_22_1","first-page":"1","volume-title":"FAST'07","author":"Schroeder B.","year":"2007","unstructured":"B. Schroeder and G. A. Gibson . Disk failures in the real world: What does an mttf of 1, 000, 000 hours mean to you ? In FAST'07 , pages 1 -- 16 , 2007 . B. Schroeder and G. A. Gibson. Disk failures in the real world: What does an mttf of 1, 000, 000 hours mean to you? In FAST'07, pages 1--16, 2007."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1145\/1272996.1273025"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISIT.2013.6620540"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1807128.1807161"},{"key":"e_1_2_1_26_1","volume-title":"FAST'12","author":"Vrable M.","year":"2012","unstructured":"M. Vrable , S. Savage , and G. M. Voelker . Bluesky : a cloud-backed file system for the enterprise . In FAST'12 , 2012 . M. Vrable, S. Savage, and G. M.Voelker. Bluesky: a cloud-backed file system for the enterprise. In FAST'12, 2012."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/1525908.1525925"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.5555\/1097871.1098163"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2013.6558427"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/2733004.2733078","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,12,28]],"date-time":"2022-12-28T09:37:33Z","timestamp":1672220253000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/2733004.2733078"}},"subtitle":["cost-saving and reliable archival storage based on volunteer resources"],"short-title":[],"issued":{"date-parts":[[2014,8]]},"references-count":29,"journal-issue":{"issue":"13","published-print":{"date-parts":[[2014,8]]}},"alternative-id":["10.14778\/2733004.2733078"],"URL":"https:\/\/doi.org\/10.14778\/2733004.2733078","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2014,8]]}}}