{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T22:43:10Z","timestamp":1777675390245,"version":"3.51.4"},"reference-count":40,"publisher":"SAGE Publications","issue":"4","license":[{"start":{"date-parts":[[2016,12,23]],"date-time":"2016-12-23T00:00:00Z","timestamp":1482451200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2017,7]]},"abstract":"<jats:p>Key\u2013value stores are being widely used as the storage system for large-scale internet services and cloud storage systems. However, they are rarely used in HPC systems, where parallel file systems are the dominant storage solution. In this study, we examine the architecture differences and performance characteristics of parallel file systems and key\u2013value stores. We propose using key\u2013value stores to optimize overall Input\/Output (I\/O) performance, especially for workloads that parallel file systems cannot handle well, such as the cases with intense data synchronization or heavy metadata operations. We conducted experiments with several synthetic benchmarks, an I\/O benchmark, and a real application. We modeled the performance of these two systems using collected data from our experiments, and we provide a predictive method to identify which system offers better I\/O performance given a specific workload. The results show that we can optimize the I\/O performance in HPC systems by utilizing key\u2013value stores.<\/jats:p>","DOI":"10.1177\/1094342016677084","type":"journal-article","created":{"date-parts":[[2017,6,22]],"date-time":"2017-06-22T06:43:24Z","timestamp":1498113804000},"page":"335-356","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":5,"title":["Rethinking key\u2013value store for parallel I\/O optimization"],"prefix":"10.1177","volume":"31","author":[{"given":"Anthony","family":"Kougkas","sequence":"first","affiliation":[{"name":"Department of Computer Science, Illinois Institute of Technology, Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hassan","family":"Eslami","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xian-He","family":"Sun","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Illinois Institute of Technology, Chicago, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rajeev","family":"Thakur","sequence":"additional","affiliation":[{"name":"Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"William","family":"Gropp","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana, IL, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2016,12,23]]},"reference":[{"key":"bibr1-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/PDSW.2008.4811892"},{"key":"bibr2-1094342016677084","unstructured":"Braam PJ, (2014) The Lustre storage architecture. Available at: ftp:\/\/ftp.uni-duisburg.de\/linux\/filesys\/Lustre\/lustre.pdf. (accessed 13 February 2015)."},{"key":"bibr3-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/2538542.2538565"},{"key":"bibr4-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2009.5161029"},{"key":"bibr5-1094342016677084","first-page":"391","volume-title":"Proceedings of the 4th annual Linux showcase and conference","author":"Carns PH","year":"2000"},{"key":"bibr6-1094342016677084","unstructured":"Ching A (2014) HPIO I\/O benchmark. Available at: http:\/\/goo.gl\/OEaaiB. (accessed 10 January 2015)."},{"key":"bibr7-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/1323293.1294281"},{"key":"bibr8-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/1362622.1362659"},{"key":"bibr9-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/2342356.2342360"},{"key":"bibr10-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/945445.945450"},{"key":"bibr11-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/242223.242300"},{"key":"bibr12-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/SC.Companion.2012.17"},{"key":"bibr13-1094342016677084","unstructured":"Group TT (2012) EMC object-based storage for active archiving and application development. Technical report, The TANEJA Group, Inc. Available at: http:\/\/www.emc.com\/collateral\/analyst-reports\/emc-atmosecosystem-taneja-group-tech-brief-final-ar.pdf."},{"key":"bibr14-1094342016677084","unstructured":"Hyperdex (n.d.) Hyperdex performance benchmarks. Available at: http:\/\/hyperdex.org\/performance\/."},{"key":"bibr15-1094342016677084","first-page":"335","author":"Kimpe D","year":"2014","journal-title":"High Performance Parallel I\/O"},{"key":"bibr16-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/1773912.1773922"},{"key":"bibr17-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2013.110"},{"key":"bibr18-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2011.85"},{"key":"bibr19-1094342016677084","unstructured":"MongoDB (n.d.) MongoDB. Available at: https:\/\/www.mongodb.com\/white-papers. (accessed 2 April 2015)."},{"key":"bibr20-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2010.18"},{"key":"bibr21-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/2188286.2188321"},{"key":"bibr22-1094342016677084","unstructured":"Nuclear Engineering Division of Argonne National Laboratory (2014) PLFS I\/O traces. Available at: https:\/\/www.mcs.anl.gov\/\u223cthakur\/pio-benchmarks.html (accessed 23 November 2016)."},{"key":"bibr23-1094342016677084","unstructured":"OrangeFS (2014) Orange file system. OrangeFS 2.8.8 Current version is 2.9.6 Retrieved from:http:\/\/www.orangefs.org."},{"key":"bibr24-1094342016677084","first-page":"13","volume-title":"9th USENIX conference on file and storage technologies","volume":"11","author":"Patil S","year":"2011"},{"key":"bibr25-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2014.25"},{"key":"bibr26-1094342016677084","volume-title":"Proceedings of USENIX conference on file and storage technologies","author":"Schmuck FB","year":"2002"},{"key":"bibr27-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2010.5496972"},{"issue":"2","key":"bibr28-1094342016677084","first-page":"6","volume":"35","author":"Shvachko KV","year":"2010","journal-title":"login"},{"key":"bibr29-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/1996130.1996138"},{"key":"bibr30-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/2063384.2063407"},{"key":"bibr31-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/383059.383071"},{"key":"bibr32-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1145\/1713072.1713077"},{"key":"bibr33-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/ICCSNT.2011.6182030"},{"key":"bibr34-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/CLUSTER.2012.54"},{"key":"bibr35-1094342016677084","first-page":"307","volume-title":"Proceedings of 7th symposium on operating systems design and implementation","author":"Weil S","year":"2006"},{"key":"bibr36-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2004.22"},{"key":"bibr37-1094342016677084","first-page":"1","volume-title":"6th USENIX conference on file and storage technologies","volume":"8","author":"Welch B","year":"2008"},{"key":"bibr38-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/CCGrid.2012.136"},{"key":"bibr39-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2013.21"},{"key":"bibr40-1094342016677084","doi-asserted-by":"publisher","DOI":"10.1109\/PDSW.2014.7"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342016677084","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/1094342016677084","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342016677084","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T08:15:25Z","timestamp":1777450525000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1094342016677084"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,12,23]]},"references-count":40,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2017,7]]}},"alternative-id":["10.1177\/1094342016677084"],"URL":"https:\/\/doi.org\/10.1177\/1094342016677084","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"value":"1094-3420","type":"print"},{"value":"1741-2846","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,12,23]]}}}