{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T22:45:50Z","timestamp":1777675550421,"version":"3.51.4"},"reference-count":50,"publisher":"SAGE Publications","issue":"6","license":[{"start":{"date-parts":[[2020,5,23]],"date-time":"2020-05-23T00:00:00Z","timestamp":1590192000000},"content-version":"vor","delay-in-days":366,"URL":"http:\/\/www.sagepub.com\/licence-information-for-chorus"}],"funder":[{"DOI":"10.13039\/100006132","name":"Office of Science","doi-asserted-by":"publisher","award":["DE-AC05-00OR22725"],"award-info":[{"award-number":["DE-AC05-00OR22725"]}],"id":[{"id":"10.13039\/100006132","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of High Performance Computing Applications"],"published-print":{"date-parts":[[2019,11]]},"abstract":"<jats:p>Recently, persistent data structures, like key-value stores (KVSs), which are stored in a high-performance computing (HPC) system\u2019s nonvolatile memory, provide an attractive solution for a number of emerging challenges like limited I\/O performance. Data compression and encryption are two well-known techniques for improving several properties of such data-oriented systems. This article investigates how to efficiently integrate data compression and encryption into persistent KVSs for HPC with the ultimate goal of hiding their costs and complexity in terms of performance and ease of use. Our compression technique exploits deep memory hierarchy in an HPC system to achieve both storage reduction and performance improvement. Our encryption technique provides a practical level of security and enables sharing of sensitive data securely in complex scientific workflows with nearly imperceptible cost. We implement the proposed techniques on top of a distributed embedded KVS to evaluate the benefits and costs of incorporating these capabilities along different points in the dataflow path, illustrating differences in effective bandwidth, latency, and additional computational expense on Swiss National Supercomputing Centre\u2019s Grand Tav\u00e9 and National Energy Research Scientific Computing Center\u2019s Cori.<\/jats:p>","DOI":"10.1177\/1094342019847264","type":"journal-article","created":{"date-parts":[[2019,5,23]],"date-time":"2019-05-23T23:08:46Z","timestamp":1558652926000},"page":"1098-1112","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":9,"title":["Implementing efficient data compression and encryption in a persistent key-value store for HPC"],"prefix":"10.1177","volume":"33","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6594-6225","authenticated-orcid":false,"given":"Jungwon","family":"Kim","sequence":"first","affiliation":[{"name":"Oak Ridge National Laboratory, Oak Ridge, TN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jeffrey S","family":"Vetter","sequence":"additional","affiliation":[{"name":"Oak Ridge National Laboratory, Oak Ridge, TN, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2019,5,23]]},"reference":[{"key":"bibr1-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/ISCA.2004.1310776"},{"key":"bibr2-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/2254756.2254766"},{"key":"bibr3-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2013.189"},{"key":"bibr4-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1985.231852"},{"key":"bibr5-1094342019847264","first-page":"359","volume-title":"Proceedings of the 15th Usenix Conference on File and Storage Technologies, FAST\u201917","author":"Bj\u00f8rling M","year":"2017"},{"key":"bibr6-1094342019847264","volume-title":"Proceedings of the 25th High Performance Computing Symposium, HPC \u201917","author":"Cao L","year":"2017"},{"key":"bibr7-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/1365815.1365816"},{"key":"bibr8-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0023501"},{"key":"bibr9-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/TSC.2010.38"},{"key":"bibr10-1094342019847264","unstructured":"Chen T, Li M, Li Y, et al. (2015) MXNet: a flexible and efficient machine learning library for heterogeneous distributed systems.\n                      CoRR\n                      abs\/1512.01274."},{"key":"bibr11-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/3132847.3132916"},{"key":"bibr12-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/ISSCC.2018.8310322"},{"key":"bibr13-1094342019847264","unstructured":"Collobert R, Bengio S, Marithoz J (2002) Torch: a modular machine learning software library."},{"key":"bibr14-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2016.11"},{"key":"bibr15-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/IPDPS.2016.23"},{"issue":"124","key":"bibr16-1094342019847264","first-page":"5","volume":"2004","author":"Fitzpatrick B","year":"2004","journal-title":"Linux Journal"},{"key":"bibr17-1094342019847264","volume-title":"Proceedings of the 7th USENIX Conference on Hot Topics in Storage and File Systems, HotStorage\u201915","author":"Greenberg HN","year":"2015"},{"key":"bibr18-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-03317-9_4"},{"key":"bibr19-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/2566673"},{"key":"bibr20-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/SC.2012.77"},{"key":"bibr21-1094342019847264","first-page":"695","volume-title":"Proceedings of the 20th International Conference on Very Large Data Bases, VLDB \u201994","author":"Iyer BR","year":"1994"},{"key":"bibr22-1094342019847264","volume-title":"Intel Xeon Phi Processor High Performance Programming: Knights Landing Edition 2Nd Edition","author":"Jeffers J","year":"2016","edition":"2"},{"key":"bibr23-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/3126908.3126943"},{"key":"bibr24-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/1773912.1773922"},{"key":"bibr25-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/1989323.1989448"},{"key":"bibr26-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/1138127.1138130"},{"key":"bibr27-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2014.10.014"},{"key":"bibr28-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/MSST.2012.6232369"},{"key":"bibr29-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.3125"},{"key":"bibr30-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/2749246.2749269"},{"key":"bibr31-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/MCG.2009.120"},{"key":"bibr32-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/ISSCC.2018.8310321"},{"key":"bibr33-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/2807591.2807656"},{"key":"bibr34-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/TPDS.2015.2435788"},{"key":"bibr35-1094342019847264","first-page":"11","volume-title":"Proceedings of the 4th USENIX Conference on Hot Topics in Storage and File Systems, HotStorage\u201912","author":"Nowlan MF","year":"2012"},{"key":"bibr36-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1007\/s002360050048"},{"key":"bibr37-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/3096742"},{"key":"bibr38-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1016\/B978-012722442-8\/50087-2"},{"key":"bibr39-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/163090.163096"},{"key":"bibr40-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/1815933.1815940"},{"key":"bibr41-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/2807591.2807669"},{"key":"bibr42-1094342019847264","unstructured":"The HDF Group (1997) Hierarchical Data Format, version 5. Available at: http:\/\/www.hdfgroup.org\/HDF5\/ (accessed April 2019)."},{"key":"bibr43-1094342019847264","volume-title":"Fundamentals of Cryptology: A Professional Reference and Interactive Tutorial","author":"Tilborg HCAV","year":"1999","edition":"1"},{"key":"bibr44-1094342019847264","unstructured":"Unidata (2008) Network Common Data Form, version 4. Available at: https:\/\/www.unidata.ucar.edu\/software\/netcdf\/ (accessed April 2019)."},{"key":"bibr45-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/2503210.2503239"},{"key":"bibr46-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/2757667.2757684"},{"key":"bibr47-1094342019847264","doi-asserted-by":"publisher","DOI":"10.14778\/3025111.3025113"},{"key":"bibr48-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2015.2427795"},{"key":"bibr49-1094342019847264","volume-title":"IBM DB2 9 New Features","author":"Zikopoulos P","year":"2007","edition":"1"},{"key":"bibr50-1094342019847264","doi-asserted-by":"publisher","DOI":"10.1145\/3023362"}],"container-title":["The International Journal of High Performance Computing Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342019847264","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/1094342019847264","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342019847264","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/1094342019847264","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T08:15:50Z","timestamp":1777450550000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/1094342019847264"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,5,23]]},"references-count":50,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2019,11]]}},"alternative-id":["10.1177\/1094342019847264"],"URL":"https:\/\/doi.org\/10.1177\/1094342019847264","relation":{},"ISSN":["1094-3420","1741-2846"],"issn-type":[{"value":"1094-3420","type":"print"},{"value":"1741-2846","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,5,23]]}}}