{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,6]],"date-time":"2024-08-06T08:31:32Z","timestamp":1722933092681},"reference-count":20,"publisher":"Oxford University Press (OUP)","issue":"20","license":[{"start":{"date-parts":[[2019,3,18]],"date-time":"2019-03-18T00:00:00Z","timestamp":1552867200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,10,15]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Motivation<\/jats:title><jats:p>Genome repositories are growing faster than our storage capacities, challenging our ability to store, transmit, process and analyze them. While genomes are not very compressible individually, those repositories usually contain myriads of genomes or genome reads of the same species, thereby creating opportunities for orders-of-magnitude compression by exploiting inter-genome similarities. A useful compression system, however, cannot be only usable for archival, but it must allow direct access to the sequences, ideally in transparent form so that applications do not need to be rewritten.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We present a highly compressed filesystem that specializes in storing large collections of genomes and reads. The system obtains orders-of-magnitude compression by using Relative Lempel-Ziv, which exploits the high similarities between genomes of the same species. The filesystem transparently stores the files in compressed form, intervening the system calls of the applications without the need to modify them. A client\/server variant of the system stores the compressed files in a server, while the client\u2019s filesystem transparently retrieves and updates the data from the server. The data between client and server are also transferred in compressed form, which saves an order of magnitude network time.<\/jats:p><\/jats:sec><jats:sec><jats:title>Availability and implementation<\/jats:title><jats:p>The C++ source code of our implementation is available for download in https:\/\/github.com\/vsepulve\/relz_fs.<\/jats:p><\/jats:sec>","DOI":"10.1093\/bioinformatics\/btz192","type":"journal-article","created":{"date-parts":[[2019,3,16]],"date-time":"2019-03-16T04:01:30Z","timestamp":1552708890000},"page":"4120-4128","source":"Crossref","is-referenced-by-count":1,"title":["Compressed filesystem for managing large genome collections"],"prefix":"10.1093","volume":"35","author":[{"given":"Gonzalo","family":"Navarro","sequence":"first","affiliation":[{"name":"CeBiB\u2014Center for Biotechnology and Bioengineering , Santiago, Chile"},{"name":"Department of Computer Science, University of Chile , Santiago, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"V\u00edctor","family":"Sep\u00falveda","sequence":"additional","affiliation":[{"name":"CeBiB\u2014Center for Biotechnology and Bioengineering , Santiago, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mauricio","family":"Mar\u00edn","sequence":"additional","affiliation":[{"name":"CeBiB\u2014Center for Biotechnology and Bioengineering , Santiago, Chile"},{"name":"DIINF, University of Santiago, Santiago , Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sen\u00e9n","family":"Gonz\u00e1lez","sequence":"additional","affiliation":[{"name":"CeBiB\u2014Center for Biotechnology and Bioengineering , Santiago, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2019,3,18]]},"reference":[{"issue":"7571","key":"2023013108274994100_btz192-B1","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1038\/nature15393","article-title":"A global reference for human genetic variation","volume":"526","year":"2015","journal-title":"Nature"},{"key":"2023013108274994100_btz192-B2","first-page":"52","volume-title":"Proc. 21st International Symposium on String Processing and Information Retrieval (SPIRE)","author":"Belazzougui","year":"2014"},{"key":"2023013108274994100_btz192-B3","first-page":"83","article-title":"Queries on LZ-bounded encodings","author":"Belazzougui","year":"2015","journal-title":"Proceedings of 25th Data Compression Conference (DCC"},{"key":"2023013108274994100_btz192-B4","doi-asserted-by":"crossref","first-page":"1228","DOI":"10.1109\/TCBB.2016.2568186","article-title":"Benchmark dataset for whole genome sequence compression","volume":"14","author":"Biji","year":"2017","journal-title":"IEEE\/ACM Trans. Comput. Biol. Bioinform"},{"key":"2023013108274994100_btz192-B5","first-page":"1","article-title":"RLZAP: relative Lempel-Ziv with adaptive pointers","author":"Cox","year":"2016","journal-title":"Proceedings of 23rd International Symposium on String Processing and Information Retrieval (SPIRE)"},{"key":"2023013108274994100_btz192-B6","doi-asserted-by":"crossref","first-page":"2979","DOI":"10.1093\/bioinformatics\/btr505","article-title":"Robust relative compression of genomes with random access","volume":"27","author":"Deorowicz","year":"2011","journal-title":"Bioinformatics"},{"key":"2023013108274994100_btz192-B7","doi-asserted-by":"crossref","first-page":"11565","DOI":"10.1038\/srep11565","article-title":"GDC 2: compression of large collections of genomes","volume":"25","author":"Deorowicz","year":"2015","journal-title":"Sci. Rep."},{"key":"2023013108274994100_btz192-B8","doi-asserted-by":"crossref","first-page":"14","DOI":"10.1016\/j.tcs.2013.07.024","article-title":"Fast relative Lempel-Ziv self-index for similar sequences","volume":"532","author":"Do","year":"2014","journal-title":"Theor. Comput. Sci"},{"key":"2023013108274994100_btz192-B9","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1093\/comjnl\/bxx108","article-title":"Relative suffix trees","volume":"61","author":"Farruggia","year":"2018","journal-title":"Comput. J"},{"key":"2023013108274994100_btz192-B10","first-page":"13","volume-title":"Proceedings of 21st International Symposium on String Processing and Information Retrieval (SPIRE)","author":"Ferrada","year":"2014"},{"key":"2023013108274994100_btz192-B11","doi-asserted-by":"crossref","first-page":"552","DOI":"10.1145\/1082036.1082039","article-title":"Indexing compressed texts","volume":"52","author":"Ferragina","year":"2005","journal-title":"J. ACM"},{"key":"2023013108274994100_btz192-B12","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1007\/978-3-319-46049-9_16","article-title":"Analyzing relative Lempel-Ziv reference construction","author":"Gagie","year":"2016","journal-title":"Proceedings of 23rd International Symposium on String Processing and Information Retrieval (SPIRE)"},{"key":"2023013108274994100_btz192-B13","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1109\/TIT.1966.1053907","article-title":"Run-length encodings","volume":"12","author":"Golomb","year":"1966","journal-title":"IEEE Trans. Inf. Theory"},{"key":"2023013108274994100_btz192-B14","doi-asserted-by":"crossref","first-page":"737","DOI":"10.1109\/18.841160","article-title":"Grammar-based codes: a new class of universal lossless source codes","volume":"46","author":"Kieffer","year":"2000","journal-title":"IEEE Trans. Inf. Theory"},{"key":"2023013108274994100_btz192-B15","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1007\/978-3-642-16321-0_20","volume-title":"Proceedings of 17th International Symposium on String Processing and Information Retrieval (SPIRE)","author":"Kuruppu","year":"2010"},{"key":"2023013108274994100_btz192-B16","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1007\/978-3-642-24583-1_41","volume-title":"Proceedings of 18th International Symposium on String Processing and Information Retrieval (SPIRE)","author":"Kuruppu","year":"2011"},{"key":"2023013108274994100_btz192-B17","doi-asserted-by":"crossref","first-page":"807","DOI":"10.1145\/2872427.2883042","article-title":"Effective construction of relative Lempel-Ziv dictionaries","author":"Liao","year":"2016","journal-title":"Proceedings of 25th International Conference on World Wide Web (WWW)"},{"key":"2023013108274994100_btz192-B18","doi-asserted-by":"crossref","first-page":"935","DOI":"10.1137\/0222058","article-title":"Suffix arrays: a new method for on-line string searches","volume":"22","author":"Manber","year":"1993","journal-title":"SIAM J. Comput"},{"key":"2023013108274994100_btz192-B19","doi-asserted-by":"crossref","first-page":"e1002195.","DOI":"10.1371\/journal.pbio.1002195","article-title":"Big data: astronomical or genomical?","volume":"17","author":"Sthephens","year":"2015","journal-title":"PLoS Biol"},{"key":"2023013108274994100_btz192-B20","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1093\/comjnl\/42.3.193","article-title":"Compressing integers for fast file access","volume":"42","author":"Williams","year":"1999","journal-title":"Comput. J"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btz192\/28769951\/btz192.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/20\/4120\/48976119\/bioinformatics_35_20_4120.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/20\/4120\/48976119\/bioinformatics_35_20_4120.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,14]],"date-time":"2023-09-14T15:29:41Z","timestamp":1694705381000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/35\/20\/4120\/5384493"}},"subtitle":[],"editor":[{"given":"Jonathan","family":"Wren","sequence":"additional","affiliation":[],"role":[{"role":"editor","vocabulary":"crossref"}]}],"short-title":[],"issued":{"date-parts":[[2019,3,18]]},"references-count":20,"journal-issue":{"issue":"20","published-print":{"date-parts":[[2019,10,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btz192","relation":{},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,10,15]]},"published":{"date-parts":[[2019,3,18]]}}}