{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,27]],"date-time":"2026-09-27T07:22:25Z","timestamp":1790493745225,"version":"4.1.0"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"S9","license":[{"start":{"date-parts":[[2019,11,1]],"date-time":"2019-11-01T00:00:00Z","timestamp":1572566400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2019,11,22]],"date-time":"2019-11-22T00:00:00Z","timestamp":1574380800000},"content-version":"vor","delay-in-days":21,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2019,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Motivation<\/jats:title>\n                <jats:p>Current NGS techniques are becoming exponentially cheaper. As a result, there is an exponential growth of genomic data unfortunately not followed by an exponential growth of storage, leading to the necessity of compression. Most of the entropy of NGS data lies in the quality values associated to each read. Those values are often more diversified than necessary. Because of that, many tools such as Quartz or GeneCodeq, try to change (smooth) quality scores in order to improve compressibility without altering the important information they carry for downstream analysis like SNP calling.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>We use the FM-Index, a type of compressed suffix array, to reduce the storage requirements of a dictionary of k-mers and an effective smoothing algorithm to maintain high precision for SNP calling pipelines, while reducing quality scores entropy.<\/jats:p>\n                <jats:p>We present YALFF (Yet Another Lossy Fastq Filter), a tool for quality scores compression by smoothing leading to improved compressibility of FASTQ files. The succinct k-mers dictionary allows YALFF to run on consumer computers with only 5.7 GB of available free RAM. YALFF smoothing algorithm can improve genotyping accuracy while using less resources.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Availability<\/jats:title>\n                <jats:p><jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/yhhshb\/yalff\">https:\/\/github.com\/yhhshb\/yalff<\/jats:ext-link><\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12859-019-2883-5","type":"journal-article","created":{"date-parts":[[2019,11,22]],"date-time":"2019-11-22T10:02:38Z","timestamp":1574416958000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Better quality score compression through sequence-based quality smoothing"],"prefix":"10.1186","volume":"20","author":[{"given":"Yoshihiro","family":"Shibuya","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matteo","family":"Comin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2019,11,22]]},"reference":[{"key":"2883_CR1","unstructured":"Google Genomics - Store, process, explore and share. https:\/\/cloud.google.com\/genomics\/."},{"issue":"3","key":"2883_CR2","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1101\/gr.8.3.175","volume":"8","author":"B Ewing","year":"1998","unstructured":"Ewing B, Hillier L, Wendl MC, Green P. Base-Calling of Automated Sequencer Traces UsingPhred, I. Accuracy Assessment. Genome Res. 1998; 8(3):175\u201385. https:\/\/doi.org\/10.1101\/gr.8.3.175.","journal-title":"Genome Res"},{"key":"2883_CR3","volume-title":"Algorithms in Bioinformatics","author":"M Comin","year":"2014","unstructured":"Comin M, Leoni A, Schimd M. Qcluster: Extending alignment-free measures with quality values for reads clustering In: Brown D, Morgenstern B, editors. Algorithms in Bioinformatics. Berlin, Heidelberg: Springer: 2014. p. 1\u201313."},{"issue":"1","key":"2883_CR4","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s13015-014-0028-y","volume":"10","author":"M Comin","year":"2015","unstructured":"Comin M, Leoni A, Schimd M. Clustering of reads with alignment-free measures and quality values. Algoritm Mol Biol. 2015; 10(1):1\u201310.","journal-title":"Algoritm Mol Biol"},{"issue":"1","key":"2883_CR5","first-page":"41","volume":"9","author":"M Schimd","year":"2016","unstructured":"Schimd M, Comin M. Fast comparison of genomic and meta-genomic reads with alignment-free measures based on quality values. BMC Med Genomics. 2016; 9(1):41\u201350.","journal-title":"BMC Med Genomics"},{"issue":"19","key":"2883_CR6","doi-asserted-by":"publisher","first-page":"3122","DOI":"10.1093\/bioinformatics\/btv330","volume":"31","author":"G Malysa","year":"2015","unstructured":"Malysa G, Hernaez M, Ochoa I, Rao M, Ganesan K, Weissman T. QVZ: lossy compression of quality values. Bioinformatics (Oxford, England). 2015; 31(19):3122\u20139. https:\/\/doi.org\/10.1093\/bioinformatics\/btv330.","journal-title":"Bioinformatics (Oxford, England)"},{"issue":"16","key":"2883_CR7","doi-asserted-by":"publisher","first-page":"2748","DOI":"10.1093\/bioinformatics\/bty205","volume":"34","author":"L Roguski","year":"2018","unstructured":"Roguski L, Ochoa I, Hernaez M, Deorowicz S. Fastore: a space-saving solution for raw sequencing data. Bioinformatics. 2018; 34(16):2748\u201356.","journal-title":"Bioinformatics"},{"key":"2883_CR8","volume-title":"Research in Computational Molecular Biology","author":"G Holley","year":"2017","unstructured":"Holley G, Wittler R, Stoye J, Hach F. Dynamic alignment-free and reference-free read compression In: Sahinalp SC, editor. Research in Computational Molecular Biology. Cham: Springer: 2017. p. 50\u201365."},{"issue":"9","key":"2883_CR9","doi-asserted-by":"publisher","first-page":"1389","DOI":"10.1093\/bioinformatics\/btu844","volume":"31","author":"S Grabowski","year":"2015","unstructured":"Grabowski S, Deorowicz S, Roguski L. Disk-based compression of data from genome sequencing. Bioinformatics. 2015; 31(9):1389\u201395.","journal-title":"Bioinformatics"},{"issue":"23","key":"2883_CR10","doi-asserted-by":"publisher","first-page":"3051","DOI":"10.1093\/bioinformatics\/bts593","volume":"28","author":"F Hach","year":"2012","unstructured":"Hach F, Numanagi\u0107 I, Alkan C, Sahinalp SC. Scalce: boosting sequence compression algorithms using locally consistent encoding. Bioinformatics. 2012; 28(23):3051\u20137.","journal-title":"Bioinformatics"},{"issue":"20","key":"2883_CR11","doi-asserted-by":"publisher","first-page":"3124","DOI":"10.1093\/bioinformatics\/btw385","volume":"32","author":"DL Greenfield","year":"2016","unstructured":"Greenfield DL, Stegle O, Rrustemi A. GeneCodeq: quality score compression and improved genotyping using a Bayesian framework. Bioinformatics (Oxford, England). 2016; 32(20):3124\u201332. https:\/\/doi.org\/10.1093\/bioinformatics\/btw385.","journal-title":"Bioinformatics (Oxford, England)"},{"issue":"3","key":"2883_CR12","doi-asserted-by":"publisher","first-page":"240","DOI":"10.1038\/nbt.3170","volume":"33","author":"YW Yu","year":"2015","unstructured":"Yu YW, Yorukoglu D, Peng J, Berger B. Quality score compression improves genotyping accuracy. Nat Biotechnol. 2015; 33(3):240\u20133. https:\/\/doi.org\/10.1038\/nbt.3170.","journal-title":"Nat Biotechnol"},{"issue":"3","key":"2883_CR13","doi-asserted-by":"publisher","first-page":"e59190","DOI":"10.1371\/journal.pone.0059190","volume":"8","author":"James K. Bonfield","year":"2013","unstructured":"Bonfield JK, Mahoney MV. Compression of fastq and sam format sequencing data. PloS ONE. 2013. https:\/\/doi.org\/10.1371\/journal.pone.0059190.","journal-title":"PLoS ONE"},{"key":"2883_CR14","unstructured":"Quality scores for next-generation sequencing, illumina inc.Technical report. 2011."},{"issue":"15","key":"2883_CR15","doi-asserted-by":"publisher","first-page":"2130","DOI":"10.1093\/bioinformatics\/btu183","volume":"30","author":"R C\u00e1novas","year":"2014","unstructured":"C\u00e1novas R, Moffat A, Turpin A. Lossy compression of quality scores in genomic data. Bioinformatics. 2014; 30(15):2130\u20136. https:\/\/doi.org\/10.1093\/bioinformatics\/btu183.","journal-title":"Bioinformatics"},{"key":"2883_CR16","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1186\/1471-2105-14-187","volume":"14","author":"I Ochoa","year":"2013","unstructured":"Ochoa I, Asnani H, Bharadia D, Chowdhury M, Weissman T, Yona G. QualComp: a new lossy compressor for quality scores based on rate distortion theory. BMC Bioinformatics. 2013; 14:187. https:\/\/doi.org\/10.1186\/1471-2105-14-187.","journal-title":"BMC Bioinformatics"},{"issue":"2","key":"2883_CR17","first-page":"183","volume":"18","author":"I Ochoa","year":"2017","unstructured":"Ochoa I, Hernaez M, Goldfeder R, Weissman T, Ashley E. Effect of lossy compression of quality scores on variant calling. Brief Bioinform. 2017; 18(2):183\u201394.","journal-title":"Brief Bioinform"},{"issue":"1","key":"2883_CR18","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1093\/bioinformatics\/btt257","volume":"30","author":"L Janin","year":"2014","unstructured":"Janin L, Rosone G, Cox AJ. Adaptive reference-free compression of sequence quality scores. Bioinformatics. 2014; 30(1):24\u201330.","journal-title":"Bioinformatics"},{"key":"2883_CR19","doi-asserted-by":"publisher","first-page":"288","DOI":"10.1186\/s12859-015-0709-7","volume":"16","author":"G Benoit","year":"2015","unstructured":"Benoit G, Lemaitre C, Lavenier D, Drezen E, Dayris T, Uricaru R, Rizk G. Reference-free compression of high throughput sequencing data with a probabilistic de Bruijn graph. BMC Bioinformatics. 2015; 16:288. https:\/\/doi.org\/10.1186\/s12859-015-0709-7.","journal-title":"BMC Bioinformatics"},{"key":"2883_CR20","volume-title":"Research in Computational Molecular Biology","author":"YW Yu","year":"2014","unstructured":"Yu YW, Yorukoglu D, Berger B. Traversing the k-mer landscape of ngs read datasets for quality score sparsification In: Sharan R, editor. Research in Computational Molecular Biology. Cham: Springer: 2014. p. 385\u201399."},{"key":"2883_CR21","unstructured":"Burrows M, Wheeler DJ. A block-sorting lossless data compression algorithm. Technical report. 1994."},{"issue":"9","key":"2883_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/1471-2105-15-S9-S1","volume":"15","author":"M Comin","year":"2014","unstructured":"Comin M, Schimd M. Assembly-free genome comparison based on next-generation sequencing reads and variable length patterns. BMC Bioinformatics. 2014; 15(9):1\u201310.","journal-title":"BMC Bioinformatics"},{"issue":"4","key":"2883_CR23","doi-asserted-by":"publisher","first-page":"628","DOI":"10.1109\/TCBB.2014.2306830","volume":"11","author":"M Comin","year":"2014","unstructured":"Comin M, Verzotto D. Beyond fixed-resolution alignment-free measures for mammalian enhancers sequence comparison. IEEE\/ACM Trans Comput Biol Bioinforma. 2014; 11(4):628\u201337. https:\/\/doi.org\/10.1109\/TCBB.2014.2306830.","journal-title":"IEEE\/ACM Trans Comput Biol Bioinforma"},{"issue":"10","key":"2883_CR24","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1186\/s12864-017-4273-6","volume":"18","author":"S Girotto","year":"2017","unstructured":"Girotto S, Comin M, Pizzi C. Higher recall in metagenomic sequence classification exploiting overlapping reads. BMC Genomics. 2017; 18(10):917.","journal-title":"BMC Genomics"},{"key":"2883_CR25","volume-title":"Biomedical Engineering Systems and Technologies","author":"J Qian","year":"2018","unstructured":"Qian J, Marchiori D, Comin M. Fast and sensitive classification of short metagenomic reads with skraken In: Peixoto N, Silveira M, Ali HH, Maciel C, van den Broek EL, editors. Biomedical Engineering Systems and Technologies. Cham: Springer: 2018. p. 212\u201326."},{"key":"2883_CR26","doi-asserted-by":"publisher","DOI":"10.5220\/0007369100210029","volume-title":"Proceedings of the 12th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 3: BIOINFORMATICS","author":"Y Shibuya","year":"2019","unstructured":"Shibuya Y, Comin M. Indexing K-mers in Linear-space for Quality Value Compression. In: Proceedings of the 12th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 3: BIOINFORMATICS. Set\u00fabal: SciTePress: 2019. p. 21\u20139. https:\/\/doi.org\/10.5220\/0007369100210029."},{"issue":"14","key":"2883_CR27","doi-asserted-by":"publisher","first-page":"1754","DOI":"10.1093\/bioinformatics\/btp324","volume":"25","author":"H Li","year":"2009","unstructured":"Li H, Durbin R. Fast and accurate short read alignment with Burrows-Wheeler transform. Bioinformatics (Oxford, England). 2009; 25(14):1754\u201360. https:\/\/doi.org\/10.1093\/bioinformatics\/btp324.","journal-title":"Bioinformatics (Oxford, England)"},{"issue":"5","key":"2883_CR28","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1093\/bioinformatics\/btp698","volume":"26","author":"H Li","year":"2010","unstructured":"Li H, Durbin R. Fast and accurate long-read alignment with Burrows-Wheeler transform. Bioinformatics (Oxford, England). 2010; 26(5):589\u201395. https:\/\/doi.org\/10.1093\/bioinformatics\/btp698.","journal-title":"Bioinformatics (Oxford, England)"},{"issue":"6","key":"2883_CR29","doi-asserted-by":"publisher","first-page":"764","DOI":"10.1093\/bioinformatics\/btr011","volume":"27","author":"G Mar\u00e7ais","year":"2011","unstructured":"Mar\u00e7ais G, Kingsford C. A fast, lock-free approach for efficient parallel counting of occurrences of k-mers. Bioinformatics. 2011; 27(6):764\u201370. https:\/\/doi.org\/10.1093\/bioinformatics\/btr011.","journal-title":"Bioinformatics"},{"key":"2883_CR30","volume-title":"Proceedings of the 41st Annual Symposium on Foundations of Computer Science. FOCS \u201900","author":"P Ferragina","year":"2000","unstructured":"Ferragina P, Manzini G. Opportunistic Data Structures with Applications. In: Proceedings of the 41st Annual Symposium on Foundations of Computer Science. FOCS \u201900. Washington: IEEE Computer Society: 2000. p. 390. http:\/\/dl.acm.org\/citation.cfm?id=795666.796543."},{"issue":"4","key":"2883_CR31","doi-asserted-by":"publisher","first-page":"552","DOI":"10.1145\/1082036.1082039","volume":"52","author":"P Ferragina","year":"2005","unstructured":"Ferragina P, Manzini G. Indexing Compressed Text. J ACM. 2005; 52(4):552\u201381. https:\/\/doi.org\/10.1145\/1082036.1082039.","journal-title":"J ACM"},{"issue":"14","key":"2883_CR32","doi-asserted-by":"publisher","first-page":"1754","DOI":"10.1093\/bioinformatics\/btp324","volume":"25","author":"H Li","year":"2009","unstructured":"Li H, Durbin R. Fast and accurate short read alignment with burrows\u2013wheeler transform. Bioinformatics. 2009; 25(14):1754\u201360.","journal-title":"Bioinformatics"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-019-2883-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s12859-019-2883-5\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s12859-019-2883-5.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,11,21]],"date-time":"2020-11-21T00:12:32Z","timestamp":1605917552000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/s12859-019-2883-5"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11]]},"references-count":32,"journal-issue":{"issue":"S9","published-print":{"date-parts":[[2019,11]]}},"alternative-id":["2883"],"URL":"https:\/\/doi.org\/10.1186\/s12859-019-2883-5","relation":{},"ISSN":["1471-2105"],"issn-type":[{"value":"1471-2105","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11]]},"assertion":[{"value":"30 April 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 May 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"22 November 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}},{"value":"Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Publisher\u2019s Note"}}],"article-number":"302"}}