{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,26]],"date-time":"2026-02-26T20:34:31Z","timestamp":1772138071314,"version":"3.50.1"},"reference-count":15,"publisher":"Oxford University Press (OUP)","issue":"1","license":[{"start":{"date-parts":[[2024,12,24]],"date-time":"2024-12-24T00:00:00Z","timestamp":1734998400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100014989","name":"Chan Zuckerberg Initiative","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100014989","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,12,26]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>The Variant Call Format (VCF) is widely used in genome sequencing but scales poorly. For instance, we estimate a 150\u2009000 genome VCF would occupy 900 TiB, making it costly and complicated to produce, analyze, and store. The issue stems from VCF\u2019s requirement to densely represent both reference-genotypes and allele-indexed arrays. These requirements lead to unnecessary data duplication and, ultimately, very large files.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>To address these challenges, we introduce the Scalable Variant Call Representation (SVCR). This representation reduces file sizes by ensuring they scale linearly with samples. SVCR\u2019s linear scaling relies on two techniques, both necessary for linearity: local allele indices and reference blocks, which were first introduced by the Genomic Variant Call Format. SVCR is also lossless and mergeable, allowing for N\u2009+\u20091 and N\u2009+\u2009K incremental joint-calling. We present two implementations of SVCR: SVCR-VCF, which encodes SVCR in VCF format, and VDS, which uses Hail\u2019s native format. Our experiments confirm the linear scalability of SVCR-VCF and VDS, in contrast to the super-linear growth seen with standard VCF files. We also discuss the VDS Combiner, a scalable, open-source tool for producing a VDS from GVCFs and unique features of VDS which enable rapid data analysis. SVCR, and VDS in particular, ensure the scientific community can generate, analyze, and disseminate genetics datasets with millions of samples.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>https:\/\/github.com\/hail-is\/hail\/.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btae746","type":"journal-article","created":{"date-parts":[[2024,12,19]],"date-time":"2024-12-19T15:29:02Z","timestamp":1734622142000},"source":"Crossref","is-referenced-by-count":6,"title":["The scalable variant call representation: enabling genetic analysis beyond one million genomes"],"prefix":"10.1093","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1287-3434","authenticated-orcid":false,"given":"Timothy","family":"Poterba","sequence":"first","affiliation":[{"name":"Program in Medical and Population Genetics, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Analytic and Translational Genetics Unit, Massachusetts General Hospital , Boston, MA 02114,","place":["United States"]},{"name":"Stanley Center for Psychiatric Research, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]}]},{"given":"Christopher","family":"Vittal","sequence":"additional","affiliation":[{"name":"Program in Medical and Population Genetics, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Analytic and Translational Genetics Unit, Massachusetts General Hospital , Boston, MA 02114,","place":["United States"]},{"name":"Stanley Center for Psychiatric Research, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]}]},{"given":"Daniel","family":"King","sequence":"additional","affiliation":[{"name":"Program in Medical and Population Genetics, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Analytic and Translational Genetics Unit, Massachusetts General Hospital , Boston, MA 02114,","place":["United States"]},{"name":"Stanley Center for Psychiatric Research, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Novo Nordisk Foundation Center for Genomic Mechanisms of Disease, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]}]},{"given":"Daniel","family":"Goldstein","sequence":"additional","affiliation":[{"name":"Program in Medical and Population Genetics, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Analytic and Translational Genetics Unit, Massachusetts General Hospital , Boston, MA 02114,","place":["United States"]},{"name":"Stanley Center for Psychiatric Research, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]}]},{"given":"Jacqueline I","family":"Goldstein","sequence":"additional","affiliation":[{"name":"Program in Medical and Population Genetics, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Analytic and Translational Genetics Unit, Massachusetts General Hospital , Boston, MA 02114,","place":["United States"]},{"name":"Stanley Center for Psychiatric Research, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]}]},{"given":"Patrick","family":"Schultz","sequence":"additional","affiliation":[{"name":"Program in Medical and Population Genetics, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Analytic and Translational Genetics Unit, Massachusetts General Hospital , Boston, MA 02114,","place":["United States"]},{"name":"Stanley Center for Psychiatric Research, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2878-4671","authenticated-orcid":false,"given":"Konrad J","family":"Karczewski","sequence":"additional","affiliation":[{"name":"Program in Medical and Population Genetics, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Analytic and Translational Genetics Unit, Massachusetts General Hospital , Boston, MA 02114,","place":["United States"]},{"name":"Stanley Center for Psychiatric Research, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Novo Nordisk Foundation Center for Genomic Mechanisms of Disease, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]}]},{"given":"Cotton","family":"Seed","sequence":"additional","affiliation":[{"name":"Program in Medical and Population Genetics, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Analytic and Translational Genetics Unit, Massachusetts General Hospital , Boston, MA 02114,","place":["United States"]},{"name":"Stanley Center for Psychiatric Research, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Novo Nordisk Foundation Center for Genomic Mechanisms of Disease, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1513-6077","authenticated-orcid":false,"given":"Benjamin M","family":"Neale","sequence":"additional","affiliation":[{"name":"Program in Medical and Population Genetics, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Analytic and Translational Genetics Unit, Massachusetts General Hospital , Boston, MA 02114,","place":["United States"]},{"name":"Stanley Center for Psychiatric Research, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]},{"name":"Novo Nordisk Foundation Center for Genomic Mechanisms of Disease, Broad Institute of MIT and Harvard , Cambridge, MA 02142,","place":["United States"]}]}],"member":"286","published-online":{"date-parts":[[2024,12,24]]},"reference":[{"key":"2025012104112209400_btae746-B1","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1186\/s13742-015-0047-8","article-title":"Second-generation PLINK: rising to the challenge of larger and richer datasets","volume":"4","author":"Chang","year":"2015","journal-title":"Gigascience"},{"key":"2025012104112209400_btae746-B2","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1038\/s41586-023-06045-0","article-title":"A genomic mutational constraint map using variation in 76,156 human genomes","volume":"625","author":"Chen","year":"2024","journal-title":"Nature"},{"key":"2025012104112209400_btae746-B3","doi-asserted-by":"publisher","first-page":"2156","DOI":"10.1093\/bioinformatics\/btr330","article-title":"The variant call format and VCFtools","volume":"27","author":"Danecek","year":"2011","journal-title":"Bioinformatics"},{"key":"2025012104112209400_btae746-B5","doi-asserted-by":"crossref","first-page":"e25","DOI":"10.1016\/j.euroneuro.2022.07.058","article-title":"Replacing GWAS arrays: capturing genomic diversity with a novel whole-exome plus low-pass whole genome product","volume":"63","author":"Howrigan","year":"2022","journal-title":"Eur Neuropsychopharmacol"},{"key":"2025012104112209400_btae746-B6","doi-asserted-by":"publisher","first-page":"434","DOI":"10.1038\/s41586-020-2308-7","article-title":"The mutational constraint spectrum quantified from variation in 141,456 humans","volume":"581","author":"Karczewski","year":"2020","journal-title":"Nature"},{"key":"2025012104112209400_btae746-B7","doi-asserted-by":"publisher","first-page":"796","DOI":"10.1101\/gr.278378.12338749656","article-title":"A harmonized public resource of deeply sequenced diverse human genomes","volume":"34","author":"Koenig","year":"2024","journal-title":"Genome Res"},{"key":"2025012104112209400_btae746-B8","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1038\/nmeth.3654","article-title":"Efficient genotype compression and analysis of large genetic-variation data sets","volume":"13","author":"Layer","year":"2016","journal-title":"Nat Methods"},{"key":"2025012104112209400_btae746-B9","doi-asserted-by":"publisher","first-page":"4248","DOI":"10.1093\/bioinformatics\/btab378","article-title":"Sparse allele vectors and the savvy software suite","volume":"37","author":"LeFaive","year":"2021","journal-title":"Bioinformatics"},{"key":"2025012104112209400_btae746-B10","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1038\/nature19057","article-title":"Analysis of protein-coding genetic variation in 60,706 humans","volume":"536","author":"Lek","year":"2016","journal-title":"Nature"},{"key":"2025012104112209400_btae746-B11","doi-asserted-by":"crossref","first-page":"590","DOI":"10.1093\/bioinformatics\/btv613","article-title":"BGT: efficient and flexible genotype query across many samples","volume":"32","author":"Li","year":"2016","journal-title":"Bioinformatics"},{"key":"2025012104112209400_btae746-B12","doi-asserted-by":"publisher","author":"Lin","DOI":"10.1101\/343970,"},{"key":"2025012104112209400_btae746-B13","doi-asserted-by":"publisher","first-page":"5537","DOI":"10.1093\/bioinformatics\/btaa1004","article-title":"Sparse project VCF: efficient encoding of population genotype matrices","volume":"36","author":"Lin","year":"2021","journal-title":"Bioinformatics"},{"key":"2025012104112209400_btae746-B14","doi-asserted-by":"publisher","first-page":"2068","DOI":"10.1016\/j.ajhg.2023.10.011","article-title":"CHARR efficiently estimates contamination from DNA sequencing data","volume":"110","author":"Lu","year":"2023","journal-title":"Am J Hum Genet"},{"key":"2025012104112209400_btae746-B15","doi-asserted-by":"publisher","first-page":"349","DOI":"10.14778\/3025111.3025117","article-title":"The TileDB array data storage manager","volume":"10","author":"Papadopoulos","year":"2016","journal-title":"Proc VLDB Endow"},{"key":"2025012104112209400_btae746-B17","doi-asserted-by":"publisher","first-page":"5582","DOI":"10.1093\/bioinformatics\/btaa1081","article-title":"Accurate, scalable cohort variant calls using DeepVariant and GLnexus","volume":"36","author":"Yun","year":"2021","journal-title":"Bioinformatics"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/advance-article-pdf\/doi\/10.1093\/bioinformatics\/btae746\/61271962\/btae746.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/41\/1\/btae746\/61271962\/btae746.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/41\/1\/btae746\/61271962\/btae746.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,20]],"date-time":"2025-01-20T23:11:41Z","timestamp":1737414701000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/doi\/10.1093\/bioinformatics\/btae746\/7932121"}},"subtitle":[],"editor":[{"given":"Russell","family":"Schwartz","sequence":"additional","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2024,12,24]]},"references-count":15,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,12,26]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btae746","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2024.01.09.574205","asserted-by":"object"}]},"ISSN":["1367-4811"],"issn-type":[{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,1]]},"published":{"date-parts":[[2024,12,24]]},"article-number":"btae746"}}