{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,25]],"date-time":"2026-06-25T01:18:59Z","timestamp":1782350339820,"version":"3.54.5"},"reference-count":37,"publisher":"Oxford University Press (OUP)","issue":"6","license":[{"start":{"date-parts":[[2021,8,4]],"date-time":"2021-08-04T00:00:00Z","timestamp":1628035200000},"content-version":"vor","delay-in-days":1,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["1457106"],"award-info":[{"award-number":["1457106"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000054","name":"National Cancer Institute","doi-asserted-by":"publisher","award":["R01CA229618"],"award-info":[{"award-number":["R01CA229618"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["U01HG007598"],"award-info":[{"award-number":["U01HG007598"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"National Science Foundation","doi-asserted-by":"publisher","award":["2041984"],"award-info":[{"award-number":["2041984"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2021,11,5]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Estimating cell type composition of blood and tissue samples is a biological challenge relevant in both laboratory studies and clinical care. In recent years, a number of computational tools have been developed to estimate cell type abundance using gene expression data. Although these tools use a variety of approaches, they all leverage expression profiles from purified cell types to evaluate the cell type composition within samples. In this study, we compare 12 cell type quantification tools and evaluate their performance while using each of 10 separate reference profiles. Specifically, we have run each tool on over 4000 samples with known cell type proportions, spanning both immune and stromal cell types. A total of 12 of these represent in vitro synthetic mixtures and 300 represent in silico synthetic mixtures prepared using single-cell data. A final 3728 clinical samples have been collected from the Framingham cohort, for which cell populations have been quantified using electrical impedance cell counting. When tools are applied to the Framingham dataset, the tool Estimating the Proportions of Immune and Cancer cells (EPIC) produces the highest correlation, whereas Gene Expression Deconvolution Interactive Tool (GEDIT) produces the lowest error. The best tool for other datasets is varied, but CIBERSORT and GEDIT most consistently produce accurate results. We find that optimal reference depends on the tool used, and report suggested references to be used with each tool. Most tools return results within minutes, but on large datasets runtimes for CIBERSORT can exceed hours or even days. We conclude that deconvolution methods are capable of returning high-quality results, but that proper reference selection is critical.<\/jats:p>","DOI":"10.1093\/bib\/bbab265","type":"journal-article","created":{"date-parts":[[2021,6,25]],"date-time":"2021-06-25T07:08:36Z","timestamp":1624604916000},"source":"Crossref","is-referenced-by-count":27,"title":["Systematic evaluation of transcriptomics-based deconvolution methods and references using thousands of clinical samples"],"prefix":"10.1093","volume":"22","author":[{"given":"Brian B","family":"Nadel","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Meritxell","family":"Oliva","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin L","family":"Shou","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Keith","family":"Mitchell","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Feiyang","family":"Ma","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dennis J","family":"Montoya","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alice","family":"Mouton","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sarah","family":"Kim-Hellmuth","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Barbara E","family":"Stranger","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matteo","family":"Pellegrini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4770-3443","authenticated-orcid":false,"given":"Serghei","family":"Mangul","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2021,8,3]]},"reference":[{"key":"2022080300105082900_ref1","doi-asserted-by":"crossref","first-page":"938","DOI":"10.1038\/nm.3909","article-title":"The prognostic landscape of genes and infiltrating immune cells across human cancers","volume":"21","author":"Gentles","year":"2015","journal-title":"Nat Med"},{"issue":"4","key":"2022080300105082900_ref2","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1038\/nrc3245","article-title":"The immune contexture in human tumours: impact on clinical outcome","volume":"12","author":"Fridman","year":"2012","journal-title":"Nat Rev Cancer"},{"issue":"1","key":"2022080300105082900_ref3","doi-asserted-by":"crossref","first-page":"174","DOI":"10.1186\/s13059-016-1028-7","article-title":"Comprehensive analyses of tumor immunity: implications for cancer immunotherapy","volume":"17","author":"Li","year":"2016","journal-title":"Genome Biol"},{"issue":"4","key":"2022080300105082900_ref4","doi-asserted-by":"crossref","first-page":"631","DOI":"10.1016\/j.molcel.2017.01.023","article-title":"Comparative analysis of single-cell RNA sequencing methods","volume":"65","author":"Ziegenhain","year":"2017","journal-title":"Mol Cell"},{"issue":"1","key":"2022080300105082900_ref5","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1186\/s13059-018-1593-z","article-title":"Understanding tumor ecosystems by single-cell sequencing: promises and limitations","volume":"19","author":"Ren","year":"2018","journal-title":"Genome Biol"},{"key":"2022080300105082900_ref6","doi-asserted-by":"crossref","first-page":"317","DOI":"10.3389\/fgene.2019.00317","article-title":"Single-cell RNA-Seq technologies and related computational data analysis","volume":"10","author":"Chen","year":"2019","journal-title":"Front Genet"},{"issue":"5","key":"2022080300105082900_ref7","doi-asserted-by":"crossref","first-page":"779","DOI":"10.1016\/j.celrep.2014.02.021","article-title":"Sorting out the FACS: a devil in the details","volume":"6","author":"Hines","year":"2014","journal-title":"Cell Rep"},{"issue":"8","key":"2022080300105082900_ref8","doi-asserted-by":"crossref","first-page":"1083","DOI":"10.1093\/bioinformatics\/btt090","article-title":"DeconRNASeq: a statistical framework for deconvolution of heterogeneous tissue samples based on mRNA-Seq data","volume":"29","author":"Gong","year":"2013","journal-title":"Bioinformatics"},{"issue":"2","key":"2022080300105082900_ref9","doi-asserted-by":"crossref","first-page":"720","DOI":"10.1002\/msb.134947","article-title":"Digital cell quantification identifies global immune cell dynamics during influenza infection","volume":"10","author":"Altboum","year":"2014","journal-title":"Mol Syst Biol"},{"key":"2022080300105082900_ref10","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1038\/nmeth.3337","article-title":"Robust enumeration of cell subsets from tissue expression profiles","volume":"12","author":"Newman","year":"2015","journal-title":"Nat Methods"},{"issue":"1","key":"2022080300105082900_ref11","doi-asserted-by":"crossref","first-page":"218","DOI":"10.1186\/s13059-016-1070-5","article-title":"Estimating the population abundance of tissue-infiltrating immune and stromal cell populations using gene expression","volume":"17","author":"Becht","year":"2016","journal-title":"Genome Biol"},{"key":"2022080300105082900_ref12","doi-asserted-by":"crossref","first-page":"220","DOI":"10.1186\/s13059-017-1349-1","article-title":"xCell: digitally portraying the tissue cellular heterogeneity landscape","volume":"18","author":"Aran","year":"2017","journal-title":"Genome Biol"},{"key":"2022080300105082900_ref13","doi-asserted-by":"publisher","first-page":"2093","DOI":"10.1093\/bioinformatics\/bty926","article-title":"dtangle: accurate and robust cell type deconvolution","volume":"35","author":"Hunt","year":"2019","journal-title":"Bioinformatics"},{"key":"2022080300105082900_ref14","doi-asserted-by":"publisher","DOI":"10.1101\/223180","article-title":"Molecular and pharmacological modulators of the tumor immune contexture revealed by deconvolution of RNA-seq data","author":"Finotello","year":"2019","journal-title":"Genome Med"},{"key":"2022080300105082900_ref15","doi-asserted-by":"publisher","DOI":"10.7554\/eLife.26476","article-title":"Simultaneous enumeration of cancer and immune cell types from bulk tumor gene expression data","volume":"6","author":"Racle","year":"2017","journal-title":"Elife"},{"key":"2022080300105082900_ref16","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1186\/s12859-019-3307-2","article-title":"Guidelines for cell-type heterogeneity quantification based on a comparative analysis of reference-free DNA methylation deconvolution software","volume":"21","author":"Decamps","year":"2020","journal-title":"BMC Bioinformatics"},{"issue":"1","key":"2022080300105082900_ref17","doi-asserted-by":"crossref","first-page":"1393","DOI":"10.1038\/s41467-019-09406-4","article-title":"Systematic benchmarking of omics computational tools","volume":"10","author":"Mangul","year":"2019","journal-title":"Nat Commun"},{"issue":"2","key":"2022080300105082900_ref18","doi-asserted-by":"crossref","DOI":"10.1093\/gigascience\/giab002","article-title":"The Gene Expression Deconvolution Interactive Tool (GEDIT): accurate cell type quantification from gene expression data","volume":"10","author":"Nadel","year":"2021","journal-title":"Giga Science"},{"issue":"14","key":"2022080300105082900_ref19","doi-asserted-by":"crossref","first-page":"i436","DOI":"10.1093\/bioinformatics\/btz363","article-title":"Comprehensive evaluation of transcriptome-based cell-type quantification methods for immuno-oncology","volume":"35","author":"Sturm","year":"2019","journal-title":"Bioinformatics"},{"key":"2022080300105082900_ref20","doi-asserted-by":"publisher","first-page":"6238","DOI":"10.1101\/437533","article-title":"Comprehensive benchmarking and integration of tumour microenvironment cell estimation methods","volume":"79","author":"Jimenez-Sanchez","year":"2019","journal-title":"Cancer Res"},{"key":"2022080300105082900_ref21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1038\/s41467-020-20288-9","article-title":"Benchmarking of cell type deconvolution pipelines for transcriptomics data","volume":"11","author":"Cobos","year":"2020","journal-title":"Nat Commun"},{"key":"2022080300105082900_ref22","doi-asserted-by":"crossref","first-page":"279","DOI":"10.2105\/AJPH.41.3.279","article-title":"Epidemiological approaches to heart disease: the Framingham study","volume":"41","author":"Dawber","year":"1951","journal-title":"Am J Public Health"},{"issue":"4","key":"2022080300105082900_ref23","doi-asserted-by":"crossref","first-page":"518","DOI":"10.1016\/0091-7435(75)90037-7","article-title":"The Framingham offspring study design and preliminary data","volume":"4","author":"Feinleib","year":"1975","journal-title":"Prev Med"},{"issue":"11","key":"2022080300105082900_ref24","doi-asserted-by":"crossref","first-page":"1328","DOI":"10.1093\/aje\/kwm021","article-title":"The Third Generation Cohort of the National Heart, Lung, and Blood Institute\u2019s Framingham Heart Study: design, recruitment, and initial examination","volume":"165","author":"Splansky","year":"2007","journal-title":"Am J Epidemiol"},{"key":"2022080300105082900_ref25","article-title":"CIBERSORT website","author":"AbsCIBERSORT","year":"2018"},{"key":"2022080300105082900_ref26","volume-title":"Solving Least Squares Problems","year":"1995"},{"issue":"1","key":"2022080300105082900_ref27","doi-asserted-by":"crossref","first-page":"824","DOI":"10.1186\/s12864-017-4167-7","article-title":"SaVanT: a web-based tool for the sample-level visualization of molecular signatures in gene expression profiles","volume":"18","author":"Lopez","year":"2017","journal-title":"BMC Genomics"},{"key":"2022080300105082900_ref28","article-title":"nnls: the Lawson-Hanson algorithm for non-negative least squares (NNLS)","author":"Mullen","journal-title":"R package version 1.4"},{"issue":"7","key":"2022080300105082900_ref29","doi-asserted-by":"crossref","first-page":"1611","DOI":"10.1016\/j.cell.2017.10.044","article-title":"Single-cell transcriptomic analysis of primary and metastatic tumor ecosystems in head and neck cancer","volume":"171","author":"Puram","year":"2017","journal-title":"Cell"},{"issue":"24","key":"2022080300105082900_ref30","doi-asserted-by":"crossref","first-page":"3842","DOI":"10.1093\/bioinformatics\/btw535","article-title":"ImmQuant: a user-friendly tool for inferring immune cell-type composition from gene-expression data","volume":"32","author":"Frishberg","year":"2016","journal-title":"Bioinformatics"},{"issue":"1","key":"2022080300105082900_ref31","doi-asserted-by":"crossref","first-page":"4735","DOI":"10.1038\/s41467-018-07242-6","article-title":"Leveraging heterogeneity across multiple datasets increases cell-mixture deconvolution accuracy and reduces biological and technical biases","volume":"9","author":"Vallania","year":"2018","journal-title":"Nat Commun"},{"issue":"1","key":"2022080300105082900_ref32","doi-asserted-by":"crossref","first-page":"14049","DOI":"10.1038\/ncomms14049","article-title":"Massively parallel digital transcriptional profiling of single cells","volume":"8","author":"Zheng","year":"2017","journal-title":"Nat Commun"},{"issue":"10","key":"2022080300105082900_ref33","doi-asserted-by":"crossref","first-page":"1487","DOI":"10.3324\/haematol.2013.094243","article-title":"BLUEPRINT: mapping human blood cell epigenomes","volume":"98","author":"Martens","year":"2013","journal-title":"Haematologica"},{"issue":"1","key":"2022080300105082900_ref34","doi-asserted-by":"crossref","first-page":"632","DOI":"10.1186\/1471-2164-14-632","article-title":"An expression atlas of human primary cells: inference of gene function from coexpression networks","volume":"14","author":"Mabbott","year":"2013","journal-title":"BMC Genomics"},{"issue":"11","key":"2022080300105082900_ref35","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pgen.1006423","article-title":"Survey of the heritability and sparse architecture of gene expression traits across human tissues","volume":"12","author":"Wheeler","year":"2016","journal-title":"PLoS Genet"},{"key":"2022080300105082900_ref36","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1093\/oxfordjournals.aje.a112813","article-title":"An investigation of coronary heart disease in families. The Framingham offspring study","volume-title":"Am J Epidemiol","author":"","year":"1979"},{"key":"2022080300105082900_ref37","first-page":"1328","article-title":"The Third Generation Cohort of the National Heart, Lung, and Blood Institute's Framingham Heart Study: Design, Recruitment, and Initial Examination","volume-title":"Am J Epidemiol","author":"","year":"2007"}],"container-title":["Briefings in Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/22\/6\/bbab265\/42242154\/bbab265.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bib\/article-pdf\/22\/6\/bbab265\/42242154\/bbab265.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,8,2]],"date-time":"2022-08-02T22:54:32Z","timestamp":1659480872000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bib\/article\/doi\/10.1093\/bib\/bbab265\/6338547"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,3]]},"references-count":37,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2021,11,5]]}},"URL":"https:\/\/doi.org\/10.1093\/bib\/bbab265","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2021.03.09.434660","asserted-by":"object"}]},"ISSN":["1467-5463","1477-4054"],"issn-type":[{"value":"1467-5463","type":"print"},{"value":"1477-4054","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2021,11]]},"published":{"date-parts":[[2021,8,3]]},"article-number":"bbab265"}}