{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,14]],"date-time":"2026-07-14T01:48:37Z","timestamp":1783993717507,"version":"3.55.0"},"reference-count":37,"publisher":"Oxford University Press (OUP)","issue":"14","license":[{"start":{"date-parts":[[2019,7,8]],"date-time":"2019-07-08T00:00:00Z","timestamp":1562544000000},"content-version":"vor","delay-in-days":7,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Austrian Cancer Aid","award":["17003"],"award-info":[{"award-number":["17003"]}]},{"DOI":"10.13039\/501100002428","name":"Austrian Science Fund","doi-asserted-by":"publisher","award":["T 974-B30"],"award-info":[{"award-number":["T 974-B30"]}],"id":[{"id":"10.13039\/501100002428","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2019,7,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Motivation<\/jats:title>\n                    <jats:p>The composition and density of immune cells in the tumor microenvironment (TME) profoundly influence tumor progression and success of anti-cancer therapies. Flow cytometry, immunohistochemistry staining or single-cell sequencing are often unavailable such that we rely on computational methods to estimate the immune-cell composition from bulk RNA-sequencing (RNA-seq) data. Various methods have been proposed recently, yet their capabilities and limitations have not been evaluated systematically. A general guideline leading the research community through cell type deconvolution is missing.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We developed a systematic approach for benchmarking such computational methods and assessed the accuracy of tools at estimating nine different immune- and stromal cells from bulk RNA-seq samples. We used a single-cell RNA-seq dataset of \u223c11\u00a0000 cells from the TME to simulate bulk samples of known cell type proportions, and validated the results using independent, publicly available gold-standard estimates. This allowed us to analyze and condense the results of more than a hundred thousand predictions to provide an exhaustive evaluation across seven computational methods over nine cell types and \u223c1800 samples from five simulated and real-world datasets. We demonstrate that computational deconvolution performs at high accuracy for well-defined cell-type signatures and propose how fuzzy cell-type signatures can be improved. We suggest that future efforts should be dedicated to refining cell population definitions and finding reliable signatures.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Availability and implementation<\/jats:title>\n                    <jats:p>A snakemake pipeline to reproduce the benchmark is available at https:\/\/github.com\/grst\/immune_deconvolution_benchmark. An R package allows the community to perform integrated deconvolution using different methods (https:\/\/grst.github.io\/immunedeconv).<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Supplementary information<\/jats:title>\n                    <jats:p>Supplementary data are available at Bioinformatics online.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/bioinformatics\/btz363","type":"journal-article","created":{"date-parts":[[2019,5,9]],"date-time":"2019-05-09T15:21:53Z","timestamp":1557415313000},"page":"i436-i445","source":"Crossref","is-referenced-by-count":810,"title":["Comprehensive evaluation of transcriptome-based cell-type quantification methods for immuno-oncology"],"prefix":"10.1093","volume":"35","author":[{"given":"Gregor","family":"Sturm","sequence":"first","affiliation":[{"name":"Chair of Experimental Bioinformatics, TUM School of Life Sciences Weihenstephan, Technical University of Munich, Freising, Germany"},{"name":"Pieris Pharmaceuticals GmbH, Freising, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Francesca","family":"Finotello","sequence":"additional","affiliation":[{"name":"Biocenter, Division of Bioinformatics, Medical University of Innsbruck, Innsbruck, Austria"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Florent","family":"Petitprez","sequence":"additional","affiliation":[{"name":"Cordeliers Research Centre, UMRS_1138, INSERM, University Paris-Descartes, Sorbonne University, Paris, France"},{"name":"Programme Cartes d\u2019Identit\u00e9 des Tumeurs, Ligue Nationale Contre le Cancer, Paris, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jitao David","family":"Zhang","sequence":"additional","affiliation":[{"name":"Roche Pharma Research and Early Development, Pharmaceutical Sciences, Roche Innovation Center Basel, F. Hoffmann-La Roche Ltd, Basel, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jan","family":"Baumbach","sequence":"additional","affiliation":[{"name":"Chair of Experimental Bioinformatics, TUM School of Life Sciences Weihenstephan, Technical University of Munich, Freising, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wolf H","family":"Fridman","sequence":"additional","affiliation":[{"name":"Cordeliers Research Centre, UMRS_1138, INSERM, University Paris-Descartes, Sorbonne University, Paris, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Markus","family":"List","sequence":"additional","affiliation":[{"name":"Big Data in BioMedicine Group, Chair of Experimental Bioinformatis, TUM School of Life Sciences Weihenstephan, Technical University of Munich, Freising, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tatsiana","family":"Aneichyk","sequence":"additional","affiliation":[{"name":"Pieris Pharmaceuticals GmbH, Freising, Germany"},{"name":"Independent Data Lab UG, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2019,7,5]]},"reference":[{"key":"2023062712375773400_btz363-B1","author":"Aran","year":"2018"},{"key":"2023062712375773400_btz363-B2","doi-asserted-by":"crossref","first-page":"220.","DOI":"10.1186\/s13059-017-1349-1","article-title":"xcell: digitally portraying the tissue cellular heterogeneity landscape","volume":"18","author":"Aran","year":"2017","journal-title":"Genome Biol"},{"key":"2023062712375773400_btz363-B3","doi-asserted-by":"crossref","first-page":"1969","DOI":"10.1093\/bioinformatics\/bty019","article-title":"Computational deconvolution of transcriptomics data from mixed cell populations","volume":"34","author":"Avila Cobos","year":"2018","journal-title":"Bioinformatics"},{"key":"2023062712375773400_btz363-B4","doi-asserted-by":"crossref","first-page":"1293","DOI":"10.1016\/j.cell.2018.05.060","article-title":"Single-cell map of diverse immune phenotypes in the breast tumor microenvironment","volume":"174","author":"Azizi","year":"2018","journal-title":"Cell"},{"key":"2023062712375773400_btz363-B5","doi-asserted-by":"crossref","first-page":"218.","DOI":"10.1186\/s13059-016-1070-5","article-title":"Estimating the population abundance of tissue-infiltrating immune and stromal cell populations using gene expression","volume":"17","author":"Becht","year":"2016","journal-title":"Genome Biol"},{"key":"2023062712375773400_btz363-B6","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1111\/imm.12117","article-title":"Human dendritic cell subsets","volume":"140","author":"Collin","year":"2013","journal-title":"Immunology"},{"key":"2023062712375773400_btz363-B7","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1093\/bioinformatics\/bts635","article-title":"STAR: ultrafast universal RNA-seq aligner","volume":"29","author":"Dobin","year":"2013","journal-title":"Bioinformatics"},{"key":"2023062712375773400_btz363-B8","doi-asserted-by":"crossref","first-page":"e66629.","DOI":"10.1371\/journal.pone.0066629","article-title":"LifeMap discovery: the embryonic development, stem cells, and regenerative medicine research portal","volume":"8","author":"Edgar","year":"2013","journal-title":"PLoS One"},{"key":"2023062712375773400_btz363-B9","doi-asserted-by":"crossref","first-page":"1031","DOI":"10.1007\/s00262-018-2150-z","article-title":"Quantifying tumor-infiltrating immune cells from transcriptomics data","volume":"67","author":"Finotello","year":"2018","journal-title":"Cancer Immunol. Immunother"},{"key":"2023062712375773400_btz363-B119","article-title":"Molecular and pharmacological modulators of the tumor immune contexture revealed by deconvolution of RNA-seq data","author":"Finotello","year":"2017","journal-title":"Genome Medicine"},{"key":"2023062712375773400_btz363-B219","doi-asserted-by":"crossref","first-page":"298","DOI":"10.1038\/nrc3245","article-title":"The immune contexture in human tumours: impact on clinical outcome","volume":"12","author":"Fridman","year":"2012","journal-title":"Nat. Rev. Cancer"},{"key":"2023062712375773400_btz363-B12","doi-asserted-by":"crossref","first-page":"747","DOI":"10.1038\/nrc4015","article-title":"Precision medicine for cancer with next-generation functional diagnostics","volume":"15","author":"Friedman","year":"2015","journal-title":"Nat. Rev. Cancer"},{"key":"2023062712375773400_btz363-B13","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1038\/s41592-018-0046-7","article-title":"Bioconda: sustainable and comprehensive software distribution for the life sciences","volume":"15","author":"Gr\u00fcning","year":"2018","journal-title":"Nat. Methods"},{"key":"2023062712375773400_btz363-B14","doi-asserted-by":"crossref","first-page":"2811","DOI":"10.1093\/bioinformatics\/btu393","article-title":"Circlize implements and enhances circular visualization in R","volume":"30","author":"Gu","year":"2014","journal-title":"Bioinformatics"},{"key":"2023062712375773400_btz363-B15","doi-asserted-by":"crossref","first-page":"978","DOI":"10.1038\/s41591-018-0045-3","article-title":"Global characterization of T cells in non-small-cell lung cancer by single-cell sequencing","volume":"24","author":"Guo","year":"2018","journal-title":"Nat. Med"},{"key":"2023062712375773400_btz363-B16","doi-asserted-by":"crossref","first-page":"7.","DOI":"10.1186\/1471-2105-14-7","article-title":"GSVA: gene set variation analysis for microarray and RNA-seq data","volume":"14","author":"H\u00e4nzelmann","year":"2013","journal-title":"BMC Bioinformatics"},{"key":"2023062712375773400_btz363-B17","doi-asserted-by":"crossref","first-page":"e0118528.","DOI":"10.1371\/journal.pone.0118528","article-title":"A cell-based systems biology assessment of human blood to monitor immune responses after influenza vaccination","volume":"10","author":"Hoek","year":"2015","journal-title":"PLoS One"},{"key":"2023062712375773400_btz363-B18","doi-asserted-by":"crossref","first-page":"118","DOI":"10.1093\/biostatistics\/kxj037","article-title":"Adjusting batch effects in microarray expression data using empirical bayes methods","volume":"8","author":"Johnson","year":"2007","journal-title":"Biostatistics"},{"key":"2023062712375773400_btz363-B19","doi-asserted-by":"crossref","first-page":"2520","DOI":"10.1093\/bioinformatics\/bts480","article-title":"Snakemake\u2014a scalable bioinformatics workflow engine","volume":"28","author":"Koster","year":"2012","journal-title":"Bioinformatics"},{"key":"2023062712375773400_btz363-B20","doi-asserted-by":"crossref","first-page":"1277","DOI":"10.1038\/s41591-018-0096-5","article-title":"Phenotype molding of stromal cells in the lung tumor microenvironment","volume":"24","author":"Lambrechts","year":"2018","journal-title":"Nat. Med"},{"key":"2023062712375773400_btz363-B21","doi-asserted-by":"crossref","first-page":"D19","DOI":"10.1093\/nar\/gkq1019","article-title":"The sequence read archive","volume":"39","author":"Leinonen","year":"2011","journal-title":"Nucleic Acids Res"},{"key":"2023062712375773400_btz363-B22","doi-asserted-by":"crossref","first-page":"323.","DOI":"10.1186\/1471-2105-12-323","article-title":"RSEM: accurate transcript quantification from RNA-Seq data with or without a reference genome","volume":"12","author":"Li","year":"2011","journal-title":"BMC Bioinformatics"},{"key":"2023062712375773400_btz363-B23","doi-asserted-by":"crossref","first-page":"174.","DOI":"10.1186\/s13059-016-1028-7","article-title":"Comprehensive analyses of tumor immunity: implications for cancer immunotherapy","volume":"17","author":"Li","year":"2016","journal-title":"Genome Biol"},{"key":"2023062712375773400_btz363-B24","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1016\/j.coi.2016.06.006","article-title":"High-throughput genomic profiling of tumor-infiltrating leukocytes","volume":"41","author":"Newman","year":"2016","journal-title":"Curr. Opin. Immunol"},{"key":"2023062712375773400_btz363-B25","doi-asserted-by":"crossref","first-page":"453.","DOI":"10.1038\/nmeth.3337","article-title":"Robust enumeration of cell subsets from tissue expression profiles","volume":"12","author":"Newman","year":"2015","journal-title":"Nat. Methods"},{"key":"2023062712375773400_btz363-B26","author":"Newman","year":"2018"},{"key":"2023062712375773400_btz363-B27","doi-asserted-by":"crossref","first-page":"390","DOI":"10.3389\/fonc.2018.00390","article-title":"Quantitative analyses of the tumor microenvironment composition and orientation in the era of precision medicine","volume":"8","author":"Petitprez","year":"2018","journal-title":"Front. Oncol"},{"key":"2023062712375773400_btz363-B28","doi-asserted-by":"crossref","first-page":"981","DOI":"10.1007\/s00262-017-2058-z","article-title":"Transcriptomic analysis of the tumor microenvironment to guide prognosis and immunotherapies","volume":"67","author":"Petitprez","year":"2018","journal-title":"Cancer Immunol. Immunother"},{"key":"2023062712375773400_btz363-B29","doi-asserted-by":"crossref","first-page":"e26476","DOI":"10.7554\/eLife.26476","article-title":"Simultaneous enumeration of cancer and immune cell types from bulk tumor gene expression data","volume":"6","author":"Racle","year":"2017","journal-title":"Elife"},{"key":"2023062712375773400_btz363-B30","doi-asserted-by":"crossref","first-page":"998","DOI":"10.1016\/j.cell.2018.10.038","article-title":"Defining T cell states associated with response to checkpoint immunotherapy in melanoma","volume":"175","author":"Sade-Feldman","year":"2018","journal-title":"Cell"},{"key":"2023062712375773400_btz363-B31","doi-asserted-by":"crossref","first-page":"2032.","DOI":"10.1038\/s41467-017-02289-3","article-title":"Estimation of immune cell content in tumour tissue using single-cell RNA-seq data","volume":"8","author":"Schelker","year":"2017","journal-title":"Nat. Commun"},{"key":"2023062712375773400_btz363-B32","doi-asserted-by":"crossref","first-page":"817","DOI":"10.1038\/nmeth.3035","article-title":"Single-cell genome-wide bisulfite sequencing for assessing epigenetic heterogeneity","volume":"11","author":"Smallwood","year":"2014","journal-title":"Nat. Methods"},{"key":"2023062712375773400_btz363-B33","doi-asserted-by":"crossref","first-page":"105.","DOI":"10.1186\/s12859-017-1511-5","article-title":"A comparison of reference-based algorithms for correcting cell-type heterogeneity in epigenome-wide association studies","volume":"18","author":"Teschendorff","year":"2017","journal-title":"BMC Bioinformatics"},{"key":"2023062712375773400_btz363-B34","doi-asserted-by":"crossref","first-page":"D331","DOI":"10.1093\/nar\/gkw1108","article-title":"Expansion of the gene ontology knowledgebase and resources","volume":"45","year":"2017","journal-title":"Nucleic Acids Res"},{"key":"2023062712375773400_btz363-B35","doi-asserted-by":"crossref","first-page":"4373","DOI":"10.1126\/science.aah4573","article-title":"Single-cell RNA-seq reveals new types of human blood dendritic cells, monocytes, and progenitors","volume":"356","author":"Villani","year":"2017","journal-title":"Science"},{"key":"2023062712375773400_btz363-B36","doi-asserted-by":"crossref","first-page":"277.","DOI":"10.1186\/s12864-017-3661-2","article-title":"Detect tissue heterogeneity in gene expression data with BioQC","volume":"18","author":"Zhang","year":"2017","journal-title":"BMC Genomics"},{"key":"2023062712375773400_btz363-B37","doi-asserted-by":"crossref","first-page":"1342","DOI":"10.1016\/j.cell.2017.05.035","article-title":"Landscape of infiltrating T cells in liver cancer revealed by single-cell sequencing","volume":"169","author":"Zheng","year":"2017","journal-title":"Cell"}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/14\/i436\/50720414\/bioinformatics_35_14_i436.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/35\/14\/i436\/50720414\/bioinformatics_35_14_i436.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,27]],"date-time":"2023-06-27T08:38:40Z","timestamp":1687855120000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/35\/14\/i436\/5529146"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,7]]},"references-count":37,"journal-issue":{"issue":"14","published-print":{"date-parts":[[2019,7,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btz363","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/463828","asserted-by":"object"}]},"ISSN":["1367-4803","1367-4811"],"issn-type":[{"value":"1367-4803","type":"print"},{"value":"1367-4811","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2019,7]]},"published":{"date-parts":[[2019,7]]}}}