{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,23]],"date-time":"2026-04-23T07:23:03Z","timestamp":1776928983346,"version":"3.51.2"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1009935","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,3,21]],"date-time":"2022-03-21T00:00:00Z","timestamp":1647820800000}}],"reference-count":40,"publisher":"Public Library of Science (PLoS)","issue":"3","license":[{"start":{"date-parts":[[2022,3,9]],"date-time":"2022-03-09T00:00:00Z","timestamp":1646784000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>Gene set enrichment tests (a.k.a. functional enrichment analysis) are among the most frequently used methods in computational biology. Despite this popularity, there are concerns that these methods are being applied incorrectly and the results of some peer-reviewed publications are unreliable. These problems include the use of inappropriate background gene lists, lack of false discovery rate correction and lack of methodological detail. To ascertain the frequency of these issues in the literature, we performed a screen of 186 open-access research articles describing functional enrichment results. We find that 95% of analyses using over-representation tests did not implement an appropriate background gene list or did not describe this in the methods. Failure to perform p-value correction for multiple tests was identified in 43% of analyses. Many studies lacked detail in the methods section about the tools and gene sets used. An extension of this survey showed that these problems are not associated with journal or article level bibliometrics. Using seven independent RNA-seq datasets, we show misuse of enrichment tools alters results substantially. In conclusion, most published functional enrichment studies suffered from one or more major flaws, highlighting the need for stronger standards for enrichment analysis.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1009935","type":"journal-article","created":{"date-parts":[[2022,3,9]],"date-time":"2022-03-09T18:41:45Z","timestamp":1646851305000},"page":"e1009935","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":111,"title":["Urgent need for consistent standards in functional enrichment analysis"],"prefix":"10.1371","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1777-4964","authenticated-orcid":true,"given":"Kaumadi","family":"Wijesooriya","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3047-8887","authenticated-orcid":true,"given":"Sameer A.","family":"Jadaan","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1232-2627","authenticated-orcid":true,"given":"Kaushalya L.","family":"Perera","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3637-4007","authenticated-orcid":true,"given":"Tanuveer","family":"Kaur","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7688-6974","authenticated-orcid":true,"given":"Mark","family":"Ziemann","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2022,3,9]]},"reference":[{"key":"pcbi.1009935.ref001","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1038\/75556","article-title":"Gene ontology: tool for the unification of biology. The Gene Ontology Consortium.","volume":"25","author":"M Ashburner","year":"2000","journal-title":"Nat Genet"},{"key":"pcbi.1009935.ref002","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1093\/nar\/28.1.27","article-title":"KEGG: kyoto encyclopedia of genes and genomes","volume":"28","author":"M Kanehisa","year":"2000","journal-title":"Nucleic Acids Res"},{"key":"pcbi.1009935.ref003","doi-asserted-by":"crossref","first-page":"R70","DOI":"10.1186\/gb-2003-4-10-r70","article-title":"Identifying biological themes within lists of genes with EASE","volume":"4","author":"DA Hosack","year":"2003","journal-title":"Genome Biol"},{"key":"pcbi.1009935.ref004","first-page":"98","article-title":"Global functional profiling of gene expression","volume":"81","author":"S Draghici","year":"2003","journal-title":"Genomics"},{"key":"pcbi.1009935.ref005","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1038\/nrg2363","article-title":"Use and misuse of the gene ontology annotations","volume":"9","author":"SY Rhee","year":"2008","journal-title":"Nat Rev Genet"},{"key":"pcbi.1009935.ref006","doi-asserted-by":"crossref","first-page":"202","DOI":"10.1186\/1479-7364-4-3-202","article-title":"An introduction to effective use of enrichment analysis software","volume":"4","author":"H Tipney","year":"2010","journal-title":"Hum Genomics"},{"key":"pcbi.1009935.ref007","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1007\/978-1-60761-175-2_6","article-title":"Gene set enrichment analysis","volume":"563","author":"CA Tilford","year":"2009","journal-title":"Methods Mol Biol"},{"key":"pcbi.1009935.ref008","doi-asserted-by":"crossref","first-page":"e0193515","DOI":"10.1371\/journal.pone.0193515","article-title":"From SNPs to pathways: Biological interpretation of type 2 diabetes (T2DM) genome wide association study (GWAS) results.","volume":"13","author":"E Cirillo","year":"2018","journal-title":"PLoS One"},{"key":"pcbi.1009935.ref009","doi-asserted-by":"crossref","first-page":"15545","DOI":"10.1073\/pnas.0506580102","article-title":"Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles","volume":"102","author":"A Subramanian","year":"2005","journal-title":"Proc Natl Acad Sci U S A"},{"key":"pcbi.1009935.ref010","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1186\/s12864-020-06856-9","article-title":"Mitch: Multi-contrast pathway enrichment for multi-omics and single-cell profiling data","volume":"21","author":"A Kaspi","year":"2020","journal-title":"BMC Genomics"},{"key":"pcbi.1009935.ref011","doi-asserted-by":"crossref","first-page":"5146","DOI":"10.1093\/bioinformatics\/btz447","article-title":"Gene set enrichment for reproducible science: comparison of CERNO and eight other algorithms","volume":"35","author":"J Zyla","year":"2019","journal-title":"Bioinformatics"},{"key":"pcbi.1009935.ref012","doi-asserted-by":"crossref","first-page":"654","DOI":"10.3389\/fgene.2020.00654","article-title":"Gene set analysis: Challenges, opportunities, and future research.","volume":"11","author":"F Maleki","year":"2020","journal-title":"Front Genet."},{"key":"pcbi.1009935.ref013","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1186\/s12859-021-04124-5","article-title":"Popularity and performance of bioinformatics software: the case of gene set analysis","volume":"22","author":"C Xie","year":"2021","journal-title":"BMC Bioinformatics"},{"key":"pcbi.1009935.ref014","doi-asserted-by":"crossref","first-page":"980","DOI":"10.1093\/bioinformatics\/btm051","article-title":"Analyzing gene expression data in terms of gene sets: methodological issues","volume":"23","author":"JJ Goeman","year":"2007","journal-title":"Bioinformatics"},{"key":"pcbi.1009935.ref015","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1177\/0962280209351908","article-title":"Gene set enrichment analysis made simple","volume":"18","author":"RA Irizarry","year":"2009","journal-title":"Stat Methods Med Res"},{"key":"pcbi.1009935.ref016","doi-asserted-by":"crossref","first-page":"472","DOI":"10.1177\/0962280212460441","article-title":"The limitations of simple gene set enrichment analysis assuming gene independence","volume":"25","author":"P Tamayo","year":"2016","journal-title":"Stat Methods Med Res"},{"key":"pcbi.1009935.ref017","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1186\/s12864-021-07502-8","article-title":"Pathway size matters: the influence of pathway granularity on over-representation (enrichment analysis) statistics","volume":"22","author":"PD Karp","year":"2021","journal-title":"BMC Genomics"},{"key":"pcbi.1009935.ref018","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1186\/s13059-015-0761-7","article-title":"Multiple sources of bias confound functional enrichment analysis of global -omics data","volume":"16","author":"JA Timmons","year":"2015","journal-title":"Genome Biol"},{"key":"pcbi.1009935.ref019","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1016\/j.cels.2015.12.004","article-title":"The Molecular Signatures Database (MSigDB) hallmark gene set collection","volume":"1","author":"A Liberzon","year":"2015","journal-title":"Cell Syst"},{"key":"pcbi.1009935.ref020","doi-asserted-by":"crossref","first-page":"281","DOI":"10.1093\/bib\/bbr049","article-title":"Gene set enrichment analysis: performance evaluation and usage guidelines","volume":"13","author":"JH Hung","year":"2012","journal-title":"Brief Bioinform"},{"key":"pcbi.1009935.ref021","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1038\/ng1201-365","article-title":"Minimum information about a microarray experiment (MIAME)-toward standards for microarray data","volume":"29","author":"A Brazma","year":"2001","journal-title":"Nat Genet"},{"key":"pcbi.1009935.ref022","doi-asserted-by":"crossref","first-page":"S27","DOI":"10.1186\/1471-2164-11-S4-S27","article-title":"Advancing standards for bioinformatics activities: persistence, reproducibility, disambiguation and Minimum Information About a Bioinformatics investigation (MIABi).","volume":"11","author":"TW Tan","year":"2010","journal-title":"BMC Genomics"},{"key":"pcbi.1009935.ref023","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1126\/science.1213847","article-title":"Reproducible research in computational science","volume":"334","author":"RD Peng","year":"2011","journal-title":"Science"},{"key":"pcbi.1009935.ref024","doi-asserted-by":"crossref","first-page":"2163","DOI":"10.1038\/s41598-021-81794-4","article-title":"Valproic acid influences the expression of genes implicated with hyperglycaemia-induced complement and coagulation pathways","volume":"11","author":"MB Felisbino","year":"2021","journal-title":"Sci Rep"},{"key":"pcbi.1009935.ref025","doi-asserted-by":"crossref","first-page":"406","DOI":"10.1186\/s13059-014-0406-2","article-title":"DNMT inhibitors reverse a specific signature of aberrant promoter DNA methylation and associated gene silencing in AML","volume":"15","author":"K Lund","year":"2014","journal-title":"Genome Biol"},{"key":"pcbi.1009935.ref026","doi-asserted-by":"crossref","first-page":"1271","DOI":"10.1101\/gr.168781.113","article-title":"Vascular histone deacetylation by pharmacological HDAC inhibition","volume":"24","author":"H Rafehi","year":"2014","journal-title":"Genome Res"},{"key":"pcbi.1009935.ref027","doi-asserted-by":"crossref","first-page":"4471","DOI":"10.1007\/s00018-014-1651-y","article-title":"Deep sequencing reveals novel Set7 networks","volume":"71","author":"ST Keating","year":"2014","journal-title":"Cell Mol Life Sci"},{"key":"pcbi.1009935.ref028","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1016\/j.mito.2020.07.003","article-title":"OXPHOS bioenergetic compensation does not explain disease penetrance in Leber hereditary optic neuropathy","volume":"54","author":"MIG Lopez Sanchez","year":"2020","journal-title":"Mitochondrion"},{"key":"pcbi.1009935.ref029","doi-asserted-by":"crossref","first-page":"1036","DOI":"10.1016\/j.cell.2020.04.026","article-title":"Imbalanced Host Response to SARS-CoV-2 Drives Development of COVID-19","volume":"181","author":"D Blanco-Melo","year":"2020","journal-title":"Cell"},{"key":"pcbi.1009935.ref030","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1080\/15592294.2017.1371892","article-title":"Systems approach to the pharmacological actions of HDAC inhibitors reveals EP300 activities and convergent mechanisms of regulation in diabetes","volume":"12","author":"H Rafehi","year":"2017","journal-title":"Epigenetics"},{"key":"pcbi.1009935.ref031","doi-asserted-by":"crossref","first-page":"e109","DOI":"10.1093\/nar\/gkw263","article-title":"iGEMS: an integrated model for identification of alternative exon usage events","volume":"44","author":"S Sood","year":"2016","journal-title":"Nucleic Acids Res"},{"issue":"1","key":"pcbi.1009935.ref032","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1186\/s13059-021-02337-8","article-title":"Best practices on the differential expression analysis of multi-species RNA-seq","volume":"22","author":"M Chung","year":"2021","journal-title":"Genome Biol"},{"key":"pcbi.1009935.ref033","first-page":"6461","article-title":"Effect of low-expression gene filtering on detection of differentially expressed genes in RNA-seq data","volume":"2015","author":"Y Sha","year":"2015","journal-title":"Annu Int Conf IEEE Eng Med Biol Soc"},{"key":"pcbi.1009935.ref034","doi-asserted-by":"crossref","first-page":"W90","DOI":"10.1093\/nar\/gkw377","article-title":"Enrichr: a comprehensive gene set enrichment analysis web server 2016 update","volume":"44","author":"MV Kuleshov","year":"2016","journal-title":"Nucleic Acids Res"},{"key":"pcbi.1009935.ref035","doi-asserted-by":"crossref","first-page":"e1003343","DOI":"10.1371\/journal.pcbi.1003343","article-title":"Ten quick tips for using the gene ontology","volume":"9","author":"JA Blake","year":"2013","journal-title":"PLoS Comput Biol"},{"key":"pcbi.1009935.ref036","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1007\/978-1-4939-3743-1_13","article-title":"Gene-Category Analysis","volume":"1446","author":"S. Bauer","year":"2017","journal-title":"Methods Mol Biol"},{"key":"pcbi.1009935.ref037","doi-asserted-by":"crossref","first-page":"giz022","DOI":"10.1093\/gigascience\/giz022","article-title":"Digital expression explorer 2: a repository of uniformly processed RNA sequencing data","volume":"8","author":"M Ziemann","year":"2019","journal-title":"Gigascience"},{"key":"pcbi.1009935.ref038","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1186\/s13059-014-0550-8","article-title":"Moderated estimation of fold change and dispersion for RNA-seq data with DESeq2","volume":"15","author":"MI Love","year":"2014","journal-title":"Genome Biol"},{"key":"pcbi.1009935.ref039","first-page":"D498","article-title":"The reactome pathway knowledgebase","volume":"48","author":"B Jassal","year":"2020","journal-title":"Nucleic Acids Res"},{"key":"pcbi.1009935.ref040","doi-asserted-by":"crossref","first-page":"284","DOI":"10.1089\/omi.2011.0118","article-title":"clusterProfiler: an R package for comparing biological themes among gene clusters","volume":"16","author":"G Yu","year":"2012","journal-title":"OMICS"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1009935","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2022,3,21]],"date-time":"2022-03-21T00:00:00Z","timestamp":1647820800000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009935","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2022,3,21]],"date-time":"2022-03-21T17:43:34Z","timestamp":1647884614000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1009935"}},"subtitle":[],"editor":[{"given":"Melissa L.","family":"Kemp","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2022,3,9]]},"references-count":40,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,3,9]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1009935","relation":{"new_version":[{"id-type":"doi","id":"10.1371\/journal.pcbi.1009935","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,9]]}}}