{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,26]],"date-time":"2025-10-26T21:14:17Z","timestamp":1761513257840,"version":"3.37.3"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"1","content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Bioinformatics"],"published-print":{"date-parts":[[2013,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec><jats:title>Background<\/jats:title><jats:p>In investigating differentially expressed genes or other selected features, researchers conduct hypothesis tests to determine which biological categories, such as those of the Gene Ontology (GO), are enriched for the selected features. Multiple comparison procedures (MCPs) are commonly used to prevent excessive false positive rates. Traditional MCPs, e.g., the Bonferroni method, go to the opposite extreme: strictly controlling a family-wise error rate, resulting in excessive false negative rates. Researchers generally prefer the more balanced approach of instead controlling the false discovery rate (FDR). However, the q-values that methods of FDR control assign to biological categories tend to be too low to reliably estimate the probability that a biological category is not enriched for the preselected features. Thus, we study an application of the other estimators of that probability, which is called the local FDR (LFDR).<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>We considered five LFDR estimators for detecting enriched GO terms: a binomial-based estimator (BBE), a maximum likelihood estimator (MLE), a normalized MLE (NMLE), a histogram-based estimator assuming a theoretical null hypothesis (HBE), and a histogram-based estimator assuming an empirical null hypothesis (HBE-EN). Since NMLE depends not only on the data but also on the specified value of<jats:italic>\u03a0<\/jats:italic><jats:sub>0<\/jats:sub>, the proportion of non-enriched GO terms, it is only advantageous when either<jats:italic>\u03a0<\/jats:italic><jats:sub>0<\/jats:sub>is already known with sufficient accuracy or there are data for only 1 GO term. By contrast, the other estimators work without specifying<jats:italic>\u03a0<\/jats:italic><jats:sub>0<\/jats:sub>but require data for at least 2 GO terms. Our simulation studies yielded the following summaries of the relative performance of each of those four estimators. HBE and HBE-EN produced larger biases for 2, 4, 8, 32, and 100 GO terms than BBE and MLE. BBE has the lowest bias if<jats:italic>\u03a0<\/jats:italic><jats:sub>0<\/jats:sub>is 1 and if the number of GO terms is between 2 and 32. The bias of MLE is no worse than that of BBE for 100 GO terms even when the ideal number of components in its underlying mixture model is unknown, but has high bias when the number of GO terms is small compared to the number of estimated parameters. For unknown values of<jats:italic>\u03a0<\/jats:italic><jats:sub>0<\/jats:sub>, BBE has the lowest bias for a small number of GO terms (2-32 GO terms), and MLE has the lowest bias for a medium number of GO terms (100 GO terms).<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>For enrichment detection, we recommend estimating the LFDR by MLE given at least a medium number of GO terms, by BBE given a small number of GO terms, and by NMLE given either only 1 GO term or precise knowledge of<jats:italic>\u03a0<\/jats:italic><jats:sub>0<\/jats:sub>.<\/jats:p><\/jats:sec>","DOI":"10.1186\/1471-2105-14-87","type":"journal-article","created":{"date-parts":[[2013,3,6]],"date-time":"2013-03-06T15:45:42Z","timestamp":1362584742000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Empirical Bayes estimation of posterior probabilities of enrichment: A comparative study of five estimators of the local false discovery rate"],"prefix":"10.1186","volume":"14","author":[{"given":"Zhenyu","family":"Yang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zuojing","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David R","family":"Bickel","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2013,3,6]]},"reference":[{"key":"5877_CR1","doi-asserted-by":"publisher","first-page":"881","DOI":"10.1126\/science.1156409","volume":"322","author":"D Altshuler","year":"2008","unstructured":"Altshuler D, Daly MJ, Lander ES: Genetic mapping in human disease. Science 2008, 322: 881-888. 10.1126\/science.1156409","journal-title":"Science"},{"issue":"7","key":"5877_CR2","doi-asserted-by":"publisher","first-page":"509","DOI":"10.1038\/nrg2363","volume":"9","author":"SY Rhee","year":"2008","unstructured":"Rhee SY, Wood V, Dolinski K, Draghici S: Use and misuse of the gene ontology annotations. Nat Rev Genet 2008,9(7):509-515. 10.1038\/nrg2363","journal-title":"Nat Rev Genet"},{"key":"5877_CR3","doi-asserted-by":"publisher","first-page":"27","DOI":"10.1093\/nar\/28.1.27","volume":"28","author":"M Kanehisa","year":"2000","unstructured":"Kanehisa M, Goto S: KEGG: Kyoto Encyclopedia of Genes and Genome. Nucleic Acids Res 2000, 28: 27-30. 10.1093\/nar\/28.1.27","journal-title":"Nucleic Acids Res"},{"key":"5877_CR4","doi-asserted-by":"publisher","first-page":"P3","DOI":"10.1186\/gb-2003-4-5-p3","volume":"4","author":"G Dennis","year":"2003","unstructured":"Dennis G, Sherman BT, Hosack DA, Yang J, Gao W, Lane HC, Lempicki R: DAVID: Database for Annotation, Visualization, and Integrated Discovery. Genome Biol 2003, 4: P3. 10.1186\/gb-2003-4-5-p3","journal-title":"Genome Biol"},{"key":"5877_CR5","doi-asserted-by":"publisher","first-page":"R7","DOI":"10.1186\/gb-2003-4-1-r7","volume":"4","author":"SW Doniger","year":"2003","unstructured":"Doniger SW, Salomonis N, Dahlquist KD, Vranizan K, Lawlor SC, Conklin BR: MAPPFinder: using gene ontology and GenMAPP to create a global gene-expression profile from microarray data. Genome Biol 2003, 4: R7. 10.1186\/gb-2003-4-1-r7","journal-title":"Genome Biol"},{"key":"5877_CR6","doi-asserted-by":"publisher","first-page":"266","DOI":"10.1006\/geno.2002.6698","volume":"79","author":"P Khatri","year":"2002","unstructured":"Khatri P, Draghici S, Ostermeier G, Krawetz S: Profiling gene expression using onto-express. Genomics 2002, 79: 266-270. 10.1006\/geno.2002.6698","journal-title":"Genomics"},{"key":"5877_CR7","doi-asserted-by":"publisher","first-page":"R28","DOI":"10.1186\/gb-2003-4-4-r28","volume":"4","author":"BR Zeeberg","year":"2003","unstructured":"Zeeberg BR, Feng W, Wang G, Wang MD, Fojo AT, Sunshine M, Narasimhan S, Kane DW, Reinhold WC, Lababidi S, Bussey KJ, Riss J, Barrett JC, Weinstein JN: GoMiner: a resource for biological interpretation of genomic and proteomic data. Genome Biol 2003, 4: R28. 10.1186\/gb-2003-4-4-r28","journal-title":"Genome Biol"},{"key":"5877_CR8","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1093\/nar\/gkn923","volume":"37","author":"DW Huang","year":"2009","unstructured":"Huang DW, Sherman BT, Lempicki RA: Bioinformatics enrichment tools paths toward the comprehensive functional analysis of large gene lists. Nucleic Acids Res 2009, 37: 1-13. 10.1093\/nar\/gkn923","journal-title":"Nucleic Acids Res"},{"key":"5877_CR9","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","volume":"57","author":"Y Benjamini","year":"1995","unstructured":"Benjamini Y, Hochberg Y: Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B 1995, 57: 289-300.","journal-title":"J R Stat Soc B"},{"key":"5877_CR10","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1186\/1471-2164-11-96","volume":"11","author":"JL Min","year":"2010","unstructured":"Min JL, Barrett A, Watts T, Pettersson FH, Lockstone HE, Lindgren CM, Taylor JM, Allen M, Zondervan KT, McCarthy MI: Variability of gene expression profiles in human blood and lymphoblastoid cell lines. BMC Genomics 2010, 11: 96. 10.1186\/1471-2164-11-96","journal-title":"BMC Genomics"},{"key":"5877_CR11","doi-asserted-by":"publisher","first-page":"R93","DOI":"10.1186\/bcr2192","volume":"10","author":"F Reyal","year":"2008","unstructured":"Reyal F, van Vliet MH, Armstrong NJ, Horlings HM, de Visser KE, Kok M, Teschendorff AE, Mook S, van\u2019t Veer L, Caldas C, Salmon RJ, Vijver MJVD, Wessels LFA: A comprehensive analysis of prognostic signatures reveals the high predictive capacity of the proliferation, immune response and RNA splicing modules in breast cancer. Breast Cancer Res 2008, 10: R93. 10.1186\/bcr2192","journal-title":"Breast Cancer Res"},{"key":"5877_CR12","doi-asserted-by":"publisher","first-page":"1461","DOI":"10.1016\/j.ecoenv.2011.05.001","volume":"74","author":"R Wang","year":"2011","unstructured":"Wang R, Bencic D, Lazorchak J, Villeneuve D, Ankley GT: Transcriptional regulatory dynamics of the hypothalamic-pituitary-gonadal axis and its peripheral pathways as impacted by the 3-beta HSD inhibitor trilostane in zebrafish (Danio rerio). Ecotoxicol Environ Saf 2011, 74: 1461-1470. 10.1016\/j.ecoenv.2011.05.001","journal-title":"Ecotoxicol Environ Saf"},{"key":"5877_CR13","doi-asserted-by":"publisher","first-page":"2013","DOI":"10.1214\/aos\/1074290335","volume":"31","author":"JD Storey","year":"2003","unstructured":"Storey JD: The positive false discovery rate: a Bayesian interpretation and the q-value. Ann Stat 2003, 31: 2013-2035. 10.1214\/aos\/1074290335","journal-title":"Ann Stat"},{"key":"5877_CR14","doi-asserted-by":"publisher","first-page":"7483","DOI":"10.1093\/nar\/gkp813","volume":"37","author":"WJ Hong","year":"2009","unstructured":"Hong WJ, Tibshirani R, Chu G: Local false discovery rate facilitates comparison of different microarray experiments. Nucleic Acids Res 2009, 37: 7483-7497. 10.1093\/nar\/gkp813","journal-title":"Nucleic Acids Res"},{"key":"5877_CR15","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1198\/016214504000000089","volume":"99","author":"B Efron","year":"2004","unstructured":"Efron B: Large-scale simultaneous hypothesis testing: The choice of a null hypothesis. J Am Stat Assoc 2004, 99: 96-104. 10.1198\/016214504000000089","journal-title":"J Am Stat Assoc"},{"key":"5877_CR16","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511761362","volume-title":"Large-Scale Inference: Empirical Bayes Methods for Estimation,Testing, and Prediction Cambridge","author":"B Efron","year":"2010","unstructured":"Efron B: Large-Scale Inference: Empirical Bayes Methods for Estimation,Testing, and Prediction Cambridge. Cambridge University Press; 2010."},{"key":"5877_CR17","unstructured":"Bickel DR: Simple estimators of false discovery rates given as few as one or two p-values without strong parametric assumptions. Stat Appl Genet Mol Biol in press in press"},{"key":"5877_CR18","first-page":"arXiv:1104.0341","volume-title":"Small-scale inference: empirical Bayes and confidence methods for as few as a single comparison","author":"DR Bickel","year":"2011","unstructured":"Bickel DR: Small-scale inference: empirical Bayes and confidence methods for as few as a single comparison. Tech Rep, Ottawa Inst Syst Biol; 2011:arXiv:1104.0341-arXiv:1104.0341."},{"issue":"5","key":"5877_CR19","doi-asserted-by":"crossref","first-page":"art. 4","DOI":"10.1515\/1544-6115.1807","volume":"11","author":"M Padilla","year":"2012","unstructured":"Padilla M, Bickel DR: Empirical Bayes methods corrected for small numbers of tests. Stat Appl Genet Mol Biol 2012,11(5):art. 4.","journal-title":"Stat Appl Genet Mol Biol"},{"key":"5877_CR20","doi-asserted-by":"crossref","DOI":"10.1093\/oso\/9780198506508.001.0001","volume-title":"Likelihood Methods in Statistics Oxford","author":"T Severini","year":"2000","unstructured":"Severini T: Likelihood Methods in Statistics Oxford. Oxford University Press; 2000."},{"key":"5877_CR21","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4899-3210-5","volume-title":"Inference and Asymptotics","author":"OE Barndorff-Nielsen","year":"1994","unstructured":"Barndorff-Nielsen OE, Cox DR: Inference and Asymptotics. London: CRC Press; 1994."},{"key":"5877_CR22","doi-asserted-by":"publisher","first-page":"70","DOI":"10.1002\/gepi.1124","volume":"23","author":"B Efron","year":"2002","unstructured":"Efron B, Tibshirani R: Empirical Bayes methods and false discovery rates for microarrays. Genet Epidemiol 2002, 23: 70-86. 10.1002\/gepi.1124","journal-title":"Genet Epidemiol"},{"key":"5877_CR23","doi-asserted-by":"publisher","first-page":"229","DOI":"10.1186\/1471-2105-8-229","volume":"8","author":"C Dalmasso","year":"2007","unstructured":"Dalmasso C, Bar-Hen A, Bro\u00ebt P: A constrained polynomial regression procedure for estimating the local false discovery rate. BMC Bioinformatics 2007, 8: 229. 10.1186\/1471-2105-8-229","journal-title":"BMC Bioinformatics"},{"key":"5877_CR24","doi-asserted-by":"publisher","first-page":"364","DOI":"10.1093\/imamat\/2.4.364","volume":"2","author":"IJ Good","year":"1966","unstructured":"Good IJ: How to estimate probabilities. IMA J Appl Math 1966, 2: 364-383. 10.1093\/imamat\/2.4.364","journal-title":"IMA J Appl Math"},{"key":"5877_CR25","doi-asserted-by":"publisher","first-page":"3865","DOI":"10.1093\/bioinformatics\/bti626","volume":"21","author":"Y Pawitan","year":"2005","unstructured":"Pawitan Y, Murthy K, Michiels S, Ploner A: Bias in the estimation of false discovery rate in microarray studies. Bioinformatics 2005, 21: 3865-3872. 10.1093\/bioinformatics\/bti626","journal-title":"Bioinformatics"},{"key":"5877_CR26","doi-asserted-by":"publisher","first-page":"422","DOI":"10.1214\/09-AOAS276","volume":"4","author":"O Muralidharan","year":"2010","unstructured":"Muralidharan O: An empirical Bayes mixture method for effect size and false discovery rate estimation. Ann Appl Stat 2010, 4: 422-438.","journal-title":"Ann Appl Stat"},{"key":"5877_CR27","volume-title":"IEEE\/ACM Trans Comput Biol Bioinformatics","author":"Y Yang","year":"2012","unstructured":"Yang Y, Aghababazadeh FA, Bickel DR: Parametric estimation of the local false discovery rate for identifying genetic associations. IEEE\/ACM Trans Comput Biol Bioinformatics 2012. online ahead of print at http:\/\/dx.doi.org\/10.1109\/TCBB.2012.140 online ahead of print at"},{"key":"5877_CR28","doi-asserted-by":"crossref","DOI":"10.7551\/mitpress\/4643.001.0001","volume-title":"The Minimum Description Length Principle","author":"PD Gr\u00fcnwald","year":"2007","unstructured":"Gr\u00fcnwald PD: The Minimum Description Length Principle. London: MIT Press; 2007."},{"key":"5877_CR29","doi-asserted-by":"publisher","first-page":"610","DOI":"10.1002\/cjs.10109","volume":"39","author":"DR Bickel","year":"2011","unstructured":"Bickel DR: A predictive approach to measuring the strength of statistical evidence for single and multiple comparisons. Can J Stat 2011, 39: 610-631. 10.1002\/cjs.10109","journal-title":"Can J Stat"},{"key":"5877_CR30","volume-title":"Int Stat Rev","author":"DR Bickel","year":"2013","unstructured":"Bickel DR: Minimax-optimal strength of statistical evidence for a composite alternative hypothesis. Int Stat Rev 2013. in press. 2011 version available at arXiv:1101.0305 in press. 2011 version available at arXiv:1101.0305"},{"key":"5877_CR31","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1016\/j.jmva.2004.02.004","volume":"90","author":"D Scholtens","year":"2004","unstructured":"Scholtens D, Miron A, Merchant FM, Miller A, Miron PL, Iglehart JD, Gentleman R: Analyzing factorial designed microarray experiments. J Multivariate Anal 2004, 90: 19-43. 10.1016\/j.jmva.2004.02.004","journal-title":"J Multivariate Anal"},{"key":"5877_CR32","doi-asserted-by":"publisher","DOI":"10.1007\/0-387-29362-0","volume-title":"Bioinformatics and Computational Biology Solutions Using R and Bioconductor","author":"R Gentleman","year":"2005","unstructured":"Gentleman R, Carey V, Huber W, Irizarry R, Dudoit S (Eds): Bioinformatics and Computational Biology Solutions Using R and Bioconductor. New York: Springer; 2005."},{"issue":"3","key":"5877_CR33","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1093\/bioinformatics\/btg405","volume":"20","author":"L Gautier","year":"2004","unstructured":"Gautier L, Cope L, Bolstad BM, Irizarry RA: Affy\u2014analysis of Affymetrix Gene Chip data at the probe level. Bioinformatics 2004,20(3):307-315. 10.1093\/bioinformatics\/btg405","journal-title":"Bioinformatics"},{"key":"5877_CR34","doi-asserted-by":"publisher","first-page":"R80","DOI":"10.1186\/gb-2004-5-10-r80","volume":"5","author":"RC Gentleman","year":"2004","unstructured":"Gentleman RC, Carey VJ, Bates DM, Bolstad B, Dettling M, Dudoit S, Ellis B, Gautier L, Ge Y, Gentry J, Hornik K, Hothorn T, Huber W, Iacus S, Irizarry R, Leisch F, Li C, Maechler M, Rossini AJ, Sawitzki G, Smith C, Smyth G, Tierney L, Yang JYH, Zhang J: Bioconductor: open software development for computational biology and bioinformatics. Genome Biol 2004, 5: R80. 10.1186\/gb-2004-5-10-r80","journal-title":"Genome Biol"},{"key":"5877_CR35","volume-title":"Exploratory Data Analysis","author":"JW Tukey","year":"1977","unstructured":"Tukey JW: Exploratory Data Analysis. Reading: Addison-Wesley; 1977."},{"key":"5877_CR36","doi-asserted-by":"publisher","first-page":"363","DOI":"10.1111\/j.1541-0420.2010.01491.x","volume":"67","author":"DR Bickel","year":"2011","unstructured":"Bickel DR: Estimating the null distribution to adjust observed confidence levels for genome-scale screening. Biometrics 2011, 67: 363-370. 10.1111\/j.1541-0420.2010.01491.x","journal-title":"Biometrics"},{"key":"5877_CR37","volume-title":"Theory of Probability","author":"H Jeffreys","year":"1948","unstructured":"Jeffreys H: Theory of Probability. London: Oxford University Press; 1948."},{"key":"5877_CR38","first-page":"1147","volume":"22","author":"DR Bickel","year":"2012","unstructured":"Bickel DR: The strength of statistical evidence for composite hypotheses: inference to the best explanation. Statistica Sinica 2012, 22: 1147-1198.","journal-title":"Statistica Sinica"}],"container-title":["BMC Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/1471-2105-14-87.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,6]],"date-time":"2024-05-06T23:07:13Z","timestamp":1715036833000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcbioinformatics.biomedcentral.com\/articles\/10.1186\/1471-2105-14-87"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2013,3,6]]},"references-count":38,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2013,12]]}},"alternative-id":["5877"],"URL":"https:\/\/doi.org\/10.1186\/1471-2105-14-87","relation":{},"ISSN":["1471-2105"],"issn-type":[{"type":"electronic","value":"1471-2105"}],"subject":[],"published":{"date-parts":[[2013,3,6]]},"assertion":[{"value":"20 December 2011","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 February 2013","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"6 March 2013","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"87"}}