{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,19]],"date-time":"2025-02-19T05:22:39Z","timestamp":1739942559783,"version":"3.37.3"},"reference-count":38,"publisher":"Oxford University Press (OUP)","issue":"8","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2010,4,15]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Background: The statistical power or multiple Type II error rate in large-scale multiple testing problems as, for example, in gene expression microarray experiments, depends on typically unknown parameters and is therefore difficult to assess a priori. However, it has been suggested to estimate the multiple Type II error rate post hoc, based on the observed data.<\/jats:p><jats:p>Methods: We consider a class of post hoc estimators that are functions of the estimated proportion of true null hypotheses among all hypotheses. Numerous estimators for this proportion have been proposed and we investigate the statistical properties of the derived multiple Type II error rate estimators in an extensive simulation study.<\/jats:p><jats:p>Results: The performance of the estimators in terms of the mean squared error depends sensitively on the distributional scenario. Estimators based on empirical distributions of the null hypotheses are superior in the presence of strongly correlated test statistics.<\/jats:p><jats:p>Availability: R-code to compute all considered estimators based on P-values and supplementary material is available on the authors web page http:\/\/statistics.msi.meduniwien.ac.at\/index.php?page=pageszfnr<\/jats:p><jats:p>Contact: \u00a0martin.posch@meduniwien.ac.at<\/jats:p><jats:p>Supplementary information: \u00a0Supplementary data are available at Bioinformatics online.<\/jats:p>","DOI":"10.1093\/bioinformatics\/btq085","type":"journal-article","created":{"date-parts":[[2010,2,27]],"date-time":"2010-02-27T01:13:51Z","timestamp":1267233231000},"page":"1050-1056","source":"Crossref","is-referenced-by-count":7,"title":["<i>Post hoc<\/i>power estimation in large-scale multiple testing problems"],"prefix":"10.1093","volume":"26","author":[{"given":"Sonja","family":"Zehetmayer","sequence":"first","affiliation":[{"name":"Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Spitalgasse 23, A-1090 Vienna, Austria"}]},{"given":"Martin","family":"Posch","sequence":"additional","affiliation":[{"name":"Center for Medical Statistics, Informatics and Intelligent Systems, Medical University of Vienna, Spitalgasse 23, A-1090 Vienna, Austria"}]}],"member":"286","published-online":{"date-parts":[[2010,2,25]]},"reference":[{"key":"2023012508081870000_B1","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"J. R. Stat. Soc. B"},{"key":"2023012508081870000_B2","doi-asserted-by":"crossref","first-page":"60","DOI":"10.3102\/10769986025001060","article-title":"On the adaptive control of the false discovery fate in multiple testing with independent statistics","volume":"25","author":"Benjamini","year":"2000","journal-title":"J. Educ. Behav. Stat."},{"key":"2023012508081870000_B3","doi-asserted-by":"crossref","first-page":"1165","DOI":"10.1214\/aos\/1013699998","article-title":"The control of the false discovery rate in multiple testing under dependency","volume":"29","author":"Benjamini","year":"2001","journal-title":"Ann. Stat."},{"key":"2023012508081870000_B4","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1093\/biomet\/93.3.491","article-title":"Adaptive linear step-up procedures that control the false discovery rate","volume":"93","author":"Benjamini","year":"2006","journal-title":"Biometrika"},{"key":"2023012508081870000_B5","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1186\/1471-2105-6-199","article-title":"A comparative review of estimates of the proportion unchanged genes and the false discovery rate","volume":"6","author":"Broberg","year":"2005","journal-title":"BMC Bioinformatics"},{"key":"2023012508081870000_B6","first-page":"861","article-title":"Choosing the lesser evil: trade-off between false discovery rate and non-discovery rate","volume":"18","author":"Craiu","year":"2008","journal-title":"Stat. Sin."},{"key":"2023012508081870000_B7","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1093\/bioinformatics\/bti063","article-title":"A simple procedure for estimating the false discovery rate","volume":"21","author":"Dalmasso","year":"2005","journal-title":"Bioinformatics"},{"key":"2023012508081870000_B8","doi-asserted-by":"crossref","first-page":"774","DOI":"10.1111\/j.0006-341X.2004.00228.x","article-title":"Multiple-testing strategy for analyzing cDNA array data on gene expression","volume":"60","author":"Delongchamp","year":"2004","journal-title":"Biometrics"},{"key":"2023012508081870000_B9","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1198\/016214506000001211","article-title":"Correlation and large-scale simultaneous significance testing","volume":"102","author":"Efron","year":"2007","journal-title":"JASA"},{"key":"2023012508081870000_B10","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1214\/009053606000001460","article-title":"Size, power and false discovery rates","volume":"35","author":"Efron","year":"2007","journal-title":"Ann. Stat."},{"key":"2023012508081870000_B11","doi-asserted-by":"crossref","DOI":"10.1198\/jasa.2010.tm09129","article-title":"Correlated z-values and the accuracy of large-scale statistical estimates","author":"Efron","year":"2010","journal-title":"JASA"},{"key":"2023012508081870000_B12","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1111\/1467-9868.00347","article-title":"Operating characteristics and extensions of the false discovery rate procedure","volume":"64","author":"Genovese","year":"2002","journal-title":"J. R. Stat. Soc. B"},{"key":"2023012508081870000_B13","first-page":"5979","article-title":"Estrogen receptor status in breast cancer is associated with remarkably distinct gene expression patterns","volume":"61","author":"Gruvberger","year":"2001","journal-title":"Cancer Res."},{"key":"2023012508081870000_B14","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1198\/000313001300339897","article-title":"The abuse of power: the pervasive fallacy of power calculations for data analysis","volume":"55","author":"Hoenig","year":"2001","journal-title":"Am. Stat."},{"key":"2023012508081870000_B15","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1081\/BIP-120024202","article-title":"Comparison of methods for estimating the number of true hypotheses in multiplicity testing","volume":"13","author":"Hsueh","year":"2003","journal-title":"J. Biopharm. Stat."},{"key":"2023012508081870000_B16","doi-asserted-by":"crossref","first-page":"15044","DOI":"10.1073\/pnas.251547398","article-title":"Gene expression in papillary thyroid carcinoma reveals highly consistent profiles","volume":"98","author":"Huang","year":"2001","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012508081870000_B17","doi-asserted-by":"crossref","first-page":"495","DOI":"10.1198\/016214507000000167","article-title":"Estimating the null and the proportion of nonnull effects in large-scale multiple comparisons","volume":"102","author":"Jin","year":"2007","journal-title":"JASA"},{"key":"2023012508081870000_B18","doi-asserted-by":"crossref","first-page":"1594","DOI":"10.1214\/009053604000000030","article-title":"Needles and straw in haystacks: empirical Bayes estimates of possibly sparse sequences","volume":"32","author":"Johnstone","year":"2004","journal-title":"Ann. Stat."},{"key":"2023012508081870000_B19","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1111\/j.1467-9868.2005.00515.x","article-title":"Estimating the proportion of true null hypotheses, with application to dna microarray data","volume":"67","author":"Langaas","year":"2005","journal-title":"J. R. Stat. Soc. B"},{"key":"2023012508081870000_B20","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1214\/009053605000000741","article-title":"Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses","volume":"34","author":"Meinshausen","year":"2006","journal-title":"Ann. Stat."},{"key":"2023012508081870000_B21","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1073\/pnas.0510115103","article-title":"Analysis of gene expression in pathophysiological states: balancing false discovery and false negative rates","volume":"103","author":"Norris","year":"2006","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012508081870000_B22","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1111\/j.1467-9868.2005.00509.x","article-title":"Variance of the number of false discoveries","volume":"67","author":"Owen","year":"2005","journal-title":"J. R. Stat. Soc. B"},{"key":"2023012508081870000_B23","doi-asserted-by":"crossref","first-page":"1620","DOI":"10.1093\/bioinformatics\/btg227","article-title":"The effect of replication on gene expression microarray experiments","volume":"19","author":"Pavlidis","year":"2003","journal-title":"Bioinformatics"},{"key":"2023012508081870000_B24","doi-asserted-by":"crossref","first-page":"3017","DOI":"10.1093\/bioinformatics\/bti448","article-title":"False discovery rate, sensitivity and sample size for microarray studies","volume":"21","author":"Pawitan","year":"2005","journal-title":"Bioinformatics"},{"key":"2023012508081870000_B25","first-page":"836","article-title":"Hunting for significance with the false discovery rate","volume":"104","author":"Posch","year":"2009","journal-title":"JASA"},{"key":"2023012508081870000_B26","doi-asserted-by":"crossref","first-page":"1737","DOI":"10.1093\/bioinformatics\/bth160","article-title":"Improving false discovery rate estimation","volume":"20","author":"Pounds","year":"2004","journal-title":"Bioinformatics"},{"key":"2023012508081870000_B27","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1093\/bioinformatics\/btg148","article-title":"Estimating the occurrence of false positives and false negatives in microarray studies by approximating and partitioning the empirical distribution of p-values","volume":"19","author":"Pounds","year":"2003","journal-title":"Bioinformatics"},{"volume-title":"R: A language and environment for statistical computing.","year":"2009","author":"R Development Core Team","key":"2023012508081870000_B28"},{"key":"2023012508081870000_B29","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1016\/j.jspi.2003.06.019","article-title":"FDR-controlling stepwise procedures and their false negatives rates","volume":"125","author":"Sarkar","year":"2004","journal-title":"J. Stat. Plan. Infer."},{"key":"2023012508081870000_B30","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1093\/biomet\/69.3.493","article-title":"Plots of p-values to evaluate many tests simultaneously","volume":"69","author":"Schweder","year":"1982","journal-title":"Biometrika"},{"key":"2023012508081870000_B31","doi-asserted-by":"crossref","first-page":"161","DOI":"10.1002\/pst.301","article-title":"Power and sample size when multiple endpoints are considered","volume":"6","author":"Senn","year":"2007","journal-title":"Pharm. Stat."},{"key":"2023012508081870000_B32","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1111\/j.1467-9868.2004.00439.x","article-title":"Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach","volume":"66","author":"Storey","year":"2004","journal-title":"J. R. Stat. Soc. B"},{"key":"2023012508081870000_B33","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1111\/1467-9868.00346","article-title":"A direct approach to false discovery rates","volume":"64","author":"Storey","year":"2002","journal-title":"J. R. Stat. Soc. B"},{"key":"2023012508081870000_B34","doi-asserted-by":"crossref","first-page":"9440","DOI":"10.1073\/pnas.1530509100","article-title":"Statistical significance for genomewide studies","volume":"100","author":"Storey","year":"2003","journal-title":"Proc. Natl Acad. Sci. USA"},{"key":"2023012508081870000_B35","doi-asserted-by":"crossref","first-page":"303","DOI":"10.1186\/1471-2105-9-303","article-title":"A unified approach to false discovery rate estimation","volume":"9","author":"Strimmer","year":"2008","journal-title":"BMC Bioinformatics"},{"key":"2023012508081870000_B36","doi-asserted-by":"crossref","first-page":"94","DOI":"10.1002\/bimj.200510311","article-title":"Combining adaptive designs with control of the false discovery rate - a generalized definition for a global p-value","volume":"49","author":"Victor","year":"2007","journal-title":"Biometrical J."},{"key":"2023012508081870000_B37","doi-asserted-by":"crossref","first-page":"3771","DOI":"10.1093\/bioinformatics\/bti604","article-title":"Two-stage designs for experiments with a large number of hypotheses","volume":"21","author":"Zehetmayer","year":"2005","journal-title":"Bioinformatics"},{"key":"2023012508081870000_B38","doi-asserted-by":"crossref","first-page":"4145","DOI":"10.1002\/sim.3300","article-title":"Optimized multi-stage designs controlling the false discovery or the family wise error rate","volume":"27","author":"Zehetmayer","year":"2008","journal-title":"Stat. Med."}],"container-title":["Bioinformatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/26\/8\/1050\/48857602\/bioinformatics_26_8_1050.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article-pdf\/26\/8\/1050\/48857602\/bioinformatics_26_8_1050.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,2,18]],"date-time":"2025-02-18T22:14:20Z","timestamp":1739916860000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/bioinformatics\/article\/26\/8\/1050\/206827"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2010,2,25]]},"references-count":38,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2010,4,15]]}},"URL":"https:\/\/doi.org\/10.1093\/bioinformatics\/btq085","relation":{},"ISSN":["1367-4811","1367-4803"],"issn-type":[{"type":"electronic","value":"1367-4811"},{"type":"print","value":"1367-4803"}],"subject":[],"published-other":{"date-parts":[[2010,4,15]]},"published":{"date-parts":[[2010,2,25]]}}}