{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T00:33:20Z","timestamp":1790037200459,"version":"4.0.1"},"reference-count":56,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2024,3,23]],"date-time":"2024-03-23T00:00:00Z","timestamp":1711152000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,12,1]]},"abstract":"<jats:title>Summary<\/jats:title>\n               <jats:p>We introduce a multiple testing procedure that controls the median of the proportion of false discoveries in a flexible way. The procedure requires only a vector of p-values as input and is comparable to the Benjamini\u2013Hochberg method, which controls the mean of the proportion of false discoveries. Our method allows free choice of one or several values of $ \\alpha $ after seeing the data, unlike the Benjamini\u2013Hochberg procedure, which can be very anti-conservative when $ \\alpha $ is chosen post hoc. We prove these claims and illustrate them with simulations. The proposed procedure is inspired by a popular estimator of the total number of true hypotheses. We adapt this estimator to provide simultaneously median unbiased estimators of the proportion of false discoveries, valid for finite samples. This simultaneity allows for the claimed flexibility. Our approach does not assume independence. The time complexity of our method is linear in the number of hypotheses, after sorting the p-values.<\/jats:p>","DOI":"10.1093\/biomet\/asae018","type":"journal-article","created":{"date-parts":[[2024,3,21]],"date-time":"2024-03-21T16:24:00Z","timestamp":1711038240000},"page":"1129-1150","source":"Crossref","is-referenced-by-count":2,"title":["Flexible control of the median of the false discovery proportion"],"prefix":"10.1093","volume":"111","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-9811-1336","authenticated-orcid":false,"given":"Jesse","family":"Hemerik","sequence":"first","affiliation":[{"name":"Econometric Institute, Erasmus University , Burg. Oudlaan 50 , 3062 PA Rotterdam,","place":["The Netherlands"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aldo","family":"Solari","sequence":"additional","affiliation":[{"name":"Department of Economics, Ca\u2019 Foscari University , Cannaregio 873 , 30121 Venice,","place":["Italy"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jelle J","family":"Goeman","sequence":"additional","affiliation":[{"name":"Department of Biomedical Data Sciences, Leiden University Medical Center , Einthovenweg 20 , 2333 ZC Leiden,","place":["The Netherlands"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2024,3,23]]},"reference":[{"key":"2024112207521120800_asae018-B1","doi-asserted-by":"crossref","first-page":"2311","DOI":"10.1002\/sim.9725","article-title":"Permutation-based true discovery proportions for fMRI cluster analysis","volume":"42","author":"Andreella","year":"2023","journal-title":"Statist. Med"},{"key":"2024112207521120800_asae018-B2","doi-asserted-by":"crossref","first-page":"2055","DOI":"10.1214\/15-AOS1337","article-title":"Controlling the false discovery rate via knockoffs","volume":"43","author":"Barber","year":"2015","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B3","author":"Basu","year":"2023"},{"key":"2024112207521120800_asae018-B4","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: A practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"J. R. Statist. Soc. B"},{"key":"2024112207521120800_asae018-B5","doi-asserted-by":"crossref","first-page":"1165","DOI":"10.1214\/aos\/1013699998","article-title":"The control of the false discovery rate in multiple testing under dependency","volume":"29","author":"Benjamini","year":"2001","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/BF02778266","article-title":"Global and multiple test procedures using ordered p-values\u2014a review","volume":"45","author":"Bernhard","year":"2004","journal-title":"Statist. Papers"},{"key":"2024112207521120800_asae018-B7","doi-asserted-by":"crossref","first-page":"119492","DOI":"10.1016\/j.neuroimage.2022.119492","article-title":"Notip: Non-parametric true discovery proportion control for brain imaging","volume":"260","author":"Blain","year":"2022","journal-title":"NeuroImage"},{"key":"2024112207521120800_asae018-B8","doi-asserted-by":"crossref","first-page":"1281","DOI":"10.1214\/19-AOS1847","article-title":"Post hoc confidence bounds on false positives using reference families","volume":"48","author":"Blanchard","year":"2020","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B9","doi-asserted-by":"crossref","first-page":"1141","DOI":"10.1214\/14-AOS1302","article-title":"New procedures controlling the false discovery proportion via Romano\u2013Wolf\u2019s heuristic","volume":"43","author":"Delattre","year":"2015","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B10","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-642-45182-9","volume-title":"Simultaneous Statistical Inference: With Applications in the Life Sciences","author":"Dickhaus","year":"2014"},{"key":"2024112207521120800_asae018-B11","doi-asserted-by":"crossref","first-page":"882","DOI":"10.1214\/19-EJS1544","article-title":"Variability and stability of the false discovery proportion","volume":"13","author":"Ditzhaus","year":"2019","journal-title":"Electron. J. Statist."},{"key":"2024112207521120800_asae018-B12","doi-asserted-by":"crossref","first-page":"4244","DOI":"10.1214\/20-EJS1771","article-title":"Controlling the false discovery exceedance for heterogeneous tests","volume":"14","author":"D\u00f6hler","year":"2020","journal-title":"Electron. J. Statist."},{"key":"2024112207521120800_asae018-B13","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1198\/016214506000001211","article-title":"Correlation and large-scale simultaneous significance testing","volume":"102","author":"Efron","year":"2007","journal-title":"J. Am. Statist. Assoc"},{"key":"2024112207521120800_asae018-B14","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1177\/0962280206079046","article-title":"A review of modern multiple hypothesis testing, with particular attention to the false discovery proportion","volume":"17","author":"Farcomeni","year":"2008","journal-title":"Statist. Meth. Med. Res"},{"key":"2024112207521120800_asae018-B15","doi-asserted-by":"crossref","first-page":"1035","DOI":"10.1214\/009053604000000283","article-title":"A stochastic process approach to false discovery control","volume":"32","author":"Genovese","year":"2004","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B16","doi-asserted-by":"crossref","first-page":"1408","DOI":"10.1198\/016214506000000339","article-title":"Exceedance control of the false discovery proportion","volume":"101","author":"Genovese","year":"2006","journal-title":"J. Am. Statist. Assoc"},{"key":"2024112207521120800_asae018-B17","doi-asserted-by":"crossref","first-page":"1218","DOI":"10.1214\/20-AOS1999","article-title":"Only closed testing procedures are admissible for controlling false discovery proportions","volume":"49","author":"Goeman","year":"2021","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B18","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1093\/biomet\/asz041","article-title":"Simultaneous control of all false discovery proportions in large-scale multiple hypothesis testing","volume":"106","author":"Goeman","year":"2019","journal-title":"Biometrika"},{"key":"2024112207521120800_asae018-B19","doi-asserted-by":"crossref","first-page":"584","DOI":"10.1214\/11-STS356","article-title":"Multiple testing for exploratory research","volume":"26","author":"Goeman","year":"2011","journal-title":"Statist. Sci."},{"key":"2024112207521120800_asae018-B20","doi-asserted-by":"crossref","first-page":"1946","DOI":"10.1002\/sim.6082","article-title":"Multiple hypothesis testing in genomics","volume":"33","author":"Goeman","year":"2014","journal-title":"Statist. Med."},{"key":"2024112207521120800_asae018-B21","author":"Gr\u00fcnwald","year":"2023"},{"key":"2024112207521120800_asae018-B22","doi-asserted-by":"crossref","first-page":"1070","DOI":"10.1214\/14-AOS1214","article-title":"Further results on controlling the false discovery proportion","volume":"42","author":"Guo","year":"2014","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2202\/1544-6115.1247","article-title":"A generalized Sidak-Holm procedure and control of generalized error rates under independence","volume":"6","author":"Guo","year":"2007","journal-title":"Statist. Appl. Genet. Molec. Biol"},{"key":"2024112207521120800_asae018-B24","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1093\/rapstu\/raaa003","article-title":"An evaluation of alternative multiple testing methods for finance applications","volume":"10","author":"Harvey","year":"2020","journal-title":"Rev. Asset Pricing Stud"},{"key":"2024112207521120800_asae018-B25","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1111\/rssb.12238","article-title":"False discovery proportion estimation by permutations: Confidence for significance analysis of microarrays","volume":"80","author":"Hemerik","year":"2018","journal-title":"J. R. Statist. Soc."},{"key":"2024112207521120800_asae018-B26","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1093\/biomet\/asz021","article-title":"Permutation-based simultaneous confidence bounds for the false discovery proportion","volume":"106","author":"Hemerik","year":"2019","journal-title":"Biometrika"},{"key":"2024112207521120800_asae018-B27","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1007\/s10463-021-00797-0","article-title":"On the usage of randomized p-values in the Schweder\u2013Spj\u00f8tvoll estimator","volume":"74","author":"Hoang","year":"2022","journal-title":"Ann. Inst. Statist. Math"},{"key":"2024112207521120800_asae018-B28","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1002\/sim.4780090710","article-title":"More powerful procedures for multiple significance testing","volume":"9","author":"Hochberg","year":"1990","journal-title":"Statist. Med."},{"key":"2024112207521120800_asae018-B29","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1177\/0959354304043638","article-title":"Alphabet soup: Blurring the distinctions between p\u2019s and $ \\alpha $\u2019s in psychological research","volume":"14","author":"Hubbard","year":"2004","journal-title":"Theory Psychol"},{"key":"2024112207521120800_asae018-B30","author":"Katsevich","year":"2018"},{"key":"2024112207521120800_asae018-B31","doi-asserted-by":"crossref","first-page":"3465","DOI":"10.1214\/19-AOS1938","article-title":"Simultaneous high-probability bounds on the false discovery proportion in structured, regression and online settings","volume":"48","author":"Katsevich","year":"2020","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B32","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1111\/j.1467-9868.2005.00515.x","article-title":"Estimating the proportion of true null hypotheses, with application to DNA microarray data","volume":"67","author":"Langaas","year":"2005","journal-title":"J. R. Statist. Soc. B"},{"key":"2024112207521120800_asae018-B33","doi-asserted-by":"crossref","first-page":"1138","DOI":"10.1214\/009053605000000084","article-title":"Generalizations of the familywise error rate","volume":"33","author":"Lehmann","year":"2005","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B34","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1111\/rssb.12274","volume":"80","author":"Lei","year":"2018","journal-title":"AdaPT. J. R. Statist. Soc. B"},{"key":"2024112207521120800_asae018-B35","doi-asserted-by":"crossref","first-page":"253","DOI":"10.1093\/biomet\/asaa064","article-title":"A general interactive framework for false discovery rate control under structural constraints","volume":"108","author":"Lei","year":"2021","journal-title":"Biometrika"},{"key":"2024112207521120800_asae018-B36","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1080\/01621459.2016.1180989","article-title":"Accumulation tests for FDR control in ordered hypothesis testing","volume":"112","author":"Li","year":"2017","journal-title":"J. Am. Statist. Assoc"},{"key":"2024112207521120800_asae018-B37","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1111\/j.1467-9868.2011.01001.x","article-title":"Adaptive and dynamic adaptive procedures for false discovery rate control and estimation","volume":"74","author":"Liang","year":"2012","journal-title":"J. R. Statist. Soc. B"},{"key":"2024112207521120800_asae018-B38","author":"Luo","year":"2022"},{"key":"2024112207521120800_asae018-B39","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1111\/j.1467-9469.2005.00488.x","article-title":"False discovery control for multiple tests of association under general dependence","volume":"33","author":"Meinshausen","year":"2006","journal-title":"Scand. J. Statist"},{"key":"2024112207521120800_asae018-B40","doi-asserted-by":"crossref","first-page":"373","DOI":"10.1214\/009053605000000741","article-title":"Estimating the proportion of false null hypotheses among a large number of independently tested hypotheses","volume":"34","author":"Meinshausen","year":"2006","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B41","doi-asserted-by":"crossref","first-page":"107758","DOI":"10.1016\/j.csda.2023.107758","article-title":"Exceedance control of the false discovery proportion via high precision inversion method of Berk-Jones statistics","volume":"185","author":"Miecznikowski","year":"2023","journal-title":"Comp. Statist. Data Anal"},{"key":"2024112207521120800_asae018-B42","doi-asserted-by":"crossref","first-page":"1065","DOI":"10.1214\/08-EJS207","article-title":"Asymptotic properties of false discovery rate controlling procedures under independence","volume":"2","author":"Neuvial","year":"2008","journal-title":"Electron. J. Statist."},{"key":"2024112207521120800_asae018-B43","author":"R Development Core Team","year":"2024"},{"key":"2024112207521120800_asae018-B44","author":"Rajchert","year":"2022"},{"key":"2024112207521120800_asae018-B45","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1093\/oxfordjournals.aje.a112510","article-title":"Estimating prevalence from the results of a screening test","volume":"107","author":"Rogan","year":"1978","journal-title":"Am. J. Epidemiol"},{"key":"2024112207521120800_asae018-B46","doi-asserted-by":"crossref","first-page":"1850","DOI":"10.1214\/009053606000000461","article-title":"Stepup procedures for control of generalizations of the familywise error rate","volume":"34","author":"Romano","year":"2006","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B47","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1017\/S0266466608080171","article-title":"Formalized data snooping based on generalized error rates","volume":"24","author":"Romano","year":"2008","journal-title":"Economet. Theory"},{"key":"2024112207521120800_asae018-B48","doi-asserted-by":"crossref","first-page":"1378","DOI":"10.1214\/009053606000001622","article-title":"Control of generalized error rates in multiple testing","volume":"35","author":"Romano","year":"2007","journal-title":"Ann. Statist."},{"key":"2024112207521120800_asae018-B49","author":"Roquain","year":"2011"},{"key":"2024112207521120800_asae018-B50","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1201\/9780429030888-8","volume-title":"Prevalence estimation. In Handbook of Multiple Comparisons","author":"Rosenblatt","year":"2021"},{"key":"2024112207521120800_asae018-B51","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1093\/biomet\/asq075","article-title":"The effect of correlation in false discovery rate estimation","volume":"98","author":"Schwartzman","year":"2011","journal-title":"Biometrika"},{"key":"2024112207521120800_asae018-B52","doi-asserted-by":"crossref","first-page":"493","DOI":"10.1093\/biomet\/69.3.493","article-title":"Plots of p-values to evaluate many tests simultaneously","volume":"69","author":"Schweder","year":"1982","journal-title":"Biometrika"},{"key":"2024112207521120800_asae018-B53","doi-asserted-by":"crossref","first-page":"776","DOI":"10.1002\/bimj.201500253","article-title":"Minimally adaptive BH: A tiny but uniform improvement of the procedure of Benjamini and Hochberg","volume":"59","author":"Solari","year":"2017","journal-title":"Biomet. J"},{"key":"2024112207521120800_asae018-B54","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1111\/1467-9868.00346","article-title":"A direct approach to false discovery rates","volume":"64","author":"Storey","year":"2002","journal-title":"J. R. Statist. Soc. B"},{"key":"2024112207521120800_asae018-B55","doi-asserted-by":"crossref","first-page":"15","DOI":"10.2202\/1544-6115.1042","article-title":"Augmentation procedures for control of the generalized family-wise error rate and tail probabilities for the proportion of false positives","volume":"3","author":"van der Laan","year":"2004","journal-title":"Statist. Appl. Genet. Molec. Biol"},{"key":"2024112207521120800_asae018-B56","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1093\/jrsssb\/qkad019","article-title":"Permutation-based true discovery guarantee by sum tests","volume":"64","author":"Vesely","year":"2023","journal-title":"J. R. Statist. Soc. B"}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/biomet\/advance-article-pdf\/doi\/10.1093\/biomet\/asae018\/57944680\/asae018.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/111\/4\/1129\/60783801\/asae018.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/111\/4\/1129\/60783801\/asae018.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,22]],"date-time":"2024-11-22T07:55:33Z","timestamp":1732262133000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/111\/4\/1129\/7634288"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3,23]]},"references-count":56,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2024,3,23]]},"published-print":{"date-parts":[[2024,12,1]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asae018","relation":{},"ISSN":["0006-3444","1464-3510"],"issn-type":[{"value":"0006-3444","type":"print"},{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,12]]},"published":{"date-parts":[[2024,3,23]]}}}