{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T21:50:12Z","timestamp":1790891412596,"version":"4.1.0"},"reference-count":42,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T00:00:00Z","timestamp":1677196800000},"content-version":"vor","delay-in-days":409,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["1945266"],"award-info":[{"award-number":["1945266"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"NSERC","doi-asserted-by":"publisher","award":["RGPIN-2018-03823"],"award-info":[{"award-number":["RGPIN-2018-03823"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000038","name":"NSERC","doi-asserted-by":"publisher","award":["RGPAS-2018-522590"],"award-info":[{"award-number":["RGPAS-2018-522590"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,7,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>E-values have gained attention as potential alternatives to p-values as measures of uncertainty, significance and evidence. In brief, e-values are realized by random variables with expectation at most one under the null; examples include betting scores, (point null) Bayes factors, likelihood ratios and stopped supermartingales. We design a natural analogue of the Benjamini-Hochberg (BH) procedure for false discovery rate (FDR) control that utilizes e-values, called the e-BH procedure, and compare it with the standard procedure for p-values. One of our central results is that, unlike the usual BH procedure, the e-BH procedure controls the FDR at the desired level\u2014with no correction\u2014for any dependence structure between the e-values. We illustrate that the new procedure is convenient in various settings of complicated dependence, structured and post-selection hypotheses, and multi-armed bandit problems. Moreover, the BH procedure is a special case of the e-BH procedure through calibration between p-values and e-values. Overall, the e-BH procedure is a novel, powerful and general tool for multiple testing under dependence, that is complementary to the BH procedure, each being an appropriate choice in different applications.<\/jats:p>","DOI":"10.1111\/rssb.12489","type":"journal-article","created":{"date-parts":[[2022,1,11]],"date-time":"2022-01-11T12:32:15Z","timestamp":1641904335000},"page":"822-852","source":"Crossref","is-referenced-by-count":94,"title":["False Discovery Rate Control with E-values"],"prefix":"10.1093","volume":"84","author":[{"given":"Ruodu","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Statistics and Actuarial Science, University of Waterloo , Waterloo , Ontario , Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aaditya","family":"Ramdas","sequence":"additional","affiliation":[{"name":"Departments of Statistics and Machine Learning, Carnegie Mellon University , Pittsburgh , Pennsylvania , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2022,1,11]]},"reference":[{"issue":"5","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"2055","DOI":"10.1214\/15-AOS1337","article-title":"Controlling the false discovery rate via knockoffs","volume":"43","author":"Barber","year":"2015","journal-title":"The Annals of Statistics"},{"issue":"4","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"1247","DOI":"10.1111\/rssb.12218","article-title":"The p-filter: multilayer false discovery rate control for grouped hypotheses","volume":"79","author":"Barber","year":"2017","journal-title":"Journal of the Royal Statistical Society Series B"},{"issue":"1","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1111\/j.1540-6261.2009.01527.x","article-title":"False discoveries in mutual fund performance: measuring luck in estimated alphas","volume":"65","author":"Barras","year":"2010","journal-title":"Journal of Finance"},{"key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"Journal of the Royal Statistical Society B"},{"issue":"3","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1111\/1467-9469.00072","article-title":"Multiple hypotheses testing with weights","volume":"24","author":"Benjamini","year":"1997","journal-title":"Scandinavian Journal of Statistics"},{"key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"1165","DOI":"10.1214\/aos\/1013699998","article-title":"The control of the false discovery rate in multiple testing under dependency","volume":"29","author":"Benjamini","year":"2001","journal-title":"The Annals of Statistics"},{"key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"963","DOI":"10.1214\/08-EJS180","article-title":"Two simple sufficient conditions for FDR control","volume":"2","author":"Blanchard","year":"2008","journal-title":"Electronic Journal of Statistics"},{"issue":"3","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"1141","DOI":"10.1214\/14-AOS1302","article-title":"New procedures controlling the false discovery proportion via Romano\u2013Wolf's heuristic","volume":"43","author":"Delattre","year":"2015","journal-title":"The Annals of Statistics"},{"issue":"2","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"979","DOI":"10.1214\/07-AOS571","article-title":"Likelihood ratio tests and singularities","volume":"37","author":"Drton","year":"2009","journal-title":"The Annals of Statistics"},{"issue":"328","key":"2023022501101603300_","first-page":"1278","article-title":"Student's t-test under symmetry conditions","volume":"64","author":"Efron","year":"1969","journal-title":"Journal of the American Statistical Association"},{"issue":"4","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"763","DOI":"10.1007\/s00780-015-0273-z","article-title":"Aggregation-robustness and model uncertainty of regulatory risk measures","volume":"19","author":"Embrechts","year":"2015","journal-title":"Finance and Stochastics"},{"issue":"2","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1214\/07-AOS569","article-title":"On the false discovery rate and an asymptotically optimal rejection curve","volume":"37","author":"Finner","year":"2009","journal-title":"The Annals of Statistics"},{"key":"2023022501101603300_","volume-title":"Conditional calibration for false discovery rate control under dependence","author":"Fithian","year":"2020"},{"key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"1035","DOI":"10.1214\/009053604000000283","article-title":"A stochastic process approach to false discovery control","volume":"32","author":"Genovese","year":"2004","journal-title":"The Annals of Statistics"},{"key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"1408","DOI":"10.1198\/016214506000000339","article-title":"Exceedance control of the false discovery proportion","volume":"101","author":"Genovese","year":"2006","journal-title":"Journal of the American Statistical Association"},{"key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"584","DOI":"10.1214\/11-STS356","article-title":"Multiple testing for exploratory research","volume":"26","author":"Goeman","year":"2011","journal-title":"Statistical Science"},{"key":"2023022501101603300_","doi-asserted-by":"crossref","DOI":"10.1109\/ITA50056.2020.9244948","volume-title":"Safe testing","author":"Gr\u00fcnwald","year":"2020"},{"issue":"3","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"1070","DOI":"10.1214\/14-AOS1214","article-title":"Further results on controlling the false discovery proportion","volume":"42","author":"Guo","year":"2014","journal-title":"The Annals of Statistics"},{"key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1214\/18-PS321","article-title":"Time-uniform Chernoff bounds via nonnegative supermartingales","volume":"17","author":"Howard","year":"2020","journal-title":"Probability Surveys"},{"issue":"2","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"1055","DOI":"10.1214\/20-AOS1991","article-title":"Time-uniform, nonparametric, nonasymptotic confidence sequences","volume":"49","author":"Howard","year":"2021","journal-title":"The Annals of Statistics"},{"key":"2023022501101603300_","first-page":"3660","volume-title":"A bandit approach to sequential experimental design with false discovery control","author":"Jamieson","year":"2018"},{"issue":"4","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"2029","DOI":"10.1214\/13-AOS1134","article-title":"Central limit theorems for classical likelihood ratio tests for high-dimensional normal distributions","volume":"41","author":"Jiang","year":"2013","journal-title":"The Annals of Statistics"},{"issue":"430","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1080\/01621459.1995.10476572","article-title":"Bayes factors","volume":"90","author":"Kass","year":"1995","journal-title":"Journal of the American Statistical Association"},{"key":"2023022501101603300_","article-title":"Filtering the rejection set while preserving false discovery rate control","author":"Katsevich","year":"2021","journal-title":"Journal of the American Statistical Association"},{"key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"2790","DOI":"10.1214\/18-AOS1765","article-title":"A unified treatment of multiple testing with prior knowledge using the p-filter","volume":"47","author":"Ramdas","year":"2019","journal-title":"The Annals of Statistics"},{"issue":"1","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1093\/biomet\/asy066","article-title":"A sequential algorithm for false discovery rate control on directed acyclic graphs","volume":"106","author":"Ramdas","year":"2019","journal-title":"Biometrika"},{"key":"2023022501101603300_","volume-title":"Admissible anytime-valid sequential inference must rely on nonnegative martingales","author":"Ramdas","year":"2020"},{"key":"2023022501101603300_","article-title":"Testing exchangeability: fork-convex hulls, supermartingales and e-processes","author":"Ramdas","year":"2021","journal-title":"International Journal of Approximate Reasoning"},{"issue":"4","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"1072","DOI":"10.1016\/j.jspi.2007.03.058","article-title":"Two-stage stepup procedures controlling FDR","volume":"138","author":"Sarkar","year":"2008","journal-title":"Journal of Statistical Planning and Inference"},{"issue":"2","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1111\/rssa.12647","article-title":"Testing by betting: a strategy for statistical and scientific communication","volume":"184","author":"Shafer","year":"2021","journal-title":"Journal of the Royal Statistical Society: Series A (Statistics in Society)"},{"issue":"1","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1214\/10-STS347","article-title":"Test martingales, Bayes factors and p-values","volume":"26","author":"Shafer","year":"2011","journal-title":"Statistical Science"},{"issue":"3","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1111\/1467-9868.00346","article-title":"A direct approach to false discovery rates","volume":"64","author":"Storey","year":"2002","journal-title":"Journal of the Royal Statistical Society: Series B (Statistical Methodology)"},{"key":"2023022501101603300_","volume-title":"The FDR-linking theorem","author":"Su","year":"2018"},{"key":"2023022501101603300_","volume-title":"Confidence and discoveries with e-values","author":"Vovk","year":"2019"},{"issue":"4","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1093\/biomet\/asaa027","article-title":"Combining p-values via averaging","volume":"107","author":"Vovk","year":"2020","journal-title":"Biometrika"},{"issue":"3","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"1736","DOI":"10.1214\/20-AOS2020","article-title":"E-values: calibration, combination and applications","volume":"49","author":"Vovk","year":"2021","journal-title":"The Annals of Statistics"},{"key":"2023022501101603300_","volume-title":"Admissible ways of merging p-values under arbitrary dependence","author":"Vovk","year":"2020"},{"issue":"29","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"16880","DOI":"10.1073\/pnas.1922664117","article-title":"Universal inference","volume":"117","author":"Wasserman","year":"2020","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"2023022501101603300_","first-page":"20204","article-title":"Confidence sequences for sampling without replacement","author":"Waudby-Smith","year":"2020","journal-title":"Neural Information Processing Systems"},{"key":"2023022501101603300_","volume-title":"Estimating means of bounded random variables by betting","author":"Waudby-Smith","year":"2020"},{"issue":"1","key":"2023022501101603300_","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1214\/aoms\/1177732360","article-title":"The large-sample distribution of the likelihood ratio for testing composite hypotheses","volume":"9","author":"Wilks","year":"1938","journal-title":"The Annals of Mathematical Statistics"},{"key":"2023022501101603300_","article-title":"A unified framework for bandit multiple testing","author":"Xu","year":"2021","journal-title":"Neural Information Processing Systems"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/rssb.12489","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/rssb.12489","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/84\/3\/822\/49322390\/jrsssb_84_3_822.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/84\/3\/822\/49322390\/jrsssb_84_3_822.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,16]],"date-time":"2024-09-16T01:53:42Z","timestamp":1726451622000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/84\/3\/822\/7056146"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,11]]},"references-count":42,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,1,11]]},"published-print":{"date-parts":[[2022,7,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12489","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,7]]},"published":{"date-parts":[[2022,1,11]]}}}