{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T06:04:43Z","timestamp":1790661883365,"version":"4.1.0"},"reference-count":43,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2022,4,12]],"date-time":"2022-04-12T00:00:00Z","timestamp":1649721600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,9,1]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Multiple testing problems (MTPs) are a staple of modern statistical analysis. The fundamental objective of MTPs is to reject as many false null hypotheses as possible (that is, maximize some notion of power), subject to controlling an overall measure of false discovery, like family-wise error rate (FWER) or false discovery rate (FDR). In this paper we provide generalizations to MTPs of the optimal Neyman-Pearson test for a single hypothesis. We show that for simple hypotheses, for both FWER and FDR and relevant notions of power, finding the optimal multiple testing procedure can be formulated as infinite dimensional binary programs and can in principle be solved for any number of hypotheses. We also characterize maximin rules for complex alternatives, and demonstrate that such rules can be found in practice, leading to improved practical procedures compared to existing alternatives that guarantee strong error control on the entire parameter space. We demonstrate the usefulness of these novel rules for identifying which studies contain signal in numerical experiments as well as in application to clinical trials with multiple studies. In various settings, the increase in power from using optimal and maximin procedures can range from 15% to more than 100%.<\/jats:p>","DOI":"10.1111\/rssb.12507","type":"journal-article","created":{"date-parts":[[2022,4,12]],"date-time":"2022-04-12T13:14:23Z","timestamp":1649769263000},"page":"1105-1128","source":"Crossref","is-referenced-by-count":6,"title":["Optimal and Maximin Procedures for Multiple Testing Problems"],"prefix":"10.1093","volume":"84","author":[{"given":"Saharon","family":"Rosset","sequence":"first","affiliation":[{"name":"Department of Statistics and Operations Research , Tel-Aviv , Israel"},{"name":"Tel-Aviv University , Tel-Aviv , Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ruth","family":"Heller","sequence":"additional","affiliation":[{"name":"Department of Statistics and Operations Research , Tel-Aviv , Israel"},{"name":"Tel-Aviv University , Tel-Aviv , Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amichai","family":"Painsky","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering , Tel-Aviv , Israel"},{"name":"Tel-Aviv University , Tel-Aviv , Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ehud","family":"Aharoni","sequence":"additional","affiliation":[{"name":"IBM Haifa Research Labs , Haifa , Israel"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2022,4,12]]},"reference":[{"key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"1983","DOI":"10.1214\/17-EJS1277","article-title":"Distribution-free multiple testing","volume":"11","author":"Arias-Castro","year":"2017","journal-title":"Electronic Journal of Statistics"},{"issue":"6, SI","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"708","DOI":"10.1002\/bimj.200900299","article-title":"Simultaneous and selective inference: current successes and future challenges","volume":"52","author":"Benjamini","year":"2010","journal-title":"Biometrical Journal"},{"issue":"1","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate \u2013 a practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"Journal of the Royal Statisical Society series B - statistical methodology"},{"issue":"2","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"399","DOI":"10.1093\/biomet\/asp006","article-title":"Optimal testing of multiple hypotheses with common effect direction","volume":"96","author":"Bittman","year":"2009","journal-title":"Biometrika"},{"issue":"1","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"79","DOI":"10.2307\/3318603","article-title":"Generalized neyman-pearson lemma via convex duality","volume":"7","author":"Cvitanic","year":"2001","journal-title":"Bernoulli"},{"issue":"22","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"2115","DOI":"10.1056\/NEJMra1709701","article-title":"Multiplicity considerations in clinical trials","volume":"378","author":"Dmitrienko","year":"2018","journal-title":"New England Journal of Medicine"},{"issue":"4","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"753","DOI":"10.1093\/biomet\/asv050","article-title":"Optimal multiple testing under a gaussian prior on the effect sizes","volume":"102","author":"Dobriban","year":"2015","journal-title":"Biometrika"},{"issue":"2","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"3336","DOI":"10.1214\/19-EJS1578","article-title":"Adaptive p-value weighting with power optimality","volume":"13","author":"Durand","year":"2019","journal-title":"Electronic Journal of Statistics"},{"issue":"1","key":"2023030906301119000_","first-page":"1","article-title":"Microarrays, empirical bayes and the two-groups model","volume":"23","author":"Efron","year":"2008","journal-title":"Statistical Science"},{"key":"2023030906301119000_","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511761362","volume-title":"Large-scale inference","author":"Efron","year":"2010"},{"issue":"456","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"1151","DOI":"10.1198\/016214501753382129","article-title":"Empirical Bayes analysis of a microarray experiment","volume":"96","author":"Efron","year":"2001","journal-title":"Journal of the American Statistical Association"},{"key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"596","DOI":"10.1214\/07-AOS569","article-title":"On the false discovery rate and an asymptotically optimal rejection curve","volume":"37","author":"Finner","year":"2009","journal-title":"The Annals of Statistics"},{"issue":"3","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"499","DOI":"10.1111\/1467-9868.00347","article-title":"Operating characteristics and extensions of the false discovery rate procedure","volume":"64","author":"Genovese","year":"2002","journal-title":"Journal of the Royal Statisical Society series B - statistical methodology"},{"issue":"2","key":"2023030906301119000_","first-page":"1218","article-title":"Only closed testing procedures are admissible for controlling false discovery proportions","volume":"49","author":"Goeman","year":"2020","journal-title":"Annals of Statistics"},{"issue":"1","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1111\/rssb.12403","article-title":"Optimal control of false discovery criteria in the two-group model","volume":"83","author":"Heller","year":"2021","journal-title":"Journal of the Royal Statistical Society, Series B"},{"key":"2023030906301119000_","volume-title":"Optimal multiple testing and design in clinical trials","author":"Heller","year":"2021"},{"key":"2023030906301119000_","volume-title":"The cochrane hand book for systematic reviews of interventions version 5.1","author":"Higgins","year":"2011"},{"issue":"4","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"800","DOI":"10.1093\/biomet\/75.4.800","article-title":"A sharper bonferroni procedure for multiple tests of significance","volume":"75","author":"Hochberg","year":"1988","journal-title":"Biometrika"},{"issue":"2","key":"2023030906301119000_","first-page":"65","article-title":"A simple sequentially rejective multiple test procedure","volume":"6","author":"Holm","year":"1979","journal-title":"Scandinavian Journal of Statistics"},{"issue":"2","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"383","DOI":"10.1093\/biomet\/75.2.383","article-title":"A stagewise rejective multiple test procedure based on a modified bonferroni test","volume":"75","author":"Hommel","year":"1988","journal-title":"Biometrika"},{"issue":"8","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"696","DOI":"10.1371\/journal.pmed.0020124","article-title":"Why most published research findings are false","volume":"2","author":"Ioannidis","year":"2005","journal-title":"PLOS Medicine"},{"issue":"2","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"511","DOI":"10.2307\/2532142","article-title":"Procedures for two-sample comparisons with multiple endpoints controlling the experimentwise error rate","volume":"47","author":"Lehmacher","year":"1991","journal-title":"Biometrics"},{"key":"2023030906301119000_","volume-title":"Testing statistical hypotheses","author":"Lehmann","year":"2005"},{"issue":"3","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"1084","DOI":"10.1214\/009053605000000066","article-title":"On optimality of stepdown and stepup multiple test procedures","volume":"33","author":"Lehmann","year":"2005","journal-title":"The Annals of Statistics"},{"issue":"3","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1093\/biomet\/63.3.655","article-title":"Closed testing procedures with special reference to ordered analysis of variance","volume":"63","author":"Marcus","year":"1976","journal-title":"Biometrika"},{"issue":"4","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"355","DOI":"10.1214\/ss\/1009212517","article-title":"The emperor's new tests","volume":"14","author":"Perlman","year":"1999","journal-title":"Statistical Science"},{"key":"2023030906301119000_","volume-title":"Numerical recipes 3rd edition: the art of scientific computing","author":"Press","year":"2007"},{"key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1214\/09-EJS430","article-title":"Optimal weighting for false discovery rate control","volume":"3","author":"Roquain","year":"2009","journal-title":"Electronic Journal of Statistics"},{"key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1016\/j.jspi.2014.07.001","article-title":"Uniformly most powerful tests for simultaneously detecting a treatment effect in the overall population and at least one subpopulation","volume":"155","author":"Rosenblum","year":"2014","journal-title":"Journal of Statistical Planning and Inference"},{"issue":"507","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"1216","DOI":"10.1080\/01621459.2013.879063","article-title":"Optimal tests of treatment effects for the overall population and two subpopulations in randomized trials, using sparse linear programming","volume":"109","author":"Rosenblum","year":"2014","journal-title":"Journal of the American Statistical Association"},{"issue":"4","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"1224","DOI":"10.3150\/10-BEJ249","article-title":"Testing composite hypotheses via convex duality","volume":"16","author":"Rudloff","year":"2010","journal-title":"Bernoulli"},{"key":"2023030906301119000_","first-page":"626","article-title":"Rectangular confidence regions for the means of multivariate normal distributions","volume":"62","author":"Sidak","year":"1967","journal-title":"Journal of the American Statistical Association"},{"issue":"3","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"751","DOI":"10.1093\/biomet\/73.3.751","article-title":"An improved bonferroni procedure for multiple tests of significance","volume":"73","author":"Simes","year":"1986","journal-title":"Biometrika"},{"issue":"11","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"1359","DOI":"10.1177\/0956797611417632","article-title":"False-positive psychology: undisclosed flexibility in data collection and analysis allows presenting anything as significant","volume":"22","author":"Simmons","year":"2011","journal-title":"Psychological Science"},{"issue":"4","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"776","DOI":"10.1002\/bimj.201500253","article-title":"Minimally adaptive bh: a tiny but uniform improvement of the procedure of benjamini and hochberg","volume":"59","author":"Solari","year":"2017","journal-title":"Biometrical Journal"},{"issue":"2","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1214\/aoms\/1177692621","article-title":"On the optimality of some multiple comparison procedures","volume":"43","author":"Spjotvoll","year":"1972","journal-title":"The Annals of Mathematical Statistics"},{"issue":"6","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"2013","DOI":"10.1214\/aos\/1074290335","article-title":"The positive false discovery rate: a bayesian interpretation and the q-value","volume":"31","author":"Storey","year":"2003","journal-title":"Annals of Statistics"},{"issue":"3","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1111\/j.1467-9868.2007.005592.x","article-title":"The optimal discovery procedure: a new approach to simultaneous significance testing","volume":"69","author":"Storey","year":"2007","journal-title":"Journal of the Royal Statisical Society Series B - Statistical Methodology"},{"key":"2023030906301119000_","volume-title":"The American soldier: adjustment during army life","author":"Stouffer","year":"1949"},{"issue":"479","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1198\/016214507000000545","article-title":"Oracle and adaptive compound decision rules for false discovery rate control","volume":"102","author":"Sun","year":"2007","journal-title":"Journal of the American Statistical Association"},{"issue":"1","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/S0378-3758(01)00077-5","article-title":"Optimally weighted, fixed sequence and gatekeeper multiple testing procedures","volume":"99","author":"Westfall","year":"2001","journal-title":"Journal of Statistical Planning and Inference"},{"issue":"18","key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"2107","DOI":"10.1002\/(SICI)1097-0258(19980930)17:18<2107::AID-SIM910>3.0.CO;2-W","article-title":"Using prior information to allocate significance levels for multiple endpoints","volume":"17","author":"Westfall","year":"1998","journal-title":"Statistics in Medicine"},{"key":"2023030906301119000_","doi-asserted-by":"crossref","first-page":"417","DOI":"10.4310\/SII.2011.v4.n4.a1","article-title":"Optimal false discovery rate control for dependent data","volume":"4","author":"Xie","year":"2011","journal-title":"Statistics and its Interface"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/rssb.12507","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/rssb.12507","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/84\/4\/1105\/49463035\/jrsssb_84_4_1105.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/84\/4\/1105\/49463035\/jrsssb_84_4_1105.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,22]],"date-time":"2024-09-22T01:18:32Z","timestamp":1726967912000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/84\/4\/1105\/7073263"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,12]]},"references-count":43,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,4,12]]},"published-print":{"date-parts":[[2022,9,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12507","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,9]]},"published":{"date-parts":[[2022,4,12]]}}}