{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T18:26:04Z","timestamp":1770143164676,"version":"3.49.0"},"reference-count":50,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2023,12,26]],"date-time":"2023-12-26T00:00:00Z","timestamp":1703548800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/pages\/standard-publication-reuse-rights"}],"funder":[{"DOI":"10.13039\/501100007246","name":"Council of higher education","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100007246","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2024,7,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>We consider the problem of inference on the signs of n&amp;gt;1 parameters. We aim to provide 1\u2212\u03b1 post hoc confidence bounds on the number of positive and negative (or non-positive) parameters, with a simultaneous guarantee, for all subsets of parameters. We suggest to start by using the data to select the direction of the hypothesis test for each parameter; then, adjust the p-values of the one-sided hypotheses for the selection, and use the adjusted p-values for simultaneous inference on the selected n one-sided hypotheses. The adjustment is straightforward assuming the p-values of one-sided hypotheses have densities with monotone likelihood ratio, and are mutually independent. We show the bounds we provide are tighter (often by a great margin) than existing alternatives, and that they can be obtained by at most a polynomial time. We demonstrate their usefulness in the evaluation of treatment effects across studies or subgroups. Specifically, we provide a tight lower bound on the number of studies which are beneficial, as well as on the number of studies which are harmful (or non-beneficial), and in addition conclude on the effect direction of individual studies, while guaranteeing that the probability of at least one wrong inference is at most 0.05.<\/jats:p>","DOI":"10.1093\/jrsssb\/qkad137","type":"journal-article","created":{"date-parts":[[2023,12,26]],"date-time":"2023-12-26T13:08:42Z","timestamp":1703596122000},"page":"650-670","source":"Crossref","is-referenced-by-count":2,"title":["Simultaneous directional inference"],"prefix":"10.1093","volume":"86","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4329-1015","authenticated-orcid":false,"given":"Ruth","family":"Heller","sequence":"first","affiliation":[{"name":"Department of Statistics and Operations Research, Tel-Aviv University , Tel-Aviv , Israel"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Aldo","family":"Solari","sequence":"additional","affiliation":[{"name":"Department of Economics, Management and Statistics, University of Milano-Bicocca , Milan , Italy"},{"name":"Department of Economics, Ca\u2019 Foscari University of Venice , Venice , Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2023,12,26]]},"reference":[{"issue":"3","key":"2024071202433706700_qkad137-B1","doi-asserted-by":"crossref","first-page":"677","DOI":"10.1093\/biomet\/asaa013","article-title":"Adaptive critical value for constrained likelihood ratio testing","volume":"107","author":"Al Mohamad","year":"2020","journal-title":"Biometrika"},{"issue":"5","key":"2024071202433706700_qkad137-B2","doi-asserted-by":"crossref","first-page":"2055","DOI":"10.1214\/15-AOS1337","article-title":"Controlling the false discovery rate via knockoffs","volume":"43","author":"Barber","year":"2015","journal-title":"The Annals of Statistics"},{"issue":"1","key":"2024071202433706700_qkad137-B3","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1007\/BF01894737","article-title":"Multiple testing of pairs of one-sided hypotheses","volume":"33","author":"Bauer","year":"1986","journal-title":"Metrika"},{"issue":"6","key":"2024071202433706700_qkad137-B4","doi-asserted-by":"crossref","first-page":"708","DOI":"10.1002\/bimj.200900299","article-title":"Simultaneous and selective inference: Current successes and future challenges","volume":"52","author":"Benjamini","year":"2010","journal-title":"Biometrical Journal"},{"issue":"1","key":"2024071202433706700_qkad137-B5","doi-asserted-by":"crossref","first-page":"297","DOI":"10.1111\/rssb.12028","article-title":"Selective inference on multiple families of hypotheses","volume":"76","author":"Benjamini","year":"2014","journal-title":"Journal of the Royal Statistical Society. Series B (Statistical Methodology)"},{"issue":"4","key":"2024071202433706700_qkad137-B6","doi-asserted-by":"crossref","first-page":"1215","DOI":"10.1111\/j.1541-0420.2007.00984.x","article-title":"Screening for partial conjunction hypotheses","volume":"64","author":"Benjamini","year":"2008","journal-title":"Biometrics"},{"issue":"1","key":"2024071202433706700_qkad137-B7","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: A practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"Journal of the Royal Statistical Society: Series B (Methodological)"},{"issue":"469","key":"2024071202433706700_qkad137-B8","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1198\/016214504000001907","article-title":"False discovery rate-adjusted multiple confidence intervals for selected parameters","volume":"100","author":"Benjamini","year":"2005","journal-title":"Journal of the American Statistical Association"},{"issue":"3","key":"2024071202433706700_qkad137-B9","doi-asserted-by":"crossref","first-page":"1281","DOI":"10.1214\/19-AOS1847","article-title":"Post hoc confidence bounds on false positives using reference families","volume":"48","author":"Blanchard","year":"2020","journal-title":"The Annals of Statistics"},{"issue":"1","key":"2024071202433706700_qkad137-B10","doi-asserted-by":"crossref","first-page":"102","DOI":"10.1214\/22-EJS2100","article-title":"Testing partial conjunction hypotheses under dependency, with applications to meta-analysis","volume":"17","author":"Bogomolov","year":"2023","journal-title":"Electronic Journal of Statistics"},{"issue":"4","key":"2024071202433706700_qkad137-B11","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1214\/23-STS892","article-title":"Replicability across multiple studies","volume":"38","author":"Bogomolov","year":"2023","journal-title":"Statistical Science"},{"issue":"1","key":"2024071202433706700_qkad137-B12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3102\/00346543073001001","article-title":"The effects of modified school calendars on student achievement and on school and community attitudes","volume":"73","author":"Cooper","year":"2003","journal-title":"Review of Educational Research"},{"issue":"3","key":"2024071202433706700_qkad137-B13","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1093\/biomet\/asz082","article-title":"Fast closed testing for exchangeable local tests","volume":"107","author":"Dobriban","year":"2020","journal-title":"Biometrika"},{"issue":"2","key":"2024071202433706700_qkad137-B14","first-page":"e65","article-title":"Gaining power in multiple testing of interval hypotheses via conditionalization","volume":"21","author":"Ellis","year":"2020","journal-title":"Biostatistics"},{"key":"2024071202433706700_qkad137-B15","author":"Finner","year":"1994"},{"issue":"1","key":"2024071202433706700_qkad137-B16","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1214\/aos\/1018031111","article-title":"Stepwise multiple test procedures and control of directional errors","volume":"27","author":"Finner","year":"1999","journal-title":"The Annals of Statistics"},{"issue":"4","key":"2024071202433706700_qkad137-B17","doi-asserted-by":"crossref","first-page":"1194","DOI":"10.1214\/aos\/1031689023","article-title":"The partitioning principle: A powerful tool in multiple decision theory","volume":"30","author":"Finner","year":"2002","journal-title":"Annals of Statistics"},{"key":"2024071202433706700_qkad137-B18","first-page":"57","volume-title":"Handbook of multiple comparisons","author":"Finner","year":"2022"},{"issue":"4","key":"2024071202433706700_qkad137-B19","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1200\/JCO.1983.1.4.227","article-title":"Influence of tumor estrogen and progesterone receptor levels on the response to tamoxifen and chemotherapy in primary breast cancer","volume":"1","author":"Fisher","year":"1983","journal-title":"Journal of Clinical Oncology"},{"issue":"2","key":"2024071202433706700_qkad137-B20","doi-asserted-by":"crossref","first-page":"361","DOI":"10.2307\/2530862","article-title":"Testing for qualitative interactions between treatment effects and patient subsets","volume":"41","author":"Gail","year":"1985","journal-title":"Biometrics"},{"issue":"476","key":"2024071202433706700_qkad137-B21","doi-asserted-by":"crossref","first-page":"1408","DOI":"10.1198\/016214506000000339","article-title":"Exceedance control of the false discovery proportion","volume":"101","author":"Genovese","year":"2006","journal-title":"Journal of the American Statistical Association"},{"issue":"2","key":"2024071202433706700_qkad137-B22","doi-asserted-by":"crossref","first-page":"1218","DOI":"10.1214\/20-AOS1999","article-title":"Only closed testing procedures are admissible for controlling false discovery proportions","volume":"49","author":"Goeman","year":"2021","journal-title":"The Annals of Statistics"},{"issue":"4","key":"2024071202433706700_qkad137-B23","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1093\/biomet\/asz041","article-title":"Simultaneous control of all false discovery proportions in large-scale multiple hypothesis testing","volume":"106","author":"Goeman","year":"2019","journal-title":"Biometrika"},{"issue":"4","key":"2024071202433706700_qkad137-B24","doi-asserted-by":"crossref","first-page":"584","DOI":"10.1214\/11-STS356","article-title":"Multiple testing for exploratory research","volume":"26","author":"Goeman","year":"2011","journal-title":"Statistical Science"},{"key":"2024071202433706700_qkad137-B25","author":"Goeman","year":"2022"},{"issue":"20","key":"2024071202433706700_qkad137-B26","doi-asserted-by":"crossref","first-page":"2117","DOI":"10.1002\/sim.4002","article-title":"Three-sided hypothesis testing: Simultaneous testing of superiority, equivalence and inferiority","volume":"29","author":"Goeman","year":"2010","journal-title":"Statistics in Medicine"},{"key":"2024071202433706700_qkad137-B27","first-page":"1","volume-title":"Reports on public health and medical subjects","author":"Greenwood","year":"1926"},{"issue":"August 2015","key":"2024071202433706700_qkad137-B28","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/j.jspi.2015.02.009","article-title":"On stepwise control of directional errors under independence and some dependence","volume":"163","author":"Guo","year":"2015","journal-title":"Journal of Statistical Planning and Inference"},{"issue":"10","key":"2024071202433706700_qkad137-B29","doi-asserted-by":"crossref","first-page":"2184","DOI":"10.1080\/00949655.2021.2022678","article-title":"Combining independent p-values in replicability analysis: A comparative study","volume":"92","author":"Hoang","year":"2022","journal-title":"Journal of Statistical Computation and Simulation"},{"issue":"2","key":"2024071202433706700_qkad137-B30","first-page":"65","article-title":"A simple sequentially rejective multiple test procedure","volume":"6","author":"Holm","year":"1979","journal-title":"Scandinavian Journal of Statistics"},{"issue":"7","key":"2024071202433706700_qkad137-B31","doi-asserted-by":"crossref","first-page":"e010247","DOI":"10.1136\/bmjopen-2015-010247","article-title":"Plea for routinely presenting prediction intervals in meta-analysis","volume":"6","author":"IntHout","year":"2016","journal-title":"BMJ Open"},{"issue":"4","key":"2024071202433706700_qkad137-B32","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1037\/1082-989X.5.4.411","article-title":"A sensible formulation of the significance test","volume":"5","author":"Jones","year":"2000","journal-title":"Psychol Methods"},{"issue":"2","key":"2024071202433706700_qkad137-B33","doi-asserted-by":"crossref","first-page":"272","DOI":"10.1214\/aoms\/1177728259","article-title":"The theory of decision procedures for distributions with monotone likelihood ratio","volume":"27","author":"Karlin","year":"1956","journal-title":"The Annals of Mathematical Statistics"},{"issue":"1","key":"2024071202433706700_qkad137-B34","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1002\/jrsm.35","article-title":"Fixed effects and variance components estimation in three-level meta-analysis","volume":"2","author":"Konstantopoulos","year":"2011","journal-title":"Research Synthesis Methods"},{"key":"2024071202433706700_qkad137-B35","volume-title":"Testing statistical hypotheses","author":"Lehmann","year":"2005","edition":"3rd ed."},{"issue":"4","key":"2024071202433706700_qkad137-B36","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1111\/rssb.12274","article-title":"AdaPT: An interactive procedure for multiple testing with side information","volume":"80","author":"Lei","year":"2018","journal-title":"Journal of the Royal Statistical Society Series B: Statistical Methodology"},{"key":"2024071202433706700_qkad137-B37","author":"Leung","year":"2023"},{"key":"2024071202433706700_qkad137-B38","author":"Li","year":"2022"},{"issue":"3","key":"2024071202433706700_qkad137-B39","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1016\/S0167-7152(96)00036-3","article-title":"Control of directional errors with step-up multiple tests","volume":"31","author":"Liu","year":"1997","journal-title":"Statistics & Probability Letters"},{"issue":"3","key":"2024071202433706700_qkad137-B40","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1093\/biomet\/63.3.655","article-title":"Closed testing procedures with special reference to ordered analysis of variance","volume":"63","author":"Marcus","year":"1976","journal-title":"Biometrika"},{"key":"2024071202433706700_qkad137-B41","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1214\/lnms\/1196285628","volume-title":"Recent developments in multiple comparison procedures, IMS lecture notes - Monograph series","author":"Sarkar","year":"2004"},{"issue":"346","key":"2024071202433706700_qkad137-B42","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1080\/01621459.1974.10482970","article-title":"Bidirectional unbiased procedures","volume":"69","author":"Shaffer","year":"1974","journal-title":"Journal of the American Statistical Association"},{"issue":"6","key":"2024071202433706700_qkad137-B43","doi-asserted-by":"crossref","first-page":"1342","DOI":"10.1214\/aos\/1176345205","article-title":"Control of directional errors with stagewise multiple test procedures","volume":"8","author":"Shaffer","year":"1980","journal-title":"The Annals of Statistics"},{"issue":"395","key":"2024071202433706700_qkad137-B44","doi-asserted-by":"crossref","first-page":"826","DOI":"10.1080\/01621459.1986.10478341","article-title":"Modified sequentially rejective multiple test procedures","volume":"81","author":"Shaffer","year":"1986","journal-title":"Journal of the American Statistical Association"},{"key":"2024071202433706700_qkad137-B45","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/978-1-4612-3818-8_8","volume-title":"Statistical decision theory and related topics IV","author":"Stefansson","year":"1988"},{"issue":"2","key":"2024071202433706700_qkad137-B46","doi-asserted-by":"crossref","first-page":"750","DOI":"10.1111\/sjos.12614","article-title":"Large-scale simultaneous inference under dependence","volume":"50","author":"Tian","year":"2023","journal-title":"Scandinavian Journal of Statistics"},{"issue":"1","key":"2024071202433706700_qkad137-B47","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1214\/ss\/1177011945","article-title":"The philosophy of multiple comparisons","volume":"6","author":"Tukey","year":"1991","journal-title":"Statistical Science"},{"issue":"3","key":"2024071202433706700_qkad137-B48","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18637\/jss.v036.i03","article-title":"Conducting meta-analyses in R with the metafor package","volume":"36","author":"Viechtbauer","year":"2010","journal-title":"Journal of Statistical Software"},{"issue":"4","key":"2024071202433706700_qkad137-B49","doi-asserted-by":"crossref","first-page":"791","DOI":"10.1093\/biomet\/asaa027","article-title":"Combining p-values via averaging","volume":"107","author":"Vovk","year":"2020","journal-title":"Biometrika"},{"issue":"527","key":"2024071202433706700_qkad137-B50","doi-asserted-by":"crossref","first-page":"1291","DOI":"10.1080\/01621459.2018.1497499","article-title":"Multiple testing when many p-values are uniformly conservative, with application to testing qualitative interaction in educational interventions","volume":"114","author":"Zhao","year":"2019","journal-title":"Journal of the American Statistical Association"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/86\/3\/650\/58504423\/qkad137.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/86\/3\/650\/58504423\/qkad137.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T02:44:27Z","timestamp":1720752267000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/86\/3\/650\/7499129"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,26]]},"references-count":50,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2023,12,26]]},"published-print":{"date-parts":[[2024,7,12]]}},"URL":"https:\/\/doi.org\/10.1093\/jrsssb\/qkad137","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024,7]]},"published":{"date-parts":[[2023,12,26]]}}}