{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T19:59:36Z","timestamp":1791403176906,"version":"4.3.3"},"reference-count":44,"publisher":"Institute of Mathematical Statistics","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Ann. Statist."],"published-print":{"date-parts":[[2022,8,1]]},"abstract":"<jats:p>Replicability is a fundamental quality of scientific discoveries: we are interested in those signals that are detectable in different laboratories, different populations, across time etc. Unlike meta-analysis which accounts for experimental variability but does not guarantee replicability, testing a partial conjunction (PC) null aims specifically to identify the signals that are discovered in multiple studies. In many contemporary applications, for example, comparing multiple high-throughput genetic experiments, a large number M of PC nulls need to be tested simultaneously, calling for a multiple comparisons correction. However, standard multiple testing adjustments on the M PC p-values can be severely conservative, especially when M is large and the signals are sparse. We introduce AdaFilter, a new multiple testing procedure that increases power by adaptively filtering out unlikely candidates of PC nulls. We prove that AdaFilter can control FWER and FDR as long as data across studies are independent, and has much higher power than other existing methods. We illustrate the application of AdaFilter with three examples: microarray studies of Duchenne muscular dystrophy, single-cell RNA sequencing of T cells in lung cancer tumors and GWAS for metabolomics.<\/jats:p>","DOI":"10.1214\/21-aos2139","type":"journal-article","created":{"date-parts":[[2022,8,29]],"date-time":"2022-08-29T15:39:37Z","timestamp":1661787577000},"page":"1890-1909","source":"Crossref","is-referenced-by-count":16,"title":["Detecting multiple replicating signals using adaptive filtering procedures"],"prefix":"10.1214","volume":"50","author":[{"given":"Jingshu","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Statistics, The University of Chicago"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lin","family":"Gui","sequence":"additional","affiliation":[{"name":"Department of Statistics, The University of Chicago"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Weijie J.","family":"Su","sequence":"additional","affiliation":[{"name":"Department of Statistics and Data Science, University of Pennsylvania"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chiara","family":"Sabatti","sequence":"additional","affiliation":[{"name":"Department of Statistics, Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Art B.","family":"Owen","sequence":"additional","affiliation":[{"name":"Department of Statistics, Stanford University"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"108","reference":[{"key":"6","doi-asserted-by":"publisher","unstructured":"<small>Bogdan, M.<\/small>, <small>van den Berg, E.<\/small>, <small>Sabatti, C.<\/small>, <small>Su, W.<\/small> and <small>Cand\u00e8s, E. J.<\/small> (2015). SLOPE\u2014adaptive variable selection via convex optimization. <i>Ann. Appl. Stat.<\/i> <b>9<\/b> 1103\u20131140.","DOI":"10.1214\/15-AOAS842"},{"key":"44","doi-asserted-by":"publisher","unstructured":"<small>ZHAO, S. D.<\/small> and <small>NGUYEN, Y. T.<\/small> (2020). Nonparametric false discovery rate control for identifying simultaneous signals. <i>Electron. J. Stat.<\/i> <b>14<\/b> 110\u2013142.","DOI":"10.1214\/19-EJS1663"},{"key":"29","doi-asserted-by":"publisher","unstructured":"<small>LEI, L.<\/small> and <small>FITHIAN, W.<\/small> (2018). AdaPT: An interactive procedure for multiple testing with side information. <i>J. R. Stat. Soc. Ser. B. Stat. Methodol.<\/i> <b>80<\/b> 649\u2013679.","DOI":"10.1111\/rssb.12253"},{"key":"12","doi-asserted-by":"publisher","unstructured":"<small>FERREIRA, J. A.<\/small> and <small>ZWINDERMAN, A. H.<\/small> (2006). On the Benjamini\u2013Hochberg method. <i>Ann. Statist.<\/i> <b>34<\/b> 1827\u20131849.","DOI":"10.1214\/009053606000000425"},{"key":"21","doi-asserted-by":"publisher","unstructured":"<small>HELLER, R.<\/small> and <small>YEKUTIELI, D.<\/small> (2014). Replicability analysis for genome-wide association studies. <i>Ann. Appl. Stat.<\/i> <b>8<\/b> 481\u2013498.","DOI":"10.1214\/13-AOAS697"},{"key":"1","doi-asserted-by":"crossref","unstructured":"<small>BAKER, M.<\/small> (2016). 1,500 scientists lift the lid on reproducibility. <i>Nature<\/i> <b>533<\/b> 452\u2013454.","DOI":"10.1038\/533452a"},{"key":"2","doi-asserted-by":"publisher","unstructured":"<small>BARBER, R. F.<\/small> and <small>RAMDAS, A.<\/small> (2017). The <i>p<\/i>-filter: Multilayer false discovery rate control for grouped hypotheses. <i>J. R. Stat. Soc. Ser. B. Stat. Methodol.<\/i> <b>79<\/b> 1247\u20131268.","DOI":"10.1111\/rssb.12218"},{"key":"3","doi-asserted-by":"crossref","unstructured":"<small>BEGLEY, C. G.<\/small> and <small>ELLIS, L. M.<\/small> (2012). Drug development: Raise standards for preclinical cancer research. <i>Nature<\/i> <b>483<\/b> 531\u2013533.","DOI":"10.1038\/483531a"},{"key":"7","doi-asserted-by":"publisher","unstructured":"<small>BOGOMOLOV, M.<\/small> and <small>HELLER, R.<\/small> (2018). Assessing replicability of findings across two studies of multiple features. <i>Biometrika<\/i> <b>105<\/b> 505\u2013516.","DOI":"10.1093\/biomet\/asy029"},{"key":"8","doi-asserted-by":"publisher","unstructured":"<small>DJORDJILOVI\u0106, V.<\/small>, <small>HEMERIK, J.<\/small> and <small>THORESEN, M.<\/small> (2022). On optimal two-stage testing of multiple mediators. <i>Biom. J.<\/i>","DOI":"10.1002\/bimj.202100190"},{"key":"9","doi-asserted-by":"publisher","unstructured":"<small>DJORDJILOVI\u0106, V.<\/small>, <small>PAGE, C. M.<\/small>, <small>GRAN, J. M.<\/small>, <small>N\u00d8ST, T. H.<\/small>, <small>SANDANGER, T. M.<\/small>, <small>VEIER\u00d8D, M. B.<\/small> and <small>THORESEN, M.<\/small> (2019). Global test for high-dimensional mediation: Testing groups of potential mediators. <i>Stat. Med.<\/i> <b>38<\/b> 3346\u20133360.","DOI":"10.1002\/sim.8199"},{"key":"14","doi-asserted-by":"crossref","unstructured":"<small>FRISTON, K. J.<\/small>, <small>PENNY, W. D.<\/small> and <small>GLASER, D. E.<\/small> (2005). Conjunction revisited. <i>NeuroImage<\/i> <b>25<\/b> 661\u2013667.","DOI":"10.1016\/j.neuroimage.2005.01.013"},{"key":"15","unstructured":"<small>GIRI, A.<\/small>, <small>HELLWEGE, J. N.<\/small>, <small>KEATON, J. M.<\/small>, <small>PARK, J.<\/small>, <small>QIU, C.<\/small>, <small>WARREN, H. R.<\/small>, <small>TORSTENSON, E. S.<\/small>, <small>KOVESDY, C. P.<\/small>, <small>SUN, Y. V.<\/small> et al. (2019). Trans-ethnic association study of blood pressure determinants in over 750,000 individuals. <i>Nat. Genet.<\/i> <b>51<\/b> 51\u201362."},{"key":"17","unstructured":"<small>GUO, X.<\/small>, <small>ZHANG, Y.<\/small>, <small>ZHENG, L.<\/small>, <small>ZHENG, C.<\/small>, <small>SONG, J.<\/small>, <small>ZHANG, Q.<\/small>, <small>KANG, B.<\/small>, <small>LIU, Z.<\/small>, <small>JIN, L.<\/small> et al. (2018). Global characterization of T cells in non-small-cell lung cancer by single-cell sequencing. <i>Nat. Med.<\/i> <b>24<\/b> 978\u2013985."},{"key":"18","doi-asserted-by":"publisher","unstructured":"<small>HASIN, Y.<\/small>, <small>SELDIN, M.<\/small> and <small>LUSIS, A.<\/small> (2017). Multi-omics approaches to disease. <i>Genome Biol.<\/i> <b>18<\/b> 83.","DOI":"10.1186\/s13059-017-1215-1"},{"key":"19","doi-asserted-by":"crossref","unstructured":"<small>HELLER, R.<\/small>, <small>BOGOMOLOV, M.<\/small> and <small>BENJAMINI, Y.<\/small> (2014). Deciding whether follow-up studies have replicated findings in a preliminary large-scale omics study. <i>Proc. Natl. Acad. Sci. USA<\/i> <b>111<\/b> 16262\u201316267.","DOI":"10.1073\/pnas.1314814111"},{"key":"20","doi-asserted-by":"crossref","unstructured":"<small>HELLER, R.<\/small>, <small>GOLLAND, Y.<\/small>, <small>MALACH, R.<\/small> and <small>BENJAMINI, Y.<\/small> (2007). Conjunction group analysis: An alternative to mixed\/random effect analysis. <i>NeuroImage<\/i> <b>37<\/b> 1178\u20131185.","DOI":"10.1016\/j.neuroimage.2007.05.051"},{"key":"22","unstructured":"<small>HUME, D.<\/small> (2003). <i>A Treatise of Human Nature<\/i>. Courier Corporation."},{"key":"23","unstructured":"<small>INTRODUCTION TO REPLICABILITY IN SCIENCE<\/small>. Available at <a href=\"http:\/\/www.replicability.tau.ac.il\/index.php\/replicability-in-science.html\">http:\/\/www.replicability.tau.ac.il\/index.php\/replicability-in-science.html<\/a>. Accessed: 2018-08-29."},{"key":"24","unstructured":"<small>IRIZARRY, R. A.<\/small>, <small>HOBBS, B.<\/small>, <small>COLLIN, F.<\/small>, <small>BEAZER-BARCLAY, Y. D.<\/small>, <small>ANTONELLIS, K. J.<\/small>, <small>SCHERF, U.<\/small> et al. (2003). Exploration, normalization, and summaries of high density oligonucleotide array probe level data. <i>Biostatistics<\/i> <b>4<\/b> 249\u2013264."},{"key":"25","doi-asserted-by":"publisher","unstructured":"<small>JALJULI, I.<\/small>, <small>BENJAMINI, Y.<\/small>, <small>SHENHAV, L.<\/small>, <small>PANAGIOTOU, O.<\/small> and <small>HELLER, R.<\/small> (2022). Quantifying replicability and consistency in systematic reviews. <i>Stat. Biopharm. Res.<\/i>.","DOI":"10.1080\/19466315.2022.2050291"},{"key":"26","doi-asserted-by":"publisher","unstructured":"<small>KARMAKAR, B.<\/small>, <small>SMALL, D. S.<\/small> and <small>ROSENBAUM, P. R.<\/small> (2021). Reinforced designs: Multiple instruments plus control groups as evidence factors in an observational study of the effectiveness of Catholic schools. <i>J. Amer. Statist. Assoc.<\/i> <b>116<\/b> 82\u201392.","DOI":"10.1080\/01621459.2020.1745811"},{"key":"27","doi-asserted-by":"crossref","unstructured":"<small>KOTELNIKOVA, E.<\/small>, <small>SHKROB, M. A.<\/small>, <small>PYATNITSKIY, M. A.<\/small>, <small>FERLINI, A.<\/small> and <small>DARASELIA, N.<\/small> (2012). Novel approach to meta-analysis of microarray datasets reveals muscle remodeling-related drug targets and biomarkers in Duchenne muscular dystrophy. <i>PLoS Comput. Biol.<\/i> <b>8<\/b> e1002365.","DOI":"10.1371\/journal.pcbi.1002365"},{"key":"28","unstructured":"<small>LEHRER, J.<\/small> (2010). The truth wears off. <i>New Yorker<\/i>."},{"key":"31","doi-asserted-by":"publisher","unstructured":"<small>LIU, Z.<\/small>, <small>SHEN, J.<\/small>, <small>BARFIELD, R.<\/small>, <small>SCHWARTZ, J.<\/small>, <small>BACCARELLI, A. A.<\/small> and <small>LIN, X.<\/small> (2022). Large-scale hypothesis testing for causal mediation effects with applications in genome-wide epigenetic studies. <i>J. Amer. Statist. Assoc.<\/i> <b>117<\/b> 67\u201381.","DOI":"10.1080\/01621459.2021.1914634"},{"key":"32","doi-asserted-by":"publisher","unstructured":"<small>MARIGORTA, U. M.<\/small> and <small>NAVARRO, A.<\/small> (2013). High trans-ethnic replicability of GWAS results implies common causal variants. <i>PLoS Genet.<\/i> <b>9<\/b> e1003566.","DOI":"10.1371\/journal.pgen.1003566"},{"key":"33","doi-asserted-by":"crossref","unstructured":"<small>MOONESINGHE, R.<\/small>, <small>KHOURY, M. J.<\/small> and <small>JANSSENS, A. C. J. W.<\/small> (2007). Most published research findings are false\u2014but a little replication goes a long way. <i>PLoS Med.<\/i> <b>4<\/b> e28.","DOI":"10.1371\/journal.pmed.0040028"},{"key":"34","doi-asserted-by":"publisher","unstructured":"<small>OWEN, A. B.<\/small> (2009). Karl Pearson\u2019s meta-analysis revisited. <i>Ann. Statist.<\/i> <b>37<\/b> 3867\u20133892.","DOI":"10.1214\/09-AOS697"},{"key":"35","doi-asserted-by":"crossref","unstructured":"<small>PRICE, C. J.<\/small> and <small>FRISTON, K. J.<\/small> (1997). Cognitive conjunction: A new approach to brain activation experiments. <i>NeuroImage<\/i> <b>5<\/b> 261\u2013270.","DOI":"10.1006\/nimg.1997.0269"},{"key":"36","doi-asserted-by":"publisher","unstructured":"<small>SMYTH, G. K.<\/small> (2004). Linear models and empirical Bayes methods for assessing differential expression in microarray experiments. <i>Stat. Appl. Genet. Mol. Biol.<\/i> <b>3<\/b> Art. 3, 29.","DOI":"10.2202\/1544-6115.1027"},{"key":"38","unstructured":"<small>TUKEY, J. W.<\/small> (1953). The problem of multiple comparisons. Technical report, Princeton Univ."},{"key":"39","doi-asserted-by":"crossref","unstructured":"<small>URBUT, S. M.<\/small>, <small>WANG, G.<\/small>, <small>CARBONETTO, P.<\/small> and <small>STEPHENS, M.<\/small> (2019). Flexible statistical methods for estimating and testing effects in genomic studies with multiple conditions. <i>Nat. Genet.<\/i> <b>51<\/b> 187\u2013195.","DOI":"10.1038\/s41588-018-0268-8"},{"key":"40","doi-asserted-by":"crossref","unstructured":"<small>WANG, J.<\/small>, <small>GUI, L.<\/small>, <small>SU, W. J.<\/small>, <small>SABATTI, C.<\/small> and <small>OWEN, A. B.<\/small> (2022). Supplement to \u201cDetecting multiple replicating signals using adaptive filtering procedures.\u201d <a href=\"https:\/\/doi.org\/10.1214\/21-AOS2139SUPP\">https:\/\/doi.org\/10.1214\/21-AOS2139SUPP<\/a>","DOI":"10.1214\/21-AOS2139SUPP"},{"key":"42","doi-asserted-by":"crossref","unstructured":"<small>XIANG, D.<\/small>, <small>ZHAO, S. D.<\/small> and <small>CAI, T. T.<\/small> (2019). Signal classification for the integrative analysis of multiple sequences of large-scale multiple tests. <i>J. R. Stat. Soc. Ser. B. Stat. Methodol.<\/i> <b>81<\/b> 707\u2013734.","DOI":"10.1111\/rssb.12323"},{"key":"43","doi-asserted-by":"crossref","unstructured":"<small>ZHANG, N. R.<\/small>, <small>SENBABAOGLU, Y.<\/small> and <small>LI, J. Z.<\/small> (2010). Joint estimation of DNA copy number from multiple platforms. <i>Bioinformatics<\/i> <b>26<\/b> 153\u2013160.","DOI":"10.1093\/bioinformatics\/btp653"},{"key":"5","doi-asserted-by":"crossref","unstructured":"<small>Benjamini, Y.<\/small> and <small>Hochberg, Y.<\/small> (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. <i>J. Roy. Statist. Soc. Ser. B<\/i> <b>57<\/b> 289\u2013300.","DOI":"10.1111\/j.2517-6161.1995.tb02031.x"},{"key":"4","doi-asserted-by":"publisher","unstructured":"<small>Benjamini, Y.<\/small> and <small>Heller, R.<\/small> (2008). Screening for partial conjunction hypotheses. <i>Biometrics<\/i> <b>64<\/b> 1215\u20131222.","DOI":"10.1111\/j.1541-0420.2007.00984.x"},{"key":"16","doi-asserted-by":"publisher","unstructured":"<small>Goeman, J. J.<\/small> and <small>Solari, A.<\/small> (2010). The sequential rejection principle of familywise error control. <i>Ann. Statist.<\/i> <b>38<\/b> 3782\u20133810.","DOI":"10.1214\/10-AOS829"},{"key":"37","doi-asserted-by":"publisher","unstructured":"<small>Sun, W.<\/small>, <small>Reich, B. J.<\/small>, <small>Cai, T. T.<\/small>, <small>Guindani, M.<\/small> and <small>Schwartzman, A.<\/small> (2015). False discovery control in large-scale spatial multiple testing. <i>J. R. Stat. Soc. Ser. B. Stat. Methodol.<\/i> <b>77<\/b> 59\u201383.","DOI":"10.1111\/rssb.12064"},{"key":"41","doi-asserted-by":"publisher","unstructured":"<small>Wang, J.<\/small> and <small>Owen, A. B.<\/small> (2019). Admissibility in partial conjunction testing. <i>J. Amer. Statist. Assoc.<\/i> <b>114<\/b> 158\u2013168.","DOI":"10.1080\/01621459.2017.1385465"},{"key":"13","doi-asserted-by":"publisher","unstructured":"<small>Flutre, T.<\/small>, <small>Wen, X.<\/small>, <small>Pritchard, J.<\/small> and <small>Stephens, M.<\/small> (2013). A statistical framework for joint eQTL analysis in multiple tissues. <i>PLoS Genet.<\/i> <b>9<\/b> e1003486.","DOI":"10.1371\/journal.pgen.1003486"},{"key":"11","doi-asserted-by":"publisher","unstructured":"<small>Efron, B.<\/small> (2010). <i>Large-Scale Inference<\/i>: <i>Empirical Bayes Methods for Estimation, Testing, and Prediction<\/i>. <i>Institute of Mathematical Statistics<\/i> (<i>IMS<\/i>) <i>Monographs<\/i> <b>1<\/b>. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/CBO9780511761362"},{"key":"10","doi-asserted-by":"publisher","unstructured":"<small>Dudoit, S.<\/small> and <small>van der Laan, M. J.<\/small> (2008). <i>Multiple Testing Procedures with Applications to Genomics<\/i>. <i>Springer Series in Statistics<\/i>. Springer, New York.","DOI":"10.1007\/978-0-387-49317-6"},{"key":"30","doi-asserted-by":"crossref","unstructured":"<small>Li, A.<\/small> and <small>Barber, R. F.<\/small> (2019). Multiple testing with the structure-adaptive Benjamini\u2013Hochberg algorithm. <i>J. R. Stat. Soc. Ser. B. Stat. Methodol.<\/i> <b>81<\/b> 45\u201374.","DOI":"10.1111\/rssb.12298"}],"container-title":["The Annals of Statistics"],"original-title":[],"deposited":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T19:26:44Z","timestamp":1791401204000},"score":1,"resource":{"primary":{"URL":"https:\/\/projecteuclid.org\/journals\/annals-of-statistics\/volume-50\/issue-4\/Detecting-multiple-replicating-signals-using-adaptive-filtering-procedures\/10.1214\/21-AOS2139.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,8,1]]},"references-count":44,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,8,1]]}},"URL":"https:\/\/doi.org\/10.1214\/21-aos2139","relation":{},"ISSN":["0090-5364","2168-8966"],"issn-type":[{"value":"0090-5364","type":"print"},{"value":"2168-8966","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,8,1]]}}}