{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T06:05:42Z","timestamp":1790661942026,"version":"4.1.0"},"reference-count":52,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2018,6,4]],"date-time":"2018-06-04T00:00:00Z","timestamp":1528070400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2018,9,1]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>We consider the problem of multiple-hypothesis testing with generic side information: for each hypothesis Hi we observe both a p-value pi and some predictor xi encoding contextual information about the hypothesis. For large-scale problems, adaptively focusing power on the more promising hypotheses (those more likely to yield discoveries) can lead to much more powerful multiple-testing procedures. We propose a general iterative framework for this problem, the adaptive p-value thresholding procedure which we call AdaPT, which adaptively estimates a Bayes optimal p-value rejection threshold and controls the false discovery rate in finite samples. At each iteration of the procedure, the analyst proposes a rejection threshold and observes partially censored p-values, estimates the false discovery proportion below the threshold and proposes another threshold, until the estimated false discovery proportion is below \u03b1. Our procedure is adaptive in an unusually strong sense, permitting the analyst to use any statistical or machine learning method she chooses to estimate the optimal threshold, and to switch between different models at each iteration as information accrues. We demonstrate the favourable performance of AdaPT by comparing it with state of the art methods in five real applications and two simulation studies.<\/jats:p>","DOI":"10.1111\/rssb.12274","type":"journal-article","created":{"date-parts":[[2018,6,5]],"date-time":"2018-06-05T02:02:22Z","timestamp":1528164142000},"page":"649-679","source":"Crossref","is-referenced-by-count":96,"title":["AdaPT: An Interactive Procedure for Multiple Testing with Side Information"],"prefix":"10.1093","volume":"80","author":[{"given":"Lihua","family":"Lei","sequence":"first","affiliation":[{"name":"University of California , Berkeley , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"William","family":"Fithian","sequence":"additional","affiliation":[{"name":"University of California , Berkeley , USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2018,6,4]]},"reference":[{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/S0167-9473(01)00046-9","article-title":"A mixture model approach for the analysis of microarray gene expression data","volume":"39","author":"Allison","year":"2002","journal-title":"Computnl Statist. Data Anal."},{"issue":"10","key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"R106","DOI":"10.1186\/gb-2010-11-10-r106","article-title":"Differential expression analysis for sequence count data","volume":"11","author":"Anders","year":"2010","journal-title":"Genome Biol."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"1983","DOI":"10.1214\/17-EJS1277","article-title":"Distribution-free multiple testing","volume":"11","author":"Arias-Castro","year":"2017","journal-title":"Electron. J. Statist."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"2055","DOI":"10.1214\/15-AOS1337","article-title":"Controlling the false discovery rate via knockoffs","volume":"43","author":"Barber","year":"2015","journal-title":"Ann. Statist."},{"key":"2023022002492333600_","article-title":"A knockoff filter for high-dimensional selective inference","volume-title":"Preprint arXiv:1602.03574","author":"Barber","year":"2016"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"J. R. Statist. Soc."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1111\/1467-9469.00072","article-title":"Multiple hypotheses testing with weights","volume":"24","author":"Benjamini","year":"1997","journal-title":"Scand. J. Statist."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"802","DOI":"10.1214\/12-AOS1077","article-title":"Valid post-selection inference","volume":"41","author":"Berk","year":"2013","journal-title":"Ann. Statist."},{"issue":"3","key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"e17820","DOI":"10.1371\/journal.pone.0017820","article-title":"Evaluating gene expression in c57bl\/6j and dba\/2j mouse striatum using RNA-seq and microarrays","volume":"6","author":"Bottomly","year":"2011","journal-title":"PLOS One"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"9546","DOI":"10.1073\/pnas.0914005107","article-title":"Independent filtering increases detection power for high-throughput experiments","volume":"107","author":"Bourgon","year":"2010","journal-title":"Proc. Natn. Acad. Sci. USA"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1101\/gr.108662.110","article-title":"Conservation of an RNA regulatory map between drosophila and mammals","volume":"21","author":"Brooks","year":"2011","journal-title":"Genome Res."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"1846","DOI":"10.1093\/bioinformatics\/btm254","article-title":"GEOquery: a bridge between the gene expression omnibus (geo) and bioconductor","volume":"23","author":"Davis","year":"2007","journal-title":"Bioinformatics"},{"key":"2023022002492333600_","article-title":"Hyperplexing: a method for higher-order multiplexed quantitative proteomics provides a map of the dynamic response to rapamycin in yeast","volume":"5","author":"Dephoure","year":"2012","journal-title":"Sci. Signlng"},{"key":"2023022002492333600_","article-title":"A general convex framework for multiple testing with prior information","volume-title":"Preprint arXiv:1603.05334","author":"Dobriban","year":"2016"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"753","DOI":"10.1093\/biomet\/asv050","article-title":"Optimal multiple testing under a Gaussian prior on the effect sizes","volume":"102","author":"Dobriban","year":"2015","journal-title":"Biometrika"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","DOI":"10.1201\/9780367807849","volume-title":"An Introduction to Generalized Linear Models","author":"Dobson","year":"2008"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"1262","DOI":"10.1214\/14-AOS1222","article-title":"Single-index modulated multiple testing","volume":"42","author":"Du","year":"2014","journal-title":"Ann. Statist."},{"key":"2023022002492333600_","first-page":"117","article-title":"Preserving statistical validity in adaptive data analysis","volume-title":"Proc. 47th A. Symp. Theory of Computing","author":"Dwork","year":"2015"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1214\/009053606000001460","article-title":"Size, power and false discovery rates","volume":"35","author":"Efron","year":"2007","journal-title":"Ann. Statist."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"1151","DOI":"10.1198\/016214501753382129","article-title":"Empirical Bayes analysis of a microarray experiment","volume":"96","author":"Efron","year":"2001","journal-title":"J. Am. Statist. Ass."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"714","DOI":"10.1214\/08-AOAS158","article-title":"Unsupervised empirical Bayesian multiple testing with external covariates","volume":"2","author":"Ferkingstad","year":"2008","journal-title":"Ann. Appl. Statist."},{"key":"2023022002492333600_","article-title":"Optimal inference after model selection","volume-title":"Preprint arXiv:1410.2597","author":"Fithian","year":"2014"},{"issue":"12","key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"e1005728","DOI":"10.1371\/journal.pgen.1005728","article-title":"Genome-wide scan informed by age-related disease identifies loci for exceptional human longevity","volume":"11","author":"Fortney","year":"2015","journal-title":"PLOS Genet."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"article 449","DOI":"10.1186\/1471-2105-12-449","article-title":"Recount: a multi-experiment resource of analysis-ready RNA-seq gene count datasets","volume":"12","author":"Frazee","year":"2011","journal-title":"BMC Bioinform."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"509","DOI":"10.1093\/biomet\/93.3.509","article-title":"False discovery control with p-value weighting","volume":"93","author":"Genovese","year":"2006","journal-title":"Biometrika"},{"key":"2023022002492333600_","article-title":"genefilter: genefilter: methods for filtering genes from high-throughput experiments","volume-title":"R Package Version 1.54.2","author":"Gentleman","year":"2016"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1111\/rssb.12122","article-title":"Sequential selection procedures and false discovery rate control","volume":"78","author":"Grazier G\u2019Sell","year":"2016","journal-title":"J. R. Statist. Soc."},{"issue":"6","key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"e99625","DOI":"10.1371\/journal.pone.0099625","article-title":"RNA-seq transcriptome profiling identifies crispld2 as a glucocorticoid responsive gene that modulates cytokine function in airway smooth muscle cells","volume":"9","author":"Himes","year":"2014","journal-title":"PLOS One"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"1215","DOI":"10.1198\/jasa.2010.tm09329","article-title":"False discovery rate control with groups","volume":"105","author":"Hu","year":"2010","journal-title":"J. Am. Statist. Ass."},{"key":"2023022002492333600_","article-title":"pasilla: data package with per-exon and per-gene read counts of RNA-seq samples of Pasilla knock-down by Brooks et al., Genome Research 2011","volume-title":"R Package Version 0.12.0","author":"Huber","year":"2016"},{"key":"2023022002492333600_","article-title":"Covariate-powered weighted multiple testing with false discovery rate control","volume-title":"Preprint arXiv:1701.05179","author":"Ignatiadis","year":"2017"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1038\/nmeth.3885","article-title":"Data-driven hypothesis weighting increases detection power in genome-scale multiple testing","volume":"13","author":"Ignatiadis","year":"2016","journal-title":"Nat. Meth."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1016\/j.neuroimage.2009.03.047","article-title":"Local and covariate-modulated false discovery rates applied in neuroimaging","volume":"47","author":"Lawyer","year":"2009","journal-title":"NeuroImage"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1214\/15-AOS1371","article-title":"Exact post-selection inference, with application to the lasso","volume":"44","author":"Lee","year":"2016","journal-title":"Ann. Statist."},{"key":"2023022002492333600_","first-page":"2924","article-title":"Power of ordered hypothesis testing","volume-title":"Proc. Int. Conf. Machine Learning","author":"Lei","year":"2016"},{"key":"2023022002492333600_","article-title":"STAR: a general interactive framework for fdr control under structural constraints","volume-title":"Preprint arXiv:1710.02776","author":"Lei","year":"2017"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"871","DOI":"10.1002\/gepi.20248","article-title":"Hierarchical Bayes prioritization of marker associations from a genome-wide association scan for further investigation","volume":"31","author":"Lewinger","year":"2007","journal-title":"Genet. Epidem."},{"key":"2023022002492333600_","article-title":"Multiple testing with the structure adaptive Benjamini-Hochberg algorithm","volume-title":"Preprint arXiv:1606.07926","author":"Li","year":"2016"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"837","DOI":"10.1080\/01621459.2016.1180989","article-title":"Accumulation tests for FDR control in ordered hypothesis testing","volume":"112","author":"Li","year":"2017","journal-title":"J. Am. Statist. Ass."},{"issue":"12","key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1186\/s13059-014-0550-8","article-title":"Moderated estimation of fold change and dispersion for RNA-seq data with deseq2","volume":"15","author":"Love","year":"2014","journal-title":"Genome Biol."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"640","DOI":"10.1093\/bioinformatics\/btq001","article-title":"A censored beta mixture model for the estimation of the proportion of non-differentially expressed genes","volume":"26","author":"Markitsis","year":"2010","journal-title":"Bioinformatics"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"1031","DOI":"10.1002\/sim.4780071005","article-title":"Identifying important results from multiple statistical tests","volume":"7","author":"Parker","year":"1988","journal-title":"Statist. Med."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"1236","DOI":"10.1093\/bioinformatics\/btg148","article-title":"Estimating the occurrence of false positives and false negatives in microarray studies by approximating and partitioning the empirical distribution of p-values","volume":"19","author":"Pounds","year":"2003","journal-title":"Bioinformatics"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1111\/1467-9868.00346","article-title":"A direct approach to false discovery rates","volume":"64","author":"Storey","year":"2002","journal-title":"J. R. Statist. Soc."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"347","DOI":"10.1111\/j.1467-9868.2007.005592.x","article-title":"The optimal discovery procedure: a new approach to simultaneous significance testing","volume":"69","author":"Storey","year":"2007","journal-title":"J. R. Statist. Soc."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1111\/j.1467-9868.2004.00439.x","article-title":"Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach","volume":"66","author":"Storey","year":"2004","journal-title":"J. R. Statist. Soc."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"9440","DOI":"10.1073\/pnas.1530509100","article-title":"Statistical significance for genomewide studies","volume":"100","author":"Storey","year":"2003","journal-title":"Proc. Natn. Acad. Sci. USA"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1111\/rssb.12064","article-title":"False discovery control in large-scale spatial multiple testing","volume":"77","author":"Sun","year":"2015","journal-title":"J. R. Statist. Soc."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1214\/17-AOS1564","article-title":"Selective inference with a randomized response","volume":"46","author":"Tian","year":"2018","journal-title":"Ann. Statist."},{"key":"2023022002492333600_","article-title":"The Collected Works of John W. Tukey","volume-title":"Multiple Comparisons, 1948-1983","author":"Tukey","year":"1994"},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1111\/j.1467-9868.2011.01016.x","article-title":"Adjusted Bayesian inference for selected parameters","volume":"74","author":"Yekutieli","year":"2012","journal-title":"J. R. Statist. Soc."},{"key":"2023022002492333600_","doi-asserted-by":"crossref","first-page":"2098","DOI":"10.1093\/bioinformatics\/btu145","article-title":"Covariate-modulated local false discovery rate for genome-wide association studies","volume":"30","author":"Zablocki","year":"2014","journal-title":"Bioinformatics"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/api.wiley.com\/onlinelibrary\/tdm\/v1\/articles\/10.1111%2Frssb.12274","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/80\/4\/649\/49270704\/jrsssb_80_4_649.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/80\/4\/649\/49270704\/jrsssb_80_4_649.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,7]],"date-time":"2024-07-07T08:15:41Z","timestamp":1720340141000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/80\/4\/649\/7048376"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,4]]},"references-count":52,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2018,6,4]]},"published-print":{"date-parts":[[2018,9,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12274","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2018,9]]},"published":{"date-parts":[[2018,6,4]]}}}