{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,23]],"date-time":"2026-08-23T18:47:04Z","timestamp":1787510824991,"version":"build-2736575974"},"reference-count":60,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2026,4,5]],"date-time":"2026-04-05T00:00:00Z","timestamp":1775347200000},"content-version":"vor","delay-in-days":94,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["R01-AG065276"],"award-info":[{"award-number":["R01-AG065276"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"NIH","doi-asserted-by":"publisher","award":["R01-GM139926"],"award-info":[{"award-number":["R01-GM139926"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,2,13]]},"abstract":"<jats:title>Summary<\/jats:title>\n                  <jats:p>We consider the problem of comparing a reference distribution with several other distributions. Given a sample from both the reference and the comparison groups, we aim to identify the comparison groups whose distributions differ from that of the reference group. Viewing this as a multiple-testing problem, we introduce a methodology that provides exact, distribution-free control of the false discovery rate. To do so, we introduce the concept of batch conformal p -values and demonstrate that they satisfy positive regression dependence across the groups Benjamini &amp; Yekutieli (2001), thereby enabling control of the false discovery rate through the Benjamini\u2013Hochberg procedure. The proof of positive regression dependence introduces a novel technique for the inductive construction of rank vectors with almost-sure dominance under exchangeability. We evaluate the performance of the proposed procedure through simulations. Despite being distribution-free, in some cases it shows performance comparable to methods with knowledge of the data-generating normal distribution, and it further has more power than direct approaches based on conformal out-of-distribution detection. Furthermore, we illustrate our methods on a hepatitis C treatment dataset, where they identify patient groups with large treatment effects, and on the Current Population Survey dataset, where they identify subpopulations with long working hours.<\/jats:p>","DOI":"10.1093\/biomet\/asag025","type":"journal-article","created":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T12:46:11Z","timestamp":1774701971000},"source":"Crossref","is-referenced-by-count":1,"title":["Finding distributions that differ, with false discovery rate control"],"prefix":"10.1093","volume":"113","author":[{"given":"Yonghoon","family":"Lee","sequence":"first","affiliation":[{"name":"Department of Statistics and Data Science, The Wharton School, University of Pennsylvania , 265 South 37th Street , Philadelphia, Pennsylvania 19104,","place":["U.S.A"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edgar","family":"Dobriban","sequence":"additional","affiliation":[{"name":"Department of Statistics and Data Science, The Wharton School, University of Pennsylvania , 265 South 37th Street , Philadelphia, Pennsylvania 19104,","place":["U.S.A"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric J","family":"Tchetgen Tchetgen","sequence":"additional","affiliation":[{"name":"Department of Statistics and Data Science, The Wharton School, University of Pennsylvania , 265 South 37th Street , Philadelphia, Pennsylvania 19104,","place":["U.S.A"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,4,4]]},"reference":[{"key":"2026060915372225400_asag025-B1","doi-asserted-by":"crossref","first-page":"626","DOI":"10.1139\/f01-004","article-title":"Permutation tests for univariate or multivariate analysis of variance and regression","volume":"58","author":"Anderson","year":"2001","journal-title":"Can. J. Fish. Aquat. Sci."},{"key":"2026060915372225400_asag025-B2","author":"Angelopoulos","year":"2022"},{"key":"2026060915372225400_asag025-B3","doi-asserted-by":"crossref","first-page":"1615","DOI":"10.1093\/jrsssb\/qkad072","article-title":"Semi-parametric estimation of treatment effects in randomised experiments","volume":"85","author":"Athey","year":"2023","journal-title":"J. R. Statist. Soc. B"},{"key":"2026060915372225400_asag025-B4","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1214\/22-AOS2244","article-title":"Testing for outliers with conformal $ p $-values","volume":"51","author":"Bates","year":"2023","journal-title":"Ann. Statist"},{"key":"2026060915372225400_asag025-B5","doi-asserted-by":"crossref","first-page":"405","DOI":"10.1111\/j.1467-9868.2010.00746.x","article-title":"Discovering the false discovery rate","volume":"72","author":"Benjamini","year":"2010","journal-title":"J. R. Statist. Soc. B"},{"key":"2026060915372225400_asag025-B6","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1111\/j.2517-6161.1995.tb02031.x","article-title":"Controlling the false discovery rate: a practical and powerful approach to multiple testing","volume":"57","author":"Benjamini","year":"1995","journal-title":"J. R. Statist. Soc. B"},{"key":"2026060915372225400_asag025-B8","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1093\/biomet\/93.3.491","article-title":"Adaptive linear step-up procedures that control the false discovery rate","volume":"93","author":"Benjamini","year":"2006","journal-title":"Biometrika"},{"key":"2026060915372225400_asag025-B7","doi-asserted-by":"crossref","first-page":"1165","DOI":"10.1214\/aos\/1013699998","article-title":"The control of the false discovery rate in multiple testing under dependency","volume":"29","author":"Benjamini","year":"2001","journal-title":"Ann. Statist."},{"key":"2026060915372225400_asag025-B9","doi-asserted-by":"crossref","first-page":"913","DOI":"10.1093\/biomet\/asu045","article-title":"A distribution-free two-sample run test applicable to high-dimensional data","volume":"101","author":"Biswas","year":"2014","journal-title":"Biometrika"},{"key":"2026060915372225400_asag025-B10","doi-asserted-by":"crossref","first-page":"963","DOI":"10.1214\/08-EJS180","article-title":"Two simple sufficient conditions for FDR control","volume":"2","author":"Blanchard","year":"2008","journal-title":"Electron. J. Statist."},{"key":"2026060915372225400_asag025-B11","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1214\/aos\/1176342662","article-title":"Empirical probability plots and statistical inference for nonlinear models in the two-sample case","volume":"2","author":"Doksum","year":"1974","journal-title":"Ann. Statist."},{"key":"2026060915372225400_asag025-B12","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1093\/biomet\/63.3.421","article-title":"Plotting with confidence: graphical comparisons of two populations","volume":"63","author":"Doksum","year":"1976","journal-title":"Biometrika"},{"key":"2026060915372225400_asag025-B13","doi-asserted-by":"crossref","first-page":"1096","DOI":"10.1080\/01621459.1955.10501294","article-title":"A multiple comparison procedure for comparing several treatments with a control","volume":"50","author":"Dunnett","year":"1955","journal-title":"J. Am. Statist. Assoc"},{"key":"2026060915372225400_asag025-B14","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1017\/S0021859600052862","article-title":"On the validity of Fisher\u2019s $ z $ test when applied to an actual example of non-normal data","volume":"23","author":"Eden","year":"1933","journal-title":"J. Agric. Sci"},{"key":"2026060915372225400_asag025-B15","doi-asserted-by":"crossref","first-page":"676","DOI":"10.1214\/088342304000000396","article-title":"Permutation methods: a basis for exact inference","volume":"19","author":"Ernst","year":"2004","journal-title":"Statist. Sci."},{"key":"2026060915372225400_asag025-B16","volume-title":"The Design of Experiments","author":"Fisher","year":"1935"},{"key":"2026060915372225400_asag025-B17","volume-title":"Permutation, Parametric, and Bootstrap Tests of Hypotheses","author":"Good","year":"2006"},{"key":"2026060915372225400_asag025-B18","first-page":"723","article-title":"A kernel two-sample test","volume":"13","author":"Gretton","year":"2012","journal-title":"J. Mach. Learn. Res"},{"key":"2026060915372225400_asag025-B19","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1007\/s11749-017-0571-1","article-title":"Exact testing with random permutations","volume":"27","author":"Hemerik","year":"2018","journal-title":"Test"},{"key":"2026060915372225400_asag025-B20","author":"Hothorn","year":"2020"},{"key":"2026060915372225400_asag025-B21","doi-asserted-by":"crossref","first-page":"1136","DOI":"10.1080\/01621459.2023.2177165","article-title":"A two-sample conditional distribution test using conformal prediction and weighted rank sum","volume":"119","author":"Hu","year":"2024","journal-title":"J. Am. Statist. Assoc"},{"key":"2026060915372225400_asag025-B22","doi-asserted-by":"crossref","first-page":"asaf001","DOI":"10.1093\/biomet\/asaf001","article-title":"Two-sample distribution tests in high dimensions via max-sliced Wasserstein distance and bootstrapping","volume":"112","author":"Hu","year":"2025","journal-title":"Biometrika"},{"key":"2026060915372225400_asag025-B23","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1080\/07350015.1995.10524581","article-title":"Randomization tests in econometrics","volume":"13","author":"Kennedy","year":"1995","journal-title":"J. Bus. Econ. Statist"},{"key":"2026060915372225400_asag025-B24","doi-asserted-by":"crossref","first-page":"3417","DOI":"10.1214\/19-AOS1936","article-title":"Robust multivariate nonparametric tests via projection averaging","volume":"48","author":"Kim","year":"2020","journal-title":"Ann. Statist."},{"key":"2026060915372225400_asag025-B25","author":"Kim","year":"2022"},{"key":"2026060915372225400_asag025-B26","first-page":"89","article-title":"Sulla determinazione empirica di una legge di distribuzione","volume":"4","author":"Kolmogorov","year":"1933","journal-title":"Giorn Dell\u2019inst Ital Degli Att"},{"key":"2026060915372225400_asag025-B27","doi-asserted-by":"crossref","first-page":"909","DOI":"10.1093\/biomet\/86.4.909","article-title":"Two-sample quantile tests under general conditions","volume":"86","author":"Kosorok","year":"1999","journal-title":"Biometrika"},{"key":"2026060915372225400_asag025-B28","author":"Lee","year":"2025"},{"key":"2026060915372225400_asag025-B29","author":"Lee","year":"2025"},{"key":"2026060915372225400_asag025-B30","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-70578-7","volume-title":"Testing Statistical Hypotheses","author":"Lehmann","year":"2022"},{"key":"2026060915372225400_asag025-B31","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1080\/01621459.2021.1920959","article-title":"High-dimensional MANOVA via bootstrapping and its application to functional and sparse count data","volume":"118","author":"Lin","year":"2023","journal-title":"J. Am. Statist. Assoc"},{"key":"2026060915372225400_asag025-B32","doi-asserted-by":"publisher","DOI":"10.1007\/s13171-025-00377-x","article-title":"High-dimensional simultaneous inference of quantiles","author":"Lou","year":"2025","journal-title":"Sankhya A"},{"key":"2026060915372225400_asag025-B33","author":"Magnani","year":"2024"},{"key":"2026060915372225400_asag025-B34","doi-asserted-by":"crossref","first-page":"50","DOI":"10.1214\/aoms\/1177730491","article-title":"On a test of whether one of two random variables is stochastically larger than the other","volume":"18","author":"Mann","year":"1947","journal-title":"Ann. Math. Statist."},{"key":"2026060915372225400_asag025-B35","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1214\/23-AOS2338","article-title":"Adaptive novelty detection with false discovery rate guarantee","volume":"52","author":"Marandon","year":"2024","journal-title":"Ann. Statist."},{"key":"2026060915372225400_asag025-B36","doi-asserted-by":"crossref","first-page":"4926","DOI":"10.1214\/22-EJS2050","article-title":"Semi-supervised multiple testing","volume":"16","author":"Mary","year":"2022","journal-title":"Electron. J. Statist."},{"key":"2026060915372225400_asag025-B37","volume-title":"Multivariate Permutation Tests: With Applications in Biostatistics","author":"Pesarin","year":"2001"},{"key":"2026060915372225400_asag025-B38","doi-asserted-by":"crossref","first-page":"4880","DOI":"10.1080\/03610926.2013.802350","article-title":"Some elementary theory of permutation tests","volume":"44","author":"Pesarin","year":"2015","journal-title":"Commun. Statist. A"},{"key":"2026060915372225400_asag025-B39","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1080\/10485250902807407","article-title":"Finite-sample consistency of combination-based permutation tests with application to repeated measures designs","volume":"22","author":"Pesarin","year":"2010","journal-title":"J. Nonparam. Statist"},{"key":"2026060915372225400_asag025-B40","first-page":"481","article-title":"The permutation testing approach: a review","volume":"70","author":"Pesarin","year":"2010","journal-title":"Statistica"},{"key":"2026060915372225400_asag025-B41","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1007\/s11222-011-9261-0","article-title":"A review and some new results on permutation testing for multivariate problems","volume":"22","author":"Pesarin","year":"2012","journal-title":"Statist. Comp"},{"key":"2026060915372225400_asag025-B42","doi-asserted-by":"crossref","first-page":"1368","DOI":"10.1080\/03610918.2012.625338","article-title":"On the weak consistency of permutation tests","volume":"42","author":"Pesarin","year":"2013","journal-title":"Commun. Statist. B"},{"key":"2026060915372225400_asag025-B43","doi-asserted-by":"crossref","first-page":"119","DOI":"10.2307\/2984124","article-title":"Significance tests which may be applied to samples from any populations","volume":"4","author":"Pitman","year":"1937","journal-title":"Suppl. J. R. Statist. Soc"},{"key":"2026060915372225400_asag025-B44","doi-asserted-by":"crossref","first-page":"47","DOI":"10.3390\/e19020047","article-title":"On Wasserstein two-sample testing and related families of nonparametric tests","volume":"19","author":"Ramdas","year":"2017","journal-title":"Entropy"},{"key":"2026060915372225400_asag025-B45","author":"Romano","year":"2019"},{"key":"2026060915372225400_asag025-B46","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1111\/j.1467-9868.2005.00513.x","article-title":"An exact distribution-free test comparing two multivariate distributions based on adjacency","volume":"67","author":"Rosenbaum","year":"2005","journal-title":"J. R. Statist. Soc. B"},{"key":"2026060915372225400_asag025-B47","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1214\/aos\/1015362192","article-title":"Some results on false discovery rate in stepwise multiple testing procedures","volume":"30","author":"Sarkar","year":"2002","journal-title":"Ann. Statist."},{"key":"2026060915372225400_asag025-B48","first-page":"231","volume-title":"Beyond Parametrics in Interdisciplinary Research: Festschrift in Honor of Professor Pranab K. Sen","author":"Sarkar","year":"2008"},{"key":"2026060915372225400_asag025-B49","doi-asserted-by":"crossref","first-page":"742","DOI":"10.1080\/01621459.2014.929522","article-title":"Matching a distribution by matching quantiles estimation","volume":"110","author":"Sgouropoulos","year":"2015","journal-title":"J. Am. Statist. Assoc"},{"key":"2026060915372225400_asag025-B50","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1146\/annurev.ps.46.020195.003021","article-title":"Multiple hypothesis testing","volume":"46","author":"Shaffer","year":"1995","journal-title":"Ann. Rev. Psychol"},{"key":"2026060915372225400_asag025-B51","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1214\/aoms\/1177730256","article-title":"Table for estimating the goodness of fit of empirical distributions","volume":"19","author":"Smirnov","year":"1948","journal-title":"Ann. Math. Statist."},{"key":"2026060915372225400_asag025-B52","doi-asserted-by":"publisher","author":"Snow","year":"2024","DOI":"10.58020\/d1m7-ye92"},{"key":"2026060915372225400_asag025-B53","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1214\/aoms\/1177700153","article-title":"The existence of probability measures with given marginals","volume":"36","author":"Strassen","year":"1965","journal-title":"Ann. Math. Statist."},{"key":"2026060915372225400_asag025-B54","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2307\/2331554","article-title":"The probable error of a mean","volume":"6","author":"Student","year":"1908","journal-title":"Biometrika"},{"key":"2026060915372225400_asag025-B55","first-page":"1249","article-title":"Testing for equal distributions in high dimension","volume":"5","author":"Sz\u00e9kely","year":"2004","journal-title":"InterStat"},{"key":"2026060915372225400_asag025-B56","author":"U.S. Census Bureau","year":"2024"},{"key":"2026060915372225400_asag025-B58","volume-title":"Algorithmic Learning in a Random World","author":"Vovk","year":"2005"},{"key":"2026060915372225400_asag025-B57","doi-asserted-by":"crossref","first-page":"1736","DOI":"10.1214\/20-AOS2020","article-title":"E-values: calibration, combination and applications","volume":"49","author":"Vovk","year":"2021","journal-title":"Ann. Statist."},{"key":"2026060915372225400_asag025-B59","volume-title":"Mathematical Statistics","author":"Wilks","year":"1962"},{"key":"2026060915372225400_asag025-B60","doi-asserted-by":"crossref","first-page":"1059","DOI":"10.1007\/s10463-016-0576-7","article-title":"Smoothed jackknife empirical likelihood for the difference of two quantiles","volume":"69","author":"Yang","year":"2017","journal-title":"Ann. Inst. Statist. Math"}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/biomet\/advance-article-pdf\/doi\/10.1093\/biomet\/asag025\/67787782\/asag025.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/113\/2\/asag025\/67787782\/asag025.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/biomet\/article-pdf\/113\/2\/asag025\/67787782\/asag025.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T19:37:40Z","timestamp":1781033860000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/doi\/10.1093\/biomet\/asag025\/8586883"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":60,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,2,13]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asag025","relation":{},"ISSN":["0006-3444","1464-3510"],"issn-type":[{"value":"0006-3444","type":"print"},{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026]]},"published":{"date-parts":[[2026]]},"article-number":"asag025"}}