{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,15]],"date-time":"2026-10-15T12:14:15Z","timestamp":1792066455091,"version":"4.4.0"},"reference-count":30,"publisher":"Oxford University Press (OUP)","issue":"4","license":[{"start":{"date-parts":[[2025,4,4]],"date-time":"2025-04-04T00:00:00Z","timestamp":1743724800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["281474342\/GRK2224\/2"],"award-info":[{"award-number":["281474342\/GRK2224\/2"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000001","name":"NSF","doi-asserted-by":"publisher","award":["DMS-2310718"],"award-info":[{"award-number":["DMS-2310718"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,9,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>In a Monte Carlo test, the observed dataset is fixed, and several resampled or permuted versions of the dataset are generated in order to test a null hypothesis that the original dataset is exchangeable with the resampled\/permuted ones. Sequential Monte Carlo tests aim to save computational resources by generating these additional datasets sequentially one by one and potentially stopping early. While earlier tests yield valid inference at a particular prespecified stopping rule, our work develops a new anytime-valid Monte Carlo test that can be continuously monitored, yielding a p-value or e-value at any stopping time possibly not specified in advance. It generalizes the well-known method by Besag and Clifford, allowing it to stop at any time, but also encompasses new sequential Monte Carlo tests that tend to stop sooner under the null and alternative without compromising power. The core technical advance is the development of new test martingales for testing exchangeability against a very particular alternative based on a testing by betting technique. The proposed betting strategies are guided by the derivation of a simple log-optimal betting strategy, have closed-form expressions for the wealth process, provable guarantees on resampling risk, and display excellent power in practice.<\/jats:p>","DOI":"10.1093\/jrsssb\/qkaf014","type":"journal-article","created":{"date-parts":[[2025,4,4]],"date-time":"2025-04-04T14:53:49Z","timestamp":1743778429000},"page":"1200-1220","source":"Crossref","is-referenced-by-count":4,"title":["Sequential Monte Carlo testing by betting"],"prefix":"10.1093","volume":"87","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3380-0066","authenticated-orcid":false,"given":"Lasse","family":"Fischer","sequence":"first","affiliation":[{"name":"Competence Center for Clinical Trials Bremen, University of Bremen , Linzer str. 4, Bremen 28359 ,","place":["Germany"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0497-311X","authenticated-orcid":false,"given":"Aaditya","family":"Ramdas","sequence":"additional","affiliation":[{"name":"Departments of Statistics & Data Science, and Machine Learning, Carnegie Mellon University , 5000 Forbes Ave, Pittsburgh, PA 15213 ,","place":["USA"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2025,4,4]]},"reference":[{"issue":"1","key":"2025091915475251000_qkaf014-B1","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1111\/rssb.12340","article-title":"The conditional permutation test for independence while controlling for confounders","volume":"82","author":"Berrett","year":"2020","journal-title":"Journal of the Royal Statistical Society: Series B, Statistical Methodology"},{"issue":"2","key":"2025091915475251000_qkaf014-B2","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1093\/biomet\/78.2.301","article-title":"Sequential Monte Carlo p-values","volume":"78","author":"Besag","year":"1991","journal-title":"Biometrika"},{"key":"2025091915475251000_qkaf014-B3","first-page":"65","volume-title":"Fourth Berkeley symposium on mathematical statistics and probability","author":"Breiman","year":"1961"},{"issue":"3","key":"2025091915475251000_qkaf014-B4","doi-asserted-by":"publisher","first-page":"551","DOI":"10.1111\/rssb.12265","article-title":"Panning for gold: \u2018Model-X\u2019 knockoffs for high dimensional controlled variable selection","volume":"80","author":"Cand\u00e8s","year":"2018","journal-title":"Journal of the Royal Statistical Society: Series B, Statistical Methodology"},{"issue":"3","key":"2025091915475251000_qkaf014-B5","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1214\/16-sts571","article-title":"A paradox from randomization-based causal inference","volume":"32","author":"Ding","year":"2017","journal-title":"Statistical Science"},{"issue":"1","key":"2025091915475251000_qkaf014-B6","doi-asserted-by":"publisher","first-page":"63","DOI":"10.1198\/000313002753631385","article-title":"Designing Monte Carlo implementations of permutation or bootstrap hypothesis tests","volume":"56","author":"Fay","year":"2002","journal-title":"The American Statistician"},{"issue":"4","key":"2025091915475251000_qkaf014-B7","doi-asserted-by":"publisher","first-page":"946","DOI":"10.1198\/106186007X257025","article-title":"On using truncated sequential probability ratio test boundaries for Monte Carlo implementation of hypothesis tests","volume":"16","author":"Fay","year":"2007","journal-title":"Journal of Computational and Graphical Statistics"},{"issue":"488","key":"2025091915475251000_qkaf014-B8","doi-asserted-by":"publisher","first-page":"1504","DOI":"10.1198\/jasa.2009.tm08368","article-title":"Sequential implementation of Monte Carlo tests with uniformly bounded resampling risk","volume":"104","author":"Gandy","year":"2009","journal-title":"Journal of the American Statistical Association"},{"issue":"5","key":"2025091915475251000_qkaf014-B9","doi-asserted-by":"publisher","first-page":"1091","DOI":"10.1093\/jrsssb\/qkae011","article-title":"Safe testing","volume":"86","author":"Gr\u00fcnwald","year":"2024","journal-title":"Journal of the Royal Statistical Society Series B: Statistical Methodology (with Discussion)"},{"issue":"546","key":"2025091915475251000_qkaf014-B10","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1080\/01621459.2023.2205607","article-title":"Anytime-valid tests of conditional independence under model-X","volume":"119","author":"Gr\u00fcnwald","year":"2023","journal-title":"Journal of the American Statistical Association"},{"issue":"2","key":"2025091915475251000_qkaf014-B11","doi-asserted-by":"publisher","first-page":"1055","DOI":"10.1214\/20-aos1991","article-title":"Time-uniform, nonparametric, nonasymptotic confidence sequences","volume":"49","author":"Howard","year":"2021","journal-title":"Annals of Statistics"},{"issue":"3","key":"2025091915475251000_qkaf014-B12","doi-asserted-by":"publisher","first-page":"1806","DOI":"10.1287\/opre.2021.2135","article-title":"Always valid inference: Continuous monitoring of A\/B tests","volume":"70","author":"Johari","year":"2022","journal-title":"Operations Research"},{"issue":"4","key":"2025091915475251000_qkaf014-B13","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1002\/bltj.1956.35.issue-4","article-title":"A new interpretation of information rate","volume":"35","author":"Kelly","year":"1956","journal-title":"The Bell System Technical Journal"},{"issue":"2","key":"2025091915475251000_qkaf014-B14","doi-asserted-by":"publisher","first-page":"1263","DOI":"10.1214\/12-AOS1008","article-title":"Rerandomization to improve covariate balance in experiments","volume":"40","author":"Morgan","year":"2012","journal-title":"Annals of Statistics"},{"issue":"512","key":"2025091915475251000_qkaf014-B15","doi-asserted-by":"publisher","first-page":"1412","DOI":"10.1080\/01621459.2015.1079528","article-title":"Rerandomization to balance tiers of covariates","volume":"110","author":"Morgan","year":"2015","journal-title":"Journal of the American Statistical Association"},{"issue":"4","key":"2025091915475251000_qkaf014-B16","doi-asserted-by":"publisher","first-page":"576","DOI":"10.1214\/23-STS894","article-title":"Game-theoretic statistics and safe anytime-valid inference","volume":"38","author":"Ramdas","year":"2023","journal-title":"Statistical Science"},{"key":"2025091915475251000_qkaf014-B17","article-title":"Randomized and exchangeable improvements of Markov\u2019s, Chebyshev\u2019s and Chernoff\u2019s inequalities","author":"Ramdas","year":"2024","journal-title":"Statistical Science (to appear)"},{"issue":"1","key":"2025091915475251000_qkaf014-B18","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1016\/j.ijar.2021.06.017","article-title":"Testing exchangeability: Fork-convexity, supermartingales and e-processes","volume":"141","author":"Ramdas","year":"2022","journal-title":"International Journal of Approximate Reasoning"},{"key":"2025091915475251000_qkaf014-B19","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-3692-2","volume-title":"Observational studies","author":"Rosenbaum","year":"2002"},{"issue":"371","key":"2025091915475251000_qkaf014-B20","doi-asserted-by":"publisher","first-page":"591","DOI":"10.2307\/2287653","article-title":"Comment on \u2018Randomization analysis of experimental data: The Fisher randomization test\u2019 by D. Basu","volume":"75","author":"Rubin","year":"1980","journal-title":"Journal of the American Statistical Association"},{"key":"2025091915475251000_qkaf014-B21","first-page":"4915","volume-title":"International conference on artificiall intelligence and statistics","author":"Saha","year":"2024"},{"issue":"2","key":"2025091915475251000_qkaf014-B22","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1111\/rssa.12647","article-title":"Testing by betting: A strategy for statistical and scientific communication","volume":"184","author":"Shafer","year":"2021","journal-title":"Journal of the Royal Statistical Society Series A: Statistics in Society (with Discussion)"},{"issue":"1","key":"2025091915475251000_qkaf014-B23","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1214\/10-sts347","article-title":"Test martingales, Bayes factors and p-values","volume":"26","author":"Shafer","year":"2011","journal-title":"Statistical Science"},{"issue":"2","key":"2025091915475251000_qkaf014-B24","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1080\/07474940902816601","article-title":"Power of the sequential Monte Carlo test","volume":"28","author":"Silva","year":"2009","journal-title":"Sequential Analysis"},{"key":"2025091915475251000_qkaf014-B25","doi-asserted-by":"publisher","first-page":"33","DOI":"10.1016\/j.jmva.2013.06.003","article-title":"Optimal generalized truncated sequential Monte Carlo test","volume":"121","author":"Silva","year":"2013","journal-title":"Journal of Multivariate Analysis"},{"issue":"4","key":"2025091915475251000_qkaf014-B26","doi-asserted-by":"publisher","first-page":"595","DOI":"10.1214\/20-STS817","article-title":"Testing randomness online","volume":"36","author":"Vovk","year":"2021","journal-title":"Statistical Science"},{"key":"2025091915475251000_qkaf014-B27","volume-title":"Algorithmic learning in a random world","author":"Vovk","year":"2005"},{"key":"2025091915475251000_qkaf014-B28","first-page":"768","volume-title":"Proceedings of the twentieth international conference on international conference on machine learning","author":"Vovk","year":"2003"},{"key":"2025091915475251000_qkaf014-B29","first-page":"20204","volume-title":"Advances in neural information processing systems","author":"Waudby-Smith","year":"2020"},{"issue":"1","key":"2025091915475251000_qkaf014-B30","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1093\/jrsssb\/qkad009","article-title":"Estimating means of bounded random variables by betting","volume":"86","author":"Waudby-Smith","year":"2023","journal-title":"Journal of the Royal Statistical Society Series B: Statistical Methodology (with Discussion)"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/87\/4\/1200\/62862344\/qkaf014.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/87\/4\/1200\/62862344\/qkaf014.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,19]],"date-time":"2025-09-19T19:47:59Z","timestamp":1758311279000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/87\/4\/1200\/8106328"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,4]]},"references-count":30,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,4,4]]},"published-print":{"date-parts":[[2025,9,15]]}},"URL":"https:\/\/doi.org\/10.1093\/jrsssb\/qkaf014","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2025,9]]},"published":{"date-parts":[[2025,4,4]]}}}