{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,12]],"date-time":"2025-11-12T20:52:49Z","timestamp":1762980769897,"version":"3.37.3"},"reference-count":29,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2019,10,19]],"date-time":"2019-10-19T00:00:00Z","timestamp":1571443200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,10,19]],"date-time":"2019-10-19T00:00:00Z","timestamp":1571443200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000761","name":"Imperial College London","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000761","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Comput Stat"],"published-print":{"date-parts":[[2020,9]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>We consider a statistical test whose<jats:italic>p<\/jats:italic>value can only be approximated using Monte Carlo simulations. We are interested in deciding whether the<jats:italic>p<\/jats:italic>value for an observed data set lies above or below a given threshold such as 5%. We want to ensure that the resampling risk, the probability of the (Monte Carlo) decision being different from the true decision, is uniformly bounded. This article introduces a simple open-ended method with this property, the confidence sequence method (CSM). We compare our approach to another algorithm, SIMCTEST, which also guarantees an (asymptotic) uniform bound on the resampling risk, as well as to other Monte Carlo procedures without a uniform bound. CSM is free of tuning parameters and conservative. It has the same theoretical guarantee as SIMCTEST and, in many settings, similar stopping boundaries. As it is much simpler than other methods, CSM is a useful method for practical applications.<\/jats:p>","DOI":"10.1007\/s00180-019-00927-6","type":"journal-article","created":{"date-parts":[[2019,10,19]],"date-time":"2019-10-19T18:24:55Z","timestamp":1571509495000},"page":"1373-1392","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["A simple method for implementing Monte Carlo tests"],"prefix":"10.1007","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1653-1768","authenticated-orcid":false,"given":"Dong","family":"Ding","sequence":"first","affiliation":[]},{"given":"Axel","family":"Gandy","sequence":"additional","affiliation":[]},{"given":"Georg","family":"Hahn","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2019,10,19]]},"reference":[{"issue":"3","key":"927_CR1","doi-asserted-by":"crossref","first-page":"271","DOI":"10.1080\/07350015.1984.10509394","volume":"2","author":"MM Ali","year":"1984","unstructured":"Ali MM (1984) Distributions of the sample autocorrelations when observations are from a stationary autoregressive-moving-average process. J Bus Econ Stat 2(3):271\u2013278","journal-title":"J Bus Econ Stat"},{"issue":"2","key":"927_CR2","first-page":"195","volume":"5","author":"MM Ali","year":"1987","unstructured":"Ali MM (1987) Durbin\u2013Watson and generalized Durbin-Watson tests for autocorrelations and randomness. J Bus Econ Stat 5(2):195\u2013203","journal-title":"J Bus Econ Stat"},{"issue":"2","key":"927_CR3","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1093\/biomet\/78.2.301","volume":"78","author":"J Besag","year":"1991","unstructured":"Besag J, Clifford P (1991) Sequential Monte Carlo $$p$$-values. Biometrika 78(2):301\u2013304","journal-title":"Biometrika"},{"key":"927_CR4","first-page":"3","volume":"8","author":"C Bonferroni","year":"1936","unstructured":"Bonferroni C (1936) Teoria statistica delle classi e calcolo delle probabilit\u00e0. Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commerciali di Firenze 8:3\u201362","journal-title":"Pubblicazioni del R Istituto Superiore di Scienze Economiche e Commerciali di Firenze"},{"key":"927_CR5","volume-title":"Time series analysis: forecasting and control","author":"GE Box","year":"2015","unstructured":"Box GE, Jenkins GM, Reinsel GC, Ljung GM (2015) Time series analysis: forecasting and control. Wiley, Hoboken"},{"issue":"1","key":"927_CR6","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1080\/07474930008800459","volume":"19","author":"R Davidson","year":"2000","unstructured":"Davidson R, MacKinnon JG (2000) Bootstrap tests: how many bootstraps? Econom Rev 19(1):55\u201368","journal-title":"Econom Rev"},{"key":"927_CR7","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511802843","volume-title":"Bootstrap methods and their application, vol 1 of Cambridge series in statistical and probabilistic mathematics","author":"AC Davison","year":"1997","unstructured":"Davison AC, Hinkley DV (1997) Bootstrap methods and their application, vol 1 of Cambridge series in statistical and probabilistic mathematics. Cambridge University Press, Cambridge"},{"issue":"1","key":"927_CR8","doi-asserted-by":"crossref","first-page":"63","DOI":"10.1198\/000313002753631385","volume":"56","author":"MP Fay","year":"2002","unstructured":"Fay MP, Follmann DA (2002) Designing Monte Carlo implementations of permutation or bootstrap hypothesis tests. Am Stat 56(1):63\u201370","journal-title":"Am Stat"},{"issue":"4","key":"927_CR9","doi-asserted-by":"crossref","first-page":"946","DOI":"10.1198\/106186007X257025","volume":"16","author":"MP Fay","year":"2007","unstructured":"Fay MP, Kim H-J, Hachey M (2007) On using truncated sequential probability ratio test boundaries for Monte Carlo implementation of hypothesis tests. J Comput Graph Stat 16(4):946\u2013967","journal-title":"J Comput Graph Stat"},{"issue":"488","key":"927_CR10","doi-asserted-by":"crossref","first-page":"1504","DOI":"10.1198\/jasa.2009.tm08368","volume":"104","author":"A Gandy","year":"2009","unstructured":"Gandy A (2009) Sequential implementation of Monte Carlo tests with uniformly bounded resampling risk. J Am Stat Assoc 104(488):1504\u20131511","journal-title":"J Am Stat Assoc"},{"issue":"4","key":"927_CR11","doi-asserted-by":"crossref","first-page":"1083","DOI":"10.1111\/sjos.12085","volume":"41","author":"A Gandy","year":"2014","unstructured":"Gandy A, Hahn G (2014) MMCTest\u2014a safe algorithm for implementing multiple Monte Carlo tests. Scand J Stat Theory Appl 41(4):1083\u20131101","journal-title":"Scand J Stat Theory Appl"},{"issue":"4","key":"927_CR12","doi-asserted-by":"crossref","first-page":"1046","DOI":"10.1111\/sjos.12228","volume":"43","author":"A Gandy","year":"2016","unstructured":"Gandy A, Hahn G (2016) A framework for Monte Carlo based multiple testing. Scand J Stat Theory Appl 43(4):1046\u20131063","journal-title":"Scand J Stat Theory Appl"},{"issue":"1","key":"927_CR13","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1214\/12-AOS1076","volume":"41","author":"A Gandy","year":"2013","unstructured":"Gandy A, Rubin-Delanchy P (2013) An algorithm to compute the power of Monte Carlo tests with guaranteed precision. Ann Stat 41(1):125\u2013142","journal-title":"Ann Stat"},{"issue":"3","key":"927_CR14","first-page":"219","volume":"11","author":"LJ Gleser","year":"1996","unstructured":"Gleser LJ (1996) Comment on Bootstrap confidence intervals by DiCiccio and Efron. Stat Sci 11(3):219\u2013221","journal-title":"Stat Sci"},{"issue":"7","key":"927_CR15","doi-asserted-by":"crossref","first-page":"1834","DOI":"10.1016\/j.jspi.2010.01.003","volume":"140","author":"H-J Kim","year":"2010","unstructured":"Kim H-J (2010) Bounding the resampling risk for sequential Monte Carlo implementation of hypothesis tests. J Stat Plan Inference 140(7):1834\u20131843","journal-title":"J Stat Plan Inference"},{"issue":"1","key":"927_CR16","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1111\/1467-985X.00186","volume":"164","author":"M Kulldorff","year":"2001","unstructured":"Kulldorff M (2001) Prospective time periodic geographical disease surveillance using a scan statistic. J R Stat Soc Ser A (Stat Soc) 164(1):61\u201372","journal-title":"J R Stat Soc Ser A (Stat Soc)"},{"issue":"2","key":"927_CR17","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1214\/aos\/1176343406","volume":"4","author":"TL Lai","year":"1976","unstructured":"Lai TL (1976) On confidence sequences. Ann Stat 4(2):265\u2013280","journal-title":"Ann Stat"},{"issue":"4","key":"927_CR18","doi-asserted-by":"crossref","first-page":"615","DOI":"10.1111\/0008-4085.00147","volume":"35","author":"JG MacKinnon","year":"2002","unstructured":"MacKinnon JG (2002) Bootstrap inference in econometrics. Can J Econ\/Revue canadienne d\u2019\u00e9conomique 35(4):615\u2013645","journal-title":"Can J Econ\/Revue canadienne d\u2019\u00e9conomique"},{"key":"927_CR19","first-page":"107","volume":"27","author":"M Massaro","year":"2003","unstructured":"Massaro M, Blair D (2003) Comparison of population numbers of yellow-eyed penguins, Megadyptes antipodes, on Stewart island and on adjacent cat-free islands. N Z J Ecol 27:107\u2013113","journal-title":"N Z J Ecol"},{"key":"927_CR20","doi-asserted-by":"crossref","first-page":"1397","DOI":"10.1214\/aoms\/1177696786","volume":"41","author":"H Robbins","year":"1970","unstructured":"Robbins H (1970) Statistical methods related to the law of the iterated logarithm. Ann Math Stat 41:1397\u20131409","journal-title":"Ann Math Stat"},{"issue":"11","key":"927_CR21","doi-asserted-by":"crossref","first-page":"1033","DOI":"10.1111\/2041-210X.12102","volume":"4","author":"GD Ruxton","year":"2013","unstructured":"Ruxton GD, Neuh\u00e4user M (2013) Improving the reporting of p-values generated by randomization methods. Methods Ecol Evol 4(11):1033\u20131036","journal-title":"Methods Ecol Evol"},{"key":"927_CR22","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1016\/j.jmva.2013.06.003","volume":"121","author":"I Silva","year":"2013","unstructured":"Silva I, Assun\u00e7\u00e3o R (2013) Optimal generalized truncated sequential Monte Carlo test. J Multivar Anal 121:33\u201349","journal-title":"J Multivar Anal"},{"issue":"2","key":"927_CR23","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1214\/16-BJPS339","volume":"32","author":"I Silva","year":"2018","unstructured":"Silva I, Assun\u00e7\u00e3o R (2018) Truncated sequential Monte Carlo test with exact power. Braz J Probab Stat 32(2):215\u2013238","journal-title":"Braz J Probab Stat"},{"issue":"2","key":"927_CR24","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1080\/07474940902816601","volume":"28","author":"I Silva","year":"2009","unstructured":"Silva I, Assun\u00e7\u00e3o R, Costa M (2009) Power of the sequential Monte Carlo test. Seq Anal Des Methods Appl 28(2):163\u2013174","journal-title":"Seq Anal Des Methods Appl"},{"key":"927_CR25","unstructured":"Sneek JM (1983) Some approximations to the exact distribution of sample autocorrelations for autoregressive moving average models. In: Time series analysis: theory and practice, vol 3"},{"issue":"1","key":"927_CR26","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1186\/1476-072X-4-11","volume":"4","author":"T Tango","year":"2005","unstructured":"Tango T, Takahashi K (2005) A flexibly shaped spatial scan statistic for detecting clusters. Int J Health Geogr 4(1):11","journal-title":"Int J Health Geogr"},{"issue":"2","key":"927_CR27","first-page":"115","volume":"2","author":"H Vinod","year":"1973","unstructured":"Vinod H (1973) Generalization of the Durbin\u2013Watson statistic for higher order autoregressive processes. Commun Stat Theory Methods 2(2):115\u2013144","journal-title":"Commun Stat Theory Methods"},{"key":"927_CR28","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1214\/aoms\/1177731118","volume":"16","author":"A Wald","year":"1945","unstructured":"Wald A (1945) Sequential tests of statistical hypotheses. Ann Math Stat 16:117\u2013186","journal-title":"Ann Math Stat"},{"key":"927_CR29","first-page":"28","volume":"34","author":"BL Welch","year":"1947","unstructured":"Welch BL (1947) The generalization of \u2018Student\u2019s\u2019 problem when several different population variances are involved. Biometrika 34:28\u201335","journal-title":"Biometrika"}],"container-title":["Computational Statistics"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s00180-019-00927-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s00180-019-00927-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s00180-019-00927-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,25]],"date-time":"2024-07-25T08:10:47Z","timestamp":1721895047000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s00180-019-00927-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,10,19]]},"references-count":29,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2020,9]]}},"alternative-id":["927"],"URL":"https:\/\/doi.org\/10.1007\/s00180-019-00927-6","relation":{},"ISSN":["0943-4062","1613-9658"],"issn-type":[{"type":"print","value":"0943-4062"},{"type":"electronic","value":"1613-9658"}],"subject":[],"published":{"date-parts":[[2019,10,19]]},"assertion":[{"value":"4 September 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 October 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 October 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}