{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T01:19:00Z","timestamp":1787015940035,"version":"build-2736575974"},"reference-count":57,"publisher":"Oxford University Press (OUP)","issue":"3","license":[{"start":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T00:00:00Z","timestamp":1677196800000},"content-version":"vor","delay-in-days":380,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001711","name":"Swiss National Science Foundation","doi-asserted-by":"publisher","award":["189295"],"award-info":[{"award-number":["189295"]}],"id":[{"id":"10.13039\/501100001711","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2022,7,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Replication studies are increasingly conducted but there is no established statistical criterion for replication success. We propose a novel approach combining reverse-Bayes analysis with Bayesian hypothesis testing: a sceptical prior is determined for the effect size such that the original finding is no longer convincing in terms of a Bayes factor. This prior is then contrasted to an advocacy prior (the reference posterior of the effect size based on the original study), and replication success is declared if the replication data favour the advocacy over the sceptical prior at a higher level than the original data favoured the sceptical prior over the null hypothesis. The sceptical Bayes factor is the highest level where replication success can be declared. A comparison to existing methods reveals that the sceptical Bayes factor combines several notions of replicability: it ensures that both studies show sufficient evidence against the null and penalises incompatibility of their effect estimates. Analysis of asymptotic properties and error rates, as well as case studies from the Social Sciences Replication Project show the advantages of the method for the assessment of replicability.<\/jats:p>","DOI":"10.1111\/rssb.12491","type":"journal-article","created":{"date-parts":[[2022,2,9]],"date-time":"2022-02-09T03:25:28Z","timestamp":1644377128000},"page":"879-911","source":"Crossref","is-referenced-by-count":28,"title":["The Sceptical Bayes Factor for the Assessment of Replication Success"],"prefix":"10.1093","volume":"84","author":[{"given":"Samuel","family":"Pawel","sequence":"first","affiliation":[{"name":"Epidemiology, Biostatistics and Prevention Institute, Center for Reproducible Science, University of Zurich , Zurich , Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leonhard","family":"Held","sequence":"additional","affiliation":[{"name":"Epidemiology, Biostatistics and Prevention Institute, Center for Reproducible Science, University of Zurich , Zurich , Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2022,2,9]]},"reference":[{"key":"2023022408353647300_","volume-title":"lamW: Lambert-W Function","author":"Adler","year":"2015"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"e0175302","DOI":"10.1371\/journal.pone.0175302","article-title":"Bayesian evaluation of effect size after replicating an original study","volume":"12","author":"van Aert","year":"2017","journal-title":"PloS One"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"579","DOI":"10.1126\/science.1211180","article-title":"Affirmative action policies promote women and do not harm efficiency in the laboratory","volume":"335","author":"Balafoutas","year":"2012","journal-title":"Science"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1016\/S0378-3758(01)00223-3","article-title":"Bayesian analysis and design for comparison of effect-sizes","volume":"103","author":"Bayarri","year":"2002","journal-title":"Journal of Statistical Planning and Inference"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1198\/000313002155","article-title":"Bayesian design of \u2018successful\u2019 replications","volume":"56","author":"Bayarri","year":"2002","journal-title":"The American Statistician"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"1550","DOI":"10.1214\/12-AOS1013","article-title":"Criteria for Bayesian model choice with application to variable selection","volume":"40","author":"Bayarri","year":"2012","journal-title":"The Annals of Statistics"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/S0378-3758(00)00235-4","article-title":"Discussion of \u2018Why should clinicians care about Bayesian methods?\u2019","volume":"94","author":"Berger","year":"2001","journal-title":"Journal of Statistical Planning and Inference"},{"key":"2023022408353647300_","volume-title":"Bayesian theory","author":"Bernardo","year":"2000"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"383","DOI":"10.2307\/2982063","article-title":"Sampling and Bayes' inference in scientific modelling and robustness (with discussion)","volume":"143","author":"Box","year":"1980","journal-title":"Journal of the Royal Statistical Society, Series A"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"1433","DOI":"10.1126\/science.aaf0918","article-title":"Evaluating replicability of laboratory experiments in economics","volume":"351","author":"Camerer","year":"2016","journal-title":"Science"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1038\/s41562-018-0399-z","article-title":"Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015","volume":"2","author":"Camerer","year":"2018","journal-title":"Nature Human Behavior"},{"key":"2023022408353647300_","volume-title":"Sufficiently skeptical intrinsic priors for the analysis of replication studies","author":"Consonni","year":"2019"},{"key":"2023022408353647300_","volume-title":"The sceptic and the advocate: comparing two opinions on the mean of a normal distribution","author":"Consonni","year":"2021"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","DOI":"10.7758\/9781610448864","volume-title":"The handbook of research synthesis and meta-analysis","author":"Cooper","year":"2019"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1007\/BF02124750","article-title":"On the Lambert W function","volume":"5","author":"Corless","year":"1996","journal-title":"Advances in Computational Mathematics"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1007\/s13164-018-0400-9","article-title":"Estimating the reproducibility of experimental philosophy","volume":"12","author":"Cova","year":"2018","journal-title":"Review of Philosophy and Psychology"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1080\/01621459.1982.10477856","article-title":"The well-calibrated Bayesian","volume":"77","author":"Dawid","year":"1982","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1214\/11-STS369","article-title":"Joint specification of model space and parameter space prior distributions","volume":"27","author":"Dellaportas","year":"2012","journal-title":"Statistical Science"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1038\/nature12774","article-title":"Experimental evidence for the influence of group size on cultural complexity","volume":"503","author":"Derex","year":"2013","journal-title":"Nature"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1037\/h0044139","article-title":"Bayesian statistical inference for psychological research","volume":"70","author":"Edwards","year":"1963","journal-title":"Psychological Review"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"e04333","DOI":"10.7554\/eLife.04333","article-title":"An open investigation of the reproducibility of cancer biology research","volume":"3","author":"Errington","year":"2014","journal-title":"eLife"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"e0149794","DOI":"10.1371\/journal.pone.0149794","article-title":"A Bayesian perspective on the reproducibility project: psychology","volume":"11","author":"Etz","year":"2016","journal-title":"PloS One"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"893","DOI":"10.1214\/06-BA129","article-title":"Checking for prior-data conflict","volume":"1","author":"Evans","year":"2006","journal-title":"Bayesian Analysis"},{"key":"2023022408353647300_","first-page":"3","article-title":"On the probable error of a coefficient of correlation deduced from a small sample","volume":"1","author":"Fisher","year":"1921","journal-title":"Metron"},{"key":"2023022408353647300_","volume-title":"Probability and the weighing of evidence","author":"Good","year":"1950"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"96","DOI":"10.1002\/pst.1736","article-title":"Idle thoughts of a \u2018well-calibrated\u2019 Bayesian in clinical drug development","volume":"15","author":"Grieve","year":"2016","journal-title":"Pharmaceutical Statistics"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1080\/00031305.2018.1518787","article-title":"A Bayes factor for replications of ANOVA results","volume":"73","author":"Harms","year":"2019","journal-title":"The American Statistician"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"543","DOI":"10.3102\/1076998619852953","article-title":"More than one replication study is needed for unambiguous tests of replication","volume":"44","author":"Hedges","year":"2019","journal-title":"Journal of Educational and Behavioral Statistics"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"431","DOI":"10.1111\/rssa.12493","article-title":"A new standard for the analysis and design of replication studies (with discussion)","volume":"183","author":"Held","year":"2020","journal-title":"Journal of the Royal Statistical Society: Series A (Statistics in Society)"},{"key":"2023022408353647300_","volume-title":"Reverse-Bayes methods for evidence assessment and research synthesis. Research Synthesis Methods","author":"Held","year":"2022"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","DOI":"10.1214\/21-AOAS1502","volume-title":"The assessment of replication success based on relative effect size","author":"Held","year":"2022"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1126\/science.1183532","article-title":"Lab experiments for the study of social-ecological systems","volume":"328","author":"Janssen","year":"2010","journal-title":"Science"},{"key":"2023022408353647300_","volume-title":"Theory of probability","author":"Jeffreys","year":"1961"},{"key":"2023022408353647300_","volume-title":"Continuous univariate distributions","author":"Johnson","year":"1995"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/01621459.2016.1240079","article-title":"On the reproducibility of psychological science","volume":"112","author":"Johnson","year":"2017","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1080\/01621459.1995.10476572","article-title":"Bayes factors","volume":"90","author":"Kass","year":"1995","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","DOI":"10.1002\/9781118470961","volume-title":"Statistical thinking for non-statisticians in drug regulation","author":"Kay","year":"2014"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1027\/1864-9335\/a000178","article-title":"Investigating variation in replicability","volume":"45","author":"Klein","year":"2014","journal-title":"Social Psychology"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"1830","DOI":"10.1126\/science.1190792","article-title":"The social sense: susceptibility to others' beliefs in human infants and adults","volume":"330","author":"Kovacs","year":"2010","journal-title":"Science"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"410","DOI":"10.1198\/016214507000001337","article-title":"Mixtures of g priors for Bayesian variable selection","volume":"103","author":"Liang","year":"2008","journal-title":"Journal of the American Statistical Association"},{"key":"2023022408353647300_","author":"Ly","year":"2021"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"2498","DOI":"10.3758\/s13428-018-1092-x","article-title":"Replication Bayes factors from evidence updating","volume":"51","author":"Ly","year":"2018","journal-title":"Behavior Research Methods"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"1223","DOI":"10.1002\/sim.4780071203","article-title":"Bayesian analysis of case-control studies","volume":"7","author":"Marshall","year":"1988","journal-title":"Statistics in Medicine"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1111\/rssa.12572","article-title":"New statistical metrics for multisite replication projects","volume":"183","author":"Mathur","year":"2020","journal-title":"Journal of the Royal Statistical Society: Series A (Statistics in Society)"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/S0378-3758(00)00232-9","article-title":"Why should clinicians care about Bayesian methods","volume":"94","author":"Matthews","year":"2001","journal-title":"Journal of Statistical Planning and Inference"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","DOI":"10.1214\/21-STS828","volume-title":"Power calculations for replication studies","author":"Micheloud","year":"2022"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"aac4716","DOI":"10.1126\/science.aac4716","article-title":"Estimating the reproducibility of psychological science","volume":"349","author":"Open Science Collaboration","year":"2015","journal-title":"Science"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1177\/1745691616646366","article-title":"What should researchers expect when they replicate studies? A statistical view of replicability in psychological science","volume":"11","author":"Patil","year":"2016","journal-title":"Perspectives on Psychological Science"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"e0231416","DOI":"10.1371\/journal.pone.0231416","article-title":"Probabilistic forecasting of replication studies","volume":"15","author":"Pawel","year":"2020","journal-title":"PloS One"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1111\/rssa.12544","article-title":"Discussion on the meeting on \u2018Signs and sizes: understanding and replicating statistical findings\u2019","volume":"183","author":"Pericchi","year":"2020","journal-title":"Journal of the Royal Statistical Society: Series A (Statistics in Society)"},{"key":"2023022408353647300_","volume-title":"R: A Language and Environment for Statistical Computing","author":"R Core Team","year":"2021"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1177\/0956797614567341","article-title":"Small telescopes: detectability and the evaluation of replication results","volume":"26","author":"Simonsohn","year":"2015","journal-title":"Psychological Science"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/0197-2456(86)90003-6","article-title":"Monitoring clinical trials: conditional or predictive power?","volume":"7","author":"Spiegelhalter","year":"1986","journal-title":"Controlled Clinical Trials"},{"key":"2023022408353647300_","volume-title":"Bayesian approaches to clinical trials and health-care evaluation","author":"Spiegelhalter","year":"2004"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","first-page":"1457","DOI":"10.1037\/a0036731","article-title":"Bayesian tests to quantify the result of a replication attempt","volume":"143","author":"Verhagen","year":"2014","journal-title":"Journal of Experimental Psychology: General"},{"key":"2023022408353647300_","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-24277-4","volume-title":"ggplot2: elegant graphics for data analysis","author":"Wickham","year":"2016"},{"key":"2023022408353647300_","first-page":"233","volume-title":"Bayesian inference and decision techniques: essays in Honor of Bruno de Finetti","author":"Zellner","year":"1986"}],"container-title":["Journal of the Royal Statistical Society Series B: Statistical Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/rssb.12491","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/rssb.12491","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/84\/3\/879\/49321988\/jrsssb_84_3_879.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/jrsssb\/article-pdf\/84\/3\/879\/49321988\/jrsssb_84_3_879.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,11,17]],"date-time":"2023-11-17T01:20:40Z","timestamp":1700184040000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/jrsssb\/article\/84\/3\/879\/7056121"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,2,9]]},"references-count":57,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,2,9]]},"published-print":{"date-parts":[[2022,7,1]]}},"URL":"https:\/\/doi.org\/10.1111\/rssb.12491","relation":{},"ISSN":["1369-7412","1467-9868"],"issn-type":[{"value":"1369-7412","type":"print"},{"value":"1467-9868","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2022,7]]},"published":{"date-parts":[[2022,2,9]]}}}