{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T18:24:28Z","timestamp":1790879068981,"version":"4.1.0"},"reference-count":34,"publisher":"Oxford University Press (OUP)","issue":"2","license":[{"start":{"date-parts":[[2020,1,24]],"date-time":"2020-01-24T00:00:00Z","timestamp":1579824000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100012338","name":"Alan Turing Institute","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100012338","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Health Data Research UK"},{"DOI":"10.13039\/100007421","name":"Li Ka Shing Foundation","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100007421","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000265","name":"Medical Research Council","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000265","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100000266","name":"Engineering and Physical Sciences Research Council","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100000266","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,6,1]]},"abstract":"<jats:title>Summary<\/jats:title><jats:p>In Bayesian statistics, the marginal likelihood, also known as the evidence, is used to evaluate model fit as it quantifies the joint probability of the data under the prior. In contrast, non-Bayesian models are typically compared using cross-validation on held-out data, either through $k$-fold partitioning or leave-$p$-out subsampling. We show that the marginal likelihood is formally equivalent to exhaustive leave-$p$-out crossvalidation averaged over all values of $p$ and all held-out test sets when using the log posterior predictive probability as the scoring rule. Moreover, the log posterior predictive score is the only coherent scoring rule under data exchangeability. This offers new insight into the marginal likelihood and cross-validation, and highlights the potential sensitivity of the marginal likelihood to the choice of the prior. We suggest an alternative approach using cumulative cross-validation following a preparatory training phase. Our work has connections to prequential analysis and intrinsic Bayes factors, but is motivated in a different way.<\/jats:p>","DOI":"10.1093\/biomet\/asz077","type":"journal-article","created":{"date-parts":[[2019,11,25]],"date-time":"2019-11-25T20:12:32Z","timestamp":1574712752000},"page":"489-496","source":"Crossref","is-referenced-by-count":69,"title":["On the marginal likelihood and cross-validation"],"prefix":"10.1093","volume":"107","author":[{"given":"E","family":"Fong","sequence":"first","affiliation":[{"name":"Department of Statistics, University of Oxford, 24\u201329 St Giles\u2019, Oxford OX1 3LB, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"C C","family":"Holmes","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of Oxford, 24\u201329 St Giles\u2019, Oxford OX1 3LB, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2020,1,24]]},"reference":[{"key":"2020071623322932000_B1","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1214\/09-SS054","article-title":"A survey of cross-validation procedures for model selection","volume":"4","author":"Arlot,","year":"2010","journal-title":"Statist. Surv."},{"key":"2020071623322932000_B2","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1080\/01621459.1996.10476668","article-title":"The intrinsic Bayes factor for model selection and prediction","volume":"91","author":"Berger,","year":"1996","journal-title":"J. Am. Statist. Assoc."},{"key":"2020071623322932000_B3","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1214\/lnms\/1215540968","article-title":"Objective Bayesian methods for model selection: Introduction and comparison","volume-title":"Model selection","author":"Berger,","year":"2001"},{"key":"2020071623322932000_B4","volume-title":"Bayesian Theory.","author":"Bernardo,","year":"2009"},{"key":"2020071623322932000_B5","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1214\/07-BA217","article-title":"Importance re-sampling MCMC for cross-validation in inverse problems","volume":"2","author":"Bhattacharya,","year":"2007","journal-title":"Bayesian Anal."},{"key":"2020071623322932000_B6","doi-asserted-by":"crossref","first-page":"1103","DOI":"10.1111\/rssb.12158","article-title":"A general framework for updating belief distributions","volume":"78","author":"Bissiri,","year":"2016","journal-title":"J. R. Statist. Soc. B"},{"key":"2020071623322932000_B7","doi-asserted-by":"crossref","first-page":"47","DOI":"10.1002\/cjs.10045","article-title":"An efficient computational approach for prior sensitivity analysis and cross-validation","volume":"38","author":"Bornn,","year":"2010","journal-title":"Can. J. Statist."},{"key":"2020071623322932000_B8","doi-asserted-by":"crossref","first-page":"278","DOI":"10.2307\/2981683","article-title":"Present position and potential developments: Some personal views: Statistical theory: The prequential approach","volume":"147","author":"Dawid,","year":"1984","journal-title":"J. R. Statist. Soc. A"},{"key":"2020071623322932000_B9","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1093\/oso\/9780198522669.003.0007","article-title":"Prequential analysis, stochastic complexity and Bayesian inference","volume-title":"Bayesian Statistics 4: Proceedings of the Fourth Valencia International Meeting.","author":"Dawid,","year":"1992"},{"key":"2020071623322932000_B10","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1007\/s40300-014-0039-y","article-title":"Theory and applications of proper scoring rules","volume":"72","author":"Dawid,","year":"2014","journal-title":"METRON"},{"key":"2020071623322932000_B11","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1214\/15-BA942","article-title":"Bayesian model selection based on proper scoring rules","volume":"10","author":"Dawid,","year":"2015","journal-title":"Bayesian Anal."},{"key":"2020071623322932000_B12","doi-asserted-by":"crossref","first-page":"320","DOI":"10.1080\/01621459.1975.10479865","article-title":"The predictive sample reuse method with applications","volume":"70","author":"Geisser,","year":"1975","journal-title":"J. Am. Statist. Assoc."},{"key":"2020071623322932000_B13","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1080\/01621459.1979.10481632","article-title":"A predictive approach to model selection","volume":"74","author":"Geisser,","year":"1979","journal-title":"J. Am. Statist. Assoc."},{"key":"2020071623322932000_B14","doi-asserted-by":"crossref","first-page":"997","DOI":"10.1007\/s11222-013-9416-2","article-title":"Understanding predictive information criteria for Bayesian models","volume":"24","author":"Gelman,","year":"2014","journal-title":"Statist. Comp."},{"key":"2020071623322932000_B15","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1198\/016214506000001437","article-title":"Strictly proper scoring rules, prediction, and estimation","volume":"102","author":"Gneiting,","year":"2007","journal-title":"J. Am. Statist. Assoc."},{"key":"2020071623322932000_B16","doi-asserted-by":"crossref","first-page":"1069","DOI":"10.1214\/17-BA1085","article-title":"Inconsistency of Bayesian inference for misspecified linear models, and a proposal for repairing it","volume":"12","author":"Gr\u00fcnwald,","year":"2017","journal-title":"Bayesian Anal."},{"key":"2020071623322932000_B17","first-page":"497","article-title":"Assigning a value to a power likelihood in a general Bayesian model","volume":"104","author":"Holmes,","year":"2017","journal-title":"Biometrika"},{"key":"2020071623322932000_B18","doi-asserted-by":"crossref","first-page":"773","DOI":"10.1080\/01621459.1995.10476572","article-title":"Bayes factors","volume":"90","author":"Kass,","year":"1995","journal-title":"J. Am. Statist. Assoc."},{"key":"2020071623322932000_B19","doi-asserted-by":"crossref","first-page":"343","DOI":"10.1093\/oso\/9780198504856.003.0015","article-title":"Bayesian model choice: What and why? (with Discussion)","volume-title":"Bayesian Statistics 6 (Proceedings of the Sixth Valencia International Meeting).","author":"Key,","year":"1999"},{"key":"2020071623322932000_B20","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1093\/biomet\/44.1-2.187","article-title":"A statistical paradox","volume":"44","author":"Lindley,","year":"1957","journal-title":"Biometrika"},{"key":"2020071623322932000_B21","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1093\/biomet\/asz006","article-title":"General Bayesian updating and the loss-likelihood bootstrap","volume":"106","author":"Lyddon,","year":"2019","journal-title":"Biometrika"},{"key":"2020071623322932000_B22","volume-title":"Kendall\u2019s Advanced Theory of Statistics, Volume 2B: Bayesian Inference.","author":"O\u2019Hagan,","year":"2004"},{"key":"2020071623322932000_B23","volume-title":"The Bayesian Choice: From Decision-Theoretic Foundations to Computational Implementation.","author":"Robert,","year":"2007","edition":"2nd ed"},{"key":"2020071623322932000_B24","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1086\/675729","article-title":"On the Jeffreys-Lindley paradox","volume":"81","author":"Robert,","year":"2014","journal-title":"Phil. Sci."},{"key":"2020071623322932000_B25","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1063\/1.3275622","article-title":"Computational methods for Bayesian model choice","volume":"1193","author":"Robert,","year":"2009","journal-title":"AIP Conf. Proc."},{"key":"2020071623322932000_B26","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1214\/aos\/1176344136","article-title":"Estimating the dimension of a model","volume":"6","author":"Schwarz,","year":"1978","journal-title":"Ann. Statist."},{"key":"2020071623322932000_B27","doi-asserted-by":"crossref","first-page":"486","DOI":"10.1080\/01621459.1993.10476299","article-title":"Linear model selection by cross-validation","volume":"88","author":"Shao,","year":"1993","journal-title":"J. Am. Statist. Assoc."},{"key":"2020071623322932000_B28","doi-asserted-by":"crossref","first-page":"1826","DOI":"10.1080\/01621459.2018.1518237","article-title":"Bayesian model comparison with the Hyv\u00e4rinen score: Computation and consistency","volume":"114","author":"Shao,","year":"2019","journal-title":"J. Am. Statist. Assoc."},{"key":"2020071623322932000_B29","doi-asserted-by":"crossref","first-page":"583","DOI":"10.1111\/1467-9868.00353","article-title":"Bayesian measures of model complexity and fit","volume":"64","author":"Spiegelhalter,","year":"2002","journal-title":"J. R. Statist. Soc. B"},{"key":"2020071623322932000_B30","doi-asserted-by":"crossref","first-page":"1413","DOI":"10.1007\/s11222-016-9696-4","article-title":"Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC","volume":"27","author":"Vehtari,","year":"2017","journal-title":"Statist. Comp."},{"key":"2020071623322932000_B31","doi-asserted-by":"crossref","first-page":"2339","DOI":"10.1162\/08997660260293292","article-title":"Bayesian model assessment and comparison using cross-validation predictive densities","volume":"14","author":"Vehtari,","year":"2002","journal-title":"Neural Comp."},{"key":"2020071623322932000_B32","doi-asserted-by":"crossref","first-page":"142","DOI":"10.1214\/12-SS102","article-title":"A survey of Bayesian predictive methods for model assessment, selection and comparison","volume":"6","author":"Vehtari,","year":"2012","journal-title":"Statist. Surveys."},{"key":"2020071623322932000_B33","first-page":"3571","article-title":"Asymptotic equivalence of Bayes cross validation and widely applicable information criterion in singular learning theory","volume":"11","author":"Watanabe,","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"2020071623322932000_B34","doi-asserted-by":"crossref","first-page":"465","DOI":"10.1214\/16-STS592","article-title":"Approximate models and robust decisions","volume":"31","author":"Watson,","year":"2016","journal-title":"Statist. Sci."}],"container-title":["Biometrika"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/academic.oup.com\/biomet\/article-pdf\/107\/2\/489\/33217997\/asz077.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"http:\/\/academic.oup.com\/biomet\/article-pdf\/107\/2\/489\/33217997\/asz077.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,7,27]],"date-time":"2024-07-27T18:25:08Z","timestamp":1722104708000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/biomet\/article\/107\/2\/489\/5715611"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,24]]},"references-count":34,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2020,1,24]]},"published-print":{"date-parts":[[2020,6,1]]}},"URL":"https:\/\/doi.org\/10.1093\/biomet\/asz077","relation":{},"ISSN":["0006-3444","1464-3510"],"issn-type":[{"value":"0006-3444","type":"print"},{"value":"1464-3510","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2020,6]]},"published":{"date-parts":[[2020,1,24]]}}}