{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T16:13:11Z","timestamp":1779898391000,"version":"3.53.1"},"reference-count":52,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2018,6,6]],"date-time":"2018-06-06T00:00:00Z","timestamp":1528243200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>When it is acknowledged that all candidate parameterised statistical models are misspecified relative to the data generating process, the decision maker (DM) must currently concern themselves with inference for the parameter value minimising the Kullback\u2013Leibler (KL)-divergence between the model and this process (Walker, 2013). However, it has long been known that minimising the KL-divergence places a large weight on correctly capturing the tails of the sample distribution. As a result, the DM is required to worry about the robustness of their model to tail misspecifications if they want to conduct principled inference. In this paper we alleviate these concerns for the DM. We advance recent methodological developments in general Bayesian updating (Bissiri, Holmes &amp; Walker, 2016) to propose a statistically well principled Bayesian updating of beliefs targeting the minimisation of more general divergence criteria. We improve both the motivation and the statistical foundations of existing Bayesian minimum divergence estimation (Hooker &amp; Vidyashankar, 2014; Ghosh &amp; Basu, 2016), allowing the well principled Bayesian to target predictions from the model that are close to the genuine model in terms of some alternative divergence measure to the KL-divergence. Our principled formulation allows us to consider a broader range of divergences than have previously been considered. In fact, we argue defining the divergence measure forms an important, subjective part of any statistical analysis, and aim to provide some decision theoretic rational for this selection. We illustrate how targeting alternative divergence measures can impact the conclusions of simple inference tasks, and discuss then how our methods might apply to more complicated, high dimensional models.<\/jats:p>","DOI":"10.3390\/e20060442","type":"journal-article","created":{"date-parts":[[2018,6,6]],"date-time":"2018-06-06T07:38:15Z","timestamp":1528270695000},"page":"442","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":47,"title":["Principles of Bayesian Inference Using General Divergence Criteria"],"prefix":"10.3390","volume":"20","author":[{"given":"Jack","family":"Jewson","sequence":"first","affiliation":[{"name":"Department of Statistics, University of Warwick, Coventry CV4 7AL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jim Q.","family":"Smith","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of Warwick, Coventry CV4 7AL, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Chris","family":"Holmes","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of Oxford, Oxford OX1 3LB, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2018,6,6]]},"reference":[{"key":"ref_1","unstructured":"Bernardo, J.M., and Smith, A.F. (2001). Bayesian Theory, Wiley."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1621","DOI":"10.1016\/j.jspi.2013.05.013","article-title":"Bayesian inference with misspecified models","volume":"143","author":"Walker","year":"2013","journal-title":"J. Statist. Plan. Inference"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1103","DOI":"10.1111\/rssb.12158","article-title":"A general framework for updating belief distributions","volume":"78","author":"Bissiri","year":"2016","journal-title":"J. R. Statist. Soc. Ser. B (Statist. Methodol.)"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Box, G.E. (1980). Sampling and Bayes\u2019 inference in scientific modelling and robustness. J. R. Statist. Soc. Ser. A (Gen.), 383\u2013430.","DOI":"10.2307\/2982063"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/BF02562676","article-title":"An overview of robust Bayesian analysis","volume":"3","author":"Berger","year":"1994","journal-title":"Test"},{"key":"ref_6","first-page":"465","article-title":"Approximate models and robust decisions","volume":"31","author":"Watson","year":"2016","journal-title":"Statist. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Huber, P.J., and Ronchetti, E. (1981). Robust Statistics, Series in Probability and Mathematical Statistics, John Wiley & Sons.","DOI":"10.1002\/0471725250"},{"key":"ref_8","unstructured":"Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., and Stahel, W.A. (2011). Robust Statistics: The Approach Based on Influence Functions, John Wiley & Sons."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1258","DOI":"10.1016\/j.jspi.2007.05.001","article-title":"Robust likelihood functions in Bayesian inference","volume":"138","author":"Greco","year":"2008","journal-title":"J. Statist. Plan. Inference"},{"key":"ref_10","unstructured":"Goldstein, M. (1999). Bayes Linear Analysis, CRC Press. Wiley StatsRef: Statistics Reference Online."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1725","DOI":"10.1214\/aos\/1176348368","article-title":"Empirical likelihood for linear models","volume":"19","author":"Owen","year":"1991","journal-title":"Ann. Statist."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1203","DOI":"10.1126\/science.1248506","article-title":"The parable of Google Flu: Traps in big data analysis","volume":"343","author":"Lazer","year":"2014","journal-title":"Science"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Basu, A., Shioya, H., and Park, C. (2011). Statistical Inference: The Minimum Distance Approach, CRC Press.","DOI":"10.1201\/b10956"},{"key":"ref_14","unstructured":"Miller, J.W., and Dunson, D.B. (arXiv, 2015). Robust Bayesian inference via coarsening, arXiv."},{"key":"ref_15","unstructured":"Goldstein, M. (1990). Influence and belief adjustment. Influence Diagrams, Belief Nets and Decision Analysis, Wiley."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"556","DOI":"10.1007\/s11749-014-0360-z","article-title":"Bayesian model robustness via disparities","volume":"23","author":"Hooker","year":"2014","journal-title":"Test"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"413","DOI":"10.1007\/s10463-014-0499-0","article-title":"Robust Bayes estimation using the density power divergence","volume":"68","author":"Ghosh","year":"2016","journal-title":"Ann. Inst. Statist. Math."},{"key":"ref_18","unstructured":"Ghosh, A., and Basu, A. (arXiv, 2017). General Robust Bayes Pseudo-Posterior: Exponential Convergence results with Applications, arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"O\u2019Hagan, A., Buck, C.E., Daneshkhah, A., Eiser, J.R., Garthwaite, P.H., Jenkinson, D.J., Oakley, J.E., and Rakow, T. (2006). Uncertain Judgements: Eliciting Experts\u2019 Probabilities, John Wiley & Sons.","DOI":"10.1002\/0470033312"},{"key":"ref_20","unstructured":"Winkler, R.L., and Murphy, A.H. (May, January 28). Evaluation of subjective precipitation probability forecasts. Proceedings of the First National Conference on Statistical Meteorology, Albany, NY, USA."},{"key":"ref_21","first-page":"1367","article-title":"Game theory, maximum entropy, minimum discrepancy and robust Bayesian decision theory","volume":"32","author":"Dawid","year":"2004","journal-title":"Ann. Statist."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1080\/00031305.1988.10475585","article-title":"Optimal information processing and Bayes\u2019s theorem","volume":"42","author":"Zellner","year":"1988","journal-title":"Am. Statist."},{"key":"ref_23","unstructured":"Celeux, G., Jewson, J., Josse, J., Marin, J.M., and Robert, C.P. (arXiv, 2017). Some discussions on the Read Paper \u201cBeyond subjective and objective in statistics\u201d by A. Gelman and C. Hennig, arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"967","DOI":"10.1111\/rssa.12276","article-title":"Beyond subjective and objective in statistics","volume":"180","author":"Gelman","year":"2015","journal-title":"J. R. Statist. Soc. Ser. A (Statist. Soc.)"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"403","DOI":"10.1214\/06-BA116","article-title":"Subjective Bayesian analysis: Principles and practice","volume":"1","author":"Goldstein","year":"2006","journal-title":"Bayesian Anal."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"311","DOI":"10.1080\/0094965021000033477","article-title":"The generalized Kullback-Leibler divergence and robust inference","volume":"73","author":"Park","year":"2003","journal-title":"J. Statist. Comput. Simul."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1111\/j.1467-842X.2006.00428.x","article-title":"Robust inference in parametric models using the family of generalized negative exponential disparities","volume":"48","author":"Bhandari","year":"2006","journal-title":"Aust. N. Z. J. Statist."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Smith, J.Q. (2010). Bayesian Decision Analysis: Principles and Practice, Cambridge University Press.","DOI":"10.1017\/CBO9780511779237"},{"key":"ref_29","unstructured":"Devroye, L., and Gyorfi, L. (1985). Nonparametric Density Estimation: The L1 View, John Wiley & Sons Incorporated."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"445","DOI":"10.1214\/aos\/1176343842","article-title":"Minimum Hellinger distance estimates for parametric models","volume":"5","author":"Beran","year":"1977","journal-title":"Ann. Statist."},{"key":"ref_31","unstructured":"Smith, J. (1995). Bayesian Approximations and the Hellinger Metric, Unpublished work."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"134","DOI":"10.3390\/e13010134","article-title":"Generalized alpha-beta divergences and their application to robust nonnegative matrix factorization","volume":"13","author":"Cichocki","year":"2011","journal-title":"Entropy"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2746","DOI":"10.3150\/16-BEJ826","article-title":"A generalized divergence for statistical inference","volume":"23","author":"Ghosh","year":"2017","journal-title":"Bernoulli"},{"key":"ref_34","first-page":"299","article-title":"Information-type measures of difference of probability distributions and indirect observations","volume":"2","author":"Csisz","year":"1967","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_35","unstructured":"Shun-ichi, A. (2012). Differential-Geometrical Methods in Statistics, Springer."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"440","DOI":"10.1111\/j.2517-6161.1984.tb01318.x","article-title":"Multinomial goodness-of-fit tests","volume":"46","author":"Cressie","year":"1984","journal-title":"J. R. Statist. Soc. Ser. B (Methodol.)"},{"key":"ref_37","unstructured":"Sason, I., and Verd\u00fa, S. (2015). Bounds among f-divergences. IEEE Trans. Inf. Theory, submitted."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1093\/biomet\/85.3.549","article-title":"Robust and efficient estimation by minimising a density power divergence","volume":"85","author":"Basu","year":"1998","journal-title":"Biometrika"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1111\/sjos.12168","article-title":"Minimum scoring rule inference","volume":"43","author":"Dawid","year":"2016","journal-title":"Scand. J. Statist."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"601","DOI":"10.1093\/biomet\/asv026","article-title":"Bayesian sensitivity analysis with the Fisher\u2013Rao metric","volume":"102","author":"Kurtek","year":"2015","journal-title":"Biometrika"},{"key":"ref_41","unstructured":"Silverman, B.W. (1986). Density Estimation for Statistics and Data Analysis, CRC Press."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1080\/01621459.1986.10478264","article-title":"Minimum Hellinger distance estimation for multivariate location and covariance","volume":"81","author":"Tamura","year":"1986","journal-title":"J. Am. Statist. Assoc."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1137\/1114019","article-title":"Non-parametric estimation of a multivariate probability density","volume":"14","author":"Epanechnikov","year":"1969","journal-title":"Theory Probab. Appl."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.1214\/aop\/1176995945","article-title":"On the maximal deviation of k-dimensional density estimates","volume":"4","author":"Rosenblatt","year":"1976","journal-title":"Ann. Probab."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1217","DOI":"10.1214\/aos\/1176345986","article-title":"On bandwidth variation in kernel estimates-a square root law","volume":"10","author":"Abramson","year":"1982","journal-title":"Ann. Statist."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"2795","DOI":"10.1109\/78.324744","article-title":"Nonparametric multivariate density estimation: A comparative study","volume":"42","author":"Hwang","year":"1994","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Ram, P., and Gray, A.G. (2011, January 21\u201324). Density estimation trees. Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Diego, CA, USA.","DOI":"10.1145\/2020408.2020507"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1402","DOI":"10.1080\/01621459.2013.813389","article-title":"Multivariate density estimation by bayesian sequential partitioning","volume":"108","author":"Lu","year":"2013","journal-title":"J. Am. Statist. Assoc."},{"key":"ref_49","unstructured":"Li, M., and Dunson, D.B. (arXiv, 2016). A framework for probabilistic inferences from imperfect models, arXiv."},{"key":"ref_50","first-page":"1","article-title":"Stan: A probabilistic programming language","volume":"20","author":"Carpenter","year":"2016","journal-title":"J. Statist. Softw."},{"key":"ref_51","unstructured":"Hansen, B.E. (2004). Nonparametric Conditional Density Estimation, Unpublished work."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1694","DOI":"10.1016\/j.csda.2007.05.018","article-title":"Outlier identification in high dimensions","volume":"52","author":"Filzmoser","year":"2008","journal-title":"Comput. Statist. Data Anal."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/6\/442\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:07:32Z","timestamp":1760195252000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/6\/442"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,6,6]]},"references-count":52,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2018,6]]}},"alternative-id":["e20060442"],"URL":"https:\/\/doi.org\/10.3390\/e20060442","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,6,6]]}}}