{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T21:51:33Z","timestamp":1791323493463,"version":"4.3.1"},"reference-count":48,"publisher":"Institute of Mathematical Statistics","issue":"4","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Ann. Statist."],"published-print":{"date-parts":[[2026,8,1]]},"abstract":"<jats:p>In recent years, differential privacy has been adopted by tech-companies and governmental agencies as the standard for measuring privacy in algorithms. In this article, we study differential privacy in Bayesian posterior sampling settings. We begin by considering differential privacy in the most common privatisation setting in which Laplace or Gaussian noise is injected into the output. In an effort to achieve better differential privacy, we consider adopting Huber\u2019s contamination model for use within privacy settings, and replace at random data points with samples from a heavy-tailed distribution (instead of injecting noise into the output). We derive bounds for the differential privacy level (\u03f5,\u03b4) of our approach, without requiring bounded observation and parameter spaces, a restriction commonly imposed in the literature. We further consider for our approach the effect of sample size on the privacy level and the rate at which (\u03f5,\u03b4) converges to zero. Asymptotically, our contamination approach is fully private with no information loss. We also provide examples of inference models for which our approach applies, with theoretical convergence rate analysis and simulation studies.<\/jats:p>","DOI":"10.1214\/26-aos2628","type":"journal-article","created":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T08:14:25Z","timestamp":1787040865000},"page":"1870-1894","source":"Crossref","is-referenced-by-count":0,"title":["Privacy guarantees in posterior sampling under contamination"],"prefix":"10.1214","volume":"54","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3528-4800","authenticated-orcid":false,"given":"Shenggang","family":"Hu","sequence":"first","affiliation":[{"name":"Department of Statistics, University of Warwick"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2211-233X","authenticated-orcid":false,"given":"Louis","family":"Aslett","sequence":"additional","affiliation":[{"name":"Department of Mathematical Sciences, Durham University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1734-2026","authenticated-orcid":false,"given":"Hongsheng","family":"Dai","sequence":"additional","affiliation":[{"name":"School of Mathematics, Statistics and Physics, Newcastle University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Murray","family":"Pollock","sequence":"additional","affiliation":[{"name":"School of Mathematics, Statistics and Physics, Newcastle University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Gareth O.","family":"Roberts","sequence":"additional","affiliation":[{"name":"Department of Statistics, University of Warwick"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"108","reference":[{"key":"15","doi-asserted-by":"publisher","unstructured":"<small>Durmus, A.<\/small> and <small>Moulines, \u00c9.<\/small> (2017). Nonasymptotic convergence analysis for the unadjusted Langevin algorithm. <i>Ann. Appl. Probab.<\/i> <b>27<\/b> 1551\u20131587.","DOI":"10.1214\/16-AAP1238"},{"key":"35","doi-asserted-by":"publisher","unstructured":"<small>Robert, C. P.<\/small> and <small>Casella, G.<\/small> (1999). <i>Monte Carlo Statistical Methods<\/i>. <i>Springer Texts in Statistics<\/i>. Springer, New York.","DOI":"10.1007\/978-1-4757-3071-5"},{"key":"45","doi-asserted-by":"publisher","unstructured":"<small>Wong, W. H.<\/small> and <small>Shen, X.<\/small> (1995). Probability inequalities for likelihood ratios and convergence rates of sieve MLEs. <i>Ann. Statist.<\/i> <b>23<\/b> 339\u2013362.","DOI":"10.1214\/aos\/1176324524"},{"key":"16","doi-asserted-by":"publisher","unstructured":"<small>Dwork, C.<\/small>, <small>McSherry, F.<\/small>, <small>Nissim, K.<\/small> and <small>Smith, A.<\/small> (2006). Calibrating noise to sensitivity in private data analysis. In <i>Theory of Cryptography<\/i>. <i>Lecture Notes in Computer Science<\/i> <b>3876<\/b> 265\u2013284. Springer, Berlin.","DOI":"10.1007\/11681878_14"},{"key":"17","doi-asserted-by":"publisher","unstructured":"<small>Dwork, C.<\/small> and <small>Roth, A.<\/small> (2013). The algorithmic foundations of differential privacy. <i>Found. Trends Theor. Comput. Sci.<\/i> <b>9<\/b> 211\u2013487.","DOI":"10.1561\/0400000042"},{"key":"36","doi-asserted-by":"publisher","unstructured":"<small>Rohde, A.<\/small> and <small>Steinberger, L.<\/small> (2020). Geometrizing rates of convergence under local differential privacy constraints. <i>Ann. Statist.<\/i> <b>48<\/b> 2646\u20132670.","DOI":"10.1214\/19-AOS1901"},{"key":"39","doi-asserted-by":"publisher","unstructured":"<small>SHEN, X.<\/small> and <small>WONG, W. H.<\/small> (1994). Convergence rate of sieve estimates. <i>Ann. Statist.<\/i> <b>22<\/b> 580\u2013615.","DOI":"10.1214\/aos\/1176325486"},{"key":"1","doi-asserted-by":"crossref","unstructured":"<small>ABOWD, J. M.<\/small> (2018). The US Census Bureau adopts differential privacy. In <i>Proceedings of the<\/i> 24<i>th ACM SIGKDD International Conference on Knowledge Discovery &amp; Data Mining<\/i> 2867\u20132867.","DOI":"10.1145\/3219819.3226070"},{"key":"38","doi-asserted-by":"publisher","unstructured":"<small>SHEN, X.<\/small> and <small>WASSERMAN, L.<\/small> (2001). Rates of convergence of posterior distributions. <i>Ann. Statist.<\/i> <b>29<\/b> 687\u2013714.","DOI":"10.1214\/aos\/1009210686"},{"key":"10","doi-asserted-by":"publisher","unstructured":"<small>CAI, T. T.<\/small>, <small>WANG, Y.<\/small> and <small>ZHANG, L.<\/small> (2021). The cost of privacy: Optimal rates of convergence for parameter estimation with differential privacy. <i>Ann. Statist.<\/i> <b>49<\/b> 2825\u20132850.","DOI":"10.1214\/21-aos2058"},{"key":"13","unstructured":"<small>DING, B.<\/small>, <small>KULKARNI, J.<\/small> and <small>YEKHANIN, S.<\/small> (2017). Collecting telemetry data privately. <i>Adv. Neural Inf. Process. Syst.<\/i> <b>30<\/b>."},{"key":"19","doi-asserted-by":"crossref","unstructured":"<small>ERLINGSSON, \u00da.<\/small>, <small>PIHUR, V.<\/small> and <small>KOROLOVA, A.<\/small> (2014). Rappor: Randomized aggregatable privacy-preserving ordinal response. In <i>Proceedings of the<\/i> 2014 <i>ACM SIGSAC Conference on Computer and Communications Security<\/i> 1054\u20131067.","DOI":"10.1145\/2660267.2660348"},{"key":"21","doi-asserted-by":"publisher","unstructured":"<small>Ghosal, S.<\/small> and <small>van der Vaart, A.<\/small> (2017). <i>Fundamentals of Nonparametric Bayesian Inference<\/i>. <i>Cambridge Series in Statistical and Probabilistic Mathematics<\/i> <b>44<\/b>. Cambridge Univ. Press, Cambridge.","DOI":"10.1017\/9781139029834"},{"key":"29","doi-asserted-by":"publisher","unstructured":"<small>LI, M.<\/small>, <small>BERRETT, T. B.<\/small> and <small>YU, Y.<\/small> (2023). On robustness and local differential privacy. <i>Ann. Statist.<\/i> <b>51<\/b> 717\u2013737.","DOI":"10.1214\/23-aos2267"},{"key":"43","doi-asserted-by":"publisher","unstructured":"<small>WASSERMAN, L.<\/small> (2004). <i>All of Statistics<\/i>: <i>A Concise Course in Statistical Inference<\/i>. <i>Springer Texts in Statistics<\/i>. Springer, New York.","DOI":"10.1007\/978-0-387-21736-9"},{"key":"2","unstructured":"<small>AKTAY, A.<\/small>, <small>BAVADEKAR, S.<\/small>, <small>COSSOUL, G.<\/small>, <small>DAVIS, J.<\/small>, <small>DESFONTAINES, D.<\/small>, <small>FABRIKANT, A.<\/small>, <small>GABRILOVICH, E.<\/small>, <small>GADEPALLI, K.<\/small>, <small>GIPSON, B.<\/small> et al. (2020). Google COVID-19 community mobility reports: Anonymization process description (version 1.1). arXiv Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2004.04145\">2004.04145<\/a>."},{"key":"4","doi-asserted-by":"publisher","unstructured":"<small>ANDRIEU, C.<\/small>, <small>LEE, A.<\/small>, <small>POWER, S.<\/small> and <small>WANG, A. Q.<\/small> (2026). Weak Poincar\u00e9 inequalities for Markov chains: Theory and applications. <i>Ann. Appl. Probab.<\/i> <b>36<\/b> 46\u2013107.","DOI":"10.1214\/25-AAP2185"},{"key":"5","unstructured":"<small>APPLE<\/small> (2017). Learning with Privacy at Scale. Available at <a href=\"https:\/\/docs-assets.developer.apple.com\/ml-research\/papers\/learning-with-privacy-at-scale.pdf\">https:\/\/docs-assets.developer.apple.com\/ml-research\/papers\/learning-with-privacy-at-scale.pdf<\/a>. [Online; accessed 13-September-2023]."},{"key":"6","unstructured":"<small>ARTIN, E.<\/small> (1964). <i>The Gamma Function<\/i>. <i>Athena Series<\/i>: <i>Selected Topics in Mathematics<\/i>. Holt, Rinehart and Winston, New York-Toronto-London."},{"key":"7","unstructured":"<small>BERNSTEIN, G.<\/small> and <small>SHELDON, D. R.<\/small> (2019). Differentially private Bayesian linear regression. <i>Adv. Neural Inf. Process. Syst.<\/i> <b>32<\/b>."},{"key":"8","unstructured":"<small>BISWAS, S.<\/small>, <small>DONG, Y.<\/small>, <small>KAMATH, G.<\/small> and <small>ULLMAN, J.<\/small> (2020). Coinpress: Practical private mean and covariance estimation. <i>Adv. Neural Inf. Process. Syst.<\/i> <b>33<\/b> 14475\u201314485."},{"key":"9","doi-asserted-by":"publisher","unstructured":"<small>BUN, M.<\/small> and <small>STEINKE, T.<\/small> (2016a). Concentrated differential privacy: Simplifications, extensions, and lower bounds. In <i>Theory of Cryptography. Part I<\/i>. <i>Lecture Notes in Computer Science<\/i> <b>9985<\/b> 635\u2013658. Springer, Berlin.","DOI":"10.1007\/978-3-662-53641-4_24"},{"key":"11","doi-asserted-by":"publisher","unstructured":"<small>COVINGTON, C.<\/small>, <small>HE, X.<\/small>, <small>HONAKER, J.<\/small> and <small>KAMATH, G.<\/small> (2025). Unbiased statistical estimation and valid confidence intervals under differential privacy. <i>Statist. Sinica<\/i> <b>35<\/b> 651\u2013670.","DOI":"10.5705\/ss.202022.0276"},{"key":"12","unstructured":"<small>DIMITRAKAKIS, C.<\/small>, <small>NELSON, B.<\/small>, <small>ZHANG, Z.<\/small>, <small>MITROKOTSA, A.<\/small> and <small>RUBINSTEIN, B. I. P.<\/small> (2017). Differential privacy for Bayesian inference through posterior sampling. <i>J. Mach. Learn. Res.<\/i> <b>18<\/b> Paper No. 11."},{"key":"14","unstructured":"<small>DUCHI, J.<\/small>, <small>HAQUE, S.<\/small> and <small>KUDITIPUDI, R.<\/small> (2023). A fast algorithm for adaptive private mean estimation. arXiv Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2301.07078\">2301.07078<\/a>."},{"key":"18","unstructured":"<small>DWORK, C.<\/small> and <small>ROTHBLUM, G. N.<\/small> (2016). Concentrated differential privacy. arXiv preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/1603.01887\">1603.01887<\/a>."},{"key":"20","unstructured":"<small>FOULDS, J.<\/small>, <small>GEUMLEK, J.<\/small>, <small>WELLING, M.<\/small> and <small>CHAUDHURI, K.<\/small> (2016). On the theory and practice of privacy-preserving Bayesian data analysis. In <i>Proceedings of the Thirty-Second Conference on Uncertainty in Artificial Intelligence<\/i>. <i>UAI\u2019<\/i>16 192\u2013201. AUAI Press, Arlington, VA, USA."},{"key":"22","unstructured":"<small>HALL, R.<\/small>, <small>WASSERMAN, L.<\/small> and <small>RINALDO, A.<\/small> (2013). Random differential privacy. <i>J. Priv. Confid.<\/i> <b>4<\/b>."},{"key":"23","unstructured":"<small>HEIKKIL\u00c4, M.<\/small>, <small>J\u00c4LK\u00d6, J.<\/small>, <small>DIKMEN, O.<\/small> and <small>HONKELA, A.<\/small> (2019). Differentially private Markov chain Monte Carlo. <i>Adv. Neural Inf. Process. Syst.<\/i> <b>32<\/b>."},{"key":"24","doi-asserted-by":"crossref","unstructured":"<small>HU, S.<\/small>, <small>ASLETT, L.<\/small>, <small>DAI, H.<\/small>, <small>POLLOCK, M.<\/small> and <small>ROBERTS, G. O.<\/small> (2026). Supplement to \u201cPrivacy guarantees in posterior sampling under contamination.\u201d <a href=\"https:\/\/doi.org\/10.1214\/26-AOS2628SUPP\">https:\/\/doi.org\/10.1214\/26-AOS2628SUPP<\/a>","DOI":"10.1214\/26-AOS2628SUPP"},{"key":"25","unstructured":"<small>HUANG, Z.<\/small>, <small>LIANG, Y.<\/small> and <small>YI, K.<\/small> (2021). Instance-optimal mean estimation under differential privacy. <i>Adv. Neural Inf. Process. Syst.<\/i> <b>34<\/b> 25993\u201326004."},{"key":"26","unstructured":"<small>HUBER, P. J.<\/small> (2004). <i>Robust Statistics<\/i> <b>523<\/b>. Wiley, New York."},{"key":"27","doi-asserted-by":"publisher","unstructured":"<small>KROLL, M.<\/small> (2021). On density estimation at a fixed point under local differential privacy. <i>Electron. J. Stat.<\/i> <b>15<\/b> 1783\u20131813.","DOI":"10.1214\/21-ejs1830"},{"key":"30","doi-asserted-by":"crossref","unstructured":"<small>MACHANAVAJJHALA, A.<\/small>, <small>KIFER, D.<\/small>, <small>ABOWD, J.<\/small>, <small>GEHRKE, J.<\/small> and <small>VILHUBER, L.<\/small> (2008). Privacy: Theory meets practice on the map. In 2008 <i>IEEE<\/i> 24<i>th International Conference on Data Engineering<\/i> 277\u2013286. IEEE Press, New York.","DOI":"10.1109\/ICDE.2008.4497436"},{"key":"31","unstructured":"<small>MINAMI, K.<\/small>, <small>ARAI, H.<\/small>, <small>SATO, I.<\/small> and <small>NAKAGAWA, H.<\/small> (2016). Differential privacy without sensitivity. <i>Adv. Neural Inf. Process. Syst.<\/i> <b>29<\/b>."},{"key":"32","doi-asserted-by":"publisher","unstructured":"<small>MU, W.<\/small> and <small>XIONG, S.<\/small> (2023). On Huber\u2019s contaminated model. <i>J. Complexity<\/i> <b>77<\/b> Paper No. 101745.","DOI":"10.1016\/j.jco.2023.101745"},{"key":"34","unstructured":"<small>PEREIRA, M.<\/small>, <small>KIM, A.<\/small>, <small>ALLEN, J.<\/small>, <small>WHITE, K.<\/small>, <small>FERRES, J. L.<\/small> and <small>DODHIA, R.<\/small> (2021). US broadband coverage data set: a differentially private data release. arXiv Preprint. Available at arXiv:<a href=\"https:\/\/arxiv.org\/abs\/2103.14035\">2103.14035<\/a>."},{"key":"37","unstructured":"<small>SHEFFET, O.<\/small> (2019). Old techniques in differentially private linear regression. In <i>Algorithmic Learning Theory<\/i> 2019. <i>Proc. Mach. Learn. Res<\/i>. (<i>PMLR<\/i>) <b>98<\/b> [788\u2013826]. Proceedings of Machine Learning Research PMLR."},{"key":"40","doi-asserted-by":"crossref","unstructured":"<small>TRIASTCYN, A.<\/small> and <small>FALTINGS, B.<\/small> (2019). Federated learning with Bayesian differential privacy. In 2019 <i>IEEE International Conference on Big Data<\/i> (<i>Big Data<\/i>) 2587\u20132596. IEEE Press, New York.","DOI":"10.1109\/BigData47090.2019.9005465"},{"key":"41","unstructured":"<small>TRIASTCYN, A.<\/small> and <small>FALTINGS, B.<\/small> (2020). Bayesian differential privacy for machine learning. In <i>International Conference on Machine Learning<\/i> 9583\u20139592. PMLR."},{"key":"42","unstructured":"<small>WANG, Y.-X.<\/small>, <small>FIENBERG, S.<\/small> and <small>SMOLA, A.<\/small> (2015). Privacy for free: Posterior sampling and stochastic gradient Monte Carlo. In <i>International Conference on Machine Learning<\/i> 2493\u20132502. PMLR."},{"key":"44","unstructured":"<small>WASSERMAN, L.<\/small> (2020). Lecture note 15, intermediate statistics, fall 2020. [Online; accessed 19-October-2023]."},{"key":"46","doi-asserted-by":"publisher","unstructured":"<small>YILDIRIM, S.<\/small> and <small>ERMI\u015e, B.<\/small> (2019). Exact MCMC with differentially private moves: Revisiting the penalty algorithm in a data privacy framework. <i>Stat. Comput.<\/i> <b>29<\/b> 947\u2013963.","DOI":"10.1007\/s11222-018-9847-x"},{"key":"47","unstructured":"<small>ZHANG, W.<\/small> and <small>ZHANG, R.<\/small> (2023). DP-fast MH: Private, fast, and accurate Metropolis\u2013Hastings for large-scale Bayesian inference. In <i>International Conference on Machine Learning<\/i> 41847\u201341860. PMLR."},{"key":"48","doi-asserted-by":"crossref","unstructured":"<small>ZHANG, Z.<\/small>, <small>RUBINSTEIN, B.<\/small> and <small>DIMITRAKAKIS, C.<\/small> (2016). On the differential privacy of Bayesian inference. In <i>Proceedings of the AAAI Conference on Artificial Intelligence<\/i> <b>30<\/b>.","DOI":"10.1609\/aaai.v30i1.10254"},{"key":"28","unstructured":"<small>KULKARNI, T.<\/small>, <small>J\u00c4LK\u00d6, J.<\/small>, <small>KOSKELA, A.<\/small>, <small>KASKI, S.<\/small> and <small>HONKELA, A.<\/small> (2021). Differentially private Bayesian inference for generalized linear models. In <i>International Conference on Machine Learning<\/i> 5838\u20135849. PMLR."},{"key":"3","doi-asserted-by":"publisher","unstructured":"<small>ANDRIEU, C.<\/small>, <small>LEE, A.<\/small>, <small>POWER, S.<\/small> and <small>WANG, A. Q.<\/small> (2022). Comparison of Markov chains via weak Poincar\u00e9 inequalities with application to pseudo-marginal MCMC. <i>Ann. Statist.<\/i> <b>50<\/b> 3592\u20133618.","DOI":"10.1214\/22-aos2241"},{"key":"33","unstructured":"<small>MURPHY, K. P.<\/small> (2022). <i>Probabilistic Machine Learning<\/i>: <i>An Introduction<\/i>. MIT Press, Cambridge."}],"container-title":["The Annals of Statistics"],"original-title":[],"deposited":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T21:12:18Z","timestamp":1791321138000},"score":1,"resource":{"primary":{"URL":"https:\/\/projecteuclid.org\/journals\/annals-of-statistics\/volume-54\/issue-4\/Privacy-guarantees-in-posterior-sampling-under-contamination\/10.1214\/26-AOS2628.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,8,1]]},"references-count":48,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2026,8,1]]}},"URL":"https:\/\/doi.org\/10.1214\/26-aos2628","relation":{},"ISSN":["0090-5364","2168-8966"],"issn-type":[{"value":"0090-5364","type":"print"},{"value":"2168-8966","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,8,1]]}}}