{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T22:55:55Z","timestamp":1771023355370,"version":"3.50.1"},"reference-count":25,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2021,10,12]],"date-time":"2021-10-12T00:00:00Z","timestamp":1633996800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We present new PAC-Bayesian generalisation bounds for learning problems with unbounded loss functions. This extends the relevance and applicability of the PAC-Bayes learning framework, where most of the existing literature focuses on supervised learning problems with a bounded loss function (typically assumed to take values in the interval [0;1]). In order to relax this classical assumption, we propose to allow the range of the loss to depend on each predictor. This relaxation is captured by our new notion of HYPothesis-dependent rangE (HYPE). Based on this, we derive a novel PAC-Bayesian generalisation bound for unbounded loss functions, and we instantiate it on a linear regression problem. To make our theory usable by the largest audience possible, we include discussions on actual computation, practicality and limitations of our assumptions.<\/jats:p>","DOI":"10.3390\/e23101330","type":"journal-article","created":{"date-parts":[[2021,10,12]],"date-time":"2021-10-12T08:09:32Z","timestamp":1634026172000},"page":"1330","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["PAC-Bayes Unleashed: Generalisation Bounds with Unbounded Losses"],"prefix":"10.3390","volume":"23","author":[{"given":"Maxime","family":"Haddouche","sequence":"first","affiliation":[{"name":"ENS Paris-Saclay, 91190 Gif-sur-Yvette, France"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1237-7430","authenticated-orcid":false,"given":"Benjamin","family":"Guedj","sequence":"additional","affiliation":[{"name":"Centre for Artificial Intelligence, Department of Computer Science, University College London, London WC1V 6LJ, UK"},{"name":"Inria, Lille\u2013Nord Europe Research Centre and Inria London Programme, 59800 Lille, France"}]},{"given":"Omar","family":"Rivasplata","sequence":"additional","affiliation":[{"name":"Centre for Artificial Intelligence, Department of Computer Science, University College London, London WC1V 6LJ, UK"}]},{"given":"John","family":"Shawe-Taylor","sequence":"additional","affiliation":[{"name":"Centre for Artificial Intelligence, Department of Computer Science, University College London, London WC1V 6LJ, UK"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Shawe-Taylor, J., and Williamson, R.C. (1997, January 6\u20139). A PAC analysis of a Bayes estimator. Proceedings of the 10th Annual Conference on Computational Learning Theory, Nashville, TN, USA.","DOI":"10.1145\/267460.267466"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"McAllester, D.A. (1998, January 24\u201326). Some PAC-Bayesian theorems. Proceedings of the Eleventh Annual Conference on Computational Learning Theory, Madison, WI, USA.","DOI":"10.1145\/279943.279989"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"McAllester, D.A. (1999, January 7\u20139). PAC-Bayesian model averaging. Proceedings of the Twelfth Annual Conference on Computational Learning Theory, Santa Cruz, CA, USA.","DOI":"10.1145\/307400.307435"},{"key":"ref_4","unstructured":"Guedj, B. (2019). A Primer on PAC-Bayesian Learning. arXiv."},{"key":"ref_5","unstructured":"Rivasplata, O., Kuzborskij, I., Szepesv\u00e1ri, C., and Shawe-Taylor, J. (2020, January 6\u201312). PAC-Bayes Analysis Beyond the Usual Bounds. Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, Online."},{"key":"ref_6","first-page":"233","article-title":"PAC-Bayesian Generalization Error Bounds for Gaussian Process Classification","volume":"3","author":"Seeger","year":"2002","journal-title":"J. Mach. Learn. Res."},{"key":"ref_7","first-page":"273","article-title":"Tutorial on practical prediction theory for classification","volume":"6","author":"Langford","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_8","unstructured":"Catoni, O. (2007). PAC-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning, Institute of Mathematical Statistics."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Germain, P., Lacasse, A., Laviolette, F., and Marchand, M. (2009, January 14\u201318). PAC-Bayesian Learning of Linear Classifiers. Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, QC, Canada.","DOI":"10.1145\/1553374.1553419"},{"key":"ref_10","first-page":"1","article-title":"On the properties of variational approximations of Gibbs posteriors","volume":"17","author":"Alquier","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_11","unstructured":"Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., and Garnett, R. (2016). PAC-Bayesian Theory Meets Bayesian Inference. Advances in Neural Information Processing Systems 29, Curran Associates, Inc."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"887","DOI":"10.1007\/s10994-017-5690-0","article-title":"Simpler PAC-Bayesian bounds for hostile data","volume":"107","author":"Alquier","year":"2018","journal-title":"Mach. Learn."},{"key":"ref_13","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). PAC-Bayes under potentially heavy tails. Advances in Neural Information Processing Systems 32, Curran Associates, Inc."},{"key":"ref_14","unstructured":"Kuzborskij, I., and Szepesv\u00e1ri, C. (2019). Efron-Stein PAC-Bayesian Inequalities. arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Shalaeva, V., Fakhrizadeh Esfahani, A., Germain, P., and Petreczky, M. (2020, January 7\u201312). Improved PAC-Bayesian Bounds for Linear Regression. Proceedings of the AAAI 2020\u2014Thirty-Fourth AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i04.6020"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Hutter, M., Stephan, F., Vovk, V., and Zeugmann, T. (2010). Distribution-Dependent PAC-Bayes Priors. Algorithmic Learning Theory, Springer.","DOI":"10.1007\/978-3-642-16108-7"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1016\/j.tcs.2012.10.013","article-title":"Tighter PAC-Bayes Bounds through Distribution-Dependent Priors","volume":"473","author":"Lever","year":"2013","journal-title":"Theor. Comput. Sci."},{"key":"ref_18","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). PAC-Bayes Un-Expected Bernstein Inequality. Advances in Neural Information Processing Systems 32, Curran Associates, Inc."},{"key":"ref_19","first-page":"3507","article-title":"PAC-Bayes bounds with data dependent priors","volume":"13","author":"Ambroladze","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1148","DOI":"10.1214\/11-AIHP454","article-title":"Challenging the empirical mean and empirical variance: A deviation study","volume":"48","author":"Catoni","year":"2012","journal-title":"Ann. Inst. H. Poincar\u00e9 Probab. Statist."},{"key":"ref_21","unstructured":"Catoni, O., and Giulini, I. (2017). Dimension-free PAC-Bayesian bounds for matrices, vectors, and linear least squares regression. arXiv."},{"key":"ref_22","first-page":"63","article-title":"A Derivation of n-Dimensional Spherical Coordinates","volume":"67","author":"Blumenson","year":"1960","journal-title":"Am. Math. Mon."},{"key":"ref_23","unstructured":"Winkelbauer, A. (2012). Moments and Absolute Moments of the Normal Distribution. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"538","DOI":"10.4153\/CMB-2010-107-8","article-title":"On the Horizontal Monotonicity of |\u0393(s)|","volume":"54","author":"Srinivasan","year":"2011","journal-title":"Can. Math. Bull."},{"key":"ref_25","unstructured":"Gauss, C.F. (2011). Disquisitiones Generales Circa Seriem Infinitam (reprint). Werke, Cambridge University Press."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/10\/1330\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:11:54Z","timestamp":1760166714000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/10\/1330"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,12]]},"references-count":25,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["e23101330"],"URL":"https:\/\/doi.org\/10.3390\/e23101330","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,12]]}}}