{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,28]],"date-time":"2026-09-28T18:45:50Z","timestamp":1790621150559,"version":"4.1.0"},"reference-count":52,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2017,3,16]],"date-time":"2017-03-16T00:00:00Z","timestamp":1489622400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We describe a framework to build distances by measuring the tightness of inequalities and introduce the notion of proper statistical divergences and improper pseudo-divergences. We then consider the H\u00f6lder ordinary and reverse inequalities and present two novel classes of H\u00f6lder divergences and pseudo-divergences that both encapsulate the special case of the Cauchy\u2013Schwarz divergence. We report closed-form formulas for those statistical dissimilarities when considering distributions belonging to the same exponential family provided that the natural parameter space is a cone (e.g., multivariate Gaussians) or affine (e.g., categorical distributions). Those new classes of H\u00f6lder distances are invariant to rescaling and thus do not require distributions to be normalized. Finally, we show how to compute statistical H\u00f6lder centroids with respect to those divergences and carry out center-based clustering toy experiments on a set of Gaussian distributions which demonstrate empirically that symmetrized H\u00f6lder divergences outperform the symmetric Cauchy\u2013Schwarz divergence.<\/jats:p>","DOI":"10.3390\/e19030122","type":"journal-article","created":{"date-parts":[[2017,3,16]],"date-time":"2017-03-16T11:24:28Z","timestamp":1489663468000},"page":"122","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":25,"title":["On H\u00f6lder Projective Divergences"],"prefix":"10.3390","volume":"19","author":[{"given":"Frank","family":"Nielsen","sequence":"first","affiliation":[{"name":"Computer Science Department LIX, \u00c9cole Polytechnique, 91128 Palaiseau Cedex, France"},{"name":"Sony Computer Science Laboratories Inc., Tokyo 141-0022, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ke","family":"Sun","sequence":"additional","affiliation":[{"name":"Computer, Electrical and Mathematical Sciences and Engineering Division, King Abdullah University of Science and Technology, Thuwal 23955, Saudi Arabia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"St\u00e9phane","family":"Marchand-Maillet","sequence":"additional","affiliation":[{"name":"Computer Vision and Multimedia Laboratory (Viper), University of Geneva, CH-1211 Geneva, Switzerland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2017,3,16]]},"reference":[{"key":"ref_1","unstructured":"Mitrinovic, D.S., Pecaric, J., and Fink, A.M. (2013). Classical and New Inequalities in Analysis, Springer Science & Business Media."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1229","DOI":"10.3390\/e13071229","article-title":"On accuracy of PDF divergence estimators and their applicability to representative data sampling","volume":"13","author":"Budka","year":"2011","journal-title":"Entropy"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Amari, S.I. (2016). Information Geometry and Its Applications, Springer.","DOI":"10.1007\/978-4-431-55978-8"},{"key":"ref_4","first-page":"81","article-title":"Information and accuracy attainable in the estimation of statistical parameters","volume":"37","author":"Rao","year":"1945","journal-title":"Bull. Calcutta Math. Soc."},{"key":"ref_5","first-page":"1705","article-title":"Clustering with Bregman divergences","volume":"6","author":"Banerjee","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Nielsen, F., and Nock, R. (2015, January 19\u201324). Total Jensen divergences: Definition, properties and clustering. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), South Brisbane, Australia.","DOI":"10.1109\/ICASSP.2015.7178324"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1109\/TIT.1982.1056497","article-title":"On the convexity of some divergence measures based on entropy functions","volume":"28","author":"Burbea","year":"1982","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"5455","DOI":"10.1109\/TIT.2011.2159046","article-title":"The Burbea-Rao and Bhattacharyya centroids","volume":"57","author":"Nielsen","year":"2011","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1198\/016214506000001437","article-title":"Strictly proper scoring rules, prediction, and estimation","volume":"102","author":"Gneiting","year":"2007","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_10","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, John Wiley & Sons."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2053","DOI":"10.1016\/j.jmva.2008.02.004","article-title":"Robust parameter estimation with a small bias against heavy contamination","volume":"99","author":"Fujisawa","year":"2008","journal-title":"J. Multivar. Anal."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Nielsen, F., and Nock, R. (2016, January 24\u201326). Patch Matching with Polynomial Exponential Families and Projective Divergences. Proceedings of the 9th International Conference on Similarity Search and Applications, Tokyo, Japan.","DOI":"10.1007\/978-3-319-46759-7_8"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1162\/08997660460734047","article-title":"Divergence function, duality, and convex analysis","volume":"16","author":"Zhang","year":"2004","journal-title":"Neural Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"5384","DOI":"10.3390\/e15125384","article-title":"Nonparametric information geometry: From divergence function to referential-representational biduality on statistical manifolds","volume":"15","author":"Zhang","year":"2013","journal-title":"Entropy"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"032003","DOI":"10.1088\/1751-8113\/45\/3\/032003","article-title":"A closed-form expression for the Sharma\u2013Mittal entropy of exponential families","volume":"45","author":"Nielsen","year":"2011","journal-title":"J. Phys. A Math. Theor."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"De Souza, D.C., Vigelis, R.F., and Cavalcante, C.C. (2016). Geometry Induced by a Generalization of R\u00e9nyi Divergence. Entropy, 18.","DOI":"10.3390\/e18110407"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.3150\/13-BEJ557","article-title":"Affine invariant divergences associated with proper composite scoring rules and their applications","volume":"20","author":"Kanamori","year":"2014","journal-title":"Bernoulli"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2611","DOI":"10.3390\/e16052611","article-title":"Scale-invariant divergences for density functions","volume":"16","author":"Kanamori","year":"2014","journal-title":"Entropy"},{"key":"ref_19","unstructured":"Nielsen, F., and Garcia, V. (arXiv, 2009). Statistical exponential families: A digest with flash cards, arXiv."},{"key":"ref_20","first-page":"145","article-title":"An extension of a certain theorem in inequalities","volume":"17","author":"Rogers","year":"1888","journal-title":"Messenger Math."},{"key":"ref_21","first-page":"38","article-title":"\u00dcber einen Mittelwertssatz","volume":"44","author":"Holder","year":"1889","journal-title":"Nachr. Akad. Wiss. Gottingen Math. Phys. Kl."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"2436","DOI":"10.1109\/TPAMI.2014.2324585","article-title":"Information theoretic shape matching","volume":"36","author":"Hasanbelliu","year":"2014","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","unstructured":"Nielsen, F. (2012, January 11\u201315). Closed-form information-theoretic divergences for statistical mixtures. Proceedings of the 21st International Conference on Pattern Recognition (ICPR), Tsukuba, Japan."},{"key":"ref_24","first-page":"130","article-title":"Reference duality and representation duality in information geometry","volume":"1641","author":"Zhang","year":"2015","journal-title":"Am. Inst. Phys. Conf. Ser."},{"key":"ref_25","first-page":"99","article-title":"On a measure of divergence between two statistical populations defined by their probability distributions","volume":"35","author":"Bhattacharyya","year":"1943","journal-title":"Bull. Calcutta Math. Soc."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Srivastava, A., Jermyn, I., and Joshi, S. (2007, January 17\u201322). Riemannian analysis of probability density functions with applications in vision. Proceedings of the 2007 IEEE Conference on Computer Vision and Pattern Recognition, Minneapolis, MN, USA.","DOI":"10.1109\/CVPR.2007.383188"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/LSP.2013.2288355","article-title":"On the Chi Square and Higher-Order Chi Distances for Approximating f-Divergences","volume":"1","author":"Nielsen","year":"2014","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_28","unstructured":"Nielsen, F., and Nock, R. (2011). Transactions on Computational Science XIV, Springer."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Nielsen, F., and Sun, K. (2016). Guaranteed Bounds on Information-Theoretic Measures of Univariate Mixtures Using Piecewise Log-Sum-Exp Inequalities. Entropy, 18.","DOI":"10.20944\/preprints201610.0086.v1"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"421","DOI":"10.1162\/NECO_a_00547","article-title":"Spontaneous clustering via minimum gamma-divergence","volume":"26","author":"Notsu","year":"2014","journal-title":"Neural Comput."},{"key":"ref_31","unstructured":"Rigazio, L., Tsakam, B., and Junqua, J.C. (2000, January 5\u20139). An optimal Bhattacharyya centroid algorithm for Gaussian clustering with applications in automatic speech recognition. Proceedings of the 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing, Istanbul, Turkey."},{"key":"ref_32","unstructured":"Davis, J.V., and Dhillon, I.S. (2006, January 7\u201310). Differential Entropic Clustering of Multivariate Gaussians. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Nielsen, F., and Nock, R. (2009). Emerging Trends in Visual Computing, Springer.","DOI":"10.1007\/978-3-642-00826-9"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Allamigeon, X., Gaubert, S., Goubault, E., Putot, S., and Stott, N. (2015, January 4\u20139). A scalable algebraic method to infer quadratic invariants of switched systems. Proceedings of the 12th International Conference on Embedded Software, Amsterdam, The Netherlands.","DOI":"10.1109\/EMSOFT.2015.7318262"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Sun, D.L., and F\u00e9votte, C. (2014, January 4\u20139). Alternating direction method of multipliers for non-negative matrix factorization with the beta-divergence. Proceedings of the 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Florence, Italy.","DOI":"10.1109\/ICASSP.2014.6854796"},{"key":"ref_36","first-page":"1345","article-title":"Clustering on the unit hypersphere using von Mises-Fisher distributions","volume":"6","author":"Banerjee","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_37","first-page":"154","article-title":"Von Mises-Fisher Clustering Models","volume":"32","author":"Gopal","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_38","first-page":"45","article-title":"Texture Retrieval Using Mixtures of Generalized Gaussian Distribution and Cauchy-Schwarz Divergence in Wavelet Domain","volume":"42","author":"Rami","year":"2016","journal-title":"Image Commun."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.neucom.2012.02.034","article-title":"Stochastic neighbor embedding (SNE) for dimension reduction and visualization using arbitrary divergences","volume":"90","author":"Bunte","year":"2012","journal-title":"Neurocomputing"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1343","DOI":"10.1162\/NECO_a_00110","article-title":"Divergence-based vector quantization","volume":"23","author":"Villmann","year":"2011","journal-title":"Neural Comput."},{"key":"ref_41","unstructured":"Huang, J.B., and Ahuja, N. (2012, January 11\u201315). Saliency detection via divergence analysis: A unified perspective. Proceedings of the 2012 21st International Conference on Pattern Recognition (ICPR), Tsukuba, Japan."},{"key":"ref_42","unstructured":"Pardo, L. (2005). Statistical Inference Based on Divergence Measures, CRC Press."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Basu, A., Shioya, H., and Park, C. (2011). Statistical Inference: The Minimum Distance Approach, CRC Press.","DOI":"10.1201\/b10956"},{"key":"ref_44","unstructured":"Hinton, G.E., and Roweis, S.T. (2002). Advances in Neural Information Processing Systems 15 (NIPS), MIT Press."},{"key":"ref_45","first-page":"2579","article-title":"Visualizing data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_46","unstructured":"Carreira-Perpi\u00f1\u00e1n, M.\u00c1. (2010, January 21\u201325). The Elastic Embedding Algorithm for Dimensionality Reduction. Proceedings of the International Conference on Machine Learning, Haifa, Israel."},{"key":"ref_47","unstructured":"Sun, K., and Marchand-Maillet, S. (2014, January 21\u201326). An Information Geometry of Statistical Manifold Learning. Proceedings of the International Conference on Machine Learning, Beijing, China."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1155\/S0161171201005658","article-title":"Generalizations of H\u00f6lder\u2019s inequality","volume":"26","author":"Cheung","year":"2001","journal-title":"Int. J. Math. Math. Sci."},{"key":"ref_49","unstructured":"Hazewinkel, M. (2001). Encyclopedia of Mathematics, Kluwer Academic Publishers."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1186\/1029-242X-2013-151","article-title":"Generalizations of H\u00f6lder inequalities for Csisz\u00e1r\u2019s f -divergence","volume":"2013","author":"Chen","year":"2013","journal-title":"J. Inequal. Appl."},{"key":"ref_51","unstructured":"Nielsen, F., Sun, K., and Marchand-Maillet, S. On H\u00f6lder Projective Divergences. Available online: https:\/\/www.lix.polytechnique.fr\/~nielsen\/HPD\/."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Boyd, S., and Vandenberghe, L. (2004). Convex Optimization, Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/3\/122\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:30:37Z","timestamp":1760207437000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/3\/122"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,3,16]]},"references-count":52,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2017,3]]}},"alternative-id":["e19030122"],"URL":"https:\/\/doi.org\/10.3390\/e19030122","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,3,16]]}}}