{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:27:39Z","timestamp":1760243259299,"version":"build-2065373602"},"reference-count":29,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2014,5,13]],"date-time":"2014-05-13T00:00:00Z","timestamp":1399939200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Divergence is a discrepancy measure between two objects, such as functions, vectors, matrices, and so forth. In particular, divergences defined on probability distributions are widely employed in probabilistic forecasting. As the dissimilarity measure, the divergence should satisfy some conditions. In this paper, we consider two conditions: The first one is the scale-invariance property and the second is that the divergence is approximated by the sample mean of a loss function. The first requirement is an important feature for dissimilarity measures. The divergence will depend on which system of measurements we used to measure the objects. Scale-invariant divergence is transformed in a consistent way when the system of measurements is changed to the other one. The second requirement is formalized such that the divergence is expressed by using the so-called composite score. We study the relation between composite scores and scale-invariant divergences, and we propose a new class of divergences called H\u00a8older divergence that satisfies two conditions above. We present some theoretical properties of H\u00a8older divergence. We show that H\u00a8older divergence unifies existing divergences from the viewpoint of scale-invariance.<\/jats:p>","DOI":"10.3390\/e16052611","type":"journal-article","created":{"date-parts":[[2014,5,13]],"date-time":"2014-05-13T11:44:55Z","timestamp":1399981495000},"page":"2611-2628","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Scale-Invariant Divergences for Density Functions"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6878-5850","authenticated-orcid":false,"given":"Takafumi","family":"Kanamori","sequence":"first","affiliation":[{"name":"Nagoya University, Furocho, Chikusaku, Nagoya 464-8603, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2014,5,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"338","DOI":"10.1175\/1520-0493(2004)132<0338:PFOPIT>2.0.CO;2","article-title":"Probabilistic forecasts of precipitation in terms of quantiles using NWP model output","volume":"132","author":"Bremnes","year":"2004","journal-title":"Mon. Weather Rev"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1175\/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2","article-title":"Verification of forecasts expressed in terms of probability","volume":"78","author":"Brier","year":"1950","journal-title":"Mon. Weather Rev"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"749","DOI":"10.3905\/jod.1997.407971","article-title":"An overview of value at risk","volume":"4","author":"Duffie","year":"1997","journal-title":"J. Deriv"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Kanamori, T., and Fujisawa, H. (2014). Affine invariant divergences associated with composite scores and its applications. Bernoulli, in press.","DOI":"10.3150\/13-BEJ557"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1198\/016214506000001437","article-title":"Strictly proper scoring rules, prediction, and estimation","volume":"102","author":"Gneiting","year":"2007","journal-title":"J. Am. Stat. Assoc"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"3884","DOI":"10.1109\/TSP.2010.2047340","article-title":"A study on invariance of f-divergence and its application to speech recognition","volume":"58","author":"Qiao","year":"2010","journal-title":"IEEE Trans. Signal Process"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"593","DOI":"10.1214\/12-AOS972","article-title":"Proper local scoring rules on discrete sample spaces","volume":"40","author":"Dawid","year":"2012","journal-title":"Ann. Stat"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"561","DOI":"10.1214\/12-AOS971","article-title":"Proper local scoring rules","volume":"40","author":"Parry","year":"2012","journal-title":"Ann. Stat"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1916","DOI":"10.1214\/aoms\/1177693057","article-title":"Proper scores for probability forecasters","volume":"42","author":"Hendrickson","year":"1971","journal-title":"Ann. Math. Stat"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/0041-5553(67)90040-7","article-title":"The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex prog ramming","volume":"7","author":"Bregman","year":"1967","journal-title":"USSR Comput. Math. Math. Phys"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1111\/j.2517-6161.1966.tb00626.x","article-title":"A general class of coefficients of divergence of one distribution from another","volume":"28","author":"Ali","year":"1966","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_13","first-page":"229","article-title":"Information-type measures of difference of probability distributions and indirect observation","volume":"2","year":"1967","journal-title":"Stud. Sci. Math. Hung"},{"key":"ref_14","unstructured":"Borwein, J.M., and Zhu, Q.Q.J. (2005). Techniques of Variational Analysis, Springer Science + Business Media, Incorporated."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1437","DOI":"10.1162\/089976604323057452","article-title":"Information geometry of U-boost and bregman divergence","volume":"16","author":"Murata","year":"2004","journal-title":"Neural Comput"},{"key":"ref_16","unstructured":"Collins, M., Schapire, R.E., and Singer, Y. (July, January 28). Logistic regression, adaBoost and bregman distances, Palo Alto, CA, USA."},{"key":"ref_17","first-page":"1705","article-title":"Clustering with bregman divergences","volume":"6","author":"Banerjee","year":"2005","journal-title":"J. Mach. Learn. Res"},{"key":"ref_18","unstructured":"Itakura, F., and Saito, S. (1968, January 21\u201328). Analysis synthesis telephony based on the maximum likelihood method, Tokyo, Japan."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"793","DOI":"10.1162\/neco.2008.04-08-771","article-title":"Nonnegative matrix factorization with the itakura-saito divergence: With application to music analysis","volume":"21","author":"Bertin","year":"2009","journal-title":"Neural Comput"},{"key":"ref_20","unstructured":"Fevotte, C., and Cemgil, A.T. (2009, January 24\u201328). Nonnegative matrix factorisations as probabilistic inference in composite models, Glasgow, Scotland."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"549","DOI":"10.1093\/biomet\/85.3.549","article-title":"Robust and efficient estimation by minimising a density power divergence","volume":"85","author":"Basu","year":"1998","journal-title":"Biometrika"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"865","DOI":"10.1093\/biomet\/88.3.865","article-title":"A comparison of related density-based minimum divergence estimators","volume":"88","author":"Jones","year":"2001","journal-title":"Biometrika"},{"key":"ref_23","unstructured":"Godambe, V.P., and Sprott, D.A. (1971). Comment on \u201cMeasuring Information and Uncertainty,\u201d by R. J. Buehler, Foundations of Statistical Inference."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2053","DOI":"10.1016\/j.jmva.2008.02.004","article-title":"Robust parameter estimation with a small bias against heavy contamination","volume":"99","author":"Fujisawa","year":"2008","journal-title":"J. Multivar. Anal"},{"key":"ref_25","unstructured":"Amari, S., and Nagaoka, H. (2000). Methods of Information Geometry: Translations of Mathematical Monographs, Oxford University Press."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3390\/e12061532","article-title":"Families of alpha- beta- and gamma- divergences: Flexible and robust measures of similarities","volume":"12","author":"Cichocki","year":"2010","journal-title":"Entropy"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Berger, J.O. (1985). Statistical Decision Theory and Bayesian Analysis, Springer.","DOI":"10.1007\/978-1-4757-4286-2"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"4925","DOI":"10.1109\/TIT.2009.2030485","article-title":"Alpha-divergence is unique, belonging to both f-divergence and Bregman divergence classes","volume":"55","author":"Amari","year":"2009","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1288","DOI":"10.1109\/18.605597","article-title":"About distances of discrete distributions satisfying the data processing theorem of information theory","volume":"43","author":"Pardo","year":"1997","journal-title":"IEEE Trans. Inf. Theory"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/16\/5\/2611\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:11:21Z","timestamp":1760217081000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/16\/5\/2611"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,5,13]]},"references-count":29,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2014,5]]}},"alternative-id":["e16052611"],"URL":"https:\/\/doi.org\/10.3390\/e16052611","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2014,5,13]]}}}