{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T10:21:22Z","timestamp":1773656482516,"version":"3.50.1"},"reference-count":73,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2018,5,19]],"date-time":"2018-05-19T00:00:00Z","timestamp":1526688000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>This paper is focused on f-divergences, consisting of three main contributions. The first one introduces integral representations of a general f-divergence by means of the relative information spectrum. The second part provides a new approach for the derivation of f-divergence inequalities, and it exemplifies their utility in the setup of Bayesian binary hypothesis testing. The last part of this paper further studies the local behavior of f-divergences.<\/jats:p>","DOI":"10.3390\/e20050383","type":"journal-article","created":{"date-parts":[[2018,5,21]],"date-time":"2018-05-21T04:07:30Z","timestamp":1526875650000},"page":"383","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":40,"title":["On f-Divergences: Integral Representations, Local Behavior, and Inequalities"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5681-1273","authenticated-orcid":false,"given":"Igal","family":"Sason","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Technion-Israel Institute of Technology, Haifa 3200003, Israel"}]}],"member":"1968","published-online":{"date-parts":[[2018,5,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"621","DOI":"10.1016\/j.sigpro.2012.09.003","article-title":"Divergence measures for statistical data processing\u2014An annotated bibliography","volume":"93","author":"Basseville","year":"2013","journal-title":"Signal Process."},{"key":"ref_2","unstructured":"Liese, F., and Vajda, I. (1987). Convex Statistical Distances. Teubner-Texte Zur Mathematik, Springer."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"4394","DOI":"10.1109\/TIT.2006.881731","article-title":"On divergences and informations in statistics and information theory","volume":"52","author":"Liese","year":"2006","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_4","first-page":"731","article-title":"Information, divergence and risk for binary experiments","volume":"12","author":"Reid","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Tsybakov, A.B. (2009). Introduction to Nonparametric Estimation, Springer.","DOI":"10.1007\/b13794"},{"key":"ref_6","unstructured":"Vapnik, V.N. (1998). Statistical Learning Theory, John Wiley & Sons."},{"key":"ref_7","unstructured":"Verd\u00fa, S. (2018). Information Theory, Unpublished work."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"261","DOI":"10.3390\/e10030261","article-title":"Axiomatic characterization of information measures","volume":"10","year":"2008","journal-title":"Entropy"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1111\/j.2517-6161.1966.tb00626.x","article-title":"A general class of coefficients of divergence of one distribution from another","volume":"28","author":"Ali","year":"1966","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_10","first-page":"85","article-title":"Eine Informationstheoretische Ungleichung und ihre Anwendung auf den Bewis der Ergodizit\u00e4t von Markhoffschen Ketten","volume":"8","year":"1963","journal-title":"Magyer Tud. Akad. Mat. Kutato Int. Koezl."},{"key":"ref_11","first-page":"185","article-title":"A note on Jensen\u2019s inequality","volume":"1","year":"1966","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_12","first-page":"299","article-title":"Information-type measures of difference of probability distributions and indirect observations","volume":"2","year":"1967","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_13","first-page":"329","article-title":"On topological properties of f-divergences","volume":"2","year":"1967","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1143\/JPSJ.18.328","article-title":"Markov processes and the H-theorem","volume":"18","author":"Morimoto","year":"1963","journal-title":"J. Phys. Soc. Jpn."},{"key":"ref_15","first-page":"690","article-title":"\u03c6-divergences, sufficiency, Bayes sufficiency, and deficiency","volume":"48","author":"Liese","year":"2012","journal-title":"Kybernetika"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1214\/aoms\/1177704567","article-title":"Uncertainty, information and sequential experiments","volume":"33","author":"DeGroot","year":"1962","journal-title":"Ann. Math. Stat."},{"key":"ref_17","unstructured":"Cohen, J.E., Kemperman, J.H.B., and Zb\u0103ganu, G. (1998). Comparisons of Stochastic Matrices with Applications in Information Theory, Statistics, Economics and Population, Springer."},{"key":"ref_18","first-page":"191","article-title":"A note on f-divergences","volume":"24","author":"Feldman","year":"1989","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_19","unstructured":"Guttenbrunner, C. (1992, January 26\u201331). On applications of the representation of f-divergences as averaged minimal Bayesian risk. Proceedings of the Transactions of the 11th Prague Conferences on Information Theory, Statistical Decision Functions, and Random Processes, Prague, Czechoslovakia."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1036","DOI":"10.1109\/18.256536","article-title":"Statistical information and discrimination","volume":"39","author":"Vajda","year":"1993","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Torgersen, E. (1991). Comparison of Statistical Experiments, Cambridge University Press.","DOI":"10.1017\/CBO9780511666353"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"5973","DOI":"10.1109\/TIT.2016.2603151","article-title":"f-divergence inequalities","volume":"62","author":"Sason","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1111\/j.1751-5823.2002.tb00178.x","article-title":"On choosing and bounding probability metrics","volume":"70","author":"Gibbs","year":"2002","journal-title":"Int. Stat. Rev."},{"key":"ref_24","first-page":"1295","article-title":"Some inequalities for Csisz\u00e1r-divergence measures","volume":"3","author":"Anwar","year":"2009","journal-title":"Int. J. Math. Anal."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"37359","DOI":"10.1155\/2007\/37359","article-title":"On logarithmic convexity for differences of power means","volume":"2007","author":"Simic","year":"2007","journal-title":"J. Inequal. Appl."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2671","DOI":"10.1016\/j.spl.2008.03.007","article-title":"On a new moments inequality","volume":"78","author":"Simic","year":"2008","journal-title":"Stat. Probab. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1007\/s10474-009-8205-z","article-title":"On certain new inequalities in information theory","volume":"124","author":"Simic","year":"2009","journal-title":"Acta Math. Hung."},{"key":"ref_28","unstructured":"Simic, S. (2016, May 13). Moment Inequalities of the Second and Third Orders. Preprint. Available online: http:\/\/arxiv.org\/abs\/1509.0851."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3230","DOI":"10.1109\/TIT.2011.2137353","article-title":"On pairs of f-divergences and their joint range","volume":"57","author":"Vajda","year":"2011","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sason, I., and Verd\u00fa, S. (2016, January 16\u201318). f-divergence inequalities via functional domination. Proceedings of the 2016 IEEE International Conference on the Science of Electrical Engineering, Eilat, Israel.","DOI":"10.1109\/ICSEE.2016.7806028"},{"key":"ref_31","first-page":"1","article-title":"Refinement inequalities among symmetric divergence measures","volume":"2","author":"Taneja","year":"2005","journal-title":"Aust. J. Math. Anal. Appl."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"198","DOI":"10.3390\/info4020198","article-title":"Seven means, generalized triangular discrimination, and generating divergence measures","volume":"4","author":"Taneja","year":"2013","journal-title":"Information"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1109\/TIT.2013.2288674","article-title":"Sharp inequalities for f-divergences","volume":"60","author":"Guntuboyina","year":"2014","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1858","DOI":"10.1109\/TIT.2003.813506","article-title":"A new metric for probability distributions","volume":"49","author":"Endres","year":"2003","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_35","first-page":"415","article-title":"On powers of f-divergences defining a distance","volume":"26","author":"Kafka","year":"1991","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"361","DOI":"10.3390\/info6030361","article-title":"A class of new metrics based on triangular discrimination","volume":"6","author":"Lu","year":"2015","journal-title":"Information"},{"key":"ref_37","first-page":"885","article-title":"On metric divergences of probability measures","volume":"45","author":"Vajda","year":"2009","journal-title":"Kybernetika"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"5377","DOI":"10.1109\/TIT.2010.2068710","article-title":"On Pinsker\u2019s and Vajda\u2019s type inequalities for Csisz\u00e1r\u2019s f-divergences","volume":"56","author":"Gilardoni","year":"2010","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1602","DOI":"10.1109\/18.850703","article-title":"Some inequalities for information divergence and related measures of discrimination","volume":"46","year":"2000","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Sason, I., and Verd\u00fa, S. (2015, January 11\u201315). Upper bounds on the relative entropy and R\u00e9nyi divergence as a function of total variation distance for finite alphabets. Proceedings of the 2015 IEEE Information Theory Workshop, Jeju Island, Korea.","DOI":"10.1109\/ITWF.2015.7360766"},{"key":"ref_41","unstructured":"Dragomir, S.S. (2000). Upper and lower bounds for Csisz\u00e1r f-divergence in terms of the Kullback-Leibler divergence and applications. Inequalities for Csisz\u00e1r f-Divergence in Information Theory, RGMIA Monographs, Victoria University."},{"key":"ref_42","unstructured":"Dragomir, S.S. (2000). Upper and lower bounds for Csisz\u00e1r f-divergence in terms of Hellinger discrimination and applications. Inequalities for Csisz\u00e1r f-Divergence in Information Theory, RGMIA Monographs, Victoria University."},{"key":"ref_43","unstructured":"Dragomir, S.S. (2000). An upper bound for the Csisz\u00e1r f-divergence in terms of the variational distance and applications. Inequalities for Csisz\u00e1r f-Divergence in Information Theory, RGMIA Monographs, Victoria University."},{"key":"ref_44","first-page":"97","article-title":"Some inequalities for the Kullback-Leibler and \u03c72-distances in information theory and applications","volume":"17","author":"Dragomir","year":"2001","journal-title":"Tamsui Oxf. J. Math. Sci."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"471","DOI":"10.1017\/S000497270004051X","article-title":"Bounds for the normalized Jensen functional","volume":"74","author":"Dragomir","year":"2006","journal-title":"Bull. Aust. Math. Soc."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"575","DOI":"10.1016\/j.camwa.2004.07.017","article-title":"A symmetric information divergence measure of the Csisz\u00e1r\u2019s f-divergence class and its bounds","volume":"49","author":"Kumar","year":"2005","journal-title":"Comp. Math. Appl."},{"key":"ref_47","first-page":"115","article-title":"Bounds on non-symmetric divergence measures in terms of symmetric divergence measures","volume":"29","author":"Taneja","year":"2005","journal-title":"J. Comb. Inf. Syst. Sci."},{"key":"ref_48","unstructured":"Binette, O. (2018, May 14). A note on reverse Pinsker inequalities. Preprint. Available online: http:\/\/arxiv.org\/abs\/1805.05135."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"763","DOI":"10.1016\/j.crma.2006.10.027","article-title":"On the minimum f-divergence for given total variation","volume":"343","author":"Gilardoni","year":"2006","journal-title":"C. R. Math."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1016\/j.crma.2010.02.006","article-title":"Corrigendum to the note on the minimum f-divergence for given total variation","volume":"348","author":"Gilardoni","year":"2010","journal-title":"C. R. Math."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"304","DOI":"10.3103\/S1066530716040049","article-title":"The minimum increment of f-divergences given total variation distances","volume":"25","author":"Gushchin","year":"2016","journal-title":"Math. Methods Stat."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"701","DOI":"10.1109\/TIT.2014.2387065","article-title":"Tight bounds on symmetric divergence measures and a refined bound for lossless source coding","volume":"61","author":"Sason","year":"2015","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1109\/TIT.2015.2504100","article-title":"On the R\u00e9nyi divergence, joint range of relative entropies, and a channel coding theorem","volume":"62","author":"Sason","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_54","first-page":"2629","article-title":"E\u03b3-resolvability","volume":"63","author":"Liu","year":"2017","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1561\/0100000004","article-title":"Information Theory and Statistics: A Tutorial","volume":"1","author":"Shields","year":"2004","journal-title":"Found. Trends Commun. Inf. Theory"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1860","DOI":"10.1109\/TIT.2003.813509","article-title":"On asymptotic properties of information-theoretic divergences","volume":"49","author":"Pardo","year":"2003","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"2307","DOI":"10.1109\/TIT.2010.2043769","article-title":"Channel coding rate in the finite blocklength regime","volume":"56","author":"Polyanskiy","year":"2010","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_58","first-page":"119","article-title":"Estimation des densit\u00e9s: Risque minimax","volume":"47","author":"Bretagnolle","year":"1979","journal-title":"Probab. Theory Relat. Fields"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"771","DOI":"10.1109\/TIT.1970.1054557","article-title":"Note on discrimination information and variation","volume":"16","author":"Vajda","year":"1970","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_60","first-page":"453","article-title":"An invariant form for the prior probability in estimation problems","volume":"186","author":"Jeffreys","year":"1946","journal-title":"Proc. R. Soc. Lond. Ser. A Math. Phys. Sci."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1080\/14786440009463897","article-title":"On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling","volume":"50","author":"Pearson","year":"1900","journal-title":"Lond. Edinb. Dublin Philos. Mag. J. Sci."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Le Cam, L. (1986). Asymptotic Methods in Statistical Decision Theory, Springer.","DOI":"10.1007\/978-1-4612-4946-7"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1109\/TCOM.1967.1089532","article-title":"The divergence and Bhattacharyya distance measures in signal selection","volume":"15","author":"Kailath","year":"1967","journal-title":"IEEE Trans. Commun. Technol."},{"key":"ref_64","unstructured":"Amari, S.I., and Nagaoka, H. (2000). Methods of Information Geometry, Oxford University Press."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3390\/e12061532","article-title":"Families of Alpha- Beta- and Gamma-divergences: Flexible and robust measures of similarities","volume":"12","author":"Cichocki","year":"2010","journal-title":"Entropy"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"134","DOI":"10.3390\/e13010134","article-title":"Generalized Alpha-Beta divergences and their application to robust nonnegative matrix factorization","volume":"13","author":"Cichocki","year":"2011","journal-title":"Entropy"},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"2988","DOI":"10.3390\/e17052988","article-title":"Log-determinant divergences revisited: Alpha-Beta and Gamma log-det divergences","volume":"17","author":"Cichocki","year":"2015","journal-title":"Entropy"},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1109\/18.61115","article-title":"Divergence measures based on the Shannon entropy","volume":"37","author":"Lin","year":"1991","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1109\/TIT.2015.2482978","article-title":"Dissipation of information in channels with input constraints","volume":"62","author":"Polyanskiy","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1109\/TIT.1967.1053968","article-title":"A lower bound for discrimination information in terms of variation","volume":"13","author":"Kullback","year":"1967","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"2156","DOI":"10.1214\/aoms\/1177697293","article-title":"On the optimal rate of transmitting information","volume":"40","author":"Kemperman","year":"1969","journal-title":"Ann. Math. Stat."},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1007\/BF02124750","article-title":"On the Lambert W function","volume":"5","author":"Corless","year":"1996","journal-title":"Adv. Comput. Math."},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"3797","DOI":"10.1109\/TIT.2014.2320500","article-title":"R\u00e9nyi divergence and Kullback-Leibler divergence","volume":"60","year":"2014","journal-title":"IEEE Trans. Inf. Theory"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/5\/383\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:05:00Z","timestamp":1760195100000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/5\/383"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,5,19]]},"references-count":73,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2018,5]]}},"alternative-id":["e20050383"],"URL":"https:\/\/doi.org\/10.3390\/e20050383","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,5,19]]}}}