{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,28]],"date-time":"2025-12-28T22:47:12Z","timestamp":1766962032741,"version":"build-2065373602"},"reference-count":63,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2020,5,18]],"date-time":"2020-05-18T00:00:00Z","timestamp":1589760000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The relative entropy and the chi-squared divergence are fundamental divergence measures in information theory and statistics. This paper is focused on a study of integral relations between the two divergences, the implications of these relations, their information-theoretic applications, and some generalizations pertaining to the rich class of f-divergences. Applications that are studied in this paper refer to lossless compression, the method of types and large deviations, strong data\u2013processing inequalities, bounds on contraction coefficients and maximal correlation, and the convergence rate to stationarity of a type of discrete-time Markov chains.<\/jats:p>","DOI":"10.3390\/e22050563","type":"journal-article","created":{"date-parts":[[2020,5,18]],"date-time":"2020-05-18T11:34:14Z","timestamp":1589801654000},"page":"563","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["On Relations Between the Relative Entropy and \u03c72-Divergence, Generalizations and Applications"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1446-399X","authenticated-orcid":false,"given":"Tomohiro","family":"Nishiyama","sequence":"first","affiliation":[{"name":"Independent Researcher, Tokyo 206\u20130003, Japan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5681-1273","authenticated-orcid":false,"given":"Igal","family":"Sason","sequence":"additional","affiliation":[{"name":"Faculty of Electrical Engineering, Technion\u2014Israel Institute of Technology, Technion City, Haifa 3200003, Israel"}]}],"member":"1968","published-online":{"date-parts":[[2020,5,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1080\/14786440009463897","article-title":"On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling","volume":"50","author":"Pearson","year":"1900","journal-title":"Lond. Edinb. Dublin Philos. Mag. J. Sci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1561\/0100000004","article-title":"Information Theory and Statistics: A Tutorial","volume":"1","author":"Shields","year":"2004","journal-title":"Found. Trends Commun. Inf. Theory"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1111\/j.2517-6161.1966.tb00626.x","article-title":"A general class of coefficients of divergence of one distribution from another","volume":"28","author":"Ali","year":"1966","journal-title":"J. R. Stat. Soc."},{"key":"ref_5","first-page":"85","article-title":"Eine Informationstheoretische Ungleichung und ihre Anwendung auf den Bewis der Ergodizit\u00e4t von Markhoffschen Ketten","volume":"8","year":"1963","journal-title":"Publ. Math. Inst. Hungar. Acad. Sci."},{"key":"ref_6","first-page":"299","article-title":"Information-type measures of difference of probability distributions and indirect observations","volume":"2","year":"1967","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_7","first-page":"329","article-title":"On topological properties of f-divergences","volume":"2","year":"1967","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1007\/BF02018661","article-title":"A class of measures of informativity of observation channels","volume":"2","year":"1972","journal-title":"Period. Math. Hung."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3797","DOI":"10.1109\/TIT.2014.2320500","article-title":"R\u00e9nyi divergence and Kullback\u2013Leibler divergence","volume":"60","year":"2014","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_10","unstructured":"R\u00e9nyi, A. (1961). On measures of entropy and information. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Volume 1: Contributions to the Theory of Statistics, University of California Press."},{"key":"ref_11","unstructured":"Liese, F., and Vajda, I. (1987). Convex Statistical Distances, Teubner-Texte Zur Mathematik."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"4394","DOI":"10.1109\/TIT.2006.881731","article-title":"On divergences and informations in statistics and information theory","volume":"52","author":"Liese","year":"2006","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"404","DOI":"10.1214\/aoms\/1177704567","article-title":"Uncertainty, information and sequential experiments","volume":"33","author":"DeGroot","year":"1962","journal-title":"Ann. Math. Stat."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2307","DOI":"10.1109\/TIT.2010.2043769","article-title":"Channel coding rate in the finite blocklength regime","volume":"56","author":"Polyanskiy","year":"2010","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"5973","DOI":"10.1109\/TIT.2016.2603151","article-title":"f-divergence inequalities","volume":"62","author":"Sason","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Sason, I. (2018). On f-divergences: Integral representations, local behavior, and inequalities. Entropy, 20.","DOI":"10.3390\/e20050383"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Melbourne, J., Madiman, M., and Salapaka, M.V. (2019, January 24\u201327). Relationships between certain f-divergences. Proceedings of the 57th Annual Allerton Conference on Communication, Control and Computing, Urbana, IL, USA.","DOI":"10.1109\/ALLERTON.2019.8919677"},{"key":"ref_18","unstructured":"Melbourne, J., Talukdar, S., Bhaban, S., Madiman, M., and Salapaka, M.V. (2020, April 22). The Differential Entropy of Mixtures: New Bounds and Applications. Available online: https:\/\/arxiv.org\/pdf\/1805.11257.pdf."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"112202","DOI":"10.1063\/1.4901039","article-title":"Quantum skew divergence","volume":"55","author":"Audenaert","year":"2014","journal-title":"J. Math. Phys."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1111\/j.1751-5823.2002.tb00178.x","article-title":"On choosing and bounding probability metrics","volume":"70","author":"Gibbs","year":"2002","journal-title":"Int. Stat. Rev."},{"key":"ref_21","first-page":"239","article-title":"A class of modified Pearson and Neyman statistics","volume":"19","author":"Vajda","year":"2001","journal-title":"Stat. Decis."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Le Cam, L. (1986). Asymptotic Methods in Statistical Decision Theory, Springer.","DOI":"10.1007\/978-1-4612-4946-7"},{"key":"ref_23","unstructured":"Gani, J., and Rohatgi, V.K. (1981). On the concept and measure of information contained in an observation. Contributions to Probability, Academic Press."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Nishiyama, T. (2019, November 02). A New Lower Bound for Kullback\u2013Leibler Divergence Based on Hammersley-Chapman-Robbins Bound. Available online: https:\/\/arxiv.org\/abs\/1907.00288v3.","DOI":"10.31219\/osf.io\/wa98j"},{"key":"ref_25","unstructured":"Sason, I. (2018). On Csisz\u00e1r\u2019s f-divergences and informativities with applications. Workshop on Channels, Statistics, Information, Secrecy and Randomness for the 80th birthday of I. Csisz\u00e1r, The R\u00e9nyi Institute of Mathematics, Hungarian Academy of Sciences."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"5704","DOI":"10.1109\/TIT.2018.2839743","article-title":"Comparison of channels: Criteria for domination by a symmetric channel","volume":"64","author":"Makur","year":"2018","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"2671","DOI":"10.1016\/j.spl.2008.03.007","article-title":"On a new moments inequality","volume":"78","author":"Simic","year":"2008","journal-title":"Stat. Probab. Lett."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1214\/aoms\/1177729548","article-title":"Minimum variance estimation without regularity assumptions","volume":"22","author":"Chapman","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"192","DOI":"10.1111\/j.2517-6161.1950.tb00056.x","article-title":"On estimating restricted parameters","volume":"12","author":"Hammersley","year":"1950","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_30","unstructured":"Verd\u00fa, S. Information Theory, in preparation."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"5116","DOI":"10.1109\/TIT.2014.2338852","article-title":"Beyond the entropy power inequality, via rearrangments","volume":"60","author":"Wang","year":"2014","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_32","unstructured":"Lewin, L. (1981). Polylogarithms and Associated Functions, Elsevier North Holland."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"857","DOI":"10.1214\/aop\/1039639365","article-title":"Bounding d\u00af-distance by informational divergence: A method to prove measure concentration","volume":"24","author":"Marton","year":"1996","journal-title":"Ann. Probab."},{"key":"ref_34","first-page":"9","article-title":"Distance-divergence inequalities","volume":"64","author":"Marton","year":"2014","journal-title":"IEEE Inf. Theory Soc. Newsl."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Boucheron, S., Lugosi, G., and Massart, P. (2013). Concentration Inequalities\u2014A Nonasymptotic Theory of Independence, Oxford University Press.","DOI":"10.1093\/acprof:oso\/9780199535255.001.0001"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Raginsky, M., and Sason, I. (2018). Concentration of Measure Inequalities in Information Theory, Communications and Coding: Third Edition. Foundations and Trends in Communications and Information Theory, NOW Publishers.","DOI":"10.1561\/9781680835359"},{"key":"ref_37","first-page":"768","article-title":"Sanov property, generalized I-projection and a conditional limit theorem","volume":"12","year":"1984","journal-title":"Ann. Probab."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1109\/18.54897","article-title":"Information-theoretic asymptotics of Bayes methods","volume":"36","author":"Clarke","year":"1990","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1137\/1030059","article-title":"The entropy of a Poisson distribution","volume":"30","author":"Evans","year":"1988","journal-title":"SIAM Rev."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/S0893-9659(98)00013-5","article-title":"Integral representations and asymptotic expansions for Shannon and R\u00e9nyi entropies","volume":"11","author":"Knessl","year":"1998","journal-title":"Appl. Math. Lett."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Merhav, N., and Sason, I. (2020). An integral representation of the logarithmic function with applications in information theory. Entropy, 22.","DOI":"10.3390\/e22060707"},{"key":"ref_42","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, John Wiley & Sons. [2nd ed.]."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"2505","DOI":"10.1109\/18.720546","article-title":"The method of types","volume":"44","year":"1998","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1007\/BF02124750","article-title":"On the Lambert W function","volume":"5","author":"Corless","year":"1996","journal-title":"Adv. Comput. Math."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Tamm, U. (2014, January 9\u201314). Some refelections about the Lambert W function as inverse of x\u00b7log(x). Proceedings of the 2014 IEEE Information Theory and Applications Workshop, San Diego, CA, USA.","DOI":"10.1109\/ITA.2014.6804273"},{"key":"ref_46","unstructured":"Cohen, J.E., Kemperman, J.H.B., and Zb\u0103ganu, G. (1998). Comparison of Stochastic Matrices with Applications in Information Theory, Statistics, Economics and Population Sciences, Birkh\u00e4user."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1016\/0024-3795(93)90331-H","article-title":"Relative entropy under mappings by stochastic matrices","volume":"179","author":"Cohen","year":"1993","journal-title":"Linear Algebra Its Appl."},{"key":"ref_48","unstructured":"Makur, A., and Zheng, L. (October, January 29). Bounds between contraction coefficients. Proceedings of the 53rd Annual Allerton Conference on Communication, Control and Computing, Urbana, IL, USA."},{"key":"ref_49","unstructured":"Makur, A. (2019). Information Contraction and Decomposition. [Ph.D. Thesis, MIT]."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1007\/978-1-4939-7005-6_7","article-title":"Strong data processing inequalities for channels and Bayesian networks","volume":"Volume 161","author":"Carlen","year":"2017","journal-title":"Convexity and Concentration"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"3355","DOI":"10.1109\/TIT.2016.2549542","article-title":"Strong data processing inequalities and \u03a6-Sobolev inequalities for discrete channels","volume":"62","author":"Raginsky","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Sason, I. (2019). On data-processing and majorization inequalities for f-divergences with applications. Entropy, 21.","DOI":"10.3390\/e21101022"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Csisz\u00e1r, I., and K\u00f6rner, J. (2011). Information Theory: Coding Theorems for Discrete Memoryless Systems, Cambridge University Press. [2nd ed.].","DOI":"10.1017\/CBO9780511921889"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1109\/TIT.1982.1056497","article-title":"On the convexity of some divergence measures based on entropy functions","volume":"28","author":"Burbea","year":"1982","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1109\/18.61115","article-title":"Divergence measures based on the Shannon entropy","volume":"37","author":"Lin","year":"1991","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1016\/S0016-0032(96)00063-4","article-title":"The Jensen\u2013Shannon divergence","volume":"334","author":"Pardo","year":"1997","journal-title":"J. Frankl. Inst."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"1602","DOI":"10.1109\/18.850703","article-title":"Some inequalities for information divergence and related measures of discrimination","volume":"46","year":"2000","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Nielsen, F. (2020). On a generalization of the Jensen\u2013Shannon divergence and the Jensen\u2013Shannon centroids. Entropy, 22.","DOI":"10.3390\/e22020221"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"2316","DOI":"10.1109\/TIT.2018.2877608","article-title":"Mixture models, Bayes Fisher information, and divergence measures","volume":"65","author":"Asadi","year":"2019","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_60","first-page":"207","article-title":"Maximum correlation coefficient (non-symmetric case)","volume":"2","author":"Sarmanov","year":"1962","journal-title":"Sel. Transl. Math. Stat. Probab."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1016\/j.crma.2010.02.006","article-title":"Corrigendum to the note on the minimum f-divergence for given total variation","volume":"348","author":"Gilardoni","year":"2010","journal-title":"Comptes Rendus Math."},{"key":"ref_62","first-page":"731","article-title":"Information, divergence and risk for binary experiments","volume":"12","author":"Reid","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"1860","DOI":"10.1109\/TIT.2003.813509","article-title":"On asymptotic properties of information-theoretic divergences","volume":"49","author":"Pardo","year":"2003","journal-title":"IEEE Trans. Inf. Theory"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/5\/563\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T09:30:00Z","timestamp":1760175000000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/22\/5\/563"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,18]]},"references-count":63,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2020,5]]}},"alternative-id":["e22050563"],"URL":"https:\/\/doi.org\/10.3390\/e22050563","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2020,5,18]]}}}