{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T07:47:14Z","timestamp":1762069634481,"version":"build-2065373602"},"reference-count":22,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2022,10,4]],"date-time":"2022-10-04T00:00:00Z","timestamp":1664841600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Natural Sciences and Engineering Research Council (NSERC) of Canada"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Two R\u00e9nyi-type generalizations of the Shannon cross-entropy, the R\u00e9nyi cross-entropy and the Natural R\u00e9nyi cross-entropy, were recently used as loss functions for the improved design of deep learning generative adversarial networks. In this work, we derive the R\u00e9nyi and Natural R\u00e9nyi differential cross-entropy measures in closed form for a wide class of common continuous distributions belonging to the exponential family, and we tabulate the results for ease of reference. We also summarise the R\u00e9nyi-type cross-entropy rates between stationary Gaussian processes and between finite-alphabet time-invariant Markov sources.<\/jats:p>","DOI":"10.3390\/e24101417","type":"journal-article","created":{"date-parts":[[2022,10,8]],"date-time":"2022-10-08T04:04:56Z","timestamp":1665201896000},"page":"1417","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["R\u00e9nyi Cross-Entropy Measures for Common Distributions and Processes with Memory"],"prefix":"10.3390","volume":"24","author":[{"given":"Ferenc Cole","family":"Thierrin","sequence":"first","affiliation":[{"name":"Department of Mathematics and Statistics, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7980-724X","authenticated-orcid":false,"given":"Fady","family":"Alajaji","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Statistics, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}]},{"given":"Tam\u00e1s","family":"Linder","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Statistics, Queen\u2019s University, Kingston, ON K7L 3N6, Canada"}]}],"member":"1968","published-online":{"date-parts":[[2022,10,4]]},"reference":[{"key":"ref_1","first-page":"547","article-title":"On measures of entropy and information","volume":"1","year":"1961","journal-title":"Fourth Berkeley Symp. Math. Stat. Probab."},{"key":"ref_2","unstructured":"Verd\u00fa, S. (2015, January 1\u20136). \u03b1-Mutual Information. Proceedings of the 2015 Information Theory and Applications Workshop (ITA), San Diego, CA, USA."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1007\/s42979-020-00403-9","article-title":"RGAN: R\u00e9nyi generative adversarial network","volume":"2","author":"Sarraf","year":"2021","journal-title":"SN Comput. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Valverde-Albacete, F.J., and Pel\u00e1ez-Moreno, C. (2019). The Case for shifting the R\u00e9nyi entropy. Entropy, 21.","DOI":"10.3390\/e21010046"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2473","DOI":"10.1162\/neco_a_01416","article-title":"Least kth-Order and R\u00e9nyi generative adversarial networks","volume":"33","author":"Bhatia","year":"2021","journal-title":"Neural Comput."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.ins.2013.06.018","article-title":"R\u00e9nyi divergence measures for commonly used univariate continuous distributions","volume":"249","author":"Gil","year":"2013","journal-title":"Inf. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/S0378-3758(00)00169-5","article-title":"R\u00e9nyi information, loglikelihood and an intrinsic distribution measure","volume":"93","author":"Song","year":"2001","journal-title":"J. Statist. Plann. Inference"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Thierrin, F.C., Alajaji, F., and Linder, T. (2022, January 5\u20138). On the R\u00e9nyi cross-entropy. Proceedings of the 17th Canadian Workshop on Information Theory, Ottawa, ON, Canada.","DOI":"10.1109\/CWIT55308.2022.9817662"},{"key":"ref_9","first-page":"2672","article-title":"Generative adversarial nets","volume":"27","author":"Goodfellow","year":"2014","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Bhatia, H., Paul, W., Alajaji, F., Gharesifard, B., and Burlina, P. (2020). R\u00e9nyi generative adversarial Networks. arXiv.","DOI":"10.1162\/neco_a_01416"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"122527","DOI":"10.1016\/j.physa.2019.122527","article-title":"On Jensen-R\u00e9nyi and Jeffreys-R\u00e9nyi type f-divergences induced by convex functions","volume":"548","author":"Kluza","year":"2019","journal-title":"Phys. Stat. Mech. Its Appl."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1109\/18.61115","article-title":"Divergence measures based on the Shannon entropy","volume":"31","author":"Lin","year":"1991","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_13","unstructured":"Pantazis, Y., Paul, D., Fasoulakis, M., Stylianou, Y., and Katsoulakis, M. (2020). Cumulant GAN. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Kurri, G.R., Sypherd, T., and Sankar, L. (2021, January 17\u201321). Realizing GANs via a tunable loss function. Proceedings of the IEEE Information Theory Workshop (ITW), Kanazawa, Japan.","DOI":"10.1109\/ITW48936.2021.9611499"},{"key":"ref_15","unstructured":"Kurri, G.R., Welfert, M., Sypherd, T., and Sankar, L. (July, January 26). \u03b1-GAN: Convergence and estimation guarantees. Proceedings of the IEEE International Symposium on Information Theory (ISIT), Espoo, Finland."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.1109\/18.923736","article-title":"R\u00e9nyi\u2019s divergence and entropy rates for finite alphabet Markov sources","volume":"47","author":"Rached","year":"2001","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_17","unstructured":"Seneta, E. (2006). Non-Negative Matrices and Markov Chains, Springer Science & Business Media."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Gallager, R.G. (1996). Discrete Stochastic Processes, Springer.","DOI":"10.1007\/978-1-4615-2329-1"},{"key":"ref_19","first-page":"85","article-title":"Eine Informationstheoretische Ungleichung und ihre Anwendung auf den Bewis der Ergodizitat on Markhoffschen Ketten","volume":"8","author":"Csiszar","year":"2006","journal-title":"Publ. Math. Inst. Hung. Acad. Sci. Ser. A"},{"key":"ref_20","first-page":"299","article-title":"Information-type measures of difference of probability distributions and indirect observations","volume":"2","year":"1967","journal-title":"Stud. Sci. Math. Hung."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1111\/j.2517-6161.1966.tb00626.x","article-title":"A general class of coefficients of divergence of one distribution from another","volume":"28","author":"Ali","year":"1966","journal-title":"J. R. Stat. Society. Ser. B (Methodol.)"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4394","DOI":"10.1109\/TIT.2006.881731","article-title":"On divergences and informations in statistics and information theory","volume":"52","author":"Liese","year":"2006","journal-title":"IEEE Trans. Inf. Theory"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/10\/1417\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:46:19Z","timestamp":1760143579000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/10\/1417"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,10,4]]},"references-count":22,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2022,10]]}},"alternative-id":["e24101417"],"URL":"https:\/\/doi.org\/10.3390\/e24101417","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2022,10,4]]}}}