{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,24]],"date-time":"2026-01-24T18:51:58Z","timestamp":1769280718990,"version":"3.49.0"},"reference-count":87,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2018,11,22]],"date-time":"2018-11-22T00:00:00Z","timestamp":1542844800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>This paper provides tight bounds on the R\u00e9nyi entropy of a function of a discrete random variable with a finite number of possible values, where the considered function is not one to one. To that end, a tight lower bound on the R\u00e9nyi entropy of a discrete random variable with a finite support is derived as a function of the size of the support, and the ratio of the maximal to minimal probability masses. This work was inspired by the recently published paper by Cicalese et al., which is focused on the Shannon entropy, and it strengthens and generalizes the results of that paper to R\u00e9nyi entropies of arbitrary positive orders. In view of these generalized bounds and the works by Arikan and Campbell, non-asymptotic bounds are derived for guessing moments and lossless data compression of discrete memoryless sources.<\/jats:p>","DOI":"10.3390\/e20120896","type":"journal-article","created":{"date-parts":[[2018,11,23]],"date-time":"2018-11-23T03:41:31Z","timestamp":1542944491000},"page":"896","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":27,"title":["Tight Bounds on the R\u00e9nyi Entropy via Majorization with Applications to Guessing and Compression"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5681-1273","authenticated-orcid":false,"given":"Igal","family":"Sason","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Technion\u2014Israel Institute of Technology, Haifa 3200003, Israel"}]}],"member":"1968","published-online":{"date-parts":[[2018,11,22]]},"reference":[{"key":"ref_1","unstructured":"Hardy, G.H., Littlewood, J.E., and P\u00f3lya, G. (1952). Inequalities, Cambridge University Press. [2nd ed.]."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Marshall, A.W., Olkin, I., and Arnold, B.C. (2011). Inequalities: Theory of Majorization and Its Applications, Springer. [2nd ed.].","DOI":"10.1007\/978-0-387-68276-1"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"407","DOI":"10.1214\/0883423060000000097","article-title":"Majorization: Here, there and everywhere","volume":"22","author":"Arnold","year":"2007","journal-title":"Stat. Sci."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Arnold, B.C., and Sarabia, J.M. (2018). Majorization and the Lorenz Order with Applications in Applied Mathematics and Economics, Springer. Statistics for Social and Behavioral Sciences.","DOI":"10.1007\/978-3-319-93773-1"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Cicalese, F., Gargano, L., and Vaccaro, U. (2013, January 7\u201312). Information theoretic measures of distances and their econometric applications. Proceedings of the 2013 IEEE International Symposium on Information Theory, Istanbul, Turkey.","DOI":"10.1109\/ISIT.2013.6620258"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Steele, J.M. (2004). The Cauchy-Schwarz Master Class, Cambridge University Press.","DOI":"10.1017\/CBO9780511817106"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Bhatia, R. (1997). Matrix Analysis Graduate Texts in Mathematics, Springer.","DOI":"10.1007\/978-1-4612-0653-8"},{"key":"ref_8","unstructured":"Horn, R.A., and Johnson, C.R. (2013). Matrix Analysis, Cambridge University Press. [2nd ed.]."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1109\/TIT.1978.1055890","article-title":"R\u00e9nyi\u2019s entropy and probability of error","volume":"24","author":"Raviv","year":"1978","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"633","DOI":"10.1109\/TIT.2004.825026","article-title":"Bounding the average length of optimal source codes via majorization theory","volume":"50","author":"Cicalese","year":"2004","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Cicalese, F., Gargano, L., and Vaccaro, U. (2017, January 25\u201330). How to find a joint probability distribution with (almost) minimum entropy given the marginals. Proceedings of the 2017 IEEE International Symposium on Information Theory, Aachen, Germany.","DOI":"10.1109\/ISIT.2017.8006914"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2220","DOI":"10.1109\/TIT.2017.2787181","article-title":"Bounds on the entropy of a function of a random variable and their applications","volume":"64","author":"Cicalese","year":"2018","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Cicalese, F., and Vaccaro, U. (2018, January 17\u201320). Maximum entropy interval aggregations. Proceedings of the 2018 IEEE International Symposium on Information Theory, Vail, CO, USA.","DOI":"10.1109\/ISIT.2018.8437780"},{"key":"ref_14","unstructured":"Harremo\u00ebs, P. (2004, January 10\u201313). A new look on majorization. Proceedings of the 2004 IEEE International Symposium on Information Theory and Its Applications, Parma, Italy."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"5906","DOI":"10.1109\/TIT.2010.2080452","article-title":"The interplay between entropy and variational distance","volume":"56","author":"Ho","year":"2010","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5930","DOI":"10.1109\/TIT.2010.2080891","article-title":"On the interplay between conditional entropy and error probability","volume":"56","author":"Ho","year":"2010","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Ho, S.W., and Verd\u00fa, S. (2015, January 14\u201319). Convexity\/concavity of the R\u00e9nyi entropy and \u03b1-mutual information. Proceedings of the 2015 IEEE International Symposium on Information Theory, Hong Kong, China.","DOI":"10.1109\/ISIT.2015.7282554"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"915","DOI":"10.1214\/aos\/1176350843","article-title":"Majorization, entropy and paired comparisons","volume":"16","author":"Joe","year":"1988","journal-title":"Ann. Stat."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1016\/0022-247X(90)90002-W","article-title":"Majorization and divergence","volume":"148","author":"Joe","year":"1990","journal-title":"J. Math. Anal. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Koga, H. (2013, January 9\u201313). Characterization of the smooth R\u00e9nyi entropy using majorization. Proceedings of the 2013 IEEE Information Theory Workshop, Seville, Spain.","DOI":"10.1109\/ITW.2013.6691332"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1088\/1751-8113\/46\/27\/272002","article-title":"Majorization entropic uncertainty relations","volume":"46","author":"Puchala","year":"2013","journal-title":"J. Phys. A Math. Theor."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/TIT.2017.2757496","article-title":"Arimoto-R\u00e9nyi conditional entropy and Bayesian M-ary hypothesis testing","volume":"64","author":"Sason","year":"2018","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_23","unstructured":"Verd\u00fa, S. (Information Theory, 2018). Information Theory, in preparation."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1109\/TIT.1980.1056173","article-title":"Some aspects of convexity useful in information theory","volume":"26","author":"Witsenhhausen","year":"1980","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1007\/978-3-642-25255-6_20","article-title":"Schur-convexity on generalized information entropy and its applications","volume":"Volume 7030","author":"Xi","year":"2011","journal-title":"Information Computing and Applications"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"6484","DOI":"10.1109\/TIT.2012.2206071","article-title":"Optimality of binary power control for the single cell uplink","volume":"58","author":"Inaltekin","year":"2012","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1561\/0100000026","article-title":"Majorization and matrix-monotone functions in wireless communications","volume":"3","author":"Jorshweick","year":"2006","journal-title":"Found. Trends Commun. Inf. Theory"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1561\/0100000018","article-title":"MIMO transceiver design via majorization theory","volume":"3","author":"Palomar","year":"2006","journal-title":"Found. Trends Commun. Inf. Theory"},{"key":"ref_29","unstructured":"Roventa, I. (2015). Recent Trends in Majorization Theory and Optimization: Applications to Wireless Communications, Editura Pro Universitaria & Universitaria Craiova."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Uysal, M. (2010). Applications of majorization theory in space-time cooperative communications. Cooperative Communications for Improved Wireless Network Transmission: Framework for Virtual Antenna Array Applications, IGI Global. Information Science Reference.","DOI":"10.4018\/978-1-60566-665-5"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1984","DOI":"10.1109\/18.782121","article-title":"Optimal sequences and sum capacity of synchronous CDMA systems","volume":"45","author":"Viswanath","year":"1999","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1968","DOI":"10.1109\/18.782119","article-title":"Optimal sequences, power control, and user capacity of synchronous CDMA systems with linear MMSE multiuser receivers","volume":"45","author":"Viswanath","year":"1999","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1295","DOI":"10.1109\/TIT.2002.1003823","article-title":"Optimal sequences for CDMA under colored noise: A Schur-saddle function property","volume":"48","author":"Viswanath","year":"2002","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_34","unstructured":"R\u00e9nyi, A. (1961, January 8\u20139). On measures of entropy and information. Proceedings of the 4th Berkeley Symposium on Probability Theory and Mathematical Statistics, Berkeley, CA, USA."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1109\/18.481781","article-title":"An inequality on guessing and its application to sequential decoding","volume":"42","author":"Arikan","year":"1996","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1041","DOI":"10.1109\/18.669158","article-title":"Guessing subject to distortion","volume":"44","author":"Arikan","year":"1998","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1756","DOI":"10.1109\/18.705557","article-title":"Joint source-channel coding and guessing with application to sequential decoding","volume":"44","author":"Arikan","year":"1998","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"6941","DOI":"10.1109\/TIT.2018.2856190","article-title":"Reducing guesswork via an unreliable oracle","volume":"64","author":"Burin","year":"2018","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kuzuoka, S. (arXiv, 2018). On the conditional smooth R\u00e9nyi entropy and its applications in guessing and source coding, arXiv.","DOI":"10.1109\/ISIT.2019.8849753"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1860","DOI":"10.1109\/18.782106","article-title":"The Shannon cipher system with a guessing wiretapper","volume":"45","author":"Merhav","year":"1999","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Sundaresan, R. (2007, January 24\u201329). Guessing based on length functions. Proceedings of the 2007 IEEE International Symposium on Information Theory, Nice, France.","DOI":"10.1109\/ISIT.2007.4557309"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Salamatian, S., Beirami, A., Cohen, A., and M\u00e9dard, M. (2017, January 25\u201330). Centralized versus decentralized multi-agent guesswork. Proceedings of the 2017 IEEE International Symposium on Information Theory, Aachen, Germany.","DOI":"10.1109\/ISIT.2017.8006931"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1109\/TIT.2006.887466","article-title":"Guessing under source uncertainty","volume":"53","author":"Sundaresan","year":"2007","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Bracher, A., Lapidoth, A., and Pfister, C. (2017, January 25\u201330). Distributed task encoding. Proceedings of the 2017 IEEE International Symposium on Information Theory, Aachen, Germany.","DOI":"10.1109\/ISIT.2017.8006878"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"5065","DOI":"10.1109\/TIT.2014.2329490","article-title":"Encoding tasks and R\u00e9nyi entropy","volume":"60","author":"Bunte","year":"2014","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Shayevitz, O. (August, January 31). On R\u00e9nyi measures and hypothesis testing. Proceedings of the 2011 IEEE International Symposium on Information Theory, Saint Petersburg, Russia.","DOI":"10.1109\/ISIT.2011.6034266"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Tomamichel, M., and Hayashi, M. (2016, January 10\u201315). Operational interpretation of R\u00e9nyi conditional mutual information via composite hypothesis testing against Markov distributions. Proceedings of the 2016 IEEE International Symposium on Information Theory, Barcelona, Spain.","DOI":"10.1109\/ISIT.2016.7541366"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"438","DOI":"10.1109\/TIT.2009.2034824","article-title":"The communication complexity of correlation","volume":"56","author":"Harsha","year":"2010","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Liu, J., and Verd\u00fa, S. (2018, January 17\u201322). Rejection sampling and noncausal sampling under moment constraints. Proceedings of the 2018 IEEE International Symposium on Information Theory, Vail, CO, USA.","DOI":"10.1109\/ISIT.2018.8437857"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"3616","DOI":"10.1109\/TIT.2018.2806569","article-title":"Wyner\u2019s common information under R\u00e9nyi divergence measures","volume":"64","author":"Yu","year":"2018","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"423","DOI":"10.1016\/S0019-9958(65)90332-3","article-title":"A coding theorem and R\u00e9nyi\u2019s entropy","volume":"8","author":"Campbell","year":"1965","journal-title":"Inf. Control"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Courtade, T., and Verd\u00fa, S. (July, January 29). Cumulant generating function of codeword lengths in optimal lossless compression. Proceedings of the 2014 IEEE International Symposium on Information Theory, Honolulu, HI, USA.","DOI":"10.1109\/ISIT.2014.6875283"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Courtade, T., and Verd\u00fa, S. (July, January 29). Variable-length lossy compression and channel coding: Non-asymptotic converses via cumulant generating functions. Proceedings of the 2014 IEEE International Symposium on Information Theory, Honolulu, HI, USA.","DOI":"10.1109\/ISIT.2014.6875284"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"975","DOI":"10.1109\/TIT.2016.2636154","article-title":"Equivocations, exponents, and second-order coding rates under various R\u00e9nyi information measures","volume":"63","author":"Hayashi","year":"2017","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Kuzuoka, S. (2016, January 10\u201315). On the smooth R\u00e9nyi entropy and variable-length source coding allowing errors. Proceedings of the 2016 IEEE International Symposium on Information Theory, Barcelona, Spain.","DOI":"10.1109\/ISIT.2016.7541398"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"4323","DOI":"10.1109\/TIT.2018.2803162","article-title":"Improved bounds on lossless source coding and guessing moments via R\u00e9nyi measures","volume":"64","author":"Sason","year":"2018","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"3734","DOI":"10.1109\/TIT.2018.2792495","article-title":"Analysis of ramaining uncertainties and exponents under various conditional R\u00e9nyi entropies","volume":"64","author":"Tan","year":"2018","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Tyagi, H. (2017, January 2\u20134). Coding theorems using R\u00e9nyi information measures. Proceedings of the 2017 IEEE Twenty-Third National Conference on Communications, Chennai, India.","DOI":"10.1109\/NCC.2017.8077037"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1109\/18.370121","article-title":"Generalized cutoff rates and R\u00e9nyi information measures","volume":"41","year":"1995","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_60","unstructured":"Polyanskiy, Y., and Verd\u00fa, S. (October, January 29). Arimoto channel coding converse and R\u00e9nyi divergence. Proceedings of the Forty-Eighth Annual Allerton Conference on Communication, Control and Computing, Monticello, IL, USA."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1109\/TIT.2015.2504100","article-title":"On the R\u00e9nyi divergence, joint range of relative entropies, and a channel coding theorem","volume":"62","author":"Sason","year":"2016","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_62","first-page":"208","article-title":"R\u00e9nyi resolvability and its applications to the wiretap channel","volume":"Volume 10681","author":"Yu","year":"2017","journal-title":"Lecture Notes in Computer Science, Proceedings of the 10th International Conference on Information Theoretic Security, Hong Kong, China, 29 November\u20132 December 2017"},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1109\/TIT.1973.1055007","article-title":"On the converse to the coding theorem for discrete memoryless channels","volume":"19","author":"Arimoto","year":"1973","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_64","first-page":"41","article-title":"Information measures and capacity of order \u03b1 for discrete memoryless channels","volume":"Volume 16","author":"Elias","year":"1977","journal-title":"Proceedings of the 2nd Colloquium on Information Theory"},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"8027","DOI":"10.1109\/TIT.2013.2283794","article-title":"Lower bounds on the probability of error for classical and classical-quantum channels","volume":"59","author":"Dalai","year":"2013","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1063\/1.4960099","article-title":"Strong converse theorems using R\u00e9nyi entropies","volume":"57","author":"Leditzky","year":"2016","journal-title":"J. Math. Phys."},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"1617","DOI":"10.1007\/s00220-014-2248-x","article-title":"Quantum hypothesis testing and the operational interpretation of the quantum R\u00e9nyi relative entropies","volume":"334","author":"Mosonyi","year":"2015","journal-title":"Commun. Math. Phys."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"1262","DOI":"10.1016\/j.aml.2009.01.040","article-title":"Jensen\u2019s inequality and new entropy bounds","volume":"22","author":"Simic","year":"2009","journal-title":"Appl. Math. Lett."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1109\/TIT.1972.1054899","article-title":"On variable-length-to-block coding","volume":"18","author":"Jelineck","year":"1972","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_70","unstructured":"Garey, M.R., and Johnson, D.S. (1979). Computers and Intractability: A Guide to the Theory of NP-Completness, W. H. Freedman and Company."},{"key":"ref_71","doi-asserted-by":"crossref","first-page":"2062","DOI":"10.1109\/18.641578","article-title":"Comments on \u201cAn inequality on guessing and its application to sequential decoding\u201d","volume":"43","year":"1997","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"Bracher, A., Hof, E., and Lapidoth, A. (2015, January 14\u201319). Guessing attacks on distributed-storage systems. Proceedings of the 2015 IEEE International Symposium on Information Theory, Hong-Kong, China.","DOI":"10.1109\/ISIT.2015.7282723"},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"796","DOI":"10.1109\/TIT.2012.2219036","article-title":"Guesswork, large deviations, and Shannon entropy","volume":"59","author":"Christiansen","year":"2013","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_74","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/TIT.2010.2090221","article-title":"Guessing revisited: A large deviations approach","volume":"57","author":"Hanawal","year":"2011","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_75","doi-asserted-by":"crossref","first-page":"2503","DOI":"10.1109\/TIT.2011.2110870","article-title":"The Shannon cipher system with a guessing wiretapper: General sources","volume":"57","author":"Hanawal","year":"2011","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_76","doi-asserted-by":"crossref","unstructured":"Huleihel, W., Salamatian, S., and M\u00e9dard, M. (2017, January 25\u201330). Guessing with limited memory. Proceedings of the 2017 IEEE International Symposium on Information Theory, Aachen, Germany.","DOI":"10.1109\/ISIT.2017.8006930"},{"key":"ref_77","unstructured":"Massey, J.L. (July, January 27). Guessing and entropy. Proceedings of the 1994 IEEE International Symposium on Information Theory, Trondheim, Norway."},{"key":"ref_78","unstructured":"McEliece, R.J., and Yu, Z. (1995, January 17\u201322). An inequality on entropy. Proceedings of the 1995 IEEE International Symposium on Information Theory, Whistler, BC, Canada."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"2794","DOI":"10.1109\/TIT.2004.836665","article-title":"R\u00e9nyi entropy, guesswork moments and large deviations","volume":"50","author":"Pfister","year":"2004","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"468","DOI":"10.1109\/18.904564","article-title":"Bounds on entropy in a guessing game","volume":"47","author":"Gaggia","year":"2001","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Yona, Y., and Diggavi, S. (2017, January 25\u201330). The effect of bias on the guesswork of hash functions. Proceedings of the 2017 IEEE International Symposium on Information Theory, Aachen, Germany.","DOI":"10.1109\/ISIT.2017.8006929"},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Gan, G., Ma, C., and Wu, J. (2007). Data Clustering: Theory, Algorithms, and Applications, SIAM.","DOI":"10.1137\/1.9780898718348"},{"key":"ref_83","first-page":"113","article-title":"Definition of entropy by means of a coding problem","volume":"6","author":"Campbell","year":"1966","journal-title":"Probab. Theory Relat. Field"},{"key":"ref_84","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, John Wiley & Sons. [2nd ed.]."},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"777","DOI":"10.1109\/TIT.2013.2291007","article-title":"Optimal lossless data compression: Non-asymptotics and asymptotics","volume":"60","author":"Kontoyiannis","year":"2014","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"3797","DOI":"10.1109\/TIT.2014.2320500","article-title":"R\u00e9nyi divergence and Kullback-Leibler divergence","volume":"60","year":"2014","journal-title":"IEEE Trans. Inf. Theory"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/12\/896\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:31:25Z","timestamp":1760196685000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/12\/896"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,11,22]]},"references-count":87,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2018,12]]}},"alternative-id":["e20120896"],"URL":"https:\/\/doi.org\/10.3390\/e20120896","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,11,22]]}}}