{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,1]],"date-time":"2026-02-01T03:03:26Z","timestamp":1769915006678,"version":"3.49.0"},"reference-count":36,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2018,8,31]],"date-time":"2018-08-31T00:00:00Z","timestamp":1535673600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Information &amp; Communications Technology Promotion (IITP)","award":["2017-0-00441"],"award-info":[{"award-number":["2017-0-00441"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Since the entropy is a popular randomness measure, there are many studies for the estimation of entropies for given random samples. In this paper, we propose an estimation method of the R\u00e9nyi entropy of order    \u03b1   . Since the R\u00e9nyi entropy of order    \u03b1    is a generalized entropy measure including the Shannon entropy as a special case, the proposed estimation method for R\u00e9nyi entropy can detect any significant deviation of an ergodic stationary random source\u2019s output. It is shown that the expected test value of the proposed scheme is equivalent to the R\u00e9nyi entropy of order    \u03b1   . After deriving a general representation of parameters of the proposed estimator, we discuss on the particular orders of R\u00e9nyi entropy such as     \u03b1 \u2192 1    ,     \u03b1 = 1 \/ 2    , and     \u03b1 = 2    . Because the R\u00e9nyi entropy of order 2 is the most popular one, we present an iterative estimation method for the application with stringent resource restrictions.<\/jats:p>","DOI":"10.3390\/e20090657","type":"journal-article","created":{"date-parts":[[2018,8,31]],"date-time":"2018-08-31T10:57:52Z","timestamp":1535713072000},"page":"657","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Low Complexity Estimation Method of R\u00e9nyi Entropy for Ergodic Sources"],"prefix":"10.3390","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4114-4935","authenticated-orcid":false,"given":"Young-Sik","family":"Kim","sequence":"first","affiliation":[{"name":"Department of Information and Communication Engineering, Chosun University, 309 Pilmoondae-ro Dong-gu, Gwangju 61452, Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,8,31]]},"reference":[{"key":"ref_1","unstructured":"NIST (1994, January 19). Digital signature standard (DSS). Proceedings of the Federal Information Processing Standard (FIPS PUB 186), Gaithersburg, MD, USA."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Bucci, M., and Luzzi, R. (2005). Design of testable random bit generators. International Workshop on Cryptographic Hardware and Embedded Systems, Springer.","DOI":"10.1007\/11545262_11"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Dichtl, M., and Goli\u0107, J.D. (2007). High-speed true random number generation with logic gates only. Cryptographic Hardware and Embedded Systems-CHES 2007, Springer.","DOI":"10.1007\/978-3-540-74735-2_4"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Vasyltsov, I., Hambardzumyan, E., Kim, Y.-S., and Karpinskyy, B. (2008). Fast digital TRNG based on metastable ring oscillator. International Workshop on Cryptographic Hardware and Embedded Systems, Springer.","DOI":"10.1007\/978-3-540-85053-3_11"},{"key":"ref_5","unstructured":"Killman, W., and Schindler, W. (2018, August 28). A Proposal for: Funtionality Classes and Evaluation Methodology for True (Physical) Random Number Generators. Available online: https:\/\/www.bsi.bund.de\/SharedDocs\/Downloads\/DE\/BSI\/Zertifizierung\/Interpretationen\/AIS_31_Functionality_classes_evaluation_methodology_for_true_RNG_e.pdf?__blob=publicationFile&v=1."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Schindler, W. (2001). Efficient online tests for true random number generators. International Workshop on Cryptographic Hardware and Embedded Systems, Springer.","DOI":"10.1007\/3-540-44709-1_10"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Coron, J.-S. (1999). On the security of random sources. International Workshop on Public Key Cryptography, Springer.","DOI":"10.1007\/3-540-49162-7_3"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1007\/BF00193563","article-title":"A universal statistical test for random bit generators","volume":"5","author":"Maurer","year":"1992","journal-title":"J. Cryptol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1007\/s10044-005-0247-4","article-title":"Estimation of generalized entropies with sample spacing","volume":"8","author":"Wachowiak","year":"2005","journal-title":"Pattern Anal. Appl."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Wu, Y., and Yang, P. (2015, January 14\u201319). Optimal entropy estimation on large alphabets via best polynomial approximation. Proceedings of the 2015 IEEE International Symposium on Information Theory (ISIT), Hong Kong, China.","DOI":"10.1109\/ISIT.2015.7282570"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Jiao, J., Venkat, K., Han, Y., and Weissman, T. (2015, January 14\u201319). Maximum likelihood estimation of information measures. Proceedings of the 2015 IEEE International Symposium on Information Theory (ISIT), Hong Kong, China.","DOI":"10.1109\/ISIT.2015.7282573"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Acharya, J., Orlitsky, A., Suresh, A.T., and Tyagi, H. (2015, January 4\u20136). The complexity of estimating R\u00e9nyi entropy. Proceedings of the 26th Annual ACM-SIAM Symposium Discrete Algorithms, San Diego, CA, USA.","DOI":"10.1137\/1.9781611973730.124"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Valiant, G., and Valiant, P. (2011, January 6\u20138). Estimating the unseen: An n\/log(n)-sample estimator for entropy and support size, shown optimal via new CLTs. Proceedings of the 43rd annual ACM symposium on Theory of Computing, San Jose, CA, USA.","DOI":"10.1145\/1993636.1993727"},{"key":"ref_14","unstructured":"R\u00e9nyi, A. (July, January 20). On measures of entropy and information. Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA, USA."},{"key":"ref_15","unstructured":"Erdogmus, D., Principe, J.C., Kim, S.-P., and Sanchez, J.C. (2002, January 6). A recursive R\u00e9nyi\u2019s entropy estimator. Proceedings of the 12th IEEE Workshop on Neural Networks for Signal Processing, Martigny, Switzerland."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1780","DOI":"10.1109\/TSP.2002.1011217","article-title":"An error-entropy minimization algorithm for supervised training of nonlinear adaptive systems","volume":"50","author":"Erdogmus","year":"2002","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_17","unstructured":"Xu, J.-W., Erdogmus, D.M., Ozturk, C., and Principe, J.C. (2003, January 17). Recursive R\u00e9nyi\u2019s entropy estimator for adaptive filtering. Proceedings of the 3rd IEEE International Symposium on Signal Processing and Information Technology, Darmstadt, Germany."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"734","DOI":"10.1109\/TIT.2009.2037038","article-title":"Computational detection of transcription factor binding sites through differential R\u00e9nyi entropy","volume":"56","author":"Maynou","year":"2010","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1915","DOI":"10.1109\/18.476316","article-title":"Generalized privacy amplification","volume":"41","author":"Bennett","year":"1995","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_20","unstructured":"NIST (2001). A Statistical Test Suite for Random and Pseudorandom Number Generators for Cryptographic Applications."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"324","DOI":"10.1109\/TIT.1978.1055890","article-title":"R\u00e9nyi\u2019s entropy and the probability of error","volume":"24","author":"Raviv","year":"1978","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_22","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, John Wiley & Sons. [2nd ed.]."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2153","DOI":"10.1214\/07-AOS539","article-title":"A class of R\u00e9nyi information estimators for multidemensional densities","volume":"36","author":"Leonenko","year":"2008","journal-title":"Ann. Stat."},{"key":"ref_24","unstructured":"Obremski, M., and Skorski, M. (2017). Renyi entropy estimation revisited. LIPIcs-Leibniz International Proceedings in Informatics, Schloss Dagstuhl\u2013Leibniz-Zentrum fuer Informatik."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1489","DOI":"10.1109\/TSP.2004.827202","article-title":"Adaptive blind deconvolution of linear channels using R\u00e9nyi\u2019s entropy with Parzen window estimation","volume":"52","author":"Erdogmus","year":"2004","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2971","DOI":"10.1016\/j.physa.2011.12.064","article-title":"Renyi\u2019s information transfer between financial time series","volume":"391","author":"Jizba","year":"2012","journal-title":"Phys. A Stat. Mech. Appl."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1109\/18.30993","article-title":"Estimating the information content of symbol sequences andefficient codes","volume":"35","author":"Grassberger","year":"1989","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3934\/amc.2008.2.1","article-title":"Entropy estimators with almost sure convergence and an O(n\u22121) variance","volume":"2","author":"Kaltchenko","year":"2008","journal-title":"Adv. Math. Commun."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2330","DOI":"10.1109\/TIT.2007.899533","article-title":"On the estimation of differential entropy from data located on embedded manifolds","volume":"53","author":"Nilsson","year":"2007","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Kamath, S., and Verdu, S. (2016, January 10\u201315). Estimation of entropy rate and Renyi entropy rate for Markov chains. Proceedings of the 2016 IEEE International Symposium on Information Theory (ISIT), Barcelona, Spain.","DOI":"10.1109\/ISIT.2016.7541386"},{"key":"ref_31","unstructured":"Kim, Y.-S. (2014, January 26\u201329). Estimation of R\u00e9nyi entropy of order \u03b1 based on the nearest neighbor distance. Proceedings of the 2014 International Symposium on Information Theory and Its Applications, Melbourne, Australia."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Coron, J.-S., and Naccache, D. (1999). An accurate evaluation of Maurer\u2019s universal test. International Workshop on Selected Areas in Cryptography, Springer.","DOI":"10.1007\/3-540-48892-8_5"},{"key":"ref_33","unstructured":"Graham, R.L., Knuth, D.E., and Patashnik, O. (1994). Concrete Mathematics, Addison-Wesley. [2nd ed.]."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1109\/18.481781","article-title":"An inequality on guessing and its applcation to sequential decoding","volume":"42","author":"Arikan","year":"1996","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Beirami, A., Calderbank, R., Duffy, K., and Medard, M. (2015, January 14\u201319). Quantifying computational security subject to source constraints, guesswork and inscrutability. Proceedings of the 2015 IEEE International Symposium on Information Theory (ISIT), Hong Kong, China.","DOI":"10.1109\/ISIT.2015.7282958"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"93","DOI":"10.4218\/etrij.10.0109.0141","article-title":"Linear corrector overcoming minimum distance limitation for secure TRNG from (17, 9, 5) quadratic residue code","volume":"32","author":"Kim","year":"2010","journal-title":"ETRI J."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/9\/657\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:22:21Z","timestamp":1760196141000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/9\/657"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8,31]]},"references-count":36,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2018,9]]}},"alternative-id":["e20090657"],"URL":"https:\/\/doi.org\/10.3390\/e20090657","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,8,31]]}}}