{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T09:53:29Z","timestamp":1770976409111,"version":"3.50.1"},"reference-count":25,"publisher":"MIT Press - Journals","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2012,5]]},"abstract":"<jats:p> A new nonparametric estimator of Shannon's entropy on a countable alphabet is proposed and analyzed against the well-known plug-in estimator. The proposed estimator is developed based on Turing's formula, which recovers distributional characteristics on the subset of the alphabet not covered by a size-n sample. The fundamental switch in perspective brings about substantial gain in estimation accuracy for every distribution with finite entropy. In general, a uniform variance upper bound is established for the entire class of distributions with finite entropy that decays at a rate of O(ln(n)\/n) compared to O([ln(n)]<jats:sup>2<\/jats:sup>\/n) for the plug-in. In a wide range of subclasses, the variance of the proposed estimator converges at a rate of O(1\/n), and this rate of convergence carries over to the convergence rates in mean squared errors in many subclasses. Specifically, for any finite alphabet, the proposed estimator has a bias decaying exponentially in n. Several new bias-adjusted estimators are also discussed. <\/jats:p>","DOI":"10.1162\/neco_a_00266","type":"journal-article","created":{"date-parts":[[2012,2,1]],"date-time":"2012-02-01T18:09:48Z","timestamp":1328119788000},"page":"1368-1389","source":"Crossref","is-referenced-by-count":43,"title":["Entropy Estimation in Turing's Perspective"],"prefix":"10.1162","volume":"24","author":[{"given":"Zhiyi","family":"Zhang","sequence":"first","affiliation":[{"name":"Department of Mathematics and Statistics, University of North Carolina at Charlotte, Charlotte, NC 28223, U.S.A."}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1002\/rsa.10019"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1137\/1104033"},{"key":"B3","first-page":"17","volume":"6","author":"Beirlant J.","year":"2001","journal-title":"International Journal of the Mathematical Statistics Sciences"},{"key":"B4","doi-asserted-by":"publisher","DOI":"10.1023\/A:1026096204727"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176346256"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/40.3-4.237"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1016\/0375-9601(88)90193-4"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.21236\/ADA020217"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177730196"},{"key":"B10","first-page":"95","volume-title":"Information theory in psychology","volume":"2","author":"Miller G.","year":"1955"},{"key":"B11","volume-title":"Advances in neural information processing systems","volume":"14","author":"Nemenman I.","year":"2002"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1162\/089976603321780272"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1152\/jn.00559.2007"},{"key":"B14","first-page":"547","volume-title":"Proc. 4th Berk. Symp. Math. Statist. and Prob.","volume":"1","author":"R\u00e9nyi A.","year":"1961"},{"key":"B15","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177698526"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.1088\/0305-4470\/37\/27\/L02"},{"key":"B17","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb00917.x"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.1038\/163688a0"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.80.197"},{"key":"B20","doi-asserted-by":"publisher","DOI":"10.1162\/089976600300014728"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1002\/sim.2942"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.2307\/1936227"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1080\/10485250802172126"},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.1214\/08-AOS658"},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1016\/j.jspi.2009.12.023"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/NECO_a_00266","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,12]],"date-time":"2021-03-12T21:39:15Z","timestamp":1615585155000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/24\/5\/1368-1389\/7763"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2012,5]]},"references-count":25,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2012,5]]}},"alternative-id":["10.1162\/NECO_a_00266"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_00266","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2012,5]]}}}