{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,19]],"date-time":"2026-03-19T20:05:52Z","timestamp":1773950752326,"version":"3.50.1"},"reference-count":36,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2021,6,11]],"date-time":"2021-06-11T00:00:00Z","timestamp":1623369600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Australian Research Council Centre of Excellence for Climate Extremes","award":["CE170100023"],"award-info":[{"award-number":["CE170100023"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We develop a simple Quantile Spacing (QS) method for accurate probabilistic estimation of one-dimensional entropy from equiprobable random samples, and compare it with the popular Bin-Counting (BC) and Kernel Density (KD) methods. In contrast to BC, which uses equal-width bins with varying probability mass, the QS method uses estimates of the quantiles that divide the support of the data generating probability density function (pdf) into equal-probability-mass intervals. And, whereas BC and KD each require optimal tuning of a hyper-parameter whose value varies with sample size and shape of the pdf, QS only requires specification of the number of quantiles to be used. Results indicate, for the class of distributions tested, that the optimal number of quantiles is a fixed fraction of the sample size (empirically determined to be ~0.25\u20130.35), and that this value is relatively insensitive to distributional form or sample size. This provides a clear advantage over BC and KD since hyper-parameter tuning is not required. Further, unlike KD, there is no need to select an appropriate kernel-type, and so QS is applicable to pdfs of arbitrary shape, including those with discontinuous slope and\/or magnitude. Bootstrapping is used to approximate the sampling variability distribution of the resulting entropy estimate, and is shown to accurately reflect the true uncertainty. For the four distributional forms studied (Gaussian, Log-Normal, Exponential and Bimodal Gaussian Mixture), expected estimation bias is less than 1% and uncertainty is low even for samples of as few as 100 data points; in contrast, for KD the small sample bias can be as large as \u221210% and for BC as large as \u221250%. We speculate that estimating quantile locations, rather than bin-probabilities, results in more efficient use of the information in the data to approximate the underlying shape of an unknown data generating pdf.<\/jats:p>","DOI":"10.3390\/e23060740","type":"journal-article","created":{"date-parts":[[2021,6,11]],"date-time":"2021-06-11T12:44:37Z","timestamp":1623415477000},"page":"740","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["Computing Accurate Probabilistic Estimates of One-D Entropy from Equiprobable Random Samples"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9855-2839","authenticated-orcid":false,"given":"Hoshin V.","family":"Gupta","sequence":"first","affiliation":[{"name":"Hydrology and Atmospheric Sciences, The University of Arizona, Tucson, AZ 85721, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0202-6901","authenticated-orcid":false,"given":"Mohammad Reza","family":"Ehsani","sequence":"additional","affiliation":[{"name":"Hydrology and Atmospheric Sciences, The University of Arizona, Tucson, AZ 85721, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tirthankar","family":"Roy","sequence":"additional","affiliation":[{"name":"Civil and Environmental Engineering, University of Nebraska-Lincoln, Omaha, NE 68182, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maria A.","family":"Sans-Fuentes","sequence":"additional","affiliation":[{"name":"GIDP Statistics and Data Science, The University of Arizona, Tucson, AZ 85721, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3454-8755","authenticated-orcid":false,"given":"Uwe","family":"Ehret","sequence":"additional","affiliation":[{"name":"Institute of Water and River Basin Management, Karlsruhe Institute of Technology (KIT), 76131 Karlsruhe, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ali","family":"Behrangi","sequence":"additional","affiliation":[{"name":"Hydrology and Atmospheric Sciences, The University of Arizona, Tucson, AZ 85721, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,6,11]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, Wiley.","DOI":"10.1002\/047174882X"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"5003","DOI":"10.1002\/2014WR015874","article-title":"Estimating information entropy for hydrological data: One dimensional case","volume":"50","author":"Gong","year":"2014","journal-title":"Tech. Note Water Resour. Res."},{"key":"ref_4","unstructured":"Silverman, B.W. (1986). Density Estimation for Statistics and Data Analysis, Chapman and Hall."},{"key":"ref_5","first-page":"17","article-title":"Nonparametric entropy estimation: An overview","volume":"6","author":"Beirlant","year":"1997","journal-title":"Int. J. Math. Stat. Sci."},{"key":"ref_6","unstructured":"Scott, D.W. (2008). Multivariate Density Estimation: Theory, Practice, and Visualization, John Wiley."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1049","DOI":"10.1214\/aoms\/1177700079","article-title":"A Nonparametric Estimate of a Multivariate Density Function","volume":"36","author":"Loftsgarden","year":"1965","journal-title":"Ann. Math. Stat."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1093\/biomet\/66.3.605","article-title":"Optimal and data-based histograms","volume":"66","author":"Scott","year":"1979","journal-title":"Biometrika"},{"key":"ref_9","unstructured":"Gentle, J.E. (1983). Probability Density Estimation in Higher Dimensions. Computer Science and Statistics: Proceedings of the Fifteenth Symposium on the Interface, North-Holland."},{"key":"ref_10","unstructured":"Scott, D.W. (2004). Handbook of Computational Statistics\u2014Concepts and Methods, Springer."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1024","DOI":"10.1214\/aos\/1176349654","article-title":"Averaged shifted histograms\u2014Effective nonparametric density estimators in several dimensions","volume":"13","author":"Scott","year":"1985","journal-title":"Ann. Stat."},{"key":"ref_12","first-page":"211","article-title":"Maximum likelihood estimation of a probability density function","volume":"3","author":"Wegman","year":"1975","journal-title":"Sankhy\u0101 Indian J. Stat."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"475","DOI":"10.1016\/S0167-9473(01)00073-1","article-title":"Bayesian partition modelling","volume":"38","author":"Denison","year":"2002","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1109\/LSP.2001.838216","article-title":"An algorithm for optimal partitioning of data on an interval","volume":"12","author":"Jackson","year":"2005","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"3766","DOI":"10.1109\/TIT.2005.856954","article-title":"Bayesian bin distribution inference and mutual information","volume":"51","author":"Endres","year":"2005","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_16","first-page":"635","article-title":"Exact Bayesian regression of piecewise constant functions","volume":"2","author":"Hutter","year":"2007","journal-title":"Bayesian An."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Darscheid, P., Guthke, A., and Ehret, U. (2018). A maximum-entropy method to estimate discrete distributions from samples ensuring nonzero probabilities. Entropy, 20.","DOI":"10.3390\/e20080601"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1080\/01621459.1926.10502161","article-title":"The choice of a class interval","volume":"21","author":"Sturges","year":"1926","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"453","DOI":"10.1007\/BF01025868","article-title":"On the histogram as a density estimator: L2 theory","volume":"57","author":"Freedman","year":"1981","journal-title":"Z. Wahrscheinlichkeitstheorie Verwandte Geb."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"102581","DOI":"10.1016\/j.dsp.2019.102581","article-title":"Optimal data-based binning for histograms and histogram-based probability density models","volume":"95","author":"Knuth","year":"2019","journal-title":"Digit. Signal Process."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1016\/j.sigpro.2004.11.022","article-title":"Fast kernel entropy estimation and optimization","volume":"85","author":"Schwartz","year":"2005","journal-title":"Signal Process."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Viola, P.A. (1995). Alignment by Maximization of Mutual Information. [Ph.D. Thesis, Massachusetts Institute of Technology\u2013Artificial Intelligence Laboratory].","DOI":"10.21236\/ADA299525"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1080\/00031305.1996.10473566","article-title":"Sample quantiles in statistical packages","volume":"50","author":"Hyndman","year":"1996","journal-title":"Am. Stat."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"635","DOI":"10.1093\/biomet\/69.3.635","article-title":"A new distribution-free quantile estimator","volume":"69","author":"Harrell","year":"1982","journal-title":"Biometrika"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2217","DOI":"10.1080\/03610926208828383","article-title":"A Generalized Quantile Estimator","volume":"11","author":"Kaigh","year":"1982","journal-title":"Commun. Stat. Part A Theory Methods"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1398","DOI":"10.1016\/j.csda.2005.08.004","article-title":"On quantile estimation by bootstrap","volume":"50","author":"Brodin","year":"2006","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1080\/01621459.1979.10481621","article-title":"Nonparametric Statistical Data Modeling","volume":"74","author":"Parzen","year":"1979","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"410","DOI":"10.1080\/01621459.1990.10476214","article-title":"Kernel quantile estimators","volume":"85","author":"Sheather","year":"1990","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1016\/S0378-3758(96)00110-3","article-title":"Unified estimators of smooth quantile and quantile density functions","volume":"59","author":"Cheng","year":"1997","journal-title":"J. Stat. Plan. Inference"},{"key":"ref_30","unstructured":"Park, C. (2006). Smooth Nonparametric Estimation of a Quantile Function under Right Censoring Using Beta Kernels, Department of Mathematical Sciences, Clemson University. Technical Report (TR 2006-01-CP)."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1004","DOI":"10.1080\/01621459.1985.10478217","article-title":"A smooth nonparametric estimator of a quantile function","volume":"80","author":"Yang","year":"1985","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"337","DOI":"10.1080\/03610910701790491","article-title":"A new family of nonparametric quantile estimators","volume":"37","author":"Sfakianakis","year":"2008","journal-title":"Commun. Stat. Simul. Comput."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1111\/bmsp.12198","article-title":"A new quantile estimator with weights based on a subsampling approach","volume":"73","author":"Navruz","year":"2020","journal-title":"Br. J. Math. Stat. Psychol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1016\/0167-7152(94)00190-J","article-title":"The Bernstein polynomial estimator of a smooth quantile function","volume":"24","author":"Cheng","year":"1995","journal-title":"Stat. Probab. Lett."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/10485252.2013.826355","article-title":"Estimation of the quantile function using Bernstein\u2013Durrmeyer polynomials","volume":"26","author":"Pepelyshev","year":"2014","journal-title":"J. Nonparametric Stat."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"141","DOI":"10.1007\/s00184-019-00736-3","article-title":"Nonparametric quantile estimation using surrogate models and importance sampling","volume":"83","author":"Kohler","year":"2020","journal-title":"Metrika"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/6\/740\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:13:31Z","timestamp":1760163211000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/6\/740"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,11]]},"references-count":36,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2021,6]]}},"alternative-id":["e23060740"],"URL":"https:\/\/doi.org\/10.3390\/e23060740","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,11]]}}}