{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:16:21Z","timestamp":1760238981890,"version":"build-2065373602"},"reference-count":28,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2020,9,13]],"date-time":"2020-09-13T00:00:00Z","timestamp":1599955200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Data"],"abstract":"<jats:p>In this paper, we study the information lost when a real-valued statistic is used to reduce or summarize sample data from a discrete random variable with a one-dimensional parameter. We compare the probability that a random sample gives a particular data set to the probability of the statistic\u2019s value for this data set. We focus on sufficient statistics for the parameter of interest and develop a general formula independent of the parameter for the Shannon information lost when a data sample is reduced to such a summary statistic. We also develop a measure of entropy for this lost information that depends only on the real-valued statistic but neither the parameter nor the data. Our approach would also work for non-sufficient statistics, but the lost information and associated entropy would involve the parameter. The method is applied to three well-known discrete distributions to illustrate its implementation.<\/jats:p>","DOI":"10.3390\/data5030084","type":"journal-article","created":{"date-parts":[[2020,9,13]],"date-time":"2020-09-13T21:11:32Z","timestamp":1600031492000},"page":"84","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Information Loss Due to the Data Reduction of Sample Data from Discrete Distributions"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5979-9871","authenticated-orcid":false,"given":"Maryam","family":"Moghimi","sequence":"first","affiliation":[{"name":"Center on Stochastic Modeling, Optimization, and Statistics (COSMOS), the University of Texas at Arlington, Arlington, TX 76013, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5439-6843","authenticated-orcid":false,"given":"Herbert W.","family":"Corley","sequence":"additional","affiliation":[{"name":"Center on Stochastic Modeling, Optimization, and Statistics (COSMOS), the University of Texas at Arlington, Arlington, TX 76013, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,9,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1147\/rd.53.0183","article-title":"Irreversibility and heat generation in the computing process","volume":"5","author":"Landauer","year":"1961","journal-title":"IBM J. Res. Dev."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Hodge, S.E., and Vieland, V.J. (2017). Information loss in binomial data due to data compression. Entropy, 19.","DOI":"10.3390\/e19020075"},{"key":"ref_3","unstructured":"Casella, G., and Berger, R.L. (2002). Statistical Inference, Cengage Learning. [2nd ed.]."},{"key":"ref_4","unstructured":"Pawitan, Y. (2013). All Likelihood: Statistical Modeling and Inference Using Likelihood, The Clarendon Press. [1st ed.]."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Rohatgi, V.K., and Saleh, A.K.E. (2001). An Introduction to Probability and Statistics, John Wiley & Sons, Inc.. [2nd ed.].","DOI":"10.1002\/9781118165676"},{"key":"ref_6","unstructured":"Shannon, C.E., and Weaver, W. (1964). The Mathematical Theory of Communication, The University of Illinois Press. [1st ed.]."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell. Syst. Tech. J."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4847","DOI":"10.1016\/j.ins.2011.05.020","article-title":"Representational information: A new general notion and measure of information","volume":"181","author":"Vigo","year":"2011","journal-title":"Inf. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/info4010001","article-title":"Complexity over uncertainty in generalized representational information theory (GRIT): A structure-sensitive general theory of information","volume":"4","author":"Vigo","year":"2013","journal-title":"Information"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Klir, G.J. (2006). Uncertainty and Information: Foundations of Generalized Information Theory, John Wiley & Sons, Inc.. [1st ed.].","DOI":"10.1002\/0471755575"},{"key":"ref_11","unstructured":"Devlin, K. (1991). Logic and Information, Cambridge University Press. [1st ed.]."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1037\/1089-2680.7.2.183","article-title":"Whatever happened to information theory in psychology?","volume":"7","author":"Luce","year":"2003","journal-title":"Rev. Gen. Psychol."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Floridi, L. (2011). The Philosophy of Information, Oxford University Press. [1st ed.].","DOI":"10.1093\/acprof:oso\/9780199232383.001.0001"},{"key":"ref_14","unstructured":"Garner, W.R. (1974). The Processing of Information and Structure, Wiley. [1st ed.]."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1046\/j.1466-822X.2003.00015.x","article-title":"A tribute to Claude-Shannon (1916\u20132001) and a plea for more rigorous use of species richness, species diversity and the \u201cShannon-Wiener\u201d index","volume":"12","author":"Spellerberg","year":"2003","journal-title":"Glob. Ecol. Biogeogr."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2696","DOI":"10.1016\/j.tcs.2010.04.006","article-title":"Learning and generalization with the information bottleneck","volume":"411","author":"Shamir","year":"2010","journal-title":"Theor. Comput. Sci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"261","DOI":"10.3390\/e10030261","article-title":"Axiomatic characterizations of information measures","volume":"10","year":"2008","journal-title":"Entropy"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kapur, J.N., and Kesavan, H.K. (1992). Entropy Optimization Principles and Their Applications, Water Science and Technology Library, Springer. [1st ed.].","DOI":"10.1007\/978-94-011-2430-0_1"},{"key":"ref_19","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, John Wiley & Sons, Inc.. [2nd ed.]."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ibekwe-SanJuan, F., and Dousa, T. (2014). Theories of Information, Communication and Knowledge: A Multidisciplinary Approach, Springer. [1st ed.].","DOI":"10.1007\/978-94-007-6973-1"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Johnson, J.L. (2008). Probability and Statistics for Computer Science, John Wiley & Sons, Inc.. [1st ed.].","DOI":"10.1002\/9780470050118.ecse322"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Beeler, R.A. (2015). How to Count: An Introduction to Combinatorics and Its Applications, Springer. [1st ed.].","DOI":"10.1007\/978-3-319-13844-2"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Sridharan, S., and Balakrishnan, R. (2018). Foundations of Discrete Mathematics with Algorithms and Programming, Chapman and Hall\/CRC. [1st ed.].","DOI":"10.1201\/9781351019149"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2579","DOI":"10.1016\/j.amc.2010.07.070","article-title":"Ramanujan formula for the generalized Stirling approximation","volume":"19","author":"Mortici","year":"2010","journal-title":"Appl. Math. Comput."},{"key":"ref_25","unstructured":"(2019, July 27). Multinomial Coefficient Calculator. Available online: https:\/\/mathcracker.com\/multinomial-coefficient-calculator.php."},{"key":"ref_26","unstructured":"Wan, L., Mehta, K.V., Klasky, S.A., Wolf, M., Wang, H.Y., Wang, W.H., Li, J.C., and Lin, Z. (2019, January 9\u201313). Data management challenges of exascale scientific simulations: A case study with the Gyrokinetic Toroidal Code and ADIOS. Proceedings of the 10th International Conference on Computational Methods, ICCM\u201919, Singapore."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1038\/s41586-019-1666-5","article-title":"Quantum supremacy using a programmable superconducting processor","volume":"574","author":"Arute","year":"2019","journal-title":"Nature"},{"key":"ref_28","first-page":"5","article-title":"The exact number of nonnegative integer solutions for a linear Diophantine inequality","volume":"40","author":"Mahmoudvand","year":"2010","journal-title":"IAENG Int. J. Appl. Math."}],"container-title":["Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2306-5729\/5\/3\/84\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:09:37Z","timestamp":1760177377000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2306-5729\/5\/3\/84"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,9,13]]},"references-count":28,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2020,9]]}},"alternative-id":["data5030084"],"URL":"https:\/\/doi.org\/10.3390\/data5030084","relation":{},"ISSN":["2306-5729"],"issn-type":[{"type":"electronic","value":"2306-5729"}],"subject":[],"published":{"date-parts":[[2020,9,13]]}}}