{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T04:10:59Z","timestamp":1760242259804,"version":"build-2065373602"},"reference-count":25,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2017,2,16]],"date-time":"2017-02-16T00:00:00Z","timestamp":1487203200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>This paper explores the idea of information loss through data compression, as occurs in the course of any data analysis, illustrated via detailed consideration of the Binomial distribution. We examine situations where the full sequence of binomial outcomes is retained, situations where only the total number of successes is retained, and in-between situations. We show that a familiar decomposition of the Shannon entropy H can be rewritten as a decomposition into      H  t o t a l      ,      H  l o s t      , and      H  c o m p      , or the total, lost and compressed (remaining) components, respectively. We relate this new decomposition to Landauer\u2019s principle, and we discuss some implications for the \u201cinformation-dynamic\u201d theory being developed in connection with our broader program to develop a measure of statistical evidence on a properly calibrated scale.<\/jats:p>","DOI":"10.3390\/e19020075","type":"journal-article","created":{"date-parts":[[2017,2,16]],"date-time":"2017-02-16T12:55:34Z","timestamp":1487249734000},"page":"75","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Information Loss in Binomial Data Due to Data Compression"],"prefix":"10.3390","volume":"19","author":[{"given":"Susan","family":"Hodge","sequence":"first","affiliation":[{"name":"Battelle Center for Mathematical Medicine, The Research Institute, Nationwide Children\u2019s Hospital, Columbus, OH 43215, USA"},{"name":"Department of Pediatrics, The Ohio State University, Columbus, OH 43210, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Veronica","family":"Vieland","sequence":"additional","affiliation":[{"name":"Battelle Center for Mathematical Medicine, The Research Institute, Nationwide Children\u2019s Hospital, Columbus, OH 43215, USA"},{"name":"Department of Pediatrics, The Ohio State University, Columbus, OH 43210, USA"},{"name":"Department of Statistics, The Ohio State University, Columbus, OH 43210, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,2,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1007\/s12064-013-0180-9","article-title":"Measurement of statistical evidence on an absolute scale following thermodynamic principles","volume":"132","author":"Vieland","year":"2013","journal-title":"Theory Biosci."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Vieland, V.J., and Seok, S.-C. (2016). Statistical evidence measured on a properly calibrated scale for multinomial hypothesis comparisons. Entropy, 18.","DOI":"10.3390\/e18040114"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1159\/000367599","article-title":"Evidence, temperature, and the laws of thermodynamics","volume":"78","author":"Vieland","year":"2014","journal-title":"Hum. Hered."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1147\/rd.53.0183","article-title":"Irreversibility and heat generation in the computing process","volume":"5","author":"Landauer","year":"1961","journal-title":"IBM J. Res. Dev."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"21","DOI":"10.3390\/e6010021","article-title":"The deep physics behind the second law: Information and energy as independent forms of bookkeeping","volume":"6","author":"Duncan","year":"2004","journal-title":"Entropy"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1767","DOI":"10.1007\/s10701-007-9159-z","article-title":"Information loss as a foundational principle for the second law of thermodynamics","volume":"37","author":"Duncan","year":"2007","journal-title":"Found. Phys."},{"key":"ref_7","unstructured":"Stuart, A., Ord, K., and Arnold, S. (2010). Kendall\u2019s Advancd Theory of Statistics, Classical Inference, and the Linear Model, Wiley."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell. Syst. Tech. J."},{"key":"ref_9","unstructured":"Attard, P. (arXiv, 2012). Is the information entropy the same as the statistical mechanical entropy?, arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Toffoli, T. (2016). Entropy? Honest!. Entropy, 18.","DOI":"10.3390\/e18070247"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Zwillinger, D., and Kokoska, S. (2000). CRC Standard Probability and Statistics Tables and Formulae, Chapman & Hall\/CRC.","DOI":"10.1201\/9780367802417"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"941","DOI":"10.1093\/bjps\/45.4.941","article-title":"The second law of probability dynamics","volume":"45","author":"Barrett","year":"1994","journal-title":"Br. J. Philos. Sci."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"5333","DOI":"10.3390\/e17085333","article-title":"Statistical evidence measured on a properly calibrated scale across nested and non-nested hypothesis comparisons","volume":"17","author":"Vieland","year":"2015","journal-title":"Entropy"},{"key":"ref_15","unstructured":"Jaynes, E.T. The Gibbs Paradox. Available online: http:\/\/worrydream.com\/refs\/Jaynes%20-%20The%20Gibbs%20Paradox.pdf."},{"key":"ref_16","unstructured":"Kullback, S. (1968). Information Theory and Statistics, Dover."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1349","DOI":"10.1080\/01621459.2000.10474346","article-title":"Principal information theoretic approaches","volume":"95","author":"Soofi","year":"2000","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_18","first-page":"338","article-title":"Information, weight of evidence, the singularity between probability measures and signal detection","volume":"376","author":"Osteyee","year":"1970","journal-title":"Lect. Notes Math."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Edwards, A.W.F. (1992). Likelihood: Expanded Edition, Hopkins.","DOI":"10.56021\/9780801844454"},{"key":"ref_20","unstructured":"Royall, R. (1997). Statistical Evidence: A Likelihood Paradigm, Chapman & Hall."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Taper, M.L., and Lele, S.R. (2004). The Nature of Statistical Evidence, University of Chicago.","DOI":"10.7208\/chicago\/9780226789583.001.0001"},{"key":"ref_22","first-page":"1147","article-title":"The strength of statistical evidence for composite hypotheses: Inference to the best explanation","volume":"22","author":"Bickel","year":"2012","journal-title":"Stat. Sin."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1093\/jigpal\/jzt023","article-title":"Bayesian epistemic values: Focus on surprise, measure probability!","volume":"22","author":"Stern","year":"2014","journal-title":"Log. J. IGPL"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Evans, M. (2015). Measuring Statistical Evidence Using Relative Belief, Chapman & Hall\/CRC.","DOI":"10.1201\/b18587"},{"key":"ref_25","unstructured":"Zhang, Z. (arXiv, 2009). A Law of Likelihood for Composite Hypotheses, arXiv."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/2\/75\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:28:25Z","timestamp":1760207305000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/2\/75"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,2,16]]},"references-count":25,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2017,2]]}},"alternative-id":["e19020075"],"URL":"https:\/\/doi.org\/10.3390\/e19020075","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2017,2,16]]}}}