{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,4]],"date-time":"2025-11-04T23:08:46Z","timestamp":1762297726769,"version":"build-2065373602"},"reference-count":44,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2011,3,18]],"date-time":"2011-03-18T00:00:00Z","timestamp":1300406400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In this paper we review a particular connection between information theory and group theory. We formalize the notions of information elements and information lattices, first proposed by Shannon. Exploiting this formalization, we expose a comprehensive parallelism between information lattices and subgroup lattices. Qualitatively, isomorphisms between information lattices and subgroup lattices are demonstrated. Quantitatively, a decisive approximation relation between the entropy structures of information lattices and the log-index structures of the corresponding subgroup lattices, first discovered by Chan and Yeung, is highlighted. This approximation, addressing both joint and common entropies, extends the work of Chan and Yeung on joint entropy. A consequence of this approximation result is that any continuous law holds in general for the entropies of information elements if and only if the same law holds in general for the log-indices of subgroups. As an application, by constructing subgroup counterexamples, we find surprisingly that common information, unlike joint information, obeys neither the submodularity nor the supermodularity law. We emphasize that the notion of information elements is conceptually significant\u2014formalizing it helps to reveal the deep connection between information theory and group theory. The parallelism established in this paper admits an appealing group-action explanation and provides useful insights into the intrinsic structure among information elements from a group-theoretic perspective.<\/jats:p>","DOI":"10.3390\/e13030683","type":"journal-article","created":{"date-parts":[[2011,3,18]],"date-time":"2011-03-18T10:30:28Z","timestamp":1300444228000},"page":"683-708","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["On a Connection between Information and Group Lattices"],"prefix":"10.3390","volume":"13","author":[{"given":"Hua","family":"Li","sequence":"first","affiliation":[{"name":"FICO, 3661 Valley Centre Drive, San Diego, CA 92130, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Edwin K. P.","family":"Chong","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Colorado State University, Fort Collins, CO 80523, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2011,3,18]]},"reference":[{"key":"ref_1","first-page":"105","article-title":"The lattice theory of information","volume":"1","author":"Shannon","year":"1953","journal-title":"IEEE Trans. Inform. Theory"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1992","DOI":"10.1109\/TIT.2002.1013138","article-title":"On a relation between information inequalities and group theory","volume":"48","author":"Chan","year":"2002","journal-title":"IEEE Trans. Inform. Theory"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"unstructured":"Billingsley, P. (1995). Probability and Measure, Wiley-Interscience. [3rd ed.].","key":"ref_4"},{"doi-asserted-by":"crossref","unstructured":"Shreve, S.E. (2005). Stochastic Calculus for Finance I: The Binomial Asset Pricing Model, Springer.","key":"ref_5","DOI":"10.1007\/978-0-387-22527-2"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"743","DOI":"10.1214\/009117905000000648","article-title":"The Shannon information of filtrations and the additional logarithmic utility of insiders","volume":"34","author":"Ankirchner","year":"2006","journal-title":"Ann. Probab."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1469","DOI":"10.1109\/TIT.2004.830761","article-title":"Universal compression of memoryless sources over unknown alphabets","volume":"50","author":"Orlitsky","year":"2004","journal-title":"IEEE Trans. Inform. Theory"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"843","DOI":"10.1023\/A:1007818830500","article-title":"Entropy and convergence on compact groups","volume":"13","author":"Johnson","year":"2000","journal-title":"J. Theor. Probability"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"222","DOI":"10.3390\/e11020222","article-title":"Maximum entropy on compact groups","volume":"11","year":"2009","journal-title":"Entropy"},{"doi-asserted-by":"crossref","unstructured":"Chirikjian, G.S. (2009). Stochastic Models, Information Theory, and Lie Groups: Volume 1 Classical Results and Geometric Methods, Birkhauser.","key":"ref_10","DOI":"10.1007\/978-0-8176-4803-9"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"119","DOI":"10.3934\/jgm.2010.2.119","article-title":"Information-Theoretic Inequalities on Unimodular Lie Groups","volume":"2","author":"Chirikjian","year":"2010","journal-title":"J. Geom. Mech."},{"doi-asserted-by":"crossref","unstructured":"Johnson, O. (2004). Information Theory and the Central Limit Theorem, Imperial College Press.","key":"ref_12","DOI":"10.1142\/9781860945373"},{"unstructured":"Willsky, A.S. (1973). Dynamical Systems Defined on Groups: Structural Properties and Estimation. [Ph.D. dissertation, Dept. Aeronautics and Astronautics, MIT].","key":"ref_13"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1137\/1112029","article-title":"Necessary and sufficient statistics for the family of shifts of probability distributions on continuous bicompact groups","volume":"12","author":"Maksimov","year":"1967","journal-title":"Theor. Probab. Appl."},{"key":"ref_15","first-page":"82","article-title":"Exponential families of densities on an analytic group and sufficient statistics","volume":"37","author":"Roy","year":"1975","journal-title":"Sankhy A"},{"unstructured":"Renyi, A. (1970). Foundations of Probability, Holden-Day Inc.","key":"ref_16"},{"doi-asserted-by":"crossref","unstructured":"Yan, X., Yeung, R.W., and Zhang, Z. (2007, January 24\u201329). The Capacity region for multi-source multi-sink network coding. Proceedings of the 2007 IEEE International Symposium on Information Theory, Nice, France.","key":"ref_17","DOI":"10.1109\/ISIT.2007.4557213"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2345","DOI":"10.1109\/TIT.2006.874531","article-title":"On the capacity of information networks","volume":"52","author":"Harvey","year":"2006","journal-title":"IEEE Trans. Inform. Theory"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1007\/BF02482893","article-title":"Every finite lattice can be embedded in a finite partition lattice","volume":"10","year":"1980","journal-title":"Algebra Univ."},{"key":"ref_20","first-page":"149","article-title":"Common information is far less than mutual information","volume":"2","year":"1973","journal-title":"Probl. Control Inform. Theory"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1137\/0128010","article-title":"On sequences of pairs of dependent random variables","volume":"28","author":"Witsenhausen","year":"1975","journal-title":"SIAM J. Appl. Math."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1121","DOI":"10.1109\/18.243431","article-title":"Common randomness in information theory and cryptography\u2014Part I: Secret sharing","volume":"39","author":"Ahlswede","year":"1993","journal-title":"IEEE Trans. Inform. Theory"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"225","DOI":"10.1109\/18.651026","article-title":"Common randomness in information cryptography\u2014Part II: CR capacity","volume":"44","author":"Ahlswede","year":"1998","journal-title":"IEEE Trans. Inform. Theory"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1109\/18.825796","article-title":"Common randomness and secret key generation with a helper","volume":"46","author":"Narayan","year":"2000","journal-title":"IEEE Trans. Inform. Theory"},{"unstructured":"Wolf, S., and Wullschleger, J. (2004, January 24\u201329). Zero-error information and application in cryptography. Proceedings of the 2004 IEEE Information Theory Workshop (ITW 2004), San Antonio, TX, USA.","key":"ref_25"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"648","DOI":"10.1109\/TIT.1980.1056273","article-title":"Multiple access channels with arbitrarily correlated sources","volume":"26","author":"Cover","year":"1980","journal-title":"IEEE Trans. Inform. Theory"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"47","DOI":"10.4310\/CIS.2003.v3.n1.a4","article-title":"On a new non-Shannon type information inequality","volume":"3","author":"Zhang","year":"2003","journal-title":"Comm. Inform. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"442","DOI":"10.1006\/jcss.1999.1677","article-title":"Inequalities for Shannon entropy and Kolmogorov complexity","volume":"60","author":"Hammer","year":"2000","journal-title":"J. Comput. Syst. Sci."},{"unstructured":"Niesen, U., Fragouli, C., and Tuninetti, D. (2011). On capacity of line networks. IEEE Trans. Inform. Theory, submitted for publication.","key":"ref_29"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/S0019-9958(78)91063-X","article-title":"Polymatroidal dependence structure of a set of random variables","volume":"39","author":"Fujishige","year":"1978","journal-title":"Inform. Contr."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"933","DOI":"10.1109\/18.992785","article-title":"Supermodularity and subadditivity properties of entropy on the majorization lattice","volume":"48","author":"Cicalese","year":"2002","journal-title":"IEEE Trans. Inform. Theory"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/S0304-3975(01)00032-9","article-title":"Upper semilattice of binary strings with the relation `x is simple conditional to y\u2019","volume":"271","author":"Chernov","year":"2002","journal-title":"Theor. Comput. Sci."},{"unstructured":"Dummit, D.S., and Foote, R.M. (2003). Abstract Algebra, Wiley. [3rd ed.].","key":"ref_33"},{"doi-asserted-by":"crossref","unstructured":"Yeung, R.W. (2002). A First Course in Information Theory, Kluwer Academic\/Plenum Publishers.","key":"ref_34","DOI":"10.1007\/978-1-4419-8608-5"},{"unstructured":"Oxley, J.G. (1992). Matroid Theory, Oxford University Press.","key":"ref_35"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1440","DOI":"10.1109\/18.681320","article-title":"On characterization of entropy function via information inequalities","volume":"44","author":"Zhang","year":"1998","journal-title":"IEEE Trans. Inform. Theory"},{"doi-asserted-by":"crossref","unstructured":"Dougherty, R., Freiling, C., and Zeger, K. (2006, January 9\u201314). Six New Non-Shannon information inequalities. Proceedings of the 2006 IEEE International Symposium on Information Theory, Seattle, WA, USA.","key":"ref_37","DOI":"10.1109\/ISIT.2006.261840"},{"unstructured":"Pippenger, N. (1986, January 3\u20135). What are the laws of information theory. Proceedings of the 1986 Special Problems on Communication and Computation Conference, Palo Alto, CA, USA.","key":"ref_38"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1109\/TIT.2005.860438","article-title":"Piecewise linear conditional information inequality","volume":"52","year":"2006","journal-title":"IEEE Trans. Inform. Theory"},{"doi-asserted-by":"crossref","unstructured":"Li, H., and Chong, E.K.P. (2007, January 24\u201329). On connections between group homomorphisms and the Ingleton inequality. Proceedings of the 2007 IEEE International Symposium on Information Theory, Nice, France.","key":"ref_40","DOI":"10.1109\/ISIT.2007.4557514"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"379","DOI":"10.3390\/e13020379","article-title":"Recent progresses in characterising information inequalities","volume":"13","author":"Chan","year":"2011","journal-title":"Entropy"},{"doi-asserted-by":"crossref","unstructured":"Chan, T. (2005, January 4\u20139). On the optimality of group network codes. Proceedings of the International Symposium on Information Theory, Adelaide, Australia.","key":"ref_42","DOI":"10.1109\/ISIT.2005.1523694"},{"doi-asserted-by":"crossref","unstructured":"Mao, W., Thill, M., and Hassibi, B. (2010, January 13\u201318). On group network codes: Ingleton-bound violations and independent sources. Proceedings of the 2010 IEEE International Symposium on Information Theory, Austin, TX, USA.","key":"ref_43","DOI":"10.1109\/ISIT.2010.5513749"},{"doi-asserted-by":"crossref","unstructured":"Mat\u00fa\u0161, F. (2007, January 24\u201329). Infinitely many information inequalities. Proceedings of the 2007 IEEE International Symposium on Information Theory, Nice, France.","key":"ref_44","DOI":"10.1109\/ISIT.2007.4557201"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/13\/3\/683\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:55:34Z","timestamp":1760219734000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/13\/3\/683"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,3,18]]},"references-count":44,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2011,3]]}},"alternative-id":["e13030683"],"URL":"https:\/\/doi.org\/10.3390\/e13030683","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2011,3,18]]}}}