{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T20:10:41Z","timestamp":1779912641298,"version":"3.53.1"},"reference-count":52,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2017,2,16]],"date-time":"2017-02-16T00:00:00Z","timestamp":1487203200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Williams and Beer (2010) proposed a nonnegative mutual information decomposition, based on the construction of information gain lattices, which allows separating the information that a set of variables contains about another variable into components, interpretable as the unique information of one variable, or redundant and synergy components. In this work, we extend this framework focusing on the lattices that underpin the decomposition. We generalize the type of constructible lattices and examine the relations between different lattices, for example, relating bivariate and trivariate decompositions. We point out that, in information gain lattices, redundancy components are invariant across decompositions, but unique and synergy components are decomposition-dependent. Exploiting the connection between different lattices, we propose a procedure to construct, in the general multivariate case, information gain decompositions from measures of synergy or unique information. We then introduce an alternative type of lattices, information loss lattices, with the role and invariance properties of redundancy and synergy components reversed with respect to gain lattices, and which provide an alternative procedure to build multivariate decompositions. We finally show how information gain and information loss dual lattices lead to a self-consistent unique decomposition, which allows a deeper understanding of the origin and meaning of synergy and redundancy.<\/jats:p>","DOI":"10.3390\/e19020071","type":"journal-article","created":{"date-parts":[[2017,2,16]],"date-time":"2017-02-16T12:55:34Z","timestamp":1487249734000},"page":"71","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Synergy and Redundancy in Dual Decompositions of Mutual Information Gain and Information Loss"],"prefix":"10.3390","volume":"19","author":[{"given":"Daniel","family":"Chicharro","sequence":"first","affiliation":[{"name":"Department of Neurobiology, Harvard Medical School, Boston, MA 02115, USA"},{"name":"Neural Computation Laboratory, Center for Neuroscience and Cognitive Systems@UniTn, Istituto Italiano di Tecnologia, Rovereto (TN) 38068, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1700-8909","authenticated-orcid":false,"given":"Stefano","family":"Panzeri","sequence":"additional","affiliation":[{"name":"Neural Computation Laboratory, Center for Neuroscience and Cognitive Systems@UniTn, Istituto Italiano di Tecnologia, Rovereto (TN) 38068, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2017,2,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1007\/s10827-013-0458-4","article-title":"Synergy, redundancy, and multivariate information measures: An experimentalist\u2019s perspective","volume":"36","author":"Timme","year":"2014","journal-title":"J. Comput. Neurosci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1038\/msb4100124","article-title":"Computational analysis of the synergy among multiple interacting genes","volume":"3","author":"Anastassiou","year":"2007","journal-title":"Mol. Syst. Biol."},{"key":"ref_3","first-page":"223","article-title":"Information-theoretic Sensitivity Analysis: A general method for credit assignment in complex networks","volume":"19","author":"Panzeri","year":"2008","journal-title":"J. R. Soc. Interface"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1111\/j.1749-6632.2008.03757.x","article-title":"Inference of regulatory gene interactions from expression data using three-way mutual information","volume":"1158","author":"Watkinson","year":"2009","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Oizumi, M., Albantakis, L., and Tononi, G. (2014). From the phenomenology to the mechanisms of consciousness: Integrated information theory 3.0. PLoS Comput. Biol., 10.","DOI":"10.1371\/journal.pcbi.1003588"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2488","DOI":"10.1109\/TBME.2016.2569823","article-title":"An Information-Theoretic Framework to Map the Spatiotemporal Dynamics of the Scalp Electroencephalogram","volume":"63","author":"Faes","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1038\/nrn1888","article-title":"Neural correlations, population coding and computation","volume":"7","author":"Averbeck","year":"2006","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/j.tics.2015.01.002","article-title":"Neural population coding: Combining insights from microscopic and mass signals","volume":"19","author":"Panzeri","year":"2015","journal-title":"Trends Cogn. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1038\/nn.3309","article-title":"Inferring decoding strategies from choice probabilities in the presence of correlated variability","volume":"16","author":"Haefner","year":"2013","journal-title":"Nat. Neurosci."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1016\/j.neuron.2016.12.036","article-title":"Cracking the neural code for sensory perception by combining statistics, intervention, and behavior","volume":"93","author":"Panzeri","year":"2017","journal-title":"Neuron"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Wibral, M., Vicente, R., and Lizier, J.T. (2014). Directed Information Measures in Neuroscience, Springer.","DOI":"10.1007\/978-3-642-54474-3"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1001","DOI":"10.1098\/rspb.1999.0736","article-title":"Correlations and the encoding of information in the nervous system","volume":"266","author":"Panzeri","year":"1999","journal-title":"Proc. Biol. Sci."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1088\/0954-898X\/14\/1\/303","article-title":"An exact method to quantify the information transmitted by different mechanisms of correlational coding","volume":"14","author":"Pola","year":"2003","journal-title":"Netw. Comput. Neural Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1701","DOI":"10.1109\/18.930911","article-title":"Information geometry on hierarchy of probability distributions","volume":"47","author":"Amari","year":"2001","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1016\/j.neunet.2010.05.008","article-title":"Information-theoretic methods for studying population codes","volume":"23","author":"Ince","year":"2010","journal-title":"Neural Netw."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"5195","DOI":"10.1523\/JNEUROSCI.5319-04.2005","article-title":"Synergy, Redundancy, and Independence in Population Codes, Revisited","volume":"25","author":"Latham","year":"2005","journal-title":"J. Neurosci."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"999","DOI":"10.1162\/NECO_a_00588","article-title":"A Causal Perspective on the Analysis of Signal and Noise Correlations and Their Role in Population Coding","volume":"26","author":"Chicharro","year":"2014","journal-title":"Neural Comput."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"11539","DOI":"10.1523\/JNEUROSCI.23-37-11539.2003","article-title":"Synergy, redundancy, and independence in population codes","volume":"23","author":"Schneidman","year":"2003","journal-title":"J. Neurosci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1007\/BF02289159","article-title":"Multivariate information transmission","volume":"19","author":"McGill","year":"1954","journal-title":"Psychometrika"},{"key":"ref_20","unstructured":"Bell, A.J. (2003, January 1\u20134). The co-information lattice. Proceedings of the 4th international Symposium on Independent Component Analysis and Blind Source Separation, Nara, Japan."},{"key":"ref_21","unstructured":"Williams, P.L., and Beer, R.D. (2010). Nonnegative Decomposition of Multivariate Information. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"012130","DOI":"10.1103\/PhysRevE.87.012130","article-title":"Bivariate measure of redundant information","volume":"87","author":"Harder","year":"2013","journal-title":"Phys. Rev. E"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Griffith, V., and Koch, C. (2013). Quantifying synergistic mutual information. arXiv.","DOI":"10.1007\/978-3-642-53734-9_6"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2161","DOI":"10.3390\/e16042161","article-title":"Quantifying unique information","volume":"16","author":"Bertschinger","year":"2014","journal-title":"Entropy"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ince, R.A.A. (2016). Measuring multivariate redundant information with pointwise common change in surprisal. arxiv.","DOI":"10.3390\/e19070318"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Rauh, J., Bertschinger, N., Olbrich, E., and Jost, J. (July, January 29). Reconsidering unique information: Towards a multivariate information decomposition. Proceedings of the 2014 IEEE International Symposium on Information Theory, Honolulu, HI, USA.","DOI":"10.1109\/ISIT.2014.6875230"},{"key":"ref_27","unstructured":"Williams, P.L. (2011). Information Dynamics: Its Theory and Application to Embodied Cognitive Systems. [Ph.D. Thesis, Indiana University]."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1985","DOI":"10.3390\/e16041985","article-title":"Intersection Information based on Common Randomness","volume":"16","author":"Griffith","year":"2014","journal-title":"Entropy"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Bertschinger, N., Rauh, J., Olbrich, E., and Jost, J. Shared Information\u2014New Insights and Problems in Decomposing Information in Complex Systems. Proceedings of the European Conference on Complex Systems 2012.","DOI":"10.1007\/978-3-319-00395-5_35"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"052802","DOI":"10.1103\/PhysRevE.91.052802","article-title":"Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems","volume":"91","author":"Barrett","year":"2015","journal-title":"Phys. Rev. E"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of Information Theory, Wiley. [2nd ed.].","DOI":"10.1002\/047174882X"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Ince, R.A.A. (2017). The Partial Entropy Decomposition: Decomposing multivariate entropy and mutual information via pointwise common surprisal. arxiv.","DOI":"10.3390\/e19070318"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"3501","DOI":"10.3390\/e17053501","article-title":"Information decomposition and synergy","volume":"17","author":"Olbrich","year":"2015","journal-title":"Entropy"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Perrone, P., and Ay, N. (2016). Hierarchical quantification of synergy in channels. arXiv.","DOI":"10.3389\/frobt.2015.00035"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"238701","DOI":"10.1103\/PhysRevLett.91.238701","article-title":"Network information and connected correlations","volume":"91","author":"Schneidman","year":"2003","journal-title":"Phys. Rev. Lett."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"041901","DOI":"10.1103\/PhysRevE.86.041901","article-title":"Framework to study dynamic dependencies in networks of interacting processes","volume":"86","author":"Chicharro","year":"2012","journal-title":"Phys. Rev. E"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"032904","DOI":"10.1103\/PhysRevE.91.032904","article-title":"Estimating the decomposition of predictive information in multivariate systems","volume":"91","author":"Faes","year":"2015","journal-title":"Phys. Rev. E"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"339","DOI":"10.1016\/j.neuroimage.2011.03.058","article-title":"Effective connectivity: Influence, causality and biophysical modeling","volume":"58","author":"Roebroeck","year":"2011","journal-title":"Neuroimage"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Solo, V. (2008, January 9\u201311). On causality and Mutual information. Proceedings of the 47th IEEE Conference on Decision and Control, Cancun, Mexico.","DOI":"10.1109\/CDC.2008.4738640"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1007\/s00422-011-0469-z","article-title":"On the spectral formulation of Granger causality","volume":"105","author":"Chicharro","year":"2011","journal-title":"Biol. Cybern."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"105003","DOI":"10.1088\/1367-2630\/16\/10\/105003","article-title":"Synergy and redundancy in the Granger causal analysis of dynamical networks","volume":"16","author":"Stramaglia","year":"2014","journal-title":"New J. Phys."},{"key":"ref_42","unstructured":"Williams, P.L., and Beer, R.D. (2011). Generalized Measures of Information Transfer. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Lizier, J., Flecker, B., and Williams, P. (2013, January 16\u201319). Towards a synergy-based approach to measuring information modification. Proceedings of the IEEE Symposium on Artificial Life, Singapore.","DOI":"10.1109\/ALIFE.2013.6602430"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.bandc.2015.09.004","article-title":"Partial information decomposition as a unified approach to the specification of neural goal functions","volume":"112","author":"Wibral","year":"2017","journal-title":"Brain Cogn."},{"key":"ref_45","unstructured":"Banerjee, P.K., and Griffith, V. (2015). Synergy, redundancy, and common information. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"James, R.G., and Crutchfield, J.P. (2016). Multivariate Dependence Beyond Shannon Information. arXiv.","DOI":"10.3390\/e19100531"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"64","DOI":"10.3389\/fninf.2014.00064","article-title":"Algorithms of causal inference for the analysis of effective connectivity among brain regions","volume":"8","author":"Chicharro","year":"2014","journal-title":"Front. Neuroinform."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"958","DOI":"10.1038\/nn.3419","article-title":"Neural coding during active somatosensation revealed using illusory touch","volume":"16","author":"Hires","year":"2013","journal-title":"Nat. Neurosci."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1038\/nature16442","article-title":"Acute off-target effects of neural circuit manipulations","volume":"528","author":"Otchy","year":"2015","journal-title":"Nature"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"17","DOI":"10.1142\/S0219525908001465","article-title":"Information flows in causal networks","volume":"11","author":"Ay","year":"2008","journal-title":"Adv. Complex Syst."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"605","DOI":"10.1140\/epjb\/e2010-00034-5","article-title":"Differentiating information transfer and causal effect","volume":"73","author":"Lizier","year":"2010","journal-title":"Eur. Phys. J. B."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Chicharro, D., and Ledberg, A. (2012). When Two Become One: The Limits of Causality Analysis of Brain Dynamics. PLoS ONE, 7.","DOI":"10.1371\/journal.pone.0032466"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/2\/71\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:28:23Z","timestamp":1760207303000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/2\/71"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,2,16]]},"references-count":52,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2017,2]]}},"alternative-id":["e19020071"],"URL":"https:\/\/doi.org\/10.3390\/e19020071","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,2,16]]}}}