{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,3]],"date-time":"2026-02-03T17:05:48Z","timestamp":1770138348060,"version":"3.49.0"},"reference-count":48,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2017,8,28]],"date-time":"2017-08-28T00:00:00Z","timestamp":1503878400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100009152","name":"Fondation Bertarelli","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100009152","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In a system of three stochastic variables, the Partial Information Decomposition (PID) of Williams and Beer dissects the information that two variables (sources) carry about a third variable (target) into nonnegative information atoms that describe redundant, unique, and synergistic modes of dependencies among the variables. However, the classification of the three variables into two sources and one target limits the dependency modes that can be quantitatively resolved, and does not naturally suit all systems. Here, we extend the PID to describe trivariate modes of dependencies in full generality, without introducing additional decomposition axioms or making assumptions about the target\/source nature of the variables. By comparing different PID lattices of the same system, we unveil a finer PID structure made of seven nonnegative information subatoms that are invariant to different target\/source classifications and that are sufficient to describe the relationships among all PID lattices. This finer structure naturally splits redundant information into two nonnegative components: the source redundancy, which arises from the pairwise correlations between the source variables, and the non-source redundancy, which does not, and relates to the synergistic information the sources carry about the target. The invariant structure is also sufficient to construct the system\u2019s entropy, hence it characterizes completely all the interdependencies in the system.<\/jats:p>","DOI":"10.3390\/e19090451","type":"journal-article","created":{"date-parts":[[2017,8,28]],"date-time":"2017-08-28T12:08:37Z","timestamp":1503922117000},"page":"451","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":31,"title":["Invariant Components of Synergy, Redundancy, and Unique Information among Three Variables"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4506-0600","authenticated-orcid":false,"given":"Giuseppe","family":"Pica","sequence":"first","affiliation":[{"name":"Neural Computation Laboratory, Center for Neuroscience and Cognitive Systems @UniTn, Istituto Italiano di Tecnologia, 38068 Rovereto (TN), Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0384-7699","authenticated-orcid":false,"given":"Eugenio","family":"Piasini","sequence":"additional","affiliation":[{"name":"Neural Computation Laboratory, Center for Neuroscience and Cognitive Systems @UniTn, Istituto Italiano di Tecnologia, 38068 Rovereto (TN), Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Daniel","family":"Chicharro","sequence":"additional","affiliation":[{"name":"Neural Computation Laboratory, Center for Neuroscience and Cognitive Systems @UniTn, Istituto Italiano di Tecnologia, 38068 Rovereto (TN), Italy"},{"name":"Department of Neurobiology, Harvard Medical School, Boston, MA 02115, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1700-8909","authenticated-orcid":false,"given":"Stefano","family":"Panzeri","sequence":"additional","affiliation":[{"name":"Neural Computation Laboratory, Center for Neuroscience and Cognitive Systems @UniTn, Istituto Italiano di Tecnologia, 38068 Rovereto (TN), Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,8,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A Mathematical Theory of Communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_2","unstructured":"Ay, N., Olbrich, E., Bertschinger, N., and Jost, J. (2006, January 25\u201329). A unifying framework for complexity measures of finite systems. Proceedings of the European Conference Complex Systems, Oxford, UK."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Bertschinger, N., Rauh, J., Olbrich, E., and Jost, J. (2012, January 3\u20137). Shared Information\u2014New Insights and Problems in Decomposing Information in Complex Systems. Proceedings of the Proceedings of the ECCS 2012, Brussels, Belguim.","DOI":"10.1007\/978-3-319-00395-5_35"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3257","DOI":"10.1073\/pnas.96.6.3257","article-title":"Measures of degeneracy and redundancy in biological networks","volume":"96","author":"Tononi","year":"1999","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"150486","DOI":"10.1098\/rsos.150486","article-title":"Only accessible information is useful: Insights from gradient-mediated patterning","volume":"2","author":"Tikhonov","year":"2015","journal-title":"R. Soc. Open Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1007\/s10827-013-0458-4","article-title":"Synergy, redundancy, and multivariate information measures: An experimentalist\u2019s perspective","volume":"36","author":"Timme","year":"2014","journal-title":"J. Comput. Neurosci."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1088\/0954-898X\/14\/1\/303","article-title":"An exact method to quantify the information transmitted by different mechanisms of correlational coding","volume":"14","author":"Pola","year":"2003","journal-title":"Network"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"11539","DOI":"10.1523\/JNEUROSCI.23-37-11539.2003","article-title":"Synergy, redundancy, and independence in population codes","volume":"23","author":"Schneidman","year":"2003","journal-title":"J. Neurosci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"5195","DOI":"10.1523\/JNEUROSCI.5319-04.2005","article-title":"Synergy, redundancy, and independence in population codes, revisited","volume":"25","author":"Latham","year":"2005","journal-title":"J. Neurosci."},{"key":"ref_10","unstructured":"Williams, P.L., and Beer, R.D. (arXiv, 2010). Nonnegative Decomposition of Multivariate Information, arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"2161","DOI":"10.3390\/e16042161","article-title":"Quantifying Unique Information","volume":"16","author":"Bertschinger","year":"2014","journal-title":"Entropy"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Chicharro, D., and Panzeri, S. (2017). Synergy and Redundancy in Dual Decompositions of Mutual Information Gain and Information Loss. Entropy, 19.","DOI":"10.3390\/e19020071"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1038\/msb4100124","article-title":"Computational analysis of the synergy among multiple interacting genes","volume":"3","author":"Anastassiou","year":"2007","journal-title":"Mol. Syst. Biol."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"223","DOI":"10.1098\/rsif.2007.1079","article-title":"Information-theoretic sensitivity analysis: A general method for credit assignment in complex networks","volume":"5","author":"Panzeri","year":"2008","journal-title":"J. R. Soc. Interface"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1111\/j.1749-6632.2008.03757.x","article-title":"Inference of Regulatory Gene Interactions from Expression Data Using Three-Way Mutual Information","volume":"1158","author":"Watkinson","year":"2009","journal-title":"Ann. N. Y. Acad. Sci."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2488","DOI":"10.1109\/TBME.2016.2569823","article-title":"An Information-Theoretic Framework to Map the Spatiotemporal Dynamics of the Scalp Electroencephalogram","volume":"63","author":"Faes","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"411","DOI":"10.1016\/j.neuron.2015.06.033","article-title":"How Can Single Sensory Neurons Predict Behavior?","volume":"87","author":"Pitkow","year":"2015","journal-title":"Neuron"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1038\/nn.3309","article-title":"Inferring decoding strategies from choice probabilities in the presence of correlated variability","volume":"16","author":"Haefner","year":"2013","journal-title":"Nat. Neurosci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"491","DOI":"10.1016\/j.neuron.2016.12.036","article-title":"Cracking the Neural Code for Sensory Perception by Combining Statistics, Intervention, and Behavior","volume":"93","author":"Panzeri","year":"2017","journal-title":"Neuron"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"25","DOI":"10.1016\/j.bandc.2015.09.004","article-title":"Partial information decomposition as a unified approach to the specification of neural goal functions","volume":"112","author":"Wibral","year":"2017","journal-title":"Brain Cogn."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"James, R.G., and Crutchfield, J.P. (arXiv, 2016). Multivariate Dependence Beyond Shannon Information, arXiv.","DOI":"10.3390\/e19100531"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"012130","DOI":"10.1103\/PhysRevE.87.012130","article-title":"Bivariate measure of redundant information","volume":"87","author":"Harder","year":"2013","journal-title":"Phys. Rev. E"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"052802","DOI":"10.1103\/PhysRevE.91.052802","article-title":"Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems","volume":"91","author":"Barrett","year":"2015","journal-title":"Phys. Rev. E"},{"key":"ref_24","unstructured":"Williams, P.L. (2011). Information Dynamics: Its Theory and Application to Embodied Cognitive Systems. [Ph.D. Thesis, Indiana University]."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ince, R.A.A. (2017). Measuring Multivariate Redundant Information with Pointwise Common Change in Surprisal. Entropy, 19.","DOI":"10.3390\/e19070318"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Griffith, V., and Koch, C. (2014). Quantifying Synergistic Mutual Information. Guided Self-Organization: Inception, Springer.","DOI":"10.1007\/978-3-642-53734-9_6"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1985","DOI":"10.3390\/e16041985","article-title":"Intersection Information Based on Common Randomness","volume":"16","author":"Griffith","year":"2014","journal-title":"Entropy"},{"key":"ref_28","unstructured":"Banerjee, P.K., and Griffith, V. (arXiv, 2015). Synergy, Redundancy and Common Information, arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ince, R.A.A. (arXiv, 2017). The Partial Entropy Decomposition: Decomposing multivariate entropy and mutual information via pointwise common surprisal, arXiv.","DOI":"10.3390\/e19070318"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Rauh, J., Banerjee, P.K., Olbrich, E., Jost, J., and Bertschinger, N. (arXiv, 2017). On extractable shared information, arXiv.","DOI":"10.3390\/e19070328"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Griffith, V., and Koch, C. (arXiv, 2013). Quantifying synergistic mutual information, arXiv.","DOI":"10.1007\/978-3-642-53734-9_6"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2518","DOI":"10.1109\/TBME.2016.2559578","article-title":"Synergetic and Redundant Information Flow Detected by Unnormalized Granger Causality: Application to Resting State fMRI","volume":"63","author":"Stramaglia","year":"2016","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Rosas, F., Ntranos, V., Ellison, C.J., Pollin, S., and Verhelst, M. (2016). Understanding Interdependency Through Complex Information Sharing. Entropy, 18.","DOI":"10.3390\/e18020038"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1007\/BF02289159","article-title":"Multivariate information transmission","volume":"19","author":"McGill","year":"1954","journal-title":"Psychometrika"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1016\/S0019-9958(78)90275-9","article-title":"Nonnegative entropy measures of multivariate symmetric correlations","volume":"36","author":"Han","year":"1978","journal-title":"Inf. Control"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1147\/rd.41.0066","article-title":"Information Theoretical Analysis of Multivariate Correlation","volume":"4","author":"Watanabe","year":"1960","journal-title":"IBM J. Res. Dev."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Margolin, A.A., Nemenman, I., Basso, K., Wiggins, C., Stolovitzky, G., Favera, R.D., and Califano, A. (2006). ARACNE: An Algorithm for the Reconstruction of Gene Regulatory Networks in a Mammalian Cellular Context. BMC Bioinform., 7.","DOI":"10.1186\/1471-2105-7-S1-S7"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"358","DOI":"10.1038\/nrn1888","article-title":"Neural correlations, population coding and computation","volume":"7","author":"Averbeck","year":"2006","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1038\/nrn2578","article-title":"Extracting information from neuronal populations: information theory and decoding approaches","volume":"10","author":"Quian","year":"2009","journal-title":"Nat. Rev. Neurosci."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/j.tics.2015.01.002","article-title":"Neural population coding: combining insights from microscopic and mass signals","volume":"19","author":"Panzeri","year":"2015","journal-title":"Trends Cogn. Sci."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Pearl, J. (2009). Causality: Models, Reasoning and Inference, Cambridge University Press. [2nd ed.].","DOI":"10.1017\/CBO9780511803161"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1016\/j.conb.2014.01.002","article-title":"Emerging principles of population coding: In search for the neural code","volume":"25","author":"Shamir","year":"2014","journal-title":"Curr. Opin. Neurobiol."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1038\/nature23020","article-title":"Distinct timescales of population coding across cortex","volume":"548","author":"Runyan","year":"2017","journal-title":"Nature"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Timme, N.M., Ito, S., Myroshnychenko, M., Nigam, S., Shimono, M., Yeh, F.C., Hottowy, P., Litke, A.M., and Beggs, J.M. (2016). High-Degree Neurons Feed Cortical Computations. PLoS Comput. Biol., 12.","DOI":"10.1371\/journal.pcbi.1004858"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1003","DOI":"10.1016\/j.neuron.2017.02.019","article-title":"Navigating the Neural Space in Search of the Neural Code","volume":"93","author":"Jazayeri","year":"2017","journal-title":"Neuron"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"978","DOI":"10.1016\/j.neuron.2017.05.025","article-title":"Neural Manifolds for the Control of Movement","volume":"94","author":"Gallego","year":"2017","journal-title":"Neuron"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1016\/j.neuron.2017.04.044","article-title":"Optimizing Neural Information Capacity through Discretization","volume":"94","author":"Sharpee","year":"2017","journal-title":"Neuron"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"943","DOI":"10.1016\/j.neuron.2017.05.028","article-title":"Inference in the Brain: Statistics Flowing in Redundant Population Codes","volume":"94","author":"Pitkow","year":"2017","journal-title":"Neuron"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/9\/451\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:43:31Z","timestamp":1760208211000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/9\/451"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,8,28]]},"references-count":48,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2017,9]]}},"alternative-id":["e19090451"],"URL":"https:\/\/doi.org\/10.3390\/e19090451","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,8,28]]}}}