{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,30]],"date-time":"2025-12-30T03:40:26Z","timestamp":1767066026694,"version":"build-2065373602"},"reference-count":22,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2015,5,22]],"date-time":"2015-05-22T00:00:00Z","timestamp":1432252800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Community FP7","award":["318723"],"award-info":[{"award-number":["318723"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Recently, a series of papers addressed the problem of decomposing the information of two random variables into shared information, unique information and synergistic information. Several measures were proposed, although still no consensus has been reached. Here, we compare these proposals with an older approach to define synergistic information based on the projections on exponential families containing only up to k-th order interactions. We show that these measures are not compatible with a decomposition into unique, shared and synergistic information if one requires that all terms are always non-negative (local positivity). We illustrate the difference between the two measures for multivariate Gaussians.<\/jats:p>","DOI":"10.3390\/e17053501","type":"journal-article","created":{"date-parts":[[2015,5,26]],"date-time":"2015-05-26T04:16:36Z","timestamp":1432613796000},"page":"3501-3517","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":52,"title":["Information Decomposition and Synergy"],"prefix":"10.3390","volume":"17","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3987-1271","authenticated-orcid":false,"given":"Eckehard","family":"Olbrich","sequence":"first","affiliation":[{"name":"Max Planck Institute for Mathematics in the Sciences, Inselstra\u00dfe 22, 04103 Leipzig, Germany"}]},{"given":"Nils","family":"Bertschinger","sequence":"additional","affiliation":[{"name":"Frankfurt Institute for Advanced Studies, Ruth-Moufang-Stra\u00dfe 1, 60438 Frankfurt am Main, Germany"}]},{"given":"Johannes","family":"Rauh","sequence":"additional","affiliation":[{"name":"Institute of Algebraic Geometry, Leibniz Universit\u00e4t Hannover, Welfengarten 1, 30167 Hannover, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2015,5,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"11539","DOI":"10.1523\/JNEUROSCI.23-37-11539.2003","article-title":"Synergy, redundancy, and independence in population codes","volume":"23","author":"Schneidman","year":"2003","journal-title":"J. Neurosci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"428","DOI":"10.1049\/iet-syb.2010.0009","article-title":"Multivariate dependence and genetic networks inference","volume":"4","author":"Margolin","year":"2010","journal-title":"IET Syst. Biol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1016\/j.biosystems.2007.05.018","article-title":"Autonomy: An information theoretic perspective","volume":"91","author":"Bertschinger","year":"2008","journal-title":"Biosystems"},{"key":"ref_4","unstructured":"Williams, P., and Beer, R. (2010). Nonnegative Decomposition of Multivariate Information, arXiv, 1004.2515v1."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Bertschinger, N., Rauh, J., Olbrich, E., and Jost, J. (2012, January 3\u20137). Shared Information\u2014New Insights and Problems in Decomposing Information in Complex Systems, Brussels, Belgium.","DOI":"10.1007\/978-3-319-00395-5_35"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"159","DOI":"10.1007\/978-3-642-53734-9_6","article-title":"Quantifying Synergistic Mutual Information","volume":"9","author":"Prokopenko","year":"2014","journal-title":"Guided Self-Organization: Inception"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"012130","DOI":"10.1103\/PhysRevE.87.012130","article-title":"Bivariate measure of redundant information","volume":"87","author":"Harder","year":"2013","journal-title":"Phys. Rev. E"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2161","DOI":"10.3390\/e16042161","article-title":"Quantifying unique information","volume":"16","author":"Bertschinger","year":"2014","journal-title":"Entropy"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1701","DOI":"10.1109\/18.930911","article-title":"Information geometry on hierarchy of probability distributions","volume":"47","author":"Amari","year":"2001","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"238701","DOI":"10.1103\/PhysRevLett.91.238701","article-title":"Network information and connected correlations","volume":"91","author":"Schneidman","year":"2003","journal-title":"Phys. Rev. Lett."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"037103","DOI":"10.1063\/1.3638446","article-title":"A geometric approach to complexity","volume":"21","author":"Ay","year":"2011","journal-title":"Chaos"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"052802","DOI":"10.1103\/PhysRevE.91.052802","article-title":"Exploration of synergistic and redundant information sharing in static and dynamical Gaussian systems","volume":"91","author":"Barrett","year":"2015","journal-title":"Phys. Rev. E"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"613","DOI":"10.1016\/j.ijar.2011.01.013","article-title":"Support sets of exponential families and oriented matroids","volume":"52","author":"Rauh","year":"2011","journal-title":"Int. J Approx. Reason."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1561\/0100000004","article-title":"Information Theory and Statistics: A Tutorial","volume":"1","author":"Shields","year":"2004","journal-title":"Found. Trends in Commun. Inf. Theory."},{"key":"ref_15","first-page":"261","article-title":"The Multiinformation Function as a Tool for Measuring Stochastic Dependence","volume":"89","author":"Jordan","year":"1998","journal-title":"Learning in Graphical Models"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1147\/rd.41.0066","article-title":"Information Theoretical Analysis of Multivariate Correlation","volume":"4","author":"Watanabe","year":"1960","journal-title":"IBM J. Res. Dev."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"026201","DOI":"10.1103\/PhysRevE.79.026201","article-title":"Complexity Measures from Interaction Structures","volume":"79","author":"Kahle","year":"2009","journal-title":"Phys. Rev. E"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2758","DOI":"10.1523\/JNEUROSCI.13-07-02758.1993","article-title":"How independent are the messages carried by adjacent inferior temporal cortical neurons?","volume":"13","author":"Gawne","year":"1993","journal-title":"J. Neurosci."},{"key":"ref_19","unstructured":"Kearns, M., Solla, S., and Cohn, D. (1999). Advances in Neural Information Processing Systems 11, MIT Press."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Dietterich, T., Becker, S., and Ghahramani, Z. (2002). Advances in Neural Information Processing Systems 14, MIT Press.","DOI":"10.7551\/mitpress\/1120.001.0001"},{"key":"ref_21","unstructured":"Bell, A.J. (2003, January 1\u20134). The Co-Information Lattice, Nara, Japan."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"97","DOI":"10.1007\/BF02289159","article-title":"Multivariate information transmission","volume":"19","author":"McGill","year":"1954","journal-title":"Psychometrika"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/5\/3501\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T20:46:49Z","timestamp":1760215609000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/17\/5\/3501"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,5,22]]},"references-count":22,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2015,5]]}},"alternative-id":["e17053501"],"URL":"https:\/\/doi.org\/10.3390\/e17053501","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2015,5,22]]}}}