{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T16:49:09Z","timestamp":1780505349672,"version":"3.54.1"},"reference-count":21,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2017,2,22]],"date-time":"2017-02-22T00:00:00Z","timestamp":1487721600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Seventh Framework Programme","award":["FP7-ICT-318121"],"award-info":[{"award-number":["FP7-ICT-318121"]}]},{"name":"Seventh Framework Programme","award":["FP7-ICT-317534"],"award-info":[{"award-number":["FP7-ICT-317534"]}]},{"DOI":"10.13039\/501100006769","name":"Russian Science Foundation","doi-asserted-by":"publisher","award":["14-21-00137"],"award-info":[{"award-number":["14-21-00137"]}],"id":[{"id":"10.13039\/501100006769","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Quantifying synergy among stochastic variables is an important open problem in information theory. Information synergy occurs when multiple sources together predict an outcome variable better than the sum of single-source predictions. It is an essential phenomenon in biology such as in neuronal networks and cellular regulatory processes, where different information flows integrate to produce a single response, but also in social cooperation processes as well as in statistical inference tasks in machine learning. Here we propose a metric of synergistic entropy and synergistic information from first principles. The proposed measure relies on so-called synergistic random variables (SRVs) which are constructed to have zero mutual information about individual source variables but non-zero mutual information about the complete set of source variables. We prove several basic and desired properties of our measure, including bounds and additivity properties. In addition, we prove several important consequences of our measure, including the fact that different types of synergistic information may co-exist between the same sets of variables. A numerical implementation is provided, which we use to demonstrate that synergy is associated with resilience to noise. Our measure may be a marked step forward in the study of multivariate information theory and its numerous applications.<\/jats:p>","DOI":"10.3390\/e19020085","type":"journal-article","created":{"date-parts":[[2017,2,22]],"date-time":"2017-02-22T11:36:58Z","timestamp":1487763418000},"page":"85","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":45,"title":["Quantifying Synergistic Information Using Intermediate Stochastic Variables"],"prefix":"10.3390","volume":"19","author":[{"given":"Rick","family":"Quax","sequence":"first","affiliation":[{"name":"Computational Science Lab, University of Amsterdam, 1098 XH Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0876-4457","authenticated-orcid":false,"given":"Omri","family":"Har-Shemesh","sequence":"additional","affiliation":[{"name":"Computational Science Lab, University of Amsterdam, 1098 XH Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3848-5395","authenticated-orcid":false,"given":"Peter","family":"Sloot","sequence":"additional","affiliation":[{"name":"The Institute for Advanced Study, University of Amsterdam, Oude Turfmarkt 147, 1012 GC Amsterdam, The Netherlands"},{"name":"Advanced Computing Lab, ITMO University, Kronverkskiy pr. 49, 197101 Saint Petersburg, Russia"},{"name":"Complexity Institute, Nanyang Technological University, 639673 Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2017,2,22]]},"reference":[{"key":"ref_1","unstructured":"Shannon, C.E., and Weaver, W. (1963). Mathematical Theory of Communication, University Illinois Press."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1147\/rd.41.0066","article-title":"Information theoretical analysis of multivariate correlation","volume":"4","author":"Watanabe","year":"1960","journal-title":"IBM J. Res. Dev."},{"key":"ref_3","unstructured":"Williams, P.L., and Beer, R.D. (2010). Nonnegative decomposition of multivariate information. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1007\/s10827-013-0458-4","article-title":"Synergy, redundancy, and multivariate information measures: An experimentalist\u2019s perspective","volume":"36","author":"Timme","year":"2014","journal-title":"J. Comput. Neurosci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"11539","DOI":"10.1523\/JNEUROSCI.23-37-11539.2003","article-title":"Synergy, redundancy, and independence in population codes","volume":"23","author":"Schneidman","year":"2003","journal-title":"J. Neurosci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1985","DOI":"10.3390\/e16041985","article-title":"Intersection Information Based on Common Randomness","volume":"16","author":"Griffith","year":"2014","journal-title":"Entropy"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lizier, J.T., Flecker, B., and Williams, P.L. (2013, January 16\u201319). Towards a synergy-based approach to measuring information modification. Proceedings of the 2013 IEEE Symposium on Artificial Life (ALIFE), Singapore.","DOI":"10.1109\/ALIFE.2013.6602430"},{"key":"ref_8","unstructured":"Bertschinger, N., Rauh, J., Olbrich, E., and Jost, J. (2013). Proceedings of the European Conference on Complex Systems 2012, Springer."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"3501","DOI":"10.3390\/e17053501","article-title":"Information Decomposition and Synergy","volume":"17","author":"Olbrich","year":"2015","journal-title":"Entropy"},{"key":"ref_10","unstructured":"Griffith, V., and Koch, C. (2014). Guided Self-Organization: Inception, Springer."},{"key":"ref_11","unstructured":"Brukner, C., Zukowski, M., and Zeilinger, A. (2001). The essence of entanglement. arXiv."},{"key":"ref_12","unstructured":"Cover, T.M., and Thomas, J.A. (1991). Elements of Information Theory, Wiley-Interscience."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wibral, M., Lizier, J.T., and Priesemann, V. (2015). Bits from Brains for Biologically Inspired Computing. Front. Robot. AI.","DOI":"10.3389\/frobt.2015.00005"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2161","DOI":"10.3390\/e16042161","article-title":"Quantifying unique information","volume":"16","author":"Bertschinger","year":"2014","journal-title":"Entropy"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1109\/TIT.1975.1055346","article-title":"The common information of two dependent random variables","volume":"21","author":"Wyner","year":"1975","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1137\/0131026","article-title":"Values and bounds for the common information of two discrete random variables","volume":"31","author":"Witsenhausen","year":"1976","journal-title":"SIAM J. Appl. Math."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Xu, G., Liu, W., and Chen, B. (2011, January 23\u201325). Wyners common information for continuous random variables\u2014A lossy source coding interpretation. Proceedings of the 45th Annual Conference on Information Sciences and Systems (CISS), Baltimore, MD, USA.","DOI":"10.1109\/CISS.2011.5766249"},{"key":"ref_18","first-page":"149","article-title":"Common information is far less than mutual information","volume":"2","year":"1973","journal-title":"Probl. Control Inf. Theory"},{"key":"ref_19","unstructured":"Karhunen, K. (1946). Zur Spektraltheorie Stochastischer Prozesse, Finnish Academy of Science and Letters. (In Germany)."},{"key":"ref_20","unstructured":"Loeve, M. (1994). Probability Theory, Springer. [4th ed.]. Graduate Texts in Mathematics (Book 46)."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1016\/j.probengmech.2005.05.007","article-title":"Simulation of strongly non-Gaussian processes using Karhunen\u2013Loeve expansion","volume":"20","author":"Phoon","year":"2005","journal-title":"Probabilistic Eng. Mech."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/2\/85\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:28:54Z","timestamp":1760207334000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/2\/85"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,2,22]]},"references-count":21,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2017,2]]}},"alternative-id":["e19020085"],"URL":"https:\/\/doi.org\/10.3390\/e19020085","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,2,22]]}}}