{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T01:26:37Z","timestamp":1772587597702,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2017,11,24]],"date-time":"2017-11-24T00:00:00Z","timestamp":1511481600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Information theory is often utilized to capture both linear as well as nonlinear relationships between any two parts of a dynamical complex system. Recently, an extension to classical information theory called partial information decomposition has been developed, which allows one to partition the information that two subsystems have about a third one into unique, redundant and synergistic contributions. Here, we apply a recent estimator of partial information decomposition to characterize the dynamics of two different complex systems. First, we analyze the distribution of information in triplets of spins in the 2D Ising model as a function of temperature. We find that while redundant information obtains a maximum at the critical point, synergistic information peaks in the disorder phase. Secondly, we characterize 1D elementary cellular automata rules based on the information distribution between neighboring cells. We describe several clusters of rules with similar partial information decomposition. These examples illustrate how the partial information decomposition provides a characterization of the emergent dynamics of complex systems in terms of the information distributed across their interacting units.<\/jats:p>","DOI":"10.3390\/e19120636","type":"journal-article","created":{"date-parts":[[2017,11,24]],"date-time":"2017-11-24T06:39:25Z","timestamp":1511505565000},"page":"636","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Analyzing Information Distribution in Complex Systems"],"prefix":"10.3390","volume":"19","author":[{"given":"Sten","family":"Sootla","sequence":"first","affiliation":[{"name":"Institute of Computer Science, University of Tartu, Ulikooli 17, 50090 Tartu, Estonia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dirk","family":"Theis","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, University of Tartu, Ulikooli 17, 50090 Tartu, Estonia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Raul","family":"Vicente","sequence":"additional","affiliation":[{"name":"Institute of Computer Science, University of Tartu, Ulikooli 17, 50090 Tartu, Estonia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2017,11,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1016\/j.ins.2012.04.016","article-title":"Local measures of information storage in complex distributed computation","volume":"208","author":"Lizier","year":"2012","journal-title":"Inf. Sci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"461","DOI":"10.1103\/PhysRevLett.85.461","article-title":"Measuring Information Transfer","volume":"85","author":"Schreiber","year":"2000","journal-title":"Phys. Rev. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1007\/s10827-010-0262-3","article-title":"Transfer entropy\u2014A model-free measure of effective connectivity for the neurosciences","volume":"30","author":"Vicente","year":"2011","journal-title":"J. Comput. Neurosci."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Wibral, M., Vicente, R., and Lindner, M. (2014). Transfer Entropy in Neuroscience, Springer.","DOI":"10.1007\/978-3-642-54474-3_1"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Wibral, M., Vicente, R., and Lizier, J.T. (2014). Directed Information Measures in Neuroscience, Springer.","DOI":"10.1007\/978-3-642-54474-3"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"037109","DOI":"10.1063\/1.3486801","article-title":"Information modification and particle collisions in distributed computation","volume":"20","author":"Lizier","year":"2010","journal-title":"Chaos"},{"key":"ref_7","unstructured":"Williams, P.L., and Beer, R.D. (2010). Nonnegative Decomposition of Multivariate Information. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"2161","DOI":"10.3390\/e16042161","article-title":"Quantifying Unique Information","volume":"16","author":"Bertschinger","year":"2014","journal-title":"Entropy"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"012130","DOI":"10.1103\/PhysRevE.87.012130","article-title":"Bivariate measure of redundant information","volume":"87","author":"Harder","year":"2013","journal-title":"Phys. Rev. E"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Prokopenko, M. (2014). Quantifying Synergistic Mutual Information. Guided Self-Organization: Inception, Springer.","DOI":"10.1007\/978-3-642-53734-9"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Ince, R.A. (2017). The Partial Entropy Decomposition: Decomposing Multivariate Entropy and Mutual Information via Pointwise Common Surprisal. arXiv.","DOI":"10.3390\/e19070318"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"5","DOI":"10.3389\/frobt.2015.00005","article-title":"Bits from brains for biologically inspired computing","volume":"2","author":"Wibral","year":"2015","journal-title":"Front. Robot. AI"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wibral, M., Priesemann, V., Kay, J.W., Lizier, J.T., and Phillips, W.A. (2015). Partial Information Decomposition as a Unified Approach to the Specification of Neural Goal Functions. arXiv.","DOI":"10.1186\/1471-2202-16-S1-P199"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Makkeh, A., Theis, D.O., and Vicente, R. (2017). Bivariate Partial Information Decomposition: The Optimization Perspective. Entropy, 19.","DOI":"10.3390\/e19100530"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Boyd, S., and Vandenberghe, L. (2004). Convex Optimization, Cambridge University Press.","DOI":"10.1017\/CBO9780511804441"},{"key":"ref_16","unstructured":"Andersen, M.S., Dahl, J., and Vandenberghe, L. (2017, November 02). CVXOPT: A Python Package for Convex Optimization. Available online: http:\/\/cvxopt.org\/."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1007\/s00407-004-0088-3","article-title":"History of the Lenz-Ising Model 1920-1950: From Ferromagnetic to Cooperative Phenomena","volume":"59","author":"Niss","year":"2005","journal-title":"Arch. Hist. Exact Sci."},{"key":"ref_18","unstructured":"Huang, K. (1987). Statistical Mechanics, John Wiley & Sons. [2nd ed.]."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/0196-8858(86)90028-X","article-title":"Random Sequence Generation by Cellular Automata","volume":"7","author":"Wolfram","year":"1986","journal-title":"Adv. Appl. Math."},{"key":"ref_20","first-page":"305","article-title":"A Model of City Traffic Based on Elementary Cellular Automata","volume":"19","author":"David","year":"2011","journal-title":"Complex Syst."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.25088\/ComplexSystems.15.1.1","article-title":"Universality in Elementary Cellular Automata","volume":"15","author":"Cook","year":"2004","journal-title":"Complex Syst."},{"key":"ref_22","unstructured":"Weisstein, E.W. (2017, May 04). Elementary Cellular Automaton. Available online: http:\/\/mathworld.wolfram.com\/ElementaryCellularAutomaton.html."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/0167-2789(84)90245-8","article-title":"Universality and Complexity in Cellular Automata","volume":"10D","author":"Wolfram","year":"1984","journal-title":"Phys. D Nonlinear Phenom."},{"key":"ref_24","unstructured":"Wolfram, S. (2002). A New Kind of Science, Wolfram Media Inc."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1063\/1.1703954","article-title":"Time-dependent statistics of the Ising model","volume":"4","author":"Glauber","year":"1963","journal-title":"J. Math. Phys."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"177203","DOI":"10.1103\/PhysRevLett.111.177203","article-title":"Information flow in a kinetic Ising model peaks in the disordered phase","volume":"111","author":"Barnett","year":"2013","journal-title":"Phys. Rev. Lett."},{"key":"ref_27","unstructured":"Barnett, L. (2017, April 06). A Commentary on Information Flow in a Kinetic Ising Model Peaks in the Disordered Phase. Available online: http:\/\/users.sussex.ac.uk\/~lionelb\/Ising_TE_commentary.html."},{"key":"ref_28","unstructured":"Lizier, J.T., Prokopenko, M., and Zomaya, A.Y. (2008, January 5\u20138). The information dynamics of phase transitions in random boolean networks. Proceedings of the Eleventh International Conference on the Simulation and Synthesis of Living Systems (ALife XI), Winchester, UK."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"051125","DOI":"10.1103\/PhysRevE.75.051125","article-title":"Mutual information as a tool for identifying phase transitions in dynamical complex systems with limited data","volume":"75","author":"Wicks","year":"2007","journal-title":"Phys. Rev. E"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"18009","DOI":"10.1209\/0295-5075\/87\/18009","article-title":"Phase-transition-like behaviour of information measures in financial markets","volume":"87","author":"Bossomaier","year":"2009","journal-title":"EPL"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"555","DOI":"10.1140\/epjb\/e2011-10905-8","article-title":"The aggregate complexity of decisions in the game of Go","volume":"80","author":"Bossomaier","year":"2011","journal-title":"Eur. Phys. J. B"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1186\/2194-3206-1-9","article-title":"Information and phase transitions in socio-economic systems","volume":"1","author":"Bossomaier","year":"2013","journal-title":"Complex Adapt. Syst. Model."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1007\/BF02330576","article-title":"Mutual information of Ising systems","volume":"35","author":"Matsuda","year":"1996","journal-title":"Int. J. Theor. Phys."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"026110","DOI":"10.1103\/PhysRevE.77.026110","article-title":"Local information transfer as a spatiotemporal filter for complex systems","volume":"77","author":"Lizier","year":"2008","journal-title":"Phys. Rev. E"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Chliamovitch, G., Chopard, B., and Dupuis, A. (2014, January 22\u201325). On the Dynamics of Multi-information in Cellular Automata. Proceedings of the Cellular Automata\u201411th International Conference on Cellular Automata for Research and Industry (ACRI) 2014, Krakow, Poland.","DOI":"10.1007\/978-3-319-11520-7_10"},{"key":"ref_36","unstructured":"Courbariaux, M., and Bengio, Y. (2016). BinaryNet: Training Deep Neural Networks with Weights and Activations Constrained to +1 or \u22121. arXiv."},{"key":"ref_37","unstructured":"Lecun, Y., Cortes, C., and Burges, C.J. (2017, May 04). The MNIST Database of Handwritten Digits. Available online: http:\/\/yann.lecun.com\/exdb\/mnist\/."},{"key":"ref_38","unstructured":"Sorngard, B. (2014). Information Theory for Analyzing Neural Networks. [Master\u2019s Thesis, Norwegian University of Science and Technology]."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"11508","DOI":"10.1073\/pnas.1514188112","article-title":"Thermodynamics and signatures of criticality in a network of neurons","volume":"112","author":"Mora","year":"2015","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Davidor, Y., Schwefel, H.P., and M\u00e4nner, R. (1994). A genetic Algorithm discovers particle-based computation in cellular automata. Parallel Problem Solving from Nature\u2014PPSN III: International Conference on Evolutionary Computation, Proceedings of the Third Conference on Parallel Problem Solving from Nature, Jerusalem, Israel, 9\u201314 October 1994, Springer.","DOI":"10.1007\/3-540-58484-6"},{"key":"ref_41","unstructured":"Shwartz-Ziv, R., and Tishby, N. (2017). Opening the Black Box of Deep Neural Networks via Information. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Tax, T., Mediano, P.A., and Shanahan, M. (2017). The Partial Information Decomposition of Generative Neural Network Models. Entropy, 19.","DOI":"10.3390\/e19090474"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/12\/636\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:51:11Z","timestamp":1760208671000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/12\/636"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,11,24]]},"references-count":42,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2017,12]]}},"alternative-id":["e19120636"],"URL":"https:\/\/doi.org\/10.3390\/e19120636","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,11,24]]}}}