{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,20]],"date-time":"2026-10-20T16:49:53Z","timestamp":1792514993869,"version":"4.5.0"},"reference-count":47,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2022,12,23]],"date-time":"2022-12-23T00:00:00Z","timestamp":1671753600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China (NSFC)","doi-asserted-by":"crossref","award":["61673070"],"award-info":[{"award-number":["61673070"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["www.mdpi.com"],"crossmark-restriction":true},"short-container-title":["Entropy"],"abstract":"<jats:p>Conventional studies of causal emergence have revealed that stronger causality can be obtained on the macro-level than the micro-level of the same Markovian dynamical systems if an appropriate coarse-graining strategy has been conducted on the micro-states. However, identifying this emergent causality from data is still a difficult problem that has not been solved because the appropriate coarse-graining strategy can not be found easily. This paper proposes a general machine learning framework called Neural Information Squeezer to automatically extract the effective coarse-graining strategy and the macro-level dynamics, as well as identify causal emergence directly from time series data. By using invertible neural network, we can decompose any coarse-graining strategy into two separate procedures: information conversion and information discarding. In this way, we can not only exactly control the width of the information channel, but also can derive some important properties analytically. We also show how our framework can extract the coarse-graining functions and the dynamics on different levels, as well as identify causal emergence from the data on several exampled systems.<\/jats:p>","DOI":"10.3390\/e25010026","type":"journal-article","created":{"date-parts":[[2022,12,27]],"date-time":"2022-12-27T04:35:43Z","timestamp":1672115743000},"page":"26","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":20,"title":["Neural Information Squeezer for Causal Emergence"],"prefix":"10.3390","volume":"25","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7402-7482","authenticated-orcid":false,"given":"Jiang","family":"Zhang","sequence":"first","affiliation":[{"name":"School of Systems Sciences, Beijing Normal University, Beijing 100875, China"},{"name":"Swarma Research, Beijing 100085, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0343-3488","authenticated-orcid":false,"given":"Kaiwei","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Systems Sciences, Beijing Normal University, Beijing 100875, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,12,23]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Holland, J.H. (1999). Emergence: From Chaos to Order, Basic Books. Illustrated edition.","DOI":"10.1093\/oso\/9780198504092.001.0001"},{"key":"ref_2","first-page":"375","article-title":"Weak Emergence","volume":"11","author":"Bedau","year":"1997","journal-title":"Philos. Perspect."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Pearl, J. (2009). Causality: Models of Reasoning and Inference, Cambridge University Press. [2nd ed.].","DOI":"10.1017\/CBO9780511803161"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"424","DOI":"10.2307\/1912791","article-title":"Investigating Causal Relations by Econometric Models and Cross-spectral Methods","volume":"37","author":"Granger","year":"1969","journal-title":"Econometrica"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"19790","DOI":"10.1073\/pnas.1314922110","article-title":"Quantifying causal emergence shows that macro can beat micro","volume":"110","author":"Hoel","year":"2013","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Hoel, E.P. (2017). When the Map Is Better Than the Territory. Entropy, 19.","DOI":"10.3390\/e19050188"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Tononi, G., and Sporns, O. (2003). Measuring information integration. BMC Neurosci., 4.","DOI":"10.1186\/1471-2202-4-31"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Varley, T., and Hoel, E. (2021). Emergence as the conversion of information: A unifying theory. arXiv.","DOI":"10.1098\/rsta.2021.0150"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Chvykov, P., and Hoel, E. (2021). Causal Geometry. Entropy, 23.","DOI":"10.3390\/e23010024"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Rosas, F.E., Mediano, P.A.M., Jensen, H.J., Seth, A.K., Barrett, A.B., Carhart-Harris, R.L., and Bor, D. (2020). Reconciling emergences: An information-theoretic approach to identify causal emergence in multivariate data. PLoS Comput. Biol., 16.","DOI":"10.1371\/journal.pcbi.1008289"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Varley, T.F. (2022). Flickering emergences: The question of locality in information-theoretic approaches to emergence. arXiv.","DOI":"10.3390\/e25010054"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.anbehav.2022.04.015","article-title":"Interactions and information: Exploring task allocation in ant colonies using network analysis","volume":"189","author":"Swain","year":"2022","journal-title":"Anim. Behav."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1093\/intbio\/zyab020","article-title":"Evolution and emergence: Higher order information structure in protein interactomes across the tree of life","volume":"13","author":"Klein","year":"2021","journal-title":"Integr. Biol."},{"key":"ref_14","first-page":"25","article-title":"Causal emergence of task information from dynamic network interactions in the human brain","volume":"31","author":"Ravi","year":"2022","journal-title":"Rev. Neurosci."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"799","DOI":"10.1111\/2041-210X.13805","article-title":"Exploring noise, degeneracy and determinism in biological networks with the einet package","volume":"13","author":"Klein","year":"2022","journal-title":"Methods Ecol. Evol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"8932526","DOI":"10.1155\/2020\/8932526","article-title":"The Emergence of Informative Higher Scales in Complex Networks","volume":"2020","author":"Klein","year":"2020","journal-title":"Complexity"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"20210150","DOI":"10.1098\/rsta.2021.0150","article-title":"Emergence as the conversion of information: A unifying theory","volume":"380","author":"Varley","year":"2022","journal-title":"Philos. Trans. R. Soc. A"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"354","DOI":"10.1038\/nature24270","article-title":"Mastering the game of Go without human knowledge","volume":"550","author":"Silver","year":"2017","journal-title":"Nature"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1038\/s41586-019-0912-1","article-title":"Deep learning and process understanding for data-driven Earth system science","volume":"566","author":"Reichstein","year":"2019","journal-title":"Nature"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"706","DOI":"10.1038\/s41586-019-1923-7","article-title":"Improved protein structure prediction using potentials from deep learning","volume":"577","author":"Senior","year":"2020","journal-title":"Nature"},{"key":"ref_22","unstructured":"Tank, A., Covert, I., Foti, N., Shojaie, A., and Fox, E. (2018). Neural Granger Causality. arXiv."},{"key":"ref_23","unstructured":"L\u00f6we, S., Madras, D., Zemel, R., and Welling, M. (2020). Amortized causal discovery: Learning to infer causal graphs from time-series data. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"524","DOI":"10.3389\/fgene.2019.00524","article-title":"Review of Causal Discovery Methods Based on Graphical Models","volume":"10","author":"Glymour","year":"2019","journal-title":"Front. Genet."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2192","DOI":"10.1038\/s41467-017-02288-4","article-title":"Model-free inference of direct network interactions from nonlinear collective dynamics","volume":"8","author":"Casadiego","year":"2017","journal-title":"Nat. Commun."},{"key":"ref_26","unstructured":"Sanchez-Gonzalez, A., Heess, N., Springenberg, J.T., Merel, J., Riedmiller, M., Hadsell, R., and Battaglia, P. (2018, January 10\u201315). Graph networks as learnable physics engines for inference and control. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1007\/s41109-019-0194-4","article-title":"A general deep learning framework for network reconstruction and dynamics learning","volume":"4","author":"Zhang","year":"2019","journal-title":"Appl. Netw. Sci."},{"key":"ref_28","unstructured":"Kipf, T., Fetaya, E., Wang, K.C., Welling, M., and Zemel, R. (2018, January 10\u201315). Neural relational inference for interacting systems. Proceedings of the International Conference on Machine Learning, Stockholm, Sweden."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Chen, B., Huang, K., Raghupathi, S., Chandratreya, I., Du, Q., and Lipson, H. (2021). Discovering State Variables Hidden in Experimental Data. arXiv.","DOI":"10.1038\/s43588-022-00281-6"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"578","DOI":"10.1038\/s41567-018-0081-4","article-title":"Mutual information, neural networks and the renormalization group","volume":"14","author":"Ringel","year":"2018","journal-title":"Nat. Phys."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"260601","DOI":"10.1103\/PhysRevLett.121.260601","article-title":"Neural Network Renormalization Group","volume":"121","author":"Li","year":"2018","journal-title":"Phys. Rev. Lett."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"023369","DOI":"10.1103\/PhysRevResearch.2.023369","article-title":"Machine learning holographic mapping by neural network renormalization group","volume":"2","author":"Hu","year":"2020","journal-title":"Phys. Rev. Res."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"035009","DOI":"10.1088\/2632-2153\/ac8393","article-title":"RG-Flow: A hierarchical and explainable flow model based on renormalization group and sparse prior","volume":"3","author":"Hu","year":"2022","journal-title":"Mach. Learn. Sci. Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"240603","DOI":"10.1103\/PhysRevLett.127.240603","article-title":"Statistical physics through the lens of real-space mutual information","volume":"127","author":"Ringel","year":"2021","journal-title":"Phys. Rev. Lett."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1007\/s41237-016-0008-2","article-title":"Causal feature learning: An overview","volume":"44","author":"Chalupka","year":"2017","journal-title":"Behaviormetrika"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"612","DOI":"10.1109\/JPROC.2021.3058954","article-title":"Toward causal representation learning","volume":"109","author":"Locatello","year":"2021","journal-title":"Proc. IEEE"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1016\/0004-3702(94)90014-0","article-title":"Causality and model abstraction","volume":"67","author":"Iwasaki","year":"1994","journal-title":"Artif. Intell."},{"key":"ref_38","unstructured":"Rubenstein, P.K., Weichwald, S., Bongers, S., Mooij, J., Janzing, D., Grosse-Wentrup, M., and Sch\u00f6lkopf, B. (2017). Causal consistency of structural equation models. arXiv."},{"key":"ref_39","unstructured":"Beckers, S., Eberhardt, F., and Halpern, J.Y. (2020, January 3\u20136). Approximate causal abstractions. Proceedings of the Uncertainty in Artificial Intelligence, Virtual."},{"key":"ref_40","unstructured":"Beckers, S., Eberhardt, F., and Halpern, J.Y. (2019). Approximate Causal Abstraction. arXiv."},{"key":"ref_41","first-page":"3362","article-title":"Coupling-based invertible neural networks are universal diffeomorphism approximators","volume":"33","author":"Teshima","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_42","unstructured":"Teshima, T., Tojo, K., Ikeda, M., Ishikawa, I., and Oono, K. (2017). Universal approximation property of neural ordinary differential equations. arXiv."},{"key":"ref_43","unstructured":"Dinh, L., Sohl-Dickstein, J., and Bengio, S. (2016). Density estimation using real nvp. arXiv."},{"key":"ref_44","unstructured":"Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv."},{"key":"ref_45","unstructured":"Shwartz-Ziv, R., and Tishby, N. (2017). Opening the black box of deep neural networks via information. arXiv."},{"key":"ref_46","unstructured":"Williams, P.L., and Beer, R.D. (2017). Nonnegative decomposition of multivariate information. arXiv."},{"key":"ref_47","unstructured":"Geiger, B.C., and Kubin, G. (2011). On the information loss in memoryless systems: The multivariate case. arXiv."}],"updated-by":[{"DOI":"10.3390\/e25101387","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2022,12,23]],"date-time":"2022-12-23T00:00:00Z","timestamp":1671753600000}}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/1\/26\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,3]],"date-time":"2025-08-03T22:14:02Z","timestamp":1754259242000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/1\/26"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,12,23]]},"references-count":47,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["e25010026"],"URL":"https:\/\/doi.org\/10.3390\/e25010026","relation":{"correction":[{"id-type":"doi","id":"10.3390\/e25101387","asserted-by":"object"}]},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,12,23]]}}}