{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:52:29Z","timestamp":1760147549969,"version":"build-2065373602"},"reference-count":66,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,2,14]],"date-time":"2023-02-14T00:00:00Z","timestamp":1676332800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The existence of the typical set is key for data compression strategies and for the emergence of robust statistical observables in macroscopic physical systems. Standard approaches derive its existence from a restricted set of dynamical constraints. However, given its central role underlying the emergence of stable, almost deterministic statistical patterns, a question arises whether typical sets exist in much more general scenarios. We demonstrate here that the typical set can be defined and characterized from general forms of entropy for a much wider class of stochastic processes than was previously thought. This includes processes showing arbitrary path dependence, long range correlations or dynamic sampling spaces, suggesting that typicality is a generic property of stochastic processes, regardless of their complexity. We argue that the potential emergence of robust properties in complex stochastic systems provided by the existence of typical sets has special relevance to biological systems.<\/jats:p>","DOI":"10.3390\/e25020350","type":"journal-article","created":{"date-parts":[[2023,2,15]],"date-time":"2023-02-15T02:31:23Z","timestamp":1676428283000},"page":"350","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["The Typical Set and Entropy in Stochastic Systems with Arbitrary Phase Space Growth"],"prefix":"10.3390","volume":"25","author":[{"given":"Rudolf","family":"Hanel","sequence":"first","affiliation":[{"name":"Complexity Science Hub Vienna, Josefst\u00e4dter Strasse 39, 1080 Vienna, Austria"},{"name":"Section for Science of Complex Systems, Medical University of Vienna, Spitalgasse 23, 1090 Vienna, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bernat","family":"Corominas-Murtra","sequence":"additional","affiliation":[{"name":"Institute of Biology, University of Graz, Holteigasse 6, 8010 Graz, Austria"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,2,14]]},"reference":[{"key":"ref_1","unstructured":"Morowitz, H.J. (1968). Energy Flow in Biology: Biological Organization as a Problem in Thermal Physics, Academic Press."},{"key":"ref_2","unstructured":"Bialek, W. (2012). Biophysics: Searching for Principles, Princeton University Press."},{"key":"ref_3","unstructured":"Sol\u00e9, R., and Goodwin, B. (2000). Signs of Life, Basic Books; Perseus Group."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"20180395","DOI":"10.1098\/rsif.2018.0395","article-title":"Zipf\u2019s Law, unbounded complexity and open-ended evolution","volume":"15","author":"Seoane","year":"2018","journal-title":"J. R. Soc. Interface"},{"key":"ref_5","unstructured":"Maynard Smith, J., and Szathm\u00e1ry, E. (1995). The Major Transitions in Evolution, Freeman."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Bonner, J.T. (1988). The Evolution of Complexity by Means of Natural Selection, Princeton University Press.","DOI":"10.1515\/9780691222110"},{"key":"ref_7","unstructured":"Wolpert, L., Jessell, T., Lawrence, P., Meyerowitz, E., Robertson, E., and Smith, J. (2007). Principles of Development, Oxford University Press. [3rd ed.]."},{"key":"ref_8","unstructured":"G\u00f6tschl, J. (1996). Evolution and Progress in Democracies. Theory and Decision Library, Springer."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"363","DOI":"10.1162\/106454600300103683","article-title":"Open Problems in Artificial Life","volume":"6","author":"Bedau","year":"2000","journal-title":"Artif. Life"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1023\/B:ORIG.0000016440.53346.dc","article-title":"A universal definition of life: Autonomy and open-ended evolution","volume":"34","author":"Moreno","year":"2004","journal-title":"Orig. Life Evol. Biosph."},{"key":"ref_11","first-page":"67","article-title":"Enabling conditions for \u201copen-ended\u201d evolution","volume":"23","author":"Moreno","year":"2008","journal-title":"Biol. Philos."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"624","DOI":"10.1098\/rsif.2011.0479","article-title":"Computability, G\u00f6del\u2019s incompleteness theorem, and an inherent limit on the predictability of evolution","volume":"9","author":"Day","year":"2012","journal-title":"J. R. Soc. Interface"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1162\/artl_a_00291","article-title":"An Overview of Open-Ended Evolution: Editorial Introduction to the Open-Ended Evolution II Special Issue","volume":"25","author":"Packard","year":"2019","journal-title":"Artif. Life"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1162\/artl_a_00276","article-title":"Evolved Open-Endedness, Not Open-Ended Evolution","volume":"25","author":"Pattee","year":"2019","journal-title":"Artif. Life"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"4219","DOI":"10.1242\/dev.003798","article-title":"Stochastic patterning in the mouse pre-implantation embryo","volume":"134","author":"Dietrich","year":"2007","journal-title":"Development"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"344","DOI":"10.1038\/nature18958","article-title":"Asymmetric division of contractile domains couples cell positioning and fate specification","volume":"536","author":"Maitre","year":"2016","journal-title":"Nature"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Giammona, J., and Camp\u00e0s, O. (2021). Physical constraints on early blastomere packings. PLoS Comput. Biol., 17.","DOI":"10.1371\/journal.pcbi.1007994"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"5890","DOI":"10.1038\/srep05890","article-title":"The dynamics of correlated novelties","volume":"4","author":"Tria","year":"2014","journal-title":"Sci. Rep."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Loreto, V., Servedio, V.D.P., Strogatz, S.H., and Tria, F. (2016). Dynamics on expanding spaces: Modeling the emergence of novelties. Creat. Universality Lang., 59\u201383.","DOI":"10.1007\/978-3-319-24403-7_5"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"248301","DOI":"10.1103\/PhysRevLett.125.248301","article-title":"Interacting Discovery Processes on Complex Networks","volume":"125","author":"Iacopini","year":"2020","journal-title":"Phys. Rev. Lett."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5348","DOI":"10.1073\/pnas.1420946112","article-title":"Understanding scaling through history-dependent processes with collapsing sample space","volume":"112","author":"Hanel","year":"2015","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"11223","DOI":"10.1038\/s41598-017-09836-4","article-title":"Sample space reducing cascading processes produce the full spectrum of scaling exponents","volume":"7","author":"Hanel","year":"2017","journal-title":"Sci. Rep."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"102502","DOI":"10.1103\/PhysRevLett.126.102502","article-title":"Power-Law Intensity Distribution of \u03b3-Decay Cascades: Nuclear Structure as a Scale-Free Random Network","volume":"126","author":"Fujii","year":"2021","journal-title":"Phys. Rev. Lett."},{"key":"ref_24","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, John Wiley & Sons."},{"key":"ref_25","unstructured":"Ash, R.B. (2012). Information Theory, Dover Publications."},{"key":"ref_26","unstructured":"Pathria, R.K. (2002). Statistical Mechanics, Oxford University Press."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Ben-Menahem, Y., and Hemmo, M. (2012). Probability in Physics, Springer.","DOI":"10.1007\/978-3-642-21329-8"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/0378-4371(93)90336-3","article-title":"Macroscopic Laws, Microscopic Dynamics, Time\u2019s Arrow and Boltzmann\u2019s Entropy","volume":"194","author":"Lebowitz","year":"1993","journal-title":"Physica A"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ledoux, M. (2005). The Concentration of Measure Phenomenon, American Mathematical Society.","DOI":"10.1090\/surv\/089"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Battermann, R. (2001). The Devil in the Details: Asymptotic Reasoning in Explanation, Reduction, and Emergence, Oxford University Press.","DOI":"10.1093\/0195146476.001.0001"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"997","DOI":"10.1086\/605800","article-title":"Typicality and the Approach to Equilibrium in Boltzmannian Statistical Mechanics","volume":"76","author":"Frigg","year":"2009","journal-title":"Philos. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"270","DOI":"10.1016\/0196-8858(89)90014-6","article-title":"A history-dependent random sequence defined by Ulam","volume":"10","year":"1989","journal-title":"Adv. Appl. Math."},{"key":"ref_33","unstructured":"Pitman, J. (2006). Combinatorial Stochastic Processes, Springer."},{"key":"ref_34","first-page":"1105","article-title":"History-dependent random processes","volume":"464","author":"Clifford","year":"2008","journal-title":"Proc. R. Soc. Lond. A"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1016\/j.physa.2018.02.078","article-title":"Unidirectional random growth with resetting","volume":"499","year":"2018","journal-title":"Phys. A Stat. Mech. Its Appl."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"375002","DOI":"10.1088\/1751-8121\/aad57b","article-title":"Statistical mechanics of exploding phase spaces: Ontic open systems","volume":"51","author":"Jensen","year":"2018","journal-title":"J. Phys. A Math. Theor."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1127","DOI":"10.1038\/s41467-021-21272-7","article-title":"Thermodynamics of structure-forming systems","volume":"12","author":"Korbel","year":"2021","journal-title":"Nat. Commun."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1109\/TIT.1974.1055248","article-title":"Source coding theorems without the ergodic assumption","volume":"20","author":"Gray","year":"1974","journal-title":"IEEE Trans. Inform. Theory"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1633","DOI":"10.1109\/18.850707","article-title":"Universal coding of nonstationary sources","volume":"46","author":"Visweswariah","year":"2000","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"688","DOI":"10.1162\/neco.2008.01-08-700","article-title":"Information in the Non-Stationary Case","volume":"21","author":"Vu","year":"2009","journal-title":"Neural Comput."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1109\/MSP.2013.2265914","article-title":"Time-frequence processing of nonstationary signals: Advanced TFD design to aid diagnosis with highlights from medical applications","volume":"30","author":"Boashash","year":"2013","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Granero-Belinch\u00f3n, C., Roux, S.G., and Garnier, N.B. (2019). Information Theory for Non-Stationary Processes with Stationary Increments. Entropy, 21.","DOI":"10.3390\/e21121223"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/S0375-9601(00)00337-6","article-title":"Axioms and uniqueness theorem for Tsallis entropy","volume":"271","author":"Abe","year":"2000","journal-title":"Phys. Lett. A"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"20006","DOI":"10.1209\/0295-5075\/93\/20006","article-title":"A comprehensive classification of complex statistical systems and an axiomatic derivation of their entropy and distribution functions","volume":"93","author":"Hanel","year":"2011","journal-title":"Europhys. Lett."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"123101","DOI":"10.1088\/1742-5468\/aa967f","article-title":"Uniqueness and characterization theorems for generalized entropies","volume":"12","author":"Enciso","year":"2017","journal-title":"J. Stat. Mech. Theory Exp."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"180","DOI":"10.1016\/j.aop.2015.08.013","article-title":"Beyond the Shannon\u2013Khinchin formulation: The composability axiom and the universal-group entropy","volume":"365","author":"Tempesta","year":"2016","journal-title":"Ann. Phys."},{"key":"ref_47","first-page":"20160143","article-title":"Formal groups and Z-entropies","volume":"472","author":"Tempesta","year":"2016","journal-title":"Proc. R. Soc. Lond. A"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"032124","DOI":"10.1103\/PhysRevE.96.032124","article-title":"The three faces of entropy for complex systems\u2014Information, thermodynamics and the maxent principle","volume":"96","author":"Thurner","year":"2017","journal-title":"Phys. Rev. E"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"042126","DOI":"10.1103\/PhysRevE.101.042126","article-title":"When Shannon and Khinchin meet Shore and Johnson: Equivalence of information theory and statistical inference axiomatics","volume":"101","author":"Jizba","year":"2020","journal-title":"Phys. Rev. E"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"120601","DOI":"10.1103\/PhysRevLett.122.120601","article-title":"Maximum Entropy Principle in Statistical Inference: Case for Non-Shannonian Entropies","volume":"122","author":"Korbel","year":"2019","journal-title":"Phys. Rev. Lett."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Jizba, P., and Korbel, J. (2017). On the Uniqueness Theorem for Pseudo-Additive Entropies. Entropy, 19.","DOI":"10.3390\/e19110605"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"50003","DOI":"10.1209\/0295-5075\/96\/50003","article-title":"When do generalized entropies apply? How phase space volume determines entropy","volume":"96","author":"Hanel","year":"2011","journal-title":"Europhys. Lett."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Jensen, H.J., and Tempesta, P. (2018). Group entropies: From phase space geometry to entropy functionals via group theory. Entropy, 20.","DOI":"10.3390\/e20100804"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"093007","DOI":"10.1088\/1367-2630\/aadcbe","article-title":"Classification of complex systems by their sample-space scaling exponents","volume":"20","author":"Korbel","year":"2018","journal-title":"New J. Phys."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1140\/epjst\/e2020-900190-x","article-title":"Information geometry of scaling expansions of non-exponentially growing configuration spaces","volume":"229","author":"Korbel","year":"2020","journal-title":"Eur. Phys. J. Spec. Top."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"5324","DOI":"10.3390\/e15125324","article-title":"Generalized (c,d)-entropy and aging random walks","volume":"15","author":"Hanel","year":"2013","journal-title":"Entropy"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"084112","DOI":"10.1063\/1.4961485","article-title":"Learning the mechanisms of chemical disequilibria","volume":"145","author":"Nicholson","year":"2016","journal-title":"J. Chem. Phys."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"15516","DOI":"10.1038\/s41598-020-72422-8","article-title":"Generalized entropies, density of states, and non-extensivity","volume":"10","author":"Balogh","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Gardiner, C.W. (1983). Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences, Springer.","DOI":"10.1007\/978-3-662-02377-8"},{"key":"ref_60","unstructured":"Feller, W. (1991). An Introduction to Probability Theory and Its Applications, Wiley."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech."},{"key":"ref_62","unstructured":"Khinchin, A. (1957). Mathematical Foundations of Information Theory, Dover."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"066118","DOI":"10.1103\/PhysRevE.80.066118","article-title":"Statistical mechanics of the \u201cChinese restaurant process\u201d: Lack of self-averaging, anomalous finite-size effects, and condensation","volume":"80","author":"Bassetti","year":"2009","journal-title":"Phys. Rev. E"},{"key":"ref_64","unstructured":"Abramowitz, M., and Stegun, I. (1964). Handbook of Mathematical Functions. National Bureau of Standards, U.S. Government Printing Office."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"126","DOI":"10.1016\/S0375-9601(01)00127-X","article-title":"Nonextensive thermodynamic relations","volume":"281","author":"Abe","year":"2001","journal-title":"Phys. Lett. A"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"430","DOI":"10.1016\/j.physa.2006.04.001","article-title":"Temperature of nonextensive systems: Tsallis entropy as Clausius entropy","volume":"368","author":"Abe","year":"2006","journal-title":"Phys. A Stat. Mech. Its Appl."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/2\/350\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:34:58Z","timestamp":1760121298000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/2\/350"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,14]]},"references-count":66,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,2]]}},"alternative-id":["e25020350"],"URL":"https:\/\/doi.org\/10.3390\/e25020350","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2023,2,14]]}}}