{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,26]],"date-time":"2026-04-26T11:49:27Z","timestamp":1777204167576,"version":"3.51.4"},"reference-count":125,"publisher":"Cambridge University Press (CUP)","issue":"3","license":[{"start":{"date-parts":[[2014,3,28]],"date-time":"2014-03-28T00:00:00Z","timestamp":1395964800000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Math. Struct. Comp. Sci."],"published-print":{"date-parts":[[2014,6]]},"abstract":"<jats:p>Statistical entropy was introduced by Shannon as a basic concept in information theory measuring the average missing information in a random source. Extended into an entropy rate, it gives bounds in coding and compression theorems. In this paper, I describe how statistical entropy and entropy rate relate to other notions of entropy that are relevant to probability theory (entropy of a discrete probability distribution measuring its unevenness), computer sciences (algorithmic complexity), the ergodic theory of dynamical systems (Kolmogorov\u2013Sinai or metric entropy) and statistical physics (Boltzmann entropy). Their mathematical foundations and correlates (the entropy concentration, Sanov, Shannon\u2013McMillan\u2013Breiman, Lempel\u2013Ziv and Pesin theorems) clarify their interpretation and offer a rigorous basis for maximum entropy principles. Although often ignored, these mathematical perspectives give a central position to entropy and relative entropy in statistical laws describing generic collective behaviours, and provide insights into the notions of randomness, typicality and disorder. The relevance of entropy beyond the realm of physics, in particular for living systems and ecosystems, is yet to be demonstrated.<\/jats:p>","DOI":"10.1017\/s0960129512000783","type":"journal-article","created":{"date-parts":[[2014,4,1]],"date-time":"2014-04-01T09:46:24Z","timestamp":1396345584000},"source":"Crossref","is-referenced-by-count":170,"title":["Shannon entropy: a rigorous notion at the crossroads between probability, information theory, dynamical systems and statistical physics"],"prefix":"10.1017","volume":"24","author":[{"given":"ANNICK","family":"LESNE","sequence":"first","affiliation":[]}],"member":"56","published-online":{"date-parts":[[2014,3,28]]},"reference":[{"key":"S0960129512000783_ref71","first-page":"1","article-title":"Three approaches to the quantitative definition of information.","volume":"1","author":"Kolmogorov","year":"1965","journal-title":"Problems of Information Transmission"},{"key":"S0960129512000783_ref42","doi-asserted-by":"publisher","DOI":"10.1016\/S0375-9601(97)00855-4"},{"key":"S0960129512000783_ref96","doi-asserted-by":"publisher","DOI":"10.1111\/j.0906-7590.2008.5203.x"},{"key":"S0960129512000783_ref124","doi-asserted-by":"publisher","DOI":"10.1007\/s10955-005-7576-y"},{"key":"S0960129512000783_ref48","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1099-0526(199609\/10)2:1<44::AID-CPLX10>3.0.CO;2-X"},{"key":"S0960129512000783_ref49","first-page":"387","volume-title":"Nonextensive Entropy \u2013 Interdisciplinary Applications","author":"Gell-Mann","year":"2003"},{"key":"S0960129512000783_ref114","doi-asserted-by":"publisher","DOI":"10.1007\/BF01341281"},{"key":"S0960129512000783_ref89","doi-asserted-by":"publisher","DOI":"10.1016\/S0019-9958(66)80018-9"},{"key":"S0960129512000783_ref92","doi-asserted-by":"publisher","DOI":"10.1016\/0960-0779(94)90016-7"},{"key":"S0960129512000783_ref122","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1977.1055714"},{"key":"S0960129512000783_ref46","doi-asserted-by":"publisher","DOI":"10.1063\/1.2372713"},{"key":"S0960129512000783_ref73","first-page":"327","volume-title":"Proceedings Sixth Berkeley Symposium","author":"Krieger","year":"1972"},{"key":"S0960129512000783_ref1","doi-asserted-by":"publisher","DOI":"10.1214\/aop\/1176991794"},{"key":"S0960129512000783_ref58","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-03709-6"},{"key":"S0960129512000783_ref82","doi-asserted-by":"publisher","DOI":"10.1017\/S0960129507005944"},{"key":"S0960129512000783_ref93","volume-title":"Self-organization in nonequilibrium systems","author":"Nicolis","year":"1977"},{"key":"S0960129512000783_ref18","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0061408"},{"key":"S0960129512000783_ref117","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1981.1056374"},{"key":"S0960129512000783_ref3","doi-asserted-by":"publisher","DOI":"10.1142\/5299"},{"key":"S0960129512000783_ref55","doi-asserted-by":"crossref","unstructured":"Gray R. M. (1990) Entropy and information theory, Springer. (Available at http:\/\/ee.stanford.edu\/~gray\/it.html.)","DOI":"10.1007\/978-1-4757-3982-4_2"},{"key":"S0960129512000783_ref64","doi-asserted-by":"publisher","DOI":"10.1146\/annurev.pc.31.100180.003051"},{"key":"S0960129512000783_ref40","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-39668-0_4"},{"key":"S0960129512000783_ref41","unstructured":"Feldman D. P. (2002) A brief introduction to information theory, excess entropy and computational mechanics. (Available online at http:\/\/hornacek.coa.edu\/dave\/.)"},{"key":"S0960129512000783_ref103","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.65.046124"},{"key":"S0960129512000783_ref95","doi-asserted-by":"publisher","DOI":"10.7208\/chicago\/9780226662237.001.0001"},{"key":"S0960129512000783_ref88","doi-asserted-by":"publisher","DOI":"10.1016\/j.physd.2010.02.001"},{"key":"S0960129512000783_ref26","volume-title":"Introduction to modern statistical mechanics","author":"Chandler","year":"1987"},{"key":"S0960129512000783_ref2","volume-title":"Methods of information geometry","author":"Amari","year":"2000"},{"key":"S0960129512000783_ref36","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-36351-4_14"},{"key":"S0960129512000783_ref112","doi-asserted-by":"publisher","DOI":"10.1007\/BF01019776"},{"key":"S0960129512000783_ref21","volume-title":"Thermodynamics and thermostatics","author":"Callen","year":"1985"},{"key":"S0960129512000783_ref83","doi-asserted-by":"publisher","DOI":"10.1186\/1742-4682-5-21"},{"key":"S0960129512000783_ref119","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1103\/RevModPhys.50.221","article-title":"General properties of entropy.","volume":"50","author":"Werhl","year":"1978","journal-title":"Reviews of Modern Physics"},{"key":"S0960129512000783_ref53","doi-asserted-by":"publisher","DOI":"10.1016\/j.physa.2006.07.034"},{"key":"S0960129512000783_ref30","doi-asserted-by":"publisher","DOI":"10.1119\/1.1990764"},{"key":"S0960129512000783_ref106","volume-title":"What is life? The physical aspect of the living cell","author":"Schr\u00f6dinger","year":"1944"},{"key":"S0960129512000783_ref33","volume-title":"Information theory, coding theorems for discrete memoryless systems","author":"Csisz\u00e1r","year":"1981"},{"key":"S0960129512000783_ref111","volume-title":"Functional integration: basics and applications (1996 Carg\u00e8se summer school)","author":"Sokal","year":"1997"},{"key":"S0960129512000783_ref17","doi-asserted-by":"crossref","DOI":"10.1063\/1.3059856","volume-title":"Science and Information Theory","author":"Brillouin","year":"1956"},{"key":"S0960129512000783_ref16","doi-asserted-by":"publisher","DOI":"10.1063\/1.1721463"},{"key":"S0960129512000783_ref28","doi-asserted-by":"publisher","DOI":"10.1023\/A:1004604804070"},{"key":"S0960129512000783_ref79","doi-asserted-by":"publisher","DOI":"10.1017\/S0143385700001528"},{"key":"S0960129512000783_ref76","doi-asserted-by":"publisher","DOI":"10.1147\/rd.53.0183"},{"key":"S0960129512000783_ref115","doi-asserted-by":"publisher","DOI":"10.1016\/j.physrep.2009.05.002"},{"key":"S0960129512000783_ref94","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-39668-0_5"},{"key":"S0960129512000783_ref110","first-page":"768","article-title":"On the concept of entropy for dynamical systems (in Russian).","volume":"124","author":"Sinai","year":"1959","journal-title":"Doklady Akademii Nauk SSSR"},{"key":"S0960129512000783_ref50","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1515\/9781400865222.37","volume-title":"Entropy","author":"Georgii","year":"2003"},{"key":"S0960129512000783_ref54","doi-asserted-by":"publisher","DOI":"10.1007\/BF00668821"},{"key":"S0960129512000783_ref31","doi-asserted-by":"publisher","DOI":"10.1214\/aop\/1176996454"},{"key":"S0960129512000783_ref75","volume-title":"Invariances d'\u00e9chelle","author":"Lagu\u00ebs","year":"2008"},{"key":"S0960129512000783_ref52","doi-asserted-by":"publisher","DOI":"10.1090\/surv\/101"},{"key":"S0960129512000783_ref11","first-page":"373","article-title":"\u00dcber die Beziehung zwisschen dem zweiten Haubtsatze der mechanischen W\u00e4rmetheorie und der Wahrscheinlichkeitsrechnung respektive dem S\u00e4tzen \u00fcber das W\u00e4rmegleichgewicht. (\u2018On the Relation between the Second Law of the Mechanical Theory of Heat and the Probability Calculus with respect to the Propositions about Heat-Equivalence\u2019.)","volume":"76","author":"Boltzmann","year":"1877","journal-title":"Wiener Berichte"},{"key":"S0960129512000783_ref4","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511524691"},{"key":"S0960129512000783_ref5","article-title":"Limit theorems for sequences of random trees","author":"Balding","year":"2008","journal-title":"TEST"},{"key":"S0960129512000783_ref6","first-page":"119","volume-title":"Entropy","author":"Balian","year":"2004"},{"key":"S0960129512000783_ref7","doi-asserted-by":"publisher","DOI":"10.1016\/j.shpsb.2005.02.001"},{"key":"S0960129512000783_ref8","first-page":"063101","article-title":"Applications of the principle of maximum entropy: from physics to ecology","volume":"22","author":"Banavar","year":"2010","journal-title":"Journal of Physics: Condensed Matter"},{"key":"S0960129512000783_ref47","doi-asserted-by":"publisher","DOI":"10.1007\/s10955-004-3455-1"},{"key":"S0960129512000783_ref9","doi-asserted-by":"crossref","unstructured":"Blanc J. L. , Pezard L. and Lesne A. (2011) Mutual information rate of pair of symbolic sequences.","DOI":"10.1103\/PhysRevE.84.036214"},{"key":"S0960129512000783_ref116","doi-asserted-by":"publisher","DOI":"10.1038\/scientificamerican0971-179"},{"key":"S0960129512000783_ref10","first-page":"40","volume-title":"Proceedings of the Second french conference on Computational Neuroscience (Neurocomp'08)","author":"Blanc","year":"2008"},{"key":"S0960129512000783_ref12","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177706899"},{"key":"S0960129512000783_ref13","first-page":"159","article-title":"Science of chaos or chaos in science.","volume":"17","author":"Bricmont","year":"1995","journal-title":"Physicalia Magazine"},{"key":"S0960129512000783_ref14","doi-asserted-by":"publisher","DOI":"10.1063\/1.1699951"},{"key":"S0960129512000783_ref39","doi-asserted-by":"publisher","DOI":"10.1080\/00018730210155133"},{"key":"S0960129512000783_ref15","doi-asserted-by":"publisher","DOI":"10.1063\/1.1699952"},{"key":"S0960129512000783_ref19","first-page":"127","article-title":"Entropy and the complexity of the trajectory of a dynamical system.","volume":"44","author":"Brudno","year":"1983","journal-title":"Transactions of the Moscow Mathematical Society"},{"key":"S0960129512000783_ref57","doi-asserted-by":"publisher","DOI":"10.1086\/650718"},{"key":"S0960129512000783_ref20","volume-title":"What to my wondering eyes","author":"Buten","year":"1989"},{"key":"S0960129512000783_ref22","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511535291"},{"key":"S0960129512000783_ref23","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-1039-9"},{"key":"S0960129512000783_ref24","volume-title":"Ludwig Boltzmann \u2013 The man who trusted atoms","author":"Cercignani","year":"1998"},{"key":"S0960129512000783_ref25","doi-asserted-by":"publisher","DOI":"10.1145\/321356.321363"},{"key":"S0960129512000783_ref32","doi-asserted-by":"publisher","DOI":"10.1109\/18.720546"},{"key":"S0960129512000783_ref125","first-page":"151","volume-title":"Frontiers of nonequilibrium statistical physics","author":"Zurek","year":"1984"},{"key":"S0960129512000783_ref35","first-page":"86","volume-title":"Proceedings of the seventh International Conference on Cognitive Modeling","author":"Dessalles","year":"2006"},{"key":"S0960129512000783_ref29","volume-title":"Elements of information theory","author":"Cover","year":"2006"},{"key":"S0960129512000783_ref37","doi-asserted-by":"publisher","DOI":"10.1002\/andp.19103381612"},{"key":"S0960129512000783_ref38","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4613-8533-2"},{"key":"S0960129512000783_ref43","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-36351-4_5"},{"key":"S0960129512000783_ref44","doi-asserted-by":"publisher","DOI":"10.1111\/j.1420-9101.2009.01775.x"},{"key":"S0960129512000783_ref45","doi-asserted-by":"publisher","DOI":"10.1063\/1.166320"},{"key":"S0960129512000783_ref56","doi-asserted-by":"publisher","DOI":"10.1007\/s10955-004-2271-y"},{"key":"S0960129512000783_ref85","volume-title":"A generative theory of shape","author":"Leyton","year":"2001"},{"key":"S0960129512000783_ref59","doi-asserted-by":"publisher","DOI":"10.1142\/1676"},{"key":"S0960129512000783_ref60","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRev.106.620"},{"key":"S0960129512000783_ref61","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRev.108.171"},{"key":"S0960129512000783_ref62","doi-asserted-by":"publisher","DOI":"10.1007\/BF00709116"},{"key":"S0960129512000783_ref63","first-page":"15","volume-title":"The Maximum Entropy Formalism","author":"Jaynes","year":"1979"},{"key":"S0960129512000783_ref51","volume-title":"Philosophical theories of probability","author":"Gillies","year":"2000"},{"key":"S0960129512000783_ref65","doi-asserted-by":"publisher","DOI":"10.1109\/PROC.1982.12425"},{"key":"S0960129512000783_ref66","volume-title":"Papers on probability, statistics and statistical physics","author":"Jaynes","year":"1982"},{"key":"S0960129512000783_ref67","volume-title":"Characterization problems in mathematical statistics","author":"Kagan","year":"1973"},{"key":"S0960129512000783_ref72","doi-asserted-by":"publisher","DOI":"10.1090\/S0002-9947-1970-0259068-3"},{"key":"S0960129512000783_ref74","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729694"},{"key":"S0960129512000783_ref78","doi-asserted-by":"publisher","DOI":"10.1016\/0378-4371(93)90336-3"},{"key":"S0960129512000783_ref80","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1976.1055501"},{"key":"S0960129512000783_ref101","volume-title":"Thermodynamic formalism","author":"Ruelle","year":"1978"},{"key":"S0960129512000783_ref27","volume-title":"The mechanical theory of heat \u2013 with its applications to the steam engine and to physical properties of bodies","author":"Clausius","year":"1865"},{"key":"S0960129512000783_ref81","volume-title":"Renormalization methods","author":"Lesne","year":"1998"},{"key":"S0960129512000783_ref84","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.79.046208"},{"key":"S0960129512000783_ref86","volume-title":"Processus stochastiques et mouvement brownien","author":"L\u00e9vy","year":"1965"},{"key":"S0960129512000783_ref123","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1978.1055934"},{"key":"S0960129512000783_ref87","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-2606-0"},{"key":"S0960129512000783_ref97","doi-asserted-by":"publisher","DOI":"10.1016\/j.ecolmodel.2005.03.026"},{"key":"S0960129512000783_ref98","volume-title":"Thermodynamics of irreversible processes","author":"Prigogine","year":"1967"},{"key":"S0960129512000783_ref99","doi-asserted-by":"publisher","DOI":"10.1109\/18.923736"},{"key":"S0960129512000783_ref102","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.0630567100"},{"key":"S0960129512000783_ref104","first-page":"11","article-title":"On the probability of large deviations of random variables (in Russian)","volume":"42","author":"Sanov","year":"1957","journal-title":"Matematicheskii Sbornik"},{"key":"S0960129512000783_ref105","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevLett.102.250602"},{"key":"S0960129512000783_ref107","doi-asserted-by":"publisher","DOI":"10.1016\/j.physe.2009.06.045"},{"key":"S0960129512000783_ref108","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"S0960129512000783_ref91","first-page":"33","article-title":"Le concept de fonctionnelle d'opacit\u00e9 d'une statistique. \u00c9tude des relations entre la loi des grands nombres, l'entropie informationnelle et l'entropie statistique.","volume":"32","author":"Mugur-Sch\u00e4chter","year":"1980","journal-title":"Annales de l'IHP, section A"},{"key":"S0960129512000783_ref109","doi-asserted-by":"publisher","DOI":"10.1103\/PhysRevE.59.1459"},{"key":"S0960129512000783_ref113","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1978.1055913"},{"key":"S0960129512000783_ref118","doi-asserted-by":"publisher","DOI":"10.1023\/A:1023674330295"},{"key":"S0960129512000783_ref120","doi-asserted-by":"publisher","DOI":"10.1017\/S0143385700007653"},{"key":"S0960129512000783_ref121","doi-asserted-by":"publisher","DOI":"10.1109\/18.45281"},{"key":"S0960129512000783_ref70","unstructured":"Kay J. J. (1984) Self-organization in living systems, Ph.D. thesis, Systems Design Engineering, University of Waterloo, Ontario."},{"key":"S0960129512000783_ref34","volume-title":"Theory of probability \u2013 a critical introduction treatment","author":"de Finetti","year":"1970"},{"key":"S0960129512000783_ref90","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729028"},{"key":"S0960129512000783_ref100","doi-asserted-by":"publisher","DOI":"10.2307\/3214649"},{"key":"S0960129512000783_ref77","doi-asserted-by":"publisher","DOI":"10.1063\/1.881363"},{"key":"S0960129512000783_ref68","volume-title":"Nonlinear time series analysis","author":"Kantz","year":"1997"},{"key":"S0960129512000783_ref69","volume-title":"A first course in stochastic processes","author":"Karlin","year":"1975"}],"container-title":["Mathematical Structures in Computer Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0960129512000783","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2019,8,9]],"date-time":"2019-08-09T03:28:58Z","timestamp":1565321338000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0960129512000783\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,3,28]]},"references-count":125,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2014,6]]}},"alternative-id":["S0960129512000783"],"URL":"https:\/\/doi.org\/10.1017\/s0960129512000783","relation":{},"ISSN":["0960-1295","1469-8072"],"issn-type":[{"value":"0960-1295","type":"print"},{"value":"1469-8072","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,3,28]]},"article-number":"e240311"}}