{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T22:41:43Z","timestamp":1760222503782,"version":"build-2065373602"},"reference-count":14,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2001,2,1]],"date-time":"2001-02-01T00:00:00Z","timestamp":980985600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Certain aspects of the history, derivation, and physical application of the information-theoretic entropy concept are discussed. Pre-dating Shannon, the concept is traced back to Pauli. A derivation from first principles is given, without use of approximations. The concept depends on the underlying degree of randomness. In physical applications, this translates to dependence on the experimental apparatus available. An example illustrates how this dependence affects Prigogine's proposal for the use of the Second Law of Thermodynamics as a selection principle for the breaking of time symmetry. The dependence also serves to yield a resolution of the so-called ``Gibbs Paradox.'' Extension of the concept from the discrete to the continuous case is discussed. The usual extension is shown to be dimensionally incorrect. Correction introduces a reference density, leading to the concept of Kullback entropy. Practical relativistic considerations suggest a possible proper reference density.<\/jats:p>","DOI":"10.3390\/e3010001","type":"journal-article","created":{"date-parts":[[2008,10,25]],"date-time":"2008-10-25T13:44:42Z","timestamp":1224942282000},"page":"1-11","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Some Observations on the Concepts of Information-Theoretic Entropy and Randomness"],"prefix":"10.3390","volume":"3","author":[{"given":"Jonathan D.H.","family":"Smith","sequence":"first","affiliation":[{"name":"Department of Mathematics, Iowa State University, Ames, IA 50011, USA"}]}],"member":"1968","published-online":{"date-parts":[[2001,2,1]]},"reference":[{"key":"ref_1","unstructured":"Ash, R.B. (1965). Information Theory, Interscience."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Feynman, R.P., Leighton, R.B., and Sands, M. (1963). The Feynman Lectures on Physics, Addison-Wesley.","DOI":"10.1063\/1.3051743"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Buck, B., and Macaulay, V.A. (1991). Maximum Entropy in Action, Clarendon Press.","DOI":"10.1093\/oso\/9780198539414.001.0001"},{"key":"ref_4","unstructured":"Ford, K.W. (1963). Statistical Physics, 1962 Brandeis Lectures."},{"key":"ref_5","unstructured":"Kullback, S. (1959). Information Theory and Statistics, Wiley."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Li, M., and Vitanyi, P. (1997). An Introduction to Kolmogorov Complexity and its Applications, Springer.","DOI":"10.1007\/978-1-4757-2606-0"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"602","DOI":"10.1016\/S0019-9958(66)80018-9","article-title":"The definition of random sequences","volume":"9","year":"1966","journal-title":"Information and Control"},{"key":"ref_8","unstructured":"O\u2019Neill, E.L. (1963). Introduction to Statistical Optics, Addison-Wesley."},{"key":"ref_9","unstructured":"Pauli, W. (1933). Handbuch der Physik, xxiv\/1, 151."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"4590","DOI":"10.1073\/pnas.80.14.4590","article-title":"The Second Law as a selection principle: the microscopic theory of dissipative processes in quantum systems","volume":"80","author":"Prigogine","year":"1983","journal-title":"Proc. Nat. Acad. Sci."},{"key":"ref_11","unstructured":"Rumer, Yu.B., and Ryvkin, M.Sh. (1977). Termodinamika, Statisticheskaya Fizika i Kinetika, English Translation (1980): Thermodynamics, Statistical Physics, and Kinetics, Mir, Moscow."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell System Tech. J."},{"key":"ref_13","unstructured":"Tolman, R.C. (1938). The Principles of Statistical Mechanics, Oxford University Press."},{"key":"ref_14","first-page":"105","article-title":"Can an (individual) sequence of zeros and ones be random?","volume":"45","author":"Semenov","year":"1990","journal-title":"Uspekhi Mat. Nauk"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/3\/1\/1\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T22:27:41Z","timestamp":1760221661000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/3\/1\/1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2001,2,1]]},"references-count":14,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2001,3]]}},"alternative-id":["e3010001"],"URL":"https:\/\/doi.org\/10.3390\/e3010001","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2001,2,1]]}}}