{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T02:40:49Z","timestamp":1775011249763,"version":"3.50.1"},"reference-count":57,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2021,10,14]],"date-time":"2021-10-14T00:00:00Z","timestamp":1634169600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001807","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de S\u00e3o Paulo","doi-asserted-by":"publisher","award":["2018\/19150-6"],"award-info":[{"award-number":["2018\/19150-6"]}],"id":[{"id":"10.13039\/501100001807","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001807","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de S\u00e3o Paulo","doi-asserted-by":"publisher","award":["2014\/50851-0"],"award-info":[{"award-number":["2014\/50851-0"]}],"id":[{"id":"10.13039\/501100001807","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003593","name":"Conselho Nacional de Desenvolvimento Cient\u00edfico e Tecnol\u00f3gico","doi-asserted-by":"publisher","award":["465755\/2014-3"],"award-info":[{"award-number":["465755\/2014-3"]}],"id":[{"id":"10.13039\/501100003593","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Entropy is a concept that emerged in the 19th century. It used to be associated with heat harnessed by a thermal machine to perform work during the Industrial Revolution. However, there was an unprecedented scientific revolution in the 20th century due to one of its most essential innovations, i.e., the information theory, which also encompasses the concept of entropy. Therefore, the following question is naturally raised: \u201cwhat is the difference, if any, between concepts of entropy in each field of knowledge?\u201d There are misconceptions, as there have been multiple attempts to conciliate the entropy of thermodynamics with that of information theory. Entropy is most commonly defined as \u201cdisorder\u201d, although it is not a good analogy since \u201corder\u201d is a subjective human concept, and \u201cdisorder\u201d cannot always be obtained from entropy. Therefore, this paper presents a historical background on the evolution of the term \u201centropy\u201d, and provides mathematical evidence and logical arguments regarding its interconnection in various scientific areas, with the objective of providing a theoretical review and reference material for a broad audience.<\/jats:p>","DOI":"10.3390\/e23101340","type":"journal-article","created":{"date-parts":[[2021,10,14]],"date-time":"2021-10-14T09:10:49Z","timestamp":1634202649000},"page":"1340","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":46,"title":["Entropy: From Thermodynamics to Information Processing"],"prefix":"10.3390","volume":"23","author":[{"given":"Jord\u00e3o","family":"Natal","sequence":"first","affiliation":[{"name":"Signal Processing Laboratory, Department of Electrical and Computing Engineering, University of S\u00e3o Paulo (USP), S\u00e3o Carlos 3566-590, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9129-0087","authenticated-orcid":false,"given":"Ivonete","family":"\u00c1vila","sequence":"additional","affiliation":[{"name":"Laboratory of Combustion and Carbon Captur, Department of Energy, School of Engineering, State University of S\u00e3o Paulo (Unesp), S\u00e3o Carlos 3566-590, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Victor Batista","family":"Tsukahara","sequence":"additional","affiliation":[{"name":"Signal Processing Laboratory, Department of Electrical and Computing Engineering, University of S\u00e3o Paulo (USP), S\u00e3o Carlos 3566-590, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marcelo","family":"Pinheiro","sequence":"additional","affiliation":[{"name":"Versatus Studio, S\u00e3o Carlos 4011-002, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0137-6678","authenticated-orcid":false,"given":"Carlos Dias","family":"Maciel","sequence":"additional","affiliation":[{"name":"Signal Processing Laboratory, Department of Electrical and Computing Engineering, University of S\u00e3o Paulo (USP), S\u00e3o Carlos 3566-590, Brazil"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,10,14]]},"reference":[{"key":"ref_1","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, John Wiley & Sons."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Greven, A., Keller, G., and Warnecke, G. (2014). Entropy, Princeton University Press.","DOI":"10.2307\/j.ctv3f8rnj"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1103\/RevModPhys.50.221","article-title":"General properties of entropy","volume":"50","author":"Wehrl","year":"1978","journal-title":"Rev. Mod. Phys."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Demirel, Y., and Gerbaud, V. (2019). Nonequilibrium Thermodynamics: Transport and Rate Processes in Physical, Chemical and Biological Systems, Elsevier.","DOI":"10.1016\/B978-0-444-64112-0.00002-2"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"e00596","DOI":"10.1016\/j.heliyon.2018.e00596","article-title":"An introduction to the maximum entropy approach and its application to inference problems in biology","volume":"4","year":"2018","journal-title":"Heliyon"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"67a","DOI":"10.1016\/j.bpj.2017.11.415","article-title":"Role of Conformational Entropy in Extremely High Affinity Protein Interactions","volume":"114","author":"Caro","year":"2018","journal-title":"Biophys. J."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1931","DOI":"10.3390\/e16041931","article-title":"Information in biological systems and the fluctuation theorem","volume":"16","author":"Demirel","year":"2014","journal-title":"Entropy"},{"key":"ref_9","unstructured":"Brooks, D.R., Wiley, E.O., and Brooks, D. (1988). Evolution as Entropy, University of Chicago Press."},{"key":"ref_10","unstructured":"Maldacena, J. (2018). Black hole entropy and quantum mechanics. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"109","DOI":"10.3847\/1538-4357\/aad5e1","article-title":"Supermassive Black Holes with High Accretion Rates in Active Galactic Nuclei. VII. Reconstruction of Velocity-delay Maps by the Maximum Entropy Method","volume":"864","author":"Xiao","year":"2018","journal-title":"Astrophys. J."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Bousso, R. (2018). Black hole entropy and the Bekenstein bound. arXiv.","DOI":"10.1142\/9789811203961_0012"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"871","DOI":"10.1177\/0954408916646139","article-title":"Shape effect of nanosize particles in unsteady mixed convection flow of nanofluid over disk with entropy generation","volume":"231","author":"Zeeshan","year":"2017","journal-title":"Proc. Inst. Mech. Eng. Part E J. Process Mech. Eng."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"610","DOI":"10.1109\/LSP.2016.2542881","article-title":"Dispersion entropy: A measure for time-series analysis","volume":"23","author":"Rostaghi","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1016\/j.ymssp.2016.03.016","article-title":"Identification of multiple faults in rotating machinery based on minimum entropy deconvolution combined with spectral kurtosis","volume":"81","author":"He","year":"2016","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Degaetano-Ortlieb, S., and Teich, E. (2017, January 10). Modeling intra-textual variation with entropy and surprisal: Topical vs. stylistic patterns. Proceedings of the Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, Vancouver, BC, Canada.","DOI":"10.18653\/v1\/W17-2209"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Reynar, J.C., and Ratnaparkhi, A. (1997). A maximum entropy approach to identifying sentence boundaries. Proceedings of the Fifth Conference on Applied Natural Language Processing, Association for Computational Linguistics.","DOI":"10.3115\/974557.974561"},{"key":"ref_18","unstructured":"Campbell, J. (1982). Grammatical Man: Information, Entropy, Language, and Life, Simon and Schuster."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Tame, J.R. (2019). On Entropy as Mixed-Up-Ness. Approaches to Entropy, Springer.","DOI":"10.1007\/978-981-13-2315-7"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"20150230","DOI":"10.1098\/rsta.2015.0230","article-title":"What is information?","volume":"374","author":"Adami","year":"2016","journal-title":"Philos. Trans. R. Soc. A Math. Phys. Eng. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"129","DOI":"10.1016\/j.energy.2016.01.071","article-title":"Misuse of thermodynamic entropy in economics","volume":"100","author":"Kovalev","year":"2016","journal-title":"Energy"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Hayflick, L. (2007). Entropy explains aging, genetic determinism explains longevity, and undefined terminology explains misunderstanding both. PLoS Genet., 3.","DOI":"10.1371\/journal.pgen.0030220"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1007\/BF00140964","article-title":"Entropy and nonsense","volume":"1","author":"Morowitz","year":"1986","journal-title":"Biol. Philos."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.3390\/e15041152","article-title":"Entropy and entropy production: Old misconceptions and new breakthroughs","volume":"15","author":"Martyushev","year":"2013","journal-title":"Entropy"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1093\/bjps\/54.2.291","article-title":"The von Neumann entropy: A reply to Shenker","volume":"54","author":"Henderson","year":"2003","journal-title":"Br. J. Philos. Sci."},{"key":"ref_26","first-page":"21","article-title":"What students\u2019 understand from entropy?: A review of selected literature","volume":"2","author":"Sozbilir","year":"2003","journal-title":"J. Balt. Sci. Educ."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1080\/00107517008202196","article-title":"Entropy and disorder","volume":"11","author":"Wright","year":"1970","journal-title":"Contemp. Phys."},{"key":"ref_28","unstructured":"Schrodinger, E. (1968). Order, disorder and entropy. Modern Systems Research for the Behavioral Scientist, Aldine."},{"key":"ref_29","first-page":"197","article-title":"Hidden Information, Energy Dispersion and Disorder: Does Entropy Really Measure Disorder?","volume":"8","author":"Soubane","year":"2018","journal-title":"World"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1088\/0143-0807\/20\/3\/308","article-title":"Sadi Carnot, Founder of the Second Law of Thermodynamics\u2019","volume":"20","author":"Erlichson","year":"1999","journal-title":"Eur. J. Phys."},{"key":"ref_31","unstructured":"Clausius, R. (1960). On the Motive Power of Heat, and on the Laws Which Can Be Deduced from It for the Theory of Heat, Annalen der Physik."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1021\/ed043p85","article-title":"Mathematical methods in elementary thermodynamics","volume":"43","author":"Blinder","year":"1966","journal-title":"J. Chem. Educ."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Boltzmann, L. (1974). The second law of thermodynamics. Theoretical Physics and Philosophical Problems, Springer.","DOI":"10.1007\/978-94-010-2091-6"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1002\/mawe.201200673","article-title":"Energy and entropy of crystals, melts and glasses or what is wrong in Kauzmann\u2019s paradox?","volume":"43","author":"Hoffmann","year":"2012","journal-title":"Mater. Werkst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"391","DOI":"10.1119\/1.1971557","article-title":"Gibbs vs. Boltzmann entropies","volume":"33","author":"Jaynes","year":"1965","journal-title":"Am. J. Phys."},{"key":"ref_36","unstructured":"Jones, D.S. (1979). Elementary Information Theory, Oxford University Press."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"179","DOI":"10.1038\/scientificamerican0971-179","article-title":"Energy and information","volume":"225","author":"Tribus","year":"1971","journal-title":"Sci. Am."},{"key":"ref_38","unstructured":"Williams, P.L., and Beer, R.D. (2010). Nonnegative decomposition of multivariate information. arXiv."},{"key":"ref_39","first-page":"369","article-title":"On tables of random numbers","volume":"25","author":"Kolmogorov","year":"1963","journal-title":"Sankhy\u0101 Indian J. Stat. Ser. A"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"595","DOI":"10.3390\/e13030595","article-title":"Entropy measures vs. Kolmogorov complexity","volume":"13","author":"Teixeira","year":"2011","journal-title":"Entropy"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zenil, H. (2020). Towards Demystifying Shannon Entropy, Lossless Compression and Approaches to Statistical Machine Learning. Proceedings, 47.","DOI":"10.3390\/proceedings2020047024"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"771","DOI":"10.1017\/S0960129511000521","article-title":"Algorithmic thermodynamics","volume":"22","author":"Baez","year":"2012","journal-title":"Math. Struct. Comput. Sci."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Zenil, H., Kiani, N.A., and Tegn\u00e9r, J. (2019). The thermodynamics of network coding, and an algorithmic refinement of the principle of maximum entropy. Entropy, 21.","DOI":"10.3390\/e21060560"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"012308","DOI":"10.1103\/PhysRevE.96.012308","article-title":"Low-algorithmic-complexity entropy-deceiving graphs","volume":"96","author":"Zenil","year":"2017","journal-title":"Phys. Rev. E"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"53143","DOI":"10.4249\/scholarpedia.53143","article-title":"Algorithmic Information Dynamics","volume":"15","author":"Zenil","year":"2020","journal-title":"Scholarpedia"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1160","DOI":"10.1016\/j.isci.2019.07.043","article-title":"An algorithmic information calculus for causal discovery and reprogramming systems","volume":"19","author":"Zenil","year":"2019","journal-title":"Iscience"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Zenil, H., Hern\u00e1ndez-Orozco, S., Kiani, N.A., Soler-Toscano, F., Rueda-Toicen, A., and Tegn\u00e9r, J. (2018). A decomposition method for global evaluation of shannon entropy and local estimations of algorithmic complexity. Entropy, 20.","DOI":"10.3390\/e20080605"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"1114","DOI":"10.1119\/1.19094","article-title":"What if entropy were dimensionless?","volume":"67","author":"Leff","year":"1999","journal-title":"Am. J. Phys."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Bernard, T.N., Shi, E.L., Gentle, K., Hakim, A., Hammett, G.W., Stoltzfus-Dueck, T., and Taylor, E.I. (2018). Gyrokinetic continuum simulations of plasma turbulence in the Texas Helimak. arXiv.","DOI":"10.1063\/1.5085457"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"205001","DOI":"10.1103\/PhysRevLett.114.205001","article-title":"Threefold increase of the bulk electron temperature of plasma discharges in a magnetic mirror device","volume":"114","author":"Bagryansky","year":"2015","journal-title":"Phys. Rev. Lett."},{"key":"ref_51","unstructured":"Maxwell, J.C., and Pesic, P. (2001). Theory of Heat, Courier Corporation."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"840","DOI":"10.1007\/BF01341281","article-title":"\u00dcber die Entropieverminderung in einem thermodynamischen System bei Eingriffen intelligenter Wesen","volume":"53","author":"Szilard","year":"1929","journal-title":"Z. F\u00fcr Phys."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1147\/rd.53.0183","article-title":"Irreversibility and heat generation in the computing process","volume":"5","author":"Landauer","year":"1961","journal-title":"IBM J. Res. Dev."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"180602","DOI":"10.1103\/PhysRevLett.109.180602","article-title":"Fluctuation theorem with information exchange: Role of correlations in stochastic thermodynamics","volume":"109","author":"Sagawa","year":"2012","journal-title":"Phys. Rev. Lett."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"620","DOI":"10.1103\/PhysRev.106.620","article-title":"Information theory and statistical mechanics","volume":"106","author":"Jaynes","year":"1957","journal-title":"Phys. Rev."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Klauder, J.R., and Skagerstam, B.S. (1985). Coherent States: Applications in Physics and Mathematical Physics, World Scientific.","DOI":"10.1142\/0096"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1119\/1.19071","article-title":"Thermodynamics and an Introduction to Thermostatistics","volume":"66","author":"Callen","year":"1998","journal-title":"Am. J. Phys."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/10\/1340\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:14:21Z","timestamp":1760166861000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/10\/1340"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,10,14]]},"references-count":57,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["e23101340"],"URL":"https:\/\/doi.org\/10.3390\/e23101340","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,10,14]]}}}