{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T05:17:37Z","timestamp":1772774257506,"version":"3.50.1"},"reference-count":42,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2014,6,30]],"date-time":"2014-06-30T00:00:00Z","timestamp":1404086400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>This paper presents a comprehensive introduction and systematic derivation of the evolutionary equations for absolute entropy H and relative entropy D, some of which exist sporadically in the literature in different forms under different subjects, within the framework of dynamical systems. In general, both H and D are dissipated, and the dissipation bears a form reminiscent of the Fisher information; in the absence of stochasticity, dH\/dt is connected to the rate of phase space expansion, and D stays invariant, i.e., the separation of two probability density functions is always conserved. These formulas are validated with linear systems, and put to application with the Lorenz system and a large-dimensional stochastic quasi-geostrophic flow problem. In the Lorenz case, H falls at a constant rate with time, implying that H will eventually become negative, a situation beyond the capability of the commonly used computational technique like coarse-graining and bin counting. For the stochastic flow problem, it is first reduced to a computationally tractable low-dimensional system, using a reduced model approach, and then handled through ensemble prediction. Both the Lorenz system and the stochastic flow system are examples of self-organization in the light of uncertainty reduction. The latter particularly shows that, sometimes stochasticity may actually enhance the self-organization process.<\/jats:p>","DOI":"10.3390\/e16073605","type":"journal-article","created":{"date-parts":[[2014,6,30]],"date-time":"2014-06-30T10:34:47Z","timestamp":1404124487000},"page":"3605-3634","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Entropy Evolution and Uncertainty Estimation with Dynamical Systems"],"prefix":"10.3390","volume":"16","author":[{"given":"X.","family":"Liang","sequence":"first","affiliation":[{"name":"School of Marine Sciences and School of Mathematics and Statistics, Nanjing University of Information Science and Technology (Nanjing Institute of Meteorology), 219 Ningliu Blvd,Nanjing 210044, China"},{"name":"China Institute for Advanced Study, Central University of Finance and Economics, 39 South College Ave, Beijing 100081, China\u00a0"}]}],"member":"1968","published-online":{"date-parts":[[2014,6,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J."},{"key":"ref_2","first-page":"620","article-title":"Information theory and statistical mechanics","volume":"106","author":"Jaynes","year":"1957","journal-title":"PhysRev"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Georgescu-Roegen, N. (1971). The Entropy and the Economic Process, Harvard University Press.","DOI":"10.4159\/harvard.9780674281653"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.3390\/e1010001","article-title":"Diversity and entropy","volume":"1","author":"Lin","year":"1999","journal-title":"Entropy"},{"key":"ref_5","first-page":"1528","article-title":"On the definition of entropy production via examples","volume":"41","author":"Maes","year":"2000","journal-title":"MathPhys"},{"key":"ref_6","first-page":"701","article-title":"How should one define entropy production for nonequilibrium quantum spin systems?","volume":"14","author":"Ruelle","year":"2002","journal-title":"MathPhys"},{"key":"ref_7","first-page":"175","article-title":"A note on the entropy production formula","volume":"327","author":"Pillet","year":"2003","journal-title":"ContempMath"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"680","DOI":"10.1063\/1.1781911","article-title":"Entropy production in nonequilibrium stationary states: A point of view","volume":"14","author":"Gallavotti","year":"2004","journal-title":"Chaos"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1016\/j.physd.2004.01.008","article-title":"On the (Boltzmann) entropy of nonequilibrium systems","volume":"224","author":"Goldstein","year":"2004","journal-title":"Physica D"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1285","DOI":"10.1590\/S0103-97332006000700029","article-title":"Entropy production in nonequilibrium systems described by a Fokker\u2013Planck equation","volume":"36","year":"2006","journal-title":"Braz. J. Phys"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1063\/1.2207716","article-title":"On entropy production for controlled Markovian evolution","volume":"47","author":"Pavon","year":"2006","journal-title":"J. Math. Phys"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2570","DOI":"10.3390\/e15072570","article-title":"Fact-checking Ziegler\u2019s maximum entropy production principle beyond the linear regimes and towards steady states","volume":"15","author":"Polettini","year":"2013","journal-title":"Entropy"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.3390\/e15041152","article-title":"Entropy and entropy production: Old misconceptions and new breakthroughs","volume":"15","author":"Martyushev","year":"2013","journal-title":"Entropy"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1111\/j.2153-3490.1965.tb01424.x","article-title":"A study of the predictability of a 28-variable atmospheric model","volume":"17","author":"Lorenz","year":"1965","journal-title":"Tellus"},{"key":"ref_15","first-page":"739","article-title":"Stochastic dynamic prediction","volume":"21","author":"Epstein","year":"1969","journal-title":"Tellus"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1175\/1520-0493(1974)102<0409:TSOMCF>2.0.CO;2","article-title":"Theoretical skill of Monte Carlo forecasts","volume":"102","author":"Leith","year":"1974","journal-title":"MonWeather Rev"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"286","DOI":"10.1175\/1520-0469(1997)054<0286:OPOFEC>2.0.CO;2","article-title":"Optimal prediction of forecast error covariances through singular vectors","volume":"54","author":"Ehrendorfer","year":"1997","journal-title":"J. AtmosSci"},{"key":"ref_18","first-page":"762","article-title":"The dynamics of error growth and predictability in a model of the Gulf Stream. II. Ensemble prediction","volume":"29","author":"Moore","year":"1999","journal-title":"J. PhysOceanogr"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"3133","DOI":"10.1175\/1520-0442(1999)012<3133:ACFFPS>2.0.CO;2","article-title":"A conceptual framework for predictability studies","volume":"12","author":"Schneider","year":"1999","journal-title":"J. Clim"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1088\/0034-4885\/63\/2\/201","article-title":"Predicting uncertainty in forecasts of weather and climate","volume":"63","author":"Palmer","year":"2000","journal-title":"Rep. ProgPhys"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2057","DOI":"10.1175\/1520-0469(2002)059<2057:MDPUUR>2.0.CO;2","article-title":"Measuring dynamical prediction utility using relative entropy","volume":"59","author":"Kleeman","year":"2002","journal-title":"J. AtmosSci"},{"key":"ref_22","first-page":"249","article-title":"Predictability, uncertainty, and hyperbolicity in the ocean","volume":"41","author":"Kirwan","year":"2003","journal-title":"Int. J. EngSci"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1016\/j.jcp.2006.02.010","article-title":"Uncertainty estimation and prediction for interdisciplinary ocean dynamics","volume":"217","author":"Lermusiaux","year":"2006","journal-title":"J. Comput. Phys"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1012","DOI":"10.1109\/TPDS.2011.64","article-title":"Many task computing for real-time uncertainty prediction and data assimilation in the ocean","volume":"22","author":"Evangelinos","year":"2011","journal-title":"IEEE Trans. Parallel. DistrSyst"},{"key":"ref_25","unstructured":"Cover, T.M., and Thomas, J.A. (1991). Elements of Information Theory, Wiley."},{"key":"ref_26","first-page":"45","article-title":"The rate of entropy change in non-Hamiltonian systems","volume":"111A","author":"Andrey","year":"1985","journal-title":"PhysLett"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"244101","DOI":"10.1103\/PhysRevLett.95.244101","article-title":"Information transfer between dynamical system components","volume":"95","author":"Liang","year":"2005","journal-title":"Phys. Rev. Lett"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"138701","DOI":"10.1103\/PhysRevLett.93.138701","article-title":"Liouville Dynamics and the conservation of information","volume":"93","author":"Plastino","year":"2004","journal-title":"Phys. Rev. Lett"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.physd.2012.12.011","article-title":"Local predictability and information flow in complex dynamical systems","volume":"248","author":"Liang","year":"2013","journal-title":"Physica D"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3927","DOI":"10.1103\/PhysRevE.56.3927","article-title":"Minimum Kullback entropy approach to the Fokker\u2013Planck equation","volume":"56","author":"Plastino","year":"1997","journal-title":"Phys. Rev. E"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.physd.2007.04.002","article-title":"A rigorous formalism of information transfer between dynamical system components. I. Discrete mapping","volume":"231","author":"Liang","year":"2007","journal-title":"Physica D"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Friston, K., and Ao, P. (2012). Free energy, value, and attractors. Comput. Math. Methods Med.","DOI":"10.1155\/2012\/937860"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Duan, J., Gao, H., and Schmalfu, B. (2002). Stochastic dynamics of a coupled atmosphere-ocean model. StochDyn.","DOI":"10.1142\/S0219493702000467"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1002\/cpa.1014","article-title":"A mathematical framework for stochastic climate models","volume":"54","author":"Majda","year":"2001","journal-title":"Commun. Pure Appl. Math"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.ocemod.2014.04.002","article-title":"Stochastic parameterization of ocean mesoscale eddies","volume":"79","author":"Mana","year":"2014","journal-title":"Ocean Model"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Pedlosky, J. (1979). Geophysical Fluid Dynamics, Springer. [2nd ed].","DOI":"10.1007\/978-1-4684-0071-7"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Vallis, G.K. (2006). Atmospheric and Oceanic Fluid Dynamics, Cambridge University Press.","DOI":"10.1017\/CBO9780511790447"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/j.dynatmoce.2011.03.003","article-title":"Uncertainty generation in deterministic flows: Theory and application with an atmospheric jet stream model","volume":"52","author":"Liang","year":"2011","journal-title":"Dyn. Atmos. Ocean"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2025","DOI":"10.1175\/1520-0469(1996)053<2025:GSTPIA>2.0.CO;2","article-title":"Generalized stability theory. Part I. Autonomous operators","volume":"53","author":"Farrell","year":"1996","journal-title":"J. AtmosSci"},{"key":"ref_40","unstructured":"Preisendorfer, R.W. (1988). Principal Component Analysis in Meteorology and Oceanography, Elsevier."},{"key":"ref_41","first-page":"1551","article-title":"A non-equilibrium free energy theorem for deterministic systems","volume":"101","author":"Evans","year":"2003","journal-title":"MolPhys"},{"key":"ref_42","unstructured":"Bar-Yam, Y. (1997). Dynamics of Complex Systems, Perseus Books."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/16\/7\/3605\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:13:06Z","timestamp":1760217186000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/16\/7\/3605"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,6,30]]},"references-count":42,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2014,7]]}},"alternative-id":["e16073605"],"URL":"https:\/\/doi.org\/10.3390\/e16073605","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,6,30]]}}}