{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,12]],"date-time":"2026-05-12T22:05:23Z","timestamp":1778623523251,"version":"3.51.4"},"reference-count":51,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2014,4,29]],"date-time":"2014-04-29T00:00:00Z","timestamp":1398729600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/3.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>H-theorem states that the entropy production is nonnegative and, therefore, the entropy of a closed system should monotonically change in time. In information processing, the entropy production is positive for random transformation of signals (the information processing lemma). Originally, the H-theorem and the information processing lemma were proved for the classical Boltzmann-Gibbs-Shannon entropy and for the correspondent divergence (the relative entropy). Many new entropies and divergences have been proposed during last decades and for all of them the H-theorem is needed. This note proposes a simple and general criterion to check whether the H-theorem is valid for a convex divergence H and demonstrates that some of the popular divergences obey no H-theorem. We consider systems with n states Ai that obey first order kinetics (master equation). A convex function H is a Lyapunov function for all master equations with given equilibrium if and only if its conditional minima properly describe the equilibria of pair transitions Ai \u21cc Aj . This theorem does not depend on the principle of detailed balance and is valid for general Markov kinetics. Elementary analysis of pair equilibria demonstrate that the popular Bregman divergences like Euclidian distance or Itakura-Saito distance in the space of distribution cannot be the universal Lyapunov functions for the first-order kinetics and can increase in Markov processes. Therefore, they violate the second law and the information processing lemma. In particular, for these measures of information (divergences) random manipulation with data may add information to data. The main results are extended to nonlinear generalized mass action law kinetic equations.<\/jats:p>","DOI":"10.3390\/e16052408","type":"journal-article","created":{"date-parts":[[2014,4,30]],"date-time":"2014-04-30T04:40:45Z","timestamp":1398832845000},"page":"2408-2432","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["General H-theorem and Entropies that Violate the Second Law"],"prefix":"10.3390","volume":"16","author":[{"given":"Alexander","family":"Gorban","sequence":"first","affiliation":[{"name":"Department of Mathematics, University of Leicester, University Road, Leicester, LE1 7RH, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2014,4,29]]},"reference":[{"key":"ref_1","unstructured":"R\u00e9nyi, A. (,  1960). On measures of entropy and information, Berkeley, CA, USA."},{"key":"ref_2","first-page":"85","article-title":"Eine informationstheoretische Ungleichung und ihre Anwendung auf den Beweis der Ergodizit\u00a8at von Markoffschen Ketten","volume":"8","year":"1963","journal-title":"Magyar. Tud. Akad. Mat. Kutato Int. Kozl"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1143\/JPSJ.18.328","article-title":"Markov processes and the H-theorem","volume":"12","author":"Morimoto","year":"1963","journal-title":"J. Phys. Soc. Jap"},{"key":"ref_4","unstructured":"Callen, H.B. (1985). Thermodynamics and an Introduction to Themostatistics, Wiley. [2nd ed.]."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"375","DOI":"10.1190\/1.1440265","article-title":"The relationship between maximum entropy spectra and maximum likelihood spectra","volume":"37","author":"Burg","year":"1972","journal-title":"Geophysics"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"440","DOI":"10.1111\/j.2517-6161.1984.tb01318.x","article-title":"Multinomial Goodness of Fit Tests","volume":"46","author":"Cressie","year":"1984","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"479","DOI":"10.1007\/BF01016429","article-title":"Possible generalization of Boltzmann-Gibbs statistics","volume":"52","author":"Tsallis","year":"1988","journal-title":"J. Stat. Phys"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Abe, S., and Okamoto, Y. (2001). Nonextensive Statistical Mechanics and its Applications, Springer.","DOI":"10.1007\/3-540-40919-X"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1532","DOI":"10.3390\/e12061532","article-title":"Families of alpha- beta- and gamma- divergences: Flexible and robust measures of similarities","volume":"12","author":"Cichocki","year":"2010","journal-title":"Entropy"},{"key":"ref_10","first-page":"337","article-title":"A summary of entropy statistics","volume":"31","author":"Esteban","year":"1995","journal-title":"Kybernetica"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.3390\/e12051145","article-title":"Entropy: The Markov ordering approach","volume":"12","author":"Gorban","year":"2010","journal-title":"Entropy"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"200","DOI":"10.1016\/0041-5553(67)90040-7","article-title":"The relaxation method of finding the common points of convex sets and its application to the solution of problems in convex programming","volume":"7","author":"Bregman","year":"1967","journal-title":"USSR Comput. Math. Math. Phys"},{"key":"ref_13","first-page":"1705","article-title":"Clustering with Bregman divergences","volume":"6","author":"Banerjee","year":"2005","journal-title":"J. Mach. Learn. Res"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Csisz\u00e1r, I., and Mat\u00fa\u0161, F. (Generalized minimizers of convex integral functionals, Bregman distance, Pythagorean identities, 2012). Generalized minimizers of convex integral functionals, Bregman distance, Pythagorean identities, arXiv:1202.0666 [math.OC].","DOI":"10.1007\/978-3-642-40020-9_32"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"379","DOI":"10.1002\/j.1538-7305.1948.tb01338.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon","year":"1948","journal-title":"Bell Syst. Tech. J"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1090\/conm\/149\/01262","article-title":"Majorization, monotonicity of relative entropy and stochastic matrices","volume":"149","author":"Cohen","year":"1993","journal-title":"Contemp. Math"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"211","DOI":"10.1016\/0024-3795(93)90331-H","article-title":"Relative entropy under mappings by stochastic matrices","volume":"179","author":"Cohen","year":"1993","journal-title":"Linear Algebra Appl"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"380","DOI":"10.1016\/S0378-4371(03)00578-8","article-title":"Monotonically equivalent entropies and solution of additivity equation","volume":"328","author":"Gorban","year":"2003","journal-title":"Physica A"},{"key":"ref_19","unstructured":"Leung, C.S., Lee, M., and Chan, J.H. (2009, January 1\u20135). Divergence, Optimization, Geometry, Bankok, Thailand."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Meyn, S.R. (2007). Control Techniques for Complex Networks, Cambridge University Press.","DOI":"10.1017\/CBO9780511804410"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Meyn, S.R., and Tweedie, R.L. (2009). Markov Chains and Stochastic Stability, Cambridge University Press.","DOI":"10.1017\/CBO9780511626630"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1016\/0009-2509(74)80195-8","article-title":"Dynamics of open chemical systems and the algebraic structure of the underlying reaction network","volume":"29","author":"Feinberg","year":"1974","journal-title":"Chem. Eng. Sci"},{"key":"ref_23","first-page":"193","article-title":"Finding weakly reversible realizations of chemical reaction networks using optimization","volume":"67","author":"Hangos","year":"2012","journal-title":"Computer"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.1016\/j.physa.2010.11.030","article-title":"Kinetic path summation, multi-sheeted extension of master equation, and evaluation of ergodicity coefficient","volume":"390","author":"Gorban","year":"2011","journal-title":"Physica A"},{"key":"ref_25","unstructured":"Van Kampen, N.G. (1981). Stochastic processes in physics and chemistry, North-Holland."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1111","DOI":"10.1016\/j.physa.2012.11.028","article-title":"Local equivalence of reversible and general Markov kinetics","volume":"392","author":"Gorban","year":"2013","journal-title":"Physica A"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1438","DOI":"10.1016\/j.camwa.2013.01.004","article-title":"Maxallent: Maximizers of all entropies and uncertainty of uncertainty","volume":"65","author":"Gorban","year":"2013","journal-title":"Comput. Math. Appl"},{"key":"ref_28","unstructured":"Golub, G.H., and van Loan, C.F. (1996). Matrix Computations, Johns Hopkins University Press."},{"key":"ref_29","unstructured":"Gorban, A.N. (1984). Equilibrium encircling. Equations of Chemical Kinetics and Their Thermodynamic Analysis, Nauka. (in Russian)."},{"key":"ref_30","unstructured":"Yablonskii, G.S., Bykov, V.I., Gorban, A.N., and Elokhin, V.I. (1991). Kinetic Models of Catalytic Reactions, Elsevier."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Grmela, M., and \u00d6ttinger, H.C. (1997). Dynamics and thermodynamics of complex fluids. I. Development of a general formalism. Phys. Rev. E, 56.","DOI":"10.1103\/PhysRevE.56.6620"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"976","DOI":"10.1016\/j.physd.2012.02.008","article-title":"Fluctuations in extended mass-action-law dynamics","volume":"241","author":"Grmela","year":"2012","journal-title":"Physica D"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1016\/j.physd.2011.12.002","article-title":"Supercritical fluid thermodynamics from equations of state","volume":"241","author":"Giovangigli","year":"2012","journal-title":"Physica D"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"966","DOI":"10.3390\/e13050966","article-title":"The Michaelis-Menten-Stueckelberg Theorem","volume":"13","author":"Gorban","year":"2011","journal-title":"Entropy"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"772","DOI":"10.3390\/e12040772","article-title":"Engineering model reduction and entropy-based Lyapunov functions in chemical reaction kinetics","volume":"12","author":"Hangos","year":"2010","journal-title":"Entropy"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1016\/0016-0032(84)90052-8","article-title":"The macrodynamics of open systems and the variational principle of the local potential","volume":"318","author":"Orlov","year":"1984","journal-title":"J. Franklin Inst-Eng. Appl. Math"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/BF00251866","article-title":"On chemical kinetics of a certain class","volume":"46","author":"Feinberg","year":"1972","journal-title":"Arch. Rat. Mechan. Anal"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1318","DOI":"10.1016\/j.physa.2012.10.009","article-title":"Thermodynamics in the limit of irreversible reactions","volume":"392","author":"Gorban","year":"2013","journal-title":"Physica A"},{"key":"ref_39","first-page":"153","article-title":"Neuer Beweis zweier S\u00e4tze \u00fcber das W\u00e4rmegleichgewicht unter mehratomigen Gasmolek\u00fcle","volume":"95","author":"Boltzmann","year":"1887","journal-title":"Sitzungsberichte der Kaiserlichen Akademie der Wissenschaften in Wien"},{"key":"ref_40","first-page":"115","article-title":"\u00dcber das Gleichgewicht der lebendigen Kraft unter Gasmolek\u00fclen","volume":"95","author":"Lorentz","year":"1887","journal-title":"Sitzungsberichte der Kaiserlichen Akademie der Wissenschaften in Wien"},{"key":"ref_41","first-page":"577","article-title":"Theoreme H et unitarite de S","volume":"25","author":"Stueckelberg","year":"1952","journal-title":"Helv. Phys. Acta"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1007\/BF00251225","article-title":"General mass action kinetics","volume":"47","author":"Horn","year":"1972","journal-title":"Arch. Ration. Mech. Anal"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1163","DOI":"10.1007\/s10910-011-9804-9","article-title":"Finding complex balanced and detailed balanced realizations of chemical reaction networks","volume":"49","author":"Hangos","year":"2011","journal-title":"J. Math. Chem"},{"key":"ref_44","unstructured":"Kalpazidou, S.L. (2006). Cycle Representations of Markov Processes, Springer."},{"key":"ref_45","unstructured":"Rockafellar, R.T. (1997). Convex Analysis, Princeton University Press."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Kolokoltsov, V.N. (2010). Nonlinear Markov Processes and Kinetic Equations, Cambridge University Press.","DOI":"10.1017\/CBO9780511760303"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1553","DOI":"10.1287\/opre.19.7.1553","article-title":"A review of quasi-convex functions","volume":"19","author":"Greenberg","year":"1971","journal-title":"Oper. Res"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1137\/1009007","article-title":"Seven kinds of convexity","volume":"9","author":"Ponstein","year":"1967","journal-title":"SIAM Rev"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1007\/s004540010069","article-title":"On directional convexity","volume":"25","year":"2001","journal-title":"Discret. Comput. Geom"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"540","DOI":"10.1287\/moor.21.3.540","article-title":"Directional-quasi-convexity, asymmetric Schur-convexity and optimality of consecutive partitions","volume":"21","author":"Hwang","year":"1996","journal-title":"Math. Oper. Res"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Marshall, A.W., Olkin, I., and Arnold, B.C. (2011). Inequalities: Theory of Majorization and its Applications, Springer.","DOI":"10.1007\/978-0-387-68276-1"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/16\/5\/2408\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T21:10:49Z","timestamp":1760217049000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/16\/5\/2408"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2014,4,29]]},"references-count":51,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2014,5]]}},"alternative-id":["e16052408"],"URL":"https:\/\/doi.org\/10.3390\/e16052408","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2014,4,29]]}}}