{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,23]],"date-time":"2026-01-23T18:05:37Z","timestamp":1769191537073,"version":"3.49.0"},"reference-count":48,"publisher":"MDPI AG","issue":"12","license":[{"start":{"date-parts":[[2022,11,27]],"date-time":"2022-11-27T00:00:00Z","timestamp":1669507200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000781","name":"European Research Council","doi-asserted-by":"publisher","award":["678082"],"award-info":[{"award-number":["678082"]}],"id":[{"id":"10.13039\/501100000781","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>We define common thermodynamic concepts purely within the framework of general Markov chains and derive Jarzynski\u2019s equality and Crooks\u2019 fluctuation theorem in this setup. In particular, we regard the discrete-time case, which leads to an asymmetry in the definition of work that appears in the usual formulation of Crooks\u2019 fluctuation theorem. We show how this asymmetry can be avoided with an additional condition regarding the energy protocol. The general formulation in terms of Markov chains allows transferring the results to other application areas outside of physics. Here, we discuss how this framework can be applied in the context of decision-making. This involves the definition of the relevant quantities, the assumptions that need to be made for the different fluctuation theorems to hold, as well as the consideration of discrete trajectories instead of the continuous trajectories, which are relevant in physics.<\/jats:p>","DOI":"10.3390\/e24121731","type":"journal-article","created":{"date-parts":[[2022,11,28]],"date-time":"2022-11-28T03:28:49Z","timestamp":1669606129000},"page":"1731","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Jarzyski\u2019s Equality and Crooks\u2019 Fluctuation Theorem for General Markov Chains with Application to Decision-Making Systems"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-8176-0591","authenticated-orcid":false,"given":"Pedro","family":"Hack","sequence":"first","affiliation":[{"name":"Institute of Neural Information Processing, Ulm University, 89081 Ulm, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2906-3577","authenticated-orcid":false,"given":"Sebastian","family":"Gottwald","sequence":"additional","affiliation":[{"name":"Institute of Neural Information Processing, Ulm University, 89081 Ulm, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8637-6652","authenticated-orcid":false,"given":"Daniel A.","family":"Braun","sequence":"additional","affiliation":[{"name":"Institute of Neural Information Processing, Ulm University, 89081 Ulm, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,11,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Seifert, U. (2012). Stochastic thermodynamics, fluctuation theorems and molecular machines. Rep. Prog. Phys., 75.","DOI":"10.1088\/0034-4885\/75\/12\/126001"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"329","DOI":"10.1146\/annurev-conmatphys-062910-140506","article-title":"Equalities and inequalities: Irreversibility and the second law of thermodynamics at the nanoscale","volume":"2","author":"Jarzynski","year":"2011","journal-title":"Annu. Rev. Condens. Matter Phys."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"331","DOI":"10.1140\/epjb\/e2008-00254-2","article-title":"Nonequilibrium work relations: Foundations and applications","volume":"64","author":"Jarzynski","year":"2008","journal-title":"Eur. Phys. J. B"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Jarzynski, C. (1997). Equilibrium free-energy differences from nonequilibrium measurements: A master-equation approach. Phys. Rev. E, 56.","DOI":"10.1103\/PhysRevE.56.5018"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Jarzynski, C. (2004). Nonequilibrium work theorem for a system strongly coupled to a thermal environment. J. Stat. Mech. Theory Exp., 2004.","DOI":"10.1088\/1742-5468\/2004\/09\/P09005"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Jarzynski, C. (1997). Nonequilibrium equality for free energy differences. Phys. Rev. Lett., 78.","DOI":"10.1103\/PhysRevLett.78.2690"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Crooks, G.E. (1999). Entropy production fluctuation theorem and the nonequilibrium work relation for free energy differences. Phys. Rev. E, 60.","DOI":"10.1103\/PhysRevE.60.2721"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1481","DOI":"10.1023\/A:1023208217925","article-title":"Nonequilibrium measurements of free energy differences for microscopically reversible Markovian systems","volume":"90","author":"Crooks","year":"1998","journal-title":"J. Stat. Phys."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1038\/nature04061","article-title":"Verification of the Crooks fluctuation theorem and recovery of RNA folding free energies","volume":"437","author":"Collin","year":"2005","journal-title":"Nature"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1832","DOI":"10.1126\/science.1071152","article-title":"Equilibrium information from nonequilibrium measurements in an experimental test of Jarzynski\u2019s equality","volume":"296","author":"Liphardt","year":"2002","journal-title":"Science"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Saira, O.P., Yoon, Y., Tanttu, T., M\u00f6tt\u00f6nen, M., Averin, D., and Pekola, J.P. (2012). Test of the Jarzynski and Crooks fluctuation relations in an electronic system. Phys. Rev. Lett., 109.","DOI":"10.1103\/PhysRevLett.109.180601"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Douarche, F., Ciliberto, S., Petrosyan, A., and Rabbiosi, I. (2005). An experimental test of the Jarzynski equality in a mechanical experiment. Europhys. Lett., 70.","DOI":"10.1209\/epl\/i2005-10024-4"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1038\/nphys3197","article-title":"Experimental test of the quantum Jarzynski equality with a trapped-ion system","volume":"11","author":"An","year":"2015","journal-title":"Nat. Phys."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Smith, A., Lu, Y., An, S., Zhang, X., Zhang, J.N., Gong, Z., Quan, H., Jarzynski, C., and Kim, K. (2018). Verification of the quantum nonequilibrium work relation in the presence of decoherence. New J. Phys., 20.","DOI":"10.1088\/1367-2630\/aa9cd6"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Crooks, G.E. (2000). Path-ensemble averages in systems driven far from equilibrium. Phys. Rev. E, 61.","DOI":"10.1103\/PhysRevE.61.2361"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Buscemi, F., and Scarani, V. (2021). Fluctuation theorems from Bayesian retrodiction. Phys. Rev. E, 103.","DOI":"10.1103\/PhysRevE.103.052111"},{"key":"ref_17","unstructured":"Crooks, G.E. (1999). Excursions in Statistical Dynamics, University of California, Berkeley."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Goldt, S., and Seifert, U. (2017). Stochastic thermodynamics of learning. Phys. Rev. Lett., 118.","DOI":"10.1103\/PhysRevLett.118.010601"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Perunov, N., Marsland, R.A., and England, J.L. (2016). Statistical physics of adaptation. Phys. Rev. X, 6.","DOI":"10.1103\/PhysRevX.6.021036"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"919","DOI":"10.1038\/nnano.2015.250","article-title":"Dissipative adaptation in driven self-assembly","volume":"10","author":"England","year":"2015","journal-title":"Nat. Nanotechnol."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Still, S., Sivak, D.A., Bell, A.J., and Crooks, G.E. (2012). Thermodynamics of prediction. Phys. Rev. Lett., 109.","DOI":"10.1103\/PhysRevLett.109.120604"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ortega, P.A., and Braun, D.A. (2013). Thermodynamics as a theory of decision-making with information-processing costs. Proc. R. Soc. A Math. Phys. Eng. Sci., 469.","DOI":"10.1098\/rspa.2012.0683"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Parr, T., Da Costa, L., and Friston, K. (2020). Markov blankets, information geometry and stochastic thermodynamics. Philos. Trans. R. Soc., 378.","DOI":"10.1098\/rsta.2019.0159"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Da Costa, L., Friston, K., Heins, C., and Pavliotis, G.A. (2021). Bayesian mechanics for stationary processes. Proc. R. Soc. A, 477.","DOI":"10.1098\/rspa.2021.0518"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1371\/journal.pcbi.1008420","article-title":"The two kinds of free energy and the Bayesian revolution","volume":"16","author":"Gottwald","year":"2020","journal-title":"PLoS Comput. Biol."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Boyd, A.B., Crutchfield, J.P., and Gu, M. (2022). Thermodynamic machine learning through maximum work production. New J. Phys., 24.","DOI":"10.1088\/1367-2630\/ac4309"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wolpert, D.H. (2006). Information theory\u2014The bridge connecting bounded rational game theory and statistical physics. Complex Engineered Systems, Springer.","DOI":"10.1007\/3-540-32834-3_12"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Tishby, N., and Polani, D. (2011). Information theory of decisions and actions. Perception-Action Cycle, Springer.","DOI":"10.1007\/978-1-4419-1452-1_19"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Ortega, P.A., and Braun, D.A. (2011, January 15\u201318). Information, utility and bounded rationality. Proceedings of the International Conference on Artificial General Intelligence, San Francisco, CA, USA.","DOI":"10.1007\/978-3-642-22887-2_28"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Genewein, T., Leibfried, F., Grau-Moya, J., and Braun, D.A. (2015). Bounded rationality, abstraction, and hierarchical decision-making: An information-theoretic optimality principle. Front. Robot. AI, 2.","DOI":"10.3389\/frobt.2015.00027"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wolpert, D.H. (2016). The free energy requirements of biological organisms; implications for evolution. Entropy, 18.","DOI":"10.3390\/e18040138"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Grau-Moya, J., Kr\u00fcger, M., and Braun, D.A. (2018). Non-equilibrium relations for bounded rational decision-making in changing environments. Entropy, 20.","DOI":"10.3390\/e20010001"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Levin, D.A., and Peres, Y. (2017). Markov Chains and Mixing Times, American Mathematical Soc.","DOI":"10.1090\/mbk\/107"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Yang, Y.J., and Qian, H. (2020). Unified formalism for entropy production and fluctuation relations. Phys. Rev. E, 101.","DOI":"10.1103\/PhysRevE.101.022129"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Cohen, E., and Mauzerall, D. (2004). A note on the Jarzynski equality. J. Stat. Mech. Theory Exp., 2004.","DOI":"10.1088\/1742-5468\/2004\/07\/P07006"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Ge, H., and Qian, M. (2007). Generalized Jarzynski\u2019s equality in inhomogeneous Markov chains. J. Math. Phys., 48.","DOI":"10.1063\/1.2722561"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Jaynes, E.T. (1957). Information theory and statistical mechanics. Phys. Rev., 106.","DOI":"10.1103\/PhysRev.106.620"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Jaynes, E.T. (2003). Probability Theory: The Logic of Science, Cambridge University Press.","DOI":"10.1017\/CBO9780511790423"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1016\/0022-247X(90)90002-W","article-title":"Majorization and divergence","volume":"148","author":"Joe","year":"1990","journal-title":"J. Math. Anal. Appl."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Gottwald, S., and Braun, D.A. (2019). Bounded rational decision-making from elementary computations that reduce uncertainty. Entropy, 21.","DOI":"10.3390\/e21040375"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Hack, P., Lindig-Leon, C., Gottwald, S., and Braun, D.A. (2022). Thermodynamic fluctuation theorems govern human sensorimotor learning. arXiv.","DOI":"10.1038\/s41598-023-27736-8"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1111","DOI":"10.1152\/jn.00635.2011","article-title":"Facilitation of learning induced by both random and gradual visuomotor task variation","volume":"107","author":"Turnham","year":"2012","journal-title":"J. Neurophysiol."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Crooks, G.E. (2011). On thermodynamic and microscopic reversibility. J. Stat. Mech. Theory Exp., 2011.","DOI":"10.1088\/1742-5468\/2011\/07\/P07008"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Tolman, R.C. (1925). The principle of microscopic reversibility. Proc. Natl. Acad. Sci. USA, 11.","DOI":"10.1073\/pnas.11.7.436"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.physrep.2011.09.002","article-title":"Stochastic theory of nonequilibrium steady states and its applications. Part I","volume":"510","author":"Zhang","year":"2012","journal-title":"Phys. Rep."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Tang, Y., Yuan, R., Chen, J., and Ao, P. (2015). Work relations connecting nonequilibrium steady states without detailed balance. Phys. Rev. E, 91.","DOI":"10.1103\/PhysRevE.91.042108"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"604","DOI":"10.1126\/science.aac8167","article-title":"Broken detailed balance at mesoscopic scales in active biological systems","volume":"352","author":"Battle","year":"2016","journal-title":"Science"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"327","DOI":"10.1080\/00031305.1995.10476177","article-title":"Understanding the Metropolis-Hastings algorithm","volume":"49","author":"Chib","year":"1995","journal-title":"Am. Stat."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/12\/1731\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:27:51Z","timestamp":1760146071000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/24\/12\/1731"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,27]]},"references-count":48,"journal-issue":{"issue":"12","published-online":{"date-parts":[[2022,12]]}},"alternative-id":["e24121731"],"URL":"https:\/\/doi.org\/10.3390\/e24121731","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,11,27]]}}}