{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,4]],"date-time":"2026-07-04T00:13:49Z","timestamp":1783124029664,"version":"3.54.6"},"reference-count":174,"publisher":"Springer Science and Business Media LLC","issue":"1-2","license":[{"start":{"date-parts":[[2024,4,1]],"date-time":"2024-04-01T00:00:00Z","timestamp":1711929600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,1]],"date-time":"2024-04-01T00:00:00Z","timestamp":1711929600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Karlsruher Institut f\u00fcr Technologie (KIT)"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Math Meth Oper Res"],"published-print":{"date-parts":[[2024,4]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The paper provides an overview of the theory and applications of risk-sensitive Markov decision processes. The term \u2019risk-sensitive\u2019 refers here to the use of the Optimized Certainty Equivalent as a means to measure expectation and risk. This comprises the well-known entropic risk measure and Conditional Value-at-Risk. We restrict our considerations to stationary problems with an infinite time horizon. Conditions are given under which optimal policies exist and solution procedures are explained. We present both the theory when the Optimized Certainty Equivalent is applied recursively as well as the case where it is applied to the cumulated reward. Discounted as well as non-discounted models are reviewed.<\/jats:p>","DOI":"10.1007\/s00186-024-00857-0","type":"journal-article","created":{"date-parts":[[2024,4,1]],"date-time":"2024-04-01T09:12:10Z","timestamp":1711962730000},"page":"141-178","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":26,"title":["Markov decision processes with risk-sensitive criteria: an overview"],"prefix":"10.1007","volume":"99","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0077-3444","authenticated-orcid":false,"given":"Nicole","family":"B\u00e4uerle","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8480-7850","authenticated-orcid":false,"given":"Anna","family":"Ja\u015bkiewicz","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,4,1]]},"reference":[{"key":"857_CR1","doi-asserted-by":"publisher","first-page":"226","DOI":"10.1016\/j.ejor.2006.07.016","volume":"182","author":"S Ahmed","year":"2007","unstructured":"Ahmed S, \u00c7akmak U, Shapiro A (2007) Coherent risk measures in inventory problems. Eur J Oper Res 182:226\u2013238","journal-title":"Eur J Oper Res"},{"issue":"2","key":"857_CR2","doi-asserted-by":"publisher","first-page":"961","DOI":"10.1137\/151002630","volume":"55","author":"V Anantharam","year":"2017","unstructured":"Anantharam V, Borkar VS (2017) A variational formula for risk-sensitive reward. SIAM J Control Optim 55(2):961\u2013988","journal-title":"SIAM J Control Optim"},{"issue":"2","key":"857_CR3","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1016\/j.jet.2004.05.008","volume":"125","author":"EW Anderson","year":"2005","unstructured":"Anderson EW (2005) The dynamics of risk-sensitive allocations. J Econ Theory 125(2):93\u2013150","journal-title":"J Econ Theory"},{"key":"857_CR4","doi-asserted-by":"crossref","unstructured":"Arapostathis A, Borkar VS (2021) Linear and dynamic programs for risk-sensitive cost minimization. In: Proceedings of the 60th IEEE conference on decision and control. IEEE, pp 3042\u20133047","DOI":"10.1109\/CDC45484.2021.9683319"},{"issue":"4","key":"857_CR5","doi-asserted-by":"publisher","first-page":"1458","DOI":"10.1007\/s10959-015-0616-x","volume":"29","author":"A Arapostathis","year":"2016","unstructured":"Arapostathis A, Borkar VS, Kumar SK (2016) Risk-sensitive control and an abstract Collatz\u2013Wielandt formula. J Theor Probab 29(4):1458\u20131484","journal-title":"J Theor Probab"},{"key":"857_CR6","unstructured":"Arrow KJ (1971) The theory of risk aversion. In: Essays in the theory of risk-bearing. North Holland, pp 90\u2013120"},{"key":"857_CR7","doi-asserted-by":"publisher","first-page":"149","DOI":"10.4064\/am2317-1-2017","volume":"44","author":"H Asienkiewicz","year":"2017","unstructured":"Asienkiewicz H, Ja\u015bkiewicz A (2017) A note on a new class of recursive utilities in Markov decision processes. Applicationes Mathematicae 44:149\u2013161","journal-title":"Applicationes Mathematicae"},{"key":"857_CR8","doi-asserted-by":"publisher","first-page":"295","DOI":"10.1007\/s10957-014-0555-1","volume":"165","author":"\u0141 Balbus","year":"2015","unstructured":"Balbus \u0141, Ja\u015bkiewicz A, Nowak AS (2015) The dynamics of risk-sensitive allocations. J Optim Theory Appl 165:295\u2013315","journal-title":"J Optim Theory Appl"},{"key":"857_CR9","doi-asserted-by":"publisher","first-page":"565","DOI":"10.1007\/s00186-006-0135-8","volume":"65","author":"C Barz","year":"2007","unstructured":"Barz C, Waldmann KH (2007) Risk-sensitive capacity control in revenue management. Math Methods Oper Res 65:565\u2013579","journal-title":"Math Methods Oper Res"},{"issue":"4","key":"857_CR10","doi-asserted-by":"publisher","first-page":"880","DOI":"10.1287\/moor.1080.0324","volume":"33","author":"A Basu","year":"2008","unstructured":"Basu A, Bhattacharyya T, Borkar VS (2008) A learning algorithm for risk-sensitive cost. Math Oper Res 33(4):880\u2013898","journal-title":"Math Oper Res"},{"issue":"3","key":"857_CR11","doi-asserted-by":"publisher","first-page":"1757","DOI":"10.1287\/moor.2021.1187","volume":"47","author":"N B\u00e4uerle","year":"2022","unstructured":"B\u00e4uerle N, Glauner A (2022) Distributionally robust Markov decision processes and their connection to risk measures. Math Oper Res 47(3):1757\u20131780","journal-title":"Math Oper Res"},{"issue":"3","key":"857_CR12","doi-asserted-by":"publisher","first-page":"953","DOI":"10.1016\/j.ejor.2021.04.030","volume":"296","author":"N B\u00e4uerle","year":"2022","unstructured":"B\u00e4uerle N, Glauner A (2022) Markov decision processes with recursive risk measures. Eur J Oper Res 296(3):953\u2013966","journal-title":"Eur J Oper Res"},{"issue":"1","key":"857_CR13","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1016\/j.ejor.2014.10.046","volume":"242","author":"N B\u00e4uerle","year":"2015","unstructured":"B\u00e4uerle N, Ja\u015bkiewicz A (2015) Risk-sensitive dividend problems. Eur J Oper Res 242(1):161\u2013171","journal-title":"Eur J Oper Res"},{"key":"857_CR14","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1016\/j.insmatheco.2017.01.006","volume":"73","author":"N B\u00e4uerle","year":"2017","unstructured":"B\u00e4uerle N, Ja\u015bkiewicz A (2017) Optimal dividend payout model with risk sensitive preferences. Insurance Math Econom 73:82\u201393","journal-title":"Insurance Math Econom"},{"key":"857_CR15","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1016\/j.jet.2017.11.005","volume":"173","author":"N B\u00e4uerle","year":"2018","unstructured":"B\u00e4uerle N, Ja\u015bkiewicz A (2018) Stochastic optimal growth model with risk sensitive preferences. J Econ Theory 173:181\u2013200","journal-title":"J Econ Theory"},{"key":"857_CR16","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1007\/s00186-008-0267-0","volume":"70","author":"N B\u00e4uerle","year":"2009","unstructured":"B\u00e4uerle N, Mundt A (2009) Dynamic mean-risk optimization in a binomial model. Math Methods Oper Res 70:219\u2013239","journal-title":"Math Methods Oper Res"},{"key":"857_CR17","doi-asserted-by":"publisher","first-page":"361","DOI":"10.1007\/s00186-011-0367-0","volume":"74","author":"N B\u00e4uerle","year":"2011","unstructured":"B\u00e4uerle N, Ott J (2011) Markov decision processes with average-value-at-risk criteria. Math Methods Oper Res 74:361\u2013379","journal-title":"Math Methods Oper Res"},{"key":"857_CR18","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-18324-9","volume-title":"Markov decision processes with applications to finance","author":"N B\u00e4uerle","year":"2011","unstructured":"B\u00e4uerle N, Rieder U (2011) Markov decision processes with applications to finance. Springer, Berlin"},{"issue":"1","key":"857_CR19","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1287\/moor.2013.0601","volume":"39","author":"N B\u00e4uerle","year":"2014","unstructured":"B\u00e4uerle N, Rieder U (2014) More risk-sensitive Markov decision processes. Math Oper Res 39(1):105\u2013120","journal-title":"Math Oper Res"},{"key":"857_CR20","unstructured":"B\u00e4uerle N, Rieder U (2015) Partially observable risk-sensitive stopping problems in discrete time. In: Piunovskiy AB (ed) Modern trends of controlled stochastic processes: theory and Applications, vol II. Luniver Press, pp 12\u201331"},{"issue":"4","key":"857_CR21","doi-asserted-by":"publisher","first-page":"1180","DOI":"10.1287\/moor.2016.0844","volume":"42","author":"N B\u00e4uerle","year":"2017","unstructured":"B\u00e4uerle N, Rieder U (2017) Partially observable risk-sensitive Markov decision processes. Math Oper Res 42(4):1180\u20131196","journal-title":"Math Oper Res"},{"issue":"12","key":"857_CR22","doi-asserted-by":"publisher","first-page":"3299","DOI":"10.1109\/TAC.2015.2416926","volume":"60","author":"GK Befekadu","year":"2015","unstructured":"Befekadu GK, Gupta V, Antsaklis PJ (2015) Risk-sensitive control under Markov modulated denial-of-service (DoS) attack strategies. IEEE Trans Autom Control 60(12):3299\u20133304","journal-title":"IEEE Trans Autom Control"},{"issue":"3","key":"857_CR23","doi-asserted-by":"publisher","first-page":"449","DOI":"10.1111\/j.1467-9965.2007.00311.x","volume":"17","author":"A Ben-Tal","year":"2007","unstructured":"Ben-Tal A, Teboulle M (2007) An old-new concept of convex risk measures: the optimized certainty equivalent. Math Financ 17(3):449\u2013476","journal-title":"Math Financ"},{"key":"857_CR24","doi-asserted-by":"publisher","first-page":"23","DOI":"10.2307\/1909829","volume":"22","author":"D Bernoulli","year":"1954","unstructured":"Bernoulli D (1954) Exposition of a new theory on the measurement of risk. Econometrica 22:23\u201336","journal-title":"Econometrica"},{"key":"857_CR25","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1007\/s001860050094","volume":"50","author":"T Bielecki","year":"1999","unstructured":"Bielecki T, Hern\u00e1ndez-Hern\u00e1ndez D, Pliska SR (1999) Risk sensitive control of finite state Markov chains in discrete time, with applications to portfolio management. Math Methods Oper Res 50:167\u2013188","journal-title":"Math Methods Oper Res"},{"key":"857_CR26","doi-asserted-by":"crossref","unstructured":"Bielecki T, Hernandez-Hernandez D, Pliska SR (1999b) Value iteration for controlled Markov chains with risk sensitive cost criterion. In: Proceedings of the 38th IEEE conference on decision and control. IEEE, pp 126\u2013130","DOI":"10.1109\/CDC.1999.832762"},{"key":"857_CR27","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1016\/j.arcontrol.2023.03.001","volume":"55","author":"A Biswas","year":"2023","unstructured":"Biswas A, Borkar VS (2023) Ergodic risk-sensitive control\u2014a survey. Annu Rev Control 55:118\u2013141","journal-title":"Annu Rev Control"},{"key":"857_CR28","first-page":"26","volume":"28","author":"A Biswas","year":"2022","unstructured":"Biswas A, Pradhan S (2022) Ergodic risk-sensitive control of Markov processes on countable state space revisited. ESAIM: Control Optim Cal Variat 28:26","journal-title":"ESAIM: Control Optim Cal Variat"},{"key":"857_CR29","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1016\/j.jet.2017.10.008","volume":"173","author":"G Bloise","year":"2018","unstructured":"Bloise G, Vailakis Y (2018) Convex dynamic programming with (bounded) recursive utility. J Econ Theory 173:118\u2013141","journal-title":"J Econ Theory"},{"key":"857_CR30","doi-asserted-by":"crossref","unstructured":"Bloise G, Le\u00a0Van C, Vailakis Y (2021) Do not blame Bellman: It is Koopmans\u2019 fault. SSRN 3943709","DOI":"10.2139\/ssrn.3943709"},{"issue":"3","key":"857_CR31","doi-asserted-by":"publisher","first-page":"1386","DOI":"10.1287\/mnsc.2017.2959","volume":"65","author":"A Bommier","year":"2019","unstructured":"Bommier A, Le Grand F (2019) Risk aversion and precautionary savings in dynamic settings. Manage Sci 65(3):1386\u20131397","journal-title":"Manage Sci"},{"issue":"5","key":"857_CR32","doi-asserted-by":"publisher","first-page":"339","DOI":"10.1016\/S0167-6911(01)00152-9","volume":"44","author":"VS Borkar","year":"2001","unstructured":"Borkar VS (2001) A sensitivity formula for risk-sensitive cost and the actor-critic algorithm. Syst Control Lett 44(5):339\u2013346","journal-title":"Syst Control Lett"},{"issue":"2","key":"857_CR33","doi-asserted-by":"publisher","first-page":"294","DOI":"10.1287\/moor.27.2.294.324","volume":"27","author":"VS Borkar","year":"2002","unstructured":"Borkar VS (2002) Q-learning for risk-sensitive control. Math Oper Res 27(2):294\u2013311","journal-title":"Math Oper Res"},{"key":"857_CR34","doi-asserted-by":"crossref","unstructured":"Borkar VS (2017) Linear and dynamic programming approaches to degenerate risk-sensitive reward processes. In: 56th Annual IEEE conference on decision and control. IEEE, pp 3714\u20133718","DOI":"10.1109\/CDC.2017.8264204"},{"issue":"1","key":"857_CR35","doi-asserted-by":"publisher","first-page":"192","DOI":"10.1287\/moor.27.1.192.334","volume":"27","author":"VS Borkar","year":"2002","unstructured":"Borkar VS, Meyn SP (2002) Risk-sensitive optimal control for Markov decision processes with monotone cost. Math Oper Res 27(1):192\u2013209","journal-title":"Math Oper Res"},{"issue":"3","key":"857_CR36","doi-asserted-by":"publisher","first-page":"603","DOI":"10.1287\/opre.40.3.603","volume":"40","author":"M Bouakiz","year":"1992","unstructured":"Bouakiz M, Sobel MJ (1992) Inventory control with an exponential utility criterion. Oper Res 40(3):603\u2013608","journal-title":"Oper Res"},{"key":"857_CR37","doi-asserted-by":"crossref","unstructured":"Brau-Rojas A, Cavazos-Cadena R, Fern\u00e1ndez-Gaucherand E (1998) Controlled Markov chains with risk-sensitive criteria: some (counter) examples. In: Proceedings of the 37th IEEE conference on decision and control. IEEE, pp 1853\u20131858","DOI":"10.1109\/CDC.1998.758575"},{"key":"857_CR38","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3389\/fnhum.2011.00001","volume":"5","author":"DA Braun","year":"2011","unstructured":"Braun DA, Nagengast AJ, Wolpert DM (2011) Risk-sensitivity in sensorimotor control. Front Hum Neurosci 5:1","journal-title":"Front Hum Neurosci"},{"issue":"3","key":"857_CR39","doi-asserted-by":"publisher","first-page":"1094","DOI":"10.1016\/j.ejor.2021.08.035","volume":"299","author":"S Bushaj","year":"2022","unstructured":"Bushaj S, B\u00fcy\u00fcktahtak\u0131n \u0130E, Haight RG (2022) Risk-averse multi-stage stochastic optimization for surveillance and operations planning of a forest insect infestation. Eur J Oper Res 299(3):1094\u20131110","journal-title":"Eur J Oper Res"},{"issue":"1","key":"857_CR40","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1007\/s00186-009-0285-6","volume":"71","author":"R Cavazos-Cadena","year":"2010","unstructured":"Cavazos-Cadena R (2010) Optimality equations and inequalities in a class of risk-sensitive average cost Markov decision chains. Math Methods Oper Res 71(1):47\u201384","journal-title":"Math Methods Oper Res"},{"issue":"3","key":"857_CR41","doi-asserted-by":"publisher","first-page":"1025","DOI":"10.1287\/moor.2017.0893","volume":"43","author":"R Cavazos-Cadena","year":"2018","unstructured":"Cavazos-Cadena R (2018) Characterization of the optimal risk-sensitive average cost in denumerable Markov decision chains. Math Oper Res 43(3):1025\u20131050","journal-title":"Math Oper Res"},{"issue":"2","key":"857_CR42","doi-asserted-by":"publisher","first-page":"1345","DOI":"10.1016\/j.jmaa.2017.01.084","volume":"450","author":"R Cavazos-Cadena","year":"2017","unstructured":"Cavazos-Cadena R, Cruz-Su\u00e1rez D (2017) Discounted approximations to the risk-sensitive average cost in finite Markov chains. J Math Anal Appl 450(2):1345\u20131362","journal-title":"J Math Anal Appl"},{"issue":"10","key":"857_CR43","doi-asserted-by":"publisher","first-page":"1800","DOI":"10.1109\/TAC.2000.880971","volume":"45","author":"R Cavazos-Cadena","year":"2000","unstructured":"Cavazos-Cadena R, Fern\u00e1ndez-Gaucherand E (2000) The vanishing discount approach in Markov chains with risk-sensitive criteria. IEEE Trans Autom Control 45(10):1800\u20131816","journal-title":"IEEE Trans Autom Control"},{"key":"857_CR44","doi-asserted-by":"publisher","first-page":"473","DOI":"10.1007\/s001860200229","volume":"56","author":"R Cavazos-Cadena","year":"2002","unstructured":"Cavazos-Cadena R, Hern\u00e1ndez-Hern\u00e1ndez D (2002) Solution to the risk-sensitive average optimality equation in communicating Markov decision chains with finite state space: An alternative approach. Math Methods Oper Res 56:473\u2013479","journal-title":"Math Methods Oper Res"},{"issue":"1A","key":"857_CR45","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1214\/105051604000000585","volume":"15","author":"R Cavazos-Cadena","year":"2005","unstructured":"Cavazos-Cadena R, Hern\u00e1ndez-Hern\u00e1ndez D (2005) A characterization of the optimal risk-sensitive average cost in finite controlled Markov chains. Ann Appl Probab 15(1A):175\u2013212","journal-title":"Ann Appl Probab"},{"issue":"4","key":"857_CR46","doi-asserted-by":"publisher","first-page":"254","DOI":"10.1016\/j.sysconle.2008.11.001","volume":"58","author":"R Cavazos-Cadena","year":"2009","unstructured":"Cavazos-Cadena R, Hern\u00e1ndez-Hern\u00e1ndez D (2009) Necessary and sufficient conditions for a solution to the risk-sensitive Poisson equation on a finite state space. Syst Control Lett 58(4):254\u2013258","journal-title":"Syst Control Lett"},{"issue":"1","key":"857_CR47","doi-asserted-by":"publisher","first-page":"133","DOI":"10.1287\/moor.1100.0476","volume":"36","author":"R Cavazos-Cadena","year":"2011","unstructured":"Cavazos-Cadena R, Hern\u00e1ndez-Hern\u00e1ndez D (2011) Discounted approximations for risk-sensitive average criteria in Markov decision chains with finite state space. Math Oper Res 36(1):133\u2013146","journal-title":"Math Oper Res"},{"issue":"1","key":"857_CR48","doi-asserted-by":"publisher","first-page":"224","DOI":"10.1287\/moor.2015.0723","volume":"41","author":"R Cavazos-Cadena","year":"2016","unstructured":"Cavazos-Cadena R, Hern\u00e1ndez-Hern\u00e1ndez D (2016) A characterization of the optimal certainty equivalent of the average cost via the Arrow-Pratt sensitivity function. Math Oper Res 41(1):224\u2013235","journal-title":"Math Oper Res"},{"issue":"4","key":"857_CR49","doi-asserted-by":"publisher","first-page":"905","DOI":"10.1239\/jap\/1134587805","volume":"42","author":"R Cavazos-Cadena","year":"2005","unstructured":"Cavazos-Cadena R, Montes-De-Oca R (2005) Nonstationary value iteration in controlled Markov chains with risk-sensitive average criterion. J Appl Probab 42(4):905\u2013918","journal-title":"J Appl Probab"},{"issue":"4","key":"857_CR50","doi-asserted-by":"publisher","first-page":"752","DOI":"10.1287\/moor.28.4.752.20515","volume":"28","author":"R Cavazos-Cadena","year":"2003","unstructured":"Cavazos-Cadena R, Montes-de Oca R (2003) The value iteration algorithm in risk-sensitive average Markov decision chains with finite state space. Math Oper Res 28(4):752\u2013776","journal-title":"Math Oper Res"},{"issue":"2","key":"857_CR51","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1007\/s00245-009-9080-2","volume":"61","author":"R Cavazos-Cadena","year":"2010","unstructured":"Cavazos-Cadena R, Salem-Silva F (2010) The discounted method and equivalence of average criteria for risk-sensitive Markov decision processes on borel spaces. Appl Math Optim 61(2):167\u2013190","journal-title":"Appl Math Optim"},{"issue":"6","key":"857_CR52","doi-asserted-by":"publisher","first-page":"3935","DOI":"10.1137\/13093902X","volume":"52","author":"O \u00c7avu\u015f","year":"2014","unstructured":"\u00c7avu\u015f O, Ruszczy\u0144ski A (2014) Risk-averse control of undicounted transient Markov models. SIAM J Control Optim 52(6):3935\u20133966","journal-title":"SIAM J Control Optim"},{"key":"857_CR53","doi-asserted-by":"publisher","first-page":"1525","DOI":"10.1109\/LCSYS.2021.3114126","volume":"6","author":"MP Chapman","year":"2021","unstructured":"Chapman MP, Smith KM (2021) Classical risk-averse control for a finite-horizon Borel model. IEEE Control Syst Lett 6:1525\u20131530","journal-title":"IEEE Control Syst Lett"},{"issue":"6","key":"857_CR54","doi-asserted-by":"publisher","first-page":"3720","DOI":"10.1109\/TAC.2022.3195381","volume":"68","author":"MP Chapman","year":"2023","unstructured":"Chapman MP, Fau\u00df M, Smith KM (2023) On optimizing the conditional value-at-risk of a maximum cost for risk-averse safety analysis. IEEE Trans Autom Control 68(6):3720\u20133727","journal-title":"IEEE Trans Autom Control"},{"issue":"1","key":"857_CR55","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1137\/22M1476757","volume":"61","author":"X Chen","year":"2023","unstructured":"Chen X, Wei Q (2023) Risk-sensitive average optimality for discrete-time Markov decision processes. SIAM J Control Optim 61(1):72\u2013104","journal-title":"SIAM J Control Optim"},{"key":"857_CR56","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1016\/j.ejor.2011.04.005","volume":"214","author":"S Choi","year":"2011","unstructured":"Choi S, Ruszczy\u0144ski A (2011) A multi-product risk-averse newsvendor with exponential utility function. Eur J Oper Res 214:78\u201384","journal-title":"Eur J Oper Res"},{"key":"857_CR57","unstructured":"Chow Y, Tamar A, Mannor S, et\u00a0al (2015) Risk-sensitive and robust decision-making: a CVaR optimization approach. In: Proceedings of the 28th international conference on neural information processing systems, ACMDL, pp 1522\u20131530"},{"issue":"11","key":"857_CR58","first-page":"2286","volume":"87","author":"S Chu","year":"2014","unstructured":"Chu S, Zhang Y (2014) Markov decision processes with iterated coherent risk measures. Int J Control 87(11):2286\u20132293","journal-title":"Int J Control"},{"issue":"1","key":"857_CR59","doi-asserted-by":"publisher","first-page":"49","DOI":"10.1137\/0325004","volume":"25","author":"KJ Chung","year":"1987","unstructured":"Chung KJ, Sobel MJ (1987) Discounted MDP\u2019s: distribution functions and exponential utility maximization. SIAM J Control Optim 25(1):49\u201362","journal-title":"SIAM J Control Optim"},{"key":"857_CR60","doi-asserted-by":"publisher","DOI":"10.1111\/mafi.12388","author":"A Coache","year":"2023","unstructured":"Coache A, Jaimungal S (2023) Reinforcement learning with dynamic convex risk measures. Math Financ. https:\/\/doi.org\/10.1111\/mafi.12388","journal-title":"Math Financ"},{"issue":"1","key":"857_CR61","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1239\/aap\/1035227995","volume":"30","author":"E Collins","year":"1998","unstructured":"Collins E, McNamara J (1998) Finite-horizon dynamic optimisation when the terminal reward is a concave functional of the distribution of the final state. Adv Appl Probab 30(1):122\u2013136","journal-title":"Adv Appl Probab"},{"issue":"2","key":"857_CR62","doi-asserted-by":"publisher","first-page":"301","DOI":"10.1016\/S0005-1098(98)00153-8","volume":"35","author":"SP Coraluppi","year":"1999","unstructured":"Coraluppi SP, Marcus SI (1999) Risk-sensitive and minimax control of discrete-time, finite-state Markov decision processes. Automatica 35(2):301\u2013309","journal-title":"Automatica"},{"key":"857_CR63","doi-asserted-by":"publisher","first-page":"303","DOI":"10.1007\/BF01211853","volume":"9","author":"P Dai Pra","year":"1996","unstructured":"Dai Pra P, Meneghini L, Runggaldier WJ (1996) Connections between Stochastic control and dynamic games. Math Control Signals Syst 9:303\u2013326","journal-title":"Math Control Signals Syst"},{"key":"857_CR64","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-5320-4","volume-title":"Large deviations techniques and applications","author":"A Dembo","year":"1998","unstructured":"Dembo A, Zeitouni O (1998) Large deviations techniques and applications. Springer, Berlin"},{"issue":"2","key":"857_CR65","doi-asserted-by":"publisher","first-page":"414","DOI":"10.1137\/S0363012904442616","volume":"45","author":"EV Denardo","year":"2006","unstructured":"Denardo EV, Rothblum UG (2006) A turnpike theorem for a risk-sensitive Markov decision process with stopping. SIAM J Control Optim 45(2):414\u2013431","journal-title":"SIAM J Control Optim"},{"issue":"2","key":"857_CR66","doi-asserted-by":"publisher","first-page":"374","DOI":"10.1287\/moor.1060.0240","volume":"32","author":"EV Denardo","year":"2007","unstructured":"Denardo EV, Park H, Rothblum UG (2007) Risk-sensitive and risk-neutral multiarmed bandits. Math Oper Res 32(2):374\u2013394","journal-title":"Math Oper Res"},{"issue":"1","key":"857_CR67","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1137\/S0363012997320614","volume":"38","author":"GB Di Masi","year":"1999","unstructured":"Di Masi GB, Stettner \u0141 (1999) Risk-sensitive control of discrete-time Markov processes with infinite horizon. SIAM J Control Optim 38(1):61\u201378","journal-title":"SIAM J Control Optim"},{"issue":"1","key":"857_CR68","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/S0167-6911(99)00118-8","volume":"40","author":"GB Di Masi","year":"2000","unstructured":"Di Masi GB, Stettner \u0141 (2000) Infinite horizon risk sensitive control of discrete time Markov processes with small risk. Syst Control Lett 40(1):15\u201320","journal-title":"Syst Control Lett"},{"issue":"1","key":"857_CR69","doi-asserted-by":"publisher","first-page":"231","DOI":"10.1137\/040618631","volume":"46","author":"GB Di Masi","year":"2007","unstructured":"Di Masi GB, Stettner \u0141 (2007) Infinite horizon risk sensitive control of discrete time Markov processes under minorization property. SIAM J Control Optim 46(1):231\u2013252","journal-title":"SIAM J Control Optim"},{"key":"857_CR70","unstructured":"Ding R, Feinberg EA (2022) Sequential optimization of CVaR. ArXiv preprint arXiv:2211.07288"},{"key":"857_CR71","unstructured":"Dowson O, Morton DP, Pagnoncelli BK (2020) Multistage stochastic programs with the entropic risk measure. Optim Online https:\/\/optimization-online.org\/?p=16662"},{"key":"857_CR72","doi-asserted-by":"publisher","DOI":"10.1007\/s10479-022-04977-w","author":"O Dowson","year":"2022","unstructured":"Dowson O, Morton DP, Pagnoncelli BK (2022) Incorporating convex risk measures into multistage stochastic programming algorithms. Ann Oper Res. https:\/\/doi.org\/10.1007\/s10479-022-04977-w","journal-title":"Ann Oper Res"},{"key":"857_CR73","doi-asserted-by":"publisher","first-page":"353","DOI":"10.2307\/2951600","volume":"1","author":"D Duffie","year":"1992","unstructured":"Duffie D, Epstein LG (1992) Stochastic differential utility. Econometrica J Econom Soc 1:353\u2013394","journal-title":"Econometrica J Econom Soc"},{"key":"857_CR74","doi-asserted-by":"publisher","first-page":"559","DOI":"10.1007\/s00291-014-0379-2","volume":"37","author":"J Dupa\u010dov\u00e1","year":"2015","unstructured":"Dupa\u010dov\u00e1 J, Kozm\u00edk V (2015) Structure of risk-averse multistage stochastic programs. OR Spectrum 37:559\u2013582","journal-title":"OR Spectrum"},{"issue":"4","key":"857_CR75","doi-asserted-by":"publisher","first-page":"937","DOI":"10.2307\/1913778","volume":"57","author":"LG Epstein","year":"1989","unstructured":"Epstein LG, Zin SE (1989) Substitution, risk aversion and the temporal behavior of consumption and asset returns: A theoretical framework. Econometrica 57(4):937\u2013969","journal-title":"Econometrica"},{"key":"857_CR76","first-page":"20436","volume":"34","author":"Y Fei","year":"2021","unstructured":"Fei Y, Yang Z, Chen Y et al (2021) Exponential Bellman equation and improved regret bounds for risk-sensitive reinforcement learning. Adv Neural Inf Process Syst 34:20436\u201320446","journal-title":"Adv Neural Inf Process Syst"},{"issue":"1","key":"857_CR77","doi-asserted-by":"publisher","first-page":"47","DOI":"10.1007\/s10898-016-0459-8","volume":"68","author":"Z Feinstein","year":"2017","unstructured":"Feinstein Z, Rudloff B (2017) A recursive algorithm for multivariate risk measures and a set-valued Bellman\u2019s principle. J Global Optim 68(1):47\u201369","journal-title":"J Global Optim"},{"issue":"10","key":"857_CR78","doi-asserted-by":"publisher","first-page":"1418","DOI":"10.1109\/9.633830","volume":"42","author":"E Fern\u00e1ndez-Gaucherand","year":"1997","unstructured":"Fern\u00e1ndez-Gaucherand E, Marcus SI (1997) Risk-sensitive optimal control of hidden Markov models: Structural results. IEEE Trans Autom Control 42(10):1418\u20131422","journal-title":"IEEE Trans Autom Control"},{"key":"857_CR79","volume-title":"Competitive Markov Decision Processes","author":"J Filar","year":"1997","unstructured":"Filar J, Koos V (1997) Competitive Markov Decision Processes. Springer, Berlin"},{"issue":"5","key":"857_CR80","doi-asserted-by":"publisher","first-page":"1790","DOI":"10.1137\/S0363012995291622","volume":"35","author":"WH Fleming","year":"1997","unstructured":"Fleming WH, Hern\u00e1ndez-Hern\u00e1ndez D (1997) Risk-sensitive control of finite state machines on an infinite horizon I. SIAM J Control Optim 35(5):1790\u20131810","journal-title":"SIAM J Control Optim"},{"key":"857_CR81","first-page":"355","volume":"1","author":"H F\u00f6llmer","year":"2010","unstructured":"F\u00f6llmer H, Schied A (2010) Convex and coherent risk measures. Encyclop Quant Financ 1:355\u2013363","journal-title":"Encyclop Quant Financ"},{"issue":"2","key":"857_CR82","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1016\/j.ejor.2017.05.033","volume":"263","author":"J G\u00f6nsch","year":"2017","unstructured":"G\u00f6nsch J (2017) A survey on risk-averse and robust revenue management. Eur J Oper Res 263(2):337\u2013348","journal-title":"Eur J Oper Res"},{"key":"857_CR83","doi-asserted-by":"publisher","first-page":"711","DOI":"10.1007\/s00291-018-0513-7","volume":"40","author":"J G\u00f6nsch","year":"2018","unstructured":"G\u00f6nsch J, Hassler M, Schur R (2018) Optimizing Conditional Value-at-Risk in dynamic pricing. OR Spectrum 40:711\u2013750","journal-title":"OR Spectrum"},{"key":"857_CR84","doi-asserted-by":"publisher","first-page":"102702","DOI":"10.1016\/j.jmateco.2022.102702","volume":"101","author":"A Goswami","year":"2022","unstructured":"Goswami A, Rana N, Siu TK (2022) Regime switching optimal growth model with risk sensitive preferences. J Math Econ 101:102702","journal-title":"J Math Econ"},{"issue":"4","key":"857_CR85","doi-asserted-by":"publisher","first-page":"2468","DOI":"10.1137\/140983136","volume":"26","author":"V Guigues","year":"2016","unstructured":"Guigues V (2016) Convergence analysis of sampling-based decomposition methods for risk-averse multistage stochastic convex programs. SIAM J Optim 26(4):2468\u20132494","journal-title":"SIAM J Optim"},{"issue":"4","key":"857_CR86","doi-asserted-by":"publisher","first-page":"1976","DOI":"10.1109\/TNET.2018.2856883","volume":"26","author":"X Guo","year":"2018","unstructured":"Guo X, Singh R, Kumar P et al (2018) A risk-sensitive approach for packet inter-delivery time optimization in networked cyber-physical systems. IEEE\/ACM Trans Networking 26(4):1976\u20131989","journal-title":"IEEE\/ACM Trans Networking"},{"issue":"3","key":"857_CR87","doi-asserted-by":"publisher","first-page":"437","DOI":"10.1111\/mafi.12382","volume":"33","author":"B Hambly","year":"2023","unstructured":"Hambly B, Xu R, Yang H (2023) Recent advances in reinforcement learning in finance. Math Financ 33(3):437\u2013503","journal-title":"Math Financ"},{"issue":"5","key":"857_CR88","doi-asserted-by":"publisher","first-page":"968","DOI":"10.1109\/9.384242","volume":"40","author":"LP Hansen","year":"1995","unstructured":"Hansen LP, Sargent TJ (1995) Discounted linear exponential quadratic Gaussian control. IEEE Trans Autom Control 40(5):968\u2013971","journal-title":"IEEE Trans Autom Control"},{"key":"857_CR89","unstructured":"Hau JL, Petrik M, Ghavamzadeh M (2023) Entropic risk optimization in discounted MDPs. In: International conference on artificial intelligence and statistics. PMLR, pp 47\u201376"},{"issue":"3","key":"857_CR90","doi-asserted-by":"publisher","first-page":"147","DOI":"10.1016\/S0167-6911(96)00051-5","volume":"29","author":"D Hern\u00e1ndez-Hern\u00e1ndez","year":"1996","unstructured":"Hern\u00e1ndez-Hern\u00e1ndez D, Marcus SI (1996) Risk sensitive control of Markov processes in countable state space. Syst Control Lett 29(3):147\u2013155 (Corrigendum in System and Control Letters (1998) 34:105\u2013106)","journal-title":"Syst Control Lett"},{"key":"857_CR91","doi-asserted-by":"publisher","first-page":"273","DOI":"10.1007\/s002459900126","volume":"40","author":"D Hern\u00e1ndez-Hern\u00e1ndez","year":"1999","unstructured":"Hern\u00e1ndez-Hern\u00e1ndez D, Marcus SI (1999) Existence of risk-sensitive optimal stationary policies for controlled Markov processes. Appl Math Optim 40:273\u2013285","journal-title":"Appl Math Optim"},{"key":"857_CR92","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-0729-0","volume-title":"Discrete-time Markov control processes, basic optimality criteria","author":"O Hern\u00e1ndez-Lerma","year":"1996","unstructured":"Hern\u00e1ndez-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes, basic optimality criteria. Springer, Berlin"},{"issue":"1","key":"857_CR93","doi-asserted-by":"publisher","first-page":"188","DOI":"10.1016\/j.ejor.2015.05.048","volume":"249","author":"T Homem-de-Mello","year":"2016","unstructured":"Homem-de-Mello T, Pagnoncelli BK (2016) Risk aversion in multistage stochastic programming: a modeling and algorithmic perspective. Eur J Oper Res 249(1):188\u2013199","journal-title":"Eur J Oper Res"},{"issue":"7","key":"857_CR94","doi-asserted-by":"publisher","first-page":"356","DOI":"10.1287\/mnsc.18.7.356","volume":"18","author":"RA Howard","year":"1972","unstructured":"Howard RA, Matheson JE (1972) Risk-sensitive Markov decision processes. Manage Sci 18(7):356\u2013369","journal-title":"Manage Sci"},{"key":"857_CR95","unstructured":"Huang A, Leqi L, Lipton ZC, et\u00a0al (2021) On the convergence and optimality of policy gradient for Markov coherent risk. arXiv preprint arXiv:2103.02827"},{"issue":"2","key":"857_CR96","doi-asserted-by":"publisher","first-page":"128026","DOI":"10.1016\/j.jmaa.2023.128026","volume":"533","author":"T Huang","year":"2024","unstructured":"Huang T, Chen J (2024) Markov decision processes under risk sensitivity: a discount vanishing approach. J Math Anal Appl 533(2):128026","journal-title":"J Math Anal Appl"},{"issue":"3","key":"857_CR97","doi-asserted-by":"publisher","first-page":"655","DOI":"10.1287\/moor.2014.0689","volume":"40","author":"DA Iancu","year":"2015","unstructured":"Iancu DA, Petrik M, Subramanian D (2015) Tight approximations of dynamic risk measures. Math Oper Res 40(3):655\u2013682","journal-title":"Math Oper Res"},{"issue":"1","key":"857_CR98","doi-asserted-by":"publisher","first-page":"193","DOI":"10.1006\/jmaa.1998.6192","volume":"230","author":"S Iwamoto","year":"1999","unstructured":"Iwamoto S (1999) Conditional decision processes with recursive function. J Math Anal Appl 230(1):193\u2013210","journal-title":"J Math Anal Appl"},{"issue":"1","key":"857_CR99","doi-asserted-by":"publisher","first-page":"73","DOI":"10.1016\/j.jmaa.2003.11.045","volume":"292","author":"S Iwamoto","year":"2004","unstructured":"Iwamoto S (2004) Stochastic optimization of forward recursive functions. J Math Anal Appl 292(1):73\u201383","journal-title":"J Math Anal Appl"},{"issue":"2","key":"857_CR100","doi-asserted-by":"publisher","first-page":"124","DOI":"10.1109\/TAC.1973.1100265","volume":"18","author":"D Jacobson","year":"1973","unstructured":"Jacobson D (1973) Optimal stochastic linear systems with exponential performance criteria and their relation to deterministic differential games. IEEE Trans Autom Control 18(2):124\u2013131","journal-title":"IEEE Trans Autom Control"},{"issue":"4","key":"857_CR101","doi-asserted-by":"publisher","first-page":"780","DOI":"10.1109\/9.286253","volume":"39","author":"MR James","year":"1994","unstructured":"James MR, Baras JS, Elliott RJ (1994) Risk-sensitive control and dynamic games for partially observed discrete-time nonlinear systems. IEEE Trans Autom Control 39(4):780\u2013792","journal-title":"IEEE Trans Autom Control"},{"issue":"1","key":"857_CR102","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1287\/mnsc.23.1.43","volume":"23","author":"SC Jaquette","year":"1976","unstructured":"Jaquette SC (1976) A utility criterion for Markov decision processes. Manag Sci 23(1):43\u201349","journal-title":"Manag Sci"},{"issue":"2","key":"857_CR103","doi-asserted-by":"publisher","first-page":"654","DOI":"10.1214\/105051606000000790","volume":"17","author":"A Ja\u015bkiewicz","year":"2007","unstructured":"Ja\u015bkiewicz A (2007) Average optimality for risk-sensitive control with general state space. Ann Appl Probab 17(2):654\u2013675","journal-title":"Ann Appl Probab"},{"issue":"11\u201312","key":"857_CR104","doi-asserted-by":"publisher","first-page":"663","DOI":"10.1016\/j.sysconle.2007.06.006","volume":"56","author":"A Ja\u015bkiewicz","year":"2007","unstructured":"Ja\u015bkiewicz A (2007) A note on risk-sensitive control of invariant models. Syst Control Lett 56(11\u201312):663\u2013668","journal-title":"Syst Control Lett"},{"issue":"5","key":"857_CR105","doi-asserted-by":"publisher","first-page":"531","DOI":"10.1016\/j.orl.2008.03.003","volume":"36","author":"A Ja\u015bkiewicz","year":"2008","unstructured":"Ja\u015bkiewicz A (2008) A note on negative dynamic programming for risk-sensitive control. Oper Res Lett 36(5):531\u2013534","journal-title":"Oper Res Lett"},{"issue":"2","key":"857_CR106","doi-asserted-by":"publisher","first-page":"1337","DOI":"10.1016\/j.jmaa.2014.06.028","volume":"420","author":"A Ja\u015bkiewicz","year":"2014","unstructured":"Ja\u015bkiewicz A, Nowak AS (2014) Robust Markov control processes. J Math Anal Appl 420(2):1337\u20131353","journal-title":"J Math Anal Appl"},{"key":"857_CR107","unstructured":"Jiang DR, Powell WB (2016) Practicality of nested risk measures for dynamic electric vehicle charging. ArXiv preprint arXiv:1605.02848"},{"issue":"2","key":"857_CR108","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1016\/j.camwa.2005.11.013","volume":"51","author":"Y Kadota","year":"2006","unstructured":"Kadota Y, Kurano M, Yasuda M (2006) Discounted Markov decision processes with utility constraints. Comput Math Appl 51(2):279\u2013284","journal-title":"Comput Math Appl"},{"key":"857_CR109","doi-asserted-by":"crossref","unstructured":"Koenig S, Simmons RG (1994) Risk-sensitive planning with probabilistic decision graphs. In: Principles of knowledge representation and reasoning. Elsevier, pp 363\u2013373","DOI":"10.1016\/B978-1-4832-1452-8.50129-9"},{"key":"857_CR110","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1007\/s10107-014-0787-8","volume":"152","author":"V Kozm\u00edk","year":"2015","unstructured":"Kozm\u00edk V, Morton DP (2015) Evaluating policies in risk-averse multi-stage stochastic programming. Math Program 152:275\u2013300","journal-title":"Math Program"},{"key":"857_CR111","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1007\/s00780-012-0184-1","volume":"17","author":"H Kraft","year":"2013","unstructured":"Kraft H, Seifried FT, Steffensen M (2013) Consumption-portfolio optimization with recursive utility in incomplete markets. Finance Stochast 17:161\u2013196","journal-title":"Finance Stochast"},{"issue":"1","key":"857_CR112","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1287\/moor.2.1.45","volume":"2","author":"DM Kreps","year":"1977","unstructured":"Kreps DM (1977) Decision problems with expected utility critera, I: upper and lower convergent utility. Math Oper Res 2(1):45\u201353","journal-title":"Math Oper Res"},{"issue":"3","key":"857_CR113","doi-asserted-by":"publisher","first-page":"266","DOI":"10.1287\/moor.2.3.266","volume":"2","author":"DM Kreps","year":"1977","unstructured":"Kreps DM (1977) Decision problems with expected utility criteria, II: stationarity. Math Oper Res 2(3):266\u2013274","journal-title":"Math Oper Res"},{"issue":"1","key":"857_CR114","doi-asserted-by":"publisher","first-page":"185","DOI":"10.2307\/1913656","volume":"46","author":"DM Kreps","year":"1978","unstructured":"Kreps DM, Porteus EL (1978) Temporal resolution of uncertainty and dynamic choice theory. Econometrica 46(1):185\u2013200","journal-title":"Econometrica"},{"key":"857_CR115","unstructured":"Le\u00a0Tallec Y (2007) Robust, risk-sensitive, and data-driven control of Markov decision processes. Phd thesis, Massachusetts Institute of Technology, available at https:\/\/dspace.mit.edu\/handle\/1721.1\/38598"},{"key":"857_CR116","volume-title":"Investement Science","author":"DG Luenberger","year":"2014","unstructured":"Luenberger DG (2014) Investement Science. Oxford University Press, Oxford"},{"issue":"4","key":"857_CR117","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1257\/mac.2.4.281","volume":"2","author":"Y Luo","year":"2010","unstructured":"Luo Y, Young ER (2010) Risk-sensitive consumption and savings under rational inattention. Am Econ J Macroecon 2(4):281\u2013325","journal-title":"Am Econ J Macroecon"},{"key":"857_CR118","doi-asserted-by":"publisher","first-page":"126","DOI":"10.1016\/j.ijepes.2015.02.025","volume":"72","author":"MEP Maceira","year":"2015","unstructured":"Maceira MEP, Marzano L, Penna DDJ et al (2015) Application of CVaR risk aversion approach in the expansion and operation planning and for setting the spot price in the Brazilian hydrothermal interconnected system. Int J Electr Power Energy Syst 72:126\u2013135","journal-title":"Int J Electr Power Energy Syst"},{"key":"857_CR119","unstructured":"Mannor S, Tsitsiklis J (2011) Mean-variance optimization in Markov decision processes. In: Proceedings of the 28th international conference on machine learning. ICML, pp 177\u2013184"},{"issue":"5","key":"857_CR120","doi-asserted-by":"publisher","first-page":"1776","DOI":"10.1016\/j.jet.2010.02.005","volume":"145","author":"M Marinacci","year":"2010","unstructured":"Marinacci M, Montrucchio L (2010) Unique solutions for stochastic recursive utilities. J Econ Theory 145(5):1776\u20131804","journal-title":"J Econ Theory"},{"issue":"1","key":"857_CR121","first-page":"77","volume":"7","author":"HM Markowitz","year":"1952","unstructured":"Markowitz HM (1952) Portfolio selection. J Financ 7(1):77\u201391","journal-title":"J Financ"},{"issue":"2","key":"857_CR122","doi-asserted-by":"publisher","first-page":"625","DOI":"10.1017\/apr.2021.44","volume":"54","author":"R Martyr","year":"2022","unstructured":"Martyr R, Moriarty J, Perninge M (2022) Discrete-time risk-aware optimal switching with non-adapted costs. Adv Appl Probab 54(2):625\u2013655","journal-title":"Adv Appl Probab"},{"issue":"15","key":"857_CR123","doi-asserted-by":"publisher","first-page":"105","DOI":"10.1016\/j.ifacol.2022.07.616","volume":"55","author":"M Mazouchi","year":"2022","unstructured":"Mazouchi M, Nageshrao S, Modares H (2022) Automating vehicles by risk-averse preview-based Q-learning algorithm. IFAC-PapersOnLine 55(15):105\u2013110","journal-title":"IFAC-PapersOnLine"},{"key":"857_CR124","unstructured":"Medina JR, Lee D, Hirche S (2012) Risk-sensitive optimal feedback control for haptic assistance. In: IEEE international conference on robotics and automation. IEEE, pp 1025\u20131031"},{"key":"857_CR125","doi-asserted-by":"crossref","unstructured":"Miao J (2020) Economic Dynamics in Discrete Time. MIT press","DOI":"10.1155\/2020\/4718496"},{"key":"857_CR126","first-page":"3131","volume":"4","author":"T Moldovan","year":"2012","unstructured":"Moldovan T, Abbeel P (2012) Risk aversion in Markov decision processes via near-optimal Chernoff bounds. Adv Neural Inf Process Syst 4:3131\u20133139","journal-title":"Adv Neural Inf Process Syst"},{"key":"857_CR127","unstructured":"Osogami T (2011) Iterated risk measures for risk-sensitive Markov decision processes with discounted cost. In: Proceedings of the 27th conference on uncertainty in artificial intelligence, pp 573\u2013580"},{"key":"857_CR128","unstructured":"Ott J (2010) A Markov decision model for a surveillance application and risk-sensitive Markov decision processes. PhD Thesis, Karlsruhe Institute of Technology. https:\/\/publikationen.bibliothek.kit.edu\/1000020835"},{"issue":"4","key":"857_CR129","doi-asserted-by":"publisher","first-page":"391","DOI":"10.1016\/0304-4068(95)00737-7","volume":"25","author":"H Ozaki","year":"1996","unstructured":"Ozaki H, Streufert PA (1996) Dynamic programming for non-additive stochastic objectives. J Math Econ 25(4):391\u2013442","journal-title":"J Math Econ"},{"issue":"2","key":"857_CR130","doi-asserted-by":"publisher","first-page":"695","DOI":"10.1016\/j.jbankfin.2005.04.006","volume":"30","author":"GC Pflug","year":"2006","unstructured":"Pflug GC (2006) A value-of-information approach to measuring risk in multi-period economic activity. J Bank Finance 30(2):695\u2013715","journal-title":"J Bank Finance"},{"issue":"1","key":"857_CR131","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.ejor.2015.02.033","volume":"249","author":"GC Pflug","year":"2016","unstructured":"Pflug GC, Pichler A (2016) Time-inconsistent multistage stochastic programs: Martingale bounds. Eur J Oper Res 249(1):155\u2013163","journal-title":"Eur J Oper Res"},{"key":"857_CR132","doi-asserted-by":"publisher","first-page":"161","DOI":"10.1007\/s10589-005-2058-3","volume":"32","author":"GC Pflug","year":"2005","unstructured":"Pflug GC, Ruszczy\u0144ski (2005) Measuring risk for income streams. Comput Optim Appl 32:161\u2013178","journal-title":"Comput Optim Appl"},{"issue":"4","key":"857_CR133","doi-asserted-by":"publisher","first-page":"957","DOI":"10.1287\/opre.2013.1175","volume":"61","author":"A Philpott","year":"2013","unstructured":"Philpott A, de Matos V, Finardi E (2013) On solving multistage stochastic programs with coherent risk measures. Oper Res 61(4):957\u2013970","journal-title":"Oper Res"},{"issue":"4","key":"857_CR134","doi-asserted-by":"publisher","first-page":"1287","DOI":"10.1111\/mafi.12406","volume":"33","author":"M Pitera","year":"2023","unstructured":"Pitera M, Stettner \u0141 (2023) Discrete-time risk sensitive portfolio optimization with proportional transaction costs. Math Financ 33(4):1287\u20131313","journal-title":"Math Financ"},{"key":"857_CR135","volume-title":"Examples in Markov decision processes","author":"AB Piunovskiy","year":"2013","unstructured":"Piunovskiy AB (2013) Examples in Markov decision processes. Imperial College Press, London"},{"key":"857_CR136","doi-asserted-by":"publisher","DOI":"10.1002\/9781119815068","volume-title":"Reinforcement learning and Stochastic optimization: a unified framework for sequential decisions","author":"WB Powell","year":"2022","unstructured":"Powell WB (2022) Reinforcement learning and Stochastic optimization: a unified framework for sequential decisions. Wiley, Boca Raton"},{"key":"857_CR137","doi-asserted-by":"publisher","first-page":"122","DOI":"10.2307\/1913738","volume":"32","author":"JW Pratt","year":"1964","unstructured":"Pratt JW (1964) Risk aversion in the small and in the large. Econometrica 32:122\u2013136","journal-title":"Econometrica"},{"key":"857_CR138","volume-title":"Markov decision processes: discrete stochastic dynamic programming","author":"ML Puterman","year":"2014","unstructured":"Puterman ML (2014) Markov decision processes: discrete stochastic dynamic programming. Wiley, Boca Raton"},{"key":"857_CR139","unstructured":"Ren G, Stachurski J (2018) Dynamic programming with recursive preferences: optimality and applications. ArXiv preprint arXiv:1812.05748"},{"issue":"1","key":"857_CR140","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1287\/moor.9.1.6","volume":"9","author":"UG Rothblum","year":"1984","unstructured":"Rothblum UG (1984) Multiplicative Markov decision chains. Math Oper Res 9(1):6\u201324","journal-title":"Math Oper Res"},{"key":"857_CR141","volume-title":"Real analysis","author":"HL Royden","year":"1988","unstructured":"Royden HL (1988) Real analysis. Prentice Hall, New Jersey"},{"issue":"3","key":"857_CR142","doi-asserted-by":"publisher","first-page":"743","DOI":"10.1016\/j.ejor.2013.11.037","volume":"234","author":"B Rudloff","year":"2014","unstructured":"Rudloff B, Street A, Vallad\u00e3o DM (2014) Time consistency and risk averse dynamic decision models: Definition, interpretation and practical consequences. Eur J Oper Res 234(3):743\u2013750","journal-title":"Eur J Oper Res"},{"key":"857_CR143","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1007\/s10107-010-0393-3","volume":"125","author":"A Ruszczy\u0144ski","year":"2010","unstructured":"Ruszczy\u0144ski A (2010) Risk-averse dynamic programming for Markov decision processes. Math Program 125:235\u2013261","journal-title":"Math Program"},{"key":"857_CR144","unstructured":"Sargent T, Stachurski J (2023) Dynamic Programming, Vol. I: Foundations. https:\/\/dp.quantecon.org"},{"key":"857_CR145","doi-asserted-by":"publisher","first-page":"179","DOI":"10.1007\/BF00532612","volume":"32","author":"M Sch\u00e4l","year":"1975","unstructured":"Sch\u00e4l M (1975) Conditions for optimality in dynamic programming and for the limit of n-stage optimal policies to be optimal. Zeitschrift f\u00fcr Wahrscheinlichkeitstheorie und Verwandte Gebiete 32:179\u2013196","journal-title":"Zeitschrift f\u00fcr Wahrscheinlichkeitstheorie und Verwandte Gebiete"},{"issue":"3","key":"857_CR146","doi-asserted-by":"publisher","first-page":"366","DOI":"10.1287\/moor.8.3.366","volume":"8","author":"M Sch\u00e4l","year":"1983","unstructured":"Sch\u00e4l M (1983) Stationary policies in dynamic programming models under compactness assumptions. Math Oper Res 8(3):366\u2013372","journal-title":"Math Oper Res"},{"key":"857_CR147","doi-asserted-by":"publisher","first-page":"451","DOI":"10.1057\/rpm.2015.20","volume":"14","author":"R Schlosser","year":"2015","unstructured":"Schlosser R (2015) A stochastic dynamic pricing and advertising model under risk aversion. J Revenue Pricing Manag 14:451\u2013468","journal-title":"J Revenue Pricing Manag"},{"key":"857_CR148","doi-asserted-by":"publisher","first-page":"153","DOI":"10.1057\/rpm.2015.28","volume":"15","author":"R Schlosser","year":"2016","unstructured":"Schlosser R (2016) Stochastic dynamic multi-product pricing with dynamic advertising and adoption effects. J Revenue Pric Manag 15:153\u2013169","journal-title":"J Revenue Pric Manag"},{"key":"857_CR149","doi-asserted-by":"publisher","first-page":"104997","DOI":"10.1016\/j.cor.2020.104997","volume":"123","author":"R Schlosser","year":"2020","unstructured":"Schlosser R (2020) Risk-sensitive control of Markov decision processes: A moment-based approach with target distributions. Comput Oper Res 123:104997","journal-title":"Comput Oper Res"},{"issue":"2","key":"857_CR150","doi-asserted-by":"publisher","first-page":"587","DOI":"10.1016\/j.ejor.2019.02.038","volume":"277","author":"R Schur","year":"2019","unstructured":"Schur R, G\u00f6nsch J, Hassler M (2019) Time-consistent, risk-averse dynamic pricing. Eur J Oper Res 277(2):587\u2013603","journal-title":"Eur J Oper Res"},{"issue":"3","key":"857_CR151","doi-asserted-by":"publisher","first-page":"719","DOI":"10.1016\/j.ejor.2011.11.005","volume":"219","author":"A Shapiro","year":"2012","unstructured":"Shapiro A (2012) Minimax and risk averse multistage stochastic programming. Eur J Oper Res 219(3):719\u2013726","journal-title":"Eur J Oper Res"},{"issue":"1","key":"857_CR152","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ejor.2020.03.065","volume":"288","author":"A Shapiro","year":"2021","unstructured":"Shapiro A (2021) Tutorial on risk neutral, distributionally robust and risk averse multistage stochastic programming. Eur J Oper Res 288(1):1\u201313","journal-title":"Eur J Oper Res"},{"issue":"2","key":"857_CR153","doi-asserted-by":"publisher","first-page":"375","DOI":"10.1016\/j.ejor.2012.08.022","volume":"224","author":"A Shapiro","year":"2013","unstructured":"Shapiro A, Tekaya W, da Costa JP et al (2013) Risk neutral and risk averse stochastic dual dynamic programming method. Eur J Oper Res 224(2):375\u2013391","journal-title":"Eur J Oper Res"},{"issue":"5","key":"857_CR154","doi-asserted-by":"publisher","first-page":"3652","DOI":"10.1137\/120899005","volume":"51","author":"Y Shen","year":"2013","unstructured":"Shen Y, Stannat W, Obermayer K (2013) Risk-sensitive Markov control processes. SIAM J Control Optim 51(5):3652\u20133672","journal-title":"SIAM J Control Optim"},{"key":"857_CR155","doi-asserted-by":"crossref","unstructured":"Shen Y, Stannat W, Obermayer K (2014) A unified framework for risk-sensitive Markov control processes. In: Proceedings of the 53rd IEEE Conference on Decision and Control, IEEE, pp 1073\u20131078","DOI":"10.1109\/CDC.2014.7039524"},{"issue":"2","key":"857_CR156","first-page":"205","volume":"44","author":"K Sladk\u1ef3","year":"2008","unstructured":"Sladk\u1ef3 K (2008) Growth rates and average optimality in risk-sensitive Markov decision chains. Kybernetika 44(2):205\u2013226","journal-title":"Kybernetika"},{"issue":"6","key":"857_CR157","first-page":"1218","volume":"54","author":"K Sladk\u1ef3","year":"2018","unstructured":"Sladk\u1ef3 K (2018) Risk-sensitive average optimality in Markov decision processes. Kybernetika 54(6):1218\u20131230","journal-title":"Kybernetika"},{"issue":"2","key":"857_CR158","doi-asserted-by":"publisher","first-page":"741","DOI":"10.1016\/j.ejor.2019.07.032","volume":"280","author":"A Staino","year":"2020","unstructured":"Staino A, Russo E (2020) Nested Conditional Value-at-Risk portfolio selection: a model with temporal dependence driven by market-index volatility. Eur J Oper Res 280(2):741\u2013753","journal-title":"Eur J Oper Res"},{"issue":"3","key":"857_CR159","doi-asserted-by":"publisher","first-page":"463","DOI":"10.1007\/s001860050081","volume":"50","author":"\u0141 Stettner","year":"1999","unstructured":"Stettner \u0141 (1999) Risk sensitive portfolio optimization. Math Methods Oper Res 50(3):463\u2013474","journal-title":"Math Methods Oper Res"},{"issue":"32","key":"857_CR160","doi-asserted-by":"publisher","first-page":"395","DOI":"10.4064\/am32-4-3","volume":"4","author":"\u0141 Stettner","year":"2005","unstructured":"Stettner \u0141 (2005) Discrete time risk sensitive portfolio optimization with consumption and proportional transaction costs. Applicationes Mathematicae 4(32):395\u2013404","journal-title":"Applicationes Mathematicae"},{"key":"857_CR161","doi-asserted-by":"publisher","first-page":"105627","DOI":"10.1016\/j.sysconle.2023.105627","volume":"181","author":"\u0141 Stettner","year":"2023","unstructured":"Stettner \u0141 (2023) Certainty equivalent control of discrete time Markov processes with the average reward functional. Syst Control Lett 181:105627","journal-title":"Syst Control Lett"},{"key":"857_CR162","volume-title":"Reinforcement Learning: An Introduction","author":"RS Sutton","year":"2018","unstructured":"Sutton RS, Barto AG (2018) Reinforcement Learning: An Introduction. MIT Press, Cambridge"},{"issue":"7","key":"857_CR163","doi-asserted-by":"publisher","first-page":"3323","DOI":"10.1109\/TAC.2016.2644871","volume":"62","author":"A Tamar","year":"2016","unstructured":"Tamar A, Chow Y, Ghavamzadeh M et al (2016) Sequential decision making with coherent risk. IEEE Trans Autom Control 62(7):3323\u20133338","journal-title":"IEEE Trans Autom Control"},{"key":"857_CR164","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1016\/j.cam.2016.11.052","volume":"319","author":"K U\u011furlu","year":"2017","unstructured":"U\u011furlu K (2017) Controlled Markov decision processes with AVaR criteria for unbounded costs. J Comput Appl Math 319:24\u201337","journal-title":"J Comput Appl Math"},{"key":"857_CR165","doi-asserted-by":"publisher","first-page":"275","DOI":"10.1016\/j.cam.2018.05.030","volume":"344","author":"K U\u011furlu","year":"2018","unstructured":"U\u011furlu K (2018) Robust optimal control using conditional risk mappings in infinite horizon. J Comput Appl Math 344:275\u2013287","journal-title":"J Comput Appl Math"},{"key":"857_CR166","doi-asserted-by":"publisher","DOI":"10.1515\/9781400829460","volume-title":"Theory of Games and Economic Behavior (60th Anniversary Commemorative Edition)","author":"J Von Neumann","year":"2007","unstructured":"Von Neumann J, Morgenstern O (2007) Theory of Games and Economic Behavior (60th Anniversary Commemorative Edition). Princeton University Press, Princeton"},{"issue":"1","key":"857_CR167","doi-asserted-by":"publisher","first-page":"29","DOI":"10.2307\/2937817","volume":"105","author":"P Weil","year":"1990","unstructured":"Weil P (1990) Nonexpected utility in macroeconomics. Q J Econ 105(1):29\u201342","journal-title":"Q J Econ"},{"issue":"2","key":"857_CR168","doi-asserted-by":"publisher","first-page":"367","DOI":"10.2307\/2298062","volume":"60","author":"P Weil","year":"1993","unstructured":"Weil P (1993) Precautionary savings and the permanent income hypothesis. Rev Econ Stud 60(2):367\u2013383","journal-title":"Rev Econ Stud"},{"issue":"4","key":"857_CR169","doi-asserted-by":"publisher","first-page":"764","DOI":"10.2307\/1426972","volume":"13","author":"P Whittle","year":"1981","unstructured":"Whittle P (1981) Risk-sensitive linear\/quadratic\/Gaussian control. Adv Appl Probab 13(4):764\u2013777","journal-title":"Adv Appl Probab"},{"issue":"2","key":"857_CR170","doi-asserted-by":"publisher","first-page":"639","DOI":"10.1016\/j.ejor.2019.07.022","volume":"280","author":"D Wozabal","year":"2020","unstructured":"Wozabal D, Rameseder G (2020) Optimal bidding of a virtual power plant on the spanish day-ahead and intraday market for electricity. Eur J Oper Res 280(2):639\u2013655","journal-title":"Eur J Oper Res"},{"issue":"12","key":"857_CR171","doi-asserted-by":"publisher","first-page":"2808","DOI":"10.1111\/poms.13252","volume":"29","author":"L Xia","year":"2020","unstructured":"Xia L (2020) Risk-sensitive Markov decision processes with combined metrics of mean and variance. Prod Oper Manag 29(12):2808\u20132827","journal-title":"Prod Oper Manag"},{"key":"857_CR172","doi-asserted-by":"crossref","unstructured":"Xia L, Glynn PW (2022) Risk-sensitive Markov decision processes with long-run CVaR criterion. ArXiv preprint arXiv:2210.08740","DOI":"10.1111\/poms.14077"},{"key":"857_CR173","unstructured":"Xu W, Gao X, He X (2023) Regret bounds for Markov decision processes with recursive optimized certainty equivalents. ArXiv preprint arXiv:2301.12601"},{"issue":"3","key":"857_CR174","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1287\/ijoc.2015.0684","volume":"28","author":"W Zhang","year":"2016","unstructured":"Zhang W, Rahimian H, Bayraksan G (2016) Decomposition algorithms for risk-averse multistage stochastic programs with application to water allocation under uncertainty. INFORMS J Comput 28(3):385\u2013404","journal-title":"INFORMS J Comput"}],"container-title":["Mathematical Methods of Operations Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00186-024-00857-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00186-024-00857-0\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00186-024-00857-0.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,15]],"date-time":"2024-11-15T11:28:54Z","timestamp":1731670134000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00186-024-00857-0"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4]]},"references-count":174,"journal-issue":{"issue":"1-2","published-print":{"date-parts":[[2024,4]]}},"alternative-id":["857"],"URL":"https:\/\/doi.org\/10.1007\/s00186-024-00857-0","relation":{},"ISSN":["1432-2994","1432-5217"],"issn-type":[{"value":"1432-2994","type":"print"},{"value":"1432-5217","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4]]},"assertion":[{"value":"9 November 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"27 February 2024","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 March 2024","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 April 2024","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no relevant financial or non-financial interests to disclose","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}