{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,2,21]],"date-time":"2025-02-21T03:24:10Z","timestamp":1740108250874,"version":"3.37.3"},"reference-count":41,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2020,3,2]],"date-time":"2020-03-02T00:00:00Z","timestamp":1583107200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2020,3,2]],"date-time":"2020-03-02T00:00:00Z","timestamp":1583107200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100005690","name":"Universit\u00e4t des Saarlandes","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100005690","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Math Meth Oper Res"],"published-print":{"date-parts":[[2020,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Markov decision models (MDM) used in practical applications are most often less complex than the underlying \u2018true\u2019 MDM. The reduction of model complexity is performed for several reasons. However, it is obviously of interest to know what kind of model reduction is reasonable (in regard to the optimal value) and what kind is not. In this article we propose a way how to address this question. We introduce a sort of derivative of the optimal value as a function of the transition probabilities, which can be used to measure the (first-order) sensitivity of the optimal value w.r.t. changes in the transition probabilities. \u2018Differentiability\u2019 is obtained for a fairly broad class of MDMs, and the \u2018derivative\u2019 is specified explicitly. Our theoretical findings are illustrated by means of optimization problems in inventory control and mathematical finance.<\/jats:p>","DOI":"10.1007\/s00186-020-00706-w","type":"journal-article","created":{"date-parts":[[2020,3,2]],"date-time":"2020-03-02T10:02:47Z","timestamp":1583143367000},"page":"165-197","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["First-order sensitivity of the optimal value in a Markov decision model with respect to deviations in the transition probability function"],"prefix":"10.1007","volume":"92","author":[{"given":"Patrick","family":"Kern","sequence":"first","affiliation":[]},{"given":"Axel","family":"Simroth","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7737-177X","authenticated-orcid":false,"given":"Henryk","family":"Z\u00e4hle","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2020,3,2]]},"reference":[{"key":"706_CR1","doi-asserted-by":"publisher","first-page":"201","DOI":"10.1070\/RM1967v022n06ABEH003761","volume":"22","author":"VI Averbukh","year":"1967","unstructured":"Averbukh VI, Smolyanov OG (1967) The theory of differentiation in linear topological spaces. Russ Math Surv 22:201\u2013258","journal-title":"Russ Math Surv"},{"key":"706_CR2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-18324-9","volume-title":"Markov decision processes with applications to finance","author":"N B\u00e4uerle","year":"2011","unstructured":"B\u00e4uerle N, Rieder U (2011) Markov decision processes with applications to finance. Springer, Berlin"},{"key":"706_CR3","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1016\/j.insmatheco.2013.10.015","volume":"54","author":"F Bellini","year":"2014","unstructured":"Bellini F, Klar B, M\u00fcller A, Rosazza Gianin E (2014) Generalized quantiles as risk measures. Insur Math Econ 54:41\u201348","journal-title":"Insur Math Econ"},{"key":"706_CR4","first-page":"127","volume":"15","author":"SH Cox Jr","year":"1971","unstructured":"Cox SH Jr, Nadler SB Jr (1971) Supremum norm differentiability. Ann Soc Math Pol 15:127\u2013131","journal-title":"Ann Soc Math Pol"},{"key":"706_CR5","first-page":"35","volume":"10","author":"G Dall\u2019Aglio","year":"1956","unstructured":"Dall\u2019Aglio G (1956) Sugli estremi di momentidetle funzioni di ripartizione doppia. Ann Sc Norm Super Pisa 10:35\u201374","journal-title":"Ann Sc Norm Super Pisa"},{"key":"706_CR6","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511755347","volume-title":"Real analysis and probability","author":"RM Dudley","year":"2002","unstructured":"Dudley RM (2002) Real analysis and probability. Cambridge University Press, Cambridge"},{"key":"706_CR7","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-5604-5","volume-title":"Von Mises calculus for statistical functionals","author":"LT Fernholz","year":"1983","unstructured":"Fernholz LT (1983) Von Mises calculus for statistical functionals. Springer, Berlin"},{"key":"706_CR8","doi-asserted-by":"publisher","DOI":"10.1515\/9783110218053","volume-title":"Stochastic finance. An introduction in discrete time","author":"H F\u00f6llmer","year":"2011","unstructured":"F\u00f6llmer H, Schied A (2011) Stochastic finance. An introduction in discrete time. de Gruyter, Berlin"},{"key":"706_CR9","first-page":"97","volume":"16","author":"RD Gill","year":"1989","unstructured":"Gill RD (1989) Non- and semi-parametric maximum likelihood estimators and the von mises method\u2014I. Scand J Stat 16:97\u2013128","journal-title":"Scand J Stat"},{"key":"706_CR10","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-0729-0","volume-title":"Discrete-time Markov control processes: basic optimality criteria","author":"O Hern\u00e1ndez-Lerma","year":"1996","unstructured":"Hern\u00e1ndez-Lerma O, Lasserre JB (1996) Discrete-time Markov control processes: basic optimality criteria. Springer, Berlin"},{"key":"706_CR11","series-title":"Lecture notes in economics and mathematical systems 33","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-46229-0","volume-title":"Foundations of non-stationary dynamic programming with discrete time parameter","author":"K Hinderer","year":"1970","unstructured":"Hinderer K (1970) Foundations of non-stationary dynamic programming with discrete time parameter. Lecture notes in economics and mathematical systems 33. Springer, Berlin"},{"key":"706_CR12","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1007\/s00186-005-0438-1","volume":"62","author":"K Hinderer","year":"2005","unstructured":"Hinderer K (2005) Lipschitz continuity of value functions in Markovian decision processes. Math Methods Oper Res 62:3\u201322","journal-title":"Math Methods Oper Res"},{"key":"706_CR13","unstructured":"Holfeld D, Simroth A (2017) Learning from the past\u2014risk profiler for intermodal route planning in SYNCHRO-NET. In: International conference on operations research (OR2017), Berlin"},{"key":"706_CR14","unstructured":"Holfeld D, Simroth A, Li Y, Manerba D, Tadei R (2018) Risk analysis for synchro-modal freight transportation: the SYNCHRO-NET approach. In: 7th international workshop on freight transportation and logistics (Odysseus 2018), Cagliari"},{"key":"706_CR15","first-page":"52","volume":"13","author":"LV Kantorovich","year":"1958","unstructured":"Kantorovich LV, Rubinstein GS (1958) On a space of completely additive functions. Vestnik Leningrad University 13:52\u201359","journal-title":"Vestnik Leningrad University"},{"key":"706_CR16","doi-asserted-by":"crossref","unstructured":"Kern P, Simroth A, Z\u00e4hle H (2020) Supplement to \u201cFirst-order sensitivity of the optimal value in a Markov decision model with respect to deviations in the transition probability function\u201d","DOI":"10.1007\/s00186-020-00706-w"},{"key":"706_CR17","doi-asserted-by":"publisher","first-page":"32","DOI":"10.3390\/risks4030032","volume":"4","author":"R Kiesel","year":"2016","unstructured":"Kiesel R, R\u00fchlicke R, Stahl G, Zheng J (2016) The Wasserstein metric and robustness in risk management. Risks 4:32","journal-title":"Risks"},{"key":"706_CR18","first-page":"17","volume":"27","author":"M Kolonko","year":"1983","unstructured":"Kolonko M (1983) Bounds for the regret loss in dynamic programming under adaptive control. Z Oper Res 27:17\u201337","journal-title":"Z Oper Res"},{"key":"706_CR19","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1016\/j.ssci.2016.05.004","volume":"88","author":"D Komljenovic","year":"2016","unstructured":"Komljenovic D, Gaha M, Abdul-Nour G, Langheit C, Bourgeois M (2016) Risks of extreme and rare events in asset management. Saf Sci 88:129\u2013145","journal-title":"Saf Sci"},{"key":"706_CR20","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1111\/sjos.12259","volume":"44","author":"V Kr\u00e4tschmer","year":"2017","unstructured":"Kr\u00e4tschmer V, Z\u00e4hle H (2017) Statistical inference for expectile-based risk measures. Scand J Stat 44:425\u2013454","journal-title":"Scand J Stat"},{"key":"706_CR21","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1016\/j.jmva.2011.06.005","volume":"103","author":"V Kr\u00e4tschmer","year":"2012","unstructured":"Kr\u00e4tschmer V, Schied A, Z\u00e4hle H (2012) Qualitative and infinitesimal robustness of tail-dependent statistical functionals. J Multivar Anal 103:35\u201347","journal-title":"J Multivar Anal"},{"key":"706_CR22","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jmva.2017.02.005","volume":"158","author":"V Kr\u00e4tschmer","year":"2017","unstructured":"Kr\u00e4tschmer V, Schied A, Z\u00e4hle H (2017) Domains of weak continuity of statistical functionals with a view toward robust statistics. J Multivar Anal 158:1\u201319","journal-title":"J Multivar Anal"},{"key":"706_CR23","doi-asserted-by":"publisher","first-page":"889","DOI":"10.3150\/bj\/1161614951","volume":"12","author":"JP Lemor","year":"2006","unstructured":"Lemor JP, Gobet E, Warin X (2006) Rate of convergence of an empirical regression method for solving generalized backward stochastic differential equations. Bernoulli 12:889\u2013916","journal-title":"Bernoulli"},{"key":"706_CR24","doi-asserted-by":"publisher","first-page":"247","DOI":"10.2307\/1926560","volume":"51","author":"RC Merton","year":"1969","unstructured":"Merton RC (1969) Lifetime portfolio selection under uncertainty: the continuous-time case. Rev Econ Stat 51:247\u2013257","journal-title":"Rev Econ Stat"},{"key":"706_CR25","doi-asserted-by":"publisher","first-page":"872","DOI":"10.1287\/moor.22.4.872","volume":"22","author":"A M\u00fcller","year":"1997","unstructured":"M\u00fcller A (1997) How does the value function of a Markov decision process depend on the transition probabilities ? Math Oper Res 22:872\u2013885","journal-title":"Math Oper Res"},{"key":"706_CR26","doi-asserted-by":"publisher","first-page":"429","DOI":"10.2307\/1428011","volume":"29","author":"A M\u00fcller","year":"1997","unstructured":"M\u00fcller A (1997) Integral probability metrics and their generating classes of functions. Adv Appl Probab 29:429\u2013443","journal-title":"Adv Appl Probab"},{"key":"706_CR27","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-89500-8","volume-title":"Continuous-time stochastic control and optimization with financial applications","author":"H Pham","year":"2009","unstructured":"Pham H (2009) Continuous-time stochastic control and optimization with financial applications. Springer, Berlin"},{"key":"706_CR28","doi-asserted-by":"publisher","DOI":"10.1002\/9780470316887","volume-title":"Markov decision processes: discrete stochastic dynamic programming","author":"ML Puterman","year":"1994","unstructured":"Puterman ML (1994) Markov decision processes: discrete stochastic dynamic programming. Wiley, New York"},{"key":"706_CR29","volume-title":"Probability metrics and the stability of stochastic models","author":"ST Rachev","year":"1991","unstructured":"Rachev ST (1991) Probability metrics and the stability of stochastic models. Wiley, New York"},{"key":"706_CR30","volume-title":"Delta method, infinite dimensional. Encyclopedia of statistical sciences","author":"W R\u00f6misch","year":"2004","unstructured":"R\u00f6misch W (2004) Delta method, infinite dimensional. Encyclopedia of statistical sciences. Wiley, New York"},{"key":"706_CR31","volume-title":"Functional analysis","author":"W Rudin","year":"1991","unstructured":"Rudin W (1991) Functional analysis. McGraw-Hill, New York"},{"key":"706_CR32","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-71333-3","volume-title":"Nonsmooth analysis","author":"W Schirotzek","year":"2007","unstructured":"Schirotzek W (2007) Nonsmooth analysis. Springer, Berlin"},{"key":"706_CR33","unstructured":"Sebasti\u00e3o e Silva J (1956) Le calcul diff\u00e9rentiel et int\u00e9gral dans les espaces localement convexes, r\u00e9els ou complexes, Nota I. Rendiconti, Atti della Accademia Nazionale dei Lincei, Serie VIII, Vol VIII, pp. 743\u2013750"},{"key":"706_CR34","doi-asserted-by":"publisher","first-page":"477","DOI":"10.1007\/BF00940933","volume":"66","author":"A Shapiro","year":"1990","unstructured":"Shapiro A (1990) On concepts of directional differentiability. J Optim Theory Appl 66:477\u2013487","journal-title":"J Optim Theory Appl"},{"key":"706_CR35","doi-asserted-by":"publisher","first-page":"784","DOI":"10.1137\/1118101","volume":"18","author":"SS Vallender","year":"1974","unstructured":"Vallender SS (1974) Calculation of the Wasserstein distance between probability distributions on the line. Theory Probab Appl 18:784\u2013786","journal-title":"Theory Probab Appl"},{"key":"706_CR36","doi-asserted-by":"publisher","first-page":"99","DOI":"10.2307\/1427272","volume":"20","author":"NM Van Dijk","year":"1988","unstructured":"Van Dijk NM (1988) Perturbation theory for unbounded Markov reward processes with applications to queueing. Adv Appl Probab 20:99\u2013111","journal-title":"Adv Appl Probab"},{"key":"706_CR37","doi-asserted-by":"publisher","first-page":"79","DOI":"10.2307\/1427271","volume":"20","author":"NM Van Dijk","year":"1988","unstructured":"Van Dijk NM, Puterman ML (1988) Perturbation theory for Markov reward processes with applications to queueing systems. Adv Appl Probab 20:79\u201398","journal-title":"Adv Appl Probab"},{"key":"706_CR38","volume-title":"Topics in optimal transportation","author":"C Villani","year":"2003","unstructured":"Villani C (2003) Topics in optimal transportation, vol 58. American Mathematical Society, Providence"},{"key":"706_CR39","doi-asserted-by":"publisher","first-page":"326","DOI":"10.1016\/0022-247X(77)90210-4","volume":"58","author":"J Wessels","year":"1977","unstructured":"Wessels J (1977) Markov programming by successive approximations with respect to weighted supremum norms. J Math Anal Appl 58:326\u2013335","journal-title":"J Math Anal Appl"},{"key":"706_CR40","doi-asserted-by":"publisher","first-page":"102","DOI":"10.1016\/j.psep.2015.07.004","volume":"98","author":"M Yang","year":"2015","unstructured":"Yang M, Khan F, Lye L, Amyotte P (2015) Risk assessment of rare events. Process Saf Environ Prot 98:102\u2013108","journal-title":"Process Saf Environ Prot"},{"key":"706_CR41","doi-asserted-by":"publisher","first-page":"278","DOI":"10.1137\/1128025","volume":"28","author":"VM Zolotarev","year":"1983","unstructured":"Zolotarev VM (1983) Probability metrics. Theory Probab Appl 28:278\u2013302","journal-title":"Theory Probab Appl"}],"container-title":["Mathematical Methods of Operations Research"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s00186-020-00706-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s00186-020-00706-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s00186-020-00706-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,9,27]],"date-time":"2023-09-27T23:17:09Z","timestamp":1695856629000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s00186-020-00706-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,3,2]]},"references-count":41,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,8]]}},"alternative-id":["706"],"URL":"https:\/\/doi.org\/10.1007\/s00186-020-00706-w","relation":{},"ISSN":["1432-2994","1432-5217"],"issn-type":[{"type":"print","value":"1432-2994"},{"type":"electronic","value":"1432-5217"}],"subject":[],"published":{"date-parts":[[2020,3,2]]},"assertion":[{"value":"23 January 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 September 2019","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 March 2020","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}