{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,7,24]],"date-time":"2024-07-24T06:37:36Z","timestamp":1721803056618},"reference-count":24,"publisher":"Cambridge University Press (CUP)","issue":"02","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Appl. Probab."],"published-print":{"date-parts":[[2015,6]]},"abstract":"<jats:p>This paper deals with the first passage optimality and variance minimisation problems of discrete-time Markov decision processes (MDPs) with varying discount factors and unbounded rewards\/costs. First, under suitable conditions slightly weaker than those in the previous literature on the standard (infinite horizon) discounted MDPs, we establish the existence and characterisation of the first passage expected-optimal stationary policies. Second, to further distinguish the expected-optimal stationary policies, we introduce the variance minimisation problem, prove that it is equivalent to a <jats:italic>new<\/jats:italic> first passage optimality problem of MDPs, and, thus, show the existence of a variance-optimal policy that minimises the variance over the set of all first passage expected-optimal stationary policies. Finally, we use a <jats:italic>computable<\/jats:italic>\nexample to illustrate our main results and also to show the difference between the first passage optimality here and the standard discount optimality of MDPs in the previous literature.<\/jats:p>","DOI":"10.1017\/s0021900200012560","type":"journal-article","created":{"date-parts":[[2016,3,30]],"date-time":"2016-03-30T20:26:30Z","timestamp":1459369590000},"page":"441-456","source":"Crossref","is-referenced-by-count":3,"title":["First Passage Optimality and Variance Minimisation of Markov Decision Processes with Varying Discount Factors"],"prefix":"10.1017","volume":"52","author":[{"given":"Xiao","family":"Wu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xianping","family":"Guo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"56","published-online":{"date-parts":[[2016,3,30]]},"reference":[{"key":"S0021900200012560_ref19","doi-asserted-by":"publisher","DOI":"10.1007\/s00186-008-0276-z"},{"key":"S0021900200012560_ref20","volume-title":"Markov Decision Processes","year":"1994"},{"key":"S0021900200012560_ref17","first-page":"340","volume":"5","year":"1992","journal-title":"Systems Sci. Math. Sci."},{"key":"S0021900200012560_ref16","doi-asserted-by":"publisher","DOI":"10.1007\/s00245-001-0007-9"},{"key":"S0021900200012560_ref15","doi-asserted-by":"publisher","DOI":"10.1016\/0022-247X(87)90332-5"},{"key":"S0021900200012560_ref9","volume-title":"Discrete-Time Markov Control Processes","year":"1996"},{"key":"S0021900200012560_ref14","doi-asserted-by":"publisher","DOI":"10.1007\/s10255-011-0061-2"},{"key":"S0021900200012560_ref8","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2012.01.051"},{"key":"S0021900200012560_ref13","doi-asserted-by":"publisher","DOI":"10.1016\/j.jmaa.2009.05.058"},{"key":"S0021900200012560_ref7","doi-asserted-by":"publisher","DOI":"10.3166\/EJC.18.528-538"},{"key":"S0021900200012560_ref12","doi-asserted-by":"publisher","DOI":"10.1007\/s001860050079"},{"key":"S0021900200012560_ref6","doi-asserted-by":"publisher","DOI":"10.1109\/TAC.2009.2023833"},{"key":"S0021900200012560_ref11","doi-asserted-by":"publisher","DOI":"10.1137\/S0363012998340673"},{"key":"S0021900200012560_ref5","volume-title":"Continuous-Time Markov Decision Processes","year":"2009"},{"key":"S0021900200012560_ref10","volume-title":"Further Topics on Discrete-Time Markov Control Processes","year":"1999"},{"key":"S0021900200012560_ref4","first-page":"149","volume":"14","year":"2008","journal-title":"Bol. Soc. Mat. Mexicana (3)"},{"key":"S0021900200012560_ref3","doi-asserted-by":"publisher","DOI":"10.1287\/moor.19.1.152"},{"key":"S0021900200012560_ref2","volume-title":"Finite State Markovian Decision Processes","year":"1970"},{"key":"S0021900200012560_ref1","volume-title":"Markov Decision Processes with Applications in Finance","year":"2011"},{"key":"S0021900200012560_ref24","doi-asserted-by":"publisher","DOI":"10.1006\/jmaa.1998.6015"},{"key":"S0021900200012560_ref23","first-page":"369","volume":"39","year":"2011","journal-title":"Operat. Res. Lett."},{"key":"S0021900200012560_ref22","doi-asserted-by":"publisher","DOI":"10.2307\/3213832"},{"key":"S0021900200012560_ref21","doi-asserted-by":"publisher","DOI":"10.1007\/s00186-005-0445-2"},{"key":"S0021900200012560_ref18","first-page":"143","volume":"19","year":"2004","journal-title":"Econ. Qual. Control"}],"container-title":["Journal of Applied Probability"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0021900200012560","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,4,26]],"date-time":"2023-04-26T09:27:06Z","timestamp":1682501226000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0021900200012560\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,6]]},"references-count":24,"journal-issue":{"issue":"02","published-print":{"date-parts":[[2015,6]]}},"alternative-id":["S0021900200012560"],"URL":"https:\/\/doi.org\/10.1017\/s0021900200012560","relation":{},"ISSN":["0021-9002","1475-6072"],"issn-type":[{"value":"0021-9002","type":"print"},{"value":"1475-6072","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,6]]}}}