{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T14:55:41Z","timestamp":1776783341481,"version":"3.51.2"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,5,30]],"date-time":"2024-05-30T00:00:00Z","timestamp":1717027200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,5,30]],"date-time":"2024-05-30T00:00:00Z","timestamp":1717027200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100004837","name":"Ministerio de Ciencia e Innovaci\u00f3n","doi-asserted-by":"publisher","award":["PID2021-122442NB-I00"],"award-info":[{"award-number":["PID2021-122442NB-I00"]}],"id":[{"id":"10.13039\/501100004837","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100003141","name":"Consejo Nacional de Ciencia y Tecnolog\u00eda","doi-asserted-by":"publisher","award":["87787"],"award-info":[{"award-number":["87787"]}],"id":[{"id":"10.13039\/501100003141","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Optim Theory Appl"],"published-print":{"date-parts":[[2024,8]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>This paper studies constrained Markov decision processes under the total expected discounted cost optimality criterion, with a state-action dependent discount factor that may take any value between zero and one. Both the state and the action space are assumed to be Borel spaces. By using the linear programming approach, consisting in stating the control problem as a linear problem on a set of occupation measures, we show the existence of an optimal stationary Markov policy. Our results are based on the study of both weak-strong topologies in the space of occupation measures and Young measures in the space of Markov policies.<\/jats:p>","DOI":"10.1007\/s10957-024-02453-y","type":"journal-article","created":{"date-parts":[[2024,5,30]],"date-time":"2024-05-30T14:02:26Z","timestamp":1717077746000},"page":"897-931","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Constrained Markov Decision Processes with Non-constant Discount Factor"],"prefix":"10.1007","volume":"202","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2502-9219","authenticated-orcid":false,"given":"H\u00e9ctor","family":"Jasso-Fuentes","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4677-4725","authenticated-orcid":false,"given":"Tom\u00e1s","family":"Prieto-Rumeau","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,5,30]]},"reference":[{"key":"2453_CR1","volume-title":"Infinite Dimensional Analysis: A Hitchhiker\u2019s Guide","author":"CD Aliprantis","year":"2006","unstructured":"Aliprantis, C.D., Border, K.C.: Infinite Dimensional Analysis: A Hitchhiker\u2019s Guide, 3rd edn. Springer, Berlin (2006)","edition":"3"},{"key":"2453_CR2","doi-asserted-by":"publisher","first-page":"45","DOI":"10.1007\/BF01303434","volume":"43","author":"E Altman","year":"1996","unstructured":"Altman, E.: Constrained Markov decision processes with total cost criteria: occupation measures and primal LP. Math. Methods Oper. Res. 43, 45\u201372 (1996)","journal-title":"Math. Methods Oper. Res."},{"key":"2453_CR3","doi-asserted-by":"publisher","first-page":"461","DOI":"10.1007\/s001860200209","volume":"55","author":"J Alvarez-Mena","year":"2002","unstructured":"Alvarez-Mena, J., Hern\u00e1ndez-Lerma, O.: Convergence of the optimal values of constrained Markov control processes. Math. Methods Oper. Res. 55, 461\u2013484 (2002)","journal-title":"Math. Methods Oper. Res."},{"key":"2453_CR4","volume-title":"Lectures on Young Measure Theory and Its Applications in Economics","author":"EJ Balder","year":"1998","unstructured":"Balder, E.J.: Lectures on Young Measure Theory and Its Applications in Economics. Rijksuniversiteit Utrecht. Mathematisch Instituut. Utrecht University, Utrecht (1998)"},{"key":"2453_CR5","doi-asserted-by":"publisher","first-page":"339","DOI":"10.1007\/BF01229313","volume":"1","author":"EJ Balder","year":"1991","unstructured":"Balder, E.J.: On Cournot\u2013Nash equilibrium distributions for games with differential information and discontinuous payoffs. Econ. Theory 1, 339\u2013354 (1991)","journal-title":"Econ. Theory"},{"key":"2453_CR6","doi-asserted-by":"publisher","first-page":"494","DOI":"10.1287\/moor.26.3.494.10581","volume":"26","author":"EJ Balder","year":"2001","unstructured":"Balder, E.J.: On ws-convergence of product measures. Math. Oper. Res. 26, 494\u2013518 (2001)","journal-title":"Math. Oper. Res."},{"key":"2453_CR7","volume-title":"Stochastic Optimal Control: The Discrete-Time Case","author":"D Bertsekas","year":"1978","unstructured":"Bertsekas, D., Shreve, S.E.: Stochastic Optimal Control: The Discrete-Time Case, vol. 5. Academic Press, New York (1978)"},{"key":"2453_CR8","doi-asserted-by":"publisher","first-page":"51","DOI":"10.1016\/j.orl.2008.10.005","volume":"37","author":"Y Carmon","year":"2009","unstructured":"Carmon, Y., Shwartz, A.: Markov decision processes with exponentially representable discounting. Oper. Res. Lett. 37, 51\u201355 (2009)","journal-title":"Oper. Res. Lett."},{"key":"2453_CR9","doi-asserted-by":"publisher","first-page":"1298","DOI":"10.1137\/120867925","volume":"51","author":"F Dufour","year":"2013","unstructured":"Dufour, F., Prieto-Rumeau, T.: Finite linear programming approximations of constrained discounted Markov decision processes. SIAM J. Control Optim. 51, 1298\u20131324 (2013)","journal-title":"SIAM J. Control Optim."},{"key":"2453_CR10","doi-asserted-by":"publisher","first-page":"856","DOI":"10.1016\/j.jmaa.2013.12.016","volume":"413","author":"F Dufour","year":"2014","unstructured":"Dufour, F., Prieto-Rumeau, T.: Stochastic approximations of constrained discounted Markov decision processes. J. Math. Anal. Appl. 413, 856\u2013879 (2014)","journal-title":"J. Math. Anal. Appl."},{"key":"2453_CR11","doi-asserted-by":"publisher","first-page":"945","DOI":"10.1137\/21M144565X","volume":"60","author":"F Dufour","year":"2022","unstructured":"Dufour, F., Prieto-Rumeau, T.: Stationary Markov Nash equilibria for nonzero-sum constrained ARAT Markov games. SIAM J. Control. Optim. 60, 945\u2013967 (2022)","journal-title":"SIAM J. Control. Optim."},{"key":"2453_CR12","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1051\/cocv\/2024002","volume":"30","author":"F Dufour","year":"2024","unstructured":"Dufour, F., Prieto-Rumeau, T.: Absorbing Markov decision processes. ESAIM Control Optim. Calc. Var. 30, 5 (2024)","journal-title":"ESAIM Control Optim. Calc. Var."},{"key":"2453_CR13","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4615-6746-2","volume-title":"Controlled Markov Processes","author":"EB Dynkin","year":"1979","unstructured":"Dynkin, E.B., Yushkevich, A.A.: Controlled Markov Processes. Springer, New York (1979)"},{"key":"2453_CR14","doi-asserted-by":"publisher","DOI":"10.1016\/j.automatica.2019.108582","volume":"111","author":"EA Feinberg","year":"2020","unstructured":"Feinberg, E.A., Ja\u015bkiewicz, A., Nowak, A.S.: Constrained discounted Markov decision processes with Borel state spaces. Automatica 111, 105582 (2020)","journal-title":"Automatica"},{"key":"2453_CR15","doi-asserted-by":"publisher","first-page":"163","DOI":"10.1137\/18M1194924","volume":"57","author":"EA Feinberg","year":"2019","unstructured":"Feinberg, E.A., Piunovskiy, A.: Sufficiency of deterministic policies for atomless discounted and uniformly absorbing MDPs with multiple criteria. SIAM J. Control Optim. 57, 163\u2013191 (2019)","journal-title":"SIAM J. Control Optim."},{"key":"2453_CR16","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1287\/moor.1110.0525","volume":"37","author":"EA Feinberg","year":"2012","unstructured":"Feinberg, E.A., Rothblum, U.G.: Splitting randomized stationary policies in total-reward Markov decision processes. Math. Oper. Res. 37, 129\u2013153 (2012)","journal-title":"Math. Oper. Res."},{"key":"2453_CR17","doi-asserted-by":"publisher","first-page":"152","DOI":"10.1287\/moor.19.1.152","volume":"19","author":"EA Feinberg","year":"1994","unstructured":"Feinberg, E.A., Shwartz, A.: Markov decision models with weighted discounted criteria. Math. Oper. Res. 19, 152\u2013168 (1994)","journal-title":"Math. Oper. Res."},{"key":"2453_CR18","doi-asserted-by":"publisher","DOI":"10.1515\/9783110280517","volume-title":"Young Measures and Compactness in Measure Spaces","author":"LC Florescu","year":"2012","unstructured":"Florescu, L.C., Godet-Thobie, C.: Young Measures and Compactness in Measure Spaces. De Gruyter, Berlin (2012)"},{"key":"2453_CR19","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1006\/jmaa.1994.1147","volume":"183","author":"D Hern\u00e1ndez-Hern\u00e1ndez","year":"1994","unstructured":"Hern\u00e1ndez-Hern\u00e1ndez, D., Hern\u00e1ndez-Lerma, O.: Discounted cost Markov decision processes on Borel spaces: the linear programming formulation. J. Math. Anal. Appl. 183, 335\u2013351 (1994)","journal-title":"J. Math. Anal. Appl."},{"key":"2453_CR20","doi-asserted-by":"publisher","first-page":"271","DOI":"10.1007\/s001860000071","volume":"52","author":"J Gonz\u00e1lez-Hern\u00e1ndez","year":"2000","unstructured":"Gonz\u00e1lez-Hern\u00e1ndez, J., Hern\u00e1ndez-Lerma, O.: Constrained Markov control processes in Borel spaces: the discounted case. Math. Methods Oper. Res. 52, 271\u2013285 (2000)","journal-title":"Math. Methods Oper. Res."},{"key":"2453_CR21","doi-asserted-by":"publisher","first-page":"163","DOI":"10.3233\/RDA-2012-0063","volume":"4","author":"J Gonz\u00e1lez-Hern\u00e1ndez","year":"2013","unstructured":"Gonz\u00e1lez-Hern\u00e1ndez, J., L\u00f3pez-Mart\u00ednez, R.R., Minj\u00e1rez-Sosa, J.A., Gabriel-Arguelles, J.R.: Constrained Markov control processes with randomized discounted cost criteria: occupation measures and extremal points. Risk Decis. Anal. 4, 163\u2013176 (2013)","journal-title":"Risk Decis. Anal."},{"key":"2453_CR22","doi-asserted-by":"publisher","first-page":"575","DOI":"10.1002\/oca.2089","volume":"35","author":"J Gonz\u00e1lez-Hern\u00e1ndez","year":"2014","unstructured":"Gonz\u00e1lez-Hern\u00e1ndez, J., L\u00f3pez-Mart\u00ednez, R.R., Minj\u00e1rez-Sosa, J.A., Gabriel-Arguelles, J.R.: Constrained Markov control processes with randomized discounted cost criteria: infinite linear programming approach. Optim. Control Appl. Methods 35, 575\u2013591 (2014)","journal-title":"Optim. Control Appl. Methods"},{"key":"2453_CR23","first-page":"230","volume":"23","author":"X Guo","year":"2000","unstructured":"Guo, X.: Constrained nonhomogeneous Markov decision processes with expected total reward criterion. Acta Appl. Math. Sin. (Engl. Ser.) 23, 230\u2013235 (2000)","journal-title":"Acta Appl. Math. Sin. (Engl. Ser.)"},{"key":"2453_CR24","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1137\/S0363012999361627","volume":"42","author":"O Hern\u00e1ndez-Lerma","year":"2003","unstructured":"Hern\u00e1ndez-Lerma, O., Gonz\u00e1lez-Hern\u00e1ndez, J., L\u00f3pez-Mart\u00ednez, R.R.: Constrained average cost Markov control processes in Borel spaces. SIAM J. Control. Optim. 42, 442\u2013468 (2003)","journal-title":"SIAM J. Control. Optim."},{"key":"2453_CR25","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-0729-0","volume-title":"Discrete-Time Markov Control Processes: Basic Optimality Criteria","author":"O Hern\u00e1ndez-Lerma","year":"1996","unstructured":"Hern\u00e1ndez-Lerma, O., Lasserre, J.B.: Discrete-Time Markov Control Processes: Basic Optimality Criteria. Springer, New York (1996)"},{"key":"2453_CR26","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-0561-6","volume-title":"Further Topics on Discrete-Time Markov Control Processes","author":"O Hern\u00e1ndez-Lerma","year":"1999","unstructured":"Hern\u00e1ndez-Lerma, O., Lasserre, J.B.: Further Topics on Discrete-Time Markov Control Processes. Springer, New York (1999)"},{"key":"2453_CR27","first-page":"82","volume":"53","author":"R Ilhuicatzi-Rold\u00e1n","year":"2017","unstructured":"Ilhuicatzi-Rold\u00e1n, R., Cruz-Su\u00e1rez, H., Ch\u00e1vez-Rodr\u00edguez, S.: Markov decision processes with time-varying discount factors and random horizon. Kybernetika 53, 82\u201398 (2017)","journal-title":"Kybernetika"},{"key":"2453_CR28","doi-asserted-by":"publisher","first-page":"925","DOI":"10.1080\/02331934.2022.2130699","volume":"73","author":"H Jasso-Fuentes","year":"2024","unstructured":"Jasso-Fuentes, H., L\u00f3pez-Martinez, R.R., Minj\u00e1rez-Sosa, J.A.: Some advances on constrained Markov decision processes in Borel spaces with random state-dependent discount factors. Optimization 73, 925\u2013951 (2024)","journal-title":"Optimization"},{"key":"2453_CR29","doi-asserted-by":"publisher","first-page":"377","DOI":"10.1007\/s00186-020-00716-8","volume":"92","author":"H Jasso-Fuentes","year":"2020","unstructured":"Jasso-Fuentes, H., Menaldi, J.L., Prieto-Rumeau, T.: Discrete-time control with non-constant discount factor. Math. Methods Oper. Res. 92, 377\u2013399 (2020)","journal-title":"Math. Methods Oper. Res."},{"key":"2453_CR30","doi-asserted-by":"publisher","first-page":"409","DOI":"10.1007\/s00245-018-9503-z","volume":"81","author":"H Jasso-Fuentes","year":"2020","unstructured":"Jasso-Fuentes, H., Menaldi, J.L., Prieto-Rumeau, T.: Discrete-time hybrid control in Borel spaces. Appl. Math. Optim. 81, 409\u2013441 (2020)","journal-title":"Appl. Math. Optim."},{"key":"2453_CR31","unstructured":"Jasso-Fuentes, H., Menaldi, J.L., Prieto-Rumeau, T.: Recent results on discrete-time hybrid control models with general state and action spaces. Pure Appl. Funct. Anal. (in press) (2024)"},{"key":"2453_CR32","doi-asserted-by":"crossref","unstructured":"Jasso-Fuentes, H., Salgado-Su\u00e1rez G.D.: Discrete-time hybrid control processes with unbounded costs. Submitted (2024)","DOI":"10.1007\/s00245-024-10192-9"},{"key":"2453_CR33","doi-asserted-by":"publisher","first-page":"489","DOI":"10.1007\/s00186-016-0551-3","volume":"84","author":"AF Mendoza-P\u00e9rez","year":"2016","unstructured":"Mendoza-P\u00e9rez, A.F., Jasso-Fuentes, H., De la Cruz-Courtois, O.A.: Constrained Markov decision processes in Borel spaces: from discounted to average optimality. Math. Methods Oper. Res. 84, 489\u2013525 (2016)","journal-title":"Math. Methods Oper. Res."},{"key":"2453_CR34","doi-asserted-by":"publisher","first-page":"743","DOI":"10.1007\/s11750-015-0360-5","volume":"23","author":"JA Minj\u00e1rez-Sosa","year":"2015","unstructured":"Minj\u00e1rez-Sosa, J.A.: Markov control models with unknown random state-action-dependent discount factors. TOP 23, 743\u2013772 (2015)","journal-title":"TOP"},{"key":"2453_CR35","volume-title":"Optimal Control of Random Sequences in Problems With Constraints","author":"A Piunovskiy","year":"2012","unstructured":"Piunovskiy, A.: Optimal Control of Random Sequences in Problems With Constraints. Springer, Dordrecht (2012)"},{"key":"2453_CR36","doi-asserted-by":"crossref","unstructured":"Piunovskiy, A., Zhang, Y.: On the continuity of the projection mapping from strategic measures to occupation measures in absorbing Markov decision processes. arXiv:2311.14043 (2023)","DOI":"10.1007\/s00245-024-10124-7"},{"key":"2453_CR37","doi-asserted-by":"publisher","first-page":"345","DOI":"10.1016\/0304-4149(75)90031-9","volume":"3","author":"M Sch\u00e4l","year":"1975","unstructured":"Sch\u00e4l, M.: On dynamic programming: compactness of the space of policies. Stoch. Proc. Appl. 3, 345\u2013364 (1975)","journal-title":"Stoch. Proc. Appl."},{"key":"2453_CR38","first-page":"369","volume":"39","author":"Q Wei","year":"2011","unstructured":"Wei, Q., Guo, X.: Markov decision processes with state-dependent discount factors and unbounded rewards\/costs. Oper. Res. Lett. 39, 369\u2013374 (2011)","journal-title":"Oper. Res. Lett."}],"container-title":["Journal of Optimization Theory and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10957-024-02453-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10957-024-02453-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10957-024-02453-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,20]],"date-time":"2024-11-20T19:10:32Z","timestamp":1732129832000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10957-024-02453-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,30]]},"references-count":38,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,8]]}},"alternative-id":["2453"],"URL":"https:\/\/doi.org\/10.1007\/s10957-024-02453-y","relation":{},"ISSN":["0022-3239","1573-2878"],"issn-type":[{"value":"0022-3239","type":"print"},{"value":"1573-2878","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,30]]},"assertion":[{"value":"12 July 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 May 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"30 May 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}