{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:10:09Z","timestamp":1750180209987,"version":"3.41.0"},"reference-count":6,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2023,9,28]],"date-time":"2023-09-28T00:00:00Z","timestamp":1695859200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGMETRICS Perform. Eval. Rev."],"published-print":{"date-parts":[[2023,9,28]]},"abstract":"<jats:p>This paper discusses the notion of optimality for time-average MDPs. We argue that while most authors claim to use the \"average reward\" criteria, the notion that is implicitly used is in fact the notion of what we call Bellman optimality. We show that it does not coincide with other existing notions of optimality, like gain-optimality and bias-optimality but has strong connection with canonical-policies (policies that are optimal for any finite horizons) as well as value iteration and policy iterations algorithms.<\/jats:p>","DOI":"10.1145\/3626570.3626582","type":"journal-article","created":{"date-parts":[[2023,10,2]],"date-time":"2023-10-02T22:16:57Z","timestamp":1696285017000},"page":"30-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["What is an Optimal Policy in Time-Average MDP?"],"prefix":"10.1145","volume":"51","author":[{"given":"Nicolas","family":"Gast","sequence":"first","affiliation":[{"name":"Univ. Grenoble Alpes, Inria, Grenoble, France"}]},{"given":"Bruno","family":"Gaujal","sequence":"additional","affiliation":[{"name":"Univ. Grenoble Alpes, Inria, Grenoble, France"}]},{"given":"Kimang","family":"Khun","sequence":"additional","affiliation":[{"name":"Univ. Grenoble Alpes, Inria, Grenoble, France"}]}],"member":"320","published-online":{"date-parts":[[2023,10,2]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4615-0805-2"},{"key":"e_1_2_1_2_1","volume-title":"Computing Whittle (and Gittins) index in subcubic time. arXiv preprint arXiv:2203.05207","author":"Gast Nicolas","year":"2022","unstructured":"Nicolas Gast, Bruno Gaujal, and Kimang Khun. Computing Whittle (and Gittins) index in subcubic time. arXiv preprint arXiv:2203.05207, 2022."},{"key":"e_1_2_1_4_1","volume-title":"Markov Decision Processes: Discrete Stochastic Dynamic Programming","author":"Puterman Martin L.","year":"2005","unstructured":"Martin L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. John Wiley & Sons, Inc., USA, 2nd edition, 2005.","edition":"2"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1287\/moor.3.4.308"},{"key":"e_1_2_1_6_1","volume-title":"Reinforcement learning: An introduction","author":"Sutton Richard S","year":"2018","unstructured":"Richard S Sutton and Andrew G Barto. Reinforcement learning: An introduction. MIT press, 2018."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1137\/1118099"}],"container-title":["ACM SIGMETRICS Performance Evaluation Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626570.3626582","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3626570.3626582","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:36:45Z","timestamp":1750178205000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3626570.3626582"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,28]]},"references-count":6,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2023,9,28]]}},"alternative-id":["10.1145\/3626570.3626582"],"URL":"https:\/\/doi.org\/10.1145\/3626570.3626582","relation":{},"ISSN":["0163-5999"],"issn-type":[{"type":"print","value":"0163-5999"}],"subject":[],"published":{"date-parts":[[2023,9,28]]},"assertion":[{"value":"2023-10-02","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}