{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,2]],"date-time":"2025-11-02T16:55:42Z","timestamp":1762102542987},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2020,7]]},"abstract":"<jats:p>In Markov Decision Processes (MDPs), rewards are assigned according to a function of the last state and action. This is often limiting, when the considered domain is not naturally Markovian, but becomes so after careful engineering of extended state space. The extended states record information from the past that is sufficient to assign rewards by looking just at the last state and action.  Non-Markovian Reward Decision  Processes  (NRMDPs)  extend  MDPs by allowing for non-Markovian rewards, which depend on the history of states and actions.  Non-Markovian rewards can be specified in temporal logics on finite traces such as LTLf\/LDLf, with the great advantage of a higher abstraction and succinctness; they can then be automatically compiled into an MDP with an extended state space. We contribute to the techniques to handle temporal rewards and to the solutions to engineer them. We first present an approach to compiling temporal rewards which merges the formula automata into a single transducer, sometimes saving up to an exponential number of states. We then define monitoring rewards, which add a further level of abstraction to temporal rewards by adopting the four-valued conditions of  runtime  monitoring;  we argue that our compilation technique allows for an efficient handling of monitoring rewards. Finally, we discuss application to reinforcement learning.<\/jats:p>","DOI":"10.24963\/kr.2020\/89","type":"proceedings-article","created":{"date-parts":[[2020,8,20]],"date-time":"2020-08-20T04:39:16Z","timestamp":1597898356000},"page":"860-870","source":"Crossref","is-referenced-by-count":6,"title":["Temporal Logic Monitoring Rewards via Transducers"],"prefix":"10.24963","author":[{"given":"Giuseppe","family":"De Giacomo","sequence":"first","affiliation":[{"name":"University of Rome \"La Sapienza\""}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marco","family":"Favorito","sequence":"additional","affiliation":[{"name":"University of Rome \"La Sapienza\""}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luca","family":"Iocchi","sequence":"additional","affiliation":[{"name":"University of Rome \"La Sapienza\""}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fabio","family":"Patrizi","sequence":"additional","affiliation":[{"name":"University of Rome \"La Sapienza\""}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alessandro","family":"Ronca","sequence":"additional","affiliation":[{"name":"University of Rome \"La Sapienza\""}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"10584","event":{"number":"17","sponsor":["Artificial Intelligence Journal","Principles of Knowledge Representation and Reasoning Inc.","Association for Logic Programming","Center for Perspicuous Computing","European Association for Artificial Intelligence","Ontopic - The Virtual Knowledge Graph Company"],"acronym":"KR-2020","name":"17th International Conference on Principles of Knowledge Representation and Reasoning {KR-2020}","start":{"date-parts":[[2020,9,12]]},"theme":"Artificial Intelligence","location":"Rhodes, Greece","end":{"date-parts":[[2020,9,18]]}},"container-title":["Proceedings of the Seventeenth International Conference on Principles of Knowledge Representation and Reasoning"],"original-title":[],"deposited":{"date-parts":[[2020,11,5]],"date-time":"2020-11-05T21:18:51Z","timestamp":1604611131000},"score":1,"resource":{"primary":{"URL":"https:\/\/proceedings.kr.org\/2020\/89"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2020,7]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/kr.2020\/89","relation":{},"subject":[],"published":{"date-parts":[[2020,7]]}}}