{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2024,8,7]],"date-time":"2024-08-07T07:40:14Z","timestamp":1723016414936},"publisher-location":"California","reference-count":0,"publisher":"International Joint Conferences on Artificial Intelligence Organization","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,9]]},"abstract":"<jats:p>Hierarchical policies are a key ingredient of intelligent behavior, expressing the different levels of abstraction involved in the solution of a problem. Learning hierarchical policies, however, remains a challenge, as no general learning principles have been identified for this purpose, despite the broad interest and vast literature in both model-free reinforcement learning and model-based planning. In this work, we introduce a principled method for learning hierarchical policies over classical planning domains, with no supervision from small instances. The method is based on learning to decompose problems into subproblems so that the subproblems have a lower complexity as measured by their width. Problems and subproblems are captured by means of sketch rules, and the scheme for reducing the width of sketch rules is applied iteratively until the final sketch rules have zero width and encode a general policy. We evaluate the learning method on a number of classical planning domains, analyze the resulting hierarchical policies, and prove their properties. We also show that learning hierarchical policies by learning and refining sketches iteratively is often more efficient than learning flat general policies in one shot.<\/jats:p>","DOI":"10.24963\/kr.2023\/21","type":"proceedings-article","created":{"date-parts":[[2023,7,31]],"date-time":"2023-07-31T22:27:47Z","timestamp":1690842467000},"page":"208-218","source":"Crossref","is-referenced-by-count":0,"title":["Learning Hierarchical Policies by Iteratively Reducing the Width of Sketch Rules"],"prefix":"10.24963","author":[{"given":"Dominik","family":"Drexler","sequence":"first","affiliation":[{"name":"Link\u00f6ping University"}]},{"given":"Jendrik","family":"Seipp","sequence":"additional","affiliation":[{"name":"Link\u00f6ping University"}]},{"given":"Hector","family":"Geffner","sequence":"additional","affiliation":[{"name":"RWTH Aachen University"},{"name":"Link\u00f6ping University"}]}],"member":"10584","event":{"number":"20","sponsor":["Artificial Intelligence Journal","Principles of Knowledge Representation and Reasoning Inc.","Academic College of Tel-Aviv","European Association for Artificial Intelligence","National Science Foundation"],"acronym":"KR-2023","name":"20th International Conference on Principles of Knowledge Representation and Reasoning {KR-2023}","start":{"date-parts":[[2023,9,2]]},"theme":"Artificial Intelligence","location":"Rhodes, Greece","end":{"date-parts":[[2023,9,8]]}},"container-title":["Proceedings of the Twentieth International Conference on Principles of Knowledge Representation and Reasoning"],"original-title":[],"deposited":{"date-parts":[[2023,7,31]],"date-time":"2023-07-31T22:28:08Z","timestamp":1690842488000},"score":1,"resource":{"primary":{"URL":"https:\/\/proceedings.kr.org\/2023\/21"}},"subtitle":[],"proceedings-subject":"Artificial Intelligence Research Articles","short-title":[],"issued":{"date-parts":[[2023,9]]},"references-count":0,"URL":"https:\/\/doi.org\/10.24963\/kr.2023\/21","relation":{},"subject":[],"published":{"date-parts":[[2023,9]]}}}