{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T16:44:01Z","timestamp":1792169041255,"version":"4.4.0"},"reference-count":25,"publisher":"Maximum Academic Press","issue":"1","license":[{"start":{"date-parts":[[2016,2,11]],"date-time":"2016-02-11T00:00:00Z","timestamp":1455148800000},"content-version":"unspecified","delay-in-days":41,"URL":"https:\/\/www.cambridge.org\/core\/terms"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["The Knowledge Engineering Review"],"published-print":{"date-parts":[[2016,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Recent theoretical results have justified the use of potential-based reward shaping as a way to improve the performance of multi-agent reinforcement learning (MARL). However, the question remains of how to generate a useful potential function.<\/jats:p>\n                  <jats:p>Previous research demonstrated the use of STRIPS operator knowledge to automatically generate a potential function for single-agent reinforcement learning. Following up on this work, we investigate the use of STRIPS planning knowledge in the context of MARL.<\/jats:p>\n                  <jats:p>Our results show that a potential function based on joint or individual plan knowledge can significantly improve MARL performance compared with no shaping. In addition, we investigate the limitations of individual plan knowledge as a source of reward shaping in cases where the combination of individual agent plans causes conflict.<\/jats:p>","DOI":"10.1017\/s0269888915000181","type":"journal-article","created":{"date-parts":[[2016,2,11]],"date-time":"2016-02-11T20:33:12Z","timestamp":1455222792000},"page":"44-58","source":"Crossref","is-referenced-by-count":15,"title":["Plan-based reward shaping for multi-agent reinforcement learning"],"prefix":"10.48130","volume":"31","author":[{"given":"Sam","family":"Devlin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Kudenko","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"27968","published-online":{"date-parts":[[2016,2,11]]},"reference":[{"key":"S0269888915000181_ref7","unstructured":"Devlin S. & Kudenko D. 2011. Theoretical considerations of potential-based reward shaping for multi-agent systems. In Proceedings of the Tenth Annual International Conference on Autonomous Agents and Multiagent Systems (AAMAS)."},{"key":"S0269888915000181_ref11","doi-asserted-by":"crossref","unstructured":"Grze\u015b M. & Kudenko D. 2008a. Multigrid reinforcement learning with reward shaping. In Artificial Neural Networks-ICANN 5163, 357\u2013366. Lecture Notes in Computer Science, Springer.","DOI":"10.1007\/978-3-540-87536-9_37"},{"key":"S0269888915000181_ref5","unstructured":"De Hauwere Y. , Vrancx P. & Now\u00e9 A. 2011. Solving delayed coordination problems in mas (extended abstract). In The 10th International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 1115\u20131116."},{"key":"S0269888915000181_ref9","unstructured":"De Weerdt M. , Ter Mors A. & Witteveen C. 2005. Multi-agent planning - an introduction to planning and coordination. Technical report, Delft University of Technology."},{"key":"S0269888915000181_ref12","doi-asserted-by":"crossref","unstructured":"Grze\u015b M. & Kudenko D. 2008b. Plan-based reward shaping for reinforcement learning. In Proceedings of the 4th IEEE International Conference on Intelligent Systems (IS'08), 22\u201329. IEEE.","DOI":"10.1109\/IS.2008.4670492"},{"key":"S0269888915000181_ref22","doi-asserted-by":"publisher","DOI":"10.1016\/0004-3702(94)00007-N"},{"key":"S0269888915000181_ref21","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2006.02.006"},{"key":"S0269888915000181_ref14","doi-asserted-by":"crossref","unstructured":"Marthi B. 2007. Automatic shaping and decomposition of reward functions. In Proceedings of the 24th International Conference on Machine learning, 608. ACM.","DOI":"10.1145\/1273496.1273572"},{"key":"S0269888915000181_ref1","unstructured":"Asmuth J. , Littman M. & Zinkov R. 2008. Potential-based shaping in model-based reinforcement learning. In Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence, 604\u2013609."},{"key":"S0269888915000181_ref19","unstructured":"Randl\u00f8v J. & Alstrom P. 1998. Learning to drive a bicycle using reinforcement learning and shaping. In Proceedings of the 15th International Conference on Machine Learning, 463\u2013471."},{"key":"S0269888915000181_ref4","unstructured":"Claus C. & Boutilier C. 1998. The dynamics of reinforcement learning in cooperative multiagent systems. In Proceedings of the National Conference on Artificial Intelligence, 746\u2013752."},{"key":"S0269888915000181_ref20","unstructured":"Rosenschein J. 1982. Synchronization of multi-agent plans. In Proceedings of the National Conference on Artificial Intelligence, 115\u2013119."},{"key":"S0269888915000181_ref18","doi-asserted-by":"publisher","DOI":"10.1002\/9780470316887"},{"key":"S0269888915000181_ref23","unstructured":"Sutton R. S. 1984. Temporal credit assignment in reinforcement learning. PhD thesis, Department of Computer Science, University of Massachusetts."},{"key":"S0269888915000181_ref2","unstructured":"Babes M. , de Cote E. & Littman M. 2008. Social reward shaping in the prisoner\u2019s dilemma. In Proceedings of the Seventh Annual International Conference on Autonomous Agents and Multiagent Systems (AAMAS), 3, 1389\u20131392."},{"key":"S0269888915000181_ref3","volume-title":"Dynamic Programming and Optimal Control","author":"Bertsekas","year":"2007"},{"key":"S0269888915000181_ref6","doi-asserted-by":"publisher","DOI":"10.1142\/S0219525911002998"},{"key":"S0269888915000181_ref8","unstructured":"Devlin S. & Kudenko D. 2012. Dynamic potential-based reward shaping. In Proceedings of the Eleventh Annual International Conference on Autonomous Agents and Multiagent Systems (AAMAS)."},{"key":"S0269888915000181_ref17","doi-asserted-by":"crossref","unstructured":"Peot M. & Smith D. 1992. Conditional nonlinear planning. In Artificial Intelligence Planning Systems: Proceedings of the First International Conference, 189. Morgan Kaufmann Publisher.","DOI":"10.1016\/B978-0-08-049944-4.50027-6"},{"key":"S0269888915000181_ref13","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04921-7_37"},{"key":"S0269888915000181_ref10","unstructured":"Grze\u015b M. 2010. Improving exploration in reinforcement learning through domain knowledge and parameter analysis. Technical report, University of York."},{"key":"S0269888915000181_ref15","doi-asserted-by":"publisher","DOI":"10.2307\/1969529"},{"key":"S0269888915000181_ref24","volume-title":"Reinforcement Learning: An Introduction","author":"Sutton","year":"1998"},{"key":"S0269888915000181_ref16","unstructured":"Ng A. Y. , Harada D. & Russell S. J. 1999. Policy invariance under reward transformations: theory and application to reward shaping. In Proceedings of the 16th International Conference on Machine Learning, 278\u2013287."},{"key":"S0269888915000181_ref25","unstructured":"Ziparo V. 2005. Multi-agent planning. Technical report, University of Rome."}],"container-title":["The Knowledge Engineering Review"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.cambridge.org\/core\/services\/aop-cambridge-core\/content\/view\/S0269888915000181","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T14:42:05Z","timestamp":1767624125000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.cambridge.org\/core\/product\/identifier\/S0269888915000181\/type\/journal_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2016,1]]},"references-count":25,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2016,1]]}},"alternative-id":["S0269888915000181"],"URL":"https:\/\/doi.org\/10.1017\/s0269888915000181","relation":{},"ISSN":["0269-8889","1469-8005"],"issn-type":[{"value":"0269-8889","type":"print"},{"value":"1469-8005","type":"electronic"}],"subject":[],"published":{"date-parts":[[2016,1]]}}}