{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,5]],"date-time":"2026-10-05T01:43:48Z","timestamp":1791164628749,"version":"4.1.0"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","issue":"3","license":[{"start":{"date-parts":[[2015,3,5]],"date-time":"2015-03-05T00:00:00Z","timestamp":1425513600000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/www.springer.com\/tdm"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Auton Agent Multi-Agent Syst"],"published-print":{"date-parts":[[2016,5]]},"DOI":"10.1007\/s10458-015-9292-6","type":"journal-article","created":{"date-parts":[[2015,3,4]],"date-time":"2015-03-04T10:10:53Z","timestamp":1425463853000},"page":"403-445","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["Potential-based reward shaping for finite horizon online POMDP planning"],"prefix":"10.1007","volume":"30","author":[{"given":"Adam","family":"Eck","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leen-Kiat","family":"Soh","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sam","family":"Devlin","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Kudenko","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2015,3,5]]},"reference":[{"key":"9292_CR1","unstructured":"Araya-Lopez, M., Buffet, O., Thomas, V., & Charpillet, F. (2010). A POMDP extension with belief-dependent rewards. In Proceedings of the 24th Annual Conference on Neural Information Processing Systems (NIPS\u201910) (pp. 64\u201372). Vancouver, B.C., Canada, December 6\u20139, 2010."},{"key":"9292_CR2","unstructured":"Asmuth, J., Littman, M. L., & Zinkov, R. (2008). Potential-based shaping in model-based reinforcement learning. In Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI\u201908) (pp. 604\u2013609). Chicago, IL, July 13\u201317, 2008."},{"key":"9292_CR3","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1023\/A:1009634810396","volume":"5","author":"DP Bertsekas","year":"1999","unstructured":"Bertsekas, D. P., & Castanon, D. A. (1999). Rollout algorithms for stochastic scheduling problems. Journal of Heuristics, 5, 89\u2013108.","journal-title":"Journal of Heuristics"},{"key":"9292_CR4","unstructured":"Boutilier, C. (2002). A POMDP formulation of preference elicitation problems. In Proceedings of the 18th National Conference on Artificial Intelligence (AAAI\u201902) (pp. 239\u2013246). Edmonton, Alberta, Canada, July 28\u2013August 1, 2002."},{"key":"9292_CR5","doi-asserted-by":"crossref","DOI":"10.1017\/CBO9780511804441","volume-title":"Convex optimization","author":"SP Boyd","year":"2004","unstructured":"Boyd, S. P., & Vandenberghe, L. (2004). Convex optimization. Cambridge: Cambridge University Press."},{"key":"9292_CR6","unstructured":"Devlin, S., & Kudenko, D. (2011). Theoretical considerations of potential-based reward shaping for multi-agent systems. In K. Tumer, P. Yolum, L. Sonenberg, & P. Stone (Eds.), Proceedings of the 10th International Conference on Autonomous Agents and Multiagent Sytems (AAMAS\u201911) (pp. 225\u2013232). Taipei, Taiwan, May 2\u20136, 2011."},{"key":"9292_CR7","unstructured":"Devlin, S., & Kudenko, D. (2012). Dynamic potential-based reward shaping. In V. Conitzer, M. Winikoff, L. Padgham, & W. van der Hoek (Eds.), Proceedings of the 11th International Conference on Autonomous Agents and Multiagent Systems (AAMAS\u201912). Valencia, Spain, June 6\u20138, 2012."},{"key":"9292_CR8","unstructured":"Doshi, F., & Roy, N. (2008). The permutable POMDP: Fast solutions to POMDPs for preference elicitation. In L. Padgham, D. C. Parkes, J. Muller & S. Parsons (Eds.), Proceedings of the 7th International Conference on Autonomous Agents and Multiagent Systems (AAMAS\u201908) (pp. 493\u2013500). Estoril, Portugal, May 12\u201316, 2008."},{"key":"9292_CR9","unstructured":"Eck, A., Soh, L.-K., Devlin, S., & Kudenko, D. (2013). Potential-based reward shaping for POMDPs (Extended Abstract). In T. Ito, C. Jonker, M. Gini, & O. Shehory (Eds.), Proceedings of the 12th International Conference on Autonomous Agents and Multiagent Systems (AAMAS\u201913). Saint Paul, Minnesota, May 8\u201310, 2013."},{"key":"9292_CR10","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1613\/jair.678","volume":"13","author":"M Hauskrecht","year":"2000","unstructured":"Hauskrecht, M. (2000). Value-function approximations for partially observable Markov decision proceses. Journal of Artificial Intelligence Research, 13, 33\u201394.","journal-title":"Journal of Artificial Intelligence Research"},{"key":"9292_CR11","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1016\/S0004-3702(98)00023-X","volume":"101","author":"LP Kaelbling","year":"1998","unstructured":"Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101, 99\u2013134.","journal-title":"Artificial Intelligence"},{"key":"9292_CR12","doi-asserted-by":"crossref","unstructured":"Kurniawati, H., Hsu, D., & Lee, W. S. (2008). SARSOP: Efficient point-based POMDP planning by approximating optimally reachable belief spaces. In Proceedings of the 2008 Robotics: Science and Systems Conference (RSS \u201908).","DOI":"10.15607\/RSS.2008.IV.009"},{"key":"9292_CR13","unstructured":"Mihaylova, L. et al. (2002). Active sensing for robotics\u2014A survey. In Proceedings of the 5th International Conference on Numerical Methods and Applications (NM&A\u201902). Borovets, Bulgaria, August 20\u201324, 2002."},{"key":"9292_CR14","unstructured":"Ng, A. Y., Harada, D., & Russell, S. (1999). Policy invariance under reward transformations: Theory and application to reward shaping, In Proceedings of the 16th International Conference on Machine Learning (ICML\u201999) (pp. 278\u2013287). Bled, Slovenia, June 27\u201330, 1999."},{"issue":"8","key":"9292_CR15","doi-asserted-by":"crossref","first-page":"1053","DOI":"10.1177\/0278364910369861","volume":"29","author":"SCW Ong","year":"2010","unstructured":"Ong, S. C. W., Png, S. W., Hsu, D., & Lee, W. S. (2010). Planning under uncertainty for robotic tasks with mixed observability. International Journal of Robotics Research, 29(8), 1053\u20131068.","journal-title":"International Journal of Robotics Research"},{"key":"9292_CR16","unstructured":"Pineau, J., Gordon, G., & Thrun, S. (2003). Point-based value iteration: An anytime algorithm for POMDPs. In Proceedings of the 18th International Joint Conference on Artificial Intelligence (IJCAI\u201903) (pp. 1025\u20131032). Acapulco, Mexico, August 9\u201315, 2003."},{"key":"9292_CR17","unstructured":"Ross, S., & Chaib-draa, B. (2007). AEMS: An anytime online search algorithm for approximate policy refinement in large POMDPs. In Proceedings of the 20th International Joint Conference on Artificial Intelligence (IJCAI\u201907) (pp. 2592\u20132598). Hyderabad, India, January 6\u201312, 2007."},{"key":"9292_CR18","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1613\/jair.2567","volume":"32","author":"S Ross","year":"2008","unstructured":"Ross, S., Pineau, J., Paquet, S., & Chaib-draa, B. (2008). Online planning algorithms for POMDPs. Journal of Artificial Intelligence Research, 32, 663\u2013704.","journal-title":"Journal of Artificial Intelligence Research"},{"key":"9292_CR19","unstructured":"Silver, D., & Veness, J. (2010). Monte-Carlo planning in large POMDPs. In Proceedings of the 24th Annual Conference on Neural Information Processing Systems (NIPS\u201910) (pp. 2164\u20132172). Vancouver, B.C., Canada, December 6\u20139, 2010."},{"key":"9292_CR20","unstructured":"Smith, T., & Simmons, R. (2004). Heuristic search value iteration for POMDPs. In Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence (UAI\u201904) (pp. 520\u2013527). Banff, Alberta, Canada, July 7\u201311, 2004."},{"key":"9292_CR21","unstructured":"Somani, A., Ye, N., Hsu, D., & Sun Lee, W. (2013). DESPOT: Online POMDP planning with regularization. In Advances in Neural Information Processing Systems (NIPS) 2013."},{"key":"9292_CR22","doi-asserted-by":"crossref","unstructured":"Sorg, J., Singh, S., & Lewis, R. L. (2011). Optimal rewards versus leaf-evaluation heuristics in planning agents. In Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI\u201911) (pp. 465\u2013470). San Francisco, CA, August 7\u201311, 2011.","DOI":"10.1609\/aaai.v25i1.7931"},{"key":"9292_CR23","doi-asserted-by":"crossref","unstructured":"Spaan, M. T. J., Veiga, T. S., & Lima, P. U. (2010). Active cooperative perception in networked robotic systems using POMDPs. In Proceedings of the 2010 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS\u201910) (pp. 4800\u20134805). Taipei, Taiwan, October 18\u201322.","DOI":"10.1109\/IROS.2010.5648856"},{"key":"9292_CR24","doi-asserted-by":"crossref","first-page":"393","DOI":"10.1016\/j.csl.2006.06.008","volume":"21","author":"JD Williams","year":"2007","unstructured":"Williams, J. D., & Young, S. (2007). Partially observable Markov decision processes for spoken dialog systems. Computer Speech and Language, 21, 393\u2013422.","journal-title":"Computer Speech and Language"},{"key":"9292_CR25","unstructured":"Zhang, Z. & Chen, X. (2012). FHHOP: A factored heuristic online planning algorithm for POMDPs. In Proceedings of the 28th Conference on Uncertainty in Artificial Intelligence (UAI\u201912) (pp. 934\u2013943). Catalina Island, USA, August 15\u201317, 2012."}],"container-title":["Autonomous Agents and Multi-Agent Systems"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10458-015-9292-6.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1007\/s10458-015-9292-6\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1007\/s10458-015-9292-6","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,8]],"date-time":"2023-08-08T11:38:54Z","timestamp":1691494734000},"score":1,"resource":{"primary":{"URL":"http:\/\/link.springer.com\/10.1007\/s10458-015-9292-6"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2015,3,5]]},"references-count":25,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2016,5]]}},"alternative-id":["9292"],"URL":"https:\/\/doi.org\/10.1007\/s10458-015-9292-6","relation":{},"ISSN":["1387-2532","1573-7454"],"issn-type":[{"value":"1387-2532","type":"print"},{"value":"1573-7454","type":"electronic"}],"subject":[],"published":{"date-parts":[[2015,3,5]]}}}