{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T09:52:39Z","timestamp":1777715559045,"version":"3.51.4"},"reference-count":64,"publisher":"SAGE Publications","issue":"2","license":[{"start":{"date-parts":[[2018,3,13]],"date-time":"2018-03-13T00:00:00Z","timestamp":1520899200000},"content-version":"vor","delay-in-days":405,"URL":"http:\/\/www.sagepub.com\/licence-information-for-chorus"}],"funder":[{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"publisher","award":["MURI Grant N000141110688"],"award-info":[{"award-number":["MURI Grant N000141110688"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000003","name":"Boeing","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100000003","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["The International Journal of Robotics Research"],"published-print":{"date-parts":[[2017,2]]},"abstract":"<jats:p>This work focuses on solving general multi-robot planning problems in continuous spaces with partial observability given a high-level domain description. Decentralized Partially Observable Markov Decision Processes (Dec-POMDPs) are general models for multi-robot coordination problems. However, representing and solving Dec-POMDPs is often intractable for large problems. This work extends the Dec-POMDP model to the Decentralized Partially Observable Semi-Markov Decision Process (Dec-POSMDP) to take advantage of the high-level representations that are natural for multi-robot problems and to facilitate scalable solutions to large discrete and continuous problems. The Dec-POSMDP formulation uses task macro-actions created from lower-level local actions that allow for asynchronous decision-making by the robots, which is crucial in multi-robot domains. This transformation from Dec-POMDPs to Dec-POSMDPs with a finite set of automatically-generated macro-actions allows use of efficient discrete-space search algorithms to solve them. The paper presents algorithms for solving Dec-POSMDPs, which are more scalable than previous methods since they can incorporate closed-loop belief space macro-actions in planning. These macro-actions are automatically constructed to produce robust solutions. The proposed algorithms are then evaluated on a complex multi-robot package delivery problem under uncertainty, showing that our approach can naturally represent realistic problems and provide high-quality solutions for large-scale problems.<\/jats:p>","DOI":"10.1177\/0278364917692864","type":"journal-article","created":{"date-parts":[[2017,3,13]],"date-time":"2017-03-13T09:57:35Z","timestamp":1489399055000},"page":"231-258","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":47,"title":["Decentralized control of multi-robot partially observable Markov decision processes using belief space macro-actions"],"prefix":"10.1177","volume":"36","author":[{"given":"Shayegan","family":"Omidshafiei","sequence":"first","affiliation":[{"name":"Laboratory for Information and Decision Systems (LIDS), MIT, Cambridge, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ali\u2013Akbar","family":"Agha\u2013Mohammadi","sequence":"additional","affiliation":[{"name":"Qualcomm-Research Center, San Diego, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christopher","family":"Amato","sequence":"additional","affiliation":[{"name":"Department of Computer Science at the University of New Hampshire, Durham, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shih\u2013Yuan","family":"Liu","sequence":"additional","affiliation":[{"name":"Laboratory for Information and Decision Systems (LIDS), MIT, Cambridge, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jonathan P","family":"How","sequence":"additional","affiliation":[{"name":"Laboratory for Information and Decision Systems (LIDS), MIT, Cambridge, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"John","family":"Vian","sequence":"additional","affiliation":[{"name":"Boeing Research & Technology, Seattle, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2017,3,13]]},"reference":[{"key":"bibr1-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2011.6095010"},{"key":"bibr2-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913501564"},{"key":"bibr3-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2014.6943034"},{"key":"bibr4-0278364917692864","doi-asserted-by":"crossref","unstructured":"Alighanbari M (2004) Task assignment algorithms for teams of UAVs in dynamic environments. Master\u2019s Thesis, Massachusetts Institute of Technology, USA.","DOI":"10.2514\/6.2004-5251"},{"key":"bibr5-0278364917692864","first-page":"593","volume-title":"International conference on autonomous agents and multiagent systems","author":"Amato C","year":"2009"},{"key":"bibr6-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-009-9103-z"},{"key":"bibr7-0278364917692864","doi-asserted-by":"publisher","DOI":"10.15607\/RSS.2015.XI.007"},{"key":"bibr8-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7139350"},{"key":"bibr9-0278364917692864","first-page":"1273","volume-title":"International conference on autonomous agents and multiagent systems","author":"Amato C","year":"2014"},{"key":"bibr10-0278364917692864","doi-asserted-by":"publisher","DOI":"10.3182\/20120914-2-US-4030.00029"},{"key":"bibr11-0278364917692864","unstructured":"Banker S (2013) Amazon and drones \u2013 here is why it will work. Available at: http:\/\/www.forbes.com\/sites\/stevebanker\/2013\/12\/19\/amazon-drones-here-is-why-it-will-work\/ (Accessed May 2015)."},{"key":"bibr12-0278364917692864","first-page":"23","volume-title":"Proceedings of conference of cooperative control and optimization","author":"Bellingham J","year":"2001"},{"key":"bibr13-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1613\/jair.2667"},{"key":"bibr14-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1287\/moor.27.4.819.297"},{"key":"bibr15-0278364917692864","volume-title":"Proceedings of the international symposium on artificial intelligence, robotics and automation in space","author":"Bernstein DS","year":"2001"},{"key":"bibr16-0278364917692864","first-page":"13","volume-title":"Decision and control, 2003","author":"Castanon D","year":"2003"},{"key":"bibr17-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1109\/TRO.2009.2022423"},{"key":"bibr18-0278364917692864","doi-asserted-by":"crossref","unstructured":"Cutler M, How JP (2012) Actuator constrained trajectory generation and control for variable-pitch quadrotors. In: AIAA guidance, navigation, and control conference, Minneapolis, USA, pp.4777. Reston, USA: AIAA. Available at: http:\/\/acl.mit.edu\/papers\/2012-uber-compressed.pdf.","DOI":"10.2514\/6.2012-4777"},{"key":"bibr19-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1109\/ROBOT.2005.1570273"},{"key":"bibr20-0278364917692864","first-page":"1523","volume-title":"Advances in neural information processing systems 14","author":"Guestrin C","year":"2001"},{"key":"bibr21-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1613\/jair.3171"},{"key":"bibr22-0278364917692864","volume-title":"Dynamic Probabilistic Systems Volume 2, Semi-Markov and Decision Processes","author":"Howard RA","year":"1971"},{"key":"bibr23-0278364917692864","first-page":"7","volume-title":"IEEE conference on decision and control","author":"Jin Y","year":"2003"},{"key":"bibr24-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(98)00023-X"},{"key":"bibr25-0278364917692864","doi-asserted-by":"crossref","unstructured":"Kaelbling LP, Lozano\u2013P\u00e9rez T (2012) Integrated task and motion planning in belief space. The International Journal of Robotics Research 32(9\u201310): 1194\u20131227. Available at: http:\/\/people.csail.mit.edu\/lpk\/papers\/HPNBelDraft.pdf","DOI":"10.1177\/0278364913484072"},{"key":"bibr26-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1177\/0278364913495721"},{"key":"bibr27-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1613\/jair.4649"},{"key":"bibr28-0278364917692864","volume-title":"Stochastic Systems: Estimation, Identification, and Adaptive Control","author":"Kumar PR","year":"1986"},{"key":"bibr29-0278364917692864","first-page":"1287","volume-title":"Advances in Neural Information Processing Systems 24","author":"Lim Z","year":"2011"},{"key":"bibr30-0278364917692864","first-page":"1181","volume-title":"AAAI","author":"Little I","year":"2005"},{"key":"bibr31-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1007\/s10514-012-9303-2"},{"key":"bibr32-0278364917692864","unstructured":"Liu M (2016) Efficient Bayesian nonparametric methods for model-free reinforcement learning in centralized and decentralized sequential environments. PhD Thesis, Department of Electrical and Computer Engineering, Duke University, USA."},{"key":"bibr33-0278364917692864","first-page":"100","volume-title":"Advances in Neural Information Processing Systems","author":"MacDermed LC","year":"2013"},{"key":"bibr34-0278364917692864","first-page":"541","volume-title":"Proceedings of the 16th conference on artificial intelligence (AAAI)","author":"Madani O","year":"1999"},{"key":"bibr35-0278364917692864","first-page":"113","volume-title":"AAAI fall symposium on planning with partially observable Markov decision processes","author":"Mahadevan S","year":"1998"},{"key":"bibr36-0278364917692864","first-page":"202","volume-title":"Proceedings of the 14th international conference on machine learning","author":"Mahadevan S","year":"1997"},{"key":"bibr37-0278364917692864","first-page":"716","volume-title":"AAAI","author":"Mausam","year":"2004"},{"key":"bibr38-0278364917692864","first-page":"120","volume-title":"International conference on automated planning and scheduling","author":"Mausam","year":"2005"},{"key":"bibr39-0278364917692864","doi-asserted-by":"publisher","DOI":"10.2514\/1.5791"},{"key":"bibr40-0278364917692864","first-page":"1151","volume-title":"Proceedings of the international conference on artificial intelligence (IJCAI)","author":"Montemerlo M","year":"2003"},{"key":"bibr41-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1109\/TCST.2011.2167331"},{"key":"bibr42-0278364917692864","doi-asserted-by":"publisher","DOI":"10.5117\/9789056296100"},{"key":"bibr43-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-27645-3_15"},{"key":"bibr44-0278364917692864","first-page":"341","volume":"32","author":"Oliehoek FA","year":"2008","journal-title":"Informatica"},{"key":"bibr45-0278364917692864","doi-asserted-by":"publisher","DOI":"10.2514\/6.2015-0643"},{"key":"bibr46-0278364917692864","doi-asserted-by":"crossref","unstructured":"Omidshafiei S, Agha\u2013Mohammadi A, Amato C, (2014) Decentralized control of partially observable Markov decision processes using belief space macro-actions. Technical Report, Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, USA.","DOI":"10.1109\/ICRA.2015.7140035"},{"key":"bibr47-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2015.7140035"},{"key":"bibr48-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1109\/ICRA.2016.7487751"},{"key":"bibr49-0278364917692864","volume-title":"Robotics: Science and Systems","author":"Platt R","year":"2010"},{"key":"bibr50-0278364917692864","doi-asserted-by":"crossref","unstructured":"Ponda SS, Johnson LB, How JP (2012) Distributed chance-constrained task allocation for autonomous multi-agent teams. In: American control conference, Montreal, Canada, 27\u201329 June, pp. 4528\u20134533. Piscataway, USA: IEEE. Available at: http:\/\/acl.mit.edu\/papers\/ACC2012_ChanceConstrainedCBBA_final_submitted.pdf .","DOI":"10.1109\/ACC.2012.6315626"},{"key":"bibr51-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1002\/9780470316887"},{"key":"bibr52-0278364917692864","first-page":"1619","volume-title":"NIPS","author":"Rohanimanesh K","year":"2002"},{"key":"bibr53-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4757-4321-0"},{"key":"bibr54-0278364917692864","first-page":"343","volume-title":"International conference on machine learning (ICML)","author":"Schaul T","year":"2013"},{"key":"bibr55-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1109\/ACC.2002.1023915"},{"key":"bibr56-0278364917692864","first-page":"2009","volume-title":"Proceedings of the 20th international joint conference on artificial intelligence (IJCAI)","author":"Seuken S","year":"2007"},{"key":"bibr57-0278364917692864","first-page":"326","volume-title":"Proceedings of international joint conference in artificial intelligence","author":"Smith DE","year":"1999"},{"key":"bibr58-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(99)00052-1"},{"key":"bibr59-0278364917692864","first-page":"775","volume-title":"Advances in neural information processing systems 16 (NIPS03)","author":"Theocharous G","year":"2004"},{"key":"bibr60-0278364917692864","volume-title":"Probabilistic Robotics","author":"Thrun S","year":"2005"},{"key":"bibr61-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1109\/CDC.2004.1429424"},{"key":"bibr62-0278364917692864","volume-title":"Decision Making Under Uncertainty: Theory and Application (MIT Lincoln Laboratory Series)","author":"Ure NK","year":"2013"},{"key":"bibr63-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1287\/opre.24.2.348"},{"key":"bibr64-0278364917692864","doi-asserted-by":"publisher","DOI":"10.1145\/2486001.2486020"}],"container-title":["The International Journal of Robotics Research"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364917692864","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/0278364917692864","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364917692864","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/0278364917692864","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T10:15:30Z","timestamp":1777457730000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/0278364917692864"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,2]]},"references-count":64,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2017,2]]}},"alternative-id":["10.1177\/0278364917692864"],"URL":"https:\/\/doi.org\/10.1177\/0278364917692864","relation":{},"ISSN":["0278-3649","1741-3176"],"issn-type":[{"value":"0278-3649","type":"print"},{"value":"1741-3176","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,2]]}}}