{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,20]],"date-time":"2025-12-20T22:03:48Z","timestamp":1766268228262,"version":"build-2065373602"},"reference-count":38,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2019,3,19]],"date-time":"2019-03-19T00:00:00Z","timestamp":1552953600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Huaimin Wang","award":["61751208, 61502510, 61773390, 2017JJ1001,no.41412050202"],"award-info":[{"award-number":["61751208, 61502510, 61773390, 2017JJ1001,no.41412050202"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In a decentralized multi-robot exploration problem, the robots have to cooperate effectively to map a strange environment as soon as possible without a centralized controller. In the past few decades, a set of \u201chuman-designed\u201d cooperation strategies have been proposed to address this problem, such as the well-known frontier-based approach. However, many real-world settings, especially the ones that are constantly changing, are too complex for humans to design efficient and decentralized strategies. This paper presents a novel approach, the Attention-based Communication neural network (CommAttn), to \u201clearn\u201d the cooperation strategies automatically in the decentralized multi-robot exploration problem. The communication neural network enables the robots to learn the cooperation strategies with explicit communication. Moreover, the attention mechanism we introduced additionally can precisely calculate whether the communication is necessary for each pair of agents by considering the relevance of each received message, which enables the robots to communicate only with the necessary partners. The empirical results on a simulated multi-robot disaster exploration scenario demonstrate that our proposal outperforms the traditional \u201chuman-designed\u201d methods, as well as other competing \u201clearning-based\u201d methods in the exploration task.<\/jats:p>","DOI":"10.3390\/e21030294","type":"journal-article","created":{"date-parts":[[2019,3,19]],"date-time":"2019-03-19T12:12:25Z","timestamp":1552997545000},"page":"294","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Learning to Cooperate via an Attention-Based Communication Neural Network in Decentralized Multi-Robot Exploration"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7239-1819","authenticated-orcid":false,"given":"Mingyang","family":"Geng","sequence":"first","affiliation":[{"name":"National Key Laboratory of Parallel and Distributed Processing, College of Computer, National University of Defense Technology, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5997-5169","authenticated-orcid":false,"given":"Kele","family":"Xu","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Parallel and Distributed Processing, College of Computer, National University of Defense Technology, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xing","family":"Zhou","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Parallel and Distributed Processing, College of Computer, National University of Defense Technology, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bo","family":"Ding","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Parallel and Distributed Processing, College of Computer, National University of Defense Technology, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huaimin","family":"Wang","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Parallel and Distributed Processing, College of Computer, National University of Defense Technology, Changsha 410073, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lei","family":"Zhang","sequence":"additional","affiliation":[{"name":"National Key Laboratory of Integrated Automation of Process Industry, Northeastern University, Shenyang 110000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,3,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1007\/s10514-012-9298-8","article-title":"A comparison of path planning strategies for autonomous exploration and mapping of unknown environments","volume":"33","author":"Gil","year":"2012","journal-title":"Auton. Robot."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"655","DOI":"10.1109\/TRA.2002.806024","article-title":"Advances in multi-robot systems","volume":"18","author":"Arai","year":"2002","journal-title":"IEEE Trans. Robot. Autom."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Peng, Z., Zhang, L., and Luo, T. (2018, January 21\u201323). Learning to Communicate via Supervised Attentional Message Processing. Proceedings of the 31st International Conference on Computer Animation and Social Agents, Beijing, China.","DOI":"10.1145\/3205326.3205346"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Khamis, A., Hussein, A., and Elmogy, A. (2015). Multi-robot task allocation: A review of the state-of-the-art. Cooperative Robots and Sensor Networks 2015, Springer.","DOI":"10.1007\/978-3-319-18299-5_2"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Geng, M., Li, Y., Ding, B., and Wang, H. (2018, January 8\u201313). Deep Learning-based Cooperative Trail Following for Multi-Robot System. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.","DOI":"10.1109\/IJCNN.2018.8489239"},{"key":"ref_6","unstructured":"Tai, L., Zhang, J., Liu, M., Boedecker, J., and Burgard, W. (arXiv, 2016). A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation, arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Geng, M., Zhou, X., Ding, B., Wang, H., and Zhang, L. (2018, January 13\u201316). Learning to Cooperate in Decentralized Multi-robot Exploration of Dynamic Environments. Proceedings of the International Conference on Neural Information Processing, Siem Reap, Cambodia.","DOI":"10.1007\/978-3-030-04239-4_4"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Geng, M., Liu, S., and Wu, Z. (2019). Sensor Fusion-Based Cooperative Trail Following for Autonomous Multi-Robot System. Sensors, 19.","DOI":"10.3390\/s19040823"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"4972","DOI":"10.1109\/TIE.2016.2636810","article-title":"Recent advances in consensus of multi-agent systems: A brief survey","volume":"64","author":"Qin","year":"2017","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_10","unstructured":"Foerster, J.N., Assael, Y.M., de Freitas, N., and Whiteson, S. (arXiv, 2016). Learning to communicate to solve riddles with deep distributed recurrent q-networks, arXiv."},{"key":"ref_11","unstructured":"Peng, P., Yuan, Q., Wen, Y., Yang, Y., Tang, Z., Long, H., and Wang, J. (arXiv, 2017). Multiagent bidirectionally-coordinated nets for learning to play starcraft combat games, arXiv."},{"key":"ref_12","unstructured":"Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, O.P., and Mordatch, I. (2017, January 4\u20139). Multi-agent actor-critic for mixed cooperative-competitive environments. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_13","unstructured":"Sukhbaatar, S., Szlam, A., and Fergus, R. (2016, January 5\u201310). Learning multiagent communication with backpropagation. Proceedings of the Thirtieth Conference on Neural Information Processing Systems (NIPS 2016), Barcelona, Spain."},{"key":"ref_14","unstructured":"Hoshen, Y. (2017, January 4\u20139). Vain: Attentional multi-agent predictive modeling. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2673","DOI":"10.1109\/78.650093","article-title":"Bidirectional Recurrent Neural Networks","volume":"45","author":"Schuster","year":"1997","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Yamauchi, B. (1998, January 10\u201313). Frontier-based exploration using multiple robots. Proceedings of the Second International Conference on Autonomous Agents, Minneapolis, MN, USA.","DOI":"10.1145\/280765.280773"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1177\/0278364913515309","article-title":"Coordination strategies for multi-robot exploration and mapping","volume":"33","author":"Rogers","year":"2014","journal-title":"Int. J. Robot. Res."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Visser, A., and Slamet, B.A. (2008). Balancing the information gain against the movement cost for multi-robot frontier exploration. European Robotics Symposium 2008, Springer.","DOI":"10.1007\/978-3-540-78317-6_5"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1815","DOI":"10.1007\/s11999-014-3964-4","article-title":"Current status of cost utility analyses in total joint arthroplasty: A systematic review","volume":"473","author":"Nwachukwu","year":"2015","journal-title":"Clin. Orthop. Relat. Res."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Charrow, B., Liu, S., Kumar, V., and Michael, N. (2015, January 26\u201330). Information-theoretic mapping using cauchy-schwarz quadratic mutual information. Proceedings of the 2015 IEEE International Conference on Robotics and Automation (ICRA), Seattle, WA, USA.","DOI":"10.1109\/ICRA.2015.7139865"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Jadidi, M.G., Miro, J.V., and Dissanayake, G. (1998, January 10\u201313). Mutual information-based exploration on continuous occupancy maps. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Minneapolis, MN, USA.","DOI":"10.1109\/IROS.2015.7354244"},{"key":"ref_22","unstructured":"Best, G., Cliff, O., Patten, T., Mettu, R., and Fitch, R. (2016, January 18\u201320). Decentralised Monte Carlo tree search for active perception. Proceedings of the International Workshop on the Algorithmic Foundations of Robotics, San Francisco, CA, USA."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.robot.2016.06.008","article-title":"Planning for robotic exploration based on forward simulation","volume":"83","author":"Lauri","year":"2016","journal-title":"Robot. Auton. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Tabib, W., Corah, M., Michael, N., and Whittaker, R. (2016, January 9\u201314). Computationally efficient information-theoretic exploration of pits and caves. Proceedings of the 2016 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, Korea.","DOI":"10.1109\/IROS.2016.7759548"},{"key":"ref_25","unstructured":"Sariel, S., and Balch, T. (2005, January 9\u201310). Real time auction based allocation of tasks for multi-robot exploration problem in dynamic environments. Proceedings of the Workshop on Integrating Planning into Scheduling (AAAI-05), Palo Alto, CA, USA."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Lagoudakis, M.G., Markakis, E., Kempe, D., Keskinocak, P., Kleywegt, A.J., Koenig, S., Tovey, C.A., Meyerson, A., and Jain, S. (2005, January 8\u201311). Auction-Based Multi-Robot Routing. Proceedings of the Robotics: Science and Systems, Cambridge, MA, USA.","DOI":"10.15607\/RSS.2005.I.045"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Tovey, C., Lagoudakis, M.G., Jain, S., and Koenig, S. (2005). The generation of bidding rules for auction-based robot coordination. Multi-Robot Systems. From Swarms to Intelligent Automata Volume III, Springer.","DOI":"10.1007\/1-4020-3389-3_1"},{"key":"ref_28","unstructured":"Koenig, S., Keskinocak, P., and Tovey, C.A. (2010, January 11\u201315). Progress on Agent Coordination with Cooperative Auctions. Proceedings of the Twenty-Fourth AAAI Conference on Artificial Intelligence (AAAI-10), Atlanta, GA, USA."},{"key":"ref_29","unstructured":"Koenig, S., Tovey, C., Lagoudakis, M., Markakis, V., Kempe, D., Keskinocak, P., Kleywegt, A., Meyerson, A., and Jain, S. (2006, January 16\u201320). The power of sequential single-item auctions for agent coordination. Proceedings of the National Conference on Artificial Intelligence, Boston, MA, USA."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Colby, M., Chung, J.J., and Tumer, K. (October, January 28). Implicit adaptive multi-robot coordination in dynamic environments. Proceedings of the 2015 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Hamburg, Germany.","DOI":"10.1109\/IROS.2015.7354105"},{"key":"ref_31","unstructured":"Omidshafiei, S., Pazis, J., Amato, C., How, J.P., and Vian, J. (2017, January 6\u201311). Deep decentralized multi-task multi-agent rl under partial observability. Proceedings of the International Conference on Machine Learning, Sydney, Australia."},{"key":"ref_32","unstructured":"Dilokthanakul, N., Kaplanis, C., Pawlowski, N., and Shanahan, M. (arXiv, 2017). Feature Control as Intrinsic Motivation for Hierarchical Reinforcement Learning, arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"229","DOI":"10.1007\/BF00992696","article-title":"Simple statistical gradient-following algorithms for connectionist reinforcement learning","volume":"8","author":"Williams","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_34","unstructured":"Barratt, S. (arXiv, 2017). Active Robotic Mapping through Deep Reinforcement Learning, arXiv."},{"key":"ref_35","unstructured":"Sukhbaatar, S., Szlam, A., Synnaeve, G., Chintala, S., and Fergus, R. (arXiv, 2015). Mazebase: A sandbox for learning from games, arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"829","DOI":"10.1177\/0278364902021010834","article-title":"Navigation strategies for exploring indoor environments","volume":"21","author":"Latombe","year":"2002","journal-title":"Int. J. Robot. Res."},{"key":"ref_37","unstructured":"Bahdanau, D., Cho, K., and Bengio, Y. (arXiv, 2014). Neural machine translation by jointly learning to align and translate, arXiv."},{"key":"ref_38","first-page":"2579","article-title":"Visualizing Data using t-SNE","volume":"9","author":"Maaten","year":"2008","journal-title":"J. Mach. Learn. Res."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/3\/294\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:38:53Z","timestamp":1760186333000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/3\/294"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,3,19]]},"references-count":38,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2019,3]]}},"alternative-id":["e21030294"],"URL":"https:\/\/doi.org\/10.3390\/e21030294","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2019,3,19]]}}}