{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,30]],"date-time":"2026-07-30T17:07:35Z","timestamp":1785431255740,"version":"3.56.0"},"reference-count":36,"publisher":"ASME International","issue":"4","license":[{"start":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T00:00:00Z","timestamp":1772150400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["asmedigitalcollection.asme.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,4,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Effective coordination is a critical challenge in the design of self-organizing systems (SOSs), particularly in decentralized training where explicit communication incurs high costs. Traditional approaches often rely on inter-agent communication, but this can introduce substantial system overhead. The challenge becomes even more pronounced in multiagent reinforcement learning (MARL)-based systems, where the training process happens in an end-to-end black-box manner. To address this issue, we explore alternative methods to enhance the exploration phase without relying on direct communication, thereby improving search efficiency. Therefore, the shared pool of information (SPI) is proposed in this article, which is a communication-free framework designed to provide agents with structured shared information at initialization. By offering a common foundation for exploration, SPI helps guide group action choices and facilitates more effective decision-making. This approach enables agents to learn coordinated behaviors without bringing the high costs associated with continuous communication. The efficiency of SPI is assessed and validated in the box-pushing problem, a task that requires agents to collaboratively maneuver a box toward a goal while avoiding obstacles. Our findings indicate that SPI accelerates learning, enhances coordination, increases success rates, and optimizes trajectory optimization. These results highlight SPI as a promising and scalable approach for scenarios where communication is costly or infeasible.<\/jats:p>","DOI":"10.1115\/1.4071239","type":"journal-article","created":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T16:34:11Z","timestamp":1772210051000},"update-policy":"https:\/\/doi.org\/10.1115\/crossmarkpolicy-asme","source":"Crossref","is-referenced-by-count":0,"title":["Efficient Training in Multiagent Reinforcement Learning: A Communication-Free Framework for the Box-Pushing Problem"],"prefix":"10.1115","volume":"26","author":[{"given":"David","family":"Ge","sequence":"first","affiliation":[{"id":[{"id":"https:\/\/ror.org\/00hj8s172","id-type":"ROR","asserted-by":"publisher"}],"name":"Columbia University Department of Computer Science, , New York, NY 10027"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hao","family":"Ji","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/03taz7m60","id-type":"ROR","asserted-by":"publisher"}],"name":"University of Southern California Center for Advanced Research Computing, , Los Angeles, CA 90089"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tsung-Jui","family":"Wu","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/03taz7m60","id-type":"ROR","asserted-by":"publisher"}],"name":"University of Southern California Department of Computer Science, , Los Angeles, CA 90089"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Bingling","family":"Huang","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/02avqqw26","id-type":"ROR","asserted-by":"publisher"}],"name":"California State University, Fullerton Department of Mechanical Engineering, ,","place":["Fullerton, CA 92831"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Quanchao","family":"Lu","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/01zkghx44","id-type":"ROR","asserted-by":"publisher"}],"name":"Georgia Institute of Technology School of Computer Science, , Atlanta, GA 30332"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shuo","family":"Shi","sequence":"additional","affiliation":[{"id":[{"id":"https:\/\/ror.org\/05abbep66","id-type":"ROR","asserted-by":"publisher"}],"name":"Brandeis University Michtom School of Computer Science, , Waltham, MA 02453"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"33","published-online":{"date-parts":[[2026,3,17]]},"reference":[{"issue":"2","key":"2026031712144579000_CIT0001","doi-asserted-by":"publisher","first-page":"256","DOI":"10.1134\/S106423071201008X","article-title":"Self-Organization and Multiagent Systems: I, Models of Multiagent Self-Organization","volume":"51","author":"Gorodetskii","year":"2012","journal-title":"J. Comput. Syst. Sci. Int."},{"issue":"1","key":"2026031712144579000_CIT0002","doi-asserted-by":"publisher","first-page":"102109","DOI":"10.1016\/j.aei.2023.102109","article-title":"Social Learning in Self-Organizing Systems for Complex Assembly Tasks","volume":"57","author":"Huang","year":"2023","journal-title":"Adv. Eng. Inform."},{"key":"2026031712144579000_CIT0003","first-page":"510","article-title":"Efficient and Effective Express via Contextual Cooperative Reinforcement Learning","author":"Li","year":"2019"},{"issue":"6","key":"2026031712144579000_CIT0004","doi-asserted-by":"publisher","first-page":"2192","DOI":"10.1109\/TNNLS.2018.2801880","article-title":"Distributed Economic Dispatch in Microgrids Based on Cooperative Reinforcement Learning","volume":"29","author":"Liu","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"2026031712144579000_CIT0005","first-page":"9904","article-title":"Multi-Agent Common Knowledge Reinforcement Learning","author":"de Witt","year":"2019"},{"issue":"1\u20132","key":"2026031712144579000_CIT0006","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000083","article-title":"Advances and Open Problems in Federated Learning","volume":"14","author":"Kairouz","year":"2021","journal-title":"Foundations and Trends in Machine Learning"},{"key":"2026031712144579000_CIT0007","doi-asserted-by":"crossref","DOI":"10.1115\/DETC2024-143326","article-title":"Impact of Social Learning on Teamwork Efficiency in Learning-Based Self-Organized Systems","author":"Huang","year":"2024"},{"key":"2026031712144579000_CIT0008","first-page":"746","article-title":"The Dynamics of Reinforcement Learning in Cooperative Multiagent Systems","author":"Claus","year":"1998"},{"issue":"1","key":"2026031712144579000_CIT0009","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1109\/TII.2017.2753408","article-title":"Enabling Cooperative Behavior for Building Demand Response Based on Extended Joint Action Learning","volume":"14","author":"Hurtado","year":"2018","journal-title":"IEEE Trans. Industr. Inform."},{"issue":"11","key":"2026031712144579000_CIT0010","doi-asserted-by":"publisher","first-page":"13677","DOI":"10.1007\/s10489-022-04105-y","article-title":"A Review of Cooperative Multi-Agent Deep Reinforcement Learning","volume":"53","author":"Oroojlooy","year":"2023","journal-title":"Appl. Intell."},{"key":"2026031712144579000_CIT0011","first-page":"3694","article-title":"Multi-Robot Box-Pushing: Single-Agent Q-Learning Versus Team Q-Learning","author":"Wang","year":"2006"},{"key":"2026031712144579000_CIT0012","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1007\/978-3-030-04792-4_6","volume-title":"Advances in Engineering Research and Application","author":"Rahimi","year":"2019"},{"key":"2026031712144579000_CIT0013","first-page":"119","article-title":"A New Object Path Planner for the Box Pushing Problem","author":"Parra-Gonzalez","year":"2009"},{"issue":"3\u20134","key":"2026031712144579000_CIT0014","doi-asserted-by":"publisher","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-Learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"2026031712144579000_CIT0015","first-page":"1182","article-title":"Adaptive Reinforcement Learning in Box-Pushing Robots","author":"Hwang","year":"2014"},{"key":"2026031712144579000_CIT0016","first-page":"4470","article-title":"Multi-Robot Cooperation Based on Continuous Reinforcement Learning With Two State Space Representations","author":"Yasuda","year":"2013"},{"key":"2026031712144579000_CIT0017","first-page":"1371","article-title":"An Object Transportation System With Multiple Robots and Machine Learning","author":"Wang","year":"2005"},{"key":"2026031712144579000_CIT0018","article-title":"An Introduction to Centralized Training for Decentralized Execution in Cooperative Multi-Agent Reinforcement Learning","author":"Amato","year":"2024"},{"key":"2026031712144579000_CIT0019","article-title":"Towards Efficient Collaboration via Graph Modeling in Reinforcement Learning","author":"Fan","year":"2024"},{"key":"2026031712144579000_CIT0020","article-title":"More Centralized Training, Still Decentralized Execution: Multi-Agent Conditional Policy Factorization","author":"Wang","year":"2023"},{"key":"2026031712144579000_CIT0021","article-title":"Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments","author":"Lowe","year":"2020"},{"key":"2026031712144579000_CIT0022","first-page":"2974","article-title":"Counterfactual Multi-Agent Policy Gradients","author":"Foerster","year":"2018"},{"key":"2026031712144579000_CIT0023","article-title":"Learning to Schedule Communication in Multi-Agent Reinforcement Learning","author":"Kim","year":"2019"},{"key":"2026031712144579000_CIT0024","first-page":"1801","article-title":"A New Framework for Multi-Agent Reinforcement Learning \u2013 Centralized Training and Exploration With Decentralized Execution via Policy Distillation","author":"Chen","year":"2020"},{"key":"2026031712144579000_CIT0025","first-page":"3540","article-title":"FeUdal Networks for Hierarchical Reinforcement Learning","author":"Vezhnevets","year":"2017"},{"issue":"4","key":"2026031712144579000_CIT0026","first-page":"95","article-title":"Survey on Recent Advances in Multiagent Reinforcement Learning Focusing on Decentralized Training With Decentralized Execution Framework","volume":"38","author":"Shin","year":"2023","journal-title":"Electron. Telecommun. Trends"},{"issue":"5","key":"2026031712144579000_CIT0027","doi-asserted-by":"publisher","first-page":"679","DOI":"10.1512\/iumj.1957.6.56038","article-title":"A Markovian Decision Process","volume":"6","author":"Bellman","year":"1957","journal-title":"J. Math. Mech."},{"key":"2026031712144579000_CIT0028","first-page":"1544","article-title":"Distributed Value Functions for Multi-Robot Exploration","author":"Matignon","year":"2012"},{"key":"2026031712144579000_CIT0029","doi-asserted-by":"crossref","DOI":"10.1145\/301136.301167","article-title":"General Principles of Learning-Based Multi-Agent Systems","author":"Wolpert","year":"1999"},{"key":"2026031712144579000_CIT0030","first-page":"489","article-title":"Learning to Cooperate via Policy Search","author":"Peshkin","year":"2000"},{"key":"2026031712144579000_CIT0031","first-page":"1523","article-title":"Multiagent Planning With Factored MDPs","author":"Guestrin","year":"2001"},{"key":"2026031712144579000_CIT0032","first-page":"436","article-title":"Improving Coordination With Communication in Multi-Agent Reinforcement Learning","author":"Szer","year":"2004"},{"key":"2026031712144579000_CIT0033","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2006.11438","article-title":"Deep Implicit Coordination Graphs for Multi-Agent Reinforcement Learning","author":"Li","year":"2021","journal-title":"arXiv"},{"issue":"1","key":"2026031712144579000_CIT0034","doi-asserted-by":"publisher","first-page":"101800","DOI":"10.1016\/j.aei.2022.101800","article-title":"Reward Shaping in Multiagent Reinforcement Learning for Self-Organizing Systems in Assembly Tasks","volume":"54","author":"Huang","year":"2022","journal-title":"Adv. Eng. Inform."},{"key":"2026031712144579000_CIT0035","author":"Team, P.","year":"2009"},{"issue":"2","key":"2026031712144579000_CIT0036","doi-asserted-by":"publisher","first-page":"021010","DOI":"10.1115\/1.4052800","article-title":"Knowledge Acquisition of Self-Organizing Systems With Deep Multiagent Reinforcement Learning","volume":"22","author":"Ji","year":"2022","journal-title":"ASME J. Comput. Inf. Sci. Eng."}],"container-title":["Journal of Computing and Information Science in Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4071239\/7597022\/jcise-25-1538.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article-pdf\/doi\/10.1115\/1.4071239\/7597022\/jcise-25-1538.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,17]],"date-time":"2026-03-17T16:14:52Z","timestamp":1773764092000},"score":1,"resource":{"primary":{"URL":"https:\/\/asmedigitalcollection.asme.org\/computingengineering\/article\/26\/4\/041002\/1231599\/Efficient-Training-in-Multiagent-Reinforcement"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,17]]},"references-count":36,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2026,4,1]]}},"URL":"https:\/\/doi.org\/10.1115\/1.4071239","relation":{},"ISSN":["1530-9827","1944-7078"],"issn-type":[{"value":"1530-9827","type":"print"},{"value":"1944-7078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,17]]},"article-number":"041002"}}