{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T09:11:02Z","timestamp":1780737062057,"version":"3.54.1"},"reference-count":38,"publisher":"Association for Computing Machinery (ACM)","issue":"4","funder":[{"name":"RIE2025 Industry Alignment Fund \u2013 Industry Collaboration Projects","award":["I2301E0026"],"award-info":[{"award-number":["I2301E0026"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Model. Comput. Simul."],"published-print":{"date-parts":[[2025,10,31]]},"abstract":"<jats:p>\n            In recent years, model-free multi-agent reinforcement learning (MaRL) has become a powerful tool for learning effective policies to solve optimization problems. However, individual agents may raise concerns about sharing their internal data, simulation models, and decision models in collaborative optimization. Distributed simulation (DS) and federated learning have been widely used as privacy-preserving methods to hide simulation details and maintain data and model privacy. Despite their benefits, these methods often require large amounts of interaction and data to converge, which leads to a high communication time, especially if the agents are distributed around the world. To address this issue, we propose a\n            <jats:italic toggle=\"yes\">distributed surrogate model<\/jats:italic>\n            for DS-based federated MaRL to utilize the surrogate model instead of DS during the training. This can enhance data efficiency and effectiveness to accelerate agent learning while maintaining data and model privacy. An aerospace supply chain (SC) is used as the experimental scenario to evaluate the performance of our proposed approach, in terms of SC profits, training convergence, and execution time. Experimental results show that our proposed approach can achieve higher SC profits with the same number of simulation runs, converge faster, and reduce execution time to gain the same level of SC profits.\n          <\/jats:p>","DOI":"10.1145\/3728466","type":"journal-article","created":{"date-parts":[[2025,5,15]],"date-time":"2025-05-15T07:09:19Z","timestamp":1747292959000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Privacy Meets Performance: Enhancing Distributed Simulation-based Federated Multi-agent Learning with Privacy-preserving Surrogate Model"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3831-1109","authenticated-orcid":false,"given":"Bo","family":"Zhang","sequence":"first","affiliation":[{"name":"College of Computing and Data Science, Nanyang Technological University","place":["Singapore, Singapore"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2204-0639","authenticated-orcid":false,"given":"Wen Jun","family":"Tan","sequence":"additional","affiliation":[{"name":"College of Computing and Data Science, Nanyang Technological University","place":["Singapore, Singapore"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0183-3835","authenticated-orcid":false,"given":"Wentong","family":"Cai","sequence":"additional","affiliation":[{"name":"College of Computing and Data Science, Nanyang Technological University","place":["Singapore, Singapore"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4795-5843","authenticated-orcid":false,"given":"Allan N.","family":"Zhang","sequence":"additional","affiliation":[{"name":"Singapore Institute of Manufacturing Technology (SIMTech), Agency for Science, Technology and Research (A*STAR)","place":["Singapore, Singapore"]},{"name":"School of Mechanical and Aerospace Engineering, Nanyang Technological University","place":["Singapore, Singapore"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,9,12]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1080\/00207543.2022.2098873"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.5555\/3306127.3331924"},{"key":"e_1_3_1_4_2","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1109\/PADS.2001.924622","volume-title":"Proceedings of the 15th Workshop on Parallel and Distributed Simulation","author":"Cai Wentong","year":"2001","unstructured":"Wentong Cai, Stephen J. Turner, and Boon Ping Gan. 2001. Hierarchical federations: An architecture for information hiding. In Proceedings of the 15th Workshop on Parallel and Distributed Simulation. IEEE, 67\u201374."},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cie.2011.10.003"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.apm.2020.09.037"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387107"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/2964284.2964301"},{"key":"e_1_3_1_9_2","first-page":"1","volume-title":"Proceedings of the 2019 USENIX Annual Technical Conference","author":"Duplyakin Dmitry","year":"2019","unstructured":"Dmitry Duplyakin, Robert Ricci, Aleksander Maricq, Gary Wong, Jonathon Duerig, Eric Eide, Leigh Stoller, Mike Hibler, David Johnson, Kirk Webb, et\u00a0al. 2019. The design and operation of \\(\\lbrace\\) CloudLab \\(\\rbrace\\) . In Proceedings of the 2019 USENIX Annual Technical Conference. 1\u201314."},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ress.2021.107923"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/2866577"},{"key":"e_1_3_1_12_2","doi-asserted-by":"crossref","first-page":"1245","DOI":"10.1109\/WSC.2000.899092","volume-title":"Proceedings of the 2000 Winter Simulation Conference Proceedings (Cat. No. 00CH37165)","author":"Gan Boon Ping","year":"2000","unstructured":"Boon Ping Gan, Li Liu, Sanjay Jain, Stephen John Turner, Wentong Cai, and Wen-Jing Hsu. 2000. Distributed supply chain simulation across enterprise boundaries. In Proceedings of the 2000 Winter Simulation Conference Proceedings (Cat. No. 00CH37165). IEEE, 1245\u20131251."},{"key":"e_1_3_1_13_2","unstructured":"Danijar Hafner Timothy Lillicrap Jimmy Ba and Mohammad Norouzi. 2020. Dream to Control: Learning Behaviors by Latent Imagination. In Proceedings of International Conference on Learning Representations."},{"key":"e_1_3_1_14_2","unstructured":"Danijar Hafner Timothy P. Lillicrap Mohammad Norouzi and Jimmy Ba. 2021. Mastering atari with discrete world models. In Proceedings of International Conference on Learning Representations."},{"key":"e_1_3_1_15_2","doi-asserted-by":"crossref","unstructured":"Danijar Hafner Jurgis Pasukonis Jimmy Ba and Timothy Lillicrap. 2025. Mastering diverse control tasks through world models. Nature (2025) 1\u20137.","DOI":"10.1038\/s41586-025-08744-2"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2024.3363615"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.2308\/isys-50692"},{"key":"e_1_3_1_18_2","first-page":"2961","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Iqbal Shariq","year":"2019","unstructured":"Shariq Iqbal and Fei Sha. 2019. Actor-attention-critic for multi-agent reinforcement learning. In Proceedings of the International Conference on Machine Learning. PMLR, 2961\u20132970."},{"key":"e_1_3_1_19_2","unstructured":"Roy H. Campbell Konrad Czechowski Dumitru Erhan Chelsea Finn Piotr Kozakowski Sergey Levine Afroz Mohiuddin Ryan Sepassi George Tucker and Henryk Michalewski. 2019. Model-based reinforcement learning for atari. In Proceedings of International Conference on Learning Representations. 6\u20139."},{"key":"e_1_3_1_20_2","unstructured":"Diederik P. Kingma and Max Welling. 2014. Auto-encoding variational bayes. In Proceedings of International Conference on Learning Representations Yoshua Bengio and Yann LeCun (Eds.)."},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ast.2022.107629"},{"key":"e_1_3_1_22_2","unstructured":"Yixin Liu Kai Zhang Yuan Li Zhiling Yan Chujie Gao Ruoxi Chen Zhengqing Yuan Yue Huang Hanchi Sun Jianfeng Gao et\u00a0al. 2024. Sora: A review on background technology limitations and opportunities of large vision models. arXiv:2402.17177. Retrieved from https:\/\/arxiv.org\/abs\/2402.17177"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3466170"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1177\/0037549708100530"},{"key":"e_1_3_1_25_2","article-title":"Action-conditional video prediction using deep networks in atari games","volume":"2","author":"Oh Junhyuk","year":"2015","unstructured":"Junhyuk Oh, Xiaoxiao Guo, Honglak Lee, Richard L. Lewis, and Satinder Singh. 2015. Action-conditional video prediction using deep networks in atari games. Advances in Neural Information Processing Systems 2 (2015), 2863\u20132871.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_26_2","unstructured":"Maarten G. Poirot Praneeth Vepakomma Ken Chang Jayashree Kalpathy-Cramer Rajiv Gupta and Ramesh Raskar. 2019. Split learning for collaborative deep learning in healthcare. arXiv:1912.12115. Retrieved from https:\/\/arxiv.org\/abs\/1912.12115"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","unstructured":"Zhiguang Qian Carolyn Conner Seepersad V. Roshan Joseph Janet K. Allen and C. F. Jeff Wu. 2005. Building surrogate models based on detailed and approximate simulations. Journal of Mechanical Design 128 4 (06 2005) 668\u2013677. DOI:10.1115\/1.2179459","DOI":"10.1115\/1.2179459"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"e_1_3_1_29_2","unstructured":"David Silver Thomas Hubert Julian Schrittwieser Ioannis Antonoglou Matthew Lai Arthur Guez Marc Lanctot Laurent Sifre Dharshan Kumaran Thore Graepel et\u00a0al. 2017. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv:1712.01815. Retrieved from https:\/\/arxiv.org\/abs\/1712.01815"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1108\/IJLM-06-2016-0150"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2022.3196503"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejor.2018.04.032"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-019-1724-z"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.1080\/00207543.2021.2020927"},{"key":"e_1_3_1_35_2","doi-asserted-by":"publisher","DOI":"10.1080\/00207543.2021.2023833"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3615979.3656055"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/WSC57314.2022.10015277"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.3390\/su16229996"},{"key":"e_1_3_1_39_2","first-page":"0","article-title":"Federated multi-agent reinforcement learning for improving visibility in supply chain inventory management","author":"Zhang Bo","year":"2025","unstructured":"Bo Zhang, Wen Jun Tan, Wentong Cai, and Allan N. Zhang. 2025. Federated multi-agent reinforcement learning for improving visibility in supply chain inventory management. International Journal of Production Research (Under submission) (2025), 0.","journal-title":"International Journal of Production Research (Under submission)"}],"container-title":["ACM Transactions on Modeling and Computer Simulation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3728466","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,9,12]],"date-time":"2025-09-12T11:50:10Z","timestamp":1757677810000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3728466"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,9,12]]},"references-count":38,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,10,31]]}},"alternative-id":["10.1145\/3728466"],"URL":"https:\/\/doi.org\/10.1145\/3728466","relation":{},"ISSN":["1049-3301","1558-1195"],"issn-type":[{"value":"1049-3301","type":"print"},{"value":"1558-1195","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,9,12]]},"assertion":[{"value":"2025-02-10","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-02","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-09-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}