{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,19]],"date-time":"2025-10-19T15:44:09Z","timestamp":1760888649058,"version":"build-2065373602"},"reference-count":21,"publisher":"Fuji Technology Press Ltd.","issue":"5","funder":[{"DOI":"10.13039\/501100001691","name":"Japan Society for the Promotion of Science","doi-asserted-by":"publisher","award":["21H05104a"],"award-info":[{"award-number":["21H05104a"]}],"id":[{"id":"10.13039\/501100001691","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002241","name":"Japan Science and Technology Agency","doi-asserted-by":"publisher","award":["JPMJPS2032","JPMJAX22A9"],"award-info":[{"award-number":["JPMJPS2032","JPMJAX22A9"]}],"id":[{"id":"10.13039\/501100002241","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JRM","J. Robot. Mechatron."],"published-print":{"date-parts":[[2025,10,20]]},"abstract":"<jats:p>In recent years, swarm robotic systems are expected to be used in open environments such as disaster sites.   Considering applications in open environments, systems are required   to maintain functionality even when the number of robots changes or communication among them is unstable.   On the other hand, the design methodology for systems that take these constraints into account has not been sufficiently discussed.   Our previous research proposed the use of the transformer encoder (self-attention mechanism), enabling the design of a neural network that remains effective even when the amount of observation data available to each robot varies.   However, that work did not include validation in real-world environments, and it remains unclear whether the cooperative behaviors learned using the transformer encoder can be effectively applied outside of simulation.   To address this, this paper aims to investigate the feasibility of sim-to-real transfer and examines the differences between the simulation and real-world environments.   After training a policy model for cooperative object transport in a simulation environment, we conduct real-world experiments using TurtleBot3 Burgers to evaluate its applicability, adaptability to changes in robot numbers without retraining, and differences between simulation and real-world environments.<\/jats:p>","DOI":"10.20965\/jrm.2025.p1034","type":"journal-article","created":{"date-parts":[[2025,10,19]],"date-time":"2025-10-19T15:02:06Z","timestamp":1760886126000},"page":"1034-1041","source":"Crossref","is-referenced-by-count":0,"title":["Real-World Validation of a Learned Policy for Cooperative Object Transport with the Transformer Encoder"],"prefix":"10.20965","volume":"37","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5716-6365","authenticated-orcid":true,"given":"Yuichiro","family":"Sueoka","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering, Graduate School of Engineering, The University of Osaka, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Seiya","family":"Ozaki","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Graduate School of Engineering, The University of Osaka, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yuki","family":"Kato","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Graduate School of Engineering, The University of Osaka, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Takahiro","family":"Yoshida","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering, Graduate School of Engineering, The University of Osaka, 2-1 Yamadaoka, Suita, Osaka 565-0871, Japan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"8550","published-online":{"date-parts":[[2025,10,20]]},"reference":[{"key":"key-10.20965\/jrm.2025.p1034-1","doi-asserted-by":"crossref","unstructured":"E. \u015eahin, \u201cSwarm robotics: From sources of inspiration to domains of application,\u201d E. \u015eahin and W. M. Spears (Eds.), \u201cSwarm Robotics. SR 2004. Lecture Notes in Computer Science, Vol.3342,\u201d pp. 10-20, 2005. https:\/\/doi.org\/10.1007\/978-3-540-30552-1_2","DOI":"10.1007\/978-3-540-30552-1_2"},{"key":"key-10.20965\/jrm.2025.p1034-2","doi-asserted-by":"crossref","unstructured":"J. Werfel, K. Petersen, and R. Nagpal, \u201cDesigning collective behavior in a termite-inspired robot construction team,\u201d Science, Vol.343, No.6172, pp. 754-758, 2014. https:\/\/doi.org\/10.1126\/science.1245842","DOI":"10.1126\/science.1245842"},{"key":"key-10.20965\/jrm.2025.p1034-3","doi-asserted-by":"crossref","unstructured":"K. Nagatani, M. Abe, K. Osuka, P. j. Chun, T. Okatani, M. Nishio, S. Chikushi, T. Matsubara, Y. Ikemoto, and H. Asama, \u201cInnovative technologies for infrastructure construction and maintenance through collaborative robots based on an open design approach,\u201d Advanced Robotics, Vol.35, No.11, pp. 715-722, 2021. https:\/\/doi.org\/10.1080\/01691864.2021.1929471","DOI":"10.1080\/01691864.2021.1929471"},{"key":"key-10.20965\/jrm.2025.p1034-4","unstructured":"M. Rubenstein, A. Cabrera, J. Werfel, G. Habibi, J. Mclurkin, and R. Nagpal, \u201cCollective transport of complex objects by simple robots: Theory and experiments,\u201d The 12th Int. Conf. on Autonomous Agents and Multiagent Systems (AAMAS 2013), pp. 47-54, 2013."},{"key":"key-10.20965\/jrm.2025.p1034-5","doi-asserted-by":"crossref","unstructured":"J. Chen, M. Gauci, W. Li, A. Kolling, and R. Gro\u00df, \u201cOcclusion-based cooperative transport with a swarm of miniature mobile robots,\u201d IEEE Trans. on Robotics, Vol.31, No.2, pp. 307-321, 2015. https:\/\/doi.org\/10.1109\/TRO.2015.2400731","DOI":"10.1109\/TRO.2015.2400731"},{"key":"key-10.20965\/jrm.2025.p1034-6","doi-asserted-by":"crossref","unstructured":"M. H. M. Alkilabi, A. Narayan, and E. Tuci, \u201cCooperative object transport with a swarm of e-puck robots: Robustness and scalability of evolved collective strategies,\u201d Swarm Intelligence, Vol.11, pp. 185-209, 2017. https:\/\/doi.org\/10.1007\/s11721-017-0135-8","DOI":"10.1007\/s11721-017-0135-8"},{"key":"key-10.20965\/jrm.2025.p1034-7","doi-asserted-by":"crossref","unstructured":"G. Eoh and T.-H. Park, \u201cAutomatic curriculum design for object transportation based on deep reinforcement learning,\u201d IEEE Access, Vol.9, pp. 137281-137294, 2021. https:\/\/doi.org\/10.1109\/ACCESS.2021.3118109","DOI":"10.1109\/ACCESS.2021.3118109"},{"key":"key-10.20965\/jrm.2025.p1034-8","doi-asserted-by":"crossref","unstructured":"T. Niwa, K. Shibata, and T. Jimbo, \u201cMulti-agent reinforcement learning and individuality analysis for cooperative transportation with obstacle removal,\u201d F. Matsuno, S.-i. Azuma, and M. Yamamoto (Eds.), \u201cDistributed Autonomous Robotic Systems. DARS 2021. Springer Proc. in Advanced Robotics, Vol.22,\u201d pp. 202-213, 2022. https:\/\/doi.org\/10.1007\/978-3-030-92790-5_16","DOI":"10.1007\/978-3-030-92790-5_16"},{"key":"key-10.20965\/jrm.2025.p1034-9","doi-asserted-by":"crossref","unstructured":"L. Zhang, Y. Sun, A. Barth, and O. Ma, \u201cDecentralized control of multi-robot system in cooperative object transportation using deep reinforcement learning,\u201d IEEE Access, Vol.8, pp. 184109-184119, 2020. https:\/\/doi.org\/10.1109\/ACCESS.2020.3025287","DOI":"10.1109\/ACCESS.2020.3025287"},{"key":"key-10.20965\/jrm.2025.p1034-10","unstructured":"T. Wang, H. Dong, V. Lesser, and C. Zhang, \u201cRoma: Multi-agent reinforcement learning with emergent roles,\u201d Proc. of the 37th Int. Conf. on Machine Learning (ICML\u201920), pp. 9876-9886, 2020."},{"key":"key-10.20965\/jrm.2025.p1034-11","unstructured":"T. Wang, T. Gupta, A. Mahajan, B. Peng, S. Whiteson, and C. Zhang, \u201cRode: Learning roles to decompose multi-agent tasks,\u201d Int. Conf. on Learning Representations, 2021."},{"key":"key-10.20965\/jrm.2025.p1034-12","unstructured":"S. Reed, K. Zolna, E. Parisotto, S. G. Colmenarejo, A. Novikov, G. Barth-Maron, M. Gim\u00e9nez, Y. Sulsky, J. Kay, J. T. Springenberg, T. Eccles, J. Bruce, A. Razavi, A. Edwards, N. Heess, Y. Chen, R. Hadsell, O. Vinyals, M. Bordbar, and N. de Freitas, \u201cA generalist agent,\u201d Trans. on Machine Learning Research, 2022."},{"key":"key-10.20965\/jrm.2025.p1034-13","unstructured":"M. Wen, J. G. Kuba, R. Lin, W. Zhang, Y. Wen, J. Wang, and Y. Yang, \u201cMulti-agent reinforcement learning is a sequence modeling problem,\u201d Proc. of the 36th Int. Conf. on Neural Information Processing Systems (NIPS\u201922), pp. 16509-16521, 2022."},{"key":"key-10.20965\/jrm.2025.p1034-14","unstructured":"A. Cohen, E. Teng, V.-P. Berges, R.-P. Dong, H. Henry, M. Mattar, A. Zook, and S. Ganguly, \u201cOn the use and misuse of absorbing states in multi-agent reinforcement learning,\u201d arXiv preprint, arXiv:2111.05992, 2022. https:\/\/doi.org\/10.48550\/arXiv.2111.05992"},{"key":"key-10.20965\/jrm.2025.p1034-15","unstructured":"Y. Sueoka, T. Yoshida, and K. Osuka, \u201cReinforcement learning of scalable, flexible and robust cooperative transport behavior using the transformer encoder,\u201d The 17th Int. Symposium on Distributed Autonomous Robotic Systems (DARS2024), 2024."},{"key":"key-10.20965\/jrm.2025.p1034-16","unstructured":"A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, \u0141. Kaiser, and I. Polosukhin, \u201cAttention is all you need,\u201d Proc. of the 31st Int. Conf. on Neural Information Processing Systems (NIPS\u201917), pp. 6000-6010, 2017."},{"key":"key-10.20965\/jrm.2025.p1034-17","unstructured":"S. Ozaki, T. Yoshida, Y. Kato, and Y. Sueoka, \u201cReal-world experiment of cooperative object transportation using self-attention based deep reinforcement learning,\u201d 2025 JSME Conf. on Robotics and Mechatronics, Article No.2P2-I03, 2025 (in Japanese)."},{"key":"key-10.20965\/jrm.2025.p1034-18","unstructured":"A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, J. Uszkoreit, and N. Houlsby, \u201cAn image is worth 16x16 words: Transformers for image recognition at scale,\u201d arXiv preprint, arXiv:2010.11929, 2020. https:\/\/doi.org\/10.48550\/arXiv.2010.11929"},{"key":"key-10.20965\/jrm.2025.p1034-19","unstructured":"J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, \u201cProximal policy optimization algorithms,\u201d arXiv preprint, arXiv:1707.06347, 2017. https:\/\/doi.org\/10.48550\/arXiv.1707.06347"},{"key":"key-10.20965\/jrm.2025.p1034-20","unstructured":"D. P. Kingma and J. Ba, \u201cAdam: A method for stochastic optimization,\u201d arXiv preprint, arXiv:1412.6980, 2017. https:\/\/doi.org\/10.48550\/arXiv.1412.6980"},{"key":"key-10.20965\/jrm.2025.p1034-21","unstructured":"J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel, \u201cHigh-dimensional continuous control using generalized advantage estimation,\u201d arXiv preprint, arXiv:1506.02438, 2018. https:\/\/doi.org\/10.48550\/arXiv.1506.02438"}],"container-title":["Journal of Robotics and Mechatronics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.fujipress.jp\/main\/wp-content\/themes\/Fujipress\/hyosetsu.php?ppno=robot003700050003","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,19]],"date-time":"2025-10-19T15:02:20Z","timestamp":1760886140000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.fujipress.jp\/jrm\/rb\/robot003700051034"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,20]]},"references-count":21,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2025,10,20]]},"published-print":{"date-parts":[[2025,10,20]]}},"URL":"https:\/\/doi.org\/10.20965\/jrm.2025.p1034","relation":{},"ISSN":["1883-8049","0915-3942"],"issn-type":[{"value":"1883-8049","type":"electronic"},{"value":"0915-3942","type":"print"}],"subject":[],"published":{"date-parts":[[2025,10,20]]}}}