{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,5]],"date-time":"2026-01-05T22:22:23Z","timestamp":1767651743986,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2018,10,3]],"date-time":"2018-10-03T00:00:00Z","timestamp":1538524800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Secretaria de Educacion Publica (SEP)","award":["511-6\/17-7605"],"award-info":[{"award-number":["511-6\/17-7605"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>A multi-agent system (MAS) is suitable for addressing tasks in a variety of domains without any programmed behaviors, which makes it ideal for the problems associated with the mobile robots. Reinforcement learning (RL) is a successful approach used in the MASs to acquire new behaviors; most of these select exact Q-values in small discrete state space and action space. This article presents a joint Q-function linearly fuzzified for a MAS\u2019 continuous state space, which overcomes the dimensionality problem. Also, this article gives a proof for the convergence and existence of the solution proposed by the algorithm presented. This article also discusses the numerical simulations and experimental results that were carried out to validate the proposed algorithm.<\/jats:p>","DOI":"10.3390\/sym10100461","type":"journal-article","created":{"date-parts":[[2018,10,4]],"date-time":"2018-10-04T02:19:49Z","timestamp":1538619589000},"page":"461","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Multi-Agent Reinforcement Learning Using Linear Fuzzy Model Applied to Cooperative Mobile Robots"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4778-8873","authenticated-orcid":false,"given":"David","family":"Luviano-Cruz","sequence":"first","affiliation":[{"name":"Department of industrial engineering and manufacturing, Autonomous University of Ciudad Juarez, Ciudad Juarez 32310, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8571-914X","authenticated-orcid":false,"given":"Francesco","family":"Garcia-Luna","sequence":"additional","affiliation":[{"name":"Department of industrial engineering and manufacturing, Autonomous University of Ciudad Juarez, Ciudad Juarez 32310, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2541-4595","authenticated-orcid":false,"given":"Luis","family":"P\u00e9rez-Dom\u00ednguez","sequence":"additional","affiliation":[{"name":"Department of industrial engineering and manufacturing, Autonomous University of Ciudad Juarez, Ciudad Juarez 32310, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7974-7825","authenticated-orcid":false,"given":"S. K.","family":"Gadi","sequence":"additional","affiliation":[{"name":"Faculty of mechanical and electrical engineering, Autonomous University of Coahuila, Torreon 27276, Mexico"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,10,3]]},"reference":[{"key":"ref_1","unstructured":"Sen, S., and Weiss, G. (1999). Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence, MIT Press."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1023\/A:1008942012299","article-title":"Multiagent systems: A survey from machine learning perspective","volume":"8","author":"Stone","year":"2000","journal-title":"Auton. Robots"},{"key":"ref_3","unstructured":"Wooldridge, M. (2002). An Introduction to MultiAgent Systems, John Wiley & Sons."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1440","DOI":"10.1016\/j.robot.2013.07.010","article-title":"Path planning with obstacle avoidance based on visibility binary tree algorithm","volume":"61","author":"Rashid","year":"2013","journal-title":"Robot. Auton. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","article-title":"Reinforcement Learning: A Survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"J. Artif. Intell. Res."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1049\/iet-its.2009.0070","article-title":"Reinforcement learning-based multi-agent system for network traffic signal control","volume":"4","author":"Arel","year":"2010","journal-title":"IET Intell. Transp. Syst."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Cherkassky, V., and Mulier, F. (2007). Learning from data: Concepts, Theory and Methods, Wiley-IEEE Press.","DOI":"10.1002\/9780470140529"},{"key":"ref_8","unstructured":"Sejnowski, T.J., and Hinton, G. (1999). Unsupervised learning. Unsupervised Learning: Foundations of Neural Computation, MIT Press. [1st ed.]."},{"key":"ref_9","unstructured":"Zhang, W., Ma, L., and Li, X. (2018). Multi-agent reinforcement learning based on local communication. Clust. Comput., 1\u201310."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1007\/s12555-012-0382-9","article-title":"Consensus of Linear Multi-Agent Systems Subject to Actuator Saturation","volume":"11","author":"Hu","year":"2013","journal-title":"Int. J. Control Autom. Syst."},{"key":"ref_11","unstructured":"Luviano, D., and Yu, W. (2015, January 28\u201330). Path planning in unknown environment with kernel smoothing and reinforcement learning for multi-agent systems. Proceedings of the 12th International Conference on Electrical Engineering, Computing Science and Automatic Control (CCE), Mexico City, Mexico."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Abul, O., Polat, F., and Alhajj, R. (2000). Multi-agent reinforcement learning using function approximation. IEEE Trans. Syst. Man Cybern. Part C Appl. Rev., 485\u2013497.","DOI":"10.1109\/5326.897075"},{"key":"ref_13","first-page":"217","article-title":"Learning in large cooperative multi-robots systems","volume":"16","author":"Fernandez","year":"2001","journal-title":"Int. J. Robot. Autom. Spec. Issue Comput. Intell. Tech. Coop. Robots"},{"key":"ref_14","unstructured":"Foerster, J., Nardelli, N., Farquhar, G., Afouras, T., Torr, P.H., Kohli, P., and Whiteson, S. (arXiv, 2017). Stabilising experience replay for deep multi-agent reinforcement learning, arXiv."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Tamakoshi, H., and Ishi, S. (2001). Multi agent reinforcement learning applied to a chase problem in a continuous world. Artif. Life Robot., 202\u2013206.","DOI":"10.1007\/BF02481502"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"245","DOI":"10.1016\/S0921-8890(03)00040-X","article-title":"An approach to pursuit problem on a heterogeneous multiagent system using reinforcement learning","volume":"43","author":"Ishiwaka","year":"2003","journal-title":"Robot. Auton. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Radac, M.-B., Precup, R.-E., and Roman, R.-C. (2017). Data-driven model reference control of MIMO vertical tank systems with model-free VRFT and Q-Learning. ISA Trans.","DOI":"10.1016\/j.isatra.2018.01.014"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.jprocont.2018.07.013","article-title":"Control of a bioreactor using a new partially supervised reinforcement learning algorithm","volume":"69","author":"Pandian","year":"2018","journal-title":"J. Process Control"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1007\/BF00992698","article-title":"Q-learning","volume":"8","author":"Watkins","year":"1992","journal-title":"Mach. Learn."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Nguyen, T., Nguyen, N.D., and Nahavandi, S. (arXiv, 2018). Multi-Agent Deep Reinforcement Learning with Human Strategies, arXiv.","DOI":"10.1109\/ICIT.2019.8755032"},{"key":"ref_21","unstructured":"Boutilier, C. (1996, January 17\u201320). Planning, Learning and Coordination in Multiagent Decision Processes. Proceedings of the Sixth Conference on Theoretical Aspects of Rationality and Knowledge (TARK96), De Zeeuwse Stromen, The Netherlands."},{"key":"ref_22","unstructured":"Harsanyi, J.C., and Selten, R. (1988). A General Theory of Equilibrium Selection in Games, MIT Press. [1st ed.]."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Busoniu, L., De Schutter, B., and Babuska, R. (2006, January 5\u20138). Decentralized Reinforcement Learning Control of a robotic Manipulator. Proceedings of the International Conference on Control, Automation, Robotics and Vision, Singapore.","DOI":"10.1109\/ICARCV.2006.345351"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1016\/S1389-0417(01)00015-8","article-title":"Value-function reinforcement learning in Markov games","volume":"2","author":"Littman","year":"2001","journal-title":"J. Cogn. Syst. Res."},{"key":"ref_25","unstructured":"Guestrin, C., Lagoudakis, M.G., and Parr, R. (2002, January 8\u201312). Coordinated reinforcement learning. Proceedings of the 19th International Conference on Machine Learning (ICML-2002), Sydney, Australia."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/S0004-3702(02)00121-2","article-title":"Multiagent learning using a variable learning rate","volume":"136","author":"Bowling","year":"2002","journal-title":"Artif. Intell."},{"key":"ref_27","unstructured":"Bertsekas, D.P. (2017). Dynamic Programming and optimal control vol. 2, Athena Scientific. [4th ed.]."},{"key":"ref_28","unstructured":"Istratesku, V. (2002). Fixed Point Theory: An introduction, Springer."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Melo, F.S., Meyn, S.P., and Ribeiro, M.I. (2008, January 5\u20139). An analysis of reinforcement learning with functions approximation. Proceedings of the 25th International Conference on Machine Learning (ICML-08), Helsinki, Finland.","DOI":"10.1145\/1390156.1390240"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Szepesvari., C., and Smart, W.D. (2004, January 4\u20138). Interpolation-based Q-learning. Proceedings of the 21st International Conference on Machine Learning (ICML-04), Banff, AB, Canada.","DOI":"10.1145\/1015330.1015445"},{"key":"ref_31","unstructured":"Sutton, R.S., McAllester, D.A., Singh, S.P., and Mansour, Y. (December, January 29). Policy gradient methods for reinforcement learning with function approximation. Proceedings of the 12th International Conference on Neural Information Processing Systems, Denver, CO, USA."},{"key":"ref_32","unstructured":"Bertsekas, D.P., and Tsitsiklis, J.N. (1996). Neuro-Dynamic Programming, Athena Scientific. [1st ed.]."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1007\/BF00114724","article-title":"Feature-based methods for large scale dynamic programming","volume":"22","author":"Tsitsiklis","year":"1996","journal-title":"Mach. Learn."},{"key":"ref_34","unstructured":"Kruse, R., Gebhardt, J.E., and Klowon, F. (1994). Foundations of Fuzzy Systems, John Wiley & Sons. [1st ed.]."},{"key":"ref_35","first-page":"1040","article-title":"Reinforcement learning with function approximation converges to a region","volume":"13","author":"Gordon","year":"2001","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1007\/BF00993306","article-title":"Asynchronous stochastic approximation and Q-learning","volume":"16","author":"Tsitsiklis","year":"1994","journal-title":"Mach. Learn."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"724","DOI":"10.1109\/72.159061","article-title":"Learning and tuning fuzzy logic controllers through reinforcements","volume":"3","author":"Berenji","year":"1992","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1023\/A:1017992615625","article-title":"Variable-resolution discretization in optimal control","volume":"49","author":"Munos","year":"2002","journal-title":"Mach. Learn."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"898","DOI":"10.1109\/9.133184","article-title":"An optimal one-way multi grid algorithm for discrete-time stochastic control","volume":"36","author":"Chow","year":"1991","journal-title":"IEEE Trans. Autom. Control"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"804","DOI":"10.1016\/j.automatica.2010.02.006","article-title":"Approximate Dynamic programming with fuzzy parametrization","volume":"46","author":"Busoniu","year":"2010","journal-title":"Automatica"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Vlassis, N. (2007). A concise Introduction to Multi Agent Systems and Distributed Artificial Intelligence. Synthesis Lectures in Artificial Intelligence and Machine Learning, Morgan & Claypool Publishers.","DOI":"10.1007\/978-3-031-01543-4"},{"key":"ref_42","unstructured":"(2018, January 15). K-Team Corporation. Available online: http:\/\/www-k-team.com."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Ganapathy, V., Soh, C.Y., and Lui, W.L.D. (2009, January 25\u201327). Utilization of webots and Khepera II as a Platform for neural Q-learning controllers. Proceedings of the IEEE Symposium on Industrial Electronics and Applications, Kuala Lumpur, Malaysia.","DOI":"10.1109\/ISIEA.2009.5356361"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/10\/10\/461\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:23:52Z","timestamp":1760196232000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/10\/10\/461"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,10,3]]},"references-count":43,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2018,10]]}},"alternative-id":["sym10100461"],"URL":"https:\/\/doi.org\/10.3390\/sym10100461","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2018,10,3]]}}}