{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T00:31:15Z","timestamp":1775089875120,"version":"3.50.1"},"reference-count":52,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2021,1,12]],"date-time":"2021-01-12T00:00:00Z","timestamp":1610409600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper presents the results of the design, simulation, and implementation of a virtual vehicle. Such a process employs the Unity videogame platform and its Machine Learning-Agents library. The virtual vehicle is implemented in Unity considering mechanisms that represent accurately the dynamics of a real automobile, such as motor torque curve, suspension system, differential, and anti-roll bar, among others. Intelligent agents are designed and implemented to drive the virtual automobile, and they are trained using imitation or reinforcement. In the former method, learning by imitation, a human expert interacts with an intelligent agent through a control interface that simulates a real vehicle; in this way, the human expert receives motion signals and has stereoscopic vision, among other capabilities. In learning by reinforcement, a reward function that stimulates the intelligent agent to exert a soft control over the virtual automobile is designed. In the training stage, the intelligent agents are introduced into a scenario that simulates a four-lane highway. In the test stage, instead, they are located in unknown roads created based on random spline curves. Finally, graphs of the telemetric variables are presented, which are obtained from the automobile dynamics when the vehicle is controlled by the intelligent agents and their human counterpart, both in the training and the test track.<\/jats:p>","DOI":"10.3390\/s21020492","type":"journal-article","created":{"date-parts":[[2021,1,12]],"date-time":"2021-01-12T20:11:31Z","timestamp":1610482291000},"page":"492","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Design and Implementation of Intelligent Agent Training Systems for Virtual Vehicles"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7197-8928","authenticated-orcid":false,"given":"Claudio","family":"Urrea","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, Universidad de Santiago de Chile, Av. Ecuador 3519, Estaci\u00f3n Central, Santiago 9170124, Chile"}]},{"given":"Felipe","family":"Garrido","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Universidad de Santiago de Chile, Av. Ecuador 3519, Estaci\u00f3n Central, Santiago 9170124, Chile"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1958-7289","authenticated-orcid":false,"given":"John","family":"Kern","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Universidad de Santiago de Chile, Av. Ecuador 3519, Estaci\u00f3n Central, Santiago 9170124, Chile"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1109\/TIT.1954.1057468","article-title":"Simulation of self-organizing systems by digital computer","volume":"4","author":"Farley","year":"1954","journal-title":"Trans. IRE Prof. Group Inf. Theory"},{"key":"ref_2","unstructured":"McCarthy, J., Minsky, M.L., Rochester, N., and Shannon, C.E. (2006). A proposal for the dartmouth summer research project on artificial intelligence, August 31, 1955. AI Mag., 27."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1037\/h0042519","article-title":"The perceptron: A probabilistic model for information storage and organization in the brain","volume":"65","author":"Rosenblatt","year":"1958","journal-title":"Psychol. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Pfeiffer, M., Schaeuble, M., Nieto, J., Siegwart, R., and Cadena, C. (June, January 29). From perception to decision: A data-driven approach to end-to-end motion planning for autonomous ground robots. Proceedings of the 2017 IEEE International Conference on Robotics and Automation (ICRA), Marina Bay Sands, Singapore.","DOI":"10.1109\/ICRA.2017.7989182"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Chen, C., Seff, A., Kornhauser, A., and Xiao, J. (2015, January 11\u201318). DeepDriving: Learning affordance for direct perception in autonomous driving. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Las Condes, Chile.","DOI":"10.1109\/ICCV.2015.312"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Urbaniak, K., W\u0105tr\u00f3bski, J., and Sa\u0142abun, W. (2020). Identification of players ranking in e-sport. Appl. Sci., 10.","DOI":"10.3390\/app10196768"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"7","DOI":"10.1007\/s13218-020-00647-w","article-title":"From chess and atari to starcraft and beyond: How game ai is driving the world of ai","volume":"34","author":"Risi","year":"2020","journal-title":"KI-K\u00fcnstl. Intell."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Tang, X., Song, H., Wang, W., and Yang, Y. (2020). Vehicle spatial distribution and 3D trajectory extraction algorithm in a cross-camera traffic scene. Sensors, 20.","DOI":"10.3390\/s20226517"},{"key":"ref_9","unstructured":"Juliani, A., Berges, V.P., Vckay, E., Gao, Y., Henry, H., Mattar, M., and Lange, D. (2018). Unity: A general platform for intelligent agents. Comput. Sci. Math., 52185833."},{"key":"ref_10","unstructured":"Kwon, O. (2020). Very simple statistical evidence that alphago has exceeded human limits in playing go game. Comput. Sci., 11107."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Urrea, C., Kern, J., and Alvarado, J. (2020). Design and evaluation of a new fuzzy control algorithm applied to a manipulator robot. Appl. Sci., 10.","DOI":"10.3390\/app10217482"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1109\/TG.2020.2990865","article-title":"Winning isn\u2019t everything: Enhancing game development with intelligent agents","volume":"12","author":"Zhao","year":"2020","journal-title":"IEEE Trans. Games"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"eaaw4106","DOI":"10.1126\/scirobotics.aaw4106","article-title":"Parallel testing of vehicle intelligence via virtual-real interaction","volume":"4","author":"Li","year":"2019","journal-title":"Sci. Robot."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Gao, H., Shi, G., Xie, G., and Cheng, B. (2018). Car-following method based on inverse reinforcement learning for autonomous vehicle decision-making. Int. J. Adv. Robot. Syst., 15.","DOI":"10.1177\/1729881418817162"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"444","DOI":"10.1108\/IR-07-2018-0154","article-title":"Research on decision-making of autonomous vehicle following based on reinforcement learning method","volume":"46","author":"Gao","year":"2019","journal-title":"Ind. Robot. Int. J."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Lef\u00e8bre, S., V\u00e1squez, D., and Laugier, C. (2014). A survey on motion prediction and risk assessment for intelligent vehicles. ROBOMECH J., 1.","DOI":"10.1186\/s40648-014-0001-z"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"4224","DOI":"10.1109\/TII.2018.2822828","article-title":"Object classification using CNN-based fusion of vision and LIDAR in autonomous vehicle environment","volume":"14","author":"Gao","year":"2018","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1109\/TIV.2016.2608003","article-title":"Intelligence testing for autonomous vehicles: A new approach","volume":"1","author":"Li","year":"2016","journal-title":"IEEE Trans. Intell. Veh."},{"key":"ref_19","unstructured":"Feng, D., Haase-Schutz, C., Rosenbaum, L., Hertlein, H., Glaser, C., Timm, F., Wiesbeck, W., and Dietmayer, K. (2020). Deep multi-modal object detection and semantic segmentation for autonomous driving: Datasets, methods, and challenges. IEEE Trans. Intell. Transp. Syst., 1\u201320."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1017\/S0269888900008122","article-title":"Intelligent agents: Theory and practice","volume":"10","author":"Wooldridge","year":"1995","journal-title":"Knowl. Eng. Rev."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"7115","DOI":"10.1080\/00207543.2016.1165359","article-title":"Cloud-based multi-agent architecture for effective planning and scheduling of distributed manufacturing","volume":"54","author":"Mishra","year":"2016","journal-title":"Int. J. Prod. Res."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Shah, S., Dey, D., Lovett, C., and Kapoor, A. (2018). AirSim: High-fidelity visual and physical simulation for autonomous vehicles. Distributed Auton. Robotic Syst., 621\u2013635.","DOI":"10.1007\/978-3-319-67361-5_40"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1054","DOI":"10.1109\/TNN.1998.712192","article-title":"Reinforcement learning: An introduction","volume":"9","author":"Sutton","year":"1998","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Brearcliffe, D., and Crooks, A. (2021, January 03). Creating Intelligent Agents: Combining Agent-Based Modeling with Machine Learning. Available online: https:\/\/easychair.org\/publications\/preprint\/w3H1.","DOI":"10.1007\/978-3-030-83418-0_3"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1177\/0278364904048962","article-title":"Learning motion patterns of people for compliant robot motion","volume":"24","author":"Bennewitz","year":"2005","journal-title":"Int. J. Robot. Res."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"46","DOI":"10.1016\/j.compind.2006.04.001","article-title":"Virtual hands and virtual reality multimodal platform to design safer industrial systems","volume":"58","author":"Pouliquen","year":"2007","journal-title":"Comput. Ind."},{"key":"ref_27","unstructured":"Dosovitskiy, A., Ros, G., Codevilla, F., Lopez, A., and Koltun, V. (2017, January 13\u201315). CARLA: An open urban driving simulator. Proceedings of the 1st Annual Conference on Robot Learning, Mountain View, CA, USA."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"735","DOI":"10.1109\/TLA.2018.8358649","article-title":"Design and development of control systems for an aircraft. Comparison of performances through computational simulations","volume":"16","author":"Urrea","year":"2018","journal-title":"IEEE Lat. Am. Trans."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"6814","DOI":"10.1109\/TVT.2018.2822762","article-title":"Humanlike driving: Empirical decision-making system for autonomous vehicles","volume":"67","author":"Li","year":"2018","journal-title":"IEEE Trans. Veh. Technol."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/j.compind.2019.05.001","article-title":"Industrial robot control and operator training using virtual reality interfaces","volume":"109","author":"Diez","year":"2019","journal-title":"Comput. Ind."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Urrea, C., and Saa, D. (2020). Design and implementation of a graphic simulator for calculating the inverse kinematics of a redundant planar manipulator robot. Appl. Sci., 10.","DOI":"10.3390\/app10196770"},{"key":"ref_32","first-page":"86","article-title":"Driving autonomous vehicles","volume":"75","year":"2020","journal-title":"Rev. Droit Technol. Inf."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Minhas, R.A., Javed, A., Irtaza, A., Mahmood, M.T., and Joo, Y.-B. (2019). Shot classification of field sports videos using AlexNet convolutional neural network. Appl. Sci., 9.","DOI":"10.3390\/app9030483"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Miclea, R.-C., Dughir, C., Alexa, F., Sandru, F., and Silea, I. (2020). Laser and LIDAR in A system for visibility distance estimation in fog conditions. Sensors, 20.","DOI":"10.3390\/s20216322"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"103447","DOI":"10.1016\/j.robot.2020.103447","article-title":"Development of a virtual reality simulator for a strategy for coordinating cooperative manipulator robots using cloud computing","volume":"126","author":"Urrea","year":"2020","journal-title":"Robot. Auton. Syst."},{"key":"ref_36","unstructured":"Gao, H., Zhu, J., Zhang, T., Xie, G., Kan, Z., Hao, Z., and Liu, K. (2020). Situational assessment for intelligent vehicles based on Stochastic model and Gaussian distributions in typical traffic scenarios. IEEE Trans. Syst. Man, Cybern. Syst., 1\u201311."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Palmerini, L., Klenk, J., Becker, C., and Chiari, L. (2020). Accelerometer-based fall detection using machine learning: Training and testing on real-world falls. Sensors, 20.","DOI":"10.3390\/s20226479"},{"key":"ref_38","unstructured":"Wymann, B., Espi\u00e9, E., Guionneau, C., Dimitrakakis, C., Coulom, R., and Sumner, A. (2021, January 03). Torcs, the Open Racing Car Simulator. Available online: http:\/\/torcs.sourceforge.net."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"520","DOI":"10.1016\/j.compind.2008.01.002","article-title":"An interactive data-driven driving simulator using motion blending","volume":"59","author":"Cha","year":"2008","journal-title":"Comput. Ind."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1367","DOI":"10.1007\/s10845-019-01515-7","article-title":"Multi-agent systems negotiation to deal with dynamic scheduling in disturbed industrial context","volume":"31","author":"Mezgebe","year":"2020","journal-title":"J. Intell. Manuf."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"4005","DOI":"10.1007\/s00170-019-03800-4","article-title":"Cyber-physical production systems architecture based on multi-agent\u2019s design pattern-comparison of selected approaches mapping four agent patterns","volume":"105","author":"Salazar","year":"2019","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"4330","DOI":"10.1109\/LRA.2019.2931825","article-title":"A framework for automatic initialization of multi-agent production systems using semantic web technologies","volume":"4","author":"Ocker","year":"2019","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_43","unstructured":"Ciortea, A., Mayer, S., and Michahelles, F. (2018, January 10\u201315). Repurposing manufacturing lines on the fly with multi-agent systems for the web of things. Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, Stockholm, Sweden."},{"key":"ref_44","unstructured":"Orio, G.D., Rocha, A., Ribeiro, L., and Barata, J. (2015, January 23\u201326). The PRIME semantic language: Plug and produce in standard-based manufacturing production systems. Proceedings of the International Conference on Flexible Automation and Intelligent Manufacturing 2015 (FAIM\u201915), Wolverhampton, UK."},{"key":"ref_45","first-page":"50","article-title":"Imitation learning: A survey of learning methods","volume":"50","author":"Hussein","year":"2017","journal-title":"ACM Comput. Surv. (CSUR)"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1613\/jair.301","article-title":"Reinforcement learning: A survey","volume":"4","author":"Kaelbling","year":"1996","journal-title":"J. Artif. Intell. Res."},{"key":"ref_47","unstructured":"Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013, January 9). Playing atari with deep reinforcement learning. Proceedings of the NIPS Deep Learning Workshop 2013, Lake Tahoe, NV, USA. Available online: https:\/\/www.cs.toronto.edu\/~vmnih\/docs\/dqn.pdf."},{"key":"ref_48","unstructured":"Beattie, C., Leibo, J.Z., Teplyashin, D., Ward, T., Wainwright, M., K\u00fcttler, H., Lefrancq, A., Green, S., Vald\u00e9s, V., and Sadik, A. (2021, January 03). Deepmind Lab. Available online: https:\/\/deepmind.com\/research; https:\/\/arxiv.org\/pdf\/1612.03801.pdf."},{"key":"ref_49","unstructured":"Johnson, M., Hofmann, K., Hutton, T., and Bignell, D. (2016, January 9\u201315). The Malmo Platform for Artificial Intelligence Experimentation. Proceedings of the 25th International Joint Conference on Artificial Intelligence (IJCAI), New York, NY, USA."},{"key":"ref_50","unstructured":"Rill, G. (2009). Vehicle Dynamics, University of Applied Sciences Hochschule. Available online: https:\/\/www.twirpx.com\/file\/1067713."},{"key":"ref_51","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2021, January 03). Proximal Policy Optimization Algorithms. Available online: https:\/\/arxiv.org\/pdf\/1707.06347v2.pdf."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Bain, M., and Sammut, C. (1999). A framework for behavioural cloning. Mach. Intell., 15.","DOI":"10.1093\/oso\/9780198538677.003.0006"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/2\/492\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:10:05Z","timestamp":1760159405000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/2\/492"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,12]]},"references-count":52,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["s21020492"],"URL":"https:\/\/doi.org\/10.3390\/s21020492","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,12]]}}}