{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,25]],"date-time":"2026-08-25T16:02:37Z","timestamp":1787673757330,"version":"build-2736575974"},"reference-count":24,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T00:00:00Z","timestamp":1648512000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Game cheating is a common occurrence that may degrade the experience of \u201chonest\u201d players. It can be hindered by using appropriate anticheat systems, which are being considered as a subset of security-related issues. In this paper, we implement and test an anticheat system whose main goal is to help differentiate human players from AI players. For this purpose, we first developed a multiplayer game inside game engine Unity that would serve as a framework for training the reinforcement learning agent. This agent would thus learn to differentiate human players from bots within the game. We implemented the Machine Learning Agents Toolkit library, which uses the proximal policy optimization algorithm. AI players are implemented using state machines, and perform certain actions depending on which condition is satisfied. Two experiments were carried out for testing the agent and showed promising results for identifying artificial players.<\/jats:p>","DOI":"10.3390\/info13040173","type":"journal-article","created":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T21:44:52Z","timestamp":1648590292000},"page":"173","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Anticheat System Based on Reinforcement Learning Agents in Unity"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7882-0477","authenticated-orcid":false,"given":"Mihael","family":"Lukas","sequence":"first","affiliation":[{"name":"Novena d.o.o., 10000 Zagreb, Croatia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Igor","family":"Tomicic","sequence":"additional","affiliation":[{"name":"Faculty of Organization and Informatics, University of Zagreb, 10000 Zagreb, Croatia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2507-9494","authenticated-orcid":false,"given":"Andrija","family":"Bernik","sequence":"additional","affiliation":[{"name":"Department of Multimedia, University North, 42000 Varazdin, Croatia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,29]]},"reference":[{"key":"ref_1","unstructured":"ML-Agents (2022, February 10). ML-Agents Toolkit Overview. Available online: https:\/\/github.com\/Unity-Technologies\/ml-agents\/blob\/main\/docs\/ML-Agents-Overview.md."},{"key":"ref_2","unstructured":"Mwiti, D. (2022, January 25). 10 Real-Life Applications of Reinforcement Learning. Available online: https:\/\/neptune.ai\/blog\/reinforcement-learning-applications."},{"key":"ref_3","unstructured":"Alayed, H., Frangoudes, F., and Neuman, C. (2022, February 05). Behavioral-Based Cheating Detection in Online First Person Shooters Using Machine Learning Techniques. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/6633617."},{"key":"ref_4","unstructured":"Chapel, L., Botvich, D., and Malone, D. (2022, February 05). Probabilistic Approaches to Cheating Detection in Online Games. Available online: https:\/\/www.researchgate.net\/publication\/221157498_Probabilistic_Approaches_to_Cheating_Detection_in_Online_Games."},{"key":"ref_5","unstructured":"Pao, H.K., Chen, K.T., and Chang, H.C. (2022, February 05). Game Bot Detection via Avatar Trajectory Analysis. Available online: https:\/\/ieeexplore.ieee.org\/document\/5560779."},{"key":"ref_6","unstructured":"Galli, L., Loiacono, D., Cardamone, L., and Lanzi, P. (2022, February 14). A Cheating Detection Framework for Unreal Tournament III: A Machine Learning Approach. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/6032016."},{"key":"ref_7","unstructured":"Khalifa, S. (2022, February 15). Machine Learning and Anti-Cheating in FPS Games. Available online: https:\/\/www.researchgate.net\/publication\/308785899_Machine_Learning_and_Anti-Cheating_in_FPS_Games."},{"key":"ref_8","unstructured":"Willman, M. (2022, February 16). Machine Learning to Identify Cheaters in Online Games. Available online: https:\/\/www.diva-portal.org\/smash\/get\/diva2:1431282\/FULLTEXT01.pdf."},{"key":"ref_9","unstructured":"Islam, M., Dong, B., Chandra, S., and Khan, L. (2022, February 16). GCI: A GPU Based Transfer Learning Approach for Detecting Cheats of Computer Game. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/9154512."},{"key":"ref_10","unstructured":"Platzer, C. (2022, March 23). Sequence-Based Bot Detection in Massive Multiplayer Online Games. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/6174239."},{"key":"ref_11","unstructured":"Lample, G., and Chaplot, D. (2022, March 21). Playing FPS Games with Deep Reinforcement Learning. Available online: https:\/\/www.aaai.org\/ocs\/index.php\/AAAI\/AAAI17\/paper\/view\/14456\/14385."},{"key":"ref_12","unstructured":"Unity (2022, February 06). About ProGrids. Available online: https:\/\/docs.unity3d.com\/Packages\/com.unity.progrids@3.0\/manual\/index.html."},{"key":"ref_13","unstructured":"Unity (2022, February 06). ProBuilder. Available online: https:\/\/unity.com\/features\/probuilder."},{"key":"ref_14","unstructured":"Tadevosyan, G. (2022, March 21). Unity AI Development: A Finite-State Machine Tutorial. Available online: https:\/\/www.toptal.com\/unity-unity3d\/unity-ai-development-finite-state-machine-tutorial."},{"key":"ref_15","unstructured":"Unity (2022, February 06). NavMesh Agent. Available online: https:\/\/docs.unity3d.com\/Manual\/class-NavMeshAgent.html."},{"key":"ref_16","unstructured":"Wang, Y., He, H., Wen, C., and Tan, X. (2022, February 10). Truly Proximal Policy Optimization. Available online: https:\/\/arxiv.org\/abs\/1903.07940."},{"key":"ref_17","unstructured":"ML-Agents (2022, February 10). Training with Proximal Policy Optimization. Available online: https:\/\/github.com\/miyamotok0105\/unity-ml-agents\/blob\/master\/docs\/Training-PPO.md."},{"key":"ref_18","unstructured":"Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2022, February 10). Proximal Policy Optimization Algorithms. Available online: https:\/\/arxiv.org\/abs\/1707.06347."},{"key":"ref_19","unstructured":"Mattar, M., Berges, V.P., Cohen, A., Teng, E., and Elion, C. (2022, March 21). ML-Agents v2.0 Release: Now Supports Training Complex Cooperative Behaviors. Available online: https:\/\/blog.unity.com\/technology\/ml-agents-v20-release-now-supports-training-complex-cooperative-behaviors."},{"key":"ref_20","unstructured":"ML-Agents (2022, February 11). Training Configuration File. Available online: https:\/\/github.com\/Unity-Technologies\/ml-agents\/blob\/main\/docs\/Training-Configuration-File.md#ppo-specific-configurations."},{"key":"ref_21","unstructured":"Lisowski, E. (2022, February 11). What is Entropy in Machine Learning?. Available online: https:\/\/addepto.com\/what-is-entropy-in-machine-learning\/?utm_source=rss&utm_medium=rss&utm_campaign=what-is-entropy-in-machine-learning."},{"key":"ref_22","unstructured":"ML-Agents (2022, March 21). Using TensorBoard to Observe Training. Available online: https:\/\/github.com\/Unity-Technologies\/ml-agents\/blob\/main\/docs\/Using-Tensorboard.md."},{"key":"ref_23","unstructured":"Anaconda (2022, February 11). Data Science Technology for Human Sensemaking. Available online: https:\/\/www.anaconda.com\/."},{"key":"ref_24","unstructured":"Zhang, J. (2022, March 21). Ultimate Volleyball: A Multi-Agent Reinforcement Learning Environment Built Using Unity ML-Agents. Available online: https:\/\/towardsdatascience.com\/ultimate-volleyball-a-3d-volleyball-environment-built-using-unity-ml-agents-c9d3213f3064."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/4\/173\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:46:04Z","timestamp":1760136364000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/4\/173"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,29]]},"references-count":24,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["info13040173"],"URL":"https:\/\/doi.org\/10.3390\/info13040173","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,29]]}}}