{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,1]],"date-time":"2026-04-01T18:32:51Z","timestamp":1775068371315,"version":"3.50.1"},"reference-count":58,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T00:00:00Z","timestamp":1732752000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"the UNINOVA research institute","award":["SistemasUIDB\/00066\/2020"],"award-info":[{"award-number":["SistemasUIDB\/00066\/2020"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>We live in an era characterized by Society 4.0 and Industry 4.0 where successive innovations that are more or less disruptive are occurring. Within this context, the modeling and simulation of dynamic supervisory and control systems require dealing with more sophistication and complexity, with effects in terms of development errors and higher costs. One of the most difficult aspects of simulating these systems is the handling of vision sensors. The current tools provide these sensors but in a specific and limited way. This paper describes a six-step approach to sensor virtualization. For testing the approach, a simulation platform based on game engines was developed. As contributions, the platform can simulate dynamic systems, including industrial processes with vision sensors. Furthermore, the proposed virtualization approach allows for the modeling of sensors in a systematic way, reducing the complexity and effort required to simulate this type of system.<\/jats:p>","DOI":"10.3390\/s24237610","type":"journal-article","created":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T09:21:23Z","timestamp":1732785683000},"page":"7610","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["An Approach for Modeling and Simulation of Virtual Sensors in Automatic Control Systems Using Game Engines and Machine Learning"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7477-6165","authenticated-orcid":false,"given":"Jo\u00e3o","family":"Rosas","sequence":"first","affiliation":[{"name":"NOVA School of Science and Technology, NOVA University Lisbon, Campus de Caparica, 2829-516 Caparica, Portugal"},{"name":"CTS-Uninova & LASI, Campus de Caparica, 2829-516 Caparica, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0339-6843","authenticated-orcid":false,"given":"Lu\u00eds Brito","family":"Palma","sequence":"additional","affiliation":[{"name":"NOVA School of Science and Technology, NOVA University Lisbon, Campus de Caparica, 2829-516 Caparica, Portugal"},{"name":"CTS-Uninova & LASI, Campus de Caparica, 2829-516 Caparica, Portugal"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3341-1386","authenticated-orcid":false,"given":"Rui Azevedo","family":"Antunes","sequence":"additional","affiliation":[{"name":"CTS-Uninova & LASI, Campus de Caparica, 2829-516 Caparica, Portugal"},{"name":"ESTSet\u00fabal, Instituto Polit\u00e9cnico de Set\u00fabal, Estefanilha, 2914-508 Set\u00fabal, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,28]]},"reference":[{"key":"ref_1","unstructured":"Haghighi, H., Wang, X., Jing, H., and Dianati, M. (2024). Review of the Learning-based Camera and Lidar Simulation Methods for Autonomous Driving Systems. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Lyu, Z., Kripakaran, K., Furth, M., Tang, E., Wandell, B., and Farrell, J. (2021). Validation of image systems simulation technology using a cornell box. arXiv.","DOI":"10.2352\/ISSN.2470-1173.2021.7.ISS-122"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"N\u00fcrnberg, T., Schambach, M., Uhlig, D., Heizmann, M., and Le\u00f3n, F.P. (2019, January 21). framework for the design and evaluation of computational cameras. Proceedings of the SPIE 11061, Automated Visual Inspection and Machine Vision III, Munich, Germany.","DOI":"10.1117\/12.2527599"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Mora-Salinas, R.J., and Hern\u00e1ndez, H.G.G. (2022, January 28\u201331). Virtual labs: 5 ways to connect with Factory IO for mechatronics engineering courses. Proceedings of the 2022 IEEE Global Engineering Education Conference (EDUCON), Tunis, Tunisia.","DOI":"10.1109\/EDUCON52537.2022.9766615"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Montenegro, G., Chac\u00f3n, R., Fabregas, E., Garcia, G., Schr\u00f6der, K., Marroqu\u00edn, A., Dormido-Canto, S., and Farias, G. (2022). Modeling and Control of a Spherical Robot in the CoppeliaSim Simulator. Sensors, 22.","DOI":"10.3390\/s22166020"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"102629","DOI":"10.1016\/j.simpat.2022.102629","article-title":"Andrew Farley, Jie Wang, Joshua A. Marshall, How to pick a mobile robot simulator: A quantitative comparison of CoppeliaSim, Gazebo, MORSE and Webots with a focus on accuracy of motion","volume":"120","author":"Farley","year":"2022","journal-title":"Simul. Model. Pract. Theory"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Rajesh, G., Raajini, X.M., and Dang, H. (2021). Industry 4.0 Interoperability, Analytics, Security, and Case Studies, CRC Press. [1st ed.].","DOI":"10.1201\/9781003048855"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Lu, Y. (2021). The Current Status and Developing Trends of Industry 4.0: A Review. Inf. Syst. Front., 1\u201320.","DOI":"10.1007\/s10796-021-10221-w"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1331","DOI":"10.1007\/s00170-024-13839-7","article-title":"Industry 4.0: Review and proposal for implementing a smart factory","volume":"133","author":"Wu","year":"2024","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhou, K., Liu, T., and Zhou, L. (2015, January 15\u201317). Industry 4.0: Towards future industrial opportunities and challenges. Proceedings of the 2015 12th International Conference on Fuzzy Systems and Knowledge Discovery (FSKD), Zhangjiajie, China.","DOI":"10.1109\/FSKD.2015.7382284"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Stavropoulos, G., Violos, J., Tsanakas, S., and Leivadeas, A. (2023). Enabling Artificial Intelligent Virtual Sensors in an IoT Environment. Sensors, 23.","DOI":"10.3390\/s23031328"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"V\u00e1zquez-Hurtado, C., Altamirano-Avila, E., Roman-Flores, A., and Vargas-Martinez, A. (2023, January 1\u20134). Towards a Mixed Virtual Reality Environment Implementation to Enable Industrial Robot Programming Competencies within a Cyber-Physical Factory. Proceedings of the 2023 IEEE Global Engineering Education Conference (EDUCON), Kuwait, Kuwait.","DOI":"10.1109\/EDUCON54358.2023.10125175"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Kampa, A. (2023). Modeling and Simulation of a Digital Twin of a Production System for Industry 4.0 with Work-in-Process Synchronization. Appl. Sci., 13.","DOI":"10.3390\/app132212261"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"107365","DOI":"10.1016\/j.compchemeng.2021.107365","article-title":"Recent trends on hybrid modeling for Industry 4.0","volume":"151","author":"Sansana","year":"2021","journal-title":"Comput. Chem. Eng."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/B978-0-444-64235-6.50035-8","article-title":"Modeling of an air quality monitoring network with high space-time resolution","volume":"Volume 43","author":"Friedl","year":"2018","journal-title":"Computer Aided Chemical Engineering"},{"key":"ref_16","unstructured":"Nath, S.V., Dunkin, A., Chowdhary, M., and Patel, N. (2020). Industrial Digital Transformation: Accelerate Digital Transformation with Business Optimization, AI, and Industry 4.0, Packt Publishing Ltd."},{"key":"ref_17","unstructured":"Nath, S.V., Schalkwyk, P.V., and Isaacs, D. (2021). Building Industrial Digital Twins: Design, Develop, and Deploy Digital Twin Solutions for Real-World Industries Using Azure Digital Twins, Packt Publishing Ltd."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Liyanawaduge, N.N., Kumarasinghe, E.M.H.K., Iyer, S.S., Kulatunga, A.K., and Lakmal, G. (2023, January 25\u201326). Digital Twin & Virtual Reality Enabled Conveyor System to Promote Learning Factory Concept. Proceedings of the 2023 IEEE 17th International Conference on Industrial and Information Systems (ICIIS), Peradeniya, Sri Lanka.","DOI":"10.1109\/ICIIS58898.2023.10253555"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Nov\u00e1k, P., and Vyskocil, J. (2022). Digitalized Automation Engineering of Industry 4.0 Production Systems and Their Tight Cooperation with Digital Twins. Processes, 10.","DOI":"10.3390\/pr10020404"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"103025","DOI":"10.1016\/j.technovation.2024.103025","article-title":"How will the digital twin shape the future of industry 5.0?","volume":"134","author":"Barata","year":"2024","journal-title":"Technovation"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1145\/502269.502288","article-title":"Game engines","volume":"45","author":"Lewis","year":"2002","journal-title":"Commun. ACM"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"285","DOI":"10.1007\/978-3-031-10047-5_25","article-title":"Specification and Development of Dynamic Systems and Controllers Based on Game Engines","volume":"Volume 930","author":"Palma","year":"2022","journal-title":"CONTROLO 2022: Proceedings of the 15th APCA International Conference on Automatic Control and Soft Computing, Caparica, Portugal, 6\u20138 July 2022"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Wang, Z., Han, K., and Tiwari, P. (August, January 15). Digital Twin Simulation of Connected and Automated Vehicles with the Unity Game Engine. Proceedings of the 2021 IEEE 1st International Conference on Digital Twins and Parallel Intelligence (DTPI), Beijing, China.","DOI":"10.1109\/DTPI52967.2021.9540074"},{"key":"ref_24","unstructured":"Goldstone, W. (2009). Unity Game Development Essentials, Packt Publishing Ltd."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sanders, A. (2016). An Introduction to Unreal Engine 4, AK Peters\/CRC Press.","DOI":"10.1201\/9781315382555"},{"key":"ref_26","unstructured":"Tracy, S., and Reindell, P. (2012). CryENGINE 3 Game Development: Beginner\u2019s Guide, Packt Publishing Ltd."},{"key":"ref_27","unstructured":"Bradfield, C. (2018). Godot Engine Game Development Projects: Build Five Cross-Platform 2D and 3D Games with Godot 3.0, Packt Publishing Ltd."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Cossu, S.M. (2019). Game Development With Game Maker Studio 2, Apress Media LLC.","DOI":"10.1007\/978-1-4842-5010-5"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Gusm\u00e3o, G.F., Barbosa, C.R.H., and Raposo, A.B. (2020). Development and Validation of LiDAR Sensor Simulators Based on Parallel Raycasting. Sensors, 20.","DOI":"10.3390\/s20247186"},{"key":"ref_30","first-page":"1","article-title":"Digital twin: A state-of-the-art review of its enabling technologies, applications and challenges","volume":"2","author":"Hu","year":"2021","journal-title":"J. Intell. Manuf. Spec. Equip."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s40436-021-00375-w","article-title":"State-of-the-art survey on digital twin implementations","volume":"10","author":"Liu","year":"2022","journal-title":"Adv. Manuf."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1016\/j.procir.2017.01.048","article-title":"Integrated data model and structure for the asset administration shell in industrie 4.0","volume":"60","author":"Tantik","year":"2017","journal-title":"Procedia Cirp."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Souza, B.V., Santos, S.R.B.D., Oliveira, A.M.D., and Givigi, S.N. (2023, January 17\u201320). Modelling of Virtual Sensors for Manufacturing Process using Gradient Boosting Technique. Proceedings of the 2023 IEEE International Systems Conference (SysCon), Vancouver, BC, Canada.","DOI":"10.1109\/SysCon53073.2023.10131061"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Paepae, T., Bokoro, P.N., and Kyamakya, K. (2021). From Fully Physical to Virtual Sensing for Water Quality Assessment: A Comprehensive Review of the Relevant State-of- the-Art. Sensors, 21.","DOI":"10.3390\/s21216971"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1007\/s12599-021-00689-w","article-title":"Virtual sensors","volume":"63","author":"Martin","year":"2021","journal-title":"Bus. Inf. Syst. Eng."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Mitchell, H.B. (2012). Data Fusion: Concepts and Ideas, Springer.","DOI":"10.1007\/978-3-642-27222-6"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Yuriyama, M., and Kushida, T. (2010, January 14\u201316). Sensor-Cloud Infrastructure\u2014Physical Sensor Management with Virtualized Sensors on Cloud Computing. Proceedings of the 2010 13th International Conference on Network-Based Information Systems, Takayama, Japan.","DOI":"10.1109\/NBiS.2010.32"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Bose, S., Gupta, A., Adhikary, S., and Mukherjee, N. (2015, January 22). Towards a Sensor-Cloud Infrastructure with Sensor Virtualization. Proceedings of the Second Workshop on Mobile Sensing, Computing and Communication (MSCC \u201915), Hangzhou, China.","DOI":"10.1145\/2757743.2757748"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Alam, S., Chowdhury, M.M.R., and Noll, J. (2010, January 25\u201326). SenaaS: An event-driven sensor virtualization approach for Internet of Things cloud. Proceedings of the 2010 IEEE International Conference on Networked Embedded Systems for Enterprise Applications, Suzhou, China.","DOI":"10.1109\/NESEA.2010.5678060"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1115","DOI":"10.2507\/30th.daaam.proceedings.156","article-title":"Digital Twin and Virtual Sensor for a Rotor System","volume":"30","author":"Tiainen","year":"2019","journal-title":"Ann. DAAAM Proc."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Mukhopadhyay, A., Reddy, G.R., Mukherjee, I., Gopa, G.K., Pena-Rios, A., and Biswas, P. (2021, January 13\u201315). Generating synthetic data for deep learning using VR digital twin. Proceedings of the 2021 5th International Conference on Cloud and Big Data Computing, Liverpool, UK.","DOI":"10.1145\/3481646.3481655"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"219","DOI":"10.1007\/BF02249052","article-title":"Experimental design and analysis in software engineering","volume":"1","author":"Pfleeger","year":"1995","journal-title":"Ann. Softw. Eng."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"De Silva, C.W. (2015). Sensors and Actuators: Engineering System Instrumentation, CRC Press.","DOI":"10.1201\/b18739"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Ericsson, N., Lennvall, T., \u00c5kerberg, J., and Bj\u00f6rkman, M. (2017, January 13\u201315). Custom simulation of Industrial Wireless Sensor and Actuator Network for improved efficiency during research and development. Proceedings of the 2017 22nd IEEE International Conference on Emerging Technologies and Factory Automation (ETFA), Limassol, Cyprus.","DOI":"10.1109\/ETFA.2017.8247657"},{"key":"ref_45","unstructured":"Unity Technologies (2024, August 01). Unity Asset Store\u20143D Models. Available online: https:\/\/assetstore.unity.com\/3d."},{"key":"ref_46","unstructured":"(2024, August 01). \u201cOtoh\u2019s Fruit Box\u201d, Unity Asset Store. Available online: https:\/\/assetstore.unity.com\/packages\/3d\/props\/food\/otoh-s-fruit-box-274072."},{"key":"ref_47","unstructured":"(2024, February 16). \u201cLow Poly Pack-Environment Lite\u201d, Unity Asset Store. Available online: https:\/\/assetstore.unity.com\/packages\/3d\/props\/exterior\/low-poly-pack-environment-lite-102039."},{"key":"ref_48","unstructured":"Bradski, G., and Kaehler, A. (2008). Learning OpenCV: Computer Vision with the OpenCV Library, O\u2019Reilly Media, Inc."},{"key":"ref_49","first-page":"591","article-title":"Introduction to convolutional neural network using Keras: An understanding from a statistician","volume":"26","author":"Lee","year":"2019","journal-title":"Commun. Stat. Appl. Methods"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"12","DOI":"10.3991\/ijim.v11i5.7066","article-title":"A Virtual PLC Environment for Assisting Automation Teaching and Learning","volume":"11","author":"Palma","year":"2017","journal-title":"Int. J. Interact. Mob. Technol. (iJIM)"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Palma, L.B., Brito, V., Rosas, J.A., and Gil, P.S. (2017, January 6\u20138). WEB PLC simulator for ST programming. Proceedings of the 2017 4th Experiment@International Conference (exp.at\u201917), Faro, Portugal.","DOI":"10.1109\/EXPAT.2017.7984410"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Palma, L.B., Rosas, J.A., Pecorelli, J., and Gil, P.S. (2015, January 11\u201313). Structured Text simulator for PLC in learning environment. Proceedings of the 2015 International Conference on Information Technology Based Higher Education and Training (ITHET), Lisbon, Portugal.","DOI":"10.1109\/ITHET.2015.7218039"},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"De Paolis, L., and Bourdot, P. (2018). Multi-user Industrial Training and Education Environment. Augmented Reality, Virtual Reality, and Computer Graphics. AVR 2018, Lecture Notes in Computer Science; Springer.","DOI":"10.1007\/978-3-319-95282-6"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"278","DOI":"10.1016\/j.ifacol.2019.08.221","article-title":"An Approach of Training Virtual Environment for Teaching Electro-Pneumatic Systems","volume":"52","author":"Garcia","year":"2019","journal-title":"IFAC-PapersOnLine"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Naranjo, J.E., Sanchez, D.G., Robalino-Lopez, A., Robalino-Lopez, P., Alarcon-Ortiz, A., and Garcia, M.V. (2020). A Scoping Review on Virtual Reality-Based Industrial Training. Appl. Sci., 10.","DOI":"10.3390\/app10228224"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"10620","DOI":"10.1007\/s11227-021-04212-6","article-title":"Knowledge-based virtual outdoor weather event simulator using unity 3D","volume":"78","author":"Noueihed","year":"2022","journal-title":"J. Supercomput."},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"49735","DOI":"10.1109\/ACCESS.2024.3384986","article-title":"Open-Access Platform for the Simulation of Aerial Robotic Manipulators","volume":"12","author":"Recalde","year":"2024","journal-title":"IEEE Access"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Sarumathi, S., Gokul, M., Gokulavelan, M., and Shimar Roshan, R. (2023). A Review on Virtual Environment using Unity Software. Int. J. Adv. Res. Comput. Commun. Eng., 12.","DOI":"10.17148\/IJARCCE.2023.121115"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/23\/7610\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:41:50Z","timestamp":1760114510000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/23\/7610"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,28]]},"references-count":58,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2024,12]]}},"alternative-id":["s24237610"],"URL":"https:\/\/doi.org\/10.3390\/s24237610","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,28]]}}}