{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,7]],"date-time":"2026-05-07T10:47:36Z","timestamp":1778150856787,"version":"3.51.4"},"reference-count":33,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2018,8,9]],"date-time":"2018-08-09T00:00:00Z","timestamp":1533772800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001807","name":"Funda\u00e7\u00e3o de Amparo \u00e0 Pesquisa do Estado de S\u00e3o Paulo","doi-asserted-by":"publisher","award":["grant#2017\/01055-4"],"award-info":[{"award-number":["grant#2017\/01055-4"]}],"id":[{"id":"10.13039\/501100001807","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100008982","name":"Qatar National Research Fund","doi-asserted-by":"publisher","award":["NPRP 10-901-2-370"],"award-info":[{"award-number":["NPRP 10-901-2-370"]}],"id":[{"id":"10.13039\/100008982","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>This paper presents the development of a Supervisory Control and Data Acquisition (SCADA) system testbed used for cybersecurity research. The testbed consists of a water storage tank\u2019s control system, which is a stage in the process of water treatment and distribution. Sophisticated cyber-attacks were conducted against the testbed. During the attacks, the network traffic was captured, and features were extracted from the traffic to build a dataset for training and testing different machine learning algorithms. Five traditional machine learning algorithms were trained to detect the attacks: Random Forest, Decision Tree, Logistic Regression, Na\u00efve Bayes and KNN. Then, the trained machine learning models were built and deployed in the network, where new tests were made using online network traffic. The performance obtained during the training and testing of the machine learning models was compared to the performance obtained during the online deployment of these models in the network. The results show the efficiency of the machine learning models in detecting the attacks in real time. The testbed provides a good understanding of the effects and consequences of attacks on real SCADA environments.<\/jats:p>","DOI":"10.3390\/fi10080076","type":"journal-article","created":{"date-parts":[[2018,8,9]],"date-time":"2018-08-09T10:36:31Z","timestamp":1533810991000},"page":"76","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":155,"title":["SCADA System Testbed for Cybersecurity Research Using Machine Learning Approach"],"prefix":"10.3390","volume":"10","author":[{"given":"Marcio","family":"Teixeira","sequence":"first","affiliation":[{"name":"Department of Informatics, Federal Institute of Education, Science, and Technology of Sao Paulo, Catanduva 15808-305, SP, Brazil"},{"name":"Department of Computer Science and Engineering, Washington University in Saint Louis, Saint Louis, MO 63130, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tara","family":"Salman","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Washington University in Saint Louis, Saint Louis, MO 63130, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maede","family":"Zolanvari","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Washington University in Saint Louis, Saint Louis, MO 63130, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7023-0368","authenticated-orcid":false,"given":"Raj","family":"Jain","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Washington University in Saint Louis, Saint Louis, MO 63130, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nader","family":"Meskin","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, Qatar University, Doha 2713, Qatar"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mohammed","family":"Samaka","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Qatar University, Doha 2713, Qatar"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,8,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Arag\u00f3, A.S., Mart\u00ednez, E.R., and Clares, S.S. (2014, January 11\u201312). SCADA laboratory and test-bed as a service for critical infrastructure protection. Proceedings of the 2nd International Symposium on ICS & SCADA Cyber Security Research, St P\u00f6lten, Austria.","DOI":"10.14236\/ewic\/ICSCSR2014.4"},{"key":"ref_2","unstructured":"National Communications Systems (NCS) (2018, August 08). Supervisory Control and Data Acquisition (SCADA) Systems, Technical Information Bulletin 04-1. Available online: https:\/\/www.cedengineering.com\/userfiles\/SCADA%20Systems.pdf."},{"key":"ref_3","unstructured":"Filkins, B. (2018, June 05). IT Security Spending Trends. Available online: https:\/\/www.sans.org\/reading-room\/whitepapers\/analyst\/security-spending-trends-36697."},{"key":"ref_4","unstructured":"(2018, June 05). NIST Special Publication 800-82, Revision 2, Available online: http:\/\/nvlpubs.nist.gov\/nistpubs\/SpecialPublications\/NIST.SP.800-82r2.pdf."},{"key":"ref_5","unstructured":"(2017, December 05). Modbus TCP\/IP. Available online: http:\/\/www.modbus.org\/tech.php."},{"key":"ref_6","unstructured":"(2018, August 08). Modbus Application Protocol Specification V1.1b3. Available online: http:\/\/www.modbus.org\/docs\/Modbus_Application_Protocol_V1_1b3.pdf."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Butts, J., and Shenoi, S. (2014). Industrial control system traffic data sets for intrusion detection research. Critical Infrastructure Protection VIII. ICCIP 2014. IFIP Advances in Information and Communication Technology, Springer.","DOI":"10.1007\/978-3-662-45355-1"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Miciolino, E.E., Bernieri, G., Pascucci, F., and Setola, R. (2015, January 24\u201326). Communications network analysis in a SCADA system testbed under cyber-attacks. Proceedings of the 23rd Telecommunications Forum, Belgrade, Serbia.","DOI":"10.1109\/TELFOR.2015.7377479"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Rosa, L., Cruz, T., Sim\u00f5es, P., Monteiro, E., and Lev, L. (2017, January 8\u201312). Attacking SCADA systems: A practical perspective. Proceedings of the IFIP\/IEEE Symposium on Integrated Network and Service Management (IM), Lisbon, Portugal.","DOI":"10.23919\/INM.2017.7987369"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Keliris, A., Salehghaffari, H., and Cairl, B. (2016, January 15\u201317). Machine learning-based defense against process-aware attacks on industrial control systems. Proceedings of the IEEE International Test Conference (ITC), Fort Worth, TX, USA.","DOI":"10.1109\/TEST.2016.7805855"},{"key":"ref_11","unstructured":"Tomin, N.V., Kurbatsky, V.G., Sidorov, D.N., and Zhukov, A.V. (2016, January 11\u201313). Machine learning techniques for power system security assessment. Proceedings of the IFAC Workshop on Control of Transmission and Distribution Smart Grids (CTDSG), Prague, Czech Republic."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cose.2015.09.009","article-title":"A review of cyber security risk assessment methods for SCADA systems","volume":"56","author":"Cherdantseva","year":"2016","journal-title":"Comput. Secur."},{"key":"ref_13","unstructured":"(2018, June 03). An Industrial Control System Cybersecurity Performance Testbed, Available online: http:\/\/nvlpubs.nist.gov\/nistpubs\/ir\/2015\/NIST.IR.8089.pdf."},{"key":"ref_14","unstructured":"(2018, June 03). DNP3. Available online: https:\/\/www.dnp.org\/Pages\/AboutDefault.aspx."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Darwish, I., Igbe, O., and Saadawi, T. (2016, January 20\u201322). Experimental and theoretical modeling of DNP3 attacks in smart grids. Proceedings of the 36th IEEE Sarnoff Symposium, Newark, NJ, USA.","DOI":"10.1109\/SARNOF.2015.7324661"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2178","DOI":"10.1109\/JIOT.2018.2826558","article-title":"Understanding the usage of industrial control system devices on the internet","volume":"5","author":"Li","year":"2018","journal-title":"IEEE Internet Things J."},{"key":"ref_17","unstructured":"(2018, August 08). Schneider PLC M241CE40. Available online: https:\/\/www.schneider-electric.us\/en\/product\/TM241CE40R\/controller-m241-40-io-relay-ethernet\/."},{"key":"ref_18","unstructured":"Erickson, K.T. (2011). Programmable Logic Controllers: An Emphasis on Design and Application, Dogwood Valley Press, LLC."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Mantere, M., Uusitalo, I., Sailio, M., and Noponen, S. (2012, January 26\u201329). Challenges of machine learning based monitoring for industrial control system networks. Proceedings of the 26th International Conference on Advanced Information Networking and Applications Workshops, Fukuoka, Japan.","DOI":"10.1109\/WAINA.2012.135"},{"key":"ref_20","unstructured":"Jordan, M.I., and Ng, A.Y. (2001, January 3\u20138). On discriminative vs. generative classifiers: A comparison of logistic regression and naive bayes. Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic, Vancouver, BC, Canada."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1109\/TSMCC.2008.923876","article-title":"Random-forests-based network intrusion detection systems","volume":"38","author":"Zhang","year":"2008","journal-title":"IEEE Trans. Syst. Man Cybern. Part C"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Amor, N.B., Benferhat, S., and Elouedi, Z. (2004, January 14\u201317). Naive bayes vs. decision trees in intrusion detection systems. Proceedings of the 2004 ACM Symposium on Applied Computing, Nicosia, Cyprus.","DOI":"10.1145\/967900.967989"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2617","DOI":"10.1016\/j.cor.2004.03.019","article-title":"Application of SVM and ANN for intrusion detection","volume":"32","author":"Chen","year":"2005","journal-title":"Comput. Oper. Res."},{"key":"ref_24","unstructured":"Zhang, H., Berg, A.C., Maire, M., and Malik, J. (2006, January 17\u201322). SVM-KNN: Discriminative nearest neighbor classification for visual category recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, NY, USA."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"427","DOI":"10.1016\/j.ipm.2009.03.002","article-title":"A systematic analysis of performance measures for classification tasks","volume":"45","author":"Sokolova","year":"2009","journal-title":"J. Inf. Process. Manag."},{"key":"ref_26","unstructured":"Buda, M., Maki, A., and Mazurowski, M.A. (2017, November 20). A Systematic Study of the Class Imbalance Problem in Convolutional Neural Networks. Available online: https:\/\/arxiv.org\/pdf\/1710.05381.pdf."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","article-title":"Learning from imbalanced data","volume":"21","author":"He","year":"2009","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_28","unstructured":"Calderon, P. (2017). Nmap: Network Exploration and Security Auditing Cookbook, Packet Publishing. [2nd ed.]."},{"key":"ref_29","unstructured":"(2017, January 30). Vulnerability & Exploit Database, Modbus Client Utility. Available online: https:\/\/www.rapid7.com\/db\/modules\/auxiliary\/scanner\/scada\/modbusclient."},{"key":"ref_30","unstructured":"(2017, October 20). Wireshark. Available online: https:\/\/www.wireshark.org\/."},{"key":"ref_31","unstructured":"(2017, November 10). ARGUS. Available online: https:\/\/qosient.com\/argus\/."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"460","DOI":"10.3390\/fi5040460","article-title":"Network traffic features for anomaly detection in specific industrial control system network","volume":"5","author":"Mantere","year":"2013","journal-title":"Futur. Internet"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Salman, T., Bhamare, D., Erbad, A., Jain, R., and Samaka, M. (2017, January 26\u201328). Machine learning for anomaly detection and categorization in multi-cloud environments. Proceedings of the 4th IEEE International Conference on Cyber Security and Cloud Computing, New York, NY, USA.","DOI":"10.1109\/CSCloud.2017.15"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/10\/8\/76\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:17:38Z","timestamp":1760195858000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/10\/8\/76"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,8,9]]},"references-count":33,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2018,8]]}},"alternative-id":["fi10080076"],"URL":"https:\/\/doi.org\/10.3390\/fi10080076","relation":{},"ISSN":["1999-5903"],"issn-type":[{"value":"1999-5903","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,8,9]]}}}