{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,28]],"date-time":"2026-07-28T14:43:10Z","timestamp":1785249790976,"version":"3.55.0"},"reference-count":32,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2022,1,24]],"date-time":"2022-01-24T00:00:00Z","timestamp":1642982400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Since the turn of the millennium, the volume of data has increased significantly in both industries and scientific institutions. The processing of these volumes and variety of data we are dealing with are unlikely to be accomplished with conventional software solutions. Thus, new technologies belonging to the big data processing area, able to distribute and process data in a scalable way, are integrated into classical Business Intelligence (BI) systems or replace them. Furthermore, we can benefit from big data technologies to gain knowledge about security, which can be obtained from massive databases. The paper presents a security-relevant data analysis based on the big data analytics engine Apache Spark. A prototype intrusion detection system is developed aimed at detecting data anomalies through machine learning by using the k-means algorithm for clustering analysis implemented in Sparks MLlib. The extraction of features to detect anomalies is currently challenging because the problem of detecting anomalies is not actively and exhaustively monitored. The detection of abnormal data can be effectuated by using relevant data that are already in companies\u2019 and scientific organizations\u2019 possession. Their interpretation and further processing in a continuous manner can sufficiently contribute to anomaly and intrusion detection.<\/jats:p>","DOI":"10.3390\/info13020058","type":"journal-article","created":{"date-parts":[[2022,1,25]],"date-time":"2022-01-25T20:40:18Z","timestamp":1643143218000},"page":"58","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Apache Spark and MLlib-Based Intrusion Detection System or How the Big Data Technologies Can Secure the Data"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5225-389X","authenticated-orcid":false,"given":"Otmane","family":"Azeroual","sequence":"first","affiliation":[{"name":"German Centre for Higher Education Research and Science Studies (DZHW), 10117 Berlin, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0532-3488","authenticated-orcid":false,"given":"Anastasija","family":"Nikiforova","sequence":"additional","affiliation":[{"name":"\u201cInnovative Information Technologies\u201d Laboratory, Programming Department, Faculty of Computing, University of Latvia, Raina Boulevard 19, LV-1050 Riga, Latvia"},{"name":"European Open Science Cloud (EOSC) Task Force \u201cFAIR Metrics and Data Quality\u201d, 1050 Brussels, Belgium"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1311","DOI":"10.1007\/s40747-020-00267-9","article-title":"Industry 4.0, a revolution that requires technology and national strategies","volume":"7","author":"Yang","year":"2021","journal-title":"Complex Intell. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"973","DOI":"10.1016\/j.jcss.2014.02.005","article-title":"A survey of emerging threats in cybersecurity","volume":"80","author":"Nepal","year":"2014","journal-title":"J. Comput. Syst. Sci."},{"key":"ref_3","first-page":"1","article-title":"Metadata and Data Quality Problems in the Digital Library","volume":"6","author":"Beall","year":"2005","journal-title":"J. Digit. Inf."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Yu, S., and Guo, S. (2016). Big Data Concepts, Theories, and Applications, Springer.","DOI":"10.1007\/978-3-319-27763-9"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Dorsche, J. (2015). Praxishandbuch Big Data, Springer.","DOI":"10.1007\/978-3-658-07289-6"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"9163","DOI":"10.1007\/s12652-020-02617-4","article-title":"Big data analytics as a tool for fighting pandemics: A systematic review of literature","volume":"12","author":"Corsi","year":"2021","journal-title":"J. Ambient. Intell. Humaniz. Comput."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"\u00d6zg\u00fcven, Y.M., and Eken, S. (2021). Distributed messaging and light streaming system for combating pandemics. J. Ambient. Intell. Humaniz. Comput., 1\u201315.","DOI":"10.1007\/s12652-021-03328-0"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"131723","DOI":"10.1109\/ACCESS.2020.3009876","article-title":"Data security and privacy protection for cloud storage: A survey","volume":"8","author":"Yang","year":"2020","journal-title":"IEEE Access"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Gonz\u00e1lez-Granadillo, G., Gonz\u00e1lez-Zarzosa, S., and Diaz, R. (2021). Security Information and Event Management (SIEM): Analysis, Trends, and Usage in Critical Infrastructures. Sensors, 21.","DOI":"10.3390\/s21144759"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Daskevics, A., and Nikiforova, A. (2021, January 15\u201316). ShoBeVODSDT: Shodan and Binary Edge based vulnerable open data sources detection tool or what Internet of Things Search Engines know about you. Proceedings of the International Conference on Intelligent Data Science Technologies and Applications (IDSTA2021), Tartu, Estonia.","DOI":"10.1109\/IDSTA53674.2021.9660818"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Daskevics, A., and Nikiforova, A. (2021, January 6\u20139). IoTSE-based Open Database Vulnerability inspection in three Baltic Countries: ShoBEVODSDT sees you. Proceedings of the 8th International Conference on Internet of Things: Systems, Management and Security (IOTSMS), Valencia, Spain.","DOI":"10.1109\/IOTSMS53705.2021.9704952"},{"key":"ref_12","unstructured":"Popescul, D. (2011, January 29\u201330). The Confidentiality\u2013Integrity\u2013Accessibility Triad into the Knowledge Security. A Reassessment from the Point of View of the Knowledge Contribution to Innovation. Proceedings of the 16th International Business Information Management Association Conference (Innovation and Knowledge Management, A Global Competitive Advantage), Kuala Lumpur, Malaysia."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1186\/s40537-015-0013-4","article-title":"Intrusion detection and Big Heterogeneous Data: A Survey","volume":"2","author":"Zuech","year":"2015","journal-title":"J. Big Data."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1186\/s42400-019-0038-7","article-title":"Survey of intrusion detection systems: Techniques, datasets and challenges","volume":"2","author":"Khraisat","year":"2019","journal-title":"Cybersecurity"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Borky, J.M., and Bradley, T.H. (2018). Protecting Information with Cybersecurity. Effective Model-Based Systems Engineering, Springer.","DOI":"10.1007\/978-3-319-95669-5"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Sanders, C., and Smith, J. (2014). The Practice of Applied Network Security Monitoring. Applied Network Security Monitoring, Syngress.","DOI":"10.1016\/B978-0-12-417208-1.00001-5"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Caballero, A. (2014). Information Security Essentials for IT Managers: Protecting Mission-Critical Systems. Computer and Information Security Handbook, Elsevier.","DOI":"10.1016\/B978-0-12-416688-2.00001-5"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Johnson, L. (2020). Chapter 11\u2014Security component fundamentals for assessment. Security Controls Evaluation, Testing, and Assessment Handbook, Academic Press. [2nd ed.].","DOI":"10.1016\/B978-0-12-818427-1.00011-2"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kakareka, A. (2014). Chapter 1\u2014Detecting System Intrusions. Network and System Security, Syngress. [2nd ed.].","DOI":"10.1016\/B978-0-12-416689-9.00001-0"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Scarfone, K., and Mell, P. (2007). Guide to Intrusion Detection and Prevention Systems (IDPS).","DOI":"10.6028\/NIST.SP.800-94"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"145","DOI":"10.1007\/s41060-016-0027-9","article-title":"Big data analytics on Apache Spark","volume":"1","author":"Salloum","year":"2016","journal-title":"Int. J. Data Sci. Anal."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Liu, X., Iftikhar, N., and Xie, X. (2014). Survey of real-time processing systems for big data. Proceedings of the 18th International Database Engineering & Applications Symposium (IDEAS \u201914), Porto, Portugal, 7\u20139 July 2014, Association for Computing Machinery.","DOI":"10.1145\/2628194.2628251"},{"key":"ref_23","unstructured":"White, T. (2012). Hadoop: The Definitive Guide, O\u2019Reilly Media Inc."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Sakr, S. (2016). General-purpose big data processing systems. Big Data 2.0 Processing Systems, Springer.","DOI":"10.1007\/978-3-319-38776-5"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"967","DOI":"10.1007\/s00607-016-0508-7","article-title":"A brief introduction to distributed systems","volume":"98","author":"Tanenbaum","year":"2016","journal-title":"Computing"},{"key":"ref_26","first-page":"1235","article-title":"MLlib: Machine Learning in Apache Spark","volume":"17","author":"Meng","year":"2016","journal-title":"J. Mach. Learn. Res."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"90316","DOI":"10.1109\/ACCESS.2019.2926642","article-title":"Leveraging Machine Learning and Big Data for Smart Buildings: A Comprehensive Survey","volume":"7","author":"Qolomany","year":"2019","journal-title":"IEEE Access"},{"key":"ref_28","unstructured":"Simon, M.K. (2002). Probability Distributions Involving Gaussian Random Variables. A Handbook for Engineers and Scientists, Springer."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1109\/JAS.2017.7510730","article-title":"SVM-DT-based adaptive and collaborative intrusion detection","volume":"5","author":"Teng","year":"2018","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1344","DOI":"10.1109\/JAS.2019.1911723","article-title":"Long-term traffic volume prediction based on K-means Gaussian interval type-2 fuzzy sets","volume":"6","author":"Li","year":"2019","journal-title":"IEEE\/CAA J. Autom. Sin."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Verizon (2021, December 20). 2021 Data Breach Investigations Report (DBIR). Available online: https:\/\/www.verizon.com\/business\/resources\/reports\/2021\/2021-data-breach-investigations-report.pdf.","DOI":"10.1016\/S1361-3723(21)00061-0"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Azeroual, O., and Fabre, R. (2021). Processing Big Data with Apache Hadoop in the Current Challenging Era of COVID-19. Big Data Cogn. Comput., 5.","DOI":"10.3390\/bdcc5010012"}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/58\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:06:30Z","timestamp":1760133990000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/13\/2\/58"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,24]]},"references-count":32,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2022,2]]}},"alternative-id":["info13020058"],"URL":"https:\/\/doi.org\/10.3390\/info13020058","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,1,24]]}}}