{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T04:35:22Z","timestamp":1782880522380,"version":"3.54.5"},"reference-count":49,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2017,9,19]],"date-time":"2017-09-19T00:00:00Z","timestamp":1505779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004083","name":"Ministry of Science ICT and Future Planning","doi-asserted-by":"publisher","award":["IITP-2017-2016-0-00465"],"award-info":[{"award-number":["IITP-2017-2016-0-00465"]}],"id":[{"id":"10.13039\/501100004083","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Building and Services of Information Security System Based on Advanced KREONET (Korea Research Environment Open NETwork)","award":["K-17-L01-C04-S03"],"award-info":[{"award-number":["K-17-L01-C04-S03"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Anomaly detection systems, also known as intrusion detection systems (IDSs), continuously monitor network traffic aiming to identify malicious actions. Extensive research has been conducted to build efficient IDSs emphasizing two essential characteristics. The first is concerned with finding optimal feature selection, while another deals with employing robust classification schemes. However, the advent of big data concepts in anomaly detection domain and the appearance of sophisticated network attacks in the modern era require some fundamental methodological revisions to develop IDSs. Therefore, we first identify two more significant characteristics in addition to the ones mentioned above. These refer to the need for employing specialized big data processing frameworks and utilizing appropriate datasets for validating system\u2019s performance, which is largely overlooked in existing studies. Afterwards, we set out to develop an anomaly detection system that comprehensively follows these four identified characteristics, i.e., the proposed system (i) performs feature ranking and selection using information gain and automated branch-and-bound algorithms respectively; (ii) employs logistic regression and extreme gradient boosting techniques for classification; (iii) introduces bulk synchronous parallel processing to cater computational requirements of high-speed big data networks; and; (iv) uses the Infromation Security Centre of Excellence, of the University of Brunswick real-time contemporary dataset for performance evaluation. We present experimental results that verify the efficacy of the proposed system.<\/jats:p>","DOI":"10.3390\/sym9090197","type":"journal-article","created":{"date-parts":[[2017,9,19]],"date-time":"2017-09-19T11:04:35Z","timestamp":1505819075000},"page":"197","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":19,"title":["Toward Bulk Synchronous Parallel-Based Machine Learning Techniques for Anomaly Detection in High-Speed Big Data Networks"],"prefix":"10.3390","volume":"9","author":[{"given":"Kamran","family":"Siddique","sequence":"first","affiliation":[{"name":"Department of Information and Communication Engineering, Dongguk University, Seoul 04620, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zahid","family":"Akhtar","sequence":"additional","affiliation":[{"name":"INRS-EMT, University of Quebec, Montreal, QC H5A 1K6, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Haeng-gon","family":"Lee","sequence":"additional","affiliation":[{"name":"Korea Institute of Science and Technology Information, Daejeon 34141, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Woongsup","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Information and Communication Engineering, Dongguk University, Seoul 04620, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yangwoo","family":"Kim","sequence":"additional","affiliation":[{"name":"Department of Information and Communication Engineering, Dongguk University, Seoul 04620, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2017,9,19]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Heady, R., Luger, G.F., Maccabe, A., and Servilla, M. (1990). The Architecture of a Network Level Intrusion Detection System, Department of Computer Science, College of Engineering, University of New Mexico. Technical Report.","DOI":"10.2172\/425295"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kim, D.S., and Park, J.S. (2003). Network-based intrusion detection with support vector machines. Information Networking, Springer.","DOI":"10.1007\/978-3-540-45235-5_73"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"11994","DOI":"10.1016\/j.eswa.2009.05.029","article-title":"Intrusion detection by machine learning: A review","volume":"36","author":"Tsai","year":"2009","journal-title":"Expert Syst. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Kim, D.Y., Jeong, Y.S., and Kim, S. (2017). Data-filtering system to avoid total data distortion in IoT networking. Symmetry, 9.","DOI":"10.3390\/sym9010016"},{"key":"ref_5","first-page":"75","article-title":"Data mining in intrusion detection: A comparative study of methods, types and data sets","volume":"5","author":"Azad","year":"2013","journal-title":"Int. J. Inf. Technol. Comput. Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1153","DOI":"10.1109\/COMST.2015.2494502","article-title":"A survey of data mining and machine learning methods for cyber security intrusion detection","volume":"18","author":"Buczak","year":"2016","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1145\/2627534.2627557","article-title":"Big data classification: Problems and challenges in network intrusion prediction with machine learning","volume":"41","author":"Suthaharan","year":"2014","journal-title":"ACM SIGMETRICS Perform. Eval. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1145\/2627534.2627560","article-title":"Security problems and challenges in a machine learning-based hybrid big data processing network systems","volume":"41","author":"Whitworth","year":"2014","journal-title":"ACM SIGMETRICS Perform. Eval. Rev."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1145\/2427036.2427038","article-title":"Toward scalable internet traffic measurement and analysis with hadoop","volume":"43","author":"Lee","year":"2013","journal-title":"ACM SIGCOMM Comput. Commun. Rev."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Grahn, K., Westerlund, M., and Pulkkis, G. (2017). Analytics for Network Security: A Survey and Taxonomy. Information Fusion for Cyber-Security Analytics, Springer.","DOI":"10.1007\/978-3-319-44257-0_8"},{"key":"ref_11","first-page":"24","article-title":"Big data analytics for network intrusion detection: A survey","volume":"7","author":"Wang","year":"2017","journal-title":"Int. J. Netw. Commun."},{"key":"ref_12","unstructured":"Zikopoulos, P., and Eaton, C. (2012). Understanding Big Data: Analytics for Enterprise Class Hadoop and Streaming Data, McGraw-Hill Osborne Media."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"812","DOI":"10.25046\/aj0203102","article-title":"Network intrusion detection system using apache storm","volume":"2","author":"Manzoor","year":"2017","journal-title":"Adv. Sci. Technol. Eng. Syst. J."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3489","DOI":"10.1007\/s11227-015-1615-5","article-title":"Real time intrusion detection system for ultra-high-speed big data environments","volume":"72","author":"Rathore","year":"2016","journal-title":"J. Supercomput."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Janeja, V.P., Azari, A., Namayanja, J.M., and Heilig, B. (2014, January 27\u201330). B-dids: Mining anomalies in a big-distributed intrusion detection system. Proceedings of the 2014 IEEE International Conference on Big Data (Big Data), Washington, DC, USA.","DOI":"10.1109\/BigData.2014.7004484"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1016\/j.cose.2011.12.012","article-title":"Toward developing a systematic approach to generate benchmark datasets for intrusion detection","volume":"31","author":"Shiravi","year":"2012","journal-title":"Comput. Secur."},{"key":"ref_17","unstructured":"Anderson, J.P. (1980). Computer Security Threat Monitoring and Surveillance, James P. Anderson Company. Technical Report."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2322","DOI":"10.1109\/TC.2012.105","article-title":"Traffic-aware design of a high-speed FPGA network intrusion detection system","volume":"62","author":"Pontarelli","year":"2013","journal-title":"IEEE Trans. Comput."},{"key":"ref_19","first-page":"281","article-title":"A comprehensive taxonomy of DDOS attacks and defense mechanism applying in a smart classification","volume":"7","author":"Asosheh","year":"2008","journal-title":"WSEAS Trans. Comput."},{"key":"ref_20","unstructured":"Axelsson, S. (2000). Intrusion Detection Systems: A Survey and Taxonomy, Department of Computer Engineering, Chalmers University of Technology. Technical Report."},{"key":"ref_21","unstructured":"(2017, August 16). Apache Hadoop. Available online: https:\/\/hadoop.apache.org\/."},{"key":"ref_22","unstructured":"(2017, August 16). Apache Spark. Available online: https:\/\/spark.apache.org\/."},{"key":"ref_23","unstructured":"(2017, August 16). Apache Storm. Available online: https:\/\/storm.apache.org\/."},{"key":"ref_24","unstructured":"(2017, August 16). DARPA Intrusion Detection Datasets. Available online: https:\/\/www.ll.mit.edu\/ideval\/data\/index.html."},{"key":"ref_25","unstructured":"(2017, August 16). KDD Cup 1999 Data. Available online: https:\/\/kdd.ics.uci.edu\/databases\/kddcup99\/kddcup99.html."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tavallaee, M., Bagheri, E., Lu, W., and Ghorbani, A.A. (2009, January 8\u201310). A detailed analysis of the KDD CUP 99 data set. Proceedings of the CISDA 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, Ottawa, ON, Canada.","DOI":"10.1109\/CISDA.2009.5356528"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Karim, I., Vien, Q.T., Le, T.A., and Mapp, G. (2017). A comparative experimental design and performance analysis of Snort-based Intrusion Detection System in practical computer networks. Computers, 6.","DOI":"10.3390\/computers6010006"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"981","DOI":"10.1016\/j.jcss.2014.12.012","article-title":"Improving network intrusion detection system performance through quality of service configuration and parallel technology","volume":"81","author":"James","year":"2015","journal-title":"J. Comput. Syst. Sci."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Vasiliadis, G., Polychronakis, M., and Ioannidis, S. (2011, January 17\u201321). MIDeA: A multi-parallel intrusion detection architecture. Proceedings of the 18th ACM Conference on Computer and Communications Security, Chicago, IL, USA.","DOI":"10.1145\/2046707.2046741"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/MCC.2014.53","article-title":"Enhancing big data security with collaborative intrusion detection","volume":"1","author":"Tan","year":"2014","journal-title":"IEEE Cloud Comput."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Marchal, S., Jiang, X., State, R., and Engel, T. (July, January 27). A big data architecture for large scale security monitoring. Proceedings of the 2014 IEEE International Congress on Big Data (BigData Congress), Anchorage, AK, USA.","DOI":"10.1109\/BigData.Congress.2014.18"},{"key":"ref_32","unstructured":"(2017, August 16). MAWI Working Group Traffic Archive. Available online: http:\/\/mawi.wide.ad.jp\/mawi\/."},{"key":"ref_33","first-page":"683","article-title":"Towards generating real-life datasets for network intrusion detection","volume":"17","author":"Bhuyan","year":"2015","journal-title":"IJ Netw. Secur."},{"key":"ref_34","unstructured":"(2017, August 16). The UNSW-NB15 Dataset. Available online: https:\/\/www.unsw.adfa.edu.au\/australian-centre-for-cyber-security\/cybersecurity\/ADFA-NB15-Datasets\/."},{"key":"ref_35","first-page":"18","article-title":"The evaluation of network anomaly detection systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set","volume":"25","author":"Moustafa","year":"2016","journal-title":"Inf. Secur. J."},{"key":"ref_36","unstructured":"Big Data Working Group (2017, August 16). Big Data Analytics for Security Intelligence. Available online: https:\/\/downloads.cloudsecurityalliance.org\/initiatives\/bdwg\/Big_Data_Analytics_for_Security_Intelligence.pdf."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Kalavri, V., and Vlassov, V. (2013, January 16\u201318). Mapreduce: Limitations, optimizations and open issues. Proceedings of the 2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom), Melbourne, VIC, Australia.","DOI":"10.1109\/TrustCom.2013.126"},{"key":"ref_38","unstructured":"(2017, August 16). Apache Hama. Available online: https:\/\/hama.apache.org\/."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1145\/79173.79181","article-title":"A bridging model for parallel computation","volume":"33","author":"Valiant","year":"1990","journal-title":"Commun. ACM"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s11227-017-1987-9","article-title":"Investigating Apache Hama: A bulk synchronous parallel computing framework","volume":"73","author":"Siddique","year":"2017","journal-title":"J. Supercomput."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"8879","DOI":"10.1109\/ACCESS.2016.2631549","article-title":"Apache Hama: An emerging bulk synchronous parallel computing framework for big data applications","volume":"4","author":"Siddique","year":"2016","journal-title":"IEEE Access"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Jakovits, P., Srirama, S.N., and Kromonov, I. (2013, January 1\u20135). Viability of the bulk synchronous parallel model for science on cloud. Proceedings of the 2013 International Conference on High Performance Computing and Simulation (HPCS), Helsinki, Finland.","DOI":"10.1109\/HPCSim.2013.6641391"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Hosmer, D.W., Lemeshow, S., and Sturdivant, R.X. (2013). Applied Logistic Regression, John Wiley & Sons.","DOI":"10.1002\/9781118548387"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). Xgboost: A scalable tree boosting system. Proceedings of the 22nd ACM Sigkdd International Conference on Knowledge Discovery and Data Mining, San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Liu, H., and Motoda, H. (2001). Instance Selection and Construction for Data Mining, Kluwer Academic Publishers.","DOI":"10.1007\/978-1-4757-3359-4"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"James, G., Witten, D., and Hastie, T. (2013). An Introduction to Statistical Learning: With Applications in R, Springer.","DOI":"10.1007\/978-1-4614-7138-7"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"1189","DOI":"10.1214\/aos\/1013203451","article-title":"Greedy function approximation: A gradient boosting machine","volume":"29","author":"Friedman","year":"2001","journal-title":"Ann. Stat."},{"key":"ref_48","unstructured":"Li, P. (arXiv, 2012). Robust Logitboost and Adaptive Base Class (ABC) Logitboost, arXiv."},{"key":"ref_49","unstructured":"(2017, August 16). XGBoost. Available online: https:\/\/github.com\/dmlc\/xgboost."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/9\/9\/197\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:45:19Z","timestamp":1760208319000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/9\/9\/197"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,9,19]]},"references-count":49,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2017,9]]}},"alternative-id":["sym9090197"],"URL":"https:\/\/doi.org\/10.3390\/sym9090197","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,9,19]]}}}