{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,20]],"date-time":"2026-08-20T15:13:31Z","timestamp":1787238811154,"version":"build-2736575974"},"reference-count":27,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T00:00:00Z","timestamp":1760140800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCP"],"abstract":"<jats:p>Intrusion Detection Systems (IDS) play a vital role in safeguarding networks, yet their effectiveness is often challenged, as cyberattacks evolve in new and unexpected ways. Machine learning models, although very powerful, usually perform well only on data that closely resembles what they were trained on. When faced with unfamiliar traffic, they often misclassify. In this work, we examine this generalization gap by training IDS models on one Denial-of-Service (DoS) variant, DoS Hulk, and testing them against other variants such as Goldeneye, Slowloris, and Slowhttptest. Our approach combines careful preprocessing, dimensionality reduction with Principal Component Analysis (PCA), and model training using Random Forests and Deep Neural Networks. To better understand model behavior, we tuned decision thresholds beyond the default 0.5 and found that small adjustments can significantly affect results. We also applied Shapley Additive Explanations (SHAP) to shed light on which features the models rely on, revealing a tendency to focus on fixed components that do not generalize well. Finally, using Uniform Manifold Approximation and Projection (UMAP), we visualized feature distributions and observed overlaps between training and testing datasets, but these did not translate into improved detection performance. Our findings highlight an important lesson: visual or apparent similarity between datasets does not guarantee generalization, and building robust IDS requires exposure to diverse attack patterns during training.<\/jats:p>","DOI":"10.3390\/jcp5040085","type":"journal-article","created":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T14:04:02Z","timestamp":1760537042000},"page":"85","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Evaluating the Generalization Gaps of Intrusion Detection Systems Across DoS Attack Variants"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6782-1741","authenticated-orcid":false,"given":"Roshan","family":"Jameel","sequence":"first","affiliation":[{"name":"Westford University College, Sharjah, United Arab Emirates"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Khyati","family":"Marwah","sequence":"additional","affiliation":[{"name":"Demont Institute of Management and Technology, Dubai, United Arab Emirates"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3342-4187","authenticated-orcid":false,"given":"Sheikh Mohammad","family":"Idrees","sequence":"additional","affiliation":[{"name":"Department of Computer Science (IDI), Norwegian University of Science and Technology, 2802 Gj\u00f8vik, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mariusz","family":"Nowostawski","sequence":"additional","affiliation":[{"name":"Department of Computer Science (IDI), Norwegian University of Science and Technology, 2802 Gj\u00f8vik, Norway"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,10,11]]},"reference":[{"key":"ref_1","unstructured":"Mandiant (2025, July 02). M-Trends 2024: Our View from the Frontlines. Available online: https:\/\/services.google.com\/fh\/files\/misc\/m-trends-2024.pdf."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Aijaz, I., Idrees, S.M., and Agarwal, P. (2021). An Empirical Study on Analysing DDoS Attacks in Cloud Environment. Advances in Intelligent Computing and Communication: Proceedings of ICAC 2020, Springer.","DOI":"10.1007\/978-981-16-0695-3_29"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Bace, R.G., and Mell, P. (2001). Intrusion Detection Systems.","DOI":"10.6028\/NIST.SP.800-31"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Sommer, R., and Paxson, V. (2010, January 16\u201319). Outside the closed world: On using machine learning for network intrusion detection. Proceedings of the 2010 IEEE Symposium on Security and Privacy, Oakland, CA, USA.","DOI":"10.1109\/SP.2010.25"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.cose.2019.06.005","article-title":"A survey of network-based intrusion detection data sets","volume":"86","author":"Ring","year":"2019","journal-title":"Comput. Secur."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"41525","DOI":"10.1109\/ACCESS.2019.2895334","article-title":"Deep learning approach for intelligent intrusion detection system","volume":"7","author":"Vinayakumar","year":"2019","journal-title":"IEEE Access"},{"key":"ref_7","first-page":"43","article-title":"Applying Machine Learning Algorithms for Detecting Phishing Websites: Applications of SVM, KNN, Decision Trees, and Random Forests","volume":"6","year":"2022","journal-title":"Int. J. Inf. Cybersecur."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sharafaldin, I., Lashkari, A.H., and Ghorbani, A.A. (2017). CICIDS 2017 Dataset, Canadian Institute for Cybersecurity, University of New Brunswick. Available online: https:\/\/www.unb.ca\/cic\/datasets\/ids-2017.html.","DOI":"10.13052\/jsn2445-9739.2017.009"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1016\/j.cose.2014.05.011","article-title":"An empirical comparison of botnet detection methods","volume":"45","author":"Garcia","year":"2020","journal-title":"Comput. Secur."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"McInnes, L., Healy, J., and Melville, J. (2018). UMAP: Uniform manifold approximation and projection for dimension reduction. arXiv.","DOI":"10.21105\/joss.00861"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"537","DOI":"10.1093\/comjnl\/bxt031","article-title":"Detecting Distributed Denial of Service Attacks: Methods, Tools and Future Directions","volume":"57","author":"Bhuyan","year":"2013","journal-title":"Comput. J."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1186\/s40537-021-00444-8","article-title":"Review of deep learning: Concepts, CNN architectures, challenges, applications, future directions","volume":"8","author":"Alzubaidi","year":"2021","journal-title":"J. Big Data"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1805","DOI":"10.1007\/s13369-021-06086-5","article-title":"A New Ensemble-Based Intrusion Detection System for Internet of Things","volume":"47","author":"Abbas","year":"2022","journal-title":"Arab. J. Sci. Eng."},{"key":"ref_14","first-page":"120215","article-title":"A hybrid interpretable model for anomaly detection in high-dimensional cybersecurity data","volume":"235","author":"Berahmand","year":"2024","journal-title":"Expert Syst. Appl."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Taher, K.A., Jisan, B.M.Y., and Rahman, M.M. (2019, January 10\u201312). Network intrusion detection using supervised machine learning technique with feature selection. Proceedings of the 2019 International Conference on Robotics, Electrical and Signal Processing Techniques (ICREST), Dhaka, Bangladesh.","DOI":"10.1109\/ICREST.2019.8644161"},{"key":"ref_16","first-page":"446","article-title":"A study on NSL-KDD dataset for intrusion detection system based on classification algorithms","volume":"4","author":"Dhanabal","year":"2015","journal-title":"Int. J. Adv. Res. Comput. Commun. Eng."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"104","DOI":"10.1186\/s40537-020-00382-x","article-title":"A survey and analysis of intrusion detection models based on CSE-CIC-IDS2018 Big Data","volume":"7","author":"Leevy","year":"2020","journal-title":"J. Big Data"},{"key":"ref_18","first-page":"183","article-title":"A survey of machine learning techniques for anomaly detection in cybersecurity","volume":"11","author":"Saabith","year":"2023","journal-title":"Int. J. Res. Eng. Sci."},{"key":"ref_19","first-page":"4765","article-title":"A unified approach to interpreting model predictions","volume":"30","author":"Lundberg","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Javaid, A., Niyaz, Q., Sun, W., and Alam, M. (2015, January 3\u20135). A Deep Learning Approach for Network Intrusion Detection System. Proceedings of the 9th EAI International Conference on Bio-Inspired Information and Communications Technologies (BICT), New York, NY, USA.","DOI":"10.4108\/eai.3-12-2015.2262516"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"21954","DOI":"10.1109\/ACCESS.2017.2762418","article-title":"A deep learning approach for intrusion detection using recurrent neural networks","volume":"5","author":"Yin","year":"2017","journal-title":"IEEE Access"},{"key":"ref_22","first-page":"233","article-title":"A convolutional neural network for improved anomaly- and attack-type classification in network intrusion detection systems","volume":"9","author":"Altwaijry","year":"2021","journal-title":"Biomed. Inform. Insights"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"5152","DOI":"10.1109\/TNSM.2022.3157344","article-title":"The cross-evaluation of machine learning-based network intrusion detection systems","volume":"19","author":"Apruzzese","year":"2022","journal-title":"IEEE Trans. Netw. Service Manag."},{"key":"ref_24","first-page":"108692","article-title":"Explainable Cross-domain Evaluation of ML-based Network Intrusion Detection Systems","volume":"100","author":"Layeghy","year":"2022","journal-title":"Comput. Electr. Eng."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"133","DOI":"10.1186\/s40537-024-00994-7","article-title":"Shielding networks: Enhancing intrusion detection with hybrid feature selection and stack ensemble learning","volume":"11","author":"Alsaffar","year":"2024","journal-title":"J. Big Data"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Cantone, M., Marrocco, C., and Bria, A. (2024). Machine learning in network intrusion detection: A cross-dataset generalization study. arXiv.","DOI":"10.1109\/ACCESS.2024.3472907"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Korniszuk, K., and Sawicki, B. (2024, January 10\u201313). Autoencoder-Based Anomaly Detection in Network Traffic. Proceedings of the 25th International Conference on Computational Problems of Electrical Engineering (CPEE), Stronie \u015al\u0105skie, Poland.","DOI":"10.1109\/CPEE64152.2024.10720411"}],"container-title":["Journal of Cybersecurity and Privacy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2624-800X\/5\/4\/85\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,15]],"date-time":"2025-10-15T14:31:33Z","timestamp":1760538693000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2624-800X\/5\/4\/85"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,11]]},"references-count":27,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["jcp5040085"],"URL":"https:\/\/doi.org\/10.3390\/jcp5040085","relation":{},"ISSN":["2624-800X"],"issn-type":[{"value":"2624-800X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,11]]}}}