{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,10]],"date-time":"2026-07-10T10:49:54Z","timestamp":1783680594718,"version":"3.55.0"},"reference-count":93,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2023,10,17]],"date-time":"2023-10-17T00:00:00Z","timestamp":1697500800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Dr. Yang\u2019s startup funding and NSF award","award":["2018919"],"award-info":[{"award-number":["2018919"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Utilizing machine learning (ML)-based approaches for network intrusion detection systems (NIDSs) raises valid concerns due to the inherent susceptibility of current ML models to various threats. Of particular concern are two significant threats associated with ML: adversarial attacks and distribution shifts. Although there has been a growing emphasis on researching the robustness of ML, current studies primarily concentrate on addressing specific challenges individually. These studies tend to target a particular aspect of robustness and propose innovative techniques to enhance that specific aspect. However, as a capability to respond to unexpected situations, the robustness of ML should be comprehensively built and maintained in every stage. In this paper, we aim to link the varying efforts throughout the whole ML workflow to guide the design of ML-based NIDSs with systematic robustness. Toward this goal, we conduct a methodical evaluation of the progress made thus far in enhancing the robustness of the targeted NIDS application task. Specifically, we delve into the robustness aspects of ML-based NIDSs against adversarial attacks and distribution shift scenarios. For each perspective, we organize the literature in robustness-related challenges and technical solutions based on the ML workflow. For instance, we introduce some advanced potential solutions that can improve robustness, such as data augmentation, contrastive learning, and robustness certification. According to our survey, we identify and discuss the ML robustness research gaps and future direction in the field of NIDS. Finally, we highlight that building and patching robustness throughout the life cycle of an ML-based NIDS is critical.<\/jats:p>","DOI":"10.3390\/computers12100209","type":"journal-article","created":{"date-parts":[[2023,10,17]],"date-time":"2023-10-17T08:10:19Z","timestamp":1697530219000},"page":"209","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["On the Robustness of ML-Based Network Intrusion Detection Systems: An Adversarial and Distribution Shift Perspective"],"prefix":"10.3390","volume":"12","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3646-9479","authenticated-orcid":false,"given":"Minxiao","family":"Wang","sequence":"first","affiliation":[{"name":"The Computer Engineering Program in the School of Electrical, Computer, and Biomedical Engineering, Southern Illinois University, Carbondale, IL 62901, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4837-6632","authenticated-orcid":false,"given":"Ning","family":"Yang","sequence":"additional","affiliation":[{"name":"The Information Technology Program in the School of Computing, Southern Illinois University, Carbondale, IL 62901, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8537-7082","authenticated-orcid":false,"given":"Dulaj H.","family":"Gunasinghe","sequence":"additional","affiliation":[{"name":"The Computer Engineering Program in the School of Electrical, Computer, and Biomedical Engineering, Southern Illinois University, Carbondale, IL 62901, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4869-8383","authenticated-orcid":false,"given":"Ning","family":"Weng","sequence":"additional","affiliation":[{"name":"The Computer Engineering Program in the School of Electrical, Computer, and Biomedical Engineering, Southern Illinois University, Carbondale, IL 62901, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,10,17]]},"reference":[{"key":"ref_1","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). Imagenet Classification with Deep Convolutional Neural Networks. Proceedings of the 26th Annual Conference on Neural Information Processing Systems 2012, Lake Tahoe, NA, USA."},{"key":"ref_2","unstructured":"Hannun, A., Case, C., Casper, J., Catanzaro, B., Diamos, G., Elsen, E., Prenger, R., Satheesh, S., Sengupta, S., and Coates, A. (2014). Deep speech: Scaling up end-to-end speech recognition. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"101851","DOI":"10.1016\/j.cose.2020.101851","article-title":"Building auto-encoder intrusion detection system based on random forest feature selection","volume":"95","author":"Li","year":"2020","journal-title":"Comput. Secur."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Mirsky, Y., Doitshman, T., Elovici, Y., and Shabtai, A. (2018, January 18\u201321). Kitsune: An Ensemble of Autoencoders for Online Network Intrusion Detection. Proceedings of the 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, CA, USA.","DOI":"10.14722\/ndss.2018.23204"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Tocchetti, A., Corti, L., Balayn, A., Yurrita, M., Lippmann, P., Brambilla, M., and Yang, J. (2023, August 18). AI Robustness: A Human-Centered Perspective on Technological Challenges and Opportunities, Available online: http:\/\/xxx.lanl.gov\/abs\/2210.08906.","DOI":"10.1145\/3665926"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"261","DOI":"10.1038\/s42256-019-0055-y","article-title":"Establishing the rules for building trustworthy AI","volume":"1","author":"Floridi","year":"2019","journal-title":"Nat. Mach. Intell."},{"key":"ref_7","unstructured":"Hoffman, W. (2021). Making AI Work for Cyber Defense, Center for Security and Emerging Technology."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/IOTM.001.2300106","article-title":"Toward a Reliable Evaluation of Machine Learning Schemes for Network-Based Intrusion Detection","volume":"6","author":"Viegas","year":"2023","journal-title":"IEEE Internet Things Mag."},{"key":"ref_9","unstructured":"Wei, F., Li, H., Zhao, Z., and Hu, H. (2023, January 9\u201311). XNIDS: Explaining Deep Learning-Based Network Intrusion Detection Systems for Active Intrusion Responses. Proceedings of the 32nd USENIX Security Symposium (USENIX Security 23), Anaheim, CA, USA."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"140","DOI":"10.1109\/MNET.011.2000088","article-title":"AI for beyond 5G networks: A cyber-security defense or offense enabler?","volume":"34","author":"Taleb","year":"2020","journal-title":"IEEE Netw."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1007\/s42979-021-00592-x","article-title":"Machine learning: Algorithms, real-world applications and research directions","volume":"2","author":"Sarker","year":"2021","journal-title":"SN Comput. Sci."},{"key":"ref_12","first-page":"103121","article-title":"Towards a robust and trustworthy machine learning system development: An engineering perspective","volume":"65","author":"Xiong","year":"2022","journal-title":"J. Inf. Secur. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1109\/MC.2022.3218005","article-title":"AI Maintenance: A Robustness Perspective","volume":"56","author":"Chen","year":"2023","journal-title":"Computer"},{"key":"ref_14","unstructured":"Drenkow, N., Sani, N., Shpitser, I., and Unberath, M. (2021). A systematic review of robustness in deep learning for computer vision: Mind the gap?. arXiv."},{"key":"ref_15","unstructured":"Teney, D., Lin, Y., Oh, S.J., and Abbasnejad, E. (2022). Id and ood performance are sometimes inversely correlated on real-world datasets. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3469659","article-title":"Modeling realistic adversarial attacks against network intrusion detection systems","volume":"3","author":"Apruzzese","year":"2022","journal-title":"Digit. Threat. Res. Pract. DTRAP"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Mbow, M., Sakurai, K., and Koide, H. (2022, January 10\u201312). Advances in Adversarial Attacks and Defenses in Intrusion Detection System: A Survey. Proceedings of the International Conference on Science of Cyber Security, Matsue, Japan.","DOI":"10.1007\/978-981-19-7769-5_15"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"538","DOI":"10.1109\/COMST.2022.3233793","article-title":"Adversarial machine learning for network intrusion detection systems: A comprehensive survey","volume":"25","author":"He","year":"2023","journal-title":"IEEE Commun. Surv. Tutor."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"109073","DOI":"10.1016\/j.comnet.2022.109073","article-title":"Adversarial machine learning for network intrusion detection: A comparative study","volume":"214","author":"Jmila","year":"2022","journal-title":"Comput. Netw."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e295","DOI":"10.1002\/spy2.295","article-title":"Multi-aspects AI-based modeling and adversarial learning for cybersecurity intelligence and robustness: A comprehensive overview","volume":"6","author":"Sarker","year":"2023","journal-title":"Secur. Priv."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2523813","article-title":"A survey on concept drift adaptation","volume":"46","author":"Gama","year":"2014","journal-title":"ACM Comput. Surv. CSUR"},{"key":"ref_22","first-page":"2346","article-title":"Learning under concept drift: A review","volume":"31","author":"Lu","year":"2018","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Adnan, A., Muhammed, A., Abd Ghani, A.A., Abdullah, A., and Hakim, F. (2021). An intrusion detection system for the internet of things based on machine learning: Review and challenges. Symmetry, 13.","DOI":"10.3390\/sym13061011"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Nixon, C., Sedky, M., and Hassan, M. (2021, January 6\u20139). Reviews in Online Data Stream and Active Learning for Cyber Intrusion Detection\u2014A Systematic Literature Review. Proceedings of the 2021 Sixth International Conference on Fog and Mobile Edge Computing (FMEC), Gandia, Spain.","DOI":"10.1109\/FMEC54266.2021.9732566"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Li, B., Qi, P., Liu, B., Di, S., Liu, J., Pei, J., Yi, J., and Zhou, B. (2022). Trustworthy AI: From Principles to Practices. arXiv.","DOI":"10.1145\/3555803"},{"key":"ref_26","unstructured":"Kloft, M., and Laskov, P. (2010, January 13\u201315). Online Anomaly Detection under Adversarial Impact. Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, JMLR Workshop and Conference Proceedings, Sardinia, Italy."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Clements, J., Yang, Y., Sharma, A.A., Hu, H., and Lao, Y. (2021, January 4\u20137). Rallying Adversarial Techniques against Deep Learning for Network Security. Proceedings of the 2021 IEEE Symposium Series on Computational Intelligence (SSCI), Orlando, FL, USA.","DOI":"10.1109\/SSCI50451.2021.9660011"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Wu, D., Fang, B., Wang, J., Liu, Q., and Cui, X. (2019, January 20\u201324). Evading Machine Learning Botnet Detection Models via Deep Reinforcement Learning. Proceedings of the ICC 2019\u20142019 IEEE International Conference on Communications (ICC), Shanghai, China.","DOI":"10.1109\/ICC.2019.8761337"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"3225","DOI":"10.1109\/TIFS.2022.3201377","article-title":"Tantra: Timing-based adversarial network traffic reshaping attack","volume":"17","author":"Sharon","year":"2022","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_30","first-page":"6","article-title":"When training and test sets are different: Characterizing learning transfer","volume":"30","author":"Storkey","year":"2009","journal-title":"Dataset Shift Mach. Learn."},{"key":"ref_31","unstructured":"Huyen, C. (2022). Designing Machine Learning Systems, O\u2019Reilly Media, Inc."},{"key":"ref_32","unstructured":"Bommasani, R., Hudson, D.A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M.S., Bohg, J., Bosselut, A., and Brunskill, E. (2021). On the opportunities and risks of foundation models. arXiv."},{"key":"ref_33","unstructured":"Sagawa, S., Koh, P.W., Hashimoto, T.B., and Liang, P. (2019). Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization. arXiv."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1792","DOI":"10.1109\/ACCESS.2017.2780250","article-title":"HAST-IDS: Learning hierarchical spatial-temporal features using deep neural networks to improve intrusion detection","volume":"6","author":"Wang","year":"2017","journal-title":"IEEE Access"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"876","DOI":"10.1109\/TNSM.2020.2971776","article-title":"LUCID: A practical, lightweight deep learning solution for DDoS attack detection","volume":"17","author":"Millar","year":"2020","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_36","first-page":"32854","article-title":"AnoShift: A distribution shift benchmark for unsupervised anomaly detection","volume":"35","author":"Dragoi","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Jagielski, M., Oprea, A., Biggio, B., Liu, C., Nita-Rotaru, C., and Li, B. (2018, January 20\u201324). Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning. Proceedings of the 2018 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA.","DOI":"10.1109\/SP.2018.00057"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1563","DOI":"10.1109\/TPAMI.2022.3162397","article-title":"Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses","volume":"45","author":"Goldblum","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Nguyen, T.D., Rieger, P., Miettinen, M., and Sadeghi, A.R. (2020, January 23\u201326). Poisoning Attacks on Federated Learning-Based IoT Intrusion Detection System. Proceedings of the Decentralized IoT Systems and Security (DISS), San Diego, CA, USA.","DOI":"10.14722\/diss.2020.23003"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1016\/j.future.2022.04.010","article-title":"SecFedNIDS: Robust defense for poisoning attack against federated learning-based network intrusion detection system","volume":"134","author":"Zhang","year":"2022","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"103205","DOI":"10.1016\/j.cose.2023.103205","article-title":"Two-phase Defense Against Poisoning Attacks on Federated Learning-based Intrusion Detection","volume":"129","author":"Lai","year":"2023","journal-title":"Comput. Secur."},{"key":"ref_42","unstructured":"Zhang, H., Yu, X., Ren, P., Luo, C., and Min, G. (2019). Deep adversarial learning in intrusion detection: A data augmentation enhanced framework. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Yuan, D., Ota, K., Dong, M., Zhu, X., Wu, T., Zhang, L., and Ma, J. (2020, January 7\u201311). Intrusion Detection for Smart Home Security Based on Data Augmentation with Edge Computing. Proceedings of the ICC 2020\u20142020 IEEE International Conference on Communications (ICC), Dublin, Ireland.","DOI":"10.1109\/ICC40277.2020.9148632"},{"key":"ref_44","unstructured":"Geirhos, R., Rubisch, P., Michaelis, C., Bethge, M., Wichmann, F.A., and Brendel, W. (2018). ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness. arXiv."},{"key":"ref_45","unstructured":"Hendrycks, D., Mu, N., Cubuk, E.D., Zoph, B., Gilmer, J., and Lakshminarayanan, B. (2019). Augmix: A simple data processing method to improve robustness and uncertainty. arXiv."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Hendrycks, D., Basart, S., Mu, N., Kadavath, S., Wang, F., Dorundo, E., Desai, R., Zhu, T., Parajuli, S., and Guo, M. (2021, January 20\u201325). The Many Faces of Robustness: A Critical Analysis of Uut-of-Distribution Generalization. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Nashville, TN, USA.","DOI":"10.1109\/ICCV48922.2021.00823"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Wei, J., and Zou, K. (2019). Eda: Easy data augmentation techniques for boosting performance on text classification tasks. arXiv.","DOI":"10.18653\/v1\/D19-1670"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Chen, J., Yang, Z., and Yang, D. (2020). Mixtext: Linguistically-informed interpolation of hidden space for semi-supervised text classification. arXiv.","DOI":"10.18653\/v1\/2020.acl-main.194"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Xie, R., Cao, J., Dong, E., Xu, M., Sun, K., Li, Q., Shen, L., and Zhang, M. (2023, January 28\u201330). Rosetta: Enabling Robust TLS Encrypted Traffic Classification in Diverse Network Environments with TCP-Aware Traffic Augmentation. Proceedings of the ACM Turing Award Celebration Conference, Wuhan, China.","DOI":"10.1145\/3603165.3607437"},{"key":"ref_50","unstructured":"Gao, I., Sagawa, S., Koh, P.W., Hashimoto, T., and Liang, P. (2023). Out-of-Domain Robustness via Targeted Augmentations. arXiv."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Divekar, A., Parekh, M., Savla, V., Mishra, R., and Shirole, M. (2018, January 25\u201327). Benchmarking Datasets for Anomaly-Based Network Intrusion Detection: KDD CUP 99 Alternatives. Proceedings of the 2018 IEEE 3rd International Conference on Computing, Communication and Security (ICCCS), Kathmandu, Nepal.","DOI":"10.1109\/CCCS.2018.8586840"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Deng, L., Zhao, Y., and Bao, H. (2022, January 16\u201317). A Self-supervised Adversarial Learning Approach for Network Intrusion Detection System. Proceedings of the Cyber Security, Beijing, China.","DOI":"10.1007\/978-981-19-8285-9_5"},{"key":"ref_53","unstructured":"Bostani, H., Zhao, Z., Liu, Z., and Moonsamy, V. (2023, August 12). Level Up with RealAEs: Leveraging Domain Constraints in Feature Space to Strengthen Robustness of Android Malware Detection, Available online: http:\/\/xxx.lanl.gov\/abs\/2205.15128."},{"key":"ref_54","first-page":"857","article-title":"Self-Supervised Learning: Generative or Contrastive","volume":"35","author":"Liu","year":"2023","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_55","first-page":"18661","article-title":"Supervised Contrastive Learning","volume":"Volume 33","author":"Larochelle","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Liu, L., Wang, P., Ruan, J., and Lin, J. (Res. Sq., 2022). ConFlow: Contrast Network Flow Improving Class-Imbalanced Learning in Network Intrusion Detection, Res. Sq., preprint.","DOI":"10.21203\/rs.3.rs-1572776\/v1"},{"key":"ref_57","unstructured":"Tong, L., Li, B., Hajaj, C., Xiao, C., Zhang, N., and Vorobeychik, Y. (2019, January 14\u201316). Improving Robustness of ML Classifiers against Realizable Evasion Attacks Using Conserved Features. Proceedings of the 28th USENIX Security Symposium (USENIX Security 19), Santa Clara, CA, USA."},{"key":"ref_58","unstructured":"Dodge, J., Ilharco, G., Schwartz, R., Farhadi, A., Hajishirzi, H., and Smith, N. (2020). Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv."},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Wang, J., Pan, J., AlQerm, I., and Liu, Y. (2021, January 19\u201322). Def-IDS: An Ensemble Defense Mechanism Against Adversarial Attacks for Deep Learning-Based Network Intrusion Detection. Proceedings of the 2021 International Conference on Computer Communications and Networks (ICCCN), Athens, Greece.","DOI":"10.1109\/ICCCN52240.2021.9522215"},{"key":"ref_60","first-page":"15","article-title":"Cert-RNN: Towards Certifying the Robustness of Recurrent Neural Networks","volume":"21","author":"Du","year":"2021","journal-title":"CCS"},{"key":"ref_61","unstructured":"Shi, Z., Zhang, H., Chang, K.W., Huang, M., and Hsieh, C.J. (2020). Robustness verification for transformers. arXiv."},{"key":"ref_62","unstructured":"Cohen, J., Rosenfeld, E., and Kolter, Z. (2019, January 9\u201315). Certified Adversarial Robustness via Randomized Smoothing. Proceedings of the International Conference on Machine Learning (PMLR), Long Beach, CA, USA."},{"key":"ref_63","unstructured":"Yang, G., Duan, T., Hu, J.E., Salman, H., Razenshteyn, I., and Li, J. (2020, January 13\u201318). Randomized Smoothing of All Shapes and Sizes. Proceedings of the International Conference on Machine Learning (PMLR), Virtual Event."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"110626","DOI":"10.1016\/j.knosys.2023.110626","article-title":"DI-NIDS: Domain invariant network intrusion detection system","volume":"273","author":"Layeghy","year":"2023","journal-title":"Knowl.-Based Syst."},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Qu, Y., Ma, H., Jiang, Y., and Bu, Y. (2023). A Network Intrusion Detection Method Based on Domain Confusion. Electronics, 12.","DOI":"10.3390\/electronics12051255"},{"key":"ref_66","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18\u201324). Learning Transferable Visual Models from Natural Language Supervision. Proceedings of the International Conference on Machine Learning (PMLR), Virtual Event."},{"key":"ref_67","unstructured":"Kumar, A., Raghunathan, A., Jones, R., Ma, T., and Liang, P. (2022). Fine-Tuning can Distort Pretrained Features and Underperform Out-of-Distribution. arXiv."},{"key":"ref_68","unstructured":"Gunel, B., Du, J., Conneau, A., and Stoyanov, V. (2023, July 07). Supervised Contrastive Learning for Pre-Trained Language Model Fine-Tuning, Available online: http:\/\/xxx.lanl.gov\/abs\/2011.01403."},{"key":"ref_69","doi-asserted-by":"crossref","unstructured":"Yan, Y., Li, R., Wang, S., Zhang, F., Wu, W., and Xu, W. (2021, January 1\u20136). ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Online.","DOI":"10.18653\/v1\/2021.acl-long.393"},{"key":"ref_70","doi-asserted-by":"crossref","unstructured":"Li, L., Weber, M., Xu, X., Rimanic, L., Kailkhura, B., Xie, T., Zhang, C., and Li, B. (2021, January 15\u201319). Tss: Transformation-Specific Smoothing for Robustness Certification. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, Online.","DOI":"10.1145\/3460120.3485258"},{"key":"ref_71","unstructured":"Wang, K., Wang, Z., Han, D., Chen, W., Yang, J., Shi, X., and Yin, X. (March, January 27). BARS: Local Robustness Certification for Deep Learning based Traffic Analysis Systems. Proceedings of the NDSS, San Diego, CA, USA."},{"key":"ref_72","unstructured":"Pal, A., and Sulam, J. (2023). Understanding Noise-Augmented Training for Randomized Smoothing. arXiv."},{"key":"ref_73","doi-asserted-by":"crossref","first-page":"12","DOI":"10.1007\/s10922-021-09615-7","article-title":"Towards model generalization for intrusion detection: Unsupervised machine learning techniques","volume":"30","author":"Verkerken","year":"2022","journal-title":"J. Netw. Syst. Manag."},{"key":"ref_74","doi-asserted-by":"crossref","unstructured":"Al-Riyami, S., Coenen, F., and Lisitsa, A. (2018, January 15\u201319). A Re-Evaluation of Intrusion Detection Accuracy: Alternative Evaluation Strategy. Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, Toronto, ON, Canada.","DOI":"10.1145\/3243734.3278490"},{"key":"ref_75","first-page":"815","article-title":"Cross-Datasets Evaluation of Machine Learning Models for Intrusion Detection Systems","volume":"Volume 4","author":"Lisitsa","year":"2022","journal-title":"Proceedings of the Sixth International Congress on Information and Communication Technology: ICICT 2021"},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"5152","DOI":"10.1109\/TNSM.2022.3157344","article-title":"The cross-evaluation of machine learning-based network intrusion detection systems","volume":"19","author":"Apruzzese","year":"2022","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"108692","DOI":"10.1016\/j.compeleceng.2023.108692","article-title":"Explainable Cross-domain Evaluation of ML-based Network Intrusion Detection Systems","volume":"108","author":"Layeghy","year":"2023","journal-title":"Comput. Electr. Eng."},{"key":"ref_78","doi-asserted-by":"crossref","unstructured":"Peng, X., Huang, W., and Shi, Z. (2019, January 4\u20136). Adversarial Attack against DoS Intrusion Detection: An Improved Boundary-Based Method. Proceedings of the 2019 IEEE 31st International Conference on Tools with Artificial Intelligence (ICTAI), Portland, OR, USA.","DOI":"10.1109\/ICTAI.2019.00179"},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"1962","DOI":"10.1109\/TNSM.2021.3052888","article-title":"Adversarial network traffic: Towards evaluating the robustness of deep-learning-based network traffic classification","volume":"18","author":"Sadeghzadeh","year":"2021","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"2632","DOI":"10.1109\/JSAC.2021.3087242","article-title":"Evaluating and improving adversarial robustness of machine learning-based network intrusion detectors","volume":"39","author":"Han","year":"2021","journal-title":"IEEE J. Sel. Areas Commun."},{"key":"ref_81","doi-asserted-by":"crossref","first-page":"2295","DOI":"10.1109\/TNSM.2022.3173933","article-title":"Sneaking Through Security: Mutating Live Network Traffic to Evade Learning-Based NIDS","volume":"19","author":"Tan","year":"2022","journal-title":"IEEE Trans. Netw. Serv. Manag."},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Peng, Y., Fu, G., Luo, Y., Hu, J., Li, B., and Yan, Q. (2020, January 16\u201318). Detecting Adversarial Examples for Network Intrusion Detection System with GAN. Proceedings of the 2020 IEEE 11th International Conference on Software Engineering and Service Science (ICSESS), Beijing, China.","DOI":"10.1109\/ICSESS49938.2020.9237728"},{"key":"ref_83","unstructured":"Donahue, J., Kr\u00e4henb\u00fchl, P., and Darrell, T. (2016). Adversarial feature learning. arXiv."},{"key":"ref_84","doi-asserted-by":"crossref","unstructured":"Wang, N., Chen, Y., Hu, Y., Lou, W., and Hou, Y.T. (2021, January 10\u201313). MANDA: On Adversarial Example Detection for Network Intrusion Detection System. Proceedings of the IEEE INFOCOM 2021\u2014IEEE Conference on Computer Communications, Vancouver, BC, Canada.","DOI":"10.1109\/INFOCOM42981.2021.9488874"},{"key":"ref_85","doi-asserted-by":"crossref","first-page":"1294","DOI":"10.1109\/TNET.2021.3137084","article-title":"Adversarial attacks against deep learning-based network intrusion detection systems and defense mechanisms","volume":"30","author":"Zhang","year":"2022","journal-title":"IEEE\/ACM Trans. Netw."},{"key":"ref_86","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/2766959","article-title":"Learning visual similarity for product design with convolutional neural networks","volume":"34","author":"Bell","year":"2015","journal-title":"ACM Trans. Graph. TOG"},{"key":"ref_87","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1007\/BF00116900","article-title":"Learning in the presence of concept drift and hidden contexts","volume":"23","author":"Widmer","year":"1996","journal-title":"Mach. Learn."},{"key":"ref_88","doi-asserted-by":"crossref","unstructured":"Andresini, G., Appice, A., Loglisci, C., Belvedere, V., Redavid, D., and Malerba, D. (2021, January 11\u201313). A Network Intrusion Detection System for Concept Drifting Network Traffic Data. Proceedings of the Discovery Science: 24th International Conference, DS 2021, Halifax, NS, Canada. Proceedings 24.","DOI":"10.1007\/978-3-030-88942-5_9"},{"key":"ref_89","first-page":"20","article-title":"A review on concept drift","volume":"17","author":"Kadwe","year":"2015","journal-title":"IOSR J. Comput. Eng."},{"key":"ref_90","doi-asserted-by":"crossref","unstructured":"Andresini, G., Pendlebury, F., Pierazzi, F., Loglisci, C., Appice, A., and Cavallaro, L. (2021, January 15). Insomnia: Towards Concept-Drift Robustness in Network Intrusion Detection. Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security, Virtual Event.","DOI":"10.1145\/3474369.3486864"},{"key":"ref_91","unstructured":"Yang, L., Guo, W., Hao, Q., Ciptadi, A., Ahmadzadeh, A., Xing, X., and Wang, G. (2021, January 11\u201313). CADE: Detecting and Explaining Concept Drift Samples for Security Applications. Proceedings of the 30th USENIX Security Symposium (USENIX Security 21), Online."},{"key":"ref_92","doi-asserted-by":"crossref","unstructured":"Tavallaee, M., Bagheri, E., Lu, W., and Ghorbani, A.A. (2009, January 8\u201310). A Detailed Analysis of the KDD CUP 99 Data Set. Proceedings of the 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications, Ottawa, ON, Canada.","DOI":"10.1109\/CISDA.2009.5356528"},{"key":"ref_93","first-page":"171","article-title":"Service-Independent Payload Analysis to Improve Intrusion Detection in Network Traffic","volume":"Volume 87","author":"Perona","year":"2008","journal-title":"Proceedings of the 7th Australasian Data Mining Conference"}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/12\/10\/209\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:08:24Z","timestamp":1760130504000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/12\/10\/209"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,17]]},"references-count":93,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["computers12100209"],"URL":"https:\/\/doi.org\/10.3390\/computers12100209","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,17]]}}}