{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,21]],"date-time":"2026-07-21T01:01:01Z","timestamp":1784595661682,"version":"3.55.0"},"reference-count":73,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T00:00:00Z","timestamp":1725408000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T00:00:00Z","timestamp":1725408000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100002954","name":"Universit\u00e0 degli Studi di Milano - Bicocca","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100002954","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Reliable Intell Environ"],"published-print":{"date-parts":[[2024,12]]},"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>A crucial role in the security of modern networks is played by Intrusion Detection Systems (IDSs), security devices designed to identify and mitigate attacks to networks structure. Data-driven approaches based on Machine Learning (ML) have gained more and more popularity for executing the classification tasks required by signature-based IDSs. However, typical ML models adopted for this purpose do not properly take into account the uncertainty associated with their prediction. This poses significant challenges, as they tend to produce misleadingly high classification scores for both misclassified inputs and inputs belonging to unknown classes (e.g. novel attacks), limiting the trustworthiness of existing ML-based solutions. In this paper, we argue that ML-based IDSs should always provide accurate uncertainty quantification to avoid overconfident predictions. In fact, an uncertainty-aware classification would be beneficial to enhance closed-set classification performance, would make it possible to carry out Active Learning, and would help recognize inputs of unknown classes as truly unknowns, unlocking open-set classification capabilities and Out-of-Distribution (OoD) detection. To verify it, we compare various ML-based methods for uncertainty quantification and open-set classification, either specifically designed for or tailored to the domain of network intrusion detection. Moreover, we develop a custom model based on Bayesian Neural Networks that stands out for its OoD detection capabilities and robustness, with a lower variance in the results over different scenarios, compared to other baselines, thus showing how proper uncertainty quantification can be exploited to significantly enhance the trustworthiness of ML-based IDSs.<\/jats:p>","DOI":"10.1007\/s40860-024-00238-8","type":"journal-article","created":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T19:02:16Z","timestamp":1725476536000},"page":"501-520","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Enhancing trustworthiness in ML-based network intrusion detection with uncertainty quantification"],"prefix":"10.1007","volume":"10","author":[{"given":"Jacopo","family":"Talpini","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fabio","family":"Sartori","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marco","family":"Savi","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,9,4]]},"reference":[{"key":"238_CR1","unstructured":"European Union Agency for Cybersecurity (ENISA) (2022) ENISA Threat Landscape 2022. https:\/\/www.enisa.europa.eu\/publications\/enisa-threat-landscape-2022. Accessed 03 Aug 2023"},{"key":"238_CR2","doi-asserted-by":"publisher","unstructured":"Tsimenidids S, Lagkas T, Rantos K (2022) Deep learning in IoT intrusion detection. J Netw Syst Manag 30(01). https:\/\/doi.org\/10.1007\/s10922-021-09621-9","DOI":"10.1007\/s10922-021-09621-9"},{"key":"238_CR3","doi-asserted-by":"publisher","first-page":"82721","DOI":"10.1109\/ACCESS.2019.2924045","volume":"7","author":"V Hassija","year":"2019","unstructured":"Hassija V, Chamola V, Saxena V, Jain D et al (2019) A survey on IoT security: application areas, security threats, and solution architectures. IEEE Access 7:82721\u201382743. https:\/\/doi.org\/10.1109\/ACCESS.2019.2924045","journal-title":"IEEE Access"},{"issue":"1","key":"238_CR4","doi-asserted-by":"publisher","first-page":"41","DOI":"10.1109\/TETCI.2017.2772792","volume":"2","author":"N Shone","year":"2018","unstructured":"Shone N, Ngoc TN, Phai VD, Shi Q (2018) A deep learning approach to network intrusion detection. IEEE Trans Emerg Top Comput Intell 2(1):41\u201350. https:\/\/doi.org\/10.1109\/TETCI.2017.2772792","journal-title":"IEEE Trans Emerg Top Comput Intell"},{"key":"238_CR5","doi-asserted-by":"crossref","unstructured":"Tauscher Z, Jiang Y, Zhang K, Wang J, Song H (2021) Learning to detect: a data-driven approach for network intrusion detection. In: IEEE international performance, computing, and communications conference (IPCCC). IEEE, pp 1\u20136","DOI":"10.1109\/IPCCC51483.2021.9679415"},{"key":"238_CR6","doi-asserted-by":"publisher","first-page":"305","DOI":"10.1109\/SP.2010.25","volume":"2010","author":"R Sommer","year":"2010","unstructured":"Sommer R, Paxson V (2010) Outside the closed world: on using machine learning for network intrusion detection. IEEE Symposium on Security and Privacy 2010:305\u2013316. https:\/\/doi.org\/10.1109\/SP.2010.25","journal-title":"IEEE Symposium on Security and Privacy"},{"issue":"1","key":"238_CR7","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1016\/j.jnca.2012.09.004","volume":"36","author":"H-J Liao","year":"2013","unstructured":"Liao H-J, Lin C-HR, Lin Y-C, Tung K-Y (2013) Intrusion detection system: a comprehensive review. J Netw Comput Appl 36(1):16\u201324","journal-title":"J Netw Comput Appl"},{"issue":"4","key":"238_CR8","doi-asserted-by":"publisher","first-page":"5152","DOI":"10.1109\/tnsm.2022.3157344","volume":"19","author":"G Apruzzese","year":"2022","unstructured":"Apruzzese G, Pajola L, Conti M (2022) The cross-evaluation of machine learning-based network intrusion detection systems. IEEE Trans Netw Serv Manag 19(4):5152\u20135169. https:\/\/doi.org\/10.1109\/tnsm.2022.3157344","journal-title":"IEEE Trans Netw Serv Manag"},{"key":"238_CR9","doi-asserted-by":"publisher","first-page":"90603","DOI":"10.1109\/ACCESS.2021.3090957","volume":"9","author":"T Zoppi","year":"2021","unstructured":"Zoppi T, Ceccarelli A, Bondavalli A (2021) Unsupervised algorithms to detect zero-day attacks: strategy and application. IEEE Access 9:90603\u201390615. https:\/\/doi.org\/10.1109\/ACCESS.2021.3090957","journal-title":"IEEE Access"},{"key":"238_CR10","doi-asserted-by":"crossref","unstructured":"Nguyen A, Yosinski J, Clune J (2015) Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 427\u2013436","DOI":"10.1109\/CVPR.2015.7298640"},{"key":"238_CR11","doi-asserted-by":"crossref","unstructured":"Bendale A, Boult TE (2016) Towards open set deep networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1563\u20131572","DOI":"10.1109\/CVPR.2016.173"},{"key":"238_CR12","unstructured":"Guo C, Pleiss G, Sun Y, Weinberger KQ (2017) On calibration of modern neural networks. In: International conference on machine learning, PMLR, pp 1321\u20131330"},{"key":"238_CR13","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1613\/jair.295","volume":"4","author":"DA Cohn","year":"1996","unstructured":"Cohn DA, Ghahramani Z, Jordan MI (1996) Active learning with statistical models. J Artif Intell Res 4:129\u2013145","journal-title":"J Artif Intell Res"},{"key":"238_CR14","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1109\/SmartCloud.2016.18","volume":"2016","author":"B Varghese","year":"2016","unstructured":"Varghese B, Wang N, Barbhuiya S, Kilpatrick P, Nikolopoulos DS (2016) Challenges and opportunities in edge computing. IEEE international conference on smart cloud (SmartCloud) 2016:20\u201326","journal-title":"IEEE international conference on smart cloud (SmartCloud)"},{"key":"238_CR15","doi-asserted-by":"crossref","unstructured":"Alsaedi A, Moustafa N, Tari Z, Mahmood A, Anwar A (2020) Ton-IoT telemetry dataset: a new generation dataset of Iot and IIoT for data-driven intrusion detection systems. IEEE Access","DOI":"10.1109\/ACCESS.2020.3022862"},{"key":"238_CR16","first-page":"108","volume":"1","author":"I Sharafaldin","year":"2018","unstructured":"Sharafaldin I, Lashkari AH, Ghorbani AA et al (2018) Toward generating a new intrusion detection dataset and intrusion traffic characterization. ICISSp 1:108\u2013116","journal-title":"ICISSp"},{"key":"238_CR17","doi-asserted-by":"publisher","first-page":"2287","DOI":"10.1007\/s11277-019-06986-8","volume":"111","author":"A Verma","year":"2020","unstructured":"Verma A, Ranga V (2020) Machine learning based intrusion detection systems for IoT applications. Wirel Pers Commun 111:2287\u20132310","journal-title":"Wirel Pers Commun"},{"key":"238_CR18","doi-asserted-by":"publisher","first-page":"41525","DOI":"10.1109\/ACCESS.2019.2895334","volume":"7","author":"R Vinayakumar","year":"2019","unstructured":"Vinayakumar R, Alazab M, Soman K, Poornachandran P, Al-Nemrat A, Venkatraman S (2019) Deep learning approach for intelligent intrusion detection system. IEEE Access 7:41525\u201341550","journal-title":"IEEE Access"},{"key":"238_CR19","doi-asserted-by":"publisher","unstructured":"Chen CW, Su CH, Lee KW, Bair PH (2020) Malware family classification using active learning by learning. In: 2020 22nd International conference on advanced communication technology (ICACT), pp 590\u2013595. https:\/\/doi.org\/10.23919\/ICACT48636.2020.9061419","DOI":"10.23919\/ICACT48636.2020.9061419"},{"key":"238_CR20","doi-asserted-by":"publisher","unstructured":"Hajizadeh M, Barua S, Golchin P (2023) FSA-IDS: a flow-based self-active intrusion detection system. In: NOMS 2023-2023 IEEE\/IFIP network operations and management symposium, pp 1\u20139. https:\/\/doi.org\/10.1109\/NOMS56928.2023.10154343","DOI":"10.1109\/NOMS56928.2023.10154343"},{"key":"238_CR21","doi-asserted-by":"publisher","unstructured":"Betsy SW, Murugesan A, Ganapathy NBS, Pughazendi N (2023) A novel framework for network intrusion detection in healthcare domain. In: 2023 4th International conference on signal processing and communication (ICSPC), pp 43\u201346. https:\/\/doi.org\/10.1109\/ICSPC57692.2023.10125636","DOI":"10.1109\/ICSPC57692.2023.10125636"},{"issue":"4","key":"238_CR22","doi-asserted-by":"publisher","first-page":"583","DOI":"10.3390\/sym11040583","volume":"11","author":"MA Khan","year":"2019","unstructured":"Khan MA, Karim MR, Kim Y (2019) A scalable and hybrid intrusion detection system based on the convolutional-LSTM network. Symmetry 11(4):583","journal-title":"Symmetry"},{"key":"238_CR23","doi-asserted-by":"publisher","first-page":"1649","DOI":"10.1007\/s13042-020-01264-7","volume":"12","author":"Z Zhang","year":"2021","unstructured":"Zhang Z, Zhang Y, Guo D, Song M (2021) A scalable network intrusion detection system towards detecting, discovering, and learning unknown attacks. Int J Mach Learn Cybern 12:1649\u20131665","journal-title":"Int J Mach Learn Cybern"},{"key":"238_CR24","unstructured":"Souza MMC, Pontes C, Gondim J, Garcia LPF, DaSilva L, Marotta MA (2022) A novel open set energy-based flow classifier for network intrusion detection. arXiv:2109.11224"},{"key":"238_CR25","first-page":"746","volume":"2019","author":"Y Ye","year":"2019","unstructured":"Ye Y, Zhang T, Yang C (2019) Fisher loss: a more discriminative feature learning method in classification. IEEE\/ASME international conference on advanced intelligent mechatronics (AIM) 2019:746\u2013751","journal-title":"IEEE\/ASME international conference on advanced intelligent mechatronics (AIM)"},{"issue":"1","key":"238_CR26","first-page":"723","volume":"13","author":"A Gretton","year":"2012","unstructured":"Gretton A, Borgwardt KM, Rasch MJ, Sch\u00f6lkopf B, Smola A (2012) A kernel two-sample test. J Mach Learn Res 13(1):723\u2013773","journal-title":"J Mach Learn Res"},{"key":"238_CR27","doi-asserted-by":"publisher","first-page":"175","DOI":"10.1016\/j.comcom.2022.11.001","volume":"198","author":"Y Guo","year":"2022","unstructured":"Guo Y (2022) A review of machine learning-based zero-day attack detection: challenges and future directions. Comput Commun 198:175\u2013185","journal-title":"Comput Commun"},{"issue":"23","key":"238_CR28","doi-asserted-by":"publisher","first-page":"3934","DOI":"10.3390\/electronics11233934","volume":"11","author":"S Ali","year":"2022","unstructured":"Ali S, Rehman SU, Imran A, Adeem G, Iqbal Z, Kim K-I (2022) Comparative evaluation of AI-based techniques for zero-day attacks detection. Electronics 11(23):3934","journal-title":"Electronics"},{"issue":"10","key":"238_CR29","doi-asserted-by":"publisher","first-page":"1684","DOI":"10.3390\/electronics9101684","volume":"9","author":"H Hindy","year":"2020","unstructured":"Hindy H, Atkinson R, Tachtatzis C, Colin J-N, Bayne E, Bellekens X (2020) Utilising deep learning techniques for effective zero-day attack detection. Electronics 9(10):1684","journal-title":"Electronics"},{"key":"238_CR30","first-page":"1050","volume":"2016","author":"Y Gal","year":"2016","unstructured":"Gal Y, Ghahramani Z (2016) Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. International conference on machine learning, PMLR 2016:1050\u20131059","journal-title":"International conference on machine learning, PMLR"},{"key":"238_CR31","first-page":"4697","volume":"33","author":"AG Wilson","year":"2020","unstructured":"Wilson AG, Izmailov P (2020) Bayesian deep learning and a probabilistic perspective of generalization. Adv Neural Inf Process Syst 33:4697\u20134708","journal-title":"Adv Neural Inf Process Syst"},{"key":"238_CR32","unstructured":"Murphy KP (2023) Probabilistic machine learning: advanced topics. MIT Press, Cambridge. http:\/\/probml.github.io\/book2"},{"key":"238_CR33","unstructured":"Mukhoti J, Kirsch A, van Amersfoort J, Torr PH, Gal Y (2021) Deterministic neural networks with inductive biases capture epistemic and aleatoric uncertainty, arXiv preprint. arXiv:2102.11582"},{"issue":"3","key":"238_CR34","doi-asserted-by":"publisher","first-page":"469","DOI":"10.1088\/0954-898X_6_3_011","volume":"6","author":"DJ MacKay","year":"1995","unstructured":"MacKay DJ (1995) Probable networks and plausible predictions-a review of practical Bayesian methods for supervised neural networks. Netw Comput Neural Syst 6(3):469","journal-title":"Netw Comput Neural Syst"},{"key":"238_CR35","first-page":"1184","volume":"2018","author":"S Depeweg","year":"2018","unstructured":"Depeweg S, Hernandez-Lobato J-M, Doshi-Velez F, Udluft S (2018) Decomposition of uncertainty in bayesian deep learning for efficient and risk-sensitive learning. International conference on machine learning, PMLR 2018:1184\u20131193","journal-title":"International conference on machine learning, PMLR"},{"key":"238_CR36","first-page":"1613","volume":"2015","author":"C Blundell","year":"2015","unstructured":"Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D (2015) Weight uncertainty in neural network. International conference on machine learning, PMLR 2015:1613\u20131622","journal-title":"International conference on machine learning, PMLR"},{"key":"238_CR37","unstructured":"Lakshminarayanan B, Pritzel A, Blundell C (2017) Simple and scalable predictive uncertainty estimation using deep ensembles. Adv Neural Inf Process Syst 30"},{"key":"238_CR38","doi-asserted-by":"crossref","unstructured":"Shaker MH, H\u00fcllermeier E (2020) Aleatoric and epistemic uncertainty with random forests. arXiv:2001.00893","DOI":"10.1007\/978-3-030-44584-3_35"},{"key":"238_CR39","unstructured":"Yang J, Zhou K, Li Y, Liu Z (2021) Generalized out-of-distribution detection: a survey, arXiv preprint arXiv:2110.11334"},{"key":"238_CR40","first-page":"21464","volume":"33","author":"W Liu","year":"2020","unstructured":"Liu W, Wang X, Owens J, Li Y (2020) Energy-based out-of-distribution detection. Adv Neural Inf Process Syst 33:21464\u201321475","journal-title":"Adv Neural Inf Process Syst"},{"key":"238_CR41","first-page":"9690","volume":"2020","author":"J Van Amersfoort","year":"2020","unstructured":"Van Amersfoort J, Smith L, Teh YW, Gal Y (2020) Uncertainty estimation using a single deep deterministic neural network. International conference on machine learning, PMLR 2020:9690\u20139700","journal-title":"International conference on machine learning, PMLR"},{"key":"238_CR42","first-page":"7498","volume":"33","author":"J Liu","year":"2020","unstructured":"Liu J, Lin Z, Padhy S, Tran D, Bedrax Weiss T, Lakshminarayanan B (2020) Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Adv Neural Inf Process Syst 33:7498\u20137512","journal-title":"Adv Neural Inf Process Syst"},{"key":"238_CR43","doi-asserted-by":"crossref","unstructured":"Min E, Long J, Liu Q, Cui J, Cai Z, Ma J (2018) SU-IDS: a semi-supervised and unsupervised framework for network intrusion detection. In: Cloud computing and security: 4th international conference, ICCCS 2018, Haikou, China, June 8\u201310, 2018, Revised Selected Papers, Part III 4. Springer, pp 322\u2013334","DOI":"10.1007\/978-3-030-00012-7_30"},{"key":"238_CR44","unstructured":"Jordaney R, Sharad K, Dash SK, Wang Z, Papini D, Nouretdinov I, Cavallaro L (2017) Transcend: detecting concept drift in malware classification models. In: 26th USENIX security symposium (USENIX Security 17), pp 625\u2013642"},{"key":"238_CR45","unstructured":"Houlsby N, Husz\u00e1r F, Ghahramani Z, Lengyel M (2011) Bayesian active learning for classification and preference learning, arXiv preprint arXiv:1112.5745"},{"key":"238_CR46","unstructured":"Gal Y, Islam R, Ghahramani Z (2017) Deep Bayesian active learning with image data. In: International conference on machine learning, PMLR, pp 1183\u20131192"},{"key":"238_CR47","doi-asserted-by":"crossref","unstructured":"Shen Y, Yun H, Lipton ZC, Kronrod Y, Anandkumar A (2017) Deep active learning for named entity recognition, arXiv preprint arXiv:1707.05928","DOI":"10.18653\/v1\/W17-2630"},{"key":"238_CR48","doi-asserted-by":"publisher","unstructured":"Doriguzzi-Corin R, Knob LAD, Mendozzi L, Siracusa D, Savi M (2023) Introducing packet-level analysis in programmable data planes to advance network intrusion detection. https:\/\/doi.org\/10.48550\/arXiv.2307.05936","DOI":"10.48550\/arXiv.2307.05936"},{"key":"238_CR49","unstructured":"Hendrycks D, Gimpel K (2018) A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv:1610.02136"},{"key":"238_CR50","unstructured":"Smith L, Gal Y (2018) Understanding measures of uncertainty for adversarial example detection, arXiv preprint arXiv:1803.08533"},{"key":"238_CR51","first-page":"9690","volume":"2020","author":"J Van Amersfoort","year":"2020","unstructured":"Van Amersfoort J, Smith L, Teh YW, Gal Y (2020) Uncertainty estimation using a single deep deterministic neural network. International conference on machine learning, PMLR 2020:9690\u20139700","journal-title":"International conference on machine learning, PMLR"},{"key":"238_CR52","unstructured":"Miyato T, Kataoka T, Koyama M, Yoshida Y (2018) Spectral normalization for generative adversarial networks. arXiv:1802.05957"},{"key":"238_CR53","doi-asserted-by":"crossref","unstructured":"He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. arXiv:1512.03385","DOI":"10.1109\/CVPR.2016.90"},{"key":"238_CR54","first-page":"7498","volume":"33","author":"J Liu","year":"2020","unstructured":"Liu J, Lin Z, Padhy S, Tran D, Bedrax Weiss T, Lakshminarayanan B (2020) Simple and principled uncertainty estimation with deterministic deep learning via distance awareness. Adv Neural Inf Process Syst 33:7498\u20137512","journal-title":"Adv Neural Inf Process Syst"},{"key":"238_CR55","unstructured":"Clevert D-A, Unterthiner T, Hochreiter S (2015) Fast and accurate deep network learning by exponential linear units (ELUs), arXiv preprint arXiv:1511.07289"},{"key":"238_CR56","volume-title":"Pattern recognition and machine learning","author":"CM Bishop","year":"2006","unstructured":"Bishop CM (2006) Pattern recognition and machine learning, vol 4. Springer, Berlin"},{"issue":"1","key":"238_CR57","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1007\/s11036-021-01843-0","volume":"27","author":"M Sarhan","year":"2021","unstructured":"Sarhan M, Layeghy S, Portmann M (2021) feature set for network intrusion detection system datasets. Mob Netw Appl 27(1):357\u2013370. https:\/\/doi.org\/10.1007\/s11036-021-01843-0","journal-title":"Mob Netw Appl"},{"key":"238_CR58","doi-asserted-by":"publisher","DOI":"10.1016\/j.comnet.2021.108661","volume":"203","author":"EM Campos","year":"2022","unstructured":"Campos EM, Saura PF, Gonz\u00e1lez-Vidal A, Hern\u00e1ndez-Ramos JL, Bernab\u00e9 JB, Baldini G, Skarmeta A (2022) Evaluating federated learning for intrusion detection in internet of things: review and challenges. Comput Netw 203:108661","journal-title":"Comput Netw"},{"key":"238_CR59","doi-asserted-by":"publisher","DOI":"10.1007\/s10922-021-09621-9","author":"S Tsimenidids","year":"2022","unstructured":"Tsimenidids S, Lagkas T, Rantos K (2022) Deep learning in IoT intrusion detection. J Netw Syst Manag. https:\/\/doi.org\/10.1007\/s10922-021-09621-9","journal-title":"J Netw Syst Manag"},{"key":"238_CR60","doi-asserted-by":"crossref","unstructured":"Tauscher Z, Jiang Y, Zhang K (2021) Learning to detect: a data-driven approach for network intrusion detection. In: IEEE international performance, computing, and communications conference (IPCCC). IEEE, vol 2021, pp 1\u20136","DOI":"10.1109\/IPCCC51483.2021.9679415"},{"key":"238_CR61","doi-asserted-by":"publisher","DOI":"10.17487\/rfc3954","volume-title":"Cisco systems NetFlow services export version 9","author":"B Claise","year":"2004","unstructured":"Claise B (2004) Cisco systems NetFlow services export version 9. Tech. rep, IETF RFC"},{"key":"238_CR62","unstructured":"Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167"},{"key":"238_CR63","unstructured":"Kingma DP, Ba J (2017) Adam: a method for stochastic optimization. arXiv:1412.6980"},{"key":"238_CR64","doi-asserted-by":"crossref","unstructured":"Akiba T, Sano S, Yanase T, Ohta T, Koyama M (2019) Optuna: a next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery and data mining, vol 2019, pp 2623\u20132631","DOI":"10.1145\/3292500.3330701"},{"key":"238_CR65","unstructured":"Abadi M, Barham P, Chen J, Chen Z, Davis A, Dean J, Devin M, Ghemawat S, Irving G, Isard M et\u00a0al (2016) Tensorflow: a system for large-scale machine learning. In: 12th $$\\{$$USENIX$$\\}$$ symposium on operating systems design and implementation ($$\\{$$OSDI$$\\}$$ 16), pp 265\u2013283"},{"key":"238_CR66","unstructured":"Dillon JV, Langmore I, Tran D, Brevdo E, Vasudevan S, Moore D, Patton B, Alemi A, Hoffman M, Saurous RA (2017) Tensorflow distributions, arXiv preprint arXiv:1711.10604"},{"key":"238_CR67","unstructured":"Wen Y, Vicol P, Ba J, Tran D, Grosse R (2018) Flipout: efficient pseudo-independent weight perturbations on mini-batches. arXiv:1803.04386"},{"key":"238_CR68","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825\u20132830","journal-title":"J Mach Learn Res"},{"key":"238_CR69","unstructured":"Nixon J, Dusenberry MW, Zhang L, Jerfel G, Tran D (2019) Measuring calibration in deep learning. In: CVPR workshops, pp 38\u201341"},{"key":"238_CR70","unstructured":"Shwartz-Ziv R, Goldblum M, Li YL, Bruss CB, Wilson AG (2023) On representation learning under class imbalance. https:\/\/openreview.net\/forum?id=CPDtGLmXEfy"},{"key":"238_CR71","doi-asserted-by":"crossref","unstructured":"Nguyen HT, Yadegar J, Kong B, Wei H (2012) Efficient batch-mode active learning of random forest. In: IEEE statistical signal processing workshop (SSP). IEEE, pp 596\u2013599","DOI":"10.1109\/SSP.2012.6319769"},{"key":"238_CR72","unstructured":"Smith FB, Kirsch A, Farquhar S, Gal Y, Foster A, Rainforth T (2023) Prediction-oriented Bayesian active learning. In: International conference on artificial intelligence and statistics, PMLR, pp 7331\u20137348"},{"issue":"7","key":"238_CR73","doi-asserted-by":"publisher","first-page":"1145","DOI":"10.1016\/S0031-3203(96)00142-2","volume":"30","author":"AP Bradley","year":"1997","unstructured":"Bradley AP (1997) The use of the area under the roc curve in the evaluation of machine learning algorithms. Pattern Recognit 30(7):1145\u20131159","journal-title":"Pattern Recognit"}],"container-title":["Journal of Reliable Intelligent Environments"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40860-024-00238-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s40860-024-00238-8\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s40860-024-00238-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,3]],"date-time":"2025-03-03T04:19:47Z","timestamp":1740975587000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s40860-024-00238-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,4]]},"references-count":73,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,12]]}},"alternative-id":["238"],"URL":"https:\/\/doi.org\/10.1007\/s40860-024-00238-8","relation":{},"ISSN":["2199-4668","2199-4676"],"issn-type":[{"value":"2199-4668","type":"print"},{"value":"2199-4676","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,4]]},"assertion":[{"value":"29 March 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"7 August 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 September 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}