{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T17:59:56Z","timestamp":1784743196723,"version":"3.55.0"},"reference-count":61,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T00:00:00Z","timestamp":1614038400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T00:00:00Z","timestamp":1614038400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Big Data"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Machine learning algorithms efficiently trained on intrusion detection datasets can detect network traffic capable of jeopardizing an information system. In this study, we use the CSE-CIC-IDS2018 dataset to investigate ensemble feature selection on the performance of seven classifiers. CSE-CIC-IDS2018 is big data (about 16,000,000 instances), publicly available, modern, and covers a wide range of realistic attack types. Our contribution is centered around answers to three research questions. The first question is, \u201cDoes feature selection impact performance of classifiers in terms of Area Under the Receiver Operating Characteristic Curve (AUC) and F1-score?\u201d The second question is, \u201cDoes including the Destination_Port categorical feature significantly impact performance of LightGBM and Catboost in terms of AUC and F1-score?\u201d The third question is, \u201cDoes the choice of classifier: Decision Tree (DT), Random Forest (RF), Naive Bayes (NB), Logistic Regression (LR), Catboost, LightGBM, or XGBoost, significantly impact performance in terms of AUC and F1-score?\u201d These research questions are all answered in the affirmative and provide valuable, practical information for the development of an efficient intrusion detection model. To the best of our knowledge, we are the first to use an ensemble feature selection technique with the CSE-CIC-IDS2018 dataset.<\/jats:p>","DOI":"10.1186\/s40537-021-00426-w","type":"journal-article","created":{"date-parts":[[2021,2,23]],"date-time":"2021-02-23T12:06:00Z","timestamp":1614081960000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":60,"title":["Detecting cybersecurity attacks across different network features and learners"],"prefix":"10.1186","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7079-7540","authenticated-orcid":false,"given":"Joffrey L.","family":"Leevy","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"John","family":"Hancock","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Richard","family":"Zuech","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taghi M.","family":"Khoshgoftaar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,2,23]]},"reference":[{"key":"426_CR1","doi-asserted-by":"crossref","unstructured":"Sharafaldin I, Lashkari AH, Ghorbani AA. Toward generating a new intrusion detection dataset and intrusion traffic characterization. In: ICISSP; 2018. p. 108\u2013116.","DOI":"10.5220\/0006639801080116"},{"issue":"3","key":"426_CR2","doi-asserted-by":"publisher","first-page":"357","DOI":"10.1016\/j.cose.2011.12.012","volume":"31","author":"A Shiravi","year":"2012","unstructured":"Shiravi A, Shiravi H, Tavallaee M, Ghorbani AA. Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Comput Secur. 2012;31(3):357\u201374.","journal-title":"Comput Secur"},{"key":"426_CR3","doi-asserted-by":"publisher","first-page":"636","DOI":"10.1016\/j.procs.2020.03.330","volume":"167","author":"A Thakkar","year":"2020","unstructured":"Thakkar A, Lohiya R. A review of the advancement in intrusion detection datasets. Proc Comput Sci. 2020;167:636\u201345.","journal-title":"Proc Comput Sci"},{"key":"426_CR4","doi-asserted-by":"crossref","unstructured":"Wald R, Khoshgoftaar TM, Zuech R, Napolitano A. Network traffic prediction models for near-and long-term predictions. In: 2014 IEEE International Conference on Bioinformatics and Bioengineering. IEEE; 2014. p. 362\u201368","DOI":"10.1109\/BIBE.2014.69"},{"key":"426_CR5","doi-asserted-by":"crossref","unstructured":"Najafabadi MM, Khoshgoftaar TM, Kemp C, Seliya N, Zuech R. Machine learning for detecting brute force attacks at the network level. In: 2014 IEEE International Conference on Bioinformatics and Bioengineering. IEEE; 2014. p. 379\u201385.","DOI":"10.1109\/BIBE.2014.73"},{"key":"426_CR6","doi-asserted-by":"crossref","unstructured":"Bekkar M, Djemaa HK, Alitouche TA. Evaluation measures for models assessment over imbalanced data sets. J Inf Eng Appl. 2013;3(10).","DOI":"10.5121\/ijdkp.2013.3402"},{"key":"426_CR7","doi-asserted-by":"crossref","unstructured":"Wald R, Villanustre F, Khoshgoftaar TM, Zuech R, Robinson J, Muharemagic E. Using feature selection and classification to build effective and efficient firewalls. In: Proceedings of the 2014 IEEE 15th International Conference on Information Reuse and Integration (IEEE IRI 2014). IEEE; 2014. p. 850\u201354.","DOI":"10.1109\/IRI.2014.7051979"},{"issue":"1","key":"426_CR8","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1186\/s40537-018-0151-6","volume":"5","author":"JL Leevy","year":"2018","unstructured":"Leevy JL, Khoshgoftaar TM, Bauder RA, Seliya N. A survey on addressing high-class imbalance in big data. J Big Data. 2018;5(1):42.","journal-title":"J Big Data"},{"issue":"1","key":"426_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-019-0278-0","volume":"7","author":"JL Leevy","year":"2020","unstructured":"Leevy JL, Khoshgoftaar TM. A survey and analysis of intrusion detection models based on cse-cic-ids2018 big data. J Big Data. 2020;7(1):1\u201319.","journal-title":"J Big Data"},{"key":"426_CR10","doi-asserted-by":"crossref","unstructured":"Wang H, Khoshgoftaar TM, Napolitano A. A comparative study of ensemble feature selection techniques for software defect prediction. In: 2010 Ninth International Conference on Machine Learning and Applications. IEEE; 2010. p. 135\u201340.","DOI":"10.1109\/ICMLA.2010.27"},{"key":"426_CR11","doi-asserted-by":"crossref","unstructured":"Leevy JL, Hancock J, Zuech R, Khoshgoftaar TM. Detecting cybersecurity attacks using different network features with lightgbm and xgboost learners. In: 2020 IEEE Second International Conference on Cognitive Machine Intelligence (CogMI). IEEE; 2020. p. 184\u201391.","DOI":"10.1109\/CogMI50398.2020.00032"},{"issue":"01","key":"426_CR12","doi-asserted-by":"publisher","first-page":"1650001","DOI":"10.1142\/S0218539316500017","volume":"23","author":"MM Najafabadi","year":"2016","unstructured":"Najafabadi MM, Khoshgoftaar TM, Seliya N. Evaluating feature selection methods for network intrusion detection with Kyoto data. Int J Reliab Qual Saf Eng. 2016;23(01):1650001.","journal-title":"Int J Reliab Qual Saf Eng"},{"key":"426_CR13","doi-asserted-by":"publisher","first-page":"106034","DOI":"10.1109\/ACCESS.2019.2931865","volume":"7","author":"J-S Lee","year":"2019","unstructured":"Lee J-S. Auc4. 5: Auc-based c4. 5 decision tree algorithm for imbalanced data classification. IEEE Access. 2019;7:106034\u201342.","journal-title":"IEEE Access"},{"issue":"1","key":"426_CR14","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman L. Random forests. Mach Learn. 2001;45(1):5\u201332.","journal-title":"Mach Learn"},{"issue":"2","key":"426_CR15","doi-asserted-by":"publisher","first-page":"88","DOI":"10.18201\/ijisae.2019252786","volume":"7","author":"MM Saritas","year":"2019","unstructured":"Saritas MM, Yasar A. Performance analysis of Ann and Naive Bayes classification algorithm for data classification. Int J Intell Syst Appl Eng. 2019;7(2):88\u201391.","journal-title":"Int J Intell Syst Appl Eng"},{"issue":"15","key":"426_CR16","doi-asserted-by":"publisher","first-page":"3400","DOI":"10.3390\/s19153400","volume":"19","author":"T Rymarczyk","year":"2019","unstructured":"Rymarczyk T, Koz\u0142owski E, K\u0142osowski G, Niderla K. Logistic regression for machine learning in process tomography. Sensors. 2019;19(15):3400.","journal-title":"Sensors"},{"key":"426_CR17","doi-asserted-by":"crossref","unstructured":"Hancock J, Khoshgoftaar TM. Medicare fraud detection using catboost. In: 2020 IEEE 21st International Conference on Information Reuse and Integration for Data Science (IRI). IEEE Computer Society; 2020. p. 97\u2013103.","DOI":"10.1109\/IRI49571.2020.00022"},{"issue":"1","key":"426_CR18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s40537-019-0278-0","volume":"7","author":"JT Hancock","year":"2020","unstructured":"Hancock JT, Khoshgoftaar TM. Catboost for big data: an interdisciplinary review. J Big Data. 2020;7(1):1\u201345.","journal-title":"J Big Data"},{"key":"426_CR19","doi-asserted-by":"crossref","unstructured":"Hancock J, Khoshgoftaar TM. Performance of catboost and xgboost in medicare fraud detection. In: 19th IEEE International Conference On Machine Learning And Applications (ICMLA). IEEE; 2020.","DOI":"10.1109\/ICMLA51294.2020.00095"},{"key":"426_CR20","doi-asserted-by":"crossref","unstructured":"Seiffert C, Khoshgoftaar TM, Van\u00a0Hulse J, Napolitano A. Mining data with rare events: a case study. In: 19th IEEE International Conference on Tools with Artificial Intelligence (ICTAI 2007), vol 2. IEEE; 2007. p. 132\u2013139.","DOI":"10.1109\/ICTAI.2007.71"},{"key":"426_CR21","doi-asserted-by":"crossref","unstructured":"Hua Y. An efficient traffic classification scheme using embedded feature selection and lightgbm. In: 2020 Information Communication Technologies Conference (ICTC). IEEE; 2020. p. 125\u201330.","DOI":"10.1109\/ICTC49638.2020.9123302"},{"key":"426_CR22","doi-asserted-by":"crossref","unstructured":"Yap BW, Abd\u00a0Rani K, Abd\u00a0Rahman HA, Fong S, Khairudin Z, Abdullah NN. An application of oversampling, undersampling, bagging and boosting in handling imbalanced datasets. In: Proceedings of the First International Conference on Advanced Data and Information Engineering (DaEng-2013). Springer; 2014. p. 13\u201322.","DOI":"10.1007\/978-981-4585-18-7_2"},{"key":"426_CR23","doi-asserted-by":"publisher","first-page":"33789","DOI":"10.1109\/ACCESS.2018.2841987","volume":"6","author":"I Ahmad","year":"2018","unstructured":"Ahmad I, Basheri M, Iqbal MJ, Rahim A. Performance comparison of support vector machine, random forest, and extreme learning machine for intrusion detection. IEEE Access. 2018;6:33789\u201395.","journal-title":"IEEE Access"},{"key":"426_CR24","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1016\/j.neucom.2017.02.077","volume":"248","author":"MM Baig","year":"2017","unstructured":"Baig MM, Awais MM, El-Alfy E-SM. Adaboost-based artificial neural network learning. Neurocomputing. 2017;248:120\u20136.","journal-title":"Neurocomputing"},{"key":"426_CR25","doi-asserted-by":"publisher","first-page":"16","DOI":"10.1016\/j.neucom.2018.11.097","volume":"342","author":"J Rynkiewicz","year":"2019","unstructured":"Rynkiewicz J. Asymptotic statistics for multilayer perceptron with Relu hidden units. Neurocomputing. 2019;342:16\u201323.","journal-title":"Neurocomputing"},{"issue":"4","key":"426_CR26","doi-asserted-by":"publisher","first-page":"1363","DOI":"10.1109\/JBHI.2019.2891526","volume":"23","author":"Y Zhao","year":"2019","unstructured":"Zhao Y, Li H, Wan S, Sekuboyina A, Hu X, Tetteh G, Piraud M, Menze B. Knowledge-aided convolutional neural network for small organ segmentation. IEEE J Biomed Health Inform. 2019;23(4):1363\u201373.","journal-title":"IEEE J Biomed Health Inform"},{"key":"426_CR27","first-page":"2825","volume":"12","author":"F Pedregosa","year":"2011","unstructured":"Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E. Scikit-learn: machine learning in Python. J Mach Learn Res. 2011;12:2825\u201330.","journal-title":"J Mach Learn Res"},{"key":"426_CR28","unstructured":"Abadi M, Agarwal A, Barham P, Brevdo E, Chen Z, Citro C, Corrado GS, Davis A, Dean J, Devin M, Ghemawat S, Goodfellow I, Harp A, Irving G, Isard M, Jia Y, Jozefowicz R, Kaiser L, Kudlur M, Levenberg J, Man\u00e9 D, Monga R, Moore S, Murray D, Olah C, Schuster M, Shlens J, Steiner B, Sutskever I, Talwar K, Tucker P, Vanhoucke V, Vasudevan V, Vi\u00e9gas F, Vinyals O, Warden P, Wattenberg M, Wicke M, Yu Y, Zheng X. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Software available from tensorflow.org 2015. https:\/\/www.tensorflow.org\/"},{"issue":"16","key":"426_CR29","doi-asserted-by":"publisher","first-page":"4501","DOI":"10.3390\/s20164501","volume":"20","author":"KS Huancayo Ramos","year":"2020","unstructured":"Huancayo Ramos KS, Sotelo Monge MA, Maestre Vidal J. Benchmark-based reference model for evaluating botnet detection tools driven by traffic-flow analytics. Sensors. 2020;20(16):4501.","journal-title":"Sensors"},{"key":"426_CR30","doi-asserted-by":"crossref","unstructured":"Alenazi A, Traore I, Ganame K, Woungang I. Holistic model for http botnet detection based on dns traffic analysis. In: International Conference on Intelligent, Secure, and Dependable Systems in Distributed and Cloud Environments. Springer; 2017. p. 1\u201318.","DOI":"10.1007\/978-3-319-69155-8_1"},{"key":"426_CR31","doi-asserted-by":"crossref","unstructured":"Vajda S, Santosh K. A fast k-nearest neighbor classifier using unsupervised clustering. In: International Conference on Recent Trends in Image Processing and Pattern Recognition. Springer; 2016. p. 185\u2013193.","DOI":"10.1007\/978-981-10-4859-3_17"},{"key":"426_CR32","doi-asserted-by":"crossref","unstructured":"Gupta V, Bhavsar A. Random forest-based feature importance for hep-2 cell image classification. In: Annual Conference on Medical Image Understanding and Analysis. Springer; 2017. p. 922\u2013934.","DOI":"10.1007\/978-3-319-60964-5_80"},{"key":"426_CR33","doi-asserted-by":"crossref","unstructured":"Yuanyuan S, Yongming W, Lili G, Zhongsong M, Shan J. The comparison of optimizing svm by ga and grid search. In: 2017 13th IEEE International Conference on Electronic Measurement & Instruments (ICEMI). IEEE; 2017. p. 354\u201360.","DOI":"10.1109\/ICEMI.2017.8265815"},{"key":"426_CR34","doi-asserted-by":"publisher","first-page":"101851","DOI":"10.1016\/j.cose.2020.101851","volume":"95","author":"X Li","year":"2020","unstructured":"Li X, Chen W, Zhang Q, Wu L. Building auto-encoder intrusion detection system based on random forest feature selection. Comput Secur. 2020;95:101851.","journal-title":"Comput Secur"},{"key":"426_CR35","doi-asserted-by":"crossref","unstructured":"Chen J, Xie B, Zhang H, Zhai J. Deep autoencoders in pattern recognition: a survey. Bio-inspired Computing Models And Algorithms. 2019;229.","DOI":"10.1142\/9789813143180_0009"},{"key":"426_CR36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.knosys.2016.10.022","volume":"116","author":"Z Wei","year":"2017","unstructured":"Wei Z, Wang Y, He S, Bao J. A novel intelligent method for bearing fault diagnosis based on affinity propagation clustering and adaptive feature selection. Knowl Based Syst. 2017;116:1\u201312.","journal-title":"Knowl Based Syst"},{"key":"426_CR37","doi-asserted-by":"crossref","unstructured":"Mirsky Y, Doitshman T, Elovici Y, Shabtai A. Kitsune: an ensemble of autoencoders for online network intrusion detection 2018. arXiv preprint arXiv:1802.09089","DOI":"10.14722\/ndss.2018.23204"},{"key":"426_CR38","doi-asserted-by":"crossref","unstructured":"Fitni QRS, Ramli K. Implementation of ensemble learning and feature selection for performance improvements in anomaly-based intrusion detection systems. In: 2020 IEEE International Conference on Industry 4.0, Artificial Intelligence, and Communications Technology (IAICT). IEEE; 2020. p. 118\u201324.","DOI":"10.1109\/IAICT50021.2020.9172014"},{"issue":"8","key":"426_CR39","first-page":"42","volume":"8","author":"A Fadlil","year":"2017","unstructured":"Fadlil A, Riadi I, Aji S. Ddos attacks classification using numeric attribute based Gaussian Naive Bayes. Int J Adv Comput Sci Appl (IJACSA). 2017;8(8):42\u201350.","journal-title":"Int J Adv Comput Sci Appl (IJACSA)"},{"key":"426_CR40","doi-asserted-by":"crossref","unstructured":"Elkhalil K, Kammoun A, Couillet R, Al-Naffouri TY, Alouini M-S. Asymptotic performance of regularized quadratic discriminant analysis based classifiers. In: 2017 IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP). IEEE; 2017. p. 1\u20136.","DOI":"10.1109\/MLSP.2017.8168172"},{"issue":"2013","key":"426_CR41","first-page":"332","volume":"1","author":"SM Abd Elrahman","year":"2013","unstructured":"Abd Elrahman SM, Abraham A. A review of class imbalance problem. J Netw Innov Compu. 2013;1(2013):332\u201340.","journal-title":"J Netw Innov Compu"},{"key":"426_CR42","doi-asserted-by":"publisher","first-page":"440","DOI":"10.1016\/j.physa.2016.01.056","volume":"451","author":"W-Y Zhang","year":"2016","unstructured":"Zhang W-Y, Wei Z-W, Wang B-H, Han X-P. Measuring mixing patterns in complex networks by spearman rank correlation coefficient. Phys A Statist Mech Appl. 2016;451:440\u201350.","journal-title":"Phys A Statist Mech Appl"},{"issue":"6","key":"426_CR43","doi-asserted-by":"publisher","first-page":"924","DOI":"10.1080\/10705511.2018.1449653","volume":"25","author":"D Shi","year":"2018","unstructured":"Shi D, DiStefano C, McDaniel HL, Jiang Z. Examining chi-square test statistics under conditions of large model size and ordinal data. Struct Equ Model. 2018;25(6):924\u201345.","journal-title":"Struct Equ Model"},{"key":"426_CR44","first-page":"102564","volume":"54","author":"L D\u2019hooge","year":"2020","unstructured":"D\u2019hooge L, Wauters T, Volckaert B, De Turck FF. Inter-dataset generalization strength of supervised machine learning methods for intrusion detection. J Inf Secur Appl. 2020;54:102564.","journal-title":"J Inf Secur Appl"},{"key":"426_CR45","doi-asserted-by":"crossref","unstructured":"Ta\u015fer PY, Birant KU, Birant D. Comparison of ensemble-based multiple instance learning approaches. In: 2019 IEEE International Symposium on INnovations in Intelligent SysTems and Applications (INISTA). IEEE; 2019. p. 1\u20135.","DOI":"10.1109\/INISTA.2019.8778273"},{"key":"426_CR46","doi-asserted-by":"publisher","first-page":"119","DOI":"10.1016\/j.commatsci.2019.04.051","volume":"166","author":"R Wang","year":"2019","unstructured":"Wang R, Zeng S, Wang X, Ni J. Machine learning for hierarchical prediction of elastic properties in fe-cr-al system. Comput Mater Sci. 2019;166:119\u201323.","journal-title":"Comput Mater Sci"},{"key":"426_CR47","unstructured":"Saikia T, Brox T, Schmid C. Optimized generic feature learning for few-shot classification across domains; 2020. arXiv preprint arXiv:2001.07926"},{"key":"426_CR48","first-page":"6463","volume":"83","author":"S Sulaiman","year":"2020","unstructured":"Sulaiman S, Wahid RA, Ariffin AH, Zulkifli CZ. Question classification based on cognitive levels using linear svc. Test Eng Manag. 2020;83:6463\u201370.","journal-title":"Test Eng Manag"},{"key":"426_CR49","doi-asserted-by":"crossref","unstructured":"Rahman MA, Hossain MA, Kabir MR, Sani MH, Awal MA, et\u00a0al.: Optimization of sleep stage classification using single-channel eeg signals. In: 2019 4th International Conference on Electrical Information and Communication Technology (EICT). IEEE; 2019. p. 1\u20136.","DOI":"10.1109\/EICT48899.2019.9068825"},{"key":"426_CR50","unstructured":"Zuech R, Khoshgoftaar TM. A survey on feature selection for intrusion detection. In: Proceedings of the 21st ISSAT International Conference on Reliability and Quality in Design; 2015. p. 150\u2013155."},{"key":"426_CR51","volume-title":"Data mining: practical machine learning tools and techniques","author":"IH Witten","year":"2011","unstructured":"Witten IH, Frank E, Hall MA. Data mining: practical machine learning tools and techniques. 3rd ed. San Francisco: Morgan Kaufmann Publishers Inc.; 2011.","edition":"3"},{"key":"426_CR52","first-page":"42","volume-title":"Categorical data analysis. Wiley series in probability and mathematical statistics. Applied probability and statistics, applied probability and statistics","author":"A Agresti","year":"1990","unstructured":"Agresti A. Categorical data analysis. Wiley Series in Probability and Mathematical Statistics. Applied probability and statistics, applied probability and statistics. Hoboken: Wiley; 1990. p. 42\u20133."},{"key":"426_CR53","doi-asserted-by":"crossref","unstructured":"Singh R, Kumar H, Singla R. Analysis of feature selection techniques for network traffic dataset. In: 2013 International Conference on Machine Intelligence and Research Advancement. IEEE; 2013. p. 42\u201346.","DOI":"10.1109\/ICMIRA.2013.15"},{"key":"426_CR54","doi-asserted-by":"crossref","unstructured":"Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Stat. 2001;1189\u20131232.","DOI":"10.1214\/aos\/1013203451"},{"issue":"3","key":"426_CR55","doi-asserted-by":"publisher","first-page":"876","DOI":"10.1287\/moor.2016.0831","volume":"42","author":"A Gupta","year":"2017","unstructured":"Gupta A, Nagarajan V, Ravi R. Approximation algorithms for optimal decision trees and adaptive tsp problems. Math Oper Res. 2017;42(3):876\u201396.","journal-title":"Math Oper Res"},{"key":"426_CR56","doi-asserted-by":"publisher","unstructured":"Wes McKinney: Data Structures for Statistical Computing in Python. In: St\u00e9fan van\u00a0der Walt, Jarrod Millman (eds.) Proceedings of the 9th Python in Science Conference; 2010. p. 56\u201361. https:\/\/doi.org\/10.25080\/Majora-92bf1922-00a","DOI":"10.25080\/Majora-92bf1922-00a"},{"key":"426_CR57","unstructured":"Rish I, et\u00a0al. An empirical study of the naive bayes classifier. In: IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence, vol 3; 2001. p. 41\u201346."},{"key":"426_CR58","doi-asserted-by":"crossref","unstructured":"Seliya N, Khoshgoftaar TM, Van\u00a0Hulse J. A study on the relationships of classifier performance metrics. In: Tools with Artificial Intelligence, 2009. ICTAI\u201909. 21st International Conference On. IEEE; 2009. p. 59\u201366.","DOI":"10.1109\/ICTAI.2009.25"},{"key":"426_CR59","doi-asserted-by":"publisher","DOI":"10.4135\/9781412983327","volume-title":"Analysis of variance","author":"GR Iversen","year":"1987","unstructured":"Iversen GR, Wildt AR, Norpoth H, Norpoth HP. Analysis of variance. Thousand Oak: Sage; 1987."},{"key":"426_CR60","doi-asserted-by":"publisher","first-page":"99","DOI":"10.2307\/3001913","volume":"5","author":"JW Tukey","year":"1949","unstructured":"Tukey JW. Comparing individual means in the analysis of variance. Biometrics. 1949;5:99\u2013114.","journal-title":"Biometrics"},{"key":"426_CR61","doi-asserted-by":"publisher","first-page":"105292","DOI":"10.1016\/j.knosys.2019.105292","volume":"192","author":"B Liu","year":"2020","unstructured":"Liu B, Tsoumakas G. Dealing with class imbalance in classifier chains via random undersampling. Knowl Based Syst. 2020;192:105292.","journal-title":"Knowl Based Syst"}],"container-title":["Journal of Big Data"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-021-00426-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/article\/10.1186\/s40537-021-00426-w\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"http:\/\/link.springer.com\/content\/pdf\/10.1186\/s40537-021-00426-w.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,4,1]],"date-time":"2021-04-01T09:00:15Z","timestamp":1617267615000},"score":1,"resource":{"primary":{"URL":"https:\/\/journalofbigdata.springeropen.com\/articles\/10.1186\/s40537-021-00426-w"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,2,23]]},"references-count":61,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["426"],"URL":"https:\/\/doi.org\/10.1186\/s40537-021-00426-w","relation":{},"ISSN":["2196-1115"],"issn-type":[{"value":"2196-1115","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,2,23]]},"assertion":[{"value":"17 December 2020","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"11 February 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 February 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Not applicable.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare that they have no competing interests.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"38"}}