{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,12]],"date-time":"2026-03-12T01:53:32Z","timestamp":1773280412549,"version":"3.50.1"},"reference-count":69,"publisher":"MDPI AG","issue":"4","license":[{"start":{"date-parts":[[2021,4,7]],"date-time":"2021-04-07T00:00:00Z","timestamp":1617753600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>The paper considers a solution to the problem of developing two-stage hybrid SVM-kNN classifiers with the aim to increase the data classification quality by refining the classification decisions near the class boundary defined by the SVM classifier. In the first stage, the SVM classifier with default parameters values is developed. Here, the training dataset is designed on the basis of the initial dataset. When developing the SVM classifier, a binary SVM algorithm or one-class SVM algorithm is used. Based on the results of the training of the SVM classifier, two variants of the training dataset are formed for the development of the kNN classifier: a variant that uses all objects from the original training dataset located inside the strip dividing the classes, and a variant that uses only those objects from the initial training dataset that are located inside the area containing all misclassified objects from the class dividing strip. In the second stage, the kNN classifier is developed using the new training dataset above-mentioned. The values of the parameters of the kNN classifier are determined during training to maximize the data classification quality. The data classification quality using the two-stage hybrid SVM-kNN classifier was assessed using various indicators on the test dataset. In the case of the improvement of the quality of classification near the class boundary defined by the SVM classifier using the kNN classifier, the two-stage hybrid SVM-kNN classifier is recommended for further use. The experimental results approve the feasibility of using two-stage hybrid SVM-kNN classifiers in the data classification problem. The experimental results obtained with the application of various datasets confirm the feasibility of using two-stage hybrid SVM-kNN classifiers in the data classification problem.<\/jats:p>","DOI":"10.3390\/sym13040615","type":"journal-article","created":{"date-parts":[[2021,4,7]],"date-time":"2021-04-07T21:49:06Z","timestamp":1617832146000},"page":"615","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":38,"title":["Two-Stage Hybrid Data Classifiers Based on SVM and kNN Algorithms"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4516-3746","authenticated-orcid":false,"given":"Liliya A.","family":"Demidova","sequence":"first","affiliation":[{"name":"Institute for Information Technologies, Federal State Budget Educational Institution of Higher Education \u201cMIREA\u2013Russian Technological University\u201d, 78, Vernadsky Avenye, 119454 Moscow, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,4,7]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Yu, L., Wang, S., Lai, K.K., and Zhou, L. (2008). BioInspired Credit Risk Analysis, Springer.","DOI":"10.1007\/978-3-540-77803-5"},{"key":"ref_2","first-page":"35","article-title":"Performance evaluation of SVM and K-nearest neighbor algorithm over medical data set","volume":"50","author":"Raikwal","year":"2012","journal-title":"Int. J. Comput. Appl."},{"key":"ref_3","unstructured":"Oh, J.H., Kwon, C., and Cho, S. (1995). Learning algorithms for classification: A comparison on handwritten digit recognition. Neural Networks: The Statistical Mechanics Perspective, World Scientific."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Joachims, T. (1998). Text Categorization with support vector machines: Learning with many relevant features. Lecture Notes in Computer Science, Springer.","DOI":"10.1007\/BFb0026683"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1007\/11559887_19","article-title":"SVM based learning system for information extraction","volume":"Volume 3635","author":"Li","year":"2005","journal-title":"Lecture Notes in Computer Science"},{"key":"ref_6","unstructured":"Oren, M., Papageorgiou, C., Sinha, P., Osuna, E., and Poggio, T. (1997, January 17\u201319). Pedestrian detection using wavelet templates. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico."},{"key":"ref_7","unstructured":"Osuna, E., Freund, R., and Girosi, F. (1997, January 17\u201319). Training support vector machines: An application to face detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, San Juan, Puerto Rico."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., and Friedman, J. (2009). The Elements of Statistical Learning, Springer. [2nd ed.].","DOI":"10.1007\/978-0-387-84858-7"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Mozina, M., Demsar, J., Kattan, M., and Zupan, B. (2004, January 20\u201324). Nomograms for visualization of Naive Bayesian Classifier. Proceedings of the 8th European Conference on Principles and Practice of Knowledge Discovery in Databases, Pisa, Italy.","DOI":"10.1007\/978-3-540-30116-5_32"},{"key":"ref_10","unstructured":"Hu, X., Lin, T., and Louie, E. (2003, January 16\u201318). Bitmap techniques for optimizing decision support queries and association rule algorithms. Proceedings of the 7th International Database Engineering and Applications Symposium, Hong Kong, China."},{"key":"ref_11","unstructured":"Cortez, P., and Silva, A. (2008, January 9\u201311). Using data mining to predict secondary school student performance. Proceedings of the 5th Future Business Technology Conference (FUBUTEC), Porto, Portugal."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"3207","DOI":"10.1162\/NECO_a_00052","article-title":"Deep, big, simple neural nets for handwritten digit recognition","volume":"22","author":"Meier","year":"2010","journal-title":"Neural Comput."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"2135","DOI":"10.1214\/07-AOS537","article-title":"Choice of neighbor order in nearest-neighbor classification","volume":"36","author":"Hall","year":"2008","journal-title":"Ann. Stat."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2412","DOI":"10.1021\/ci060149f","article-title":"Melting point prediction employing k-nearest neighbor algorithms and genetic parameter optimization","volume":"46","author":"Nigsch","year":"2006","journal-title":"J. Chem. Inf. Model."},{"key":"ref_16","first-page":"662","article-title":"Extended k-nearest neighbours based on evidence theory","volume":"47","author":"Wang","year":"2004","journal-title":"Computer"},{"key":"ref_17","unstructured":"Vapnik, V. (1998). Statistical Learning Theory, John Wiley & Sons."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1023\/A:1012450327387","article-title":"Choosing multiple parameters for support vector machines","volume":"46","author":"Chapelle","year":"2002","journal-title":"Mach. Learn."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1023\/A:1009715923555","article-title":"A Tutorial on Support Vector Machines for Pattern Recognition","volume":"2","author":"Burges","year":"1998","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bottou, L., and Lin, C.-J. (2007). Support Vector Machine Solvers, MIT Press.","DOI":"10.7551\/mitpress\/7496.003.0003"},{"key":"ref_21","first-page":"16","article-title":"The SVM classifier based on the modified particle swarm optimization","volume":"7","author":"Demidova","year":"2016","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_22","first-page":"294","article-title":"Big data classification using the SVM classifiers with the modified particle swarm optimization and the SVM ensembles","volume":"7","author":"Demidova","year":"2016","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Demidova, L., and Sokolova, Y. (2015, January 5\u20139). Modification of particle swarm algorithm for the problem of the SVM classifier development. Proceedings of the International Conference \u201cStability and Control Processes\u201d in Memory of V.I. Zubov, Saint-Petersburg, Russia.","DOI":"10.1109\/SCP.2015.7342242"},{"key":"ref_24","first-page":"521","article-title":"Parallel support vector machines: The cascade SVM","volume":"17","author":"Graf","year":"2005","journal-title":"Adv. Neural Inform. Process. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Meyer, O., Bischl, B., and Weihs, C. (2014). Support vector machines on large data sets: Simple parallel approaches. Data Analysis, Machine Learning and Knowledge Discovery, Studies in Classification, Data Analysis, and Knowledge Organization, Springer.","DOI":"10.1007\/978-3-319-01595-8_10"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"77","DOI":"10.14257\/ijdta.2015.8.5.07","article-title":"A map reduce based support vector machine for big data classification","volume":"8","author":"Priyadarshini","year":"2015","journal-title":"Int. J. Database Theory Appl."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"4634","DOI":"10.1109\/JSTARS.2015.2458855","article-title":"On understanding big data impacts in remotely sensed image classification using support vector machine methods","volume":"8","author":"Cavallaro","year":"2015","journal-title":"IEEE Sel. Top. Appl. Earth Obs. Remote. Sens."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Yasodha, P., and Ananthanarayanan, N.R. (2015). Analysing big data to build knowledge based system for early detection of ovarian cancer. Indian J. Sci. Technol., 8.","DOI":"10.17485\/ijst\/2015\/v8i14\/65745"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"130503","DOI":"10.1103\/PhysRevLett.113.130503","article-title":"Quantum support vector machine for big data classification","volume":"113","author":"Rebentrost","year":"2014","journal-title":"Phys. Rev. Lett."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1188","DOI":"10.1109\/72.870050","article-title":"Improvements to the SMO algorithm for SVM regression","volume":"11","author":"Shevade","year":"2000","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_31","unstructured":"Osuna, E., Freund, R., and Girosi, F. (1997, January 24\u201326). An improved training algorithm for support vector machines. Proceedings of the Neural Networks for Signal Processing VII. IEEE Signal Processing Society Workshop, Amelia Island, FL, USA."},{"key":"ref_32","unstructured":"Vishwanathan, S., and Murty, M.N. (2002, January 12\u201317). SSVM: A simple SVM algorithm. Proceedings of the International Joint Conference on Neural Networks, Honolulu, HI, USA."},{"key":"ref_33","first-page":"3","article-title":"Pegasos: Primal estimated sub-gradient solver for SVM","volume":"127","author":"Singer","year":"2010","journal-title":"Math. Program."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"James, G., Witten, D., Hastie, T., and Tibshirani, R. (2013). An Introduction to Statistical Learning with Applications in R, Springer.","DOI":"10.1007\/978-1-4614-7138-7"},{"key":"ref_35","first-page":"493","article-title":"Messy genetic algorithms. motivation analysis, and first results","volume":"5","author":"Goldberg","year":"1989","journal-title":"Complex Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"134","DOI":"10.32362\/2500-316X-2019-7-6-134-150","article-title":"Genetic clustering algorithm","volume":"7","author":"Anfyorov","year":"2020","journal-title":"Russ. Technol. J."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1023\/A:1008202821328","article-title":"Differential evolution\u2014A simple and efficient heuristic for global optimization over continuous spaces","volume":"11","author":"Storn","year":"1997","journal-title":"J. Glob. Optim."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1016\/j.agsy.2004.05.002","article-title":"Differential evolution\u2014An easy and efficient evolutionary algorithm for model optimisation","volume":"83","author":"Mayer","year":"2005","journal-title":"Agric. Syst."},{"key":"ref_39","first-page":"367","article-title":"Application of parallel particle swarm optimize support vector machine model based on hadoop framework in the analysis of railway passenger flow data in China","volume":"46","author":"Xun","year":"2015","journal-title":"Chem. Eng. Trans."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Gazi, V., and Passino, K.M. (2011). Swarm Stability and Optimization, Springer.","DOI":"10.1007\/978-3-642-18041-5"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"39","DOI":"10.14257\/ijbsbt.2015.7.1.05","article-title":"Analytics for the quality of fertility data using particle swarm optimization","volume":"7","author":"Duggal","year":"2015","journal-title":"Int. J. Bio-Sci. Bio-Technol."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"21","DOI":"10.4018\/IJSIR.2018100102","article-title":"Improving the performance of the fish school search algorithm","volume":"9","author":"Monteiro","year":"2018","journal-title":"Int. J. Swarm Intell. Res."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Demidova, L.A., and Gorchakov, A.V. (2020). A study of chaotic maps producing symmetric distributions in the fish school search optimization algorithm with exponential step decay. Symmetry, 12.","DOI":"10.3390\/sym12050784"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Demidova, L.A., and Gorchakov, A.V. (2020). Research and study of the hybrid algorithms based on the collective behavior of fish schools and classical optimization methods. Algorithms, 13.","DOI":"10.3390\/a13040085"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1109\/LGRS.2011.2160150","article-title":"SVMeFC: SVM ensemble fuzzy clustering for satellite image segmentation","volume":"9","author":"Saha","year":"2012","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_46","first-page":"583","article-title":"Cluster ensembles\u2014A knowledge reuse framework for combining multiple partitions","volume":"3","author":"Strehl","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_47","unstructured":"Eastaff, M.S., and Premalatha, P. (2015). Analysis of big data based on ensemble classification. Int. J. Adv. Netw. Appl., 191\u2013193. Available online: http:\/\/www.ijana.in\/Special%20Issue\/file41.pdf."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"2003","DOI":"10.1051\/itmconf\/20160602003","article-title":"Development of the SVM classifier ensemble for the classification accuracy increase","volume":"6","author":"Demidova","year":"2016","journal-title":"ITM Web Conf."},{"key":"ref_49","first-page":"446","article-title":"Use of fuzzy clustering algorithms ensemble for SVM classifier development","volume":"8","author":"Demidova","year":"2015","journal-title":"Int. Rev. Model. Simul."},{"key":"ref_50","unstructured":"Zhang, H., Berg, A.C., Maire, M., and Malik, J. (2006, January 17\u201322). SVM-KNN: Discriminative nearest neighbor classification for visual category recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, NY, USA."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"441","DOI":"10.1088\/1009-9271\/7\/3\/15","article-title":"Support vector machine combined with k-nearest neighbors for solar flare forecasting","volume":"7","author":"Li","year":"2007","journal-title":"Chin. J. Astron. Astrophys."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Demidova, L., and Sokolova, Y. (2017, January 11\u201315). A novel SVM-kNN technique for data classification. Proceedings of the 6th Mediterranean Conference on Embedded Computing (MECO), Bar, Montenegro.","DOI":"10.1109\/MECO.2017.7977132"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"012001","DOI":"10.1088\/1757-899X\/1027\/1\/012001","article-title":"Approbation of the data classification method based on the SVM algorithm and the k nearest neighbors algorithm","volume":"1027","author":"Demidova","year":"2021","journal-title":"IOP Conf. Ser. Mater. Sci. Eng."},{"key":"ref_54","unstructured":"Haibo, H., and Yunqian, M. (2013). Imbalanced Learning: Foundations, Algorithms, and Applications, Wiley-IEEE Press."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1007\/s13748-016-0094-0","article-title":"Learning from imbalanced data: Open challenges and future directions","volume":"5","author":"Krawczyk","year":"2016","journal-title":"Prog. Artif. Intell."},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"1443","DOI":"10.1162\/089976601750264965","article-title":"Estimating the support of a high-dimensional distribution","volume":"13","author":"Platt","year":"2001","journal-title":"Neural Comput."},{"key":"ref_57","doi-asserted-by":"crossref","unstructured":"Liu, F.T., Ting, K.M., and Zhou, Z.-H. (2008, January 15\u201319). Isolation forest. Proceedings of the 8th IEEE International Conference on Data Mining, Pisa, Italy.","DOI":"10.1109\/ICDM.2008.17"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Hubert, M., Debruyne, M., and Rousseeuw, P.J. (2017). Minimum covariance determinant and extensions. Wiley Interdiscip. Rev. Comput. Stat., 10.","DOI":"10.1002\/wics.1421"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Breunig, M.M., Kriegel, H.-P., Ng, R.T., and Sander, J. (2000, January 16\u201318). LOF: Identifying density-based local outliers. Proceedings of the ACM Sigmod International Conference on Management of Data, Dallas, TX, USA.","DOI":"10.1145\/342009.335388"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"765","DOI":"10.1007\/s13042-014-0292-7","article-title":"Two-stage extreme learning machine for high-dimensional data","volume":"7","author":"Liu","year":"2014","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"ref_61","doi-asserted-by":"crossref","unstructured":"Khan, M.A., Karim, R., and Kim, Y. (2018). A Two-stage big data analytics framework with real world applications using spark machine learning and long short-term memory network. Symmetry, 10.","DOI":"10.3390\/sym10100485"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1109\/TCC.2017.2732344","article-title":"Predicting workflow task execution time in the cloud using a two-stage machine learning approach","volume":"8","author":"Pham","year":"2020","journal-title":"IEEE Trans. Cloud Comput."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Khan, S.S., and Madden, M.G. (2010). A survey of recent trends in one class classification. Artificial Intelligence and Cognitive Science. AICS. Lecture Notes in Computer Science, Springer.","DOI":"10.1007\/978-3-642-17080-5_21"},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/j.patcog.2016.03.028","article-title":"High-dimensional and large-scale anomaly detection using a linear one-class SVM with deep learning","volume":"58","author":"Erfani","year":"2016","journal-title":"Pattern Recognit."},{"key":"ref_65","first-page":"120","article-title":"One-class support vector machines for protein-protein interactions prediction","volume":"1","author":"Alashwal","year":"2006","journal-title":"Int. J. Biomed. Sci."},{"key":"ref_66","first-page":"139","article-title":"One-class SVMs for document classification","volume":"2","author":"Manevitz","year":"2001","journal-title":"J. Mach. Learn. Res."},{"key":"ref_67","unstructured":"Li, K.-L., Huang, H.-K., Tian, S.-F., and Xu, W. (2013, January 2\u20135). Improving one-class SVM for anomaly detection. Proceedings of the 2003 International Conference on Machine Learning and Cybernetics, Xi\u2019an, China."},{"key":"ref_68","doi-asserted-by":"crossref","unstructured":"Demidova, L., and Egin, M. (2018, January 7\u201310). Improving the accuracy of the SVM classification using the Parzen classifier. Proceedings of the 7th Mediterranean Conference on Embedded Computing (MECO), Budva, Montenegro.","DOI":"10.1109\/MECO.2018.8406039"},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"012007","DOI":"10.1088\/1742-6596\/1727\/1\/012007","article-title":"The two-stage classification based on 1-SVM and RF classifiers","volume":"1727","author":"Demidova","year":"2021","journal-title":"J. Phys. Conf. Ser."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/13\/4\/615\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T14:11:58Z","timestamp":1760364718000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/13\/4\/615"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,4,7]]},"references-count":69,"journal-issue":{"issue":"4","published-online":{"date-parts":[[2021,4]]}},"alternative-id":["sym13040615"],"URL":"https:\/\/doi.org\/10.3390\/sym13040615","relation":{},"ISSN":["2073-8994"],"issn-type":[{"value":"2073-8994","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,4,7]]}}}