{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T02:12:11Z","timestamp":1772590331335,"version":"3.50.1"},"reference-count":67,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2018,9,29]],"date-time":"2018-09-29T00:00:00Z","timestamp":1538179200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100004281","name":"Narodowe Centrum Nauki","doi-asserted-by":"publisher","award":["2017\/01\/X\/ST6\/00202"],"award-info":[{"award-number":["2017\/01\/X\/ST6\/00202"]}],"id":[{"id":"10.13039\/501100004281","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>The purpose of instance selection is to reduce the data size while preserving as much useful information stored in the data as possible and detecting and removing the erroneous and redundant information. In this work, we analyze instance selection in regression tasks and apply the NSGA-II multi-objective evolutionary algorithm to direct the search for the optimal subset of the training dataset and the k-NN algorithm for evaluating the solutions during the selection process. A key advantage of the method is obtaining a pool of solutions situated on the Pareto front, where each of them is the best for certain RMSE-compression balance. We discuss different parameters of the process and their influence on the results and put special efforts to reducing the computational complexity of our approach. The experimental evaluation proves that the proposed method achieves good performance in terms of minimization of prediction error and minimization of dataset size.<\/jats:p>","DOI":"10.3390\/e20100746","type":"journal-article","created":{"date-parts":[[2018,10,2]],"date-time":"2018-10-02T08:23:50Z","timestamp":1538468630000},"page":"746","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["Multi-Objective Evolutionary Instance Selection for Regression Tasks"],"prefix":"10.3390","volume":"20","author":[{"given":"Miros\u0142aw","family":"Kordos","sequence":"first","affiliation":[{"name":"Department of Computer Science and Automatics, University of Bielsko-Bia\u0142a, ul. Willowa 2, 43-309 Bielsko-Bia\u0142a, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3926-5685","authenticated-orcid":false,"given":"Krystian","family":"\u0141apa","sequence":"additional","affiliation":[{"name":"Institute of Computational Intelligence, Cz\u0119stochowa University of Technology, 42-201 Cz\u0119stochowa, Poland"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,9,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Garc\u00eda, S., Luengo, J., and Herrera, F. (2015). Data Preprocessing in Data Mining, Springer.","DOI":"10.1007\/978-3-319-10247-4"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"417","DOI":"10.1109\/TPAMI.2011.142","article-title":"Prototype selection for nearest neighbor classification: Taxonomy and empirical study","volume":"34","author":"Garcia","year":"2012","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Blachnik, M. (2017). Instance Selection for Classifier Performance Estimation in Meta Learning. Entropy, 19.","DOI":"10.3390\/e19110583"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.inffus.2015.12.002","article-title":"Fusion of instance selection methods in regression tasks","volume":"30","author":"Blachnik","year":"2016","journal-title":"Inf. Fusion"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"787","DOI":"10.1142\/S0218001405004332","article-title":"Decision boundary preserving prototype selection for nearest neighbor classification","volume":"19","author":"Barandela","year":"2005","journal-title":"Int. J. Pattern Recog. Artif. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"586","DOI":"10.1016\/j.procs.2017.01.176","article-title":"Entropy-Based Classifier Enhancement to Handle Imbalanced Class Problem","volume":"104","author":"Kirshners","year":"2017","journal-title":"Procedia Comput. Sci."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Michalowicz, J.V., Nichols, J.M., and Bucholtz, F. (2013). Handbook of Differential Entropy, Chapman and Hall\/CRC.","DOI":"10.1201\/b15991"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Rusiecki, A., Kordos, M., Kami\u0144ski, T., and Gre\u0144, K. (2014). Training neural networks on noisy data. International Conference on Artificial Intelligence and Soft Computing, Springer.","DOI":"10.1007\/978-3-319-07173-2_13"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1109\/4235.996017","article-title":"A fast and elitist multiobjective genetic algorithm: NSGA-II","volume":"6","author":"Deb","year":"2002","journal-title":"IEEE Trans. Evolut. Comput."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1109\/TIT.1968.1054155","article-title":"The condensed nearest neighbor rule (Corresp.)","volume":"14","author":"Hart","year":"1968","journal-title":"IEEE Trans. Inf. Theory"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1023\/A:1007626913721","article-title":"Reduction techniques for instance-based learning algorithms","volume":"38","author":"Wilson","year":"2000","journal-title":"Mach. Learn."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1009","DOI":"10.1016\/S0031-3203(02)00119-X","article-title":"Finding representative patterns with ordered projections","volume":"36","author":"Riquelme","year":"2003","journal-title":"Pattern Recogn."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1023\/A:1014043630878","article-title":"Advances in instance selection for instance-based learning algorithms","volume":"6","author":"Brighton","year":"2002","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1016\/j.inffus.2015.04.002","article-title":"INFFC: An iterative class noise filter based on the fusion of classifiers with noise sensitivity control","volume":"27","author":"Galar","year":"2016","journal-title":"Inf. Fusion"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1523","DOI":"10.1016\/j.patcog.2014.10.001","article-title":"Three new instance selection methods based on local sets: A comparative study with several approaches from a bi-objective perspective","volume":"48","author":"Leyva","year":"2015","journal-title":"Pattern Recogn."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"2923","DOI":"10.3233\/JIFS-169478","article-title":"Accurate and fast prototype selection based on the notion of relevant and border prototypes","volume":"34","year":"2018","journal-title":"J. Intell. Fuzzy Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Suganthi, M., and Karunakaran, V. (2018). Instance selection and feature extraction using cuttlefish optimization algorithm and principal component analysis using decision tree. Cluster Comput., 1\u201313.","DOI":"10.1007\/s10586-018-1821-z"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Krawczyk, B., Triguero, I., Garc\u00eda, S., Wo\u017aniak, M., and Herrera, F. (2018). Instance reduction for one-class classification. Knowl. Inf. Syst., 1\u201328.","DOI":"10.1007\/s10115-018-1220-z"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Son, S.H., and Kim, J.Y. (2006). Data reduction for instance-based learning using entropy-based partitioning. International Conference on Computational Science and Its Applications, Springer.","DOI":"10.1007\/11751595_63"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kajdanowicz, T., Plamowski, S., and Kazienko, P. (2011, January 6\u20138). Training set selection using entropy based distance. Proceedings of the 2011 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT), Amman, Jordan.","DOI":"10.1109\/AEECT.2011.6132530"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"114","DOI":"10.1016\/j.eswa.2018.05.017","article-title":"Study of data transformation techniques for adapting single-label prototype selection algorithms to multi-label learning","volume":"109","year":"2018","journal-title":"Expert Syst. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Grochowski, M., and Jankowski, N. (2004). Comparison of instance selection algorithms II. Results and comments. International Conference on Artificial Intelligence and Soft Computing, Springer.","DOI":"10.1007\/978-3-540-24844-6_87"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Zhang, J., Yim, Y.S., and Yang, J. (1997). Intelligent selection of instances for prediction functions in lazy learning algorithms. Lazy Learning, Springer.","DOI":"10.1007\/978-94-017-2053-3_7"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Kordos, M., and Blachnik, M. (2012). Instance selection with neural networks for regression problems. International Conference on Artificial Neural Networks, Springer.","DOI":"10.1007\/978-3-642-33266-1_33"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1016\/j.neucom.2016.04.003","article-title":"Instance selection for regression: Adapting DROP","volume":"201","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"340","DOI":"10.1016\/j.eswa.2015.12.046","article-title":"Instance selection for regression by discretization","volume":"54","year":"2016","journal-title":"Expert Syste. Appl."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Rodriguez-Fdez, I., Mucientes, M., and Bugarin, A. (2013, January 7\u201310). An instance selection algorithm for regression and its application in variance reduction. Proceedings of the 2013 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Hyderabad, India.","DOI":"10.1109\/FUZZ-IEEE.2013.6622486"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1109\/TPAMI.2009.164","article-title":"Class conditional nearest neighbor for large margin instance selection","volume":"32","author":"Marchiori","year":"2010","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2030","DOI":"10.1016\/j.neucom.2009.11.031","article-title":"New method for instance or prototype selection using mutual information in time series prediction","volume":"73","author":"Guillen","year":"2010","journal-title":"Neurocomputing"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"926","DOI":"10.3390\/e15030926","article-title":"Mutual information-based inputs selection for electric load time series forecasting","volume":"15","year":"2013","journal-title":"Entropy"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1016\/j.neucom.2014.03.006","article-title":"A methodology for training set instance selection using mutual information in time series prediction","volume":"141","year":"2014","journal-title":"Neurocomputing"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Abdulali, A., Hassan, W., and Jeon, S. (2016). Stimuli-magnitude-adaptive sample selection for data-driven haptic modeling. Entropy, 18.","DOI":"10.3390\/e18060222"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"527","DOI":"10.1007\/s00500-003-0310-2","article-title":"Genetic algorithms for outlier detection and variable selection in linear regression models","volume":"8","author":"Tolvi","year":"2004","journal-title":"Soft Comput."},{"key":"ref_34","unstructured":"Wu, S., and Olafsson, S. (2006, January 20\u201323). Optimal instance selection for improved decision tree induction. Proceedings of the IIE Annual Conference on Institute of Industrial and Systems Engineers (IISE), Pittsburgh, PA, USA."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"381","DOI":"10.1007\/s10994-009-5161-3","article-title":"A cooperative coevolutionary algorithm for instance selection for instance-based learning","volume":"78","year":"2010","journal-title":"Mach. Learn."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"240","DOI":"10.1016\/j.knosys.2012.11.005","article-title":"Genetic algorithms in feature and instance selection","volume":"39","author":"Tsai","year":"2013","journal-title":"Knowl.-Based Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1383","DOI":"10.1109\/TSMCB.2012.2191953","article-title":"Integrating instance selection, instance weighting, and feature weighting for nearest neighbor classifiers by coevolutionary algorithms","volume":"42","author":"Derrac","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. Part B"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1007\/s10115-010-0375-z","article-title":"Cluster-based instance selection for machine classification","volume":"30","author":"Czarnowski","year":"2012","journal-title":"Knowl. Inf. Syst."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Czarnowski, I., and J\u0119drzejowicz, P. (2018). Cluster-Based Instance Selection for the Imbalanced Data Classification. International Conference on Computational Collective Intelligence, Springer.","DOI":"10.1007\/978-3-319-98446-9_18"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"863","DOI":"10.1109\/TEVC.2017.2688863","article-title":"An evolutionary multiobjective model and instance selection for support vector machines with pareto-based ensembles","volume":"21","author":"Gonzalez","year":"2017","journal-title":"IEEE Trans. Evolut. Comput."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1007\/s10044-015-0454-6","article-title":"MOPG: A multi-objective evolutionary algorithm for prototype generation","volume":"20","author":"Escalante","year":"2017","journal-title":"Pattern Anal. Appl."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"577","DOI":"10.1109\/TEVC.2013.2281535","article-title":"An evolutionary many-objective optimization algorithm using reference-point-based nondominated sorting approach, part I: Solving problems with box constraints","volume":"18","author":"Deb","year":"2014","journal-title":"IEEE Trans. Evolut. Comput."},{"key":"ref_43","unstructured":"Liu, H., and Motoda, H. (2013). Instance Selection and Construction for Data Mining, Springer US."},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Kordos, M., Rusiecki, A., and Blachnik, M. (2015, January 24\u201326). Noise reduction in regression tasks with distance, instance, attribute and density weighting. Proceedings of the 2015 IEEE 2nd International Conference on Cybernetics (CYBCONF), Gdynia, Poland.","DOI":"10.1109\/CYBConf.2015.7175909"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1145\/272991.272995","article-title":"Mersenne twister: A 623-dimensionally equidistributed uniform pseudo-random number generator","volume":"8","author":"Matsumoto","year":"1998","journal-title":"ACM Trans. Model. Comput. Simul. (TOMACS)"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"75","DOI":"10.1016\/0022-0000(86)90044-9","article-title":"Generating quasi-random sequences from semi-random sources","volume":"33","author":"Santha","year":"1986","journal-title":"J. Comput. Syst. Sci."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Pan, W., Li, K., Wang, M., Wang, J., and Jiang, B. (2014). Adaptive randomness: A new population initialization method. Math. Probl. Eng., 2014.","DOI":"10.1155\/2014\/975916"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Rahnamayan, S., and Wang, G.G. (2009, January 18\u201321). Center-based sampling for population-based algorithms. Proceedings of the 2009 IEEE Congress on Evolutionary Computation, Trondheim, Norway.","DOI":"10.1109\/CEC.2009.4983045"},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1016\/j.neucom.2013.07.021","article-title":"SW-ELM: A summation wavelet extreme learning machine algorithm with a priori parameter initialization","volume":"123","author":"Javed","year":"2014","journal-title":"Neurocomputing"},{"key":"ref_50","unstructured":"Kallel, L., and Schoenauer, M. (1997). Alternative Random Initialization in Genetic Algorithms, ICGA."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"\u0141apa, K., Cpa\u0142ka, K., and Hayashi, Y. (2017). Hybrid initialization in the process of evolutionary learning. International Conference on Artificial Intelligence and Soft Computing, Springer.","DOI":"10.1007\/978-3-319-59063-9_34"},{"key":"ref_52","unstructured":"Reunanen, J. (2012). Overfitting in Feature Selection: Pitfalls and Solutions, Doctoral Dissertaion, Aalto University."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Jabbar, H., and Khan, R. (2014). Methods to Avoid Over-fitting and Under-fitting in Supervised Machine Learning (Comparative Study). Comput. Sci. Commun. Instrum. Devices.","DOI":"10.3850\/978-981-09-5247-1_017"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1007\/s00500-015-1690-9","article-title":"Reducing Noise Impact on MLP Training","volume":"20","author":"Kordos","year":"2016","journal-title":"Soft Comput."},{"key":"ref_55","first-page":"255","article-title":"Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework","volume":"17","author":"Luengo","year":"2011","journal-title":"J. Mult.-Valued Logic Soft Comput."},{"key":"ref_56","unstructured":"Kordos, M., and Duch, W. (2004, January 1\u20133). Variable step search algorithm for MLP training. Proceedings of the 8th IASTED International Conference on Artificial Intelligence and Soft Computing, Marbella, Spain."},{"key":"ref_57","unstructured":"Riedmiller, M., and Braun, H. (April, January 28). A direct adaptive method for faster backpropagation learning: The RPROP algorithm. Proceedings of the IEEE International Conference on Neural Networks, San Francisco, CA, USA."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Refaeilzadeh, P. (2009). Cross-Validation. Encycl. Database Syst., 532\u2013538.","DOI":"10.1007\/978-0-387-39940-9_565"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"40","DOI":"10.1214\/09-SS054","article-title":"A survey of cross-validation procedures for model selection","volume":"4","author":"Arlot","year":"2010","journal-title":"Stat. Surv."},{"key":"ref_60","first-page":"1","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","year":"2006","journal-title":"J. Mach. Learn. Res."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"251","DOI":"10.1023\/A:1015516501242","article-title":"Evolutionary algorithms for multi-objective optimization: Performance assessments and comparisons","volume":"17","author":"Tan","year":"2002","journal-title":"Artif. Intell. Rev."},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1002\/(SICI)1099-1360(199801)7:1<34::AID-MCDA161>3.0.CO;2-6","article-title":"Pareto simulated annealing\u2014A metaheuristic technique for multiple-objective combinatorial optimization","volume":"7","author":"Jaszkiewicz","year":"1998","journal-title":"J. Multi-Criteria Dec. Anal."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1162\/106365600568202","article-title":"Comparison of multiobjective evolutionary algorithms: Empirical results","volume":"8","author":"Zitzler","year":"2000","journal-title":"Evolut. Comput."},{"key":"ref_64","doi-asserted-by":"crossref","unstructured":"Riquelme, N., Von L\u00fccken, C., and Baran, B. (2015, January 19\u201323). Performance metrics in multi-objective optimization. Proceedings of the 2015 Latin American Computing Conference (CLEI), Arequipa, Peru.","DOI":"10.1109\/CLEI.2015.7360024"},{"key":"ref_65","doi-asserted-by":"crossref","unstructured":"Yu, X., Lu, Y., and Yu, X. (2018). Evaluating Multiobjective Evolutionary Algorithms Using MCDM Methods. Math. Probl. Eng., 2018.","DOI":"10.1155\/2018\/9751783"},{"key":"ref_66","unstructured":"Rudzinski, F., Alonso, J., and Bustince, H. (July, January 30). Finding Sets of Non-Dominated Solutions with High Spread and Well-Balanced Distribution using Generalized Strength Pareto Evolutionary Algorithm. Proceedings of the Conference International Fuzzy Systems Association and the European Society for Fuzzy Logic and Technology (IFSA-EUSFLAT-15), Asturias, Spain."},{"key":"ref_67","doi-asserted-by":"crossref","unstructured":"Rudzi\u0144ski, F. (2017). An Application of Generalized Strength Pareto Evolutionary Algorithm for Finding a Set of Non-Dominated Solutions with High-Spread and Well-Balanced Distribution in the Logistics Facility Location Problem. International Conference on Artificial Intelligence and Soft Computing, Springer.","DOI":"10.1007\/978-3-319-59063-9_39"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/10\/746\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:23:14Z","timestamp":1760196194000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/20\/10\/746"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,9,29]]},"references-count":67,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2018,10]]}},"alternative-id":["e20100746"],"URL":"https:\/\/doi.org\/10.3390\/e20100746","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,9,29]]}}}