{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T18:07:04Z","timestamp":1772906824227,"version":"3.50.1"},"reference-count":28,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2021,3,22]],"date-time":"2021-03-22T00:00:00Z","timestamp":1616371200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Algorithms"],"abstract":"<jats:p>This study presents a novel performance metric for feature selection algorithms that is unbiased and can be used for comparative analysis across feature selection problems. The baseline fitness improvement (BFI) measure quantifies the potential value gained by applying feature selection. The BFI measure can be used to compare the performance of feature selection algorithms across datasets by measuring the change in classifier performance as a result of feature selection, with respect to the baseline where all features are included. Empirical results are presented to show that there is performance complementarity for a suite of feature selection algorithms on a variety of real world datasets. The BFI measure is a normalised performance metric that can be used to correlate problem characteristics with feature selection algorithm performance, across multiple datasets. This ability paves the way towards describing the performance space of the per-instance algorithm selection problem for feature selection algorithms.<\/jats:p>","DOI":"10.3390\/a14030100","type":"journal-article","created":{"date-parts":[[2021,3,22]],"date-time":"2021-03-22T12:55:07Z","timestamp":1616417707000},"page":"100","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["A Feature Selection Algorithm Performance Metric for Comparative Analysis"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3995-9222","authenticated-orcid":false,"given":"Werner","family":"Mostert","sequence":"first","affiliation":[{"name":"Department of Industrial Engineering, Division of Computer Science, Stellenbosch University, Stellenbosch 7600, South Africa"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6070-2632","authenticated-orcid":false,"given":"Katherine M.","family":"Malan","sequence":"additional","affiliation":[{"name":"Department of Decision Sciences, University of South Africa, Pretoria 7701, South Africa"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0242-3539","authenticated-orcid":false,"given":"Andries P.","family":"Engelbrecht","sequence":"additional","affiliation":[{"name":"Department of Industrial Engineering, Division of Computer Science, Stellenbosch University, Stellenbosch 7600, South Africa"}]}],"member":"1968","published-online":{"date-parts":[[2021,3,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Kotthoff, L. (2016). Algorithm selection for combinatorial search problems: A survey. Data Mining and Constraint Programming, Springer.","DOI":"10.1007\/978-3-319-50137-6_7"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1162\/evco_a_00242","article-title":"Automated algorithm selection: Survey and perspectives","volume":"27","author":"Kerschke","year":"2019","journal-title":"Evol. Comput."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.compeleceng.2013.11.024","article-title":"A survey on feature selection methods","volume":"40","author":"Chandrashekar","year":"2014","journal-title":"Comput. Electr. Eng."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zongker, D., and Jain, A. (1996, January 25\u201329). Algorithms for feature selection: An evaluation. Proceedings of the 13th International Conference on Pattern Recognition, Vienna, Austria.","DOI":"10.1109\/ICPR.1996.546716"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1016\/S0004-3702(97)00043-X","article-title":"Wrappers for feature subset selection","volume":"97","author":"Kohavi","year":"1997","journal-title":"Artif. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/S0065-2458(08)60520-3","article-title":"The algorithm selection problem","volume":"Volume 15","author":"Rice","year":"1976","journal-title":"Advances in Computers"},{"key":"ref_7","first-page":"1157","article-title":"An introduction to variable and feature selection","volume":"3","author":"Guyon","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_8","unstructured":"Zhao, Z., Morstatter, F., Sharma, S., Alelyani, S., Anand, A., and Liu, H. (2010). Advancing feature selection research. ASU Feature Selection Repository, Arizona State University."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"199","DOI":"10.1007\/978-1-4612-2404-4_19","article-title":"A comparative evaluation of sequential feature selection algorithms","volume":"Volume 112","author":"Aha","year":"1996","journal-title":"Learning from Data"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"2429","DOI":"10.1093\/bioinformatics\/bth267","article-title":"A comparative study of feature selection and multiclass classification methods for tissue classification based on gene expression","volume":"20","author":"Li","year":"2004","journal-title":"Bioinformatics"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1016\/j.ejor.2015.09.051","article-title":"Integer programming models for feature selection: New extensions and a randomized solution algorithm","volume":"250","author":"Bertolazzi","year":"2016","journal-title":"Eur. J. Oper. Res."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Mehri, M., Chaieb, R., Kalti, K., H\u00e9roux, P., Mullot, R., and Essoukri Ben Amara, N. (2018). A comparative study of two state-of-the-art feature selection algorithms for texture-based pixel-labeling task of ancient documents. J. Imaging, 4.","DOI":"10.3390\/jimaging4080097"},{"key":"ref_13","first-page":"147","article-title":"Insights into the feature selection problem using local optima networks","volume":"Volume 11452","author":"Mostert","year":"2019","journal-title":"Lecture Notes in Computer Science, Proceedings of the European Conference on Evolutionary Computation in Combinatorial Optimization, Leipzig, Germany, 24\u201326 April 2019"},{"key":"ref_14","first-page":"1","article-title":"Cross-disciplinary perspectives on meta-learning for algorithm selection","volume":"41","year":"2009","journal-title":"ACM Comput. Surv."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"745","DOI":"10.1613\/jair.4726","article-title":"Autofolio: An automatically configured algorithm selector","volume":"53","author":"Lindauer","year":"2015","journal-title":"J. Artif. Intell. Res."},{"key":"ref_16","unstructured":"Sakamoto, Y., Ishiguro, M., and Kitagawa, G. (1986). Akaike Information Criterion Statistics, D. Reidel."},{"key":"ref_17","first-page":"37","article-title":"Evaluation: From precision, recall and F-measure to ROC, informedness, markedness and correlation","volume":"2","author":"Powers","year":"2011","journal-title":"J. Mach. Learn. Technol."},{"key":"ref_18","unstructured":"Van Asch, V. (2013). Macro-and Micro-Averaged Evaluation Measures [Basic Draft], CLiPS."},{"key":"ref_19","first-page":"91","article-title":"Combining multiple classifiers for wrapper feature selection","volume":"1","author":"Chrysostomou","year":"2008","journal-title":"Int. J. Data Mining Model. Manag."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bajer, D., Dudjak, M., and Zori\u0107, B. (2020, January 14\u201316). Wrapper-based feature selection: How important is the wrapped classifier?. Proceedings of the 2020 International Conference on Smart Systems and Technologies (SST), Osijek, Croatia.","DOI":"10.1109\/SST49455.2020.9264072"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1007\/BF00153759","article-title":"Instance-based learning algorithms","volume":"6","author":"Aha","year":"1991","journal-title":"Mach. Learn."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1145\/1656274.1656278","article-title":"The WEKA Data Mining Software: An Update","volume":"11","author":"Hall","year":"2009","journal-title":"SIGKDD Explor. Newsl."},{"key":"ref_23","unstructured":"Lichman, M. (2013). UCI Machine Learning Repository, UCI."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Tran, B., Xue, B., and Zhang, M. (2019, January 13\u201319). Adaptive multi-subswarm optimisation for feature selection on high-dimensional classification. Proceedings of the Genetic and Evolutionary Computation Conference, ACM, Prague, Czech Republic.","DOI":"10.1145\/3321707.3321713"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1109\/TCYB.2014.2322602","article-title":"A competitive swarm optimizer for large scale optimization","volume":"45","author":"Cheng","year":"2014","journal-title":"IEEE Trans. Cybern."},{"key":"ref_26","unstructured":"Goldberg, D.E. (1989). Genetic Algorithms in Search, Optimization and Machine Learning, Addison-Wesley."},{"key":"ref_27","unstructured":"Cover, T.M., and Thomas, J.A. (2012). Elements of Information Theory, John Wiley & Sons."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"409","DOI":"10.1016\/j.patcog.2008.08.001","article-title":"Performance of feature-selection methods in the classification of high-dimension data","volume":"42","author":"Hua","year":"2009","journal-title":"Pattern Recognit."}],"container-title":["Algorithms"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/3\/100\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:39:24Z","timestamp":1760161164000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-4893\/14\/3\/100"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,22]]},"references-count":28,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,3]]}},"alternative-id":["a14030100"],"URL":"https:\/\/doi.org\/10.3390\/a14030100","relation":{},"ISSN":["1999-4893"],"issn-type":[{"value":"1999-4893","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,22]]}}}