{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,30]],"date-time":"2026-04-30T07:08:38Z","timestamp":1777532918581,"version":"3.51.4"},"reference-count":43,"publisher":"Oxford University Press (OUP)","issue":"7","license":[{"start":{"date-parts":[[2022,4,9]],"date-time":"2022-04-09T00:00:00Z","timestamp":1649462400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/academic.oup.com\/journals\/pages\/open_access\/funder_policies\/chorus\/standard_publication_model"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,7,13]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Feature selection refers to a critical preprocessing of machine learning to remove irrelevant and redundant data. According to feature selection methods, sufficient samples are usually required to select a reliable feature subset, especially considering the presence of outliers. However, sufficient samples cannot always be ensured in several real-world applications (e.g. neuroscience, bioinformatics and psychology). This study proposed a method to improve the performance of feature selection methods with ultra low-sample-size data, which is named feature selection based on data quality and variable training samples (QVT). Given that none of feature selection methods can perform optimally in all scenarios, QVT is primarily characterized by its versatility, because it can be implemented in any feature selection method. Furthermore, compared to the existing methods which tried to extract a stable feature subset for low-sample-size data by increasing the sample size or using more complicated algorithm, QVT tried to get improvement using the original data. An experiment was performed using 20 benchmark datasets, three feature selection methods and three classifiers to verify the feasibility of QVT; the results showed that using features selected by QVT is capable of achieving higher classification accuracy than using the explicit feature selection method, and significant differences exist.<\/jats:p>","DOI":"10.1093\/comjnl\/bxac033","type":"journal-article","created":{"date-parts":[[2022,4,9]],"date-time":"2022-04-09T10:16:51Z","timestamp":1649499411000},"page":"1664-1686","source":"Crossref","is-referenced-by-count":8,"title":["Improving the Performance of Feature Selection Methods with Low-Sample-Size Data"],"prefix":"10.1093","volume":"66","author":[{"given":"Wanwan","family":"Zheng","sequence":"first","affiliation":[{"name":"Institute of Industrial Internet and Internet of Things , China Academy of Information and Communications Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Mingzhe","family":"Jin","sequence":"additional","affiliation":[{"name":"Graduate School of Culture and Information Science , Doshisha University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"286","published-online":{"date-parts":[[2022,4,9]]},"reference":[{"key":"2023071709114517700_ref1","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1023\/A:1009715923555","article-title":"A tutorial on support vector machines for pattern recognition","volume":"2","author":"Burges","year":"1998","journal-title":"Data Min. Knowl. Discovery"},{"key":"2023071709114517700_ref2","volume-title":"Wiley-Interscience","author":"Cover","year":"2006","edition":"2nd"},{"key":"2023071709114517700_ref3","first-page":"1","article-title":"Evaluation of variable selection methods for random forests and omics data sets","volume":"20","author":"Degenhardt","year":"2017","journal-title":"Brief. Bioinform."},{"key":"2023071709114517700_ref4","first-page":"1","article-title":"Feature selection using LASSO","author":"Fonti","year":"2017","journal-title":"VRIJE University Amsterdam"},{"key":"2023071709114517700_ref5","first-page":"101","article-title":"A selective overview of variable selection in high dimensional feature space","volume":"20","author":"Fan","year":"2010","journal-title":"Stat. Sinica"},{"key":"2023071709114517700_ref6","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/j.eswa.2015.12.004","article-title":"Hybrid feature selection based on enhanced genetic algorithm for text categorization","volume":"49","author":"Ghareb","year":"2016","journal-title":"Expert Syst. Appl."},{"key":"2023071709114517700_ref7","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/j.eswa.2018.06.057","article-title":"Recursive memetic algorithm for gene selection in microarray data","volume":"116","author":"Ghosh","year":"2019","journal-title":"Expert Syst. Appl."},{"key":"2023071709114517700_ref8","first-page":"1157","article-title":"An introduction to variable and feature selection","volume":"3","author":"Guyon","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"2023071709114517700_ref9","volume-title":"Proceedings of the 33rd Annual International Conference of the IEEE EMBS","author":"Golugula","year":"2011"},{"key":"2023071709114517700_ref10","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1016\/j.asoc.2017.09.038","article-title":"Correlation feature selection based Improved-Binary Particle Swarm Optimization for gene selection and cancer classification","volume":"62","author":"Jain","year":"2018","journal-title":"Appl. Soft Comput."},{"key":"2023071709114517700_ref11","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1016\/j.engappai.2016.02.002","article-title":"Deep feature weighting for naive bayes and its application to text classification","volume":"52","author":"Jiang","year":"2016","journal-title":"Eng. Appl. Artif. Intel."},{"key":"2023071709114517700_ref12","first-page":"255","article-title":"Using random forest to identify a text\u2019s author","volume":"55","author":"Jin","year":"2007","journal-title":"Stat. Math."},{"key":"2023071709114517700_ref13","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1016\/j.neucom.2018.07.002","article-title":"Towards perfect text classification with Wikipedia-based semantic Na\u00efve Bayes learning","volume":"315","author":"Kim","year":"2018","journal-title":"Neurocomputing"},{"key":"2023071709114517700_ref14","doi-asserted-by":"crossref","first-page":"660","DOI":"10.1016\/j.patcog.2018.03.012","article-title":"On feature selection protocols for very low-sample-size data","volume":"81","author":"Kuncheva","year":"2018","journal-title":"Pattern Recognit."},{"key":"2023071709114517700_ref15","volume-title":"Information Theory and Statistics. Dover Publishers","author":"Kullback","year":"1997","edition":"2nd"},{"key":"2023071709114517700_ref16","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1214\/aoms\/1177729694","article-title":"On information and sufficiency","volume":"22","author":"Kullback","year":"1951","journal-title":"Ann. Math. Stat."},{"key":"2023071709114517700_ref17","doi-asserted-by":"crossref","first-page":"6611","DOI":"10.1016\/j.eswa.2014.04.033","article-title":"Study of image retrieval and classification based on adaptive features using genetic algorithm feature selection","volume":"41","author":"Lin","year":"2014","journal-title":"Expert Syst. Appl."},{"key":"2023071709114517700_ref18","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1007\/978-3-642-38067-9_15","volume-title":"Proceedings of the 11th International Workshop on Multiple Classifier Systems","author":"Li","year":"2013"},{"key":"2023071709114517700_ref19","doi-asserted-by":"crossref","first-page":"551","DOI":"10.1007\/s10115-017-1059-8","article-title":"Recent advances in feature selection and its application","volume":"53","author":"Li","year":"2017","journal-title":"Knowl. Inform. Syst."},{"key":"2023071709114517700_ref20","doi-asserted-by":"crossref","first-page":"1388","DOI":"10.1109\/TNNLS.2014.2341627","article-title":"Frel: a stable feature selection algorithm","volume":"26","author":"Li","year":"2015","journal-title":"IEEE Trans. Neural Network Learn. Syst."},{"key":"2023071709114517700_ref21","first-page":"2287","volume-title":"Proceedings of the 26th International Joint Conference on Artificial Intelligence","author":"Liu","year":"2017"},{"key":"2023071709114517700_ref22","volume-title":"Encyclopedia of Machine Learning","author":"Mladeni\u0107","year":"2011"},{"key":"2023071709114517700_ref23","doi-asserted-by":"crossref","first-page":"362","DOI":"10.7305\/automatika.53-4.281","article-title":"Breast density classification using multiple feature selection","volume":"53","author":"Mu\u0161tra","year":"2012","journal-title":"Automatika"},{"key":"2023071709114517700_ref24","volume-title":"PhD thesis of the Universite Libre de Bruxelles","author":"Meyer","year":"2009"},{"key":"2023071709114517700_ref25","doi-asserted-by":"crossref","first-page":"224","DOI":"10.1016\/j.eswa.2017.08.026","article-title":"Metaheuristic approach for an enhanced mRMR filter method for classification using drug response microarray data","volume":"90","author":"Mohamed","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"2023071709114517700_ref26","first-page":"1","article-title":"Gene regulatory network inference using fused LASSO on multiple data sets","volume":"6","author":"Omranian","year":"2015","journal-title":"Sci. Rep."},{"key":"2023071709114517700_ref27","doi-asserted-by":"crossref","first-page":"69","DOI":"10.1016\/j.eswa.2016.11.024","article-title":"Attribute clustering using rough set theory for feature selection in fault severity classification of rotating machinery","volume":"71","author":"Pacheco","year":"2017","journal-title":"Expert Syst. Appl."},{"key":"2023071709114517700_ref28","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1109\/TSMC.1973.4309285","article-title":"Fundamentals of pattern recognition","volume":"5","author":"Patrick","year":"1973","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"2023071709114517700_ref29","volume-title":"IEEE Trans. Pattern Anal. Mach. Intell.","author":"Raudys","year":"1991"},{"key":"2023071709114517700_ref30","volume-title":"Advances in Learning Theory: Methods, Models and Applications","author":"Suykens","year":"2003"},{"key":"2023071709114517700_ref31","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1023\/A:1018628609742","article-title":"Least squares support vector machine classifiers","volume":"9","author":"Suykens","year":"1999","journal-title":"Neural Process. Lett."},{"key":"2023071709114517700_ref32","doi-asserted-by":"crossref","first-page":"68","DOI":"10.1016\/j.neuroimage.2017.06.061","article-title":"Cross-validation failure: small sample sizes lead to error bars","volume":"180","author":"Varoquaux","year":"2018","journal-title":"Neuroimage"},{"key":"2023071709114517700_ref33","doi-asserted-by":"crossref","DOI":"10.1007\/978-1-4757-3264-1","volume-title":"The Nature of Statistical Learning Theory","author":"Vapnik","year":"2000"},{"key":"2023071709114517700_ref34","volume-title":"Statistical Learning Theory","author":"Vapnik","year":"1998","edition":"1st"},{"key":"2023071709114517700_ref35","first-page":"28","article-title":"The generalization of Student\u2019s problem when several different population variances are involved","volume":"34","author":"Welch","year":"1947","journal-title":"Biometrika"},{"key":"2023071709114517700_ref36","doi-asserted-by":"crossref","first-page":"522","DOI":"10.1145\/2623330.2623635","volume-title":"Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Xu","year":"2014"},{"key":"2023071709114517700_ref37","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1177\/0165551516677946","article-title":"Bayesian na\u00efve Bayes classifiers to text classification","volume":"44","author":"Xu","year":"2018","journal-title":"Journal of Information Science"},{"key":"2023071709114517700_ref38","first-page":"412","volume-title":"Proceedings of the 14th International Conference on Machine Learning","author":"Yang","year":"1997"},{"key":"2023071709114517700_ref39","first-page":"856","volume-title":"Proceedings of the 21th International Conference on Machine Learning","author":"Yu","year":"2004"},{"key":"2023071709114517700_ref40","doi-asserted-by":"crossref","first-page":"1352","DOI":"10.1109\/TKDE.2018.2789451","article-title":"Ultra high-dimensional nonlinear feature selection for big biological data","volume":"30","author":"Yamada","year":"2018","journal-title":"IEEE Transl. Knowl. Eng."},{"key":"2023071709114517700_ref41","first-page":"562","volume-title":"Proceedings of the 17th International Florida Artificial Intelligence Research Society Conference","author":"Zhang","year":"2004"},{"key":"2023071709114517700_ref42","doi-asserted-by":"crossref","first-page":"215","DOI":"10.1016\/j.patcog.2012.07.018","article-title":"Self-taught dimensionality reduction on the high-dimensional small-sized data","volume":"46","author":"Zhu","year":"2013","journal-title":"Pattern Recogn."},{"key":"2023071709114517700_ref43","doi-asserted-by":"crossref","first-page":"125","DOI":"10.1016\/j.eswa.2018.05.032","article-title":"A new subset based deep feature learning method for intelligent fault diagnosis of bearing","volume":"110","author":"Zhang","year":"2018","journal-title":"Expert Syst. Appl."}],"container-title":["The Computer Journal"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/66\/7\/1664\/50876250\/bxac033.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/comjnl\/article-pdf\/66\/7\/1664\/50876250\/bxac033.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,7,17]],"date-time":"2023-07-17T09:18:13Z","timestamp":1689585493000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/comjnl\/article\/66\/7\/1664\/6565626"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,4,9]]},"references-count":43,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,4,9]]},"published-print":{"date-parts":[[2023,7,13]]}},"URL":"https:\/\/doi.org\/10.1093\/comjnl\/bxac033","relation":{},"ISSN":["0010-4620","1460-2067"],"issn-type":[{"value":"0010-4620","type":"print"},{"value":"1460-2067","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023,7]]},"published":{"date-parts":[[2022,4,9]]}}}