{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,17]],"date-time":"2025-10-17T14:08:21Z","timestamp":1760710101538,"version":"build-2065373602"},"reference-count":39,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2020,1,10]],"date-time":"2020-01-10T00:00:00Z","timestamp":1578614400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Feature selection is a way of reducing the features of data such that, when the classification algorithm runs, it produces better accuracy. In general, conventional feature selection is quite unstable when faced with changing data characteristics. It would be inefficient to implement individual feature selection in some cases. Ensemble feature selection exists to overcome this problem. However, with the advantages of ensemble feature selection, some issues like stability, threshold, and feature aggregation still need to be overcome. We propose a new framework to deal with stability and feature aggregation. We also used an automatic threshold to see whether it was efficient or not; the results showed that the proposed method always produces the best performance in both accuracy and feature reduction. The accuracy comparison between the proposed method and other methods was 0.5\u201314% and reduced more features than other methods by 50%. The stability of the proposed method was also excellent, with an average of 0.9. However, when we applied the automatic threshold, there was no beneficial improvement compared to without an automatic threshold. Overall, the proposed method presented excellent performance compared to previous work and standard ReliefF.<\/jats:p>","DOI":"10.3390\/info11010038","type":"journal-article","created":{"date-parts":[[2020,1,10]],"date-time":"2020-01-10T10:20:29Z","timestamp":1578651629000},"page":"38","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Optimal Feature Aggregation and Combination for Two-Dimensional Ensemble Feature Selection"],"prefix":"10.3390","volume":"11","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7900-2484","authenticated-orcid":false,"given":"Machmud Roby","family":"Alhamidi","sequence":"first","affiliation":[{"name":"Faculty of Computer Science, Universitas Indonesia, Jawa Barat 16424, Indonesia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wisnu","family":"Jatmiko","sequence":"additional","affiliation":[{"name":"Faculty of Computer Science, Universitas Indonesia, Jawa Barat 16424, Indonesia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,1,10]]},"reference":[{"key":"ref_1","first-page":"1157","article-title":"An Introduction to Variable and Feature Selection","volume":"3","author":"Guyon","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"8215","DOI":"10.17148\/IJARCCE.2014.31031","article-title":"Feature Selection using ReliefF Algorithm","volume":"3","author":"Durgabai","year":"2014","journal-title":"Int. J. Adv. Res. Comput. Commun. Eng."},{"key":"ref_3","unstructured":"Kira, K., and Rendell, L.A. (1992, January 12\u201316). Feature selection problem: Traditional methods and a new algorithm. Proceedings of the Tenth National Conference on Artificial Intelligence, San Jose, CA, USA."},{"key":"ref_4","unstructured":"Bergadano, F., and De Raedt, L. (1994). Estimating attributes: Analysis and extensions of RELIEF. Proceedings of the Machine Learning: ECML-94, Springer."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1023\/A:1008280620621","article-title":"Overcoming the Myopia of Inductive Learning Algorithms with RELIEFF","volume":"7","author":"Kononenko","year":"1997","journal-title":"Appl. Intell."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"189","DOI":"10.1016\/j.jbi.2018.07.014","article-title":"Relief-based feature selection: Introduction and review","volume":"85","author":"Urbanowicz","year":"2018","journal-title":"J. Biomed. Inform."},{"key":"ref_7","unstructured":"Hall, M. (1999). Correlation-Based Feature Selection for Machine Learning. [Master\u2019s Thesis, University of Waikato Hamilton]."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"542","DOI":"10.1016\/j.jesit.2017.06.004","article-title":"Correlation based feature selection with clustering for high dimensional data","volume":"5","author":"Chormunge","year":"2018","journal-title":"J. Electr. Syst. Inf. Technol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1016\/S0004-3702(97)00043-X","article-title":"Wrappers for feature subset selection","volume":"97","author":"Kohavi","year":"1997","journal-title":"Artif. Intell."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1016\/j.eswa.2011.07.048","article-title":"Feature subset selection wrapper based on mutual information and rough sets","volume":"39","author":"Foithong","year":"2012","journal-title":"Expert Syst. Appl."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"144","DOI":"10.1016\/j.jbi.2017.11.005","article-title":"A novel bagging C4.5 algorithm based on wrapper feature selection for supporting wise clinical decision making","volume":"78","author":"Lee","year":"2018","journal-title":"J. Biomed. Inform."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"81","DOI":"10.1016\/j.knosys.2015.03.009","article-title":"Accelerating wrapper-based feature selection with K-nearest-neighbor","volume":"83","author":"Wang","year":"2015","journal-title":"Knowl.-Based Syst."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1016\/j.chemolab.2019.06.004","article-title":"Feature selection based convolutional neural network pruning and its application in calibration modeling for NIR spectroscopy","volume":"191","author":"Chen","year":"2019","journal-title":"Chemom. Intell. Lab. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"669","DOI":"10.1016\/j.jfranklin.2014.04.021","article-title":"Embedded feature-selection support vector machine for driving pattern recognition","volume":"352","author":"Zhang","year":"2015","journal-title":"J. Frankl. Inst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1818","DOI":"10.1016\/j.proeng.2012.08.109","article-title":"Feature Selection in Ischemic Heart Disease identification using feed forward neural networks","volume":"41","author":"Rajeswari","year":"2012","journal-title":"Procedia Eng."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1016\/j.patcog.2016.05.012","article-title":"Feature selection using Forest Optimization Algorithm","volume":"60","author":"Ghaemi","year":"2016","journal-title":"Pattern Recognit."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"116","DOI":"10.1016\/j.knosys.2017.02.013","article-title":"Ensemble feature selection using bi-objective genetic algorithm","volume":"123","author":"Das","year":"2017","journal-title":"Knowl.-Based Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"568","DOI":"10.1016\/j.procs.2017.12.073","article-title":"Web-Spam Features Selection Using CFS-PSO","volume":"125","author":"Singh","year":"2018","journal-title":"Procedia Comput. Sci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"612","DOI":"10.1016\/j.eswa.2014.08.014","article-title":"Gene selection from microarray gene expression data for classification of cancer subgroups employing PSO and adaptive K-nearest neighborhood technique","volume":"42","author":"Kar","year":"2015","journal-title":"Expert Syst. Appl."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1016\/j.ygeno.2016.05.001","article-title":"A hybrid gene selection approach for microarray data classification using cellular learning automata and ant colony optimization","volume":"107","author":"Mosafer","year":"2016","journal-title":"Genomics"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"300","DOI":"10.1016\/j.asoc.2016.11.021","article-title":"Ensemble of feature selection methods: A hesitant fuzzy sets approach","volume":"50","author":"Ebrahimpour","year":"2017","journal-title":"Appl. Soft Comput. J."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.compind.2018.01.014","article-title":"A novel feature selection method using fuzzy rough sets","volume":"97","author":"Sheeja","year":"2018","journal-title":"Comput. Ind."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.fss.2018.10.021","article-title":"Incremental feature weighting for fuzzy feature selection","volume":"368","author":"Wang","year":"2019","journal-title":"Fuzzy Sets Syst."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Chen, J., Mi, J., and Lin, Y. (2019). A graph approach for fuzzy-rough feature selection. Fuzzy Sets Syst., 1.","DOI":"10.1016\/j.fss.2019.07.014"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Liu, Z., Zhao, X., Li, L., Wang, X., and Wang, D. (2019). A novel multi-attribute decision making method based on the double hierarchy hesitant fuzzy linguistic generalized power aggregation operator. Information, 10.","DOI":"10.3390\/info10110339"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1471","DOI":"10.1109\/LGRS.2015.2409112","article-title":"Improving Random Forest With Ensemble of Features and Semisupervised Feature Extraction","volume":"12","author":"Xia","year":"2015","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"124","DOI":"10.1016\/j.knosys.2016.11.017","article-title":"Ensemble feature selection: Homogeneous and heterogeneous approaches","volume":"118","year":"2017","journal-title":"Knowl.-Based Syst."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.inffus.2018.11.008","article-title":"Ensembles for feature selection: A review and future trends","volume":"52","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.inffus.2018.02.007","article-title":"On developing an automatic threshold applied to feature selection ensembles","volume":"45","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"365","DOI":"10.1016\/j.ins.2018.12.033","article-title":"Ensemble feature selection using election methods and ranker clustering","volume":"480","author":"Gazda","year":"2019","journal-title":"Inf. Sci."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.ins.2019.01.064","article-title":"A new hybrid ensemble feature selection framework for machine learning-based phishing detection system","volume":"484","author":"Chiew","year":"2019","journal-title":"Inf. Sci."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Alhamidi, M.R., Arsa, D.M.S., Rachmadi, M.F., and Jatmiko, W. (2018, January 27\u201328). 2-Dimensional homogeneous distributed ensemble feature selection. Proceedings of the 2018 International Conference on Advanced Computer Science and Information Systems, ICACSIS 2018, Yogyakarta, Indonesia.","DOI":"10.1109\/ICACSIS.2018.8618266"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Dowlatshahi, M.B., Derhami, V., and Nezamabadi-Pour, H. (2017). Ensemble of filter-based rankers to guide an epsilon-greedy swarm optimizer for high-dimensional feature subset selection. Information, 8.","DOI":"10.3390\/info8040152"},{"key":"ref_34","unstructured":"Guyon, I. (2018, July 17). NIPS 2003 Workshop on Feature Extraction and Feature Selection Challenge. Available online: http:\/\/clopinet.com\/isabelle\/Projects\/NIPS2003\/#links."},{"key":"ref_35","unstructured":"(2018, April 02). Feature Selection Dataset. Available online: http:\/\/featureselection.asu.edu\/datasets.php."},{"key":"ref_36","unstructured":"Dheeru, D., and Karra Taniskidou, E. (2018, April 02). Machine Learning Repository. Available online: http:\/\/archive.ics.uci.edu\/ml."},{"key":"ref_37","unstructured":"(2018, May 10). Gene Expression Model Selector. Available online: http:\/\/gems-system.org."},{"key":"ref_38","first-page":"98","article-title":"A Survey on Feature Selection Stability Measures","volume":"5","author":"Perumal","year":"2016","journal-title":"Int. J. Comput. Inf. Technol."},{"key":"ref_39","unstructured":"Khaire, U.M., and Dhanalakshmi, R. (2019). Stability of feature selection algorithm: A review. J. King Saud Univ. Comput. Inf. Sci., in press."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/1\/38\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:19:27Z","timestamp":1760361567000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/11\/1\/38"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,1,10]]},"references-count":39,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2020,1]]}},"alternative-id":["info11010038"],"URL":"https:\/\/doi.org\/10.3390\/info11010038","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2020,1,10]]}}}