{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,28]],"date-time":"2026-02-28T04:30:50Z","timestamp":1772253050964,"version":"3.50.1"},"reference-count":60,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2023,5,3]],"date-time":"2023-05-03T00:00:00Z","timestamp":1683072000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Federal Ministry of Education and Research of Germany (BMBF) within the iCampus Cottbus project","award":["16ES1128K"],"award-info":[{"award-number":["16ES1128K"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Feature selection (FS) represents an essential step for many machine learning-based predictive maintenance (PdM) applications, including various industrial processes, components, and monitoring tasks. The selected features not only serve as inputs to the learning models but also can influence further decisions and analysis, e.g., sensor selection and understandability of the PdM system. Hence, before deploying the PdM system, it is crucial to examine the reproducibility and robustness of the selected features under variations in the input data. This is particularly critical for real-world datasets with a low sample-to-dimension ratio (SDR). However, to the best of our knowledge, stability of the FS methods under data variations has not been considered yet in the field of PdM. This paper addresses this issue with an application to tool condition monitoring in milling, where classifiers based on support vector machines and random forest were employed. We used a five-fold cross-validation to evaluate three popular filter-based FS methods, namely Fisher score, minimum redundancy maximum relevance (mRMR), and ReliefF, in terms of both stability and macro-F1. Further, for each method, we investigated the impact of the homogeneous FS ensemble on both performance indicators. To gain broad insights, we used four (2:2) milling datasets obtained from our experiments and NASA\u2019s repository, which differ in the operating conditions, sensors, SDR, number of classes, etc. For each dataset, the study was conducted for two individual sensors and their fusion. Among the conclusions: (1) Different FS methods can yield comparable macro-F1 yet considerably different FS stability values. (2) Fisher score (single and\/or ensemble) is superior in most of the cases. (3) mRMR\u2019s stability is overall the lowest, the most variable over different settings (e.g., sensor(s), subset cardinality), and the one that benefits the most from the ensemble.<\/jats:p>","DOI":"10.3390\/s23094461","type":"journal-article","created":{"date-parts":[[2023,5,4]],"date-time":"2023-05-04T02:03:18Z","timestamp":1683165798000},"page":"4461","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["On the Stability and Homogeneous Ensemble of Feature Selection for Predictive Maintenance: A Classification Application for Tool Condition Monitoring in Milling"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3924-3355","authenticated-orcid":false,"given":"Maryam","family":"Assafo","sequence":"first","affiliation":[{"name":"Department of Wireless Systems, Brandenburg University of Technology Cottbus\u2013Senftenberg, 03046 Cottbus, Germany"}]},{"given":"Jost Philipp","family":"St\u00e4dter","sequence":"additional","affiliation":[{"name":"Department of Automation Technology, Brandenburg University of Technology Cottbus\u2013Senftenberg, 03046 Cottbus, Germany"}]},{"given":"Tenia","family":"Meisel","sequence":"additional","affiliation":[{"name":"Fraunhofer IPMS, 01109 Dresden, Germany"}]},{"given":"Peter","family":"Langend\u00f6rfer","sequence":"additional","affiliation":[{"name":"Department of Wireless Systems, Brandenburg University of Technology Cottbus\u2013Senftenberg, 03046 Cottbus, Germany"},{"name":"IHP\u2014Leibniz-Institut f\u00fcr innovative Mikroelektronik, 15236 Frankfurt, Germany"}]}],"member":"1968","published-online":{"date-parts":[[2023,5,3]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1016\/j.procir.2018.12.019","article-title":"Predictive maintenance of machine tool systems using artificial intelligence techniques applied to machine condition data","volume":"80","author":"Lee","year":"2019","journal-title":"Procedia Cirp"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1016\/j.jmsy.2020.07.008","article-title":"Towards multi-model approaches to predictive maintenance: A systematic literature survey on diagnostics and prognostics","volume":"56","author":"Jimenez","year":"2020","journal-title":"J. Manuf. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2213","DOI":"10.1109\/JSYST.2019.2905565","article-title":"Data-driven methods for predictive maintenance of industrial equipment: A survey","volume":"13","author":"Zhang","year":"2019","journal-title":"IEEE Syst. J."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"90011","DOI":"10.1109\/ACCESS.2021.3091476","article-title":"A TOPSIS-Assisted Feature Selection Scheme and SOM-Based Anomaly Detection for Milling Tools Under Different Operating Conditions","volume":"9","author":"Assafo","year":"2021","journal-title":"IEEE Access"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"3299","DOI":"10.1109\/TIE.2016.2527623","article-title":"A Hybrid Feature Selection Scheme for Reducing Diagnostic Performance Deterioration Caused by Outliers in Data-Driven Diagnostics","volume":"63","author":"Kang","year":"2016","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1109\/TIE.2014.2327589","article-title":"Heterogeneous Feature Models and Feature Selection Applied to Bearing Fault Diagnosis","volume":"62","author":"Rauber","year":"2015","journal-title":"IEEE Trans. Ind. Electron."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1007\/s10845-020-01564-3","article-title":"Identification of cutting tool wear condition in turning using self-organizing map trained with imbalanced data","volume":"32","author":"Brito","year":"2021","journal-title":"J. Intell. Manuf."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"4041","DOI":"10.1016\/j.asoc.2011.03.026","article-title":"A hybrid feature selection scheme and self-organizing map model for machine health assessment","volume":"11","author":"Yu","year":"2011","journal-title":"Appl. Soft Comput."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Casusol, A.J., Zegarra, F.C., Vargas-Machuca, J., and Coronado, A.M. (2021, January 5\u20137). Optimal window size for the extraction of features for tool wear estimation. Proceedings of the 2021 IEEE XXVIII International Conference on Electronics, Electrical Engineering and Computing (INTERCON), Lima, Peru.","DOI":"10.1109\/INTERCON52678.2021.9532759"},{"key":"ref_10","first-page":"1157","article-title":"An introduction to variable and feature selection","volume":"3","author":"Guyon","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_11","unstructured":"Tang, J., Alelyani, S., and Liu, H. (2014). Data Classification: Algorithms and Applications, CRC Press."},{"key":"ref_12","first-page":"313","article-title":"Robust Feature Selection Using Ensemble Feature Selection Techniques","volume":"Volume 5212","author":"Daelemans","year":"2008","journal-title":"Machine Learning and Knowledge Discovery in Databases. ECML PKDD 2008. Lecture Notes in Computer Science"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2507","DOI":"10.1093\/bioinformatics\/btm344","article-title":"A review of feature selection techniques in bioinformatics","volume":"23","author":"Saeys","year":"2007","journal-title":"Bioinformatics"},{"key":"ref_14","unstructured":"Kalousis, A., Prados, J., and Hilario, M. (2005, January 27\u201330). Stability of feature selection algorithms. Proceedings of the Fifth IEEE International Conference on Data Mining (ICDM\u201905), Houston, TX, USA."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1777","DOI":"10.1007\/s00170-023-10846-y","article-title":"Fault feature selection for the identification of compound gear-bearing faults using firefly algorithm","volume":"125","author":"Athisayam","year":"2023","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Islam, M.R., Islam, M.M.M., and Kim, J. (2016, January 20\u201322). Feature selection techniques for increasing reliability of fault diagnosis of bearings. Proceedings of the 2016 9th International Conference on Electrical and Computer Engineering (ICECE), Dhaka, Bangladesh.","DOI":"10.1109\/ICECE.2016.7853940"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Shah, M., Borade, H., Sanghavi, V., Purohit, A., Wankhede, V., and Vakharia, V. (2023). Enhancing Tool Wear Prediction Accuracy Using Walsh\u2013Hadamard Transform, DCGAN and Dragonfly Algorithm-Based Feature Selection. Sensors, 23.","DOI":"10.3390\/s23083833"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3585","DOI":"10.1016\/j.eswa.2013.11.037","article-title":"Global geometric similarity scheme for feature selection in fault diagnosis","volume":"41","author":"Liu","year":"2014","journal-title":"Expert Syst. Appl."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.compeleceng.2013.11.024","article-title":"A survey on feature selection methods","volume":"40","author":"Chandrashekar","year":"2014","journal-title":"Comput. Electr. Eng."},{"key":"ref_20","first-page":"203","article-title":"Adjusted Measures for Feature Selection Stability for Data Sets with Similar Features","volume":"Volume 12565","author":"Bommert","year":"2020","journal-title":"Machine Learning, Optimization, and Data Science. LOD 2020. Lecture Notes in Computer Scienc"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5951","DOI":"10.1007\/s00521-019-04082-3","article-title":"Ensemble feature selection for high-dimensional data: A stability analysis across multiple domains","volume":"32","author":"Pes","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"33944","DOI":"10.1109\/ACCESS.2020.2974982","article-title":"Stability Assessment of Feature Selection Algorithms on Homogeneous Datasets: A Study for Sensor Array Optimization Problem","volume":"8","author":"Wijaya","year":"2020","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"110622","DOI":"10.1016\/j.measurement.2021.110622","article-title":"A new tool wear condition monitoring method based on deep learning under small samples","volume":"189","author":"Zhou","year":"2022","journal-title":"Measurement"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"108","DOI":"10.1016\/j.cirpj.2020.12.002","article-title":"Sensors selection for tool failure detection during machining processes: A simple accurate classification model","volume":"32","author":"Abubakr","year":"2021","journal-title":"CIRP J. Manuf. Sci. Technol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1369","DOI":"10.1109\/TII.2016.2644669","article-title":"Optimal Sensor Configuration and Feature Selection for AHU Fault Detection and Diagnosis","volume":"13","author":"Li","year":"2017","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_26","unstructured":"Agogino, A., and Goebel, K. (2022, September 27). Milling Data Set. BEST Lab, UC Berkeley. NASA Ames Prognostics Data Repository. Moffett Field, CA, USA, Available online: https:\/\/ti.arc.nasa.gov\/project\/prognostic-datarepository."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"3197","DOI":"10.1007\/s00170-018-2926-5","article-title":"Feature selection and a method to improve the performance of tool condition monitoring","volume":"100","author":"Xie","year":"2019","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1051","DOI":"10.1007\/s00170-019-03906-9","article-title":"Tool wear state recognition based on GWO\u2013SVM with feature selection of genetic algorithm","volume":"104","author":"Liao","year":"2019","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"137","DOI":"10.1016\/j.precisioneng.2020.09.025","article-title":"Tool wear monitoring in micromilling using support vector machine with vibration and sound sensors","volume":"67","author":"Gomes","year":"2021","journal-title":"Precis. Eng."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"40313","DOI":"10.1109\/ACCESS.2020.2976832","article-title":"Machinery fault diagnosis scheme using redefined dimensionless indicators and mRMR feature selection","volume":"8","author":"Hu","year":"2020","journal-title":"IEEE Access"},{"key":"ref_31","first-page":"157","article-title":"Feature selection for tool wear diagnosis using soft computing techniques","volume":"Volume 19166","author":"Goebel","year":"2000","journal-title":"ASME International Mechanical Engineering Congress and Exposition"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"109962","DOI":"10.1016\/j.measurement.2021.109962","article-title":"Effective feature selection with fuzzy entropy and similarity classifier for chatter vibration diagnosis","volume":"184","author":"Tran","year":"2021","journal-title":"Measurement"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"901","DOI":"10.1080\/00207540310001626652","article-title":"Identification of feature set for effective tool condition monitoring by acoustic emission sensing","volume":"42","author":"Sun","year":"2004","journal-title":"Int. J. Prod. Res."},{"key":"ref_34","unstructured":"Yang, H., Tieng, H., and Chen, S. (2011, January 13\u201318). Automatic feature selection and failure diagnosis for bearing faults. Proceedings of the SICE Annual Conference 2011, Tokyo, Japan."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"348","DOI":"10.1080\/10910340802293769","article-title":"A comparative study of feature selection for hidden Markov model-based micro-milling tool wear monitoring","volume":"12","author":"Zhu","year":"2008","journal-title":"Mach. Sci. Technol."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"12074","DOI":"10.1109\/TITS.2021.3109632","article-title":"A Sound-Based Fault Diagnosis Method for Railway Point Machines Based on Two-Stage Feature Selection Strategy and Ensemble Classifier","volume":"23","author":"Cao","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1016\/j.engappai.2009.09.004","article-title":"Feature extraction and selection from acoustic emission signals with an application in grinding wheel condition monitoring","volume":"23","author":"Liao","year":"2010","journal-title":"Eng. Appl. Artif. Intell."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"392","DOI":"10.1093\/bioinformatics\/btp630","article-title":"Robust biomarker identification for cancer diagnosis with ensemble feature selection methods","volume":"26","author":"Abeel","year":"2010","journal-title":"Bioinformatics"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"937","DOI":"10.1007\/s11634-017-0285-y","article-title":"Ensemble feature selection for high dimensional data: A new method and a comparative study","volume":"12","author":"Brahim","year":"2018","journal-title":"Adv. Data Anal. Classif."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Alhamidi, M.R., and Jatmiko, W. (2020). Optimal feature aggregation and combination for two-dimensional ensemble feature selection. Information, 1.","DOI":"10.3390\/info11010038"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"887","DOI":"10.1016\/j.ins.2021.08.032","article-title":"Feature selection using Fisher score and multilabel neighborhood rough sets for multilabel classification","volume":"578","author":"Sun","year":"2021","journal-title":"Inf. Sci."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"6167","DOI":"10.1007\/s10489-020-02141-0","article-title":"Iteratively local fisher score for feature selection","volume":"51","author":"Gan","year":"2021","journal-title":"Appl. Intell."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1142\/S0219720005001004","article-title":"Minimum redundancy feature selection from microarray gene expression data","volume":"3","author":"Ding","year":"2005","journal-title":"J. Bioinform. Comput. Biol."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1023\/A:1025667309714","article-title":"Theoretical and empirical analysis of ReliefF and RReliefF","volume":"53","author":"Kononenko","year":"2003","journal-title":"Mach. Learn."},{"key":"ref_45","unstructured":"Nogueira, S., and Brown, G. (2016). Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1662","DOI":"10.1093\/bioinformatics\/btp295","article-title":"Evaluating reproducibility of differential expression discoveries in microarray studies by considering correlated molecular changes","volume":"25","author":"Zhang","year":"2009","journal-title":"Bioinformatics"},{"key":"ref_47","first-page":"1060","article-title":"Stability of feature selection algorithm: A review","volume":"34","author":"Khaire","year":"2019","journal-title":"J. King Saud Univ. Comput. Inf. Sci."},{"key":"ref_48","unstructured":"Bommert, A. (2020). Integration of Feature Selection Stability in Model Fitting. [Ph.D. Thesis, TU Dortmund University]."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.inffus.2018.11.008","article-title":"Ensembles for feature selection: A review and future trends","volume":"52","year":"2019","journal-title":"Inf. Fusion"},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Bellapu, R.P., Tirumala, R., and Kurukundu, R.N. (2021, January 6\u20138). Evaluation of Homogeneous and Heterogeneous Distributed Ensemble Feature Selection Approaches for Classification of Rice Plant Diseases. Proceedings of the 2021 5th International Conference on Intelligent Computing and Control Systems (ICICCS), Madurai, India.","DOI":"10.1109\/ICICCS51141.2021.9432081"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Mo, W., Kari, T., Wang, H., Luan, L., and Gao, W. (2017, January 9\u201310). Power Transformer Fault Diagnosis Using Support Vector Machine and Particle Swarm Optimization. Proceedings of the 2017 10th International Symposium on Computational Intelligence and Design (ISCID), Hangzhou, China.","DOI":"10.1109\/ISCID.2017.165"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"73","DOI":"10.1007\/s00170-011-3504-2","article-title":"Tool condition monitoring based on numerous signal features","volume":"59","author":"Jemielniak","year":"2012","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/j.ifacol.2019.11.172","article-title":"Machine learning framework for predictive maintenance in milling","volume":"52","author":"Traini","year":"2019","journal-title":"IFAC-PapersOnLine"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"G\u00f3mez, \u00c1.L.P., Maim\u00f3, L.F., Celdr\u00e1n, A.H., and Clemente, F.J.G. (2020). Madics: A methodology for anomaly detection in industrial control systems. Symmetry, 12.","DOI":"10.3390\/sym12101583"},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"1402","DOI":"10.1016\/j.applthermaleng.2017.07.065","article-title":"Optimized neural network-based fault diagnosis strategy for VRF system in heating mode using data mining","volume":"125","author":"Guo","year":"2017","journal-title":"Appl. Therm. Eng."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Calabrese, F., Regattieri, A., Bortolini, M., Galizia, F.G., and Visentini, L. (2021). Feature-based multi-class classification and novelty detection for fault diagnosis of industrial machinery. Appl. Sci., 11.","DOI":"10.3390\/app11209580"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"37866","DOI":"10.1109\/ACCESS.2021.3063929","article-title":"An Efficient Rolling Bearing Fault Diagnosis Method Based on Spark and Improved Random Forest Algorithm","volume":"9","author":"Wan","year":"2021","journal-title":"IEEE Access"},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"32","DOI":"10.1016\/j.mfglet.2021.10.002","article-title":"Tool wear monitoring by ensemble learning and sensor fusion using power, sound, vibration, and AE signals","volume":"30","author":"Nasir","year":"2021","journal-title":"Manuf. Lett."},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"1174","DOI":"10.1016\/j.matpr.2021.02.061","article-title":"Friction stir welding tool condition monitoring using vibration signals and Random forest algorithm\u2013A Machine learning approach","volume":"46","author":"Balachandar","year":"2021","journal-title":"Mater. Today Proc."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/j.eswa.2015.08.050","article-title":"An improved global feature selection scheme for text classification","volume":"43","author":"Uysal","year":"2016","journal-title":"Expert Syst. Appl."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/9\/4461\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:28:37Z","timestamp":1760124517000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/9\/4461"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,5,3]]},"references-count":60,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2023,5]]}},"alternative-id":["s23094461"],"URL":"https:\/\/doi.org\/10.3390\/s23094461","relation":{"has-preprint":[{"id-type":"doi","id":"10.36227\/techrxiv.22047971.v3","asserted-by":"object"},{"id-type":"doi","id":"10.36227\/techrxiv.22047971","asserted-by":"object"},{"id-type":"doi","id":"10.36227\/techrxiv.22047971.v4","asserted-by":"object"}]},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,5,3]]}}}