{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,17]],"date-time":"2026-04-17T16:37:17Z","timestamp":1776443837366,"version":"3.51.2"},"reference-count":65,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2025,7,22]],"date-time":"2025-07-22T00:00:00Z","timestamp":1753142400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Ministry of Science and Higher Education of the Russian Federation, Goszadanie (State Assignment)","award":["FSER-2025-0013"],"award-info":[{"award-number":["FSER-2025-0013"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>In the modern world, there is a need to provide a better understanding of the importance or relevance of the available descriptive features for predicting target attributes to solve the feature ranking problem. Among the published works, the vast majority are devoted to the problems of feature selection and extraction, and not the problems of their ranking. In this paper, we propose a novel method based on the Bayesian approach that allows us to not only to build a methodically justified way of ranking features on small datasets, but also to methodically solve the problem of benchmarking the results obtained by various ranking algorithms. The proposed method is also model-free, since no restrictions are imposed on the model. We carry out an experimental comparison of our proposed method with the classical frequency method. For this, we use two synthetic datasets and two public medical datasets. As a result, we show that the proposed ranking method has a high level of self-consistency (stability) already at the level of 50 samples, which is greatly improved compared to classical logistic regression and SHAP ranking. All the experiments performed confirm our theoretical conclusions: with the growth of the sample, an increasing trend of mutual consistency is observed, and our method demonstrates at least comparable results, and often results superior to other methods in the values of self-consistency and monotonicity. The proposed method can be applied to a wide class of rankings of influence factors on small samples, including industrial tasks, forensics, psychology, etc.<\/jats:p>","DOI":"10.3390\/e27080773","type":"journal-article","created":{"date-parts":[[2025,7,22]],"date-time":"2025-07-22T15:00:42Z","timestamp":1753196442000},"page":"773","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Feature Ranking on Small Samples: A Bayes-Based Approach"],"prefix":"10.3390","volume":"27","author":[{"given":"Aleksandra","family":"Vatian","sequence":"first","affiliation":[{"name":"School of Translational Information Technologies, ITMO University, 197101 St. Petersburg, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Natalia","family":"Gusarova","sequence":"additional","affiliation":[{"name":"School of Translational Information Technologies, ITMO University, 197101 St. Petersburg, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ivan","family":"Tomilov","sequence":"additional","affiliation":[{"name":"School of Translational Information Technologies, ITMO University, 197101 St. Petersburg, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,7,22]]},"reference":[{"key":"ref_1","first-page":"267","article-title":"Ensemble feature ranking","volume":"Volume 3202","author":"Boulicaut","year":"2004","journal-title":"PKDD Lecture Notes in Computer Science, Proceedings of the Knowledge Discovery in Databases: PKDD 2004: 8th European Conference on Principles and Practice of Knowledge Discovery in Databases, Pisa, Italy, 20\u201324 September 2004"},{"key":"ref_2","first-page":"4379","article-title":"Feature ranking for semi-supervised learning","volume":"112","author":"Kocev","year":"2022","journal-title":"Mach. Learn."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s11004-019-09838-0","article-title":"Value of Geologically Derived Features in Machine Learning Facies Classification","volume":"52","author":"Halotel","year":"2020","journal-title":"Math. Geosci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"107921","DOI":"10.1016\/j.petrol.2020.107291","article-title":"Log data-driven model and feature ranking for water saturation prediction using machine learning approach","volume":"194","author":"Miah","year":"2020","journal-title":"J. Pet. Sci. Eng."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"105234","DOI":"10.1016\/j.ssci.2021.105234","article-title":"Ranking of occupational health and safety risks by a multi-criteria perspective: Inclusion of human factors and application of VIKOR","volume":"138","author":"Giallanza","year":"2021","journal-title":"Saf. Sci."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"629","DOI":"10.1007\/s00477-021-02069-y","article-title":"Occupational health, safety and environmental risk assessment in textile production industry through a Bayesian BWM-VIKOR approach","volume":"36","author":"Ak","year":"2022","journal-title":"Stoch. Environ. Res. Risk Assess."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Abdulla, A., Baryannis, G., and Badi, I. (2019, January 6). Weighting the Key Features Affecting Supplier Selection using Machine Learning Techniques. Proceedings of the 7th International Conference on Transport and Logistics, Ni\u0161, Serbia.","DOI":"10.20944\/preprints201912.0154.v1"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Zhang, Y., Zhang, H., and Zhang, B. (2022). An Effective Ensemble Automatic Feature Selection Method for Network Intrusion Detection. Information, 13.","DOI":"10.3390\/info13070314"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Megantara, A.A., and Ahmad, T. (2020, January 15\u201316). Feature Importance Ranking for Increasing Performance of Intrusion Detection System. Proceedings of the 3rd International Conference on Computer and Informatics Engineering (IC2IE), Yogyakarta, Indonesia.","DOI":"10.1109\/IC2IE50715.2020.9274570"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"136991","DOI":"10.1016\/j.scitotenv.2020.136991","article-title":"A machine-learning framework for predicting multiple air pollutants\u2019 concentrations via multi-target regression and feature selection","volume":"715","author":"Masmoudi","year":"2020","journal-title":"Sci. Total Environ."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"7791","DOI":"10.5194\/gmd-15-7791-2022","article-title":"Development of a regional feature selection-based machine learning system (RFSML v1.0) for air pollution forecasting over China","volume":"15","author":"Fang","year":"2022","journal-title":"Geosci. Model Dev."},{"key":"ref_12","first-page":"595","article-title":"Intelligent Clinical Decision Support for Small Patient Datasets","volume":"23","author":"Vatian","year":"2023","journal-title":"Sci. Tech. J. Inf. Technol. Mech. Opt."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Remeseiro, B., and Bolon-Canedo, V. (2019). A review of feature selection methods in medical applications. Comput. Biol. Med., 112.","DOI":"10.1016\/j.compbiomed.2019.103375"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1007\/s10115-018-1185-y","article-title":"Fisher score and Matthews correlation coefficient-based feature subset selection for heart disease diagnosis using support vector machines","volume":"58","author":"Saqlain","year":"2019","journal-title":"Knowl. Inf. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Wu, L., Hu, Y., Liu, X., Zhang, X., Chen, W., Yu, A.S.L., Kellum, J.A., Waitman, L.R., and Liu, M. (2018). Feature Ranking in Predictive Models for Hospital-Acquired Acute Kidney Injury. Sci. Rep., 8.","DOI":"10.1038\/s41598-018-35487-0"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1464","DOI":"10.1198\/jasa.2011.tm10563","article-title":"Model-Free Feature Screening for Ultrahigh Dimensional Data","volume":"106","author":"Zhu","year":"2011","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"126186","DOI":"10.1016\/j.neucom.2023.03.047","article-title":"Deep Feature Screening: Feature Selection for Ultra High-Dimensional Data via Deep Neural Networks","volume":"538","author":"Li","year":"2023","journal-title":"Neurocomputing"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"112973","DOI":"10.1016\/j.knosys.2025.112973","article-title":"VSI: An Interpretable Bayesian Feature Ranking Method Based on Vendi Score","volume":"311","author":"Mousavi","year":"2025","journal-title":"Knowl.-Based Syst."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Boulicaut, J.F., Esposito, F., Giannotti, F., and Pedreschi, D. (2004). Feature selection filters based on the permutation test. Lecture Notes in Computer Science, Proceedings of the Machine Learning: ECML 2004: 15th European Conference on Machine Learning, Pisa, Italy, 20\u201324 September 2004, Springer.","DOI":"10.1007\/b100702"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"106439","DOI":"10.1016\/j.knosys.2020.106439","article-title":"Relevance assignation feature selection method based on mutual information for machine learning","volume":"209","author":"Gao","year":"2020","journal-title":"Knowl.-Based Systems."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Bhadra, T., Mallik, S., Hasan, N., and Zhao, Z. (2022). Comparison of five supervised feature selection algorithms leading to top features and gene signatures from multi-omics data in cancer. BMC Bioinform., 23.","DOI":"10.1186\/s12859-022-04678-y"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1177\/0165551518770967","article-title":"Mutual information and sensitivity analysis for feature selection in customer targeting: A comparative study","volume":"45","author":"Barraza","year":"2019","journal-title":"J. Inf. Sci."},{"key":"ref_23","unstructured":"Quinlan, J.R. (1993). C4.5: Programs for Machine Learning, Morgan Kaufmann Publishers, Inc."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1109\/TPAMI.2005.159","article-title":"Feature Selection Based on Mutual Information: Criteria of Max-Dependency, Max-Relevance, and Min-Redundancy","volume":"27","author":"Peng","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"1518","DOI":"10.1126\/science.1205438","article-title":"Detecting novel associations in large data sets","volume":"334","author":"Reshef","year":"2011","journal-title":"Science"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.jaci.2020.04.006","article-title":"Risk factors for severity and mortality in adult COVID-19 inpatients in Wuhan","volume":"146","author":"Li","year":"2020","journal-title":"J. Allergy Clin. Immunol."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1038\/s41401-021-00627-2","article-title":"Clinical characteristics and related risk factors of disease severity in 101 COVID-19 patients hospitalized in Wuhan, China","volume":"43","author":"Liu","year":"2021","journal-title":"Acta Pharmacol. Sin."},{"key":"ref_28","first-page":"172","article-title":"Cardiovascular comorbidities as predictors for severe COVID-19 infection or death","volume":"7","author":"Phelps","year":"2021","journal-title":"Eur. Heart J."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"100180","DOI":"10.1016\/j.imu.2019.100180","article-title":"A Random Forest based predictor for medical data classification using feature ranking","volume":"15","author":"Alam","year":"2019","journal-title":"Inform. Med. Unlocked"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Joloudari, J.H., Joloudari, E.H., Saadatfar, H., Ghasemigol, M., Razavi, S.M., Mosavi, A., Nabipour, N., Shamshirband, S., and Nadai, L. (2020). Coronary Artery Disease Diagnosis; Ranking the Significant Features Using Random Trees Model. Int. J. Environ. Res. Public Health, 17.","DOI":"10.20944\/preprints202001.0220.v1"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Muthukrishnan, R., and Rohini, R. (2016, January 24). LASSO: A feature selection technique in predictive modeling for machine learning. Proceedings of the 2016 IEEE International Conference on Advances in Computer Applications (ICACA), Coimbatore, India.","DOI":"10.1109\/ICACA.2016.7887916"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"301","DOI":"10.1111\/j.1467-9868.2005.00503.x","article-title":"Regularization and variable selection via the elastic net","volume":"67","author":"Zou","year":"2005","journal-title":"J. R. Stat. Soc. Ser. B"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Chen, T., and Guestrin, C. (2016, January 13\u201317). XGBoost: A Scalable Tree Boosting System. Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD \u201916), San Francisco, CA, USA.","DOI":"10.1145\/2939672.2939785"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_35","unstructured":"Rice, J. (1995). Mathematical Statistics and Data Analysis, Duxbury Press."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Schulte, R.V., Prinsen, E.C., Hermen, H.J., and Buurke, J.H. (2021). Genetic Algorithm for Feature Selection in Lower Limb Pattern Recognition. Sec. Biomed. Robot., 8.","DOI":"10.3389\/frobt.2021.710806"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Ali, W., and Saeed, F. (2023). Hybrid Filter and Genetic Algorithm-Based Feature Selection for Improving Cancer Classification in High-Dimensional Microarray Data. Processes, 11.","DOI":"10.3390\/pr11020562"},{"key":"ref_38","unstructured":"Abdollahi, J., and Nouri-Moghaddam, B. (2021). Feature selection for medical diagnosis: Evaluation for using a hybrid stacked-genetic approach in the diagnosis of heart disease. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Xie, H., Zhang, L., Lim, C.P., Yu, Y., and Liu, H. (2021). Feature Selection Using Enhanced Particle Swarm Optimisation for Classification Models. Sensors, 21.","DOI":"10.3390\/s21051816"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Shen, X., Yang, F., Yang, P., Yang, M., Xu, L., Zhuo, J., Wang, J., Lu, D., Liu, Z., and Zheng, S. (2020). A Contrast-Enhanced Computed Tomography Based Radiomics Approach for Preoperative Differentiation of Pancreatic Cystic Neoplasm Subtypes: A Feasibility Study. Front. Oncol., 10.","DOI":"10.3389\/fonc.2020.00248"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Awad, M., and Fraihat, S. (2023). Recursive Feature Elimination with Cross-Validation with Decision Tree: Feature Selection Method for Machine Learning-Based Intrusion Detection Systems. J. Sens. Actuator Netw., 12.","DOI":"10.3390\/jsan12050067"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Nizami, I.F., Majid, M., and Khurshid, K. (2017, January 10\u201314). Efficient feature selection for Blind Image Quality Assessment based on natural scene statistics. Proceedings of the 14th International Bhurban Conference on Applied Sciences and Technology (IBCAST), Islamabad, Pakistan.","DOI":"10.1109\/IBCAST.2017.7868071"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1186\/s40537-020-00385-8","article-title":"Stable bagging feature selection on medical data","volume":"8","author":"Alelyani","year":"2021","journal-title":"J. Big Data"},{"key":"ref_44","unstructured":"Lundberg, S.M., and Lee, S.I. (2017). A Unified Approach to Interpreting Model Predictions. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"103502","DOI":"10.1016\/j.artint.2021.103502","article-title":"Explaining individual predictions when features are dependent: More accurate approximations to Shapley values","volume":"298","author":"Aas","year":"2021","journal-title":"Artif. Intell."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"72","DOI":"10.3390\/fintech1010006","article-title":"Shapley Feature Selection","volume":"1","author":"Gramegna","year":"2022","journal-title":"FinTech"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Amini, M.R., Canu, S., Fischer, A., Guns, T., Kralj Novak, P., and Tsoumakas, G. (2022). Powershap: A Power-full Shapley Feature Selection Method. Lecture Notes in Computer Science, Proceedings of the Machine Learning and Knowledge Discovery in Databases. European Conference, ECML PKDD 2022, Grenoble, France, 19\u201323 September 2022, Springer.","DOI":"10.1007\/978-3-031-26409-2"},{"key":"ref_48","unstructured":"Kumar, I.E., Venkatasubramanian, S., Scheidegger, C., and Friedler, S.A. (2020). Problems with Shapley-value-based explanations as feature importance measures. arXiv."},{"key":"ref_49","unstructured":"Yuan, H., Liu, M., Kang, L., Miao, C., and Wu, Y. (2022). An empirical study of the effect of background data size on the stability of SHapley Additive exPlanations (SHAP) for deep learning models. arXiv."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"3897","DOI":"10.1007\/s10994-022-06221-9","article-title":"A user-guided Bayesian framework for ensemble feature selection in life science applications (UBayFS)","volume":"111","author":"Jenul","year":"2022","journal-title":"Mach. Learn."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"2370","DOI":"10.1080\/02664763.2021.1902955","article-title":"Review of Bayesian selection methods for categorical predictors using JAGS","volume":"49","author":"Jreich","year":"2021","journal-title":"J. Appl. Stat."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Bartonicek, A., Wickham, S.R., Pat, N., and Conner, T.S. (2021). The value of Bayesian predictive projection for variable selection: An example of selecting lifestyle predictors of young adult well-being. BMC Public Health, 21.","DOI":"10.1186\/s12889-021-10690-3"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/sim.8107","article-title":"Bayesian variable selection based on clinical relevance weights in small sample studies\u2014Application to colon cancer","volume":"38","author":"Boulet","year":"2019","journal-title":"Stat. Med."},{"key":"ref_54","unstructured":"Wasserman, L. (2010). All of Statistics: A Concise Course in Statistical Inference, Springer."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1023\/A:1007612503587","article-title":"A Formalism for Relevance and Its Application in Feature Subset Selection","volume":"41","author":"Bell","year":"2000","journal-title":"Mach. Learn."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Cover, T.M., and Thomas, J.A. (2006). Elements of information theory, Wiley. [2nd ed.].","DOI":"10.1002\/047174882X"},{"key":"ref_57","unstructured":"van der Vaart, A.W. (1998). 10.2 Bernstein\u2013von Mises Theorem. Asymptotic Statistics, Cambridge University Press."},{"key":"ref_58","unstructured":"Hendeby, G., and Gustafsson, F. (2025, July 10). On Nonlinear Transformations of Gaussian Distributions. Available online: https:\/\/users.isy.liu.se\/en\/rt\/fredrik\/reports\/07SSPut.pdf."},{"key":"ref_59","unstructured":"Guyon, I. (2025, July 10). Design of Experiments for the NIPS 2003 Variable Selection Benchmark. Available online: https:\/\/archive.ics.uci.edu\/ml\/machine-learning-databases\/madelon\/Dataset.pdf."},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1038\/s41746-020-00372-6","article-title":"Machine learning-based prediction of COVID-19 diagnosis based on symptoms","volume":"4","author":"Zoabi","year":"2021","journal-title":"Npj Digit. Med."},{"key":"ref_61","unstructured":"COVID-19 (2025, July 10). COVID-19 Patient\u2019s Symptoms, Status, and Medical History. Available online: https:\/\/www.kaggle.com\/datasets\/meirnizri\/covid19-dataset."},{"key":"ref_62","unstructured":"Lapp, D. (2025, July 10). Heart Disease Dataset. Available online: https:\/\/www.kaggle.com\/datasets\/johnsmith88\/heart-disease-dataset."},{"key":"ref_63","unstructured":"Fedesoriano (2025, July 10). Heart Failure Prediction Dataset. Available online: https:\/\/www.kaggle.com\/datasets\/fedesoriano\/heart-failure-prediction."},{"key":"ref_64","unstructured":"UCI Machine Learning and 1 Collaborator (2025, July 10). Red Wine Quality. Available online: https:\/\/www.kaggle.com\/datasets\/uciml\/red-wine-quality-cortez-et-al-2009."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"4611","DOI":"10.21105\/joss.04611","article-title":"Fseval: A Benchmarking Framework for Feature Selection and Feature Ranking Algorithms","volume":"7","author":"Overschie","year":"2022","journal-title":"J. Open Source Softw."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/8\/773\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:14:11Z","timestamp":1760033651000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/27\/8\/773"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,22]]},"references-count":65,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,8]]}},"alternative-id":["e27080773"],"URL":"https:\/\/doi.org\/10.3390\/e27080773","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,22]]}}}