{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T13:19:49Z","timestamp":1763558389677,"version":"3.45.0"},"reference-count":55,"publisher":"MDPI AG","issue":"22","license":[{"start":{"date-parts":[[2025,11,16]],"date-time":"2025-11-16T00:00:00Z","timestamp":1763251200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"EK\u00d6P-25-3 University Research Scholarship Program of the Ministry for Culture and Innovation from the source of the National Research, Development, and Innovation Fund"},{"name":"J\u00e1nos Bolyai Research Scholarship of the Hungarian Academy of Sciences"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JCM"],"abstract":"<jats:p>Background: With the rapid emergence of novel survival models and feature selection methods, comparing them with traditional approaches is essential to define contexts of optimal performance. Methods: This study systematically evaluates nine survival models combined with nine feature selection methods for predicting the occurrence of angina pectoris using electronic health record (EHR) data from a Hungarian hospital (n = 29,655, features = 1150). Performance was assessed with the concordance index (C-index) and integrated Brier score (IBS) to compare predictive accuracy across methods. Results: Tree-based survival models, particularly gradient-boosted survival (GBS) and random survival forest (RSF), consistently outperformed conventional approaches in terms of C-index, but showed slightly worse calibration as reflected in their higher IBSs. The best-performing model was RSF, which was optimized using Bayesian hyperparameter tuning. For feature selection, tree-based methods such as Boruta and RSF-based approaches showed superior performance. We further identified clusters of feature selection methods and generated consensus feature sets. We also analyzed the internal relationships between the selected features. Survival model performance was also examined over time using the time-dependent Area Under the Curve (AUC) based on the best-performing feature set. Conclusions: Our findings highlight the substantial impact of recent methodological innovations in survival analysis, which offer significant gains in predictive accuracy and efficiency, ultimately support more robust clinical decision-making in the early identification of angina pectoris among patients with diabetes.<\/jats:p>","DOI":"10.3390\/jcm14228111","type":"journal-article","created":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T12:55:31Z","timestamp":1763556931000},"page":"8111","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Feature Selection and Model Optimization for Survival Prediction in Patients with Angina Pectoris"],"prefix":"10.3390","volume":"14","author":[{"given":"R\u00f3bert","family":"Bata","sequence":"first","affiliation":[{"name":"Department of Epidemiology, Faculty of Health Sciences, University of Debrecen, H-4032 Debrecen, Hungary"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-0593-0188","authenticated-orcid":false,"given":"Amr Sayed","family":"Ghanem","sequence":"additional","affiliation":[{"name":"Department of Epidemiology, Faculty of Health Sciences, University of Debrecen, H-4032 Debrecen, Hungary"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0554-7350","authenticated-orcid":false,"given":"Attila Csaba","family":"Nagy","sequence":"additional","affiliation":[{"name":"Department of Epidemiology, Faculty of Health Sciences, University of Debrecen, H-4032 Debrecen, Hungary"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,11,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"198","DOI":"10.1093\/jamia\/ocw042","article-title":"Opportunities and challenges in developing risk prediction models with electronic health records data: A systematic review","volume":"24","author":"Goldstein","year":"2017","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"61","DOI":"10.1146\/annurev-publhealth-032315-021353","article-title":"Using Electronic Health Records for Population Health Research: A Review of Methods and Applications","volume":"37","author":"Casey","year":"2016","journal-title":"Annu. Rev. Public Health"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"187","DOI":"10.1111\/j.2517-6161.1972.tb00899.x","article-title":"Regression Models and Life-Tables","volume":"34","author":"Cox","year":"1972","journal-title":"J. R. Stat. Soc. Ser. B (Methodol.)"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Hastie, T., Tibshirani, R., and Friedman, J. (2009). High-Dimensional Problems: pN. The Elements of Statistical Learning: Data Mining, Inference, and Prediction, Springer.","DOI":"10.1007\/b94608_18"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2663","DOI":"10.1007\/s40747-021-00637-x","article-title":"Feature dimensionality reduction: A review","volume":"8","author":"Jia","year":"2022","journal-title":"Complex Intell. Syst."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Kolasseri, A.E. (2024). BV Comparative study of machine learning and statistical survival models for enhancing cervical cancer prognosis and risk factor assessment using SEER data. Sci. Rep., 14.","DOI":"10.1038\/s41598-024-72790-5"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"759","DOI":"10.1016\/j.procs.2024.10.302","article-title":"Comparative analysis of machine learning and survival analysis for breast cancer prediction","volume":"245","author":"Emily","year":"2024","journal-title":"Procedia Comput. Sci."},{"key":"ref_8","first-page":"749","article-title":"Comparison of Survival Models for Analyzing Prognostic Factors in Gastric Cancer Patients","volume":"19","author":"Habibi","year":"2018","journal-title":"Asian Pac. J. Cancer Prev."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Wang, M., Greenberg, M., Forkert, N.D., Chekouo, T., Afriyie, G., Ismail, Z., Smith, E., and Sajobi, T.T. (2022). Dementia risk prediction in individuals with mild cognitive impairment: A comparison of Cox regression and machine learning models. BMC Med. Res. Methodol., 22.","DOI":"10.1186\/s12874-022-01754-y"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Spooner, A., Chen, E., Sowmya, A., Sachdev, P., Kochan, N.A., Trollor, J., and Brodaty, J. (2020). A comparison of machine learning methods for survival analysis of high-dimensional clinical data for dementia prediction. Sci. Rep., 10.","DOI":"10.1038\/s41598-020-77220-w"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Leger, S., Zwanenburg, A., Pilz, K., Lohaus, F., Linge, A., Z\u00f6phel, K., Kotzerke, J., Schreiber, A., Tinhofer, I., and Budach, V. (2017). A comparative study of machine learning methods for time-to-event survival data for radiomics risk modelling. Sci. Rep., 7.","DOI":"10.1038\/s41598-017-13448-3"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2080","DOI":"10.1093\/bioinformatics\/btm305","article-title":"Predicting survival from microarray data\u2014A comparative study","volume":"23","author":"Aldrin","year":"2007","journal-title":"Bioinformatics"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Steele, A.J., Denaxas, S.C., Shah, A.D., Hemingway, H., and Luscombe, N.M. (2018). Machine learning models in electronic health records can outperform conventional survival models for predicting patient mortality in coronary artery disease. PLoS ONE, 13.","DOI":"10.1101\/256008"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"5310","DOI":"10.1002\/sim.6246","article-title":"Investigating the prediction ability of survival models based on both clinical and omics data: Two case studies","volume":"33","author":"Sauerbrei","year":"2014","journal-title":"Stat. Med."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.artmed.2016.07.004","article-title":"Survival analysis for high-dimensional, heterogeneous medical data: Exploring feature extraction as an alternative to feature selection","volume":"72","author":"Conjeti","year":"2016","journal-title":"Artif. Intell. Med."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1002\/(SICI)1097-0258(19970228)16:4<385::AID-SIM380>3.0.CO;2-3","article-title":"The Lasso Method for Variable Selection in the Cox Model","volume":"16","author":"Tibshirani","year":"1997","journal-title":"Stat. Med."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Zhou, H., Xin, Y., and Li, S. (2023). A diabetes prediction model based on Boruta feature selection and ensemble learning. BMC Bioinform., 24.","DOI":"10.1186\/s12859-023-05300-5"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"106839","DOI":"10.1016\/j.csda.2019.106839","article-title":"Benchmark for filter methods for feature selection in high-dimensional classification data","volume":"143","author":"Bommert","year":"2020","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"308","DOI":"10.1080\/00031305.2020.1790217","article-title":"Feature Engineering and Selection: A Practical Approach for Predictive Models","volume":"Volume 74","author":"Kuhn","year":"2020","journal-title":"The American Statistician"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1002\/sim.4780030207","article-title":"Regression modelling strategies for improved prognostic prediction","volume":"3","author":"Harrell","year":"1984","journal-title":"Stat. Med."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1515\/dx-2023-0109","article-title":"Validity and reliability of Brier scoring for assessment of probabilistic diagnostic reasoning","volume":"12","author":"Stehouwer","year":"2025","journal-title":"Diagnosis"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4","article-title":"Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors","volume":"15","author":"Harrell","year":"1996","journal-title":"Stat. Med."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2179","DOI":"10.1002\/sim.9717","article-title":"Pitfalls of the concordance index for survival outcomes","volume":"42","author":"Hartman","year":"2023","journal-title":"Stat. Med."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2543","DOI":"10.1001\/jama.1982.03320430047030","article-title":"Evaluating the yield of medical tests","volume":"247","author":"Harrell","year":"1982","journal-title":"JAMA"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2109","DOI":"10.1002\/sim.1802","article-title":"Overall C as a measure of discrimination in survival analysis: Model specific population value and confidence interval estimation","volume":"23","author":"Pencina","year":"2004","journal-title":"Stat. Med."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1697","DOI":"10.3348\/kjr.2021.0223","article-title":"Review of Statistical Methods for Evaluating the Performance of Survival or Other Time-to-Event Prediction Models (from Conventional to Deep Learning Approaches)","volume":"22","author":"Park","year":"2021","journal-title":"Korean J. Radiol."},{"key":"ref_27","unstructured":"Steyerberg, E.W. (2009). Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating, Springer. International Statistical Review."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"128","DOI":"10.1097\/EDE.0b013e3181c30fb2","article-title":"Assessing the performance of prediction models: A framework for traditional and novel measures","volume":"21","author":"Steyerberg","year":"2010","journal-title":"Epidemiology"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Yin, H., Tino, P., Corchado, E., Byrne, W., and Yao, X. (2007). Filter Methods for Feature Selection\u2014A Comparative Study. Intelligent Data Engineering and Automated Learning\u2014IDEAL, Springer.","DOI":"10.1007\/978-3-540-77226-2"},{"key":"ref_30","first-page":"105","article-title":"Log-transformation and its implications for data analysis","volume":"26","author":"Feng","year":"2014","journal-title":"Shanghai Arch. Psychiatry"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"107525","DOI":"10.1016\/j.patcog.2020.107525","article-title":"Mutual information based feature subset selection in multivariate time series classification","volume":"108","author":"Ircio","year":"2020","journal-title":"Pattern Recognit."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1016\/j.procs.2015.12.117","article-title":"Wrapper Feature Subset Selection for Dimension Reduction Based on Ensemble Learning Algorithm","volume":"72","author":"Panthong","year":"2015","journal-title":"Procedia Comput. Sci."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Darst, B.F., Malecki, K.C., and Engelman, C.D. (2018). Using recursive feature elimination in random forest to account for correlated variables in high dimensional data. BMC Genet., 19.","DOI":"10.1186\/s12863-018-0633-8"},{"key":"ref_34","unstructured":"Wang, S., Tang, J., and Liu, H. (March, January 25). Embedded Unsupervised Feature Selection. Proceedings of the AAAI Conference on Artificial Intelligence, Philadelphia, PA, USA. Available online: https:\/\/ojs.aaai.org\/index.php\/AAAI\/article\/view\/9211."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"216969","DOI":"10.1109\/ACCESS.2020.3042119","article-title":"CBRG: A Novel Algorithm for Handling Missing Data Using Bayesian Ridge Regression and Feature Selection Based on Gain Ratio","volume":"8","author":"Mostafa","year":"2020","journal-title":"IEEE Access"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Jo, I., Lee, S., and Oh, S. (2019). Improved Measures of Redundancy and Relevance for mRMR Feature Selection. Computers, 8.","DOI":"10.3390\/computers8020042"},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"873595","DOI":"10.1155\/2013\/873595","article-title":"A Gradient Boosting Algorithm for Survival Analysis via Direct Optimization of Concordance Index","volume":"2013","author":"Chen","year":"2013","journal-title":"Comput. Math. Methods Med."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"85","DOI":"10.22283\/qbs.2017.36.2.85","article-title":"A Selective Review on Random Survival Forests for High Dimensional Data","volume":"36","author":"Wang","year":"2017","journal-title":"Quant. Bio-Sci."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1016\/S0167-9473(01)00065-2","article-title":"Stochastic gradient boosting","volume":"38","author":"Friedman","year":"2002","journal-title":"Comput. Stat. Data Anal."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"1056","DOI":"10.1016\/j.spl.2010.02.020","article-title":"Consistency of random survival forests","volume":"80","author":"Ishwaran","year":"2010","journal-title":"Stat. Probab. Lett."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Kalbfleisch, J.D., and Prentice, R.L. (2002). The Statistical Analysis of Failure Time Data, Wiley-Interscience. [2nd ed.].","DOI":"10.1002\/9781118032985"},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Klein, J.P., and Moeschberger, M.L. (2003). Refinements of the Semiparametric Proportional Hazards Model. Survival Analysis: Techniques for Censored and Truncated Data, Springer.","DOI":"10.1007\/0-387-21645-6_9"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1093\/biomet\/62.2.269","article-title":"Partial likelihood","volume":"62","author":"Cox","year":"1975","journal-title":"Biometrika"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"515","DOI":"10.1093\/biomet\/81.3.515","article-title":"Proportional hazards tests and diagnostics based on weighted residuals","volume":"81","author":"Grambsch","year":"1994","journal-title":"Biometrika"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Therneau, T.M., and Grambsch, P.M. (2000). The Cox Model. Modeling Survival Data: Extending the Cox Model, Springer.","DOI":"10.1007\/978-1-4757-3294-8"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"293","DOI":"10.1115\/1.4010337","article-title":"A Statistical Distribution Function of Wide Applicability","volume":"18","author":"Weibull","year":"2021","journal-title":"J. Appl. Mech."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Collett, D. (2015). Modelling Survival Data in Medical Research, Chapman and Hall\/CRC. [3rd ed.].","DOI":"10.1201\/b18041"},{"key":"ref_48","unstructured":"Lawless, J.F. (2011). Statistical Models and Methods for Lifetime Data, John Wiley & Sons."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Burnham, K.P., and Anderson, D.R. (2004). Model Selection and Multimodel Inference, Springer. Available online: http:\/\/link.springer.com\/10.1007\/b97636.","DOI":"10.1007\/b97636"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"841","DOI":"10.1214\/08-AOAS169","article-title":"Random survival forests","volume":"2","author":"Ishwaran","year":"2008","journal-title":"Ann. Appl. Stat."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"87","DOI":"10.1093\/bioinformatics\/btq617","article-title":"Improved performance on high-dimensional survival data by application of Survival-SVM","volume":"27","author":"Pelckmans","year":"2011","journal-title":"Bioinformatics"},{"key":"ref_52","unstructured":"Belle, V.V., Pelckmans, K., Suykens, J.A.K., and Huffel, S.V. (2008, January 23\u201325). Survival SVM: A Practical Scalable Algorithm. Proceedings of the ESANN\u20192008 proceedings, European Symposium on Artificial Neural Networks\u2014Advances in Computational Intelligence and Learning, Bruges, Belgium."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"1632","DOI":"10.1093\/bioinformatics\/btn253","article-title":"Sparse kernel methods for high-dimensional survival data","volume":"24","author":"Evers","year":"2008","journal-title":"Bioinformatics"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Matusik, P.S., Bryll, A., Pac, A., Popiela, T.J., and Matusik, P.T. (2022). Clinical Data, Chest Radiograph and Electrocardiography in the Screening for Left Ventricular Hypertrophy: The CAR2E2 Score. J. Clin. Med., 11.","DOI":"10.3390\/jcm11133585"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Cuthbert, A.R., Giles, L.C., Glonek, G., Kalisch Ellett, L.M., and Pratt, N.L. (2022). A comparison of survival models for prediction of eight-year revision risk following total knee and hip arthroplasty. BMC Med. Res. Methodol., 22.","DOI":"10.1186\/s12874-022-01644-3"}],"container-title":["Journal of Clinical Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/22\/8111\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,19]],"date-time":"2025-11-19T13:16:43Z","timestamp":1763558203000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2077-0383\/14\/22\/8111"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,16]]},"references-count":55,"journal-issue":{"issue":"22","published-online":{"date-parts":[[2025,11]]}},"alternative-id":["jcm14228111"],"URL":"https:\/\/doi.org\/10.3390\/jcm14228111","relation":{},"ISSN":["2077-0383"],"issn-type":[{"type":"electronic","value":"2077-0383"}],"subject":[],"published":{"date-parts":[[2025,11,16]]}}}