{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,13]],"date-time":"2026-10-13T06:41:11Z","timestamp":1791873671204,"version":"4.3.4"},"reference-count":58,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T00:00:00Z","timestamp":1673222400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100019170","name":"IACTEC Technological Training program","doi-asserted-by":"publisher","award":["TF INNOVA 2016-2021"],"award-info":[{"award-number":["TF INNOVA 2016-2021"]}],"id":[{"id":"10.13039\/501100019170","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100019170","name":"IACTEC Technological Training program","doi-asserted-by":"publisher","award":["POC 2014-2020"],"award-info":[{"award-number":["POC 2014-2020"]}],"id":[{"id":"10.13039\/501100019170","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007757","name":"Agencia Canaria de Investigacion, Innovacion y Sociedad de la Informaci\u00f3n (ACIISI)","doi-asserted-by":"publisher","award":["TF INNOVA 2016-2021"],"award-info":[{"award-number":["TF INNOVA 2016-2021"]}],"id":[{"id":"10.13039\/501100007757","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100007757","name":"Agencia Canaria de Investigacion, Innovacion y Sociedad de la Informaci\u00f3n (ACIISI)","doi-asserted-by":"publisher","award":["POC 2014-2020"],"award-info":[{"award-number":["POC 2014-2020"]}],"id":[{"id":"10.13039\/501100007757","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Diabetes mellitus presents a high prevalence around the world. A common and long-term derived complication is diabetic foot ulcers (DFUs), which have a global prevalence of roughly 6.3%, and a lifetime incidence of up to 34%. Infrared thermograms, covering the entire plantar aspect of both feet, can be employed to monitor the risk of developing a foot ulcer, because diabetic patients exhibit an abnormal pattern that may indicate a foot disorder. In this study, the publicly available INAOE dataset composed of thermogram images of healthy and diabetic subjects was employed to extract relevant features aiming to establish a set of state-of-the-art features that efficiently classify DFU. This database was extended and balanced by fusing it with private local thermograms from healthy volunteers and generating synthetic data via synthetic minority oversampling technique (SMOTE). State-of-the-art features were extracted using two classical approaches, LASSO and random forest, as well as two variational deep learning (DL)-based ones: concrete and variational dropout. Then, the most relevant features were detected and ranked. Subsequently, the extracted features were employed to classify subjects at risk of developing an ulcer using as reference a support vector machine (SVM) classifier with a fixed hyperparameter configuration to evaluate the robustness of the selected features. The new set of features extracted considerably differed from those currently considered state-of-the-art but provided a fair performance. Among the implemented extraction approaches, the variational DL ones, particularly the concrete dropout, performed the best, reporting an F1 score of 90% using the aforementioned SVM classifier. In comparison with features previously considered as the state-of-the-art, approximately 15% better performance was achieved for classification.<\/jats:p>","DOI":"10.3390\/s23020757","type":"journal-article","created":{"date-parts":[[2023,1,9]],"date-time":"2023-01-09T07:55:39Z","timestamp":1673250939000},"page":"757","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":21,"title":["Feature Ranking by Variational Dropout for Classification Using Thermograms from Diabetic Foot Ulcers"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2508-2845","authenticated-orcid":false,"given":"Abian","family":"Hernandez-Guedes","sequence":"first","affiliation":[{"name":"Instituto Universitario de Investigaciones Biom\u00e9dicas y Sanitarias (IUIBS), Universidad de Las Palmas de Gran Canaria, 35016 Las Palmas de Gran Canaria, Spain"},{"name":"Instituto Universitario de Microelectr\u00f3nica Aplicada (IUMA), Universidad de Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1645-0810","authenticated-orcid":false,"given":"Natalia","family":"Arteaga-Marrero","sequence":"additional","affiliation":[{"name":"Grupo Tecnolog\u00eda M\u00e9dica IACTEC, Instituto de Astrof\u00edsica de Canarias (IAC), 38205 San Crist\u00f3bal de La Laguna, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1028-6957","authenticated-orcid":false,"given":"Enrique","family":"Villa","sequence":"additional","affiliation":[{"name":"Grupo Tecnolog\u00eda M\u00e9dica IACTEC, Instituto de Astrof\u00edsica de Canarias (IAC), 38205 San Crist\u00f3bal de La Laguna, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3784-5504","authenticated-orcid":false,"given":"Gustavo M.","family":"Callico","sequence":"additional","affiliation":[{"name":"Instituto Universitario de Microelectr\u00f3nica Aplicada (IUMA), Universidad de Las Palmas de Gran Canaria, 35017 Las Palmas de Gran Canaria, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3545-2328","authenticated-orcid":false,"given":"Juan","family":"Ruiz-Alzola","sequence":"additional","affiliation":[{"name":"Instituto Universitario de Investigaciones Biom\u00e9dicas y Sanitarias (IUIBS), Universidad de Las Palmas de Gran Canaria, 35016 Las Palmas de Gran Canaria, Spain"},{"name":"Grupo Tecnolog\u00eda M\u00e9dica IACTEC, Instituto de Astrof\u00edsica de Canarias (IAC), 38205 San Crist\u00f3bal de La Laguna, Spain"},{"name":"Departamento de Se\u00f1ales y Comunicaciones, Universidad de Las Palmas de Gran Canaria, 35016 Las Palmas de Gran Canaria, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,9]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"109119","DOI":"10.1016\/j.diabres.2021.109119","article-title":"IDF Diabetes Atlas: Global, regional and country-level diabetes prevalence estimates for 2021 and projections for 2045","volume":"183","author":"Sun","year":"2022","journal-title":"Diabetes Res. Clin. Pract."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"435","DOI":"10.1016\/j.pcd.2021.02.005","article-title":"A review on current advances in machine learning based diabetes prediction","volume":"15","author":"Jaiswal","year":"2021","journal-title":"Prim. Care Diabetes"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"326","DOI":"10.1016\/j.compbiomed.2017.10.030","article-title":"Computer aided diagnosis of diabetic foot using infrared thermography: A review","volume":"91","author":"Adam","year":"2017","journal-title":"Comput. Biol. Med."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1080\/07853890.2016.1231932","article-title":"Global epidemiology of diabetic foot ulceration: A systematic review and meta-analysis","volume":"49","author":"Zhang","year":"2017","journal-title":"Ann. Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"2367","DOI":"10.1056\/NEJMra1615439","article-title":"Diabetic foot ulcers and their recurrence","volume":"376","author":"Armstrong","year":"2017","journal-title":"New Engl. J. Med."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"5","DOI":"10.17925\/EE.2021.17.1.5","article-title":"The DFUC 2020 dataset: Analysis towards diabetic foot ulcer detection","volume":"17","author":"Cassidy","year":"2021","journal-title":"touchREVIEWS Endocrinol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"e3160","DOI":"10.1002\/dmrr.3160","article-title":"Global recurrence rates in diabetic foot ulcers: A systematic review and meta-analysis","volume":"35","author":"Fu","year":"2019","journal-title":"Diabetes\/Metabolism Res. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"9808295","DOI":"10.1155\/2018\/9808295","article-title":"Establishing differences in thermographic patterns between the various complications in diabetic foot disease","volume":"2018","author":"Gatt","year":"2018","journal-title":"Int. J. Endocrinol."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"79502","DOI":"10.1109\/ACCESS.2022.3194529","article-title":"A Survey of Wound Image Analysis Using Deep Learning: Classification, Detection, and Segmentation","volume":"10","author":"Zhang","year":"2022","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"687","DOI":"10.1089\/wound.2021.0091","article-title":"Image-based artificial intelligence in wound assessment: A systematic review","volume":"11","author":"Anisuzzaman","year":"2022","journal-title":"Adv. Wound Care"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1730","DOI":"10.1109\/JBHI.2018.2868656","article-title":"Robust methods for real-time diabetic foot ulcer detection and localization on mobile devices","volume":"23","author":"Goyal","year":"2018","journal-title":"IEEE J. Biomed. Health Informatics"},{"key":"ref_12","first-page":"1531","article-title":"Classification of Diabetic Foot Ulcers Using Class Knowledge Banks","volume":"9","author":"Xu","year":"2021","journal-title":"Front. Bioeng. Biotechnol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"105055","DOI":"10.1016\/j.compbiomed.2021.105055","article-title":"Diabetic foot ulcer classification using mapped binary patterns and convolutional neural networks","volume":"140","author":"Ebsim","year":"2022","journal-title":"Comput. Biol. Med."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"103616","DOI":"10.1016\/j.compbiomed.2020.103616","article-title":"Recognition of ischaemia and infection in diabetic foot ulcers: Dataset and techniques","volume":"117","author":"Goyal","year":"2020","journal-title":"Comput. Biol. Med."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Chemello, G., Salvatori, B., Morettini, M., and Tura, A. (2022). Artificial Intelligence Methodologies Applied to Technologies for Screening, Diagnosis and Care of the Diabetic Foot: A Narrative Review. Biosensors, 12.","DOI":"10.3390\/bios12110985"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Cruz-Vega, I., Hernandez-Contreras, D., Peregrina-Barreto, H., Rangel-Magdaleno, J.d.J., and Ramirez-Cortes, J.M. (2020). Deep learning classification for diabetic foot thermograms. Sensors, 20.","DOI":"10.3390\/s20061762"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"103219","DOI":"10.1016\/j.infrared.2020.103219","article-title":"Computer aided detection of diabetic foot ulcer using asymmetry analysis of texture and temperature features","volume":"105","author":"Saminathan","year":"2020","journal-title":"Infrared Phys. Technol."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Khandakar, A., Chowdhury, M.E., Reaz, M.B.I., Ali, S.H.M., Kiranyaz, S., Rahman, T., Chowdhury, M.H., Ayari, M.A., Alfkey, R., and Bakar, A.A.A. (2022). A Novel Machine Learning Approach for Severity Classification of Diabetic Foot Complications Using Thermogram Images. Sensors, 22.","DOI":"10.3390\/s22114249"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1419","DOI":"10.6061\/clinics\/2012(12)12","article-title":"Plantar thermography is useful in the early diagnosis of diabetic neuropathy","volume":"67","author":"Balbinot","year":"2012","journal-title":"Clinics"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"160","DOI":"10.1016\/j.infrared.2014.06.001","article-title":"Application of infrared thermography in computer aided diagnosis","volume":"66","author":"Faust","year":"2014","journal-title":"Infrared Phys. Technol."},{"key":"ref_21","first-page":"1157","article-title":"An introduction to variable and feature selection","volume":"3","author":"Guyon","year":"2003","journal-title":"J. Mach. Learn. Res."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1109\/TPAMI.2005.159","article-title":"Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy","volume":"27","author":"Peng","year":"2005","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Song, L., Smola, A., Gretton, A., Borgwardt, K.M., and Bedo, J. (2007, January 20\u201324). Supervised feature selection via dependence estimation. Proceedings of the 24th International Conference on Machine Learning, Corvalis, OR, USA.","DOI":"10.1145\/1273496.1273600"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, R., and Tang, K. (2009, January 6). Feature selection for maximizing the area under the ROC curve. Proceedings of the 2009 IEEE International Conference on Data Mining Workshops, Miami, FL, USA.","DOI":"10.1109\/ICDMW.2009.25"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"273","DOI":"10.1016\/S0004-3702(97)00043-X","article-title":"Wrappers for feature subset selection","volume":"97","author":"Kohavi","year":"1997","journal-title":"Artif. Intell."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"3273","DOI":"10.1016\/j.neucom.2010.04.003","article-title":"A new wrapper feature selection approach using neural network","volume":"73","author":"Kabir","year":"2010","journal-title":"Neurocomputing"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/j.2517-6161.1996.tb02080.x","article-title":"Regression shrinkage and selection via the lasso","volume":"58","author":"Tibshirani","year":"1996","journal-title":"J. R. Stat. Soc. Ser. (Methodol.)"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1111\/wrr.13019","article-title":"Further evidence that wound size and duration are strong prognostic markers of diabetic foot ulcer healing","volume":"40","author":"Margolis","year":"2022","journal-title":"Wound Repair Regen."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"101489","DOI":"10.1109\/ACCESS.2020.2997311","article-title":"COVID-19 future forecasting using supervised machine learning models","volume":"8","author":"Rustam","year":"2020","journal-title":"IEEE Access"},{"key":"ref_30","unstructured":"Kingma, D.P., Salimans, T., and Welling, M. (2015). Variational dropout and the local reparameterization trick. Advances in Neural Information Processing Systems 28 (NIPS 2015), MIT Press."},{"key":"ref_31","unstructured":"Gal, Y., Hron, J., and Kendall, A. (2017). Concrete dropout. Advances in Neural Information Processing Systems 30, MIT Press."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Wang, L. (2005). Support Vector Machines: Theory and Applications, Springer Science & Business Media.","DOI":"10.1007\/b95439"},{"key":"ref_33","first-page":"1929","article-title":"dropout: A simple way to prevent neural networks from overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_34","unstructured":"Kingma, D.P., and Welling, M. (2013). Auto-encoding variational bayes. arXiv."},{"key":"ref_35","unstructured":"Maddison, C.J., Mnih, A., and Teh, Y.W. (2016). The concrete distribution: A continuous relaxation of discrete random variables. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational inference: A review for statisticians","volume":"112","author":"Blei","year":"2017","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_37","unstructured":"Molchanov, D., Ashukha, A., and Vetrov, D. (2017, January 6\u201311). Variational dropout sparsifies deep neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Sydney, Australia."},{"key":"ref_38","unstructured":"Chang, C.H., Rampasek, L., and Goldenberg, A. (2017). dropout feature ranking for deep learning models. arXiv."},{"key":"ref_39","unstructured":"Louizos, C., Welling, M., and Kingma, D.P. (2017). Learning sparse neural networks through L_0 regularization. arXiv."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Yamada, Y., Lindenbaum, O., Negahban, S., and Kluger, Y. (2020, January 13\u201318). Feature selection using stochastic gates. Proceedings of the International Conference on Machine Learning, PMLR, Virtual Event.","DOI":"10.32614\/CRAN.package.Rstg"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"161296","DOI":"10.1109\/ACCESS.2019.2951356","article-title":"Plantar thermogram database for the study of diabetic foot complications","volume":"7","year":"2019","journal-title":"IEEE Access"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"493","DOI":"10.2147\/DMSO.S146121","article-title":"Decompression nerve surgery for diabetic neuropathy: A structured review of published clinical trials","volume":"11","author":"Albers","year":"2018","journal-title":"Diabetes Metab. Syndr. Obes. Targets Ther."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"103187","DOI":"10.1016\/j.infrared.2020.103187","article-title":"Automatic detection of risk zones in diabetic foot soles by processing thermographic images taken in an uncontrolled environment","volume":"105","author":"Maldonado","year":"2020","journal-title":"Infrared Phys. Technol."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1613\/jair.953","article-title":"SMOTE: Synthetic minority over-sampling technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J. Artif. Intell. Res."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Villa, E., Arteaga-Marrero, N., and Ruiz-Alzola, J. (2020). Performance assessment of low-cost thermal cameras for medical applications. Sensors, 20.","DOI":"10.3390\/s20051321"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Arteaga-Marrero, N., Bodson, L.C., Hern\u00e1ndez, A., Villa, E., and Ruiz-Alzola, J. (2021). Morphological Foot Model for Temperature Pattern Analysis Proposed for Diabetic Foot Disorders. Appl. Sci., 11.","DOI":"10.3390\/app11167396"},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"242","DOI":"10.1016\/j.infrared.2017.01.010","article-title":"A quantitative index for classification of plantar thermal changes in the diabetic foot","volume":"81","year":"2017","journal-title":"Infrared Phys. Technol."},{"key":"ref_48","first-page":"585306","article-title":"Quantitative estimation of temperature variations in plantar angiosomes: A study case for diabetic foot","volume":"2014","year":"2014","journal-title":"Comput. Math. Methods Med."},{"key":"ref_49","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/j.diabres.2005.12.012","article-title":"Relationship of skin temperature to sympathetic dysfunction in diabetic at-risk feet","volume":"73","author":"Sun","year":"2006","journal-title":"Diabetes Res. Clin. Pract."},{"key":"ref_50","unstructured":"Freedman, D., Pisani, R., Purves, R., and Adhikari, A. (2022, November 29). Statistics. Available online: https:\/\/www.wiley.com\/en-au\/Statistics%2C+4th+Edition+International+Student+Edition-p-9780393930436."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","article-title":"An introduction to ROC analysis","volume":"27","author":"Fawcett","year":"2006","journal-title":"Pattern Recognit. Lett."},{"key":"ref_52","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_53","unstructured":"S\u00f8nderby, C.K., Raiko, T., Maal\u00f8e, L., S\u00f8nderby, S.K., and Winther, O. (2016). How to train deep variational autoencoders and probabilistic ladder networks. arXiv."},{"key":"ref_54","first-page":"281","article-title":"Random search for hyper-parameter optimization","volume":"13","author":"Bergstra","year":"2012","journal-title":"J. Mach. Learn. Res."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"124373","DOI":"10.1109\/ACCESS.2022.3225107","article-title":"Performance Evaluation of Deep Learning Models for Image Classification Over Small Datasets: Diabetic Foot Case Study","volume":"10","author":"Fabelo","year":"2022","journal-title":"IEEE Access"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Filipe, V., Teixeira, P., and Teixeira, A. (2022). Automatic classification of foot thermograms using machine learning techniques. Algorithms, 15.","DOI":"10.3390\/a15070236"},{"key":"ref_57","unstructured":"Selle, J., Prakash, K.V., Sai, G.A., Vinod, B., and Chellappan, K. (2021, January 4\u20136). Classification of foot thermograms using texture features and support vector machine. Proceedings of the 2021 Second International Conference on Electronics and Sustainable Communication Systems (ICESC), Coimbatore, India."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"186","DOI":"10.1002\/dmrr.2746","article-title":"Classification of diabetic foot ulcers","volume":"32","author":"Game","year":"2016","journal-title":"Diabetes\/Metab. Res. Rev."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/757\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:04:35Z","timestamp":1760119475000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/757"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,9]]},"references-count":58,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["s23020757"],"URL":"https:\/\/doi.org\/10.3390\/s23020757","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,9]]}}}