{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,23]],"date-time":"2026-03-23T10:52:34Z","timestamp":1774263154141,"version":"3.50.1"},"reference-count":26,"publisher":"MDPI AG","issue":"10","license":[{"start":{"date-parts":[[2023,10,18]],"date-time":"2023-10-18T00:00:00Z","timestamp":1697587200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"LARSyS","award":["UIDB\/50009\/2020"],"award-info":[{"award-number":["UIDB\/50009\/2020"]}]},{"name":"LARSyS","award":["UID\/EEA\/50009\/2019"],"award-info":[{"award-number":["UID\/EEA\/50009\/2019"]}]},{"name":"LARSyS","award":["M1420-09-5369-FSE-000002"],"award-info":[{"award-number":["M1420-09-5369-FSE-000002"]}]},{"name":"Portuguese Foundation for Science and Technology (FCT)","award":["UIDB\/50009\/2020"],"award-info":[{"award-number":["UIDB\/50009\/2020"]}]},{"name":"Portuguese Foundation for Science and Technology (FCT)","award":["UID\/EEA\/50009\/2019"],"award-info":[{"award-number":["UID\/EEA\/50009\/2019"]}]},{"name":"Portuguese Foundation for Science and Technology (FCT)","award":["M1420-09-5369-FSE-000002"],"award-info":[{"award-number":["M1420-09-5369-FSE-000002"]}]},{"name":"ARDITI (Ag\u00eancia Regional para o Desenvolvimento da Investiga\u00e7\u00e3o, Tecnologia e Inova\u00e7\u00e3o)","award":["UIDB\/50009\/2020"],"award-info":[{"award-number":["UIDB\/50009\/2020"]}]},{"name":"ARDITI (Ag\u00eancia Regional para o Desenvolvimento da Investiga\u00e7\u00e3o, Tecnologia e Inova\u00e7\u00e3o)","award":["UID\/EEA\/50009\/2019"],"award-info":[{"award-number":["UID\/EEA\/50009\/2019"]}]},{"name":"ARDITI (Ag\u00eancia Regional para o Desenvolvimento da Investiga\u00e7\u00e3o, Tecnologia e Inova\u00e7\u00e3o)","award":["M1420-09-5369-FSE-000002"],"award-info":[{"award-number":["M1420-09-5369-FSE-000002"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>This study presents a novel approach for kernel selection based on Kullback\u2013Leibler divergence in variational autoencoders using features generated by the convolutional encoder. The proposed methodology focuses on identifying the most relevant subset of latent variables to reduce the model\u2019s parameters. Each latent variable is sampled from the distribution associated with a single kernel of the last encoder\u2019s convolutional layer, resulting in an individual distribution for each kernel. Relevant features are selected from the sampled latent variables to perform kernel selection, which filters out uninformative features and, consequently, unnecessary kernels. Both the proposed filter method and the sequential feature selection (standard wrapper method) were examined for feature selection. Particularly, the filter method evaluates the Kullback\u2013Leibler divergence between all kernels\u2019 distributions and hypothesizes that similar kernels can be discarded as they do not convey relevant information. This hypothesis was confirmed through the experiments performed on four standard datasets, where it was observed that the number of kernels can be reduced without meaningfully affecting the performance. This analysis was based on the accuracy of the model when the selected kernels fed a probabilistic classifier and the feature-based similarity index to appraise the quality of the reconstructed images when the variational autoencoder only uses the selected kernels. Therefore, the proposed methodology guides the reduction of the number of parameters of the model, making it suitable for developing applications for resource-constrained devices.<\/jats:p>","DOI":"10.3390\/info14100571","type":"journal-article","created":{"date-parts":[[2023,10,18]],"date-time":"2023-10-18T10:36:56Z","timestamp":1697625416000},"page":"571","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["On the Use of Kullback\u2013Leibler Divergence for Kernel Selection and Interpretation in Variational Autoencoders for Feature Creation"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5107-3248","authenticated-orcid":false,"given":"F\u00e1bio","family":"Mendon\u00e7a","sequence":"first","affiliation":[{"name":"Faculty of Exact Sciences and Engineering, University of Madeira, 9000-082 Funchal, Portugal"},{"name":"Interactive Technologies Institute (ITI\/LARSyS and ARDITI), 9020-105 Funchal, Portugal"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7677-0971","authenticated-orcid":false,"given":"Sheikh Shanawaz","family":"Mostafa","sequence":"additional","affiliation":[{"name":"Interactive Technologies Institute (ITI\/LARSyS and ARDITI), 9020-105 Funchal, Portugal"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7334-3993","authenticated-orcid":false,"given":"Fernando","family":"Morgado-Dias","sequence":"additional","affiliation":[{"name":"Faculty of Exact Sciences and Engineering, University of Madeira, 9000-082 Funchal, Portugal"},{"name":"Interactive Technologies Institute (ITI\/LARSyS and ARDITI), 9020-105 Funchal, Portugal"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8512-965X","authenticated-orcid":false,"given":"Antonio G.","family":"Ravelo-Garc\u00eda","sequence":"additional","affiliation":[{"name":"Interactive Technologies Institute (ITI\/LARSyS and ARDITI), 9020-105 Funchal, Portugal"},{"name":"Institute for Technological Development and Innovation in Communications, Universidad de Las Palmas de Gran Canaria, 35001 Las Palmas de Gran Canaria, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,10,18]]},"reference":[{"key":"ref_1","unstructured":"Kingma, D., and Welling, M. (2014, January 14\u201316). Auto-encoding variational bayes. Proceedings of the 2nd International Conference on Learning Representations, ICLR 2014, Banff, AB, Canada."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"307","DOI":"10.1561\/2200000056","article-title":"An Introduction to Variational Autoencoders","volume":"12","author":"Kingma","year":"2019","journal-title":"Found. Trends\u00ae Mach. Learn."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Ternes, L., Dane, M., Gross, S., Labrie, M., Mills, G., Gray, J., Heiser, L., and Chang, Y. (2022). A multi-encoder variational autoencoder controls multiple transformational features in single-cell image analysis. Commun. Biol., 5.","DOI":"10.1038\/s42003-022-03218-x"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"101938","DOI":"10.1016\/j.artmed.2020.101938","article-title":"GANs for medical image analysis","volume":"109","author":"Kazeminia","year":"2020","journal-title":"Artif. Intell. Med."},{"key":"ref_5","unstructured":"Rezende, D., Mohamed, S., and Wierstra, D. (2014, January 21\u201326). Stochastic backpropagation and approximate inference in deep generative models. Proceedings of the 31st International Conference on Machine Learning, Bejing, China."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"153651","DOI":"10.1109\/ACCESS.2020.3018151","article-title":"Variations in Variational Autoencoders\u2014A Comparative Evaluation","volume":"8","author":"Wei","year":"2020","journal-title":"IEEE Access"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Ashiquzzaman, A., Kim, S., Lee, D., Um, T., and Kim, J. (2019, January 11\u201313). Compacting Deep Neural Networks for Light Weight IoT & SCADA Based Applications with Node Pruning. Proceedings of the 2019 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Okinawa, Japan.","DOI":"10.1109\/ICAIIC.2019.8669031"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1186\/s13634-021-00744-4","article-title":"An efficient pruning scheme of deep neural networks for Internet of Things applications","volume":"2021","author":"Qi","year":"2021","journal-title":"EURASIP J. Adv. Signal Process."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Singh, A., and Ogunfunmi, T. (2022). An Overview of Variational Autoencoders for Source Separation, Finance, and Bio-Signal Applications. Entropy, 24.","DOI":"10.3390\/e24010055"},{"key":"ref_10","unstructured":"Yeung, S., Kannan, A., Dauphin, Y., and Fei-Fei, L. (2017). Tackling Over-pruning in Variational Autoencoders. arXiv."},{"key":"ref_11","unstructured":"Razavi, A., Oord, A.v.d., Poole, B., and Vinyals, O. (2019, January 6\u20139). Preventing Posterior Collapse with delta-VAEs. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_12","unstructured":"Tomczak, J., and Welling, M. (2018, January 9\u201311). VAE with a VampPrior. Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics, Playa Blanca, Spain."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"295","DOI":"10.1109\/TNNLS.2019.2900734","article-title":"Semisupervised Text Classification by Variational Autoencoder","volume":"31","author":"Xu","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Shen, D., Liu, T., Peters, T.M., Staib, L.H., Essert, C., Zhou, S., Yap, P.-T., and Khan, A. (2019, January 13\u201317). Variational AutoEncoder for Regression: Application to Brain Aging Analysis. Proceedings of the Medical Image Computing and Computer Assisted Intervention\u2014MICCAI 2019, Shenzhen, China.","DOI":"10.1007\/978-3-030-32248-9_98"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"2131056","DOI":"10.1080\/08839514.2022.2131056","article-title":"Variational Autoencoder for Classification and Regression for Out-of-Distribution Detection in Learning-Enabled Cyber-Physical Systems","volume":"36","author":"Koutsoukos","year":"2022","journal-title":"Appl. Artif. Intell."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"108827","DOI":"10.1016\/j.knosys.2022.108827","article-title":"Application of domain-adaptive convolutional variational autoencoder for stress-state prediction","volume":"248","author":"Lee","year":"2022","journal-title":"Knowl.-Based Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"859","DOI":"10.1080\/01621459.2017.1285773","article-title":"Variational Inference: A Review for Statisticians","volume":"112","author":"Blei","year":"2016","journal-title":"J. Am. Stat. Assoc."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Nagi, J., Ducatelle, F., Caro, G., Cire\u015fan, D., Meier, U., Giusti, A., Nagi, F., Schmidhuber, J., and Gambardella, L. (2011, January 16\u201318). Max-pooling convolutional neural networks for vision-based hand gesture recognition. Proceedings of the 2011 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), Kuala Lumpur, Malaysia.","DOI":"10.1109\/ICSIPA.2011.6144164"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Noh, H., Hong, S., and Han, B. (2015, January 7\u201313). Learning deconvolution network for semantic segmentation. Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), Santiago, Chile.","DOI":"10.1109\/ICCV.2015.178"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"15711","DOI":"10.1007\/s00521-018-3455-8","article-title":"Comparison of SFS and mRMR for oximetry feature selection in obstructive sleep apnea detection","volume":"32","author":"Mostafa","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"8438","DOI":"10.1039\/C9SC01992H","article-title":"A Bayesian graph convolutional network for reliable prediction of molecular properties with uncertainty quantification","volume":"36","author":"Ryu","year":"2019","journal-title":"Chem. Sci."},{"key":"ref_22","unstructured":"Wen, Y., Vicol, P., Ba, J., Tran, D., and Grosse, R. (May, January 30). Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batches. Proceedings of the Sixth International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"2378","DOI":"10.1109\/TIP.2011.2109730","article-title":"FSIM: A Feature Similarity Index for Image Quality Assessment","volume":"20","author":"Zhang","year":"2011","journal-title":"IEEE Trans. Image Process."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1419","DOI":"10.1109\/TMI.2019.2947595","article-title":"A 3D Probabilistic Deep Learning System for Detection and Diagnosis of Lung Cancer Using Low-Dose CT Scans","volume":"39","author":"Ozdemir","year":"2019","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Amroun, H., Temkit, M., and Ammi, M. (2017, January 21\u201323). Best Feature for CNN Classification of Human Activity Using IOT Network. Proceedings of the IEEE\/ACM International Conference on and International Conference on Cyber, Physical and Social Computing (CPSCom) Green Computing and Communications (GreenCom) Exeter, Devon, UK.","DOI":"10.1109\/iThings-GreenCom-CPSCom-SmartData.2017.145"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"11073","DOI":"10.3390\/app112211073","article-title":"Hardware\/Software Co-Design for TinyML Voice-Recognition Application on Resource Frugal Edge Devices","volume":"11","author":"Know","year":"2021","journal-title":"Appl. Sci."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/14\/10\/571\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:09:20Z","timestamp":1760130560000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/14\/10\/571"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,18]]},"references-count":26,"journal-issue":{"issue":"10","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["info14100571"],"URL":"https:\/\/doi.org\/10.3390\/info14100571","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,18]]}}}