{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:44:47Z","timestamp":1760147087833,"version":"build-2065373602"},"reference-count":32,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2023,1,12]],"date-time":"2023-01-12T00:00:00Z","timestamp":1673481600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Kernel methods have played a major role in the last two decades in the modeling and visualization of complex problems in data science. The choice of kernel function remains an open research area and the reasons why some kernels perform better than others are not yet understood. Moreover, the high computational costs of kernel-based methods make it extremely inefficient to use standard model selection methods, such as cross-validation, creating a need for careful kernel design and parameter choice. These reasons justify the prior analyses of kernel matrices, i.e., mathematical objects generated by the kernel functions. This paper explores these topics from an entropic standpoint for the case of kernelized relevance vector machines (RVMs), pinpointing desirable properties of kernel matrices that increase the likelihood of obtaining good model performances in terms of generalization power, as well as relate these properties to the model\u2019s fitting ability. We also derive a heuristic for achieving close-to-optimal modeling results while keeping the computational costs low, thus providing a recipe for efficient analysis when processing resources are limited.<\/jats:p>","DOI":"10.3390\/e25010154","type":"journal-article","created":{"date-parts":[[2023,1,12]],"date-time":"2023-01-12T06:25:57Z","timestamp":1673504757000},"page":"154","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Analysis of Kernel Matrices via the von Neumann Entropy and Its Relation to RVM Performances"],"prefix":"10.3390","volume":"25","author":[{"given":"Llu\u00eds A.","family":"Belanche-Mu\u00f1oz","sequence":"first","affiliation":[{"name":"Department of Computer Science, Universitat Polit\u00e8cnica de Catalunya, 08034 Barcelona, Catalonia, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ma\u0142gorzata","family":"Wiejacha","sequence":"additional","affiliation":[{"name":"Cien.ai, Ronda Carrer de Sagu\u00e9s, 45, 08021 Barcelona, Catalonia, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1023\/A:1012450327387","article-title":"Choosing Multiple Parameters for Support Vector Machines","volume":"46","author":"Chapelle","year":"2002","journal-title":"Mach. Learn."},{"key":"ref_2","unstructured":"Tipping, M.E. (2000). The Relevance Vector Machine. Advances in Neural Information Processing Systems 12, Curran Associates, Inc."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"2526","DOI":"10.1016\/j.neucom.2010.11.037","article-title":"Parameter-insensitive kernel in extreme learning for non-linear support vector regression","volume":"74","author":"Verleysen","year":"2011","journal-title":"Neurocomputing"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"637","DOI":"10.1016\/S0893-6080(98)00032-X","article-title":"The connection between regularization operators and support vector kernels","volume":"11","author":"Smola","year":"1998","journal-title":"Neural Netw."},{"key":"ref_5","first-page":"1043","article-title":"Learning the Kernel with Hyperkernels","volume":"6","author":"Ong","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Cristianini, N., Kandola, J., Elisseeff, A., and Shawe-Taylor, J. (2002). On kernel-target alignment. Advances in Neural Information Processing Systems 14, Curran Associates, Inc.","DOI":"10.7551\/mitpress\/1120.003.0052"},{"key":"ref_7","unstructured":"Nguyen, C.H., and Ho, T.B. (2007, January 6\u201312). Kernel matrix evaluation. Proceedings of the IJCAI International Joint Conference on Artificial Intelligence, Hyderabad, India."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1023\/B:MACH.0000015879.28004.9b","article-title":"A meta-learning method to select the kernel width in support vector regression","volume":"54","author":"Soares","year":"2004","journal-title":"Mach. Learn."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Soares, C., and Brazdil, P.B. (2006, January 23\u201327). Selecting parameters of SVM using meta-learning and kernel matrix-based meta-features. Proceedings of the 2006 ACM Symposium on Applied Computing\u2014SAC\u201906, Dijon, France.","DOI":"10.1145\/1141277.1141408"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1016\/j.neucom.2006.03.004","article-title":"A meta-learning approach to automatic kernel selection for support vector machines","volume":"70","author":"Ali","year":"2006","journal-title":"Neurocomputing"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1016\/j.neucom.2011.07.005","article-title":"Combining meta-learning and search techniques to select parameters for support vector machines","volume":"75","author":"Gomes","year":"2012","journal-title":"Neurocomputing"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"41","DOI":"10.1016\/S0925-2312(02)00601-X","article-title":"Evaluation of simple performance measures for tuning SVM hyper parameters. Technical report","volume":"51","author":"Duan","year":"2001","journal-title":"Neurocomputing"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"i326","DOI":"10.1093\/bioinformatics\/bth906","article-title":"Learning kernels from biological networks by maximizing entropy","volume":"20","author":"Tsuda","year":"2004","journal-title":"Bioinformatics"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Nielsen, M.A., and Chuang, I.L. (2011). Quantum Computation and Quantum Information: 10th Anniversary Edition, Cambridge University Press.","DOI":"10.1017\/CBO9780511976667"},{"key":"ref_15","unstructured":"Wehrl, A. (2001). Entropy, the von Neumann and the von Neumann entropy. John Von Neumann, Springer."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Redei, M., and St\u00f6ltzner, M. (2001). Entropy, von Neumann and the von Neumann Entropy. John von Neumann and the Foundation of Quantum Physics, Springer.","DOI":"10.1007\/978-94-017-2012-0"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1016\/0169-7439(87)80084-9","article-title":"Principal Component Analysis","volume":"2","author":"Wold","year":"1987","journal-title":"Chemom. Intell. Lab. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2380","DOI":"10.1103\/PhysRevB.41.2380","article-title":"Maximum-entropy method for analytic continuation of quantum Monte Carlo data","volume":"41","author":"Silver","year":"1990","journal-title":"Phys. Rev. B"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Jenssen, R., Eltoft, T., Girolami, M., and Erdogmus, D. (2007). Kernel Maximum Entropy Data Transformation and an Enhanced Spectral Clustering Algorithm. Advances in Neural Information Processing Systems 19, Curran Associates, Inc.","DOI":"10.7551\/mitpress\/7503.003.0084"},{"key":"ref_20","first-page":"129","article-title":"Diffusion Kernels on Statistical Manifolds","volume":"6","author":"Lafferty","year":"2005","journal-title":"J. Mach. Learn. Res."},{"key":"ref_21","unstructured":"Lafferty, J., and Kondor, R.I. Diffusion Kernels on Graphs and Other Discrete Input Spaces. Proceedings of the ICML\u201902, Nineteenth International Conference on Machine Learning, Sydney, Australia, 8\u201312 July 2002."},{"key":"ref_22","unstructured":"Malossini, A., Segata, N., and Blanzieri, E. (2009). Kernel Integration Using Von Neumann Entropy, University of Trento."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Trajkovski, G. (2011). Quantifying Disorder in Networks: The von Neumann Entropy. Developments in Intelligent Agent Technologies and Multi-Agent Systems: Concepts and Applications, IGI Global.","DOI":"10.4018\/978-1-60960-171-3"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"9610826","DOI":"10.1155\/2019\/9610826","article-title":"Spectral Complexity of Directed Graphs and Application to Structural Decomposition","volume":"2019","author":"Fonoberov","year":"2019","journal-title":"Complexity"},{"key":"ref_25","first-page":"277","article-title":"Normalization in Support Vector Machines","volume":"13","author":"Graf","year":"2001","journal-title":"Neural Comput."},{"key":"ref_26","unstructured":"Smola, A., Ovari, Z., and Williamson, R. (2001). Regularization with dot-product kernels. Advances in Neural Information Processing Systems 13, Curran Associates, Inc."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"78","DOI":"10.1090\/S0002-9939-1965-0171902-8","article-title":"Continuity and location of zeros of linear combinations of polynomials","volume":"16","author":"Zedek","year":"1965","journal-title":"Proc. Am. Math. Soc."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1093\/qjmam\/1.1.287","article-title":"Rounding-off errors in matrix processes","volume":"1","author":"Turing","year":"1948","journal-title":"Q. J. Mech. Appl. Math."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1021","DOI":"10.1090\/S0002-9904-1947-08909-6","article-title":"Numerical inverting of matrices of high order","volume":"53","author":"Goldstine","year":"1947","journal-title":"Bull. Am. Math. Soc."},{"key":"ref_30","unstructured":"Diniz, F.B. (2017). Condition number and matrices. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1090\/S0002-9939-1953-0055639-3","article-title":"Sequential Minimax Search for a Maximum","volume":"4","author":"Kiefer","year":"1953","journal-title":"Proc. Am. Math. Soc."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1080\/00150517.1966.12431364","article-title":"Optimally proof for the symmetric fibonacci search technique","volume":"4","author":"Avriel","year":"1966","journal-title":"Fibonacci Q."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/1\/154\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:04:20Z","timestamp":1760119460000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/25\/1\/154"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,12]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["e25010154"],"URL":"https:\/\/doi.org\/10.3390\/e25010154","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2023,1,12]]}}}