{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T09:08:59Z","timestamp":1777626539069,"version":"3.51.4"},"reference-count":67,"publisher":"MIT Press","issue":"5","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Neural Computation"],"published-print":{"date-parts":[[2011,5]]},"abstract":"<jats:p>Supervised and unsupervised vector quantization methods for classification and clustering traditionally use dissimilarities, frequently taken as Euclidean distances. In this article, we investigate the applicability of divergences instead, focusing on online learning. We deduce the mathematical fundamentals for its utilization in gradient-based online vector quantization algorithms. It bears on the generalized derivatives of the divergences known as Fr\u00e9chet derivatives in functional analysis, which reduces in finite-dimensional problems to partial derivatives in a natural way. We demonstrate the application of this methodology for widely applied supervised and unsupervised online vector quantization schemes, including self-organizing maps, neural gas, and learning vector quantization. Additionally, principles for hyperparameter optimization and relevance learning for parameterized divergences in the case of supervised vector quantization are given to achieve improved classification accuracy.<\/jats:p>","DOI":"10.1162\/neco_a_00110","type":"journal-article","created":{"date-parts":[[2011,2,7]],"date-time":"2011-02-07T22:27:58Z","timestamp":1297117678000},"page":"1343-1392","source":"Crossref","is-referenced-by-count":61,"title":["Divergence-Based Vector Quantization"],"prefix":"10.1162","volume":"23","author":[{"given":"Thomas","family":"Villmann","sequence":"first","affiliation":[{"name":"University of Applied Sciences Mittweida, Department of Mathematics, Natural and Computer Sciences, 09648 Mittweida, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sven","family":"Haase","sequence":"additional","affiliation":[{"name":"University of Applied Sciences Mittweida, Department of Mathematics, Natural and Computer Sciences, 09648 Mittweida, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2008.12.026"},{"key":"B2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-5056-2"},{"key":"B3","volume-title":"Methods of information geometry","author":"Amari S.-I.","year":"2000"},{"key":"B4","first-page":"1705","volume":"6","author":"Banerjee A.","year":"2005","journal-title":"Journal of Machine Learning Research"},{"key":"B5","volume-title":"Distance measures for signal processing and pattern recognition","author":"Basseville M","year":"1988"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/85.3.549"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2009.4959891"},{"key":"B8","doi-asserted-by":"publisher","DOI":"10.1162\/089976698300017953"},{"key":"B9","doi-asserted-by":"publisher","DOI":"10.1016\/0041-5553(67)90040-7"},{"key":"B10","first-page":"87","volume-title":"Proc. of European Symposium on Artificial Neural Networks","author":"Bunte K.","year":"2010"},{"key":"B11","doi-asserted-by":"publisher","DOI":"10.3390\/e12061532"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2008.02.016"},{"key":"B13","doi-asserted-by":"publisher","DOI":"10.1002\/9780470747278"},{"key":"B14","first-page":"462","volume-title":"Advances in neural information processing systems","volume":"15","author":"Crammer K.","year":"2002"},{"key":"B15","first-page":"299","volume":"2","author":"Csisz\u00e1r I.","year":"1967","journal-title":"Studia Sci. Math. Hungaria"},{"key":"B16","volume-title":"Robustifying maximum likelihood estimation","author":"Eguchi S.","year":"2001"},{"key":"B18","volume-title":"Differential- und Integralrechnung","author":"Fichtenholz G.","year":"1964"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2008.929943"},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.1016\/j.jmva.2008.02.004"},{"key":"B22","doi-asserted-by":"publisher","DOI":"10.1016\/S0925-2312(98)00035-6"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00079-5"},{"key":"B24","unstructured":"Hegde, A., Erdogmus, D., Lehn-Schi\u00f8ler, T., Rao, Y. & Principe, J. (2004). Vector quantization by density matching in the minimum Kullback-Leibler-divergence sense. InProc. of the International Joint Conference on Artificial Neural Networks(pp. 105\u2013109). Piscataway, NJ: IEEE Press."},{"key":"B25","doi-asserted-by":"publisher","DOI":"10.1016\/B978-044450270-4\/50024-3"},{"key":"B26","volume-title":"Faithful representations and topographic maps","author":"Hulle M.M.V.","year":"2000"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1162\/089976602760128054"},{"key":"B28","doi-asserted-by":"publisher","DOI":"10.1016\/S0893-6080(02)00077-1"},{"key":"B29","first-page":"289","volume-title":"Speech synthesis","author":"Itakura F.","year":"1973"},{"key":"B30","unstructured":"Jang, E., Fyfe, C. & Ko, H. (2008). Bregman divergences and the self organising map. In C. Fyfe, D. Kim, S.Y., Lee, & H. Yin (Eds.), Intelligent data engineering and automated learning (pp. 452\u2013458). New York: Springer."},{"key":"B32","doi-asserted-by":"publisher","DOI":"10.1016\/j.jfranklin.2006.03.018"},{"key":"B33","doi-asserted-by":"crossref","DOI":"10.1515\/9783112719978","volume-title":"Funktionalanalysis in normierten R\u00e4umen","author":"Kantorowitsch I.","year":"1978"},{"key":"B34","volume-title":"Measures of information and their application","author":"Kapur J.","year":"1994"},{"key":"B35","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-97966-8"},{"key":"B36","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729694"},{"key":"B37","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-03040-6_114"},{"key":"B38","volume-title":"Proc. of Workshop on Self-Organizing Maps","author":"Lee J.","year":"2005"},{"key":"B39","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-39351-3"},{"key":"B40","doi-asserted-by":"publisher","DOI":"10.1007\/s11047-004-9619-8"},{"key":"B41","volume-title":"Convex statistical distances","author":"Liese F.","year":"1987"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2006.881731"},{"key":"B43","doi-asserted-by":"publisher","DOI":"10.1109\/TCOM.1980.1094577"},{"key":"B44","first-page":"2579","volume":"9","author":"Maaten L.","year":"2008","journal-title":"Journal of Machine Learning Research"},{"key":"B45","doi-asserted-by":"publisher","DOI":"10.1109\/72.238311"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.1162\/089976602760128045"},{"key":"B47","volume-title":"Divergence measures and message passing","author":"Minka T.","year":"2005"},{"key":"B48","volume-title":"Proc. of European Symposium on Artificial Neural Networks","author":"Mwebaze E.","year":"2010"},{"key":"B49","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2009.2018176"},{"key":"B50","volume-title":"Unsupervised adaptive filtering","author":"Principe J. C. III","year":"2000"},{"key":"B51","volume-title":"INTERSPEECH\u2014Proc. of the Annual Conference of the International Speech Communication Association (pp. 1349\u20131352)","author":"Qiao Y.","year":"2008"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.1002\/0471667196.ess3138"},{"key":"B53","volume-title":"Proceedings of the Fourth Berkeley Symposium on Mathematical Statistics and Probability","author":"Renyi A.","year":"1961"},{"key":"B54","volume-title":"Probability theory","author":"Renyi A.","year":"1970"},{"key":"B55","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2004.11.012"},{"key":"B56","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-009-5132-8"},{"key":"B57","volume-title":"Advances in neural information processing systems","volume":"8","author":"Sato A.","year":"1996"},{"key":"B58","volume-title":"Proceedings of the European Symposium on Artificial Neural Networks","author":"Schneider P.","year":"2009"},{"key":"B59","doi-asserted-by":"publisher","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"B60","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2003.11.002"},{"key":"B61","doi-asserted-by":"publisher","DOI":"10.1162\/153244303322753742"},{"key":"B62","volume-title":"Proc. of the International Conference on Machine Learning","author":"Torkkola K.","year":"2000"},{"key":"B64","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-13208-7_60"},{"key":"B65","first-page":"8","volume":"2010","author":"Villmann T.","year":"2010","journal-title":"Ulmer Informatik-Berichte"},{"key":"B66","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2006.05.026"},{"key":"B67","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2008.04.048"},{"key":"B68","doi-asserted-by":"publisher","DOI":"10.1109\/WHISPERS.2009.5289064"},{"key":"B69","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbn009"},{"key":"B70","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-02397-2_39"},{"key":"B71","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.1982.1056490"}],"container-title":["Neural Computation"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mitpressjournals.org\/doi\/pdf\/10.1162\/NECO_a_00110","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,3,2]],"date-time":"2025-03-02T12:34:50Z","timestamp":1740918890000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/neco\/article\/23\/5\/1343-1392\/7660"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,5]]},"references-count":67,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2011,5]]}},"alternative-id":["10.1162\/NECO_a_00110"],"URL":"https:\/\/doi.org\/10.1162\/neco_a_00110","relation":{},"ISSN":["0899-7667","1530-888X"],"issn-type":[{"value":"0899-7667","type":"print"},{"value":"1530-888X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,5]]}}}