{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,7]],"date-time":"2026-03-07T18:35:36Z","timestamp":1772908536775,"version":"3.50.1"},"reference-count":20,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2019,2,18]],"date-time":"2019-02-18T00:00:00Z","timestamp":1550448000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002850","name":"Fondo Nacional de Desarrollo Cient\u00edfico y Tecnol\u00f3gico","doi-asserted-by":"publisher","award":["11140892"],"award-info":[{"award-number":["11140892"]}],"id":[{"id":"10.13039\/501100002850","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Today, there is growing interest in the automatic classification of a variety of tasks, such as weather forecasting, product recommendations, intrusion detection, and people recognition. \u201cMixture-of-experts\u201d is a well-known classification technique; it is a probabilistic model consisting of local expert classifiers weighted by a gate network that is typically based on softmax functions, combined with learnable complex patterns in data. In this scheme, one data point is influenced by only one expert; as a result, the training process can be misguided in real datasets for which complex data need to be explained by multiple experts. In this work, we propose a variant of the regular mixture-of-experts model. In the proposed model, the cost classification is penalized by the Shannon entropy of the gating network in order to avoid a \u201cwinner-takes-all\u201d output for the gating network. Experiments show the advantage of our approach using several real datasets, with improvements in mean accuracy of 3\u20136% in some datasets. In future work, we plan to embed feature selection into this model.<\/jats:p>","DOI":"10.3390\/e21020190","type":"journal-article","created":{"date-parts":[[2019,2,19]],"date-time":"2019-02-19T04:08:20Z","timestamp":1550549300000},"page":"190","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":7,"title":["Mixture of Experts with Entropic Regularization for Data Classification"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5457-2157","authenticated-orcid":false,"given":"Billy","family":"Peralta","sequence":"first","affiliation":[{"name":"Department of Engineering Science, Andres Bello University, Santiago 7500971, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ariel","family":"Saavedra","sequence":"additional","affiliation":[{"name":"Department of Engineering Informatics, Catholic University of Temuco, Temuco 4781312, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luis","family":"Caro","sequence":"additional","affiliation":[{"name":"Department of Engineering Informatics, Catholic University of Temuco, Temuco 4781312, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alvaro","family":"Soto","sequence":"additional","affiliation":[{"name":"Department of Computer Sciences, Pontifical Catholic University of Chile, Santiago 7820436, Chile"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2019,2,18]]},"reference":[{"key":"ref_1","unstructured":"Bishop, C.M. (2006). Pattern Recognition and Machine Learning, Springer."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1826","DOI":"10.1073\/pnas.0808843106","article-title":"Scoring diverse cellular morphologies in image-based screens with iterative feedback and machine learning","volume":"106","author":"Jones","year":"2009","journal-title":"Proc. Natl. Acad. Sci. USA"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/360402.360406","article-title":"Web Mining Research: A Survey","volume":"2","author":"Kosala","year":"2000","journal-title":"SIGKDD Explor. Newslett."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1186\/s40537-015-0029-9","article-title":"Survey of review spam detection using machine learning techniques","volume":"2","author":"Crawford","year":"2015","journal-title":"J. Big Data"},{"key":"ref_5","unstructured":"Pazzani, M.J., Billsus, D., Kobsa, A., and Nejdl, W. (2007). Content-Based Recommendation Systems. The Adaptive Web: Methods and Strategies of Web Personalization, Springer."},{"key":"ref_6","unstructured":"Jacobs, R., and Jordan, M. (1991). Adaptive Mixture of Local Experts, Department of Brain and Cognitive Science, Massachusetts Institute of Technology."},{"key":"ref_7","first-page":"9","article-title":"A Mixture-of-experts Framework for Text Classification","volume":"Volume 71","author":"Estabrooks","year":"2001","journal-title":"Proceedings of the 2001 Workshop on Computational Natural Language Learning"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"804","DOI":"10.1016\/j.ijforecast.2010.02.015","article-title":"Mixture of MLP-experts for trend forecasting of time series: A case study of the Tehran stock exchange","volume":"27","author":"Ebrahimpour","year":"2011","journal-title":"Int. J. Forecast."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Gupta, R., Audhkhasi, K., and Narayanan, S. (2015, January 19\u201324). A mixture of experts approach towards intelligibility classification of pathological speech. Proceedings of the 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2015), South Brisbane, Australia.","DOI":"10.1109\/ICASSP.2015.7178318"},{"key":"ref_10","unstructured":"Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., and Dean, J. (2017, January 24\u201326). Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer. Proceedings of the ICLR Conference, Toulon, France."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1351","DOI":"10.1016\/j.eswa.2009.06.083","article-title":"Support vector machine based multiagent ensemble learning for credit risk evaluation","volume":"37","author":"Yu","year":"2010","journal-title":"Expert Syst. Appl."},{"key":"ref_12","unstructured":"Arbib, M.A. (1995). Winner-Take-All Mechanisms. Handbook of Brain Theory and Neural Networks, MIT Press."},{"key":"ref_13","unstructured":"Shashanka, M., Raj, B., and Smaragdis, P. (2007). Probabilistic Latent Variable Models as Non-Negative Factorizations, MERL-Mitsubishi Electric Research Laboratories. Technical Report TR2007-083."},{"key":"ref_14","unstructured":"Grandvalet, Y., and Bengio, Y. (2005). Semi-supervised Learning by Entropy Minimization. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Yang, M., and Chen, L. (2017, January 4\u20139). Discriminative Semi-Supervised Dictionary Learning with Entropy Regularization for Pattern Classification. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10736"},{"key":"ref_16","unstructured":"Moerland, P. (1997). Some Methods for Training Mixtures of Experts, IDIAP Research Institute. Technical Report."},{"key":"ref_17","unstructured":"Jordan, M.I., and Xu, L. (1993). Convergence Results for the EM Approach to Mixtures of Experts Architectures, Department of Brain and Cognitive Science, Massachusetts Institute of Technology."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1016\/j.ins.2014.01.008","article-title":"Embedded local feature selection within mixture of experts","volume":"269","author":"Peralta","year":"2014","journal-title":"Inf. Sci."},{"key":"ref_19","unstructured":"Arbib, M.A. (1995). The Handbook of Brain Theory and Neural Networks, MIT Press. [1st ed.]."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Bay, S., Kibler, D., Pazzani, M., and Smyth, P. (2000). The UCI KDD Archive of Large Data Sets for Data Mining Research and Experimentation, Department of Information and Computer Science University of California.","DOI":"10.1145\/380995.381030"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/2\/190\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T12:32:55Z","timestamp":1760185975000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/21\/2\/190"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,2,18]]},"references-count":20,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2019,2]]}},"alternative-id":["e21020190"],"URL":"https:\/\/doi.org\/10.3390\/e21020190","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,2,18]]}}}