{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T07:27:44Z","timestamp":1762327664222,"version":"build-2065373602"},"reference-count":64,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T00:00:00Z","timestamp":1762300800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>Datasets often incorporate various functional patterns related to different aspects or regimes, which are typically not equally present throughout the dataset. We propose a novel partitioning algorithm that utilizes competition between models to detect and separate these functional patterns. This competition is induced by multiple models iteratively submitting their predictions for the dataset, with the best prediction for each data point being rewarded with training on that data point. This reward mechanism amplifies each model's strengths and encourages specialization in different patterns. The specializations can then be translated into a partitioning scheme. We validate our concept with datasets with clearly distinct functional patterns, such as mechanical stress and strain data in a porous structure. Our partitioning algorithm produces valuable insights into the datasets' structure, which can serve various further applications. As a demonstration of one exemplary usage, we set up modular models consisting of multiple expert models, each learning a single partition, and compare their performance on more than twenty popular regression problems with single models learning all partitions simultaneously. Our results show significant improvements, with up to 56% loss reduction, confirming our algorithm's utility.<\/jats:p>","DOI":"10.3389\/frai.2025.1661444","type":"journal-article","created":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T06:28:42Z","timestamp":1762324122000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Functional partitioning through competitive learning"],"prefix":"10.3389","volume":"8","author":[{"given":"Marius","family":"Tacke","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthias","family":"Busch","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kevin","family":"Linka","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Christian","family":"Cyron","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Roland","family":"Aydin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2025,11,5]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","DOI":"10.1201\/b15410","author":"Aggarwal","year":"2013","journal-title":"Data Clustering: Algorithms and Applications"},{"key":"B2","doi-asserted-by":"publisher","first-page":"2001428","DOI":"10.1002\/adem.202001428","article-title":"Atomic scale structure inspired 3D-printed porous structures with tunable mechanical response","volume":"23","author":"Ambekar","year":"2021","journal-title":"Adv. Eng. Mater"},{"key":"B3","doi-asserted-by":"publisher","first-page":"1309","DOI":"10.1109\/72.536325","article-title":"A modified hme architecture for text-dependent speaker identification","volume":"7","author":"Chen","year":"1996","journal-title":"IEEE Trans. Neural Netw"},{"key":"B4","doi-asserted-by":"publisher","first-page":"1229","DOI":"10.1016\/S0893-6080(99)00043-X","article-title":"Improved learning algorithms for mixture of experts in multiclass classification","volume":"12","author":"Chen","year":"1999","journal-title":"Neural Netw"},{"volume-title":"Beijing PM2.5","year":"2017","author":"Chen","key":"B5"},{"key":"B6","article-title":"\u201cWinner-takes-all for multivariate probabilistic time series forecasting,\u201d","volume-title":"ICML 2025: The 42nd International Conference on Machine Learning","author":"Cort\u00e9s","year":"2025"},{"year":"2014","author":"Cortez","journal-title":"Student Performance","key":"B7"},{"year":"2009","author":"Cortez","journal-title":"Wine Quality","key":"B8"},{"year":"2008","author":"Cortez","journal-title":"Forest Fires.","key":"B9"},{"key":"B10","article-title":"Sparse mixture of experts as unified competitive learning","author":"Do","year":"2025","journal-title":"arXiv preprint arXiv:2503.22996"},{"key":"B11","doi-asserted-by":"publisher","first-page":"89","DOI":"10.1016\/j.neunet.2009.08.007","article-title":"Clustering: a neural network approach","volume":"23","author":"Du","year":"2010","journal-title":"Neural Netw"},{"key":"B12","doi-asserted-by":"publisher","first-page":"95","DOI":"10.1080\/01969727408546059","article-title":"Well-separated clusters and optimal fuzzy partitions","volume":"4","author":"Dunn","year":"1974","journal-title":"J. Cybern"},{"key":"B13","article-title":"Learning factored representations in a deep mixture of experts","author":"Eigen","year":"2013","journal-title":"arXiv preprint arXiv:1312.4314"},{"key":"B14","doi-asserted-by":"publisher","first-page":"6247","DOI":"10.1007\/s00521-020-05395-4","article-title":"Automatic clustering algorithms: a systematic review and bibliometric analysis of relevant literature","volume":"33","author":"Ezugwu","year":"2021","journal-title":"Neural Comput. Applic"},{"unstructured":"Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity\n          \n          1\n          39\n          \n            \n              Fedus\n              W.\n            \n            \n              Zoph\n              B.\n            \n            \n              Shazeer\n              N.\n            \n          \n          J. Mach. Learn. Res\n          23\n          2022","key":"B15"},{"year":"1987","author":"Feldmesser","journal-title":"Computer Hardware","key":"B16"},{"year":"2015","author":"Fernandes","journal-title":"Online News Popularity","key":"B17"},{"key":"B18","article-title":"Learning mixtures of experts with em: a mirror descent perspective","author":"Fruytier","year":"2024","journal-title":"arXiv preprint arXiv:2411.06056"},{"key":"B19","doi-asserted-by":"crossref","first-page":"1085","DOI":"10.1109\/IJCNN.2008.4633934","article-title":"\u201cSelf-splitting modular neural network-domain partitioning at boundaries of trained regions,\u201d","volume-title":"2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence)","author":"Gordon","year":"2008"},{"key":"B20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3757737","article-title":"Came: competitively learning a mixture-of-experts model for first-stage retrieval","volume":"43","author":"Guo","year":"2025","journal-title":"ACM Trans. Inf. Syst"},{"year":"2018","author":"Hamidieh","journal-title":"Superconductivty Data","key":"B21"},{"year":"2020","author":"Imran","journal-title":"Productivity Prediction of Garment Employees","key":"B22"},{"key":"B23","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1162\/neco.1991.3.1.79","article-title":"Adaptive mixtures of local experts","volume":"3","author":"Jacobs","year":"1991","journal-title":"Neural Comput"},{"key":"B24","doi-asserted-by":"publisher","first-page":"651","DOI":"10.1016\/j.patrec.2009.09.011","article-title":"Data clustering: 50 years beyond k-means","volume":"31","author":"Jain","year":"2010","journal-title":"Pattern Recognit. Lett"},{"year":"1988","author":"Janosi","journal-title":"Heart Disease","key":"B25"},{"key":"B26","doi-asserted-by":"publisher","first-page":"181","DOI":"10.1162\/neco.1994.6.2.181","article-title":"Hierarchical mixtures of experts and the em algorithm","volume":"6","author":"Jordan","year":"1994","journal-title":"Neural Comput"},{"key":"B27","article-title":"Mixture of experts provably detect and learn the latent cluster structure in gradient-based learning","author":"Kawata","year":"2025","journal-title":"arXiv preprint arXiv:2506.01656"},{"unstructured":"Kelly\n              M.\n            \n            \n              Longjohn\n              R.\n            \n            \n              Nottingham\n              K.\n            \n          \n          Uci Machine Learning Repository\n          \n          2024","key":"B28"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1464","DOI":"10.1109\/5.58325","article-title":"The self-organizing map","volume":"78","author":"Kohonen","year":"1990","journal-title":"Proc. IEEE"},{"key":"B30","article-title":"Improving expert specialization in mixture of experts","author":"Krishnamurthy","year":"2023","journal-title":"arXiv preprint arXiv:2302.14703"},{"key":"B31","doi-asserted-by":"publisher","first-page":"118219","DOI":"10.1016\/j.eswa.2022.118219","article-title":"Design of a modular neural network based on an improved soft subspace clustering algorithm","volume":"209","author":"Li","year":"2022","journal-title":"Expert Syst. Appl"},{"key":"B32","doi-asserted-by":"publisher","first-page":"2049","DOI":"10.1016\/j.ins.2007.01.009","article-title":"Hybridizing mixtures of experts with support vector machines: Investigation into nonlinear dynamic systems identification","volume":"177","author":"Lima","year":"2007","journal-title":"Inf. Sci"},{"volume-title":"Some Methods for Classification and Analysis of Multivariate Observations","year":"1967","author":"Macqueen","key":"B33"},{"year":"2020","author":"Matzka","journal-title":"AI4I 2020 Predictive Maintenance Dataset","key":"B34"},{"key":"B35","article-title":"\u201cAn alternative infinite mixture of gaussian process experts,\u201d","author":"Meeds","year":"2005","journal-title":"Advances in Neural Information Processing Systems"},{"year":"2016","author":"Moro","journal-title":"Facebook Metrics","key":"B36"},{"year":"1995","author":"Nash","journal-title":"Abalone. UCI Machine Learning Repository","key":"B37"},{"key":"B38","doi-asserted-by":"publisher","first-page":"738","DOI":"10.1109\/TNN.2004.826217","article-title":"Using the em algorithm to train neural networks: misconceptions and a new algorithm for multiclass classification","volume":"15","author":"Ng","year":"2004","journal-title":"IEEE Trans. Neural Netw"},{"key":"B39","article-title":"Exploring expert specialization through unsupervised training in sparse mixture of experts","author":"Nikolic","year":"2025","journal-title":"arXiv preprint arXiv:2509.10025"},{"key":"B40","first-page":"53022","article-title":"\u201cMultilinear mixture of experts: Scalable expert specialization through factorization,\u201d","author":"Oldfield","year":"2024","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1016\/j.dib.2019.104344","author":"Palechor","year":"2019","journal-title":"Estimation of Obesity Levels Based On Eating Habits and Physical Condition"},{"key":"B42","article-title":"Competesmoe-effective training of sparse mixture of experts via competition","author":"Pham","year":"2024","journal-title":"arXiv preprint arXiv:2402.02526"},{"key":"B43","article-title":"Divide, specialize, and route: a new approach to efficient ensemble learning","author":"Piwko","year":"2025","journal-title":"arXiv preprint arXiv:2506.20814"},{"year":"1993","author":"Quinlan","journal-title":"Auto MPG","key":"B44"},{"key":"B45","doi-asserted-by":"publisher","first-page":"106672","DOI":"10.1016\/j.knosys.2020.106672","article-title":"Gbk-means clustering algorithm: an improvement to the k-means algorithm based on the bargaining game","volume":"213","author":"Rezaee","year":"2021","journal-title":"Knowl. Based Syst"},{"year":"2020","author":"Sathishkumar","journal-title":"Seoul Bike Sharing Demand","key":"B46"},{"year":"1987","author":"Schlimmer","journal-title":"Automobile","key":"B47"},{"key":"B48","article-title":"Outrageously large neural networks: the sparsely-gated mixture-of-experts layer","author":"Shazeer","year":"2017","journal-title":"arXiv preprint arXiv:1701.06538"},{"year":"2014","author":"Tfekci","journal-title":"Combined Cycle Power Plant","key":"B49"},{"key":"B50","article-title":"\u201cMixtures of Gaussian processes,\u201d","author":"Tresp","year":"2000","journal-title":"Advances in Neural Information Processing Systems"},{"year":"2009","author":"Tsanas","journal-title":"Parkinsons Telemonitoring","key":"B51"},{"year":"2012","author":"Tsanas","journal-title":"Energy Efficiency","key":"B52"},{"key":"B53","doi-asserted-by":"publisher","first-page":"1223","DOI":"10.1016\/S0893-6080(02)00040-0","article-title":"Bayesian model search for mixture models based on optimizing variational bounds","volume":"15","author":"Ueda","year":"2002","journal-title":"Neural Netw"},{"key":"B54","doi-asserted-by":"publisher","first-page":"20240124","DOI":"10.1098\/rspa.2024.0124","author":"Ukorigho","year":"2025"},{"key":"B55","article-title":"\u201cBayesian methods for mixtures of experts,\u201d","author":"Waterhouse","year":"1995","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B56","doi-asserted-by":"publisher","first-page":"373","DOI":"10.1142\/S0129065795000251","article-title":"Nonlinear gated experts for time series: discovering regimes and avoiding overfitting","volume":"6","author":"Weigend","year":"1995","journal-title":"Int. J. Neural Syst"},{"year":"1995","author":"Wolberg","journal-title":"Breast Cancer Wisconsin (Prognostic)","key":"B57"},{"key":"B58","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1109\/TITB.2004.824724","article-title":"Cluster analysis of gene expression data based on self-splitting and merging competitive learning","volume":"8","author":"Wu","year":"2004","journal-title":"IEEE Trans. Inf. Technol. Biomed"},{"key":"B59","article-title":"\u201cAn alternative model for mixtures of experts,\u201d","author":"Xu","year":"1994","journal-title":"Advances in Neural Information Processing Systems"},{"year":"2007","author":"Yeh","journal-title":"Concrete Compressive Strength","key":"B60"},{"year":"2018","author":"Yeh","journal-title":"Real Estate Valuation","key":"B61"},{"key":"B62","article-title":"\u201cVariational mixture of gaussian process experts,\u201d","author":"Yuan","year":"2008","journal-title":"Advances in Neural Information Processing Systems"},{"key":"B63","doi-asserted-by":"publisher","first-page":"1177","DOI":"10.1109\/TNNLS.2012.2200299","article-title":"Twenty years of mixture of experts","volume":"23","author":"Yuksel","year":"2012","journal-title":"IEEE Trans. Neural Netw. Learn. Syst"},{"key":"B64","doi-asserted-by":"publisher","first-page":"369","DOI":"10.1109\/72.991422","article-title":"Self-splitting competitive learning: a new on-line clustering paradigm","volume":"13","author":"Zhang","year":"2002","journal-title":"IEEE Trans. Neural Netw"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1661444\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,11,5]],"date-time":"2025-11-05T06:28:51Z","timestamp":1762324131000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2025.1661444\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,11,5]]},"references-count":64,"alternative-id":["10.3389\/frai.2025.1661444"],"URL":"https:\/\/doi.org\/10.3389\/frai.2025.1661444","relation":{},"ISSN":["2624-8212"],"issn-type":[{"type":"electronic","value":"2624-8212"}],"subject":[],"published":{"date-parts":[[2025,11,5]]},"article-number":"1661444"}}