{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,18]],"date-time":"2026-01-18T08:24:11Z","timestamp":1768724651873,"version":"3.49.0"},"reference-count":45,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2017,7,6]],"date-time":"2017-07-06T00:00:00Z","timestamp":1499299200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61602207"],"award-info":[{"award-number":["61602207"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61572228"],"award-info":[{"award-number":["61572228"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61472158"],"award-info":[{"award-number":["61472158"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61373050"],"award-info":[{"award-number":["61373050"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Science Technology Development Project from Jilin Province","award":["20160101247JC"],"award-info":[{"award-number":["20160101247JC"]}]},{"name":"Zhuhai Premier Discipline Enhancement Scheme"},{"name":"Guangdong Premier Key-Discipline Enhancement Scheme"},{"name":"the Educational Commission of Jilin Province"},{"DOI":"10.13039\/501100004543","name":"China Scholarship Council","doi-asserted-by":"publisher","id":[{"id":"10.13039\/501100004543","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Overfitting is an important problem in machine learning. Several algorithms, such as the extreme learning machine (ELM), suffer from this issue when facing high-dimensional sparse data, e.g., in text classification. One common issue is that the extent of overfitting is not well quantified. In this paper, we propose a quantitative measure of overfitting referred to as the rate of overfitting (RO) and a novel model, named AdaBELM, to reduce the overfitting. With RO, the overfitting problem can be quantitatively measured and identified. The newly proposed model can achieve high performance on multi-class text classification. To evaluate the generalizability of the new model, we designed experiments based on three datasets, i.e., the 20 Newsgroups, Reuters-21578, and BioMed corpora, which represent balanced, unbalanced, and real application data, respectively. Experiment results demonstrate that AdaBELM can reduce overfitting and outperform classical ELM, decision tree, random forests, and AdaBoost on all three text-classification datasets; for example, it can achieve 62.2% higher accuracy than ELM. Therefore, the proposed model has a good generalizability.<\/jats:p>","DOI":"10.3390\/e19070330","type":"journal-article","created":{"date-parts":[[2017,7,6]],"date-time":"2017-07-06T10:55:45Z","timestamp":1499338545000},"page":"330","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":23,"title":["Overfitting Reduction of Text Classification Based on AdaBELM"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3954-1333","authenticated-orcid":false,"given":"Xiaoyue","family":"Feng","sequence":"first","affiliation":[{"name":"Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, College of Computer Science and Technology, Jilin University, Changchun 130012, China"}]},{"given":"Yanchun","family":"Liang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, College of Computer Science and Technology, Jilin University, Changchun 130012, China"},{"name":"Zhuhai Laboratory of Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Zhuhai College of Jilin University, Zhuhai 519041, China"},{"name":"Department of Electric Engineering and Computer Science, and Christopher S. Bond Life Sciences Center, University of Missouri, Columbia, MO 65211, USA"}]},{"given":"Xiaohu","family":"Shi","sequence":"additional","affiliation":[{"name":"Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, College of Computer Science and Technology, Jilin University, Changchun 130012, China"},{"name":"Zhuhai Laboratory of Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Zhuhai College of Jilin University, Zhuhai 519041, China"}]},{"given":"Dong","family":"Xu","sequence":"additional","affiliation":[{"name":"Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, College of Computer Science and Technology, Jilin University, Changchun 130012, China"},{"name":"Department of Electric Engineering and Computer Science, and Christopher S. Bond Life Sciences Center, University of Missouri, Columbia, MO 65211, USA"}]},{"given":"Xu","family":"Wang","sequence":"additional","affiliation":[{"name":"Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, College of Computer Science and Technology, Jilin University, Changchun 130012, China"}]},{"given":"Renchu","family":"Guan","sequence":"additional","affiliation":[{"name":"Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, College of Computer Science and Technology, Jilin University, Changchun 130012, China"},{"name":"Zhuhai Laboratory of Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Zhuhai College of Jilin University, Zhuhai 519041, China"}]}],"member":"1968","published-online":{"date-parts":[[2017,7,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/505282.505283","article-title":"Machine Learning in Automated Text Categorization","volume":"34","author":"Sebastiani","year":"2002","journal-title":"ACM Comput. Surv."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Joachims, T. (1998). Text Categorization with Support Vector Machines: Learning with Many Relevant Features, Springer.","DOI":"10.1007\/BFb0026683"},{"key":"ref_3","first-page":"1929","article-title":"Dropout: A Simple Way to Prevent Neural Networks from Overfitting","volume":"15","author":"Srivastava","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"135","DOI":"10.1023\/A:1007649029923","article-title":"BoosTexter: A Boosting-based System for Text Categorization","volume":"39","author":"Schapire","year":"2000","journal-title":"Mach. Learn."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Laurent, A., Camelin, N., and Raymond, C. (2014, January 12). Boosting Bonsai Trees for Efficient Features Combination: Application to Speaker Role Identification. Proceedings of the 15th Annual Conference of the International Speech Communication Association, Singapore.","DOI":"10.21437\/Interspeech.2014-16"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep Learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1320","DOI":"10.1109\/72.471375","article-title":"Stochastic Choice of Basis Functions in Adaptive Function Approximation and The Functional-Link Net","volume":"6","author":"Igelnik","year":"1995","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1109\/2.144401","article-title":"Functional-link Net Computing: Theory, System Architecture, and Functionalities","volume":"25","author":"Pao","year":"1992","journal-title":"Computer"},{"key":"ref_9","unstructured":"Huang, G.B., Zhu, Q.Y., and Siew, C.K. (2004, January 25\u201329). Extreme Learning Machine: A New Learning Scheme of Feedforward Neural Networks. Proceedings of the 2004 IEEE International Joint Conference on Neural Networks, Budapest, Hungary."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1094","DOI":"10.1016\/j.ins.2015.09.025","article-title":"A comprehensive evaluation of random vector functional link networks","volume":"367","author":"Zhang","year":"2016","journal-title":"Inf. Sci."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"489","DOI":"10.1016\/j.neucom.2005.12.126","article-title":"Extreme Learning Machine: Theory and Applications","volume":"70","author":"Huang","year":"2006","journal-title":"Neurocomputing"},{"key":"ref_12","unstructured":"(2017, March 16). Extreme Learning Machines: Random Neurons, Random Features, Kernels. Available online: http:\/\/www.ntu.edu.sg\/home\/egbhuang\/."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"107","DOI":"10.1007\/s13042-011-0019-y","article-title":"Extreme Learning Machines: A Survey","volume":"2","author":"Huang","year":"2011","journal-title":"Int. J. Mach. Learn. Cybern."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"158","DOI":"10.1109\/TNN.2009.2036259","article-title":"OP-ELM: Optimally Pruned Extreme Learning Machine","volume":"21","author":"Miche","year":"2010","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"505","DOI":"10.1109\/TNN.2010.2103956","article-title":"BELM: Bayesian Extreme Learning Machine","volume":"22","author":"Martin","year":"2011","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"386","DOI":"10.1016\/j.patcog.2010.08.009","article-title":"Realtime Training on Mobile Devices for Face Recognition Applications","volume":"44","author":"Choi","year":"2011","journal-title":"Pattern Recognit."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"836","DOI":"10.1109\/TNNLS.2013.2281839","article-title":"Sparse Bayesian Extreme Learning Machine for Multi-classification","volume":"25","author":"Luo","year":"2014","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.neucom.2012.01.041","article-title":"Optimizing Extreme Learning Machines via Ridge Regression and Batch Intrinsic Plasticity","volume":"102","author":"Neumann","year":"2013","journal-title":"Neurocomputing"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Er, M.J., Shao, Z., and Wang, N. (2014, January 6\u201311). A Fast and Effective Extreme Learning Machine Algorithm Without Tuning. Proceedings of the 2014 International Joint Conference on Neural Networks (IJCNN), Beijing, China.","DOI":"10.1109\/IJCNN.2014.6889397"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"153","DOI":"10.1016\/j.neucom.2013.08.041","article-title":"Ensemble Delta Test-Extreme Learning Machine (DT-ELM) for Regression","volume":"129","author":"Yu","year":"2014","journal-title":"Neurocomputing"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"359","DOI":"10.1016\/j.neucom.2008.01.005","article-title":"A Fast Pruned-Extreme Learning Machine for Classification Problem","volume":"72","author":"Rong","year":"2008","journal-title":"Neurocomputing"},{"key":"ref_22","unstructured":"Viola, P., and Jones, M. (2001, January 8\u201314). Rapid Object Detection Using a Boosted Cascade of Simple Features. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, CVPR 2001, Kauai, HI, USA."},{"key":"ref_23","unstructured":"Freund, Y., and Schapire, R.E. (1996, January 2). Experiments with A New Boosting Algorithm. Proceedings of the 13th International Conference of machine learning, Bari, Italy."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1016\/j.ins.2014.10.040","article-title":"A Rapid Learning Algorithm for Vehicle Classification","volume":"295","author":"Wen","year":"2015","journal-title":"Inf. Sci."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1023\/A:1007515423169","article-title":"An Empirical Comparison of Voting Classification Algorithms: Bagging, Boosting, and Variants","volume":"36","author":"Bauer","year":"1999","journal-title":"Mach. Learn."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1023\/A:1007607513941","article-title":"An Experimental Comparison of Three Methods for Constructing Ensembles of Decision Trees: Bagging, Boosting, and Randomization","volume":"40","author":"Dietterich","year":"2000","journal-title":"Mach. Learn."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.artint.2013.07.002","article-title":"On the Doubt About Margin Explanation of Boosting","volume":"203","author":"Gao","year":"2013","journal-title":"Artif. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Freund, Y., and Schapire, R.E. (1995). A Desicion-Theoretic Generalization of On-Line Learning and an Application to Boosting, Springer.","DOI":"10.1007\/3-540-59119-2_166"},{"key":"ref_29","unstructured":"Grove, A.J., and Schuurmans, D. (1998, January 26\u201330). Boosting in the Limit: Maximizing the Margin of Learned Ensembles. Proceedings of the 15th National Conference on Artificial Intelligence, Madison, WI, USA."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1023\/A:1007618119488","article-title":"Soft Margins for AdaBoost","volume":"42","author":"Onoda","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_31","unstructured":"Reyzin, L., and Schapire, R.E. How Boosting the Margin Can Also Boost Classifier Complexity. Proceedings of the 23rd International Conference on Machine Learning, New York, NY, USA."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1876","DOI":"10.1016\/j.tcs.2009.01.016","article-title":"Exploration\u2013Exploitation Tradeoff Using Variance Estimates in Multi-Armed Bandits","volume":"410","author":"Audibert","year":"2009","journal-title":"Theor. Comput. Sci."},{"key":"ref_33","first-page":"3133","article-title":"Do we Need Hundreds of Classifiers to Solve Real World Classification Problems?","volume":"15","author":"Cernadas","year":"2014","journal-title":"J. Mach. Learn. Res."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2165","DOI":"10.1109\/TCYB.2014.2366468","article-title":"Oblique Decision Tree Ensemble via Multisurface Proximal Support Vector Machine","volume":"45","author":"Zhang","year":"2015","journal-title":"IEEE Trans. Cybern."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"3429","DOI":"10.1016\/j.patcog.2014.04.001","article-title":"Random forests with ensemble of feature spaces","volume":"47","author":"Zhang","year":"2014","journal-title":"Pattern Recognit."},{"key":"ref_37","first-page":"601","article-title":"Generalized Inverse of a Matrix and Its Applications","volume":"1","author":"Rao","year":"1972","journal-title":"Berkeley Symp. Math. Stat. Probab."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s10115-007-0114-2","article-title":"Top 10 Algorithms in Data Mining","volume":"14","author":"Wu","year":"2008","journal-title":"Knowl. Inf. Syst."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1007\/BF00116037","article-title":"The Strength of Weak Learnability","volume":"5","author":"Schapire","year":"1990","journal-title":"Mach. Learn."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Deng, W., Zheng, Q., and Chen, L. (April, January 30). Regularized Extreme Learning Machine. Proceedings of the 2009 IEEE Symposium on Computational Intelligence and Data Mining, Nashville, TN, USA.","DOI":"10.1109\/CIDM.2009.4938676"},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Zhang, T. (2004, January 4\u20138). Solving Large Scale Linear Prediction Problems Using Stochastic Gradient Descent Algorithms. Proceedings of the Twenty-First International Conference on Machine Learning, New York, NY, USA.","DOI":"10.1145\/1015330.1015332"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1109\/TSMCB.2011.2168604","article-title":"Extreme Learning Machine for Regression and Multiclass Classification","volume":"42","author":"Huang","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. Part B Cybern."},{"key":"ref_43","unstructured":"(2017, March 17). Home Page for 20 Newsgroups Data Set. Available online: http:\/\/qwone.com\/~jason\/20Newsgroups\/."},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"1624","DOI":"10.1109\/TKDE.2005.198","article-title":"Document Clustering Using Locality Preserving Indexing","volume":"17","author":"Cai","year":"2005","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"627","DOI":"10.1109\/TKDE.2010.144","article-title":"Text Clustering with Seeds Affinity Propagation","volume":"23","author":"Guan","year":"2011","journal-title":"IEEE Trans. Knowl. Data Eng."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/7\/330\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T18:41:39Z","timestamp":1760208099000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/19\/7\/330"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,7,6]]},"references-count":45,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2017,7]]}},"alternative-id":["e19070330"],"URL":"https:\/\/doi.org\/10.3390\/e19070330","relation":{},"ISSN":["1099-4300"],"issn-type":[{"value":"1099-4300","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,7,6]]}}}