{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:57:10Z","timestamp":1760241430573,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2018,3,17]],"date-time":"2018-03-17T00:00:00Z","timestamp":1521244800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Equally partitioned data are essential for prediction. However, in some important cases, the data distribution is severely unbalanced. In this study, several algorithms are utilized to maximize the learning accuracy when dealing with a highly unbalanced dataset. A linguistic algorithm is applied to evaluate the input and output relationship, namely Fuzzy c-Means (FCM), which is applied as a clustering algorithm for the majority class to balance the minority class data from about 3 million cases. Each cluster is used to train several artificial neural network (ANN) models. Different techniques are applied to generate an ensemble genetic fuzzy neuro model (EGFNM) in order to select the models. The first ensemble technique, the intra-cluster EGFNM, works by evaluating the best combination from all the models generated by each cluster. Another ensemble technique is the inter-cluster model EGFNM, which is based on selecting the best model from each cluster. The accuracy of these techniques is evaluated using the receiver operating characteristic (ROC) via its area under the curve (AUC). Results show that the AUC of the unbalanced data is 0.67974. The random cluster and best ANN single model have AUCs of 0.7177 and 0.72806, respectively. For the ensemble evaluations, the intra-cluster and the inter-cluster EGFNMs produce 0.7293 and 0.73038, respectively. In conclusion, this study achieved improved results by performing the EGFNM method compared with the unbalanced training. This study concludes that selecting several best models will produce a better result compared with all models combined.<\/jats:p>","DOI":"10.3390\/sym10030071","type":"journal-article","created":{"date-parts":[[2018,3,20]],"date-time":"2018-03-20T15:59:39Z","timestamp":1521561579000},"page":"71","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Ensemble Genetic Fuzzy Neuro Model Applied for the Emergency Medical Service via Unbalanced Data Evaluation"],"prefix":"10.3390","volume":"10","author":[{"given":"Muammar","family":"Sadrawi","sequence":"first","affiliation":[{"name":"Department of Mechanical Engineering and Innovation Center for Big Data and Digital Convergence, Yuan Ze University, Taoyuan, Chung-Li 32003, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei-Zen","family":"Sun","sequence":"additional","affiliation":[{"name":"Department of Anesthesiology, College of Medicine, National Taiwan University, Taipei 100, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthew","family":"Ma","sequence":"additional","affiliation":[{"name":"Department of Emergency Medicine, College of Medicine, National Taiwan University, Taipei 100, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu-Ting","family":"Yeh","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering and Innovation Center for Big Data and Digital Convergence, Yuan Ze University, Taoyuan, Chung-Li 32003, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8515-7933","authenticated-orcid":false,"given":"Maysam","family":"Abbod","sequence":"additional","affiliation":[{"name":"Department of Electronic and Computer Engineering, Brunel University London, Uxbridge UB8 3PH, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiann-Shing","family":"Shieh","sequence":"additional","affiliation":[{"name":"Department of Mechanical Engineering and Innovation Center for Big Data and Digital Convergence, Yuan Ze University, Taoyuan, Chung-Li 32003, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2018,3,17]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1263","DOI":"10.1109\/TKDE.2008.239","article-title":"Learning from imbalanced data","volume":"21","author":"He","year":"2009","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1007730.1007733","article-title":"Editorial: Special issue on learning from imbalanced data sets","volume":"6","author":"Chawla","year":"2004","journal-title":"ACM SIGKDD Explor. Newsl."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1119","DOI":"10.1109\/TSMCB.2012.2187280","article-title":"Multi-class imbalance problems: Analysis and potential solutions","volume":"42","author":"Wang","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. Part B Cybern."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"203","DOI":"10.1023\/A:1007601015854","article-title":"Robust classification for imprecise environments","volume":"42","author":"Provost","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"195","DOI":"10.1023\/A:1007452223027","article-title":"Machine learning for the detection of oil spills in satellite radar images","volume":"30","author":"Kubat","year":"1998","journal-title":"Mach. Learn."},{"key":"ref_6","unstructured":"Ezawa, K.J., Singh, M., and Norton, S.W. (1996, January 3\u20136). Learning goal oriented Bayesian networks for telecommunications risk management. Proceedings of the ICML, Bari, Italy."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Lewis, D.D., and Catlett, J. (1994, January 10\u201313). Heterogeneous uncertainty sampling for supervised learning. Proceedings of the Eleventh International Conference on Machine Learning, New Brunswick, NJ, USA.","DOI":"10.1016\/B978-1-55860-335-6.50026-X"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"291","DOI":"10.1023\/A:1009700419189","article-title":"Adaptive fraud detection","volume":"1","author":"Fawcett","year":"1997","journal-title":"Data Min. Knowl. Discov."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"19","DOI":"10.1007\/s10844-013-0254-7","article-title":"Cost-sensitive three-way email spam filtering","volume":"42","author":"Zhou","year":"2014","journal-title":"J. Intell. Inf. Syst."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1145\/1007730.1007735","article-title":"A study of the behavior of several methods for balancing machine learning training data","volume":"6","author":"Batista","year":"2004","journal-title":"ACM SIGKDD Explor. Newsl."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Cristianini, N., and Shawe-Taylor, J. (2000). An Introduction to Support Vector Machines, Cambridge University Press.","DOI":"10.1017\/CBO9780511801389"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"321","DOI":"10.1613\/jair.953","article-title":"SMOTE: Synthetic minority over-sampling technique","volume":"16","author":"Chawla","year":"2002","journal-title":"J. Artif. Intell. Res."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"264","DOI":"10.1145\/331499.331504","article-title":"Data clustering: A review","volume":"31","author":"Jain","year":"1999","journal-title":"ACM Comput. Surv. CSUR"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1880","DOI":"10.1049\/el.2015.1541","article-title":"Improving fuzzy c-means method for unbalanced dataset","volume":"51","author":"Liu","year":"2015","journal-title":"Electron. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Zhou, B., Ha, M., and Wang, C. (2010). An improved algorithm of unbalanced data SVM. Fuzzy Information and Engineering, Springer.","DOI":"10.1007\/978-3-642-14880-4_60"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Kocyigit, Y., and Seker, H. (2012, January 5\u20137). Imbalanced data classifier by using ensemble fuzzy c-means clustering. Proceedings of the 2012 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), Hong Kong, China.","DOI":"10.1109\/BHI.2012.6211746"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Yazdani-Chamzini, A., Zavadskas, E.K., Antucheviciene, J., and Bausys, R. (2017). A Model for Shovel Capital Cost Estimation, Using a Hybrid Model of Multivariate Regression and Neural Networks. Symmetry, 9.","DOI":"10.3390\/sym9120298"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"994","DOI":"10.3846\/16111699.2012.683808","article-title":"Forecasting gold price changes by using adaptive network fuzzy inference system","volume":"13","author":"Yakhchali","year":"2012","journal-title":"J. Bus. Econ. Manag."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1162\/neco.1992.4.1.1","article-title":"Neural networks and the bias\/variance dilemma","volume":"4","author":"Geman","year":"1992","journal-title":"Neural Comput."},{"key":"ref_20","unstructured":"Xie, B., Liang, Y., and Song, L. (2017, January 20\u201322). Diverse Neural Network Learns True Target Functions. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, FL, USA."},{"key":"ref_21","unstructured":"Wang, S., and Yao, X. (April, January 30). Diversity analysis on imbalanced data sets by using ensemble models. Proceedings of the IEEE Symposium on Computational Intelligence and Data Mining (CIDM\u201909), Nashville, TN, USA."},{"key":"ref_22","unstructured":"Cunningham, P., and Carney, J. (June, January 31). Diversity versus quality in classification ensembles based on feature selection. Proceedings of the European Conference on Machine Learning, Barcelona, Spain."},{"key":"ref_23","first-page":"231","article-title":"Neural network ensembles, cross validation, and active learning","volume":"7","author":"Krogh","year":"1995","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1016\/j.inffus.2004.04.004","article-title":"Diversity creation methods: A survey and categorization","volume":"6","author":"Brown","year":"2005","journal-title":"Inf. Fusion"},{"key":"ref_25","unstructured":"Tumer, K., and Ghosh, J. (2018, January 10). Theoretical Foundations of Linear and Order Statistics Combiners for Neural Pattern Classifiers. Available online: http:\/\/citeseerx.ist.psu.edu\/viewdoc\/download?doi=10.1.1.36.3954&rep=rep1&type=pdf."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"418","DOI":"10.1109\/21.155943","article-title":"Methods of combining multiple classifiers and their applications to handwriting recognition","volume":"22","author":"Xu","year":"1992","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"66","DOI":"10.1109\/34.273716","article-title":"Decision combination in multiple classifier systems","volume":"16","author":"Ho","year":"1994","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1109\/34.667881","article-title":"On combining classifiers","volume":"20","author":"Kittler","year":"1998","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"463","DOI":"10.1109\/TSMCC.2011.2161285","article-title":"A review on ensembles for the class imbalance problem: Bagging-, boosting-, and hybrid-based approaches","volume":"42","author":"Galar","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. Part C Appl. Rev."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"3460","DOI":"10.1016\/j.patcog.2013.05.006","article-title":"EUSBoost: Enhancing ensembles for highly imbalanced data-sets by evolutionary undersampling","volume":"46","author":"Galar","year":"2013","journal-title":"Pattern Recognit."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.cmpb.2017.02.011","article-title":"A new survival status prediction system for severe trauma patients based on a multiple classifier system","volume":"142","author":"Sanz","year":"2017","journal-title":"Comput. Methods Programs Biomed."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1016\/S0004-3702(02)00190-X","article-title":"Ensembling neural networks: Many could be better than all","volume":"137","author":"Zhou","year":"2002","journal-title":"Artif. Intell."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Hornby, G.S., Globus, A., Linden, D.S., and Lohn, J.D. (2006, January 19\u201321). Automated antenna design with evolutionary algorithms. Proceedings of the AIAA Space, San Jose, CA, USA.","DOI":"10.2514\/6.2006-7242"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"85","DOI":"10.1016\/j.knosys.2016.05.033","article-title":"A multi-level approach using genetic algorithms in an ensemble of Least Squares Support Vector Machines","volume":"106","author":"Barone","year":"2016","journal-title":"Knowl. Based Syst."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Haque, M.N., Noman, N., Berretta, R., and Moscato, P. (2016). Heterogeneous ensemble combination search using genetic algorithm for class imbalanced data classification. PLoS ONE, 11.","DOI":"10.1371\/journal.pone.0146116"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1178","DOI":"10.1109\/JBHI.2015.2443799","article-title":"Demand Forecast Using Data Analytics for the Preallocation of Ambulances","volume":"20","author":"Chen","year":"2016","journal-title":"IEEE J. Biomed. Health Inform."},{"key":"ref_37","unstructured":"Shieh, J.S., Yeh, Y.T., Sun, Y.Z., Ma, M.H., Dia, C.Y., Sadrawi, M., and Abbod, M. (2017, January 29). Big Data Analysis of Emergency Medical Service Applied to Determine the Survival Rate Effective Factors and Predict the Ambulance Time Variables. Proceedings of the ISER 50th International Conference, Tokyo, Japan."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Ross, T.J. (2009). Fuzzy Logic with Engineering Applications, John Wiley & Sons.","DOI":"10.1002\/9781119994374"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1016\/S0031-3203(96)00142-2","article-title":"The use of the area under the ROC curve in the evaluation of machine learning algorithms","volume":"30","author":"Bradley","year":"1997","journal-title":"Pattern Recognit."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1016\/S0031-3203(00)00002-9","article-title":"A novel algorithm for data clustering","volume":"34","author":"Wong","year":"2001","journal-title":"Pattern Recognit."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/s10015-012-0048-y","article-title":"Ensembled neural networks applied to modeling survival rate for the patients with out-of-hospital cardiac arrest","volume":"17","author":"Jiang","year":"2012","journal-title":"Artif. Life Robot."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"79","DOI":"10.1109\/TR.2017.2749242","article-title":"Reliable control of discrete-time piecewise-affine time-delay systems via output feedback","volume":"67","author":"Qiu","year":"2018","journal-title":"IEEE Trans. Reliab."},{"key":"ref_43","first-page":"957","article-title":"A novel approach to reliable control of piecewise affine systems with actuator faults","volume":"64","author":"Qiu","year":"2017","journal-title":"IEEE Trans. Circuits Syst. II Express Briefs"}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/10\/3\/71\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T14:57:29Z","timestamp":1760194649000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/10\/3\/71"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,3,17]]},"references-count":43,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2018,3]]}},"alternative-id":["sym10030071"],"URL":"https:\/\/doi.org\/10.3390\/sym10030071","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2018,3,17]]}}}