{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:38:10Z","timestamp":1760060290298,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2025,8,20]],"date-time":"2025-08-20T00:00:00Z","timestamp":1755648000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62376094","62206177","62106145","LY23F020007","LQ22F020024","Y202248951"],"award-info":[{"award-number":["62376094","62206177","62106145","LY23F020007","LQ22F020024","Y202248951"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004731","name":"Zhejiang Provincial Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62376094","62206177","62106145","LY23F020007","LQ22F020024","Y202248951"],"award-info":[{"award-number":["62376094","62206177","62106145","LY23F020007","LQ22F020024","Y202248951"]}],"id":[{"id":"10.13039\/501100004731","id-type":"DOI","asserted-by":"publisher"}]},{"name":"General Scientific Research Project of Zhejiang Education Department","award":["62376094","62206177","62106145","LY23F020007","LQ22F020024","Y202248951"],"award-info":[{"award-number":["62376094","62206177","62106145","LY23F020007","LQ22F020024","Y202248951"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>Traditional deep neural networks (DNNs) often suffer from a time-consuming training process, which is restricted by accumulation of excessive network layers and a large amount of parameters. More neural units are required to be stacked to achieve desirable performance. Specifically, when dealing with large-scale datasets, a single DNN can hardly obtain the best performance on the available limited computing resources. To address the issues above, in this paper, a novel Parallel Hierarchical Ensemble Deep Neural Network (PH-E-DNN) is proposed to improve accuracy and efficiency of the deep network. Firstly, the fuzzy C-means algorithm (FCM) is adopted so that the large-scale dataset is separated into several small data partitions. As a benefit of the fuzzy partitioning of the FCM, several sub-models can be obtained through learning their respective data partitions and isolating them from the others. Secondly, the prediction results of each sub-model in the current level are used as the discriminative knowledge appended to original regional subsets, and predictions from each level symmetrically augment inputs for the next level. In the PH-E-DNN architecture, predictions from each level symmetrically augment inputs for the next level, creating a symmetrical flow of discriminative knowledge across the hierarchical structure. Finally, multiple regional subsets are merged to form a global augmented dataset, while multi-level parallel sub-models are stacked to organize a large-scale deep ensemble network. More importantly, only the multiple DNNs in the last level are ensembled to generate the decision result of the proposed PH-E-DNN. Extensive experiments demonstrate that the PH-E-DNN is superior to some traditional and deep learning models, only requiring a few parameters to be set, which demonstrates its efficiency and flexibility.<\/jats:p>","DOI":"10.3390\/sym17081362","type":"journal-article","created":{"date-parts":[[2025,8,20]],"date-time":"2025-08-20T13:18:14Z","timestamp":1755695894000},"page":"1362","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Divide-and-Merge Parallel Hierarchical Ensemble DNNs with Local Knowledge Augmentation"],"prefix":"10.3390","volume":"17","author":[{"given":"Zhibin","family":"Jiang","sequence":"first","affiliation":[{"name":"Department of Computer Science and Engineering, Shaoxing University, Shaoxing 312000, China"},{"name":"Institute of Artificial Intelligence, Shaoxing University, Shaoxing 312000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuai","family":"Dong","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Huzhou University, Huzhou 313000, China"},{"name":"Zhejiang Province Key Laboratory of Smart Management & Application of Modern Agricultural Resources, Huzhou University, Huzhou 313000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kaining","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Huzhou University, Huzhou 313000, China"},{"name":"Zhejiang Province Key Laboratory of Smart Management & Application of Modern Agricultural Resources, Huzhou University, Huzhou 313000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Computer Science and Engineering, Shaoxing University, Shaoxing 312000, China"},{"name":"Institute of Artificial Intelligence, Shaoxing University, Shaoxing 312000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiongtao","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Information Engineering, Huzhou University, Huzhou 313000, China"},{"name":"Zhejiang Province Key Laboratory of Smart Management & Application of Modern Agricultural Resources, Huzhou University, Huzhou 313000, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2025,8,20]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1109\/TNNLS.2018.2829819","article-title":"Deep CNN-based blind image quality predictor","volume":"30","author":"Kim","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"2306","DOI":"10.1109\/TNNLS.2016.2582798","article-title":"Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics","volume":"28","author":"Zhang","year":"2017","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"7005","DOI":"10.1109\/TII.2020.3041300","article-title":"Intelligent short-term voltage stability assessment via spatial attention rectified RNN learning","volume":"17","author":"Zhu","year":"2021","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1798","DOI":"10.1109\/TPAMI.2013.50","article-title":"Representation learning: A review and new perspectives","volume":"35","author":"Bengio","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"197","DOI":"10.1561\/2000000039","article-title":"Deep learning: Methods and applications","volume":"7","author":"Deng","year":"2014","journal-title":"Found. Trends Signal Process."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"504","DOI":"10.1126\/science.1127647","article-title":"Reducing the dimensionality of data with neural networks","volume":"313","author":"Hinton","year":"2006","journal-title":"Science"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1527","DOI":"10.1162\/neco.2006.18.7.1527","article-title":"A fast learning algorithm for deep belief nets","volume":"18","author":"Hinton","year":"2006","journal-title":"Neural Comput."},{"key":"ref_8","first-page":"3371","article-title":"Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion","volume":"11","author":"Vincent","year":"2010","journal-title":"J. Mach. Learn. Res."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1109\/TMI.2015.2458702","article-title":"Stacked sparse autoencoder (SSAE) for nuclei detection on breast cancer histopathology images","volume":"35","author":"Xu","year":"2016","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"7080","DOI":"10.1002\/int.22581","article-title":"A stacked autoencoder-based convolutional and recurrent deep neural network for detecting cyberattacks in interconnected power control systems","volume":"36","author":"Palmieri","year":"2021","journal-title":"Int. J. Intell. Syst."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1109\/TCYB.2015.2501373","article-title":"Coupled deep autoencoder for single image super-resolution","volume":"47","author":"Zeng","year":"2017","journal-title":"IEEE Trans. Cybern."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Graves, A., and Mohamed Ar Hinton, G. (2013). Speech recognition with deep recurrent neural networks. International Conference on Acoustics, Speech and Signal Processing, IEEE.","DOI":"10.1109\/ICASSP.2013.6638947"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1229","DOI":"10.1109\/TNNLS.2014.2317880","article-title":"A comprehensive review of stability analysis of continuous-time recurrent neural networks","volume":"25","author":"Zhang","year":"2014","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"10","DOI":"10.1109\/TNNLS.2017.2716952","article-title":"Broad learning system: An effective and efficient incremental learning system without the need for deep architecture","volume":"29","author":"Chen","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1191","DOI":"10.1109\/TNNLS.2018.2866622","article-title":"Universal approximation capability of broad learning system and its structural variations","volume":"30","author":"Chen","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"803","DOI":"10.1109\/TETCI.2020.2996943","article-title":"A novel ensemble deep learning approach for sleep-wake detection using heart rate variability and acceleration","volume":"5","author":"Chen","year":"2021","journal-title":"IEEE Trans. Emerg. Top. Comput. Intell."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"553","DOI":"10.1109\/TNNLS.2018.2844464","article-title":"Deep ensemble machine for video classification","volume":"30","author":"Zheng","year":"2019","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"3457","DOI":"10.1109\/TCYB.2020.3010331","article-title":"Gated stacked target-related autoencoder: A novel deep feature extraction and layerwise ensemble method for industrial soft sensor application","volume":"52","author":"Sun","year":"2022","journal-title":"IEEE Trans. Cybern."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"447","DOI":"10.1109\/TMM.2023.3266610","article-title":"Hierarchical Locality-Aware Deep Dictionary Learning for Classification","volume":"26","author":"Gou","year":"2024","journal-title":"IEEE Trans. Multimed."},{"key":"ref_20","first-page":"1562","article-title":"A width-growth model with subnetwork nodes and refinement structure for representation learning and image classification","volume":"17","author":"Zhang","year":"2021","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2337","DOI":"10.1109\/TNNLS.2017.2654357","article-title":"A parallel multiclassification algorithm for big data using an extreme learning machine","volume":"29","author":"Duan","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1074","DOI":"10.1109\/TFUZZ.2013.2280141","article-title":"A novel evolutionary kernel intuitionistic fuzzy c-means clustering algorithm","volume":"22","author":"Lin","year":"2014","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"4280","DOI":"10.1002\/int.22459","article-title":"Fuzzy decision trees embedded with evolutionary fuzzy clustering for locating users using wireless signal strength in an indoor environment","volume":"36","author":"Narayanan","year":"2021","journal-title":"Int. J. Intell. Syst."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"400","DOI":"10.1109\/TSMC.2020.2999813","article-title":"Prediction by fuzzy clustering and KNN on validation data with parallel ensemble of interpretable TSK fuzzy classifiers","volume":"52","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","article-title":"Stacked generalization","volume":"5","author":"Wolpert","year":"1992","journal-title":"Neural Netw."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1109\/TSMCB.2003.811113","article-title":"Wavelet support vector machine","volume":"34","author":"Zhang","year":"2004","journal-title":"IEEE Trans. Syst. Man Cybern. Part B (Cybern.)"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1956","DOI":"10.1109\/TBME.2008.919734","article-title":"Support vector machine-based classification scheme for myoelectric control applied to upper limb","volume":"55","author":"Oskoei","year":"2008","journal-title":"IEEE Trans. Biomed. Eng."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"513","DOI":"10.1109\/TSMCB.2011.2168604","article-title":"Extreme learning machine for regression and multiclass classification","volume":"42","author":"Huang","year":"2012","journal-title":"IEEE Trans. Syst. Man Cybern. Part B (Cybern.)"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"1411","DOI":"10.1109\/TNN.2006.880583","article-title":"A fast and accurate online sequential learning algorithm for feedforward networks","volume":"17","author":"Liang","year":"2006","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_30","first-page":"1","article-title":"Multiscale Deep Learning Reparameterized Full Waveform Inversion With the Adjoint Method","volume":"63","author":"Zhao","year":"2025","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"3353","DOI":"10.1109\/TKDE.2016.2609424","article-title":"Online bagging and boosting for imbalanced data streams","volume":"28","author":"Wang","year":"2016","journal-title":"IEEE Trans. Knowl. Data Eng."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Deng, L., and Yu, D. (2011). Deep convex net: A scalable architecture for speech pattern classification. Twelfth Annual Conference of the International Speech Communication Association, ISCA.","DOI":"10.21437\/Interspeech.2011-607"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"1527","DOI":"10.1109\/TSMC.2017.2759090","article-title":"Deep additive least squares support vector machines for classification with model transfer","volume":"49","author":"Wang","year":"2019","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"1207","DOI":"10.1109\/TFUZZ.2016.2604003","article-title":"Deep TSK fuzzy classifier with stacked generalization and triplely concise interpreta-bility guarantee for large data","volume":"25","author":"Zhou","year":"2017","journal-title":"IEEE Trans. Fuzzy Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"3039","DOI":"10.1002\/int.22388","article-title":"Entropy-based hybrid sampling ensemble learning for imbalanced data","volume":"36","author":"Li","year":"2021","journal-title":"Int. J. Intell. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1858","DOI":"10.1109\/TCYB.2014.2298235","article-title":"Sparse extreme learning machine for classification","volume":"44","author":"Bai","year":"2014","journal-title":"IEEE Trans. Cybern."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"1703","DOI":"10.1109\/TSMC.2016.2531674","article-title":"Probabilistic inference-based least squares support vector machine for modeling under noisy environment","volume":"46","author":"Fan","year":"2016","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1117","DOI":"10.1109\/TNSRE.2019.2913142","article-title":"Multiclass support matrix machines by maximizing the inter-class margin for single trial EEG classification","volume":"27","author":"Razzak","year":"2019","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1242","DOI":"10.1109\/TNNLS.2020.3041591","article-title":"Conditional random fields for multiview sequential data modeling","volume":"33","author":"Sun","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_40","unstructured":"Bache, K., and Lichman, M. (2013). UCI Machine Learning Repository, University California, School of Information and Computer Science. Available online: http:\/\/archive.ics.uci.edu\/ml."},{"key":"ref_41","first-page":"255","article-title":"Keel data-mining software tool: Data set repository, integration of algorithms and experimental analysis framework","volume":"17","author":"Derrac","year":"2011","journal-title":"J. Mult.-Valued Log. Soft Comput."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"14522","DOI":"10.1109\/JSEN.2025.3554696","article-title":"Deep Learning for Image Reconstruction in Electrical Tomography: A Review","volume":"25","author":"Li","year":"2025","journal-title":"IEEE Sens. J."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/8\/1362\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:31:43Z","timestamp":1760034703000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/17\/8\/1362"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,20]]},"references-count":42,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,8]]}},"alternative-id":["sym17081362"],"URL":"https:\/\/doi.org\/10.3390\/sym17081362","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2025,8,20]]}}}