{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:13:21Z","timestamp":1760145201075,"version":"build-2065373602"},"reference-count":26,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2024,6,30]],"date-time":"2024-06-30T00:00:00Z","timestamp":1719705600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&amp;D Program of China","doi-asserted-by":"publisher","award":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"],"award-info":[{"award-number":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Natural Science Foundation of Shanghai","award":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"],"award-info":[{"award-number":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"],"award-info":[{"award-number":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Shanghai Municipal of Science and Technology Major","award":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"],"award-info":[{"award-number":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"]}]},{"name":"HPC of School of Mathematical Sciences and the Student Innovation Center","award":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"],"award-info":[{"award-number":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"]}]},{"name":"Siyuan-1 cluster supported by the Center for High Performance Computing at Shanghai Jiao Tong University","award":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"],"award-info":[{"award-number":["2022YFA1008200","20ZR1429000","62002221","2021SHZDZX0102"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Neural networks have been extensively applied to a variety of tasks, achieving astounding results. Applying neural networks in the scientific field is an important research direction that is gaining increasing attention. In scientific applications, the scale of neural networks is generally moderate size, mainly to ensure the speed of inference during application. Additionally, comparing neural networks to traditional algorithms in scientific applications is inevitable. These applications often require rapid computations, making the reduction in neural network sizes increasingly important. Existing work has found that the powerful capabilities of neural networks are primarily due to their nonlinearity. Theoretical work has discovered that under strong nonlinearity, neurons in the same layer tend to behave similarly, a phenomenon known as condensation. Condensation offers an opportunity to reduce the scale of neural networks to a smaller subnetwork with a similar performance. In this article, we propose a condensation reduction method to verify the feasibility of this idea in practical problems, thereby validating existing theories. Our reduction method can currently be applied to both fully connected networks and convolutional networks, achieving positive results. In complex combustion acceleration tasks, we reduced the size of the neural network to 41.7% of its original scale while maintaining prediction accuracy. In the CIFAR10 image classification task, we reduced the network size to 11.5% of the original scale, still maintaining a satisfactory validation accuracy. Our method can be applied to most trained neural networks, reducing computational pressure and improving inference speed.<\/jats:p>","DOI":"10.3390\/e26070567","type":"journal-article","created":{"date-parts":[[2024,7,2]],"date-time":"2024-07-02T05:08:34Z","timestamp":1719896914000},"page":"567","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Efficient and Flexible Method for Reducing Moderate-Size Deep Neural Networks with Condensation"],"prefix":"10.3390","volume":"26","author":[{"given":"Tianyi","family":"Chen","sequence":"first","affiliation":[{"name":"School of Mathematical Sciences, Institute of Natural Sciences, MOE-LSC, Shanghai Jiao Tong University, Shanghai 200240, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhi-Qin John","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Mathematical Sciences, Institute of Natural Sciences, MOE-LSC, Shanghai Jiao Tong University, Shanghai 200240, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,6,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1038\/s43246-022-00315-6","article-title":"Graph neural networks for materials science and chemistry","volume":"3","author":"Reiser","year":"2022","journal-title":"Commun. Mater."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1007\/s12065-020-00540-3","article-title":"Convolutional neural networks in medical image understanding: A survey","volume":"15","author":"Sarvamangala","year":"2022","journal-title":"Evol. Intell."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"021001","DOI":"10.1088\/2632-2153\/abbf9a","article-title":"Graph neural networks in particle physics","volume":"2","author":"Shlomi","year":"2020","journal-title":"Mach. Learn. Sci. Technol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"221454","DOI":"10.1098\/rsos.221454","article-title":"Astronomia ex machina: A history, primer and outlook on neural networks in astronomy","volume":"10","author":"Smith","year":"2023","journal-title":"R. Soc. Open Sci."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Zhang, X.M., Liang, L., Liu, L., and Tang, M.J. (2021). Graph neural networks and their current applications in bioinformatics. Front. Genet., 12.","DOI":"10.3389\/fgene.2021.690049"},{"key":"ref_6","first-page":"12741","article-title":"Machine learning: New ideas and tools in environmental science and engineering","volume":"55","author":"Zhong","year":"2021","journal-title":"Environ. Sci. Technol."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"686","DOI":"10.1016\/j.jcp.2018.10.045","article-title":"Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations","volume":"378","author":"Raissi","year":"2019","journal-title":"J. Comput. Phys."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"e202100006","DOI":"10.1002\/gamm.202100006","article-title":"Three ways to solve partial differential equations with neural networks\u2014A review","volume":"44","author":"Blechschmidt","year":"2021","journal-title":"GAMM-Mitteilungen"},{"key":"ref_9","unstructured":"Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. (2020). Neural operator: Graph kernel network for partial differential equations. arXiv."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"193","DOI":"10.1016\/j.neucom.2020.02.015","article-title":"Solving differential equations using deep neural networks","volume":"399","author":"Michoski","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_11","unstructured":"Xu, Z.Q.J., Yao, J., Yi, Y., Hang, L., Zhang, Y., and Zhang, T. (2024). Solving multiscale dynamical systems by deep learning. arXiv."},{"key":"ref_12","unstructured":"Liu, Z., Sun, M., Zhou, T., Huang, G., and Darrell, T. (2018). Rethinking the value of network pruning. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Wang, Z., Li, C., and Wang, X. (2021, January 20\u201325). Convolutional neural network pruning with structural redundancy reduction. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.01467"},{"key":"ref_14","unstructured":"Wesseling, P. (1995). Introduction to Multigrid Methods, NASA. Technical Report."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"20200097","DOI":"10.1098\/rspa.2020.0097","article-title":"Shallow neural networks for fluid flow reconstruction with limited sensors","volume":"476","author":"Erichson","year":"2020","journal-title":"Proc. R. Soc. A"},{"key":"ref_16","first-page":"1","article-title":"Resource-efficient neural networks for embedded systems","volume":"25","author":"Roth","year":"2024","journal-title":"J. Mach. Learn. Res."},{"key":"ref_17","first-page":"1","article-title":"Phase diagram for two-layer relu neural networks at infinite-width limit","volume":"22","author":"Luo","year":"2021","journal-title":"J. Mach. Learn. Res."},{"key":"ref_18","first-page":"14848","article-title":"Embedding principle of loss landscape of deep neural networks","volume":"34","author":"Zhang","year":"2021","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_19","unstructured":"Frankle, J., and Carbin, M. (2018). The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"e20767","DOI":"10.2196\/20767","article-title":"Using Artificial Neural Network Condensation to Facilitate Adaptation of Machine Learning in Medical Settings by Reducing Computational Burden: Model Design and Evaluation Study","volume":"5","author":"Liu","year":"2021","journal-title":"JMIR Form. Res."},{"key":"ref_21","unstructured":"Cheng, H., Zhang, M., and Shi, J.Q. (2023). A survey on deep neural network pruning-taxonomy, comparison, analysis, and recommendations. arXiv."},{"key":"ref_22","unstructured":"Hanson, S., and Pratt, L. (1988). Comparing biases for minimal network construction with back-propagation. Adv. Neural Inf. Process. Syst., 1."},{"key":"ref_23","unstructured":"LeCun, Y., Denker, J., and Solla, S. (1989). Optimal brain damage. Adv. Neural Inf. Process. Syst., 2, Available online: https:\/\/proceedings.neurips.cc\/paper_files\/paper\/1989\/file\/6c9882bbac1c7093bd25041881277658-Paper.pdf."},{"key":"ref_24","unstructured":"You, Z., Yan, K., Ye, J., Ma, M., and Wang, P. (2019). Gate decorator: Global filter pruning method for accelerating deep convolutional neural networks. Adv. Neural Inf. Process. Syst., 32, Available online: https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2019\/file\/b51a15f382ac914391a58850ab343b00-Paper.pdf."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., and Chen, L.C. (2018, January 18\u201323). Mobilenetv2: Inverted residuals and linear bottlenecks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00474"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"997","DOI":"10.1007\/s10494-019-00081-5","article-title":"Quasi-DNS dataset of a piloted flame with inhomogeneous inlet conditions","volume":"104","author":"Zirwes","year":"2020","journal-title":"Flow Turbul. Combust."}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/7\/567\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:08:09Z","timestamp":1760108889000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/26\/7\/567"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,30]]},"references-count":26,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2024,7]]}},"alternative-id":["e26070567"],"URL":"https:\/\/doi.org\/10.3390\/e26070567","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2024,6,30]]}}}