{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,9]],"date-time":"2026-07-09T22:38:37Z","timestamp":1783636717849,"version":"3.55.0"},"reference-count":36,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2021,1,28]],"date-time":"2021-01-28T00:00:00Z","timestamp":1611792000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Deep neural networks have evolved significantly in the past decades and are now able to achieve better progression of sensor data. Nonetheless, most of the deep models verify the ruling maxim in deep learning\u2014bigger is better\u2014so they have very complex structures. As the models become more complex, the computational complexity and resource consumption of these deep models are increasing significantly, making them difficult to perform on resource-limited platforms, such as sensor platforms. In this paper, we observe that different layers often have different pruning requirements, and propose a differential evolutionary layer-wise weight pruning method. Firstly, the pruning sensitivity of each layer is analyzed, and then the network is compressed by iterating the weight pruning process. Unlike some other methods that deal with pruning ratio by greedy ways or statistical analysis, we establish an optimization model to find the optimal pruning sensitivity set for each layer. Differential evolution is an effective method based on population optimization which can be used to address this task. Furthermore, we adopt a strategy to recovery some of the removed connections to increase the capacity of the pruned model during the fine-tuning phase. The effectiveness of our method has been demonstrated in experimental studies. Our method compresses the number of weight parameters in LeNet-300-100, LeNet-5, AlexNet and VGG16 by 24\u00d7, 14\u00d7, 29\u00d7 and 12\u00d7, respectively.<\/jats:p>","DOI":"10.3390\/s21030880","type":"journal-article","created":{"date-parts":[[2021,1,28]],"date-time":"2021-01-28T09:03:45Z","timestamp":1611824625000},"page":"880","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":35,"title":["Differential Evolution Based Layer-Wise Weight Pruning for Compressing Deep Neural Networks"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6925-109X","authenticated-orcid":false,"given":"Tao","family":"Wu","sequence":"first","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, 127 West Youyi Road, Xi\u2019an 710072, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaoyang","family":"Li","sequence":"additional","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, 127 West Youyi Road, Xi\u2019an 710072, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Deyun","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, 127 West Youyi Road, Xi\u2019an 710072, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Na","family":"Li","sequence":"additional","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, 127 West Youyi Road, Xi\u2019an 710072, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1124-6738","authenticated-orcid":false,"given":"Jiao","family":"Shi","sequence":"additional","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, 127 West Youyi Road, Xi\u2019an 710072, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,28]]},"reference":[{"key":"ref_1","unstructured":"Canziani, A., Paszke, A., and Culurciello, E. (2016). An analysis of deep neural network models for practical applications. arXiv."},{"key":"ref_2","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012). Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_3","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_6","unstructured":"LeCun, Y., Denker, J.S., and Solla, S.A. (1990). Optimal brain damage. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_7","unstructured":"Hassibi, B., and Stork, D.G. (1993). Second order derivatives for network pruning: Optimal brain surgeon. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"389","DOI":"10.1142\/S012906570800166X","article-title":"Pruning artificial neural networks using neural complexity measures","volume":"18","author":"Jorgensen","year":"2008","journal-title":"Int. J. Neural Syst."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Molchanov, P., Mallya, A., Tyree, S., Frosio, I., and Kautz, J. (2019, January 15\u201320). Importance Estimation for Neural Network Pruning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01152"},{"key":"ref_10","unstructured":"Liu, Z., Sun, M., Zhou, T., Huang, G., and Darrell, T. (2018). Rethinking the value of network pruning. arXiv."},{"key":"ref_11","unstructured":"Lee, N., Ajanthan, T., and Torr, P.H. (2018). SNIP: Single-shot network pruning based on connection sensitivity. arXiv."},{"key":"ref_12","unstructured":"Han, S., Pool, J., Tran, J., and Dally, W. (2015). Learning both weights and connections for efficient neural network. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_13","unstructured":"Dong, X., Chen, S., and Pan, S. (2017). Learning to prune deep neural networks via layer-wise optimal brain surgeon. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1109\/72.572092","article-title":"An iterative pruning algorithm for feedforward neural networks","volume":"8","author":"Castellano","year":"1997","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"76","DOI":"10.1007\/BF01501173","article-title":"Pruning backpropagation neural networks using modern stochastic optimisation techniques","volume":"5","author":"Stepniewski","year":"1997","journal-title":"Neural Comput. Appl."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"32:1","DOI":"10.1145\/3005348","article-title":"Structured pruning of deep convolutional neural networks","volume":"13","author":"Anwar","year":"2017","journal-title":"ACM J. Emerg. Technol. Comput. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"He, Y., Zhang, X., and Sun, J. (2017, January 22\u201329). Channel pruning for accelerating very deep neural networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.155"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1109\/72.248452","article-title":"Pruning algorithms\u2014A survey","volume":"4","author":"Reed","year":"1993","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_19","unstructured":"Molchanov, P., Tyree, S., Karras, T., Aila, T., and Kautz, J. (2016). Pruning convolutional neural networks for resource efficient transfer learning. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1023\/A:1008202821328","article-title":"Differential evolution\u2014A simple and efficient heuristic for global optimization over continuous spaces","volume":"11","author":"Storn","year":"1997","journal-title":"J. Glob. Optim."},{"key":"ref_21","unstructured":"Price, K., Storn, R.M., and Lampinen, J.A. (2006). Differential Evolution: A Practical Approach to Global Optimization, Springer Science & Business Media."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"398","DOI":"10.1109\/TEVC.2008.927706","article-title":"Differential evolution algorithm with strategy adaptation for global numerical optimization","volume":"13","author":"Qin","year":"2009","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_23","unstructured":"Denil, M., Shakibi, B., Dinh, L., Ranzato, M., and De Freitas, N. (2013). Predicting parameters in deep learning. Advances in Neural Information Processing Systems, The MIT Press."},{"key":"ref_24","unstructured":"Han, S., Mao, H., and Dally, W.J. (2015). Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv."},{"key":"ref_25","unstructured":"Hu, H., Peng, R., Tai, Y.W., and Tang, C.K. (2016). Network trimming: A data-driven neuron pruning approach towards efficient deep architectures. arXiv."},{"key":"ref_26","unstructured":"Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, H.P. (2016). Pruning filters for efficient convnets. arXiv."},{"key":"ref_27","unstructured":"Chen, W., Wilson, J., Tyree, S., Weinberger, K., and Chen, Y. (2015, January 6\u201311). Compressing neural networks with the hashing trick. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_28","unstructured":"Dettmers, T. (2015). 8-bit approximations for parallelism in deep learning. arXiv."},{"key":"ref_29","unstructured":"Gupta, S., Agrawal, A., Gopalakrishnan, K., and Narayanan, P. (2015, January 6\u201311). Deep learning with limited numerical precision. Proceedings of the International Conference on Machine Learning, Lille, France."},{"key":"ref_30","unstructured":"Hinton, G., Vinyals, O., and Dean, J. (2015). Distilling the knowledge in a neural network. arXiv."},{"key":"ref_31","unstructured":"Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., and Bengio, Y. (2014). Fitnets: Hints for thin deep nets. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Yim, J., Joo, D., Bae, J., and Kim, J. (2017, January 21\u201326). A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.754"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_34","unstructured":"Krizhevsky, A., and Hinton, G. (2009). Learning Multiple Layers of Features from Tiny Images, Citeseer. Technical report."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Srinivas, S., and Babu, R.V. (2015). Data-free parameter pruning for deep neural networks. arXiv.","DOI":"10.5244\/C.29.31"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Wu, T., Shi, J., Zhou, D., Lei, Y., and Gong, M. (2019, January 10\u201313). A Multi-objective Particle Swarm Optimization for Neural Networks Pruning. Proceedings of the 2019 IEEE Congress on Evolutionary Computation, Wellington, New Zealand.","DOI":"10.1109\/CEC.2019.8790145"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/3\/880\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:16:38Z","timestamp":1760159798000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/3\/880"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,28]]},"references-count":36,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2021,2]]}},"alternative-id":["s21030880"],"URL":"https:\/\/doi.org\/10.3390\/s21030880","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,1,28]]}}}