{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,11]],"date-time":"2026-01-11T01:25:13Z","timestamp":1768094713983,"version":"3.49.0"},"reference-count":43,"publisher":"MDPI AG","issue":"17","license":[{"start":{"date-parts":[[2021,9,2]],"date-time":"2021-09-02T00:00:00Z","timestamp":1630540800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62076204"],"award-info":[{"award-number":["62076204"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Natural Science Foundation of Shaanxi Province","award":["2018JQ6003"],"award-info":[{"award-number":["2018JQ6003"]}]},{"DOI":"10.13039\/501100007128","name":"Natural Science Foundation of Shaanxi Province","doi-asserted-by":"publisher","award":["2018JQ6030"],"award-info":[{"award-number":["2018JQ6030"]}],"id":[{"id":"10.13039\/501100007128","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"publisher","award":["2017M613204"],"award-info":[{"award-number":["2017M613204"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002858","name":"China Postdoctoral Science Foundation","doi-asserted-by":"publisher","award":["2017M623246"],"award-info":[{"award-number":["2017M623246"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Deep neural networks have achieved significant development and wide applications for their amazing performance. However, their complex structure, high computation and storage resource limit their applications in mobile or embedding devices such as sensor platforms. Neural network pruning is an efficient way to design a lightweight model from a well-trained complex deep neural network. In this paper, we propose an evolutionary multi-objective one-shot filter pruning method for designing a lightweight convolutional neural network. Firstly, unlike some famous iterative pruning methods, a one-shot pruning framework only needs to perform filter pruning and model fine-tuning once. Moreover, we built a constraint multi-objective filter pruning problem in which two objectives represent the filter pruning ratio and the accuracy of the pruned convolutional neural network, respectively. A non-dominated sorting-based evolutionary multi-objective algorithm was used to solve the filter pruning problem, and it provides a set of Pareto solutions which consists of a series of different trade-off pruned models. Finally, some models are uniformly selected from the set of Pareto solutions to be fine-tuned as the output of our method. The effectiveness of our method was demonstrated in experimental studies on four designed models, LeNet and AlexNet. Our method can prune over 85%, 82%, 75%, 65%, 91% and 68% filters with little accuracy loss on four designed models, LeNet and AlexNet, respectively.<\/jats:p>","DOI":"10.3390\/s21175901","type":"journal-article","created":{"date-parts":[[2021,9,2]],"date-time":"2021-09-02T23:05:12Z","timestamp":1630623912000},"page":"5901","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":12,"title":["Evolutionary Multi-Objective One-Shot Filter Pruning for Designing Lightweight Convolutional Neural Network"],"prefix":"10.3390","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6925-109X","authenticated-orcid":false,"given":"Tao","family":"Wu","sequence":"first","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, 127 West Youyi Road, Xi\u2019an 710072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1124-6738","authenticated-orcid":false,"given":"Jiao","family":"Shi","sequence":"additional","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, 127 West Youyi Road, Xi\u2019an 710072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Deyun","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, 127 West Youyi Road, Xi\u2019an 710072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaolong","family":"Zheng","sequence":"additional","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, 127 West Youyi Road, Xi\u2019an 710072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Na","family":"Li","sequence":"additional","affiliation":[{"name":"School of Electronics and Information, Northwestern Polytechnical University, 127 West Youyi Road, Xi\u2019an 710072, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,2]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"ref_2","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_3","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Abdel-Hamid, O., Mohamed, A.R., Jiang, H., and Penn, G. (2012, January 25\u201330). Applying convolutional neural networks concepts to hybrid NN-HMM model for speech recognition. Proceedings of the 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Kyoto, Japan.","DOI":"10.1109\/ICASSP.2012.6288864"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1533","DOI":"10.1109\/TASLP.2014.2339736","article-title":"Convolutional neural networks for speech recognition","volume":"22","author":"Mohamed","year":"2014","journal-title":"IEEE Trans. Audio Speech Lang. Process."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Park, S.R., and Lee, J. (2016). A fully convolutional neural network for speech enhancement. arXiv.","DOI":"10.21437\/Interspeech.2017-1465"},{"key":"ref_8","unstructured":"Canziani, A., Paszke, A., and Culurciello, E. (2016). An analysis of deep neural network models for practical applications. arXiv."},{"key":"ref_9","unstructured":"Denil, M., Shakibi, B., Dinh, L., Ranzato, M., and De Freitas, N. (2013). Predicting parameters in deep learning. arXiv."},{"key":"ref_10","unstructured":"Cheng, Y., Wang, D., Zhou, P., and Zhang, T. (2017). A survey of model compression and acceleration for deep neural networks. arXiv."},{"key":"ref_11","unstructured":"Blalock, D., Ortiz, J.J.G., Frankle, J., and Guttag, J. (2020). What is the state of neural network pruning?. arXiv."},{"key":"ref_12","unstructured":"Han, S., Pool, J., Tran, J., and Dally, W.J. (2015). Learning both weights and connections for efficient neural networks. arXiv."},{"key":"ref_13","unstructured":"Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, H.P. (2016). Pruning filters for efficient convnets. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"He, Y., Zhang, X., and Sun, J. (2017, January 22\u201329). Channel pruning for accelerating very deep neural networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.155"},{"key":"ref_15","unstructured":"Hu, H., Peng, R., Tai, Y.W., and Tang, C.K. (2016). Network trimming: A data-driven neuron pruning approach towards efficient deep architectures. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"He, Y., Kang, G., Dong, X., Fu, Y., and Yang, Y. (2018). Soft filter pruning for accelerating deep convolutional neural networks. arXiv.","DOI":"10.24963\/ijcai.2018\/309"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Luo, J.H., Wu, J., and Lin, W. (2017, January 22\u201329). Thinet: A filter level pruning method for deep neural network compression. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.541"},{"key":"ref_18","unstructured":"Touretzky, D.S. (1990). Optimal brain damage. Advances in Neural Information Processing Systems 2, Morgan Kaufmann Publishers Inc.. Available online: https:\/\/dl.acm.org\/doi\/10.5555\/109230.109298."},{"key":"ref_19","unstructured":"Hassibi, B., and Stork, D.G. (1993). Second Order Derivatives for Network Pruning: Optimal Brain Surgeon, Morgan Kaufmann."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"519","DOI":"10.1109\/72.572092","article-title":"An iterative pruning algorithm for feedforward neural networks","volume":"8","author":"Castellano","year":"1997","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Wu, T., Li, X., Zhou, D., Li, N., and Shi, J. (2021). Differential Evolution Based Layer-Wise Weight Pruning for Compressing Deep Neural Networks. Sensors, 21.","DOI":"10.3390\/s21030880"},{"key":"ref_22","unstructured":"Lee, N., Ajanthan, T., and Torr, P.H. (2018). SNIP: Single-shot network pruning based on connection sensitivity. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Srinivas, S., and Babu, R.V. (2015). Data-free parameter pruning for deep neural networks. arXiv.","DOI":"10.5244\/C.29.31"},{"key":"ref_24","unstructured":"Frankle, J., and Carbin, M. (2018). The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Molchanov, P., Mallya, A., Tyree, S., Frosio, I., and Kautz, J. (2019, January 15\u201320). Importance Estimation for Neural Network Pruning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01152"},{"key":"ref_26","unstructured":"Liu, Z., Sun, M., Zhou, T., Huang, G., and Darrell, T. (2018). Rethinking the value of network pruning. arXiv."},{"key":"ref_27","unstructured":"Hanson, S.J., and Pratt, L.Y. (2021, August 29). Comparing Biases for Minimal Network Construction with Back-Propagation. Available online: https:\/\/proceedings.neurips.cc\/paper\/1988\/file\/1c9ac0159c94d8d0cbedc973445af2da-Paper.pdf."},{"key":"ref_28","unstructured":"Wen, W., Wu, C., Wang, Y., Chen, Y., and Li, H. (2016). Learning structured sparsity in deep neural networks. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1109\/4235.996017","article-title":"A fast and elitist multiobjective genetic algorithm: NSGA-II","volume":"6","author":"Deb","year":"2002","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1109\/TEVC.2004.826067","article-title":"Handling multiple objectives with particle swarm optimization","volume":"8","author":"Coello","year":"2004","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_31","first-page":"287","article-title":"Multi-objective particle swarm optimizers: A survey of the state-of-the-art","volume":"2","author":"Coello","year":"2006","journal-title":"Int. J. Comput. Intell. Res."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1109\/TEVC.2007.892759","article-title":"MOEA\/D: A multiobjective evolutionary algorithm based on decomposition","volume":"11","author":"Zhang","year":"2007","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"827","DOI":"10.1109\/TEVC.2013.2287153","article-title":"An evolutionary multiobjective approach to sparse reconstruction","volume":"18","author":"Li","year":"2014","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"3263","DOI":"10.1109\/TNNLS.2015.2469673","article-title":"A multiobjective sparse feature learning model for deep neural networks","volume":"26","author":"Gong","year":"2015","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"2450","DOI":"10.1109\/TNNLS.2017.2695223","article-title":"Structure learning for deep neural networks based on multiobjective optimization","volume":"29","author":"Liu","year":"2018","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1109\/TEVC.2018.2850769","article-title":"Decomposition-Based Evolutionary Multi-objective Optimization to Self-paced Learning","volume":"23","author":"Gong","year":"2018","journal-title":"IEEE Trans. Evol. Comput."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"172","DOI":"10.1016\/j.ins.2018.08.037","article-title":"A multi-objective memetic algorithm for low rank and sparse matrix decomposition","volume":"468","author":"Wu","year":"2018","journal-title":"Inf. Sci."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Lu, Z., Deb, K., Goodman, E., Banzhaf, W., and Boddeti, V.N. (2020). NSGANetV2: Evolutionary multi-objective surrogate-assisted neural architecture search. Computer Vision\u2014ECCV 2020, Springer.","DOI":"10.1007\/978-3-030-58452-8_3"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Yang, Z., Wang, Y., Chen, X., Shi, B., Xu, C., Xu, C., Tian, Q., and Xu, C. (2020, January 13\u201319). CARS: Continuous evolution for efficient neural architecture search. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00190"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Wu, T., Shi, J., Zhou, D., Lei, Y., and Gong, M. (2019, January 10\u201313). A Multi-objective Particle Swarm Optimization for Neural Networks Pruning. Proceedings of the 2019 IEEE Congress on Evolutionary Computation (CEC), Wellington, New Zealand.","DOI":"10.1109\/CEC.2019.8790145"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"89497","DOI":"10.1109\/ACCESS.2020.2990567","article-title":"Pymoo: Multi-Objective Optimization in Python","volume":"8","author":"Blank","year":"2020","journal-title":"IEEE Access"},{"key":"ref_43","unstructured":"(2021, August 29). Keras. Available online: https:\/\/keras.io\/getting_started\/faq\/."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5901\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:54:56Z","timestamp":1760165696000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/21\/17\/5901"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,2]]},"references-count":43,"journal-issue":{"issue":"17","published-online":{"date-parts":[[2021,9]]}},"alternative-id":["s21175901"],"URL":"https:\/\/doi.org\/10.3390\/s21175901","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,2]]}}}