{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T15:39:47Z","timestamp":1766158787896,"version":"build-2065373602"},"reference-count":32,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2021,8,13]],"date-time":"2021-08-13T00:00:00Z","timestamp":1628812800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Jilin Province Science and Technology Development Plan Project","award":["No. 20190201273JC"],"award-info":[{"award-number":["No. 20190201273JC"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["Nos. 62072212, 61772227"],"award-info":[{"award-number":["Nos. 62072212, 61772227"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Jilin Provincial Key Laboratory of Big Data Intelligent Computing","award":["No. 20180622002JC"],"award-info":[{"award-number":["No. 20180622002JC"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Entropy"],"abstract":"<jats:p>Deep neural networks may achieve excellent performance in many research fields. However, many deep neural network models are over-parameterized. The computation of weight matrices often consumes a lot of time, which requires plenty of computing resources. In order to solve these problems, a novel block-based division method and a special coarse-grained block pruning strategy are proposed in this paper to simplify and compress the fully connected structure, and the pruned weight matrices with a blocky structure are then stored in the format of Block Sparse Row (BSR) to accelerate the calculation of the weight matrices. First, the weight matrices are divided into square sub-blocks based on spatial aggregation. Second, a coarse-grained block pruning procedure is utilized to scale down the model parameters. Finally, the BSR storage format, which is much more friendly to block sparse matrix storage and computation, is employed to store these pruned dense weight blocks to speed up the calculation. In the following experiments on MNIST and Fashion-MNIST datasets, the trend of accuracies with different pruning granularities and different sparsity is explored in order to analyze our method. The experimental results show that our coarse-grained block pruning method can compress the network and can reduce the computational cost without greatly degrading the classification accuracy. The experiment on the CIFAR-10 dataset shows that our block pruning strategy can combine well with the convolutional networks.<\/jats:p>","DOI":"10.3390\/e23081042","type":"journal-article","created":{"date-parts":[[2021,8,13]],"date-time":"2021-08-13T05:34:46Z","timestamp":1628832886000},"page":"1042","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Coarse-Grained Pruning of Neural Network Models Based on Blocky Sparse Structure"],"prefix":"10.3390","volume":"23","author":[{"given":"Lan","family":"Huang","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"},{"name":"Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education, Jilin University, Changchun 130012, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jia","family":"Zeng","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shiqi","family":"Sun","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wencong","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4751-0708","authenticated-orcid":false,"given":"Yan","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"},{"name":"Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education, Jilin University, Changchun 130012, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4402-3346","authenticated-orcid":false,"given":"Kangping","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Jilin University, Changchun 130012, China"},{"name":"Key Laboratory of Symbolic Computation and Knowledge Engineering of the Ministry of Education, Jilin University, Changchun 130012, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,8,13]]},"reference":[{"doi-asserted-by":"crossref","unstructured":"Hu, J., Shen, L., and Sun, G. (2018, January 18\u201322). Squeeze-and-Excitation Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","key":"ref_1","DOI":"10.1109\/CVPR.2018.00745"},{"unstructured":"Real, E., Aggarwal, A., Huang, Y., and Le, Q.V. (February, January 27). Regularized Evolution for Image Classifier Architecture Search. Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, Honolulu, HI, USA.","key":"ref_2"},{"unstructured":"Ioannou, Y., Robertson, D.P., Shotton, J., Cipolla, R., and Criminisi, A. (2016, January 2\u20134). Training CNNs with Low-Rank Filters for Efficient Image Classification. Proceedings of the 4th International Conference on Learning Representations, San Juan, Puerto Rico.","key":"ref_3"},{"doi-asserted-by":"crossref","unstructured":"Luo, P., Zhu, Z., Liu, Z., Wang, X., and Tang, X. (2016, January 12\u201317). Face Model Compression by Distilling Knowledge from Neurons. Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ, USA.","key":"ref_4","DOI":"10.1609\/aaai.v30i1.10449"},{"unstructured":"Ullrich, K., Meeds, E., and Welling, M. (2017, January 24\u201326). Soft Weight-Sharing for Neural Network Compression. Proceedings of the 5th International Conference on Learning Representations, Toulon, France.","key":"ref_5"},{"unstructured":"Han, S., Pool, J., Tran, J., and Dally, W.J. (2015, January 7\u201312). Learning both Weights and Connections for Efficient Neural Network. Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems, Montreal, QC, Canada.","key":"ref_6"},{"unstructured":"Frankle, J., and Carbin, M. (2019, January 6\u20139). The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. Proceedings of the 7th International Conference on Learning Representations, New Orleans, LA, USA.","key":"ref_7"},{"doi-asserted-by":"crossref","unstructured":"Mao, H., Han, S., Pool, J., Li, W., Liu, X., Wang, Y., and Dally, W.J. (2017). Exploring the Regularity of Sparse Structure in Convolutional Neural Networks. arXiv.","key":"ref_8","DOI":"10.1109\/CVPRW.2017.241"},{"key":"ref_9","first-page":"3","article-title":"Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding","volume":"56","author":"Han","year":"2015","journal-title":"Fiber"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.neunet.2019.04.021","article-title":"Redundant feature pruning for accelerated inference in deep neural networks","volume":"118","author":"Ayinde","year":"2019","journal-title":"Neural Networks"},{"doi-asserted-by":"crossref","unstructured":"Wu, T., Shi, J., Zhou, D., Lei, Y., and Gong, M. (2019, January 10\u201313). A Multi-objective Particle Swarm Optimization for Neural Networks Pruning. Proceedings of the IEEE Congress on Evolutionary Computation, Wellington, New Zealand.","key":"ref_11","DOI":"10.1109\/CEC.2019.8790145"},{"unstructured":"Lee, N., Ajanthan, T., and Torr, P.H.S. (2019, January 6\u20139). Snip: Single-Shot Network Pruning based on Connection sensitivity. Proceedings of the 7th International Conference on Learning Representations, New Orleans, LA, USA.","key":"ref_12"},{"unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G.E. (2012, January 3\u20136). ImageNet Classification with Deep Convolutional Neural Networks. Proceedings of the Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems, Lake Tahoe, NV, USA.","key":"ref_13"},{"unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very Deep Convolutional Networks for Large-Scale Image Recognition. Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA.","key":"ref_14"},{"unstructured":"Lee, N., Ajanthan, T., Gould, S., and Torr, P.H.S. (2020, January 26\u201330). A Signal Propagation Perspective for Pruning Neural Networks at Initialization. Proceedings of the 8th International Conference on Learning Representations, Addis Ababa, Ethiopia.","key":"ref_15"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1561\/2200000016","article-title":"Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers","volume":"3","author":"Boyd","year":"2011","journal-title":"Found. Trends Mach. Learn."},{"doi-asserted-by":"crossref","unstructured":"Ren, A., Zhang, T., Ye, S., Li, J., Xu, W., Qian, X., Lin, X., and Wang, Y. (2019, January 13\u201317). ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Methods of Multipliers. Proceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, Providence, RI, USA.","key":"ref_17","DOI":"10.1145\/3297858.3304076"},{"unstructured":"Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, H.P. (2017, January 24\u201326). Pruning Filters for Efficient ConvNets. Proceedings of the 5th International Conference on Learning Representations, Toulon, France.","key":"ref_18"},{"unstructured":"Wen, W., Wu, C., Wang, Y., Chen, Y., and Li, H. (2016, January 5\u201310). Learning Structured Sparsity in Deep Neural Networks. Proceedings of the Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, Barcelona, Spain.","key":"ref_19"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1109\/TNNLS.2019.2906563","article-title":"Toward Compact ConvNets via Structure-Sparsity Regularized Filter Pruning","volume":"31","author":"Lin","year":"2020","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"doi-asserted-by":"crossref","unstructured":"Ma, X., Guo, F., Niu, W., Lin, X., Tang, J., Ma, K., Ren, B., and Wang, Y. (2020, January 7\u201312). PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-Time Execution on Mobile Devices. Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence, New York, NY, USA.","key":"ref_21","DOI":"10.1609\/aaai.v34i04.5954"},{"doi-asserted-by":"crossref","unstructured":"He, Y., Liu, P., Wang, Z., Hu, Z., and Yang, Y. (2019, January 16\u201320). Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","key":"ref_22","DOI":"10.1109\/CVPR.2019.00447"},{"doi-asserted-by":"crossref","unstructured":"Lin, M., Ji, R., Wang, Y., Zhang, Y., Zhang, B., Tian, Y., and Shao, L. (2020, January 13\u201319). HRank: Filter Pruning Using High-Rank Feature Map. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","key":"ref_23","DOI":"10.1109\/CVPR42600.2020.00160"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.neucom.2021.04.063","article-title":"CCPrune: Collaborative channel pruning for learning compact convolutional networks","volume":"451","author":"Chen","year":"2021","journal-title":"Neurocomputing"},{"unstructured":"Bell, N., and Garland, M. (2008). Efficient Sparse Matrix-Vector Multiplication on CUDA, Nvidia Corporation. NVIDIA Technical Report NVR-2008-004.","key":"ref_25"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"2623","DOI":"10.1109\/TC.2014.2366731","article-title":"Performance Optimization Using Partitioned SpMV on GPUs and Multicore CPUs","volume":"64","author":"Yang","year":"2015","journal-title":"IEEE Trans. Computers"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1016\/j.jpdc.2016.12.023","article-title":"A hybrid computing method of SpMV on CPU-GPU heterogeneous computing systems","volume":"104","author":"Yang","year":"2017","journal-title":"J. Parallel Distributed Comput."},{"doi-asserted-by":"crossref","unstructured":"Bell, N., and Garland, M. (2009, January 14\u201320). Implementing sparse matrix-vector multiplication on throughput-oriented processors. Proceedings of the ACM\/IEEE Conference on High Performance Computing, Portland, OR, USA.","key":"ref_28","DOI":"10.1145\/1654059.1654078"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-based learning applied to document recognition","volume":"86","author":"LeCun","year":"1998","journal-title":"Proc. IEEE"},{"doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","key":"ref_30","DOI":"10.1109\/CVPR.2016.90"},{"unstructured":"Zhou, H., Lan, J., Liu, R., and Yosinski, J. (2019, January 8\u201314). Deconstructing Lottery Tickets: Zeros, Signs, and the Supermask. Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems, Vancouver, BC, Canada.","key":"ref_31"},{"unstructured":"Naumov, M., Chien, L., Vandermersch, P., and Kapasi, U. (2010, January 20\u201323). Cusparse library. Proceedings of the GPU Technology Conference, San Jose, CA, USA.","key":"ref_32"}],"container-title":["Entropy"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/8\/1042\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T06:45:15Z","timestamp":1760165115000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1099-4300\/23\/8\/1042"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,13]]},"references-count":32,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2021,8]]}},"alternative-id":["e23081042"],"URL":"https:\/\/doi.org\/10.3390\/e23081042","relation":{},"ISSN":["1099-4300"],"issn-type":[{"type":"electronic","value":"1099-4300"}],"subject":[],"published":{"date-parts":[[2021,8,13]]}}}