{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T15:22:51Z","timestamp":1793114571493,"version":"4.5.0"},"reference-count":55,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2023,1,11]],"date-time":"2023-01-11T00:00:00Z","timestamp":1673395200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Key Program Project of Science and Technology Innovation of the Chinese Academy of Sciences","award":["KGFZD-135-20-03-02"],"award-info":[{"award-number":["KGFZD-135-20-03-02"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>To address the problems of large storage requirements, computational pressure, untimely data supply of off-chip memory, and low computational efficiency during hardware deployment due to the large number of convolutional neural network (CNN) parameters, we developed an innovative hardware-friendly CNN pruning method called KRP, which prunes the convolutional kernel on a row scale. A new retraining method based on LR tracking was used to obtain a CNN model with both a high pruning rate and accuracy. Furthermore, we designed a high-performance convolutional computation module on the FPGA platform to help deploy KRP pruning models. The results of comparative experiments on CNNs such as VGG and ResNet showed that KRP has higher accuracy than most pruning methods. At the same time, the KRP method, together with the GSNQ quantization method developed in our previous study, forms a high-precision hardware-friendly network compression framework that can achieve \u201clossless\u201d CNN compression with a 27\u00d7 reduction in network model storage. The results of the comparative experiments on the FPGA showed that the KRP pruning method not only requires much less storage space, but also helps to reduce the on-chip hardware resource consumption by more than half and effectively improves the parallelism of the model in FPGAs with a strong hardware-friendly feature. This study provides more ideas for the application of CNNs in the field of edge computing.<\/jats:p>","DOI":"10.3390\/s23020824","type":"journal-article","created":{"date-parts":[[2023,1,11]],"date-time":"2023-01-11T04:59:58Z","timestamp":1673413198000},"page":"824","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":30,"title":["A Hardware-Friendly High-Precision CNN Pruning Method and Its FPGA Implementation"],"prefix":"10.3390","volume":"23","author":[{"given":"Xuefu","family":"Sui","sequence":"first","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"},{"name":"School of Optoelectronics, University of Chinese Academy of Sciences, No. 19(A) Yuquan Road, Shijingshan District, Beijing 100049, China"},{"name":"Department of Key Laboratory of Computational Optical Imagine Technology, CAS, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qunbo","family":"Lv","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"},{"name":"School of Optoelectronics, University of Chinese Academy of Sciences, No. 19(A) Yuquan Road, Shijingshan District, Beijing 100049, China"},{"name":"Department of Key Laboratory of Computational Optical Imagine Technology, CAS, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3118-2017","authenticated-orcid":false,"given":"Liangjie","family":"Zhi","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"},{"name":"School of Optoelectronics, University of Chinese Academy of Sciences, No. 19(A) Yuquan Road, Shijingshan District, Beijing 100049, China"},{"name":"Department of Key Laboratory of Computational Optical Imagine Technology, CAS, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Baoyu","family":"Zhu","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"},{"name":"School of Optoelectronics, University of Chinese Academy of Sciences, No. 19(A) Yuquan Road, Shijingshan District, Beijing 100049, China"},{"name":"Department of Key Laboratory of Computational Optical Imagine Technology, CAS, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yuanbo","family":"Yang","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"},{"name":"School of Optoelectronics, University of Chinese Academy of Sciences, No. 19(A) Yuquan Road, Shijingshan District, Beijing 100049, China"},{"name":"Department of Key Laboratory of Computational Optical Imagine Technology, CAS, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"},{"name":"School of Optoelectronics, University of Chinese Academy of Sciences, No. 19(A) Yuquan Road, Shijingshan District, Beijing 100049, China"},{"name":"Department of Key Laboratory of Computational Optical Imagine Technology, CAS, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zheng","family":"Tan","sequence":"additional","affiliation":[{"name":"Aerospace Information Research Institute, Chinese Academy of Sciences, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"},{"name":"Department of Key Laboratory of Computational Optical Imagine Technology, CAS, No. 9 Dengzhuang South Road, Haidian District, Beijing 100094, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,1,11]]},"reference":[{"key":"ref_1","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1137","DOI":"10.1109\/TPAMI.2016.2577031","article-title":"Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks","volume":"39","author":"Ren","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_3","unstructured":"Duan, K., Bai, S., Xie, L., Qi, H., Huang, Q., and Tian, Q. (November, January 27). CenterNet: Keypoint Triplets for Object Detection. Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Zhao, H., Qi, X., Shen, X., Shi, J., and Jia, J. (2018). ICNet for Real-Time Semantic Segmentation on High-Resolution Images, Springer.","DOI":"10.1007\/978-3-030-01219-9_25"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Lane, N.D., Bhattacharya, S., Georgiev, P., Forlivesi, C., and Kawsar, F. (2015, January 1). An Early Resource Characterization of Deep Learning on Wearables, Smartphones and Internet-of-Things Devices. Proceedings of the 2015 International Workshop on Internet of Things towards Applications, Seoul, Republic of Korea.","DOI":"10.1145\/2820975.2820980"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Bhattacharya, S., and Lane, N.D. (2016, January 14\u201318). From smart to deep: Robust activity recognition on smartwatches using deep learning. Proceedings of the IEEE International Conference on Pervasive Computing & Communication Workshops, Sydney, Australia.","DOI":"10.1109\/PERCOMW.2016.7457169"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"2073","DOI":"10.1109\/TII.2018.2866518","article-title":"Nonlinear Output Feedback Finite-Time Control for Vehicle Active Suspension Systems","volume":"15","author":"Pan","year":"2019","journal-title":"IEEE Trans. Ind. Inform."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"1769","DOI":"10.1109\/TCSI.2018.2881162","article-title":"ChipNet: Real-Time LiDAR Processing for Drivable Region Segmentation on an FPGA","volume":"66","author":"Lyu","year":"2019","journal-title":"IEEE Trans. Circuits Syst. I Regul. Pap."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Buslaev, A., Seferbekov, S., Iglovikov, V., and Shvets, A. (2018, January 18\u201322). Fully Convolutional Network for Automatic Road Extraction from Satellite Imagery. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00035"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"4069","DOI":"10.1109\/TNNLS.2021.3055814","article-title":"FPGA-Based High-Throughput CNN Hardware Accelerator With High Computing Resource Utilization Ratio","volume":"33","author":"Huang","year":"2022","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_11","unstructured":"Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., and Adam, H. (2017). MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv."},{"key":"ref_12","first-page":"1135","article-title":"Learning both Weights and Connections for Efficient Neural Networks","volume":"Volume 28","author":"Cortes","year":"2015","journal-title":"Advances in Neural Information Processing Systems 28"},{"key":"ref_13","unstructured":"Gale, T., Elsen, E., and Hooker, S. (2019). The State of Sparsity in Deep Neural Networks. arXiv."},{"key":"ref_14","unstructured":"Mocanu, D.C., Lu, Y., and Pechenizkiy, M. (2021, January 18\u201324). Do We Actually Need Dense Over-Parameterization? In-Time Over-Parameterization in Sparse Training. Proceedings of the International Conference on Machine Learning, Online."},{"key":"ref_15","unstructured":"Jorge, P.D., Sanyal, A., Behl, H.S., Torr, P., and Dokania, P.K. (2021). Progressive Skeletonization: Trimming more fat from a network at initialization. arXiv."},{"key":"ref_16","unstructured":"Wen, W., Wu, C., Wang, Y., Chen, Y., and Li, H. (2016). Learning Structured Sparsity in Deep Neural Networks. arXiv."},{"key":"ref_17","unstructured":"Li, H., Asim, K., and Igor, D. (2017, January 24\u201326). Pruning Filters for Efficient ConvNets. Proceedings of the International Conference on Learning Representations, Toulon, France."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"He, Y., Kang, G., Dong, X., Fu, Y., and Yang, Y. (2018). Soft Filter Pruning for Accelerating Deep Convolutional Neural Networks. arXiv.","DOI":"10.24963\/ijcai.2018\/309"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., and Zhang, C. (2017, January 22\u201329). Learning Efficient Convolutional Networks through Network Slimming. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.298"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Ding, X., Hao, T., Tan, J., Liu, J., Han, J., Guo, Y., and Ding, G. (2021, January 11\u201317). ResRep: Lossless CNN Pruning via Decoupling Remembering and Forgetting. Proceedings of the International Conference on Computer Vision, Online.","DOI":"10.1109\/ICCV48922.2021.00447"},{"key":"ref_21","unstructured":"Dong, W., An, J., and Ke, X. (2016). PipeCNN: An OpenCL-Based FPGA Accelerator for Large-Scale Convolution Neuron Networks. arXiv."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"1648","DOI":"10.1109\/TCAD.2020.3023903","article-title":"OMNI: A Framework for Integrating Hardware and Software Optimizations for Sparse CNNs","volume":"40","author":"Liang","year":"2021","journal-title":"IEEE Trans. Comput. Aided Des. Integr. Circuits Syst."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Gao, S., Huang, F., Cai, W., and Huang, H. (2021, January 19\u201325). Network Pruning via Performance Maximization. Proceedings of the Computer Vision and Pattern Recognition, Nashville, TN, USA.","DOI":"10.1109\/CVPR46437.2021.00915"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Niu, W., Ma, X., Lin, S., Wang, S., Qian, X., Lin, X., Wang, Y., and Ren, B. (2020). ACM PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruning. Mach. Learn., 907\u2013922.","DOI":"10.1145\/3373376.3378534"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Ma, X., Guo, F.M., Niu, W., Lin, X., Tang, J., Ma, K., Ren, B., and Wang, Y. (2019). PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-time Execution on Mobile Devices. arXiv.","DOI":"10.1609\/aaai.v34i04.5954"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Tan, Z., Song, J., Ma, X., Tan, S., Chen, H., Miao, Y., Wu, Y., Ye, S., Wang, Y., and Li, D. (2020). PCNN: Pattern-based Fine-Grained Regular Pruning Towards Optimizing CNN Accelerators. arXiv.","DOI":"10.1109\/DAC18072.2020.9218498"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"1706","DOI":"10.1109\/TCSI.2020.3048260","article-title":"A Mixed-Pruning Based Framework for Embedded Convolutional Neural Network Acceleration","volume":"Volume 68","author":"Chang","year":"2021","journal-title":"IEEE Transactions on Circuits and Systems I: Regular Papers"},{"key":"ref_28","unstructured":"Bergamaschi, L. (2009). Iterative Methods for Sparse Linear Systems, SIAM."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Bulu, A., Fineman, J.T., Frigo, M., Gilbert, J.R., and Leiserson, C.E. (2009, January 11\u201313). Parallel sparse matrix-vector and matrix-transpose-vector multiplication using compressed sparse blocks. Proceedings of the ACM Symposium on Parallel Algorithms and Architectures, Calgary, AB, Canada.","DOI":"10.1145\/1583991.1584053"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"243","DOI":"10.1145\/3007787.3001163","article-title":"IEEE EIE: Efficient Inference Engine on Compressed Deep Neural Network","volume":"44","author":"Han","year":"2016","journal-title":"ACM SIGARCH Comput. Archit. News"},{"key":"ref_31","unstructured":"Xilinx (2018). 7 Series FPGAs Configuration User Guide (UG470), AMD."},{"key":"ref_32","unstructured":"Liu, Z., Sun, M., Zhou, T., Huang, G., and Darrell, T. (2018). Rethinking the Value of Network Pruning. arXiv."},{"key":"ref_33","first-page":"658","article-title":"Neural Networks: Tricks of the Trade","volume":"41","author":"Lecun","year":"2012","journal-title":"Can. J. Anaesth."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Smith, L. (2017, January 24\u201331). Cyclical Learning Rates for Training Neural Networks. Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), Santa Rosa, CA, USA.","DOI":"10.1109\/WACV.2017.58"},{"key":"ref_35","unstructured":"Frankle, J., and Carbin, M. (May, January 30). The Lottery Ticket Hypothesis: Finding Sparse Trainable Neural Networks. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_36","unstructured":"Malach, E., Yehudai, G., Shalev-Shwartz, S., and Shamir, O. (2020). Proving the Lottery Ticket Hypothesis: Pruning is All You Need. arXiv."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Sui, X., Lv, Q., Bai, Y., Zhu, B., Zhi, L., Yang, Y., and Tan, Z. (2022). A Hardware-Friendly Low-Bit Power-of-Two Quantization Method for CNNs and Its FPGA Implementation. Sensors, 22.","DOI":"10.3390\/s22176618"},{"key":"ref_38","unstructured":"Krizhevsky, A., and Hinton, G. (2009). Learning multiple layers of features from tiny images. Handb. Syst. Autoimmune Dis., 7."},{"key":"ref_39","unstructured":"Krizhevsky, A., Sutskever, I., and Hinton, G. (2013). ImageNet Classification with Deep Convolutional Neural Networks. Advances in Neural Information Processing Systems 25: 26th Annual Conference on Neural Information Processing Systems 2012, Curran Associates, Inc."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_41","unstructured":"Loshchilov, I., and Hutter, F. (2016). SGDR: Stochastic Gradient Descent with Warm Restarts. arXiv."},{"key":"ref_42","unstructured":"Guo, Y., Yao, A., and Chen, Y. (2016, January 5\u201310). Dynamic Network Surgery for Efficient DNNs. Proceedings of the NIPS, Barcelona Spain."},{"key":"ref_43","unstructured":"Zhou, A., Yao, A., Guo, Y., Xu, L., and Chen, Y. (2017). Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weights. arXiv."},{"key":"ref_44","unstructured":"Jia, D., Wei, D., Socher, R., Li, L.J., Kai, L., and Li, F.F. (2009, January 20\u201325). ImageNet: A large-scale hierarchical image database. Proceedings of the 2009 IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Lu, Y., Wu, Y., and Huang, J. (September, January 31). A Coarse-Grained Dual-Convolver Based CNN Accelerator with High Computing Resource Utilization. Proceedings of the 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), Genova, Italy.","DOI":"10.1109\/AICAS48895.2020.9073835"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., and Rabinovich, A. (2015, January 7\u201312). Going Deeper with Convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_47","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alche-Buc, F., Fox, E., and Garnett, R. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library, NeurIPS."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Xu, Y., Li, Y., Zhang, S., Wen, W., Wang, B., Qi, Y., Chen, Y., Lin, W., and Xiong, H. (2020). TRP: Trained Rank Pruning for Efficient Deep Neural Networks. arXiv.","DOI":"10.24963\/ijcai.2020\/136"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Zhang, C., Li, P., Sun, G., Guan, Y., and Cong, J. (2015, January 22\u201324). Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks. Proceedings of the 2015 ACM\/SIGDA International Symposium, New York, NY, USA.","DOI":"10.1145\/2684746.2689060"},{"key":"ref_50","unstructured":"Xilinx (2021). Vivado Design Suite User Guide: Synthesis, AMD."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1109\/TCSI.2020.3030663","article-title":"High Performance CNN Accelerators Based on Hardware and Algorithm Co-Optimization","volume":"68","author":"Yuan","year":"2020","journal-title":"Circuits Syst. I Regul. Pap. IEEE Trans."},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"You, W., and Wu, C. (November, January 29). An Efficient Accelerator for Sparse Convolutional Neural Networks. Proceedings of the 2019 IEEE 13th International Conference on ASIC (ASICON), Chongqing, China.","DOI":"10.1109\/ASICON47005.2019.8983560"},{"key":"ref_53","first-page":"513","article-title":"DLAU: A Scalable Deep Learning Accelerator Unit on FPGA","volume":"36","author":"Chao","year":"2017","journal-title":"IEEE Trans. Comput. Aided Des. Integr. Circuits Syst."},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3432816","article-title":"Toward Multi-FPGA Acceleration of the Neural Networks","volume":"17","author":"Biookaghazadeh","year":"2021","journal-title":"ACM J. Emerg. Technol. Comput. Syst."},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Bouguezzi, S., Fredj, H.B., Belabed, T., Valderrama, C., Faiedh, H., and Souani, C. (2021). An Efficient FPGA-Based Convolutional Neural Network for Classification: Ad-MobileNet. Electronics, 10.","DOI":"10.3390\/electronics10182272"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/824\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:06:40Z","timestamp":1760119600000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/2\/824"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,1,11]]},"references-count":55,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,1]]}},"alternative-id":["s23020824"],"URL":"https:\/\/doi.org\/10.3390\/s23020824","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,1,11]]}}}