{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,10]],"date-time":"2026-02-10T18:10:09Z","timestamp":1770747009095,"version":"3.49.0"},"reference-count":40,"publisher":"Springer Science and Business Media LLC","issue":"7","license":[{"start":{"date-parts":[[2021,3,31]],"date-time":"2021-03-31T00:00:00Z","timestamp":1617148800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,3,31]],"date-time":"2021-03-31T00:00:00Z","timestamp":1617148800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Scientific Research Platform Cultivation Project of SZIIT","award":["PT201704"],"award-info":[{"award-number":["PT201704"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61906124"],"award-info":[{"award-number":["61906124"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61871154"],"award-info":[{"award-number":["61871154"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61906103"],"award-info":[{"award-number":["61906103"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62031013"],"award-info":[{"award-number":["62031013"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Shenzhen Research Council","award":["KJYY20170724152625446"],"award-info":[{"award-number":["KJYY20170724152625446"]}]},{"DOI":"10.13039\/501100002858","name":"Chinese Postdoctoral Science Foundation","doi-asserted-by":"crossref","award":["2018M630158"],"award-info":[{"award-number":["2018M630158"]}],"id":[{"id":"10.13039\/501100002858","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["SIViP"],"published-print":{"date-parts":[[2021,10]]},"abstract":"Abstract<\/jats:title>In deep neural network compression, channel\/filter pruning is widely used for compressing the pre-trained network by judging the redundant channels\/filters. In this paper, we propose a two-step filter pruning method to judge the redundant channels\/filters layer by layer. The first step is to design a filter selection scheme based on $$\\ell _{2,1}$$<\/jats:tex-math>\n \n \u2113<\/mml:mi>\n \n 2<\/mml:mn>\n ,<\/mml:mo>\n 1<\/mml:mn>\n <\/mml:mrow>\n <\/mml:msub>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>-norm by reconstructing the feature map of current layer. More specifically, the filter selection scheme aims to solve a joint $$\\ell _{2,1}$$<\/jats:tex-math>\n \n \u2113<\/mml:mi>\n \n 2<\/mml:mn>\n ,<\/mml:mo>\n 1<\/mml:mn>\n <\/mml:mrow>\n <\/mml:msub>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>-norm minimization problem, i.e., both the regularization term and feature map reconstruction error term are constrained by $$\\ell _{2,1}$$<\/jats:tex-math>\n \n \u2113<\/mml:mi>\n \n 2<\/mml:mn>\n ,<\/mml:mo>\n 1<\/mml:mn>\n <\/mml:mrow>\n <\/mml:msub>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>-norm. The $$\\ell _{2,1}$$<\/jats:tex-math>\n \n \u2113<\/mml:mi>\n \n 2<\/mml:mn>\n ,<\/mml:mo>\n 1<\/mml:mn>\n <\/mml:mrow>\n <\/mml:msub>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>-norm regularization plays a role in the channel\/filter selection, while the $$\\ell _{2,1}$$<\/jats:tex-math>\n \n \u2113<\/mml:mi>\n \n 2<\/mml:mn>\n ,<\/mml:mo>\n 1<\/mml:mn>\n <\/mml:mrow>\n <\/mml:msub>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula>-norm feature map reconstruction error term plays a role in the robust reconstruction. In this way, the proposed filter selection scheme can learn a column-sparse coefficient representation matrix that can indicate the redundancy of filters. Since pruning the redundant filters in current layer might dramatically influence the output feature map of the following layer, the second step needs to update the filters of the following layer to assure output of feature map approximates to that of baseline. Experimental results demonstrate the effectiveness of this proposed method. For example, our pruned VGG-16 on ImageNet achieves $$4\\times $$<\/jats:tex-math>\n \n 4<\/mml:mn>\n \u00d7<\/mml:mo>\n <\/mml:mrow>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula> speedup with 0.95% top-5 accuracy drop. Our pruned ResNet-50 on ImageNet achieves $$2\\times $$<\/jats:tex-math>\n \n 2<\/mml:mn>\n \u00d7<\/mml:mo>\n <\/mml:mrow>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula> speedup with 1.56% top-5 accuracy drop. Our pruned MobileNet on ImageNet achieves $$2\\times $$<\/jats:tex-math>\n \n 2<\/mml:mn>\n \u00d7<\/mml:mo>\n <\/mml:mrow>\n <\/mml:math><\/jats:alternatives><\/jats:inline-formula> speedup with 1.20% top-5 accuracy drop.<\/jats:p>","DOI":"10.1007\/s11760-021-01888-4","type":"journal-article","created":{"date-parts":[[2021,3,31]],"date-time":"2021-03-31T20:02:44Z","timestamp":1617220964000},"page":"1555-1563","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Filter pruning-based two-step feature map reconstruction"],"prefix":"10.1007","volume":"15","author":[{"given":"Yongsheng","family":"Liang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0190-5971","authenticated-orcid":false,"given":"Wei","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuangyan","family":"Yi","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huoxiang","family":"Yang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhenyu","family":"He","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,3,31]]},"reference":[{"key":"1888_CR1","unstructured":"Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: NIPS, pp. 1097\u20131105 (2012)"},{"issue":"2","key":"1888_CR2","doi-asserted-by":"publisher","first-page":"270","DOI":"10.1162\/neco.1989.1.2.270","volume":"1","author":"RJ Williams","year":"1989","unstructured":"Williams, R.J., Zipser, D.: A learning algorithm for continually running fully recurrent neural networks. Neural Comput. 1(2), 270\u2013280 (1989)","journal-title":"Neural Comput."},{"key":"1888_CR3","doi-asserted-by":"crossref","unstructured":"Jia, Y., Shelhamer, E., Donahue, J.: Caffe: Convolutional architecture for fast feature embedding. In: ACM MM, pp. 675\u2013678 (2014)","DOI":"10.1145\/2647868.2654889"},{"key":"1888_CR4","doi-asserted-by":"crossref","unstructured":"Ciresan, D.C., Meier, U., Masci, J.: Flexible, high performance convolutional neural networks for image classification. In: IJCAI, p. 1237 (2011)","DOI":"10.1109\/IJCNN.2011.6033458"},{"key":"1888_CR5","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y.: Going deeper with convolutions. In: CVPR, pp. 1\u20139 (2015)","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"1888_CR6","unstructured":"Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)"},{"key":"1888_CR7","doi-asserted-by":"crossref","unstructured":"Taigman, Y., Yang, M., Ranzato, M.A.: Deepface: Closing the gap to human-level performance in face verification. In: CVPR, pp. 1701\u20131708 (2014)","DOI":"10.1109\/CVPR.2014.220"},{"key":"1888_CR8","unstructured":"Shang, W., Sohn, K., Almeida, D.: Understanding and improving convolutional neural networks via concatenated rectified linear units. In: ICML, pp. 2217\u20132225 (2016)"},{"key":"1888_CR9","doi-asserted-by":"crossref","unstructured":"Chatfield, K., Simonyan, K., Vedaldi, A.: Return of the devil in the details: delving deep into convolutional nets. arXiv preprint arXiv:1405.3531 (2014)","DOI":"10.5244\/C.28.6"},{"key":"1888_CR10","unstructured":"Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400 (2013)"},{"key":"1888_CR11","doi-asserted-by":"crossref","unstructured":"Lecun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. In: Proceedings of the IEEE, pp. 2278\u20132324 (1998)","DOI":"10.1109\/5.726791"},{"key":"1888_CR12","doi-asserted-by":"crossref","unstructured":"Luo, J.H., Wu, J., Lin, W.: Thinet: A filter level pruning method for deep neural network compression. In: ICCV, pp. 5058\u20135066 (2017)","DOI":"10.1109\/ICCV.2017.541"},{"key":"1888_CR13","doi-asserted-by":"publisher","first-page":"298","DOI":"10.1016\/j.neucom.2019.11.097","volume":"381","author":"X Liu","year":"2020","unstructured":"Liu, X., Zhou, Y., Zhao, J., Yao, R., Liu, B., Ma, D., Zheng, Y.: Multiobjective ResNet pruning by means of EMOAs for remote sensing scene classification. Neurocomputing 381, 298\u2013305 (2020)","journal-title":"Neurocomputing"},{"key":"1888_CR14","doi-asserted-by":"publisher","first-page":"950","DOI":"10.1016\/j.compeleceng.2018.01.036","volume":"70","author":"J Zou","year":"2018","unstructured":"Zou, J., Rui, T., Zhou, Y., Yang, C., Zhang, S.: Convolutional neural network simplification via feature map pruning. Comput. Electr. Eng.. 70, 950\u2013958 (2018)","journal-title":"Comput. Electr. Eng.."},{"key":"1888_CR15","first-page":"1","volume":"93","author":"P Singh","year":"2019","unstructured":"Singh, P., Kadi, V.S.R., Namboodiri, V.P.: Convolutional neural network simplification via feature map pruning. Image Vis. Comput. 93, 1\u201314 (2019)","journal-title":"Image Vis. Comput."},{"key":"1888_CR16","doi-asserted-by":"publisher","first-page":"106466","DOI":"10.1109\/ACCESS.2019.2932733","volume":"7","author":"Y Mao","year":"2019","unstructured":"Mao, Y., He, Z., Ma, Z., Tang, X., Wang, Z.: Efficient convolution neural networks for object tracking using separable convolution and filter pruning. IEEE Access. 7, 106466\u2013106474 (2019)","journal-title":"IEEE Access."},{"key":"1888_CR17","doi-asserted-by":"crossref","unstructured":"Jaderberg, M., Vedaldi, A., Zisserman, A.: Speeding up convolutional neural networks with low rank expansions. In: CVPR, pp. 1\u201312 (2014)","DOI":"10.5244\/C.28.88"},{"key":"1888_CR18","doi-asserted-by":"crossref","unstructured":"Lebedev, V., Lempitsky, V.: Fast convnets using group-wise brain damage. In: CVPR, pp. 2554\u20132564 (2016)","DOI":"10.1109\/CVPR.2016.280"},{"key":"1888_CR19","unstructured":"Wen, W., Wu, C., Wang, Y., Chen, Y., Li, H.: Learning structured sparsity in deep neural networks. In: NIPS, pp. 2074\u20132082 (2016)"},{"key":"1888_CR20","unstructured":"Li, H., Kadav, A., Durdanovic, I., Samet, H.: Pruning filters for efficient convnets. In: ICLR, pp. 1\u201313 (2016)"},{"key":"1888_CR21","doi-asserted-by":"crossref","unstructured":"He, Y., Zhang, X., Sun, J.: Channel pruning for accelerating very deep neural networks. In: ICCV, pp. 1\u201310 (2017)","DOI":"10.1109\/ICCV.2017.155"},{"key":"1888_CR22","unstructured":"Howard, A.G., Zhu, M., Chen, B.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. In: CVPR, pp. 1\u20139 (2017)"},{"key":"1888_CR23","doi-asserted-by":"crossref","unstructured":"Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.-C.: Mobilenetv 2: Inverted residuals and linear bottlenecks. In: CVPR, pp. 1\u201314 (2018)","DOI":"10.1109\/CVPR.2018.00474"},{"key":"1888_CR24","unstructured":"Iandola, F.N., Moskewicz, M.W., Ashraf, K., Han, S., Dally, W.J., Keutzer, K.: Squeezenet: Alexnet-level accuracy with 50x fewer parameters and<1mb model size. In: CVPR, pp. 1\u20135 (2016)"},{"key":"1888_CR25","unstructured":"Jin, J., Dundar, A., Culurciello, E.: Flattened convolutional neural networks for feedforward acceleration. In: ICLR, pp. 1\u201311 (2015)"},{"key":"1888_CR26","doi-asserted-by":"crossref","unstructured":"Rastegari, M., ordonez, V., Redmon, J., Farhadi, A.: Xnornet: Imagenet classification using binary convolutional neural networks. In: CVPR, pp. 1\u201317 (2016)","DOI":"10.1007\/978-3-319-46493-0_32"},{"key":"1888_CR27","doi-asserted-by":"crossref","unstructured":"Wang, M., Liu, B., Foroosh, H.: Factorized convolutional neural networks. In: CVPR, pp. 1\u201310 (2016)","DOI":"10.1109\/ICCVW.2017.71"},{"key":"1888_CR28","doi-asserted-by":"crossref","unstructured":"Wu, J., Leng, C., Wang, Y., Hu, Q., Cheng, J.: Quantized convolutional neural networks for mobile devices. In: CVPR, pp. 4820\u20134828 (2016)","DOI":"10.1109\/CVPR.2016.521"},{"key":"1888_CR29","doi-asserted-by":"crossref","unstructured":"Zhang, X., Zhou, X., Lin, M., Sun, J.: Shufflenet: An extremely efficient convolutional neural network for mobile devices. In: CVPR, pp. 6848\u20136856 (2018)","DOI":"10.1109\/CVPR.2018.00716"},{"key":"1888_CR30","doi-asserted-by":"crossref","unstructured":"Lin, M., Ji, R., Wang, Y., Zhang, Y., Zhang, B., Tian, Y., Shao, L.: HRank: Filter pruning using high-rank feature map. In: CVPR (2020)","DOI":"10.1109\/CVPR42600.2020.00160"},{"key":"1888_CR31","unstructured":"Misha, D., Babak, S., Laurent, D.: Predicting parameters in deep learning. In: NIPS, pp. 2148\u20132156 (2013)"},{"issue":"3","key":"1888_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3005348","volume":"13","author":"S Anwar","year":"2017","unstructured":"Anwar, S., Hwang, K., Sung, W.: Structured pruning of deep convolutional neural networks. ACM J. Emerg. Technol. Comput. Syst. 13(3), 1\u201318 (2017)","journal-title":"ACM J. Emerg. Technol. Comput. Syst."},{"key":"1888_CR33","doi-asserted-by":"publisher","first-page":"106385","DOI":"10.1109\/ACCESS.2019.2933032","volume":"7","author":"C Yang","year":"2019","unstructured":"Yang, C., Yang, Z., Khattak, A.M., Yang, L., Zhang, W., Gao, W., Wang, M.: Structured pruning of convolutional neural networks via L1 regularization. IEEE Access. 7, 106385\u2013106394 (2019)","journal-title":"IEEE Access."},{"key":"1888_CR34","doi-asserted-by":"publisher","first-page":"1347","DOI":"10.1109\/ACCESS.2019.2961960","volume":"8","author":"I Garg","year":"2020","unstructured":"Garg, I., Panda, P., Roy, K.: A low effort approach to structured CNN design using PCA. IEEE Access. 8, 1347\u20131360 (2020)","journal-title":"IEEE Access."},{"issue":"10","key":"1888_CR35","doi-asserted-by":"publisher","first-page":"2525","DOI":"10.1109\/TPAMI.2018.2858232","volume":"41","author":"J Luo","year":"2018","unstructured":"Luo, J., Zhang, H., Zhou, H., Xie, C., Wu, J., Lin, W.: Thinet: pruning CNN filters for a thinner net. IEEE Trans. Pattern Anal. Mach. Intell. 41(10), 2525\u20132538 (2018)","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"issue":"5","key":"1888_CR36","doi-asserted-by":"publisher","first-page":"779","DOI":"10.1049\/iet-ipr.2018.6191","volume":"13","author":"W Yang","year":"2019","unstructured":"Yang, W., Jin, L., Wang, S., Cu, Z., Chen, X., Chen, L.: Thinning of convolutional neural network with mixed pruning. IET Image Proc. 13(5), 779\u2013784 (2019)","journal-title":"IET Image Proc."},{"key":"1888_CR37","doi-asserted-by":"crossref","unstructured":"Zheng, Z.: Sparse locality preserving embedding. In: CISP, pp. 1\u20135 (2009)","DOI":"10.1109\/CISP.2009.5302490"},{"key":"1888_CR38","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR, pp. 770\u2013778 (2016)","DOI":"10.1109\/CVPR.2016.90"},{"key":"1888_CR39","doi-asserted-by":"crossref","unstructured":"Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Li, F.: Imagenet: A large-scale hierarchical image database. In: CVPR, pp. 248\u2013255 (2009)","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"1888_CR40","unstructured":"Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)"}],"container-title":["Signal, Image and Video Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11760-021-01888-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11760-021-01888-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11760-021-01888-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,8,26]],"date-time":"2021-08-26T05:15:05Z","timestamp":1629954905000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11760-021-01888-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,3,31]]},"references-count":40,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2021,10]]}},"alternative-id":["1888"],"URL":"https:\/\/doi.org\/10.1007\/s11760-021-01888-4","relation":{},"ISSN":["1863-1703","1863-1711"],"issn-type":[{"value":"1863-1703","type":"print"},{"value":"1863-1711","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,3,31]]},"assertion":[{"value":"5 December 2019","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 March 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"31 March 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}