{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T17:32:14Z","timestamp":1778347934014,"version":"3.51.4"},"reference-count":45,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,6,29]],"date-time":"2021-06-29T00:00:00Z","timestamp":1624924800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,6,29]],"date-time":"2021-06-29T00:00:00Z","timestamp":1624924800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"National Key R&D Program of China","award":["No. 2017YFB1301003"],"award-info":[{"award-number":["No. 2017YFB1301003"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["No. 61701439"],"award-info":[{"award-number":["No. 61701439"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Zhejiang Key Research and Development Plan","award":["No. 2019C01002"],"award-info":[{"award-number":["No. 2019C01002"]}]},{"name":"Huawei Cooperation Project"},{"name":"Zhejiang Lab","award":["2019LC0AB01"],"award-info":[{"award-number":["2019LC0AB01"]}]},{"name":"Zhejiang Provincial Natural Science Foundation of China","award":["(No. LY20F010016"],"award-info":[{"award-number":["(No. LY20F010016"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["EURASIP J. Adv. Signal Process."],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Nowadays, deep neural networks (DNNs) have been rapidly deployed to realize a number of functionalities like sensing, imaging, classification, recognition, etc. However, the computational-intensive requirement of DNNs makes it difficult to be applicable for resource-limited Internet of Things (IoT) devices. In this paper, we propose a novel pruning-based paradigm that aims to reduce the computational cost of DNNs, by uncovering a more compact structure and learning the effective weights therein, on the basis of not compromising the expressive capability of DNNs. In particular, our algorithm can achieve efficient end-to-end training that transfers a redundant neural network to a compact one with a specifically targeted compression rate directly. We comprehensively evaluate our approach on various representative benchmark datasets and compared with typical advanced convolutional neural network (CNN) architectures. The experimental results verify the superior performance and robust effectiveness of our scheme. For example, when pruning VGG on CIFAR-10, our proposed scheme is able to significantly reduce its FLOPs (floating-point operations) and number of parameters with a proportion of 76.2% and 94.1%, respectively, while still maintaining a satisfactory accuracy. To sum up, our scheme could facilitate the integration of DNNs into the common machine-learning-based IoT framework and establish distributed training of neural networks in both cloud and edge.<\/jats:p>","DOI":"10.1186\/s13634-021-00744-4","type":"journal-article","created":{"date-parts":[[2021,6,29]],"date-time":"2021-06-29T13:02:46Z","timestamp":1624971766000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":16,"title":["An efficient pruning scheme of deep neural networks for Internet of Things applications"],"prefix":"10.1186","volume":"2021","author":[{"given":"Chen","family":"Qi","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shibo","family":"Shen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Rongpeng","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhifeng","family":"Zhao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qing","family":"Liu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jing","family":"Liang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Honggang","family":"Zhang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,6,29]]},"reference":[{"issue":"5","key":"744_CR1","doi-asserted-by":"publisher","first-page":"1125","DOI":"10.1109\/JIOT.2017.2683200","volume":"4","author":"J. Lin","year":"2017","unstructured":"J. Lin, W. Yu, N. Zhang, X. Yang, H. Zhang, W. Zhao, A survey on Internet of Things: Architecture, enabling technologies, security and privacy, and applications. IEEE Internet Things J.4(5), 1125\u20131142 (2017).","journal-title":"IEEE Internet Things J."},{"key":"744_CR2","unstructured":"M. A. Al-Garadi, A. Mohamed, A. Al-Ali, X. Du, M. Guizani, A survey of machine and deep learning methods for Internet of Things (IoT) security. arXiv preprint arXiv:1807.11023 (2018)."},{"issue":"4","key":"744_CR3","doi-asserted-by":"publisher","first-page":"2923","DOI":"10.1109\/COMST.2018.2844341","volume":"20","author":"M. Mohammadi","year":"2018","unstructured":"M. Mohammadi, A. Al-Fuqaha, S. Sorour, M. Guizani, Deep learning for IoT big data and streaming analytics: a survey. IEEE Commun. Surv. Tutorials. 20(4), 2923\u20132960 (2018).","journal-title":"IEEE Commun. Surv. Tutorials"},{"issue":"6","key":"744_CR4","doi-asserted-by":"publisher","first-page":"2342","DOI":"10.1109\/JIOT.2017.2750765","volume":"4","author":"E. Park","year":"2017","unstructured":"E. Park, Y. Cho, J. Han, S. J. Kwon, Comprehensive approaches to user acceptance of Internet of Things in a smart home environment. IEEE Internet Things J.4(6), 2342\u20132350 (2017).","journal-title":"IEEE Internet Things J."},{"issue":"5","key":"744_CR5","doi-asserted-by":"publisher","first-page":"3758","DOI":"10.1109\/JIOT.2018.2844296","volume":"5","author":"O. Elijah","year":"2018","unstructured":"O. Elijah, T. A. Rahman, I. Orikumhi, C. Y. Leow, M. N. Hindia, An overview of Internet of Things (IoT) and data analytics in agriculture: Benefits and challenges. IEEE Internet Things J.5(5), 3758\u20133773 (2018).","journal-title":"IEEE Internet Things J."},{"issue":"1","key":"744_CR6","doi-asserted-by":"publisher","first-page":"96","DOI":"10.1109\/MNET.2018.1700202","volume":"32","author":"H. Li","year":"2018","unstructured":"H. Li, K. Ota, M. Dong, Learning IoT in edge: Deep learning for the Internet of Things with edge computing. IEEE Netw.32(1), 96\u2013101 (2018).","journal-title":"IEEE Netw."},{"key":"744_CR7","doi-asserted-by":"publisher","first-page":"181721","DOI":"10.1109\/ACCESS.2019.2958962","volume":"7","author":"X. Ma","year":"2019","unstructured":"X. Ma, T. Yao, M. Hu, Y. Dong, W. Liu, F. Wang, J. Liu, A survey on deep learning empowered IoT applications. IEEE Access. 7:, 181721\u2013181732 (2019).","journal-title":"IEEE Access"},{"key":"744_CR8","first-page":"1","volume-title":"The 25th Annual International Conference on Mobile Computing and Networking","author":"X. Xie","year":"2019","unstructured":"X. Xie, K. -H. Kim, in The 25th Annual International Conference on Mobile Computing and Networking. Source compression with bounded DNN perception loss for IoT edge computer vision (ACMLos Cabos, 2019), pp. 1\u201316."},{"key":"744_CR9","volume-title":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"D. Jia","year":"2009","unstructured":"D. Jia, D. Wei, S. Richard, L. Li-Jia, L. Kai, L. Fei-Fei, in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Imagenet: A large-scale hierarchical image database (IEEEMiami, 2009)."},{"key":"744_CR10","first-page":"1097","volume-title":"Advances in Neural Information Processing Systems","author":"A. Krizhevsky","year":"2012","unstructured":"A. Krizhevsky, I. Sutskever, G. E. Hinton, in Advances in Neural Information Processing Systems. Imagenet classification with deep convolutional neural networks (Curran Associates, Inc.Harrahs and Harveys, Lake Tahoe, 2012), pp. 1097\u20131105."},{"key":"744_CR11","volume-title":"International Conference on Learning Representations (ICLR)","author":"K. Simonyan","year":"2015","unstructured":"K. Simonyan, A. Zisserman, in International Conference on Learning Representations (ICLR). Very deep convolutional networks for large-scale image recognition (OpenReview.netSan Diego, 2015)."},{"key":"744_CR12","volume-title":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"C. Szegedy","year":"2015","unstructured":"C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, A. Rabinovich, in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Going deeper with convolutions (IEEEBoston, 2015)."},{"key":"744_CR13","volume-title":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"K. He","year":"2016","unstructured":"K. He, X. Zhang, S. Ren, J. Sun, in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Deep residual learning for image recognition (IEEELas Vegas Nevada, 2016)."},{"key":"744_CR14","unstructured":"A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, H. Adam, Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)."},{"key":"744_CR15","first-page":"1135","volume-title":"Advances in Neural Information Processing Systems 28","author":"S. Han","year":"2015","unstructured":"S. Han, J. Pool, J. Tran, W. Dally, in Advances in Neural Information Processing Systems 28, ed. by C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett. Learning both weights and connections for efficient neural network (Curran Associates, Inc.Montreal, 2015), pp. 1135\u20131143."},{"key":"744_CR16","volume-title":"International Conference on Learning Representations (ICLR)","author":"S. Han","year":"2016","unstructured":"S. Han, H. Mao, W. Dally, in International Conference on Learning Representations (ICLR). Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding (OpenReview.netCaribe Hilton, San Juan, 2016)."},{"key":"744_CR17","volume-title":"International Conference on Learning Representations (ICLR)","author":"P. Molchanov","year":"2017","unstructured":"P. Molchanov, S. Tyree, T. Karras, T. Aila, J. Kautz, in International Conference on Learning Representations (ICLR). Pruning convolutional neural networks for resource efficient inference (OpenReview.netPalais des Congreptune, Toulon, 2017)."},{"key":"744_CR18","volume-title":"International Conference on Learning Representations (ICLR)","author":"H. Li","year":"2017","unstructured":"H. Li, A. Kadav, I. Durdanovic, H. Samet, H. P. Graf, in International Conference on Learning Representations (ICLR). Pruning filters for efficient convnets (OpenReview.netPalais des Congreptune, Toulon, 2017)."},{"key":"744_CR19","volume-title":"The IEEE International Conference on Computer Vision (ICCV)","author":"Z. Liu","year":"2017","unstructured":"Z. Liu, J. Li, Z. Shen, G. Huang, S. Yan, C. Zhang, in The IEEE International Conference on Computer Vision (ICCV). Learning efficient convolutional networks through network slimming (IEEEVenice, 2017)."},{"key":"744_CR20","volume-title":"The IEEE International Conference on Computer Vision (ICCV)","author":"J. Luo","year":"2017","unstructured":"J. Luo, J. Wu, W. Lin, in The IEEE International Conference on Computer Vision (ICCV). Thinet: A filter level pruning method for deep neural network compression (IEEEVenice, 2017)."},{"key":"744_CR21","unstructured":"T. Dettmers, L. Zettlemoyer, Sparse networks from scratch: Faster training without losing performance. arXiv preprint arXiv:1907.04840 (2019)."},{"key":"744_CR22","unstructured":"H. Mostafa, X. Wang, in Proceedings of the 36th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol.97, ed. by K. Chaudhuri, R. Salakhutdinov. Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization (Long Beach, 2019), pp. 4646\u20134655."},{"key":"744_CR23","doi-asserted-by":"crossref","unstructured":"E. De Coninck, T. Verbelen, B. Vankeirsbilck, S. Bohez, P. Simoens, P. Demeester, B. Dhoedt, in International Internet of Things Summit. Distributed neural networks for Internet of Things: The big-little approach (Springer, 2015), pp. 484\u2013492.","DOI":"10.1007\/978-3-319-47075-7_52"},{"key":"744_CR24","unstructured":"R. Hu, Y. Guo, E. P. Ratazzi, Y. Gong, Differentially private federated learning for resource-constrained Internet of Things. arXiv preprint arXiv:2003.12705 (2020)."},{"key":"744_CR25","unstructured":"A. Krizhevsky, G. Hinton, Learning multiple layers of features from tiny images. Technical Report (2009)."},{"key":"744_CR26","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1109\/iThings-GreenCom-CPSCom-SmartData.2017.145","volume-title":"2017 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData)","author":"H. Amroun","year":"2017","unstructured":"H. Amroun, M. H. Temkit, M. Ammi, in 2017 IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData). Best feature for CNN classification of human activity using IoT network (IEEEExeter, 2017), pp. 943\u2013950."},{"key":"744_CR27","volume-title":"Advances in Neural Information Processing Systems 32","author":"X. Ding","year":"2019","unstructured":"X. Ding, G. Ding, X. Zhou, Y. Guo, J. Han, J. Liu, in Advances in Neural Information Processing Systems 32. Global sparse momentum SGD for pruning very deep neural networks (Curran Associates, Inc.Vancouver, 2019)."},{"key":"744_CR28","first-page":"2074","volume-title":"Advances in Neural Information Processing Systems 29","author":"W. Wen","year":"2016","unstructured":"W. Wen, C. Wu, Y. Wang, Y. Chen, H. Li, in Advances in Neural Information Processing Systems 29. Learning structured sparsity in deep neural networks (Curran Associates, Inc.Vancouver, 2016), pp. 2074\u20132082."},{"key":"744_CR29","volume-title":"Proceedings of the 32nd International Conference on Machine Learning (ICML)","author":"S. Ioffe","year":"2015","unstructured":"S. Ioffe, C. Szegedy, in Proceedings of the 32nd International Conference on Machine Learning (ICML). Batch normalization: Accelerating deep network training by reducing internal covariate shift (ACMLille, 2015)."},{"key":"744_CR30","first-page":"875","volume-title":"Advances in Neural Information Processing Systems 31","author":"Z. Zhuang","year":"2018","unstructured":"Z. Zhuang, M. Tan, B. Zhuang, J. Liu, Y. Guo, Q. Wu, J. Huang, J. Zhu, in Advances in Neural Information Processing Systems 31. Discrimination-aware channel pruning for deep neural networks (Curran Associates, Inc.Montreal, 2018), pp. 875\u2013886."},{"key":"744_CR31","volume-title":"The IEEE International Conference on Computer Vision (ICCV)","author":"Y. He","year":"2017","unstructured":"Y. He, X. Zhang, J. Sun, in The IEEE International Conference on Computer Vision (ICCV). Channel pruning for accelerating very deep neural networks (IEEEVenice, 2017)."},{"key":"744_CR32","unstructured":"P. Singh, V. K. Verma, P. Rai, V. P. Namboodiri, Leveraging filter correlations for deep model compression. arXiv e-prints, 1811\u201310559 (2018)."},{"key":"744_CR33","volume-title":"Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19","author":"W. Wang","year":"2019","unstructured":"W. Wang, C. Fu, J. Guo, D. Cai, X. He, in Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. Cop: Customized deep model compression via regularized correlation-based filter-level pruning (Morgan KaufmannMacao, 2019)."},{"key":"744_CR34","first-page":"3296","volume-title":"Proceedings of the IEEE International Conference on Computer Vision (ICCV)","author":"Z. Liu","year":"2019","unstructured":"Z. Liu, H. Mu, X. Zhang, Z. Guo, X. Yang, K. -T. Cheng, J. Sun, in Proceedings of the IEEE International Conference on Computer Vision (ICCV). Metapruning: Meta learning for automatic neural network channel pruning (IEEESeoul, 2019), pp. 3296\u20133305."},{"key":"744_CR35","volume-title":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"S. Lin","year":"2019","unstructured":"S. Lin, R. Ji, C. Yan, B. Zhang, L. Cao, Q. Ye, F. Huang, D. Doermann, in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Towards optimal structured CNN pruning via generative adversarial learning (IEEELong Beach, 2019)."},{"key":"744_CR36","volume-title":"International Conference on Learning Representations (ICLR)","author":"Z. Liu","year":"2019","unstructured":"Z. Liu, M. Sun, T. Zhou, G. Huang, T. Darrell, in International Conference on Learning Representations (ICLR). Rethinking the value of network pruning (OpenReview.netNew Orleans, 2019)."},{"key":"744_CR37","volume-title":"IJCAI International Joint Conference on Artificial Intelligence","author":"Y. He","year":"2018","unstructured":"Y. He, G. Kang, X. Dong, Y. Fu, Y. Yang, in IJCAI International Joint Conference on Artificial Intelligence. Soft filter pruning for accelerating deep convolutional neural networks (Morgan KaufmannStockholm, 2018)."},{"key":"744_CR38","unstructured":"Y. Wang, X. Zhang, L. Xie, J. Zhou, H. Su, B. Zhang, X. Hu, Pruning from scratch. arXiv e-prints, 1909\u201312579 (2019)."},{"key":"744_CR39","doi-asserted-by":"crossref","unstructured":"M. Lin, R. Ji, Y. Zhang, B. Zhang, Y. Wu, Y. Tian, Channel pruning via automatic structure search. arXiv e-prints, 2001\u201308565 (2020).","DOI":"10.24963\/ijcai.2020\/94"},{"issue":"55","key":"744_CR40","first-page":"1","volume":"20","author":"T. Elsken","year":"2019","unstructured":"T. Elsken, J. H. Metzen, F. Hutter, Neural architecture search: A survey. J. Mach. Learn. Res.20(55), 1\u201321 (2019).","journal-title":"J. Mach. Learn. Res."},{"key":"744_CR41","volume-title":"2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"M. Tan","year":"2019","unstructured":"M. Tan, B. Chen, R. Pang, V. Vasudevan, M. Sandler, A. Howard, Q. V. Le, in 2019 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Mnasnet: Platform-aware neural architecture search for mobile (IEEELong Beach, 2019)."},{"key":"744_CR42","volume-title":"International Conference on Learning Representations(ICLR)","author":"H. Liu","year":"2019","unstructured":"H. Liu, K. Simonyan, Y. Yang, in International Conference on Learning Representations(ICLR). DARTS: Differentiable architecture search (OpenReview.netNew Orleans, 2019)."},{"key":"744_CR43","volume-title":"International Conference on Learning Representations(ICLR)","author":"Y. Xu","year":"2020","unstructured":"Y. Xu, L. Xie, X. Zhang, X. Chen, G. -J. Qi, Q. Tian, H. Xiong, in International Conference on Learning Representations(ICLR). Pc-darts: Partial channel connections for memory-efficient architecture search (OpenReview.netVirtual Conference, Formerly Addis Ababa ETHIOPIA, 2020)."},{"key":"744_CR44","volume-title":"NIPS-W","author":"A. Paszke","year":"2017","unstructured":"A. Paszke, S. Gross, S. Chintala, G. Chanan, E. Yang, Z. DeVito, Z. Lin, A. Desmaison, L. Antiga, A. Lerer, in NIPS-W. Automatic differentiation in pytorch (Curran Associates, Inc.Long Beach, 2017)."},{"key":"744_CR45","volume-title":"The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Y. He","year":"2019","unstructured":"Y. He, P. Liu, Z. Wang, Z. Hu, Y. Yang, in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Filter pruning via geometric median for deep convolutional neural networks acceleration (IEEELong Beach, 2019)."}],"container-title":["EURASIP Journal on Advances in Signal Processing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-021-00744-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s13634-021-00744-4\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s13634-021-00744-4.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,29]],"date-time":"2021-06-29T13:18:48Z","timestamp":1624972728000},"score":1,"resource":{"primary":{"URL":"https:\/\/asp-eurasipjournals.springeropen.com\/articles\/10.1186\/s13634-021-00744-4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,29]]},"references-count":45,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["744"],"URL":"https:\/\/doi.org\/10.1186\/s13634-021-00744-4","relation":{},"ISSN":["1687-6180"],"issn-type":[{"value":"1687-6180","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,29]]},"assertion":[{"value":"29 January 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"13 June 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 June 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"31"}}