{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,12]],"date-time":"2026-07-12T02:50:44Z","timestamp":1783824644862,"version":"3.55.0"},"reference-count":33,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2017,2,9]],"date-time":"2017-02-09T00:00:00Z","timestamp":1486598400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100003725","name":"National Research Foundation of Korea","doi-asserted-by":"publisher","award":["2015R1A2A1A10056051"],"award-info":[{"award-number":["2015R1A2A1A10056051"]}],"id":[{"id":"10.13039\/501100003725","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Emerg. Technol. Comput. Syst."],"published-print":{"date-parts":[[2017,7,31]]},"abstract":"<jats:p>Real-time application of deep learning algorithms is often hindered by high computational complexity and frequent memory accesses. Network pruning is a promising technique to solve this problem. However, pruning usually results in irregular network connections that not only demand extra representation efforts but also do not fit well on parallel computation. We introduce structured sparsity at various scales for convolutional neural networks: feature map-wise, kernel-wise, and intra-kernel strided sparsity. This structured sparsity is very advantageous for direct computational resource savings on embedded computers, in parallel computing environments, and in hardware-based systems. To decide the importance of network connections and paths, the proposed method uses a particle filtering approach. The importance weight of each particle is assigned by assessing the misclassification rate with a corresponding connectivity pattern. The pruned network is retrained to compensate for the losses due to pruning. While implementing convolutions as matrix products, we particularly show that intra-kernel strided sparsity with a simple constraint can significantly reduce the size of the kernel and feature map tensors. The proposed work shows that when pruning granularities are applied in combination, we can prune the CIFAR-10 network by more than 70% with less than a 1% loss in accuracy.<\/jats:p>","DOI":"10.1145\/3005348","type":"journal-article","created":{"date-parts":[[2017,2,10]],"date-time":"2017-02-10T13:28:54Z","timestamp":1486733334000},"page":"1-18","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":500,"title":["Structured Pruning of Deep Convolutional Neural Networks"],"prefix":"10.1145","volume":"13","author":[{"given":"Sajid","family":"Anwar","sequence":"first","affiliation":[{"name":"Seoul National University, Republic of Korea, Seoul, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kyuyeon","family":"Hwang","sequence":"additional","affiliation":[{"name":"Seoul National University, Republic of Korea, Seoul, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wonyong","family":"Sung","sequence":"additional","affiliation":[{"name":"Seoul National University, Republic of Korea, Seoul, Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2017,2,9]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/78.978374"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1049\/ip-rsn:19990255"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/72.572092"},{"key":"e_1_2_1_4_1","volume-title":"Proceedings of the 10th International Workshop on Frontiers in Handwriting Recognition. Suvisoft.","author":"Chellapilla Kumar","year":"2006","unstructured":"Kumar Chellapilla , Sidd Puri , and Patrice Simard . 2006 . High performance convolutional neural networks for document processing . In Proceedings of the 10th International Workshop on Frontiers in Handwriting Recognition. Suvisoft. Kumar Chellapilla, Sidd Puri, and Patrice Simard. 2006. High performance convolutional neural networks for document processing. In Proceedings of the 10th International Workshop on Frontiers in Handwriting Recognition. Suvisoft."},{"key":"e_1_2_1_5_1","volume-title":"Cudnn: Efficient primitives for deep learning. Arxiv Preprint Arxiv:1410.0759","author":"Chetlur Sharan","year":"2014","unstructured":"Sharan Chetlur , Cliff Woolley , Philippe Vandermersch , Jonathan Cohen , John Tran , Bryan Catanzaro , and Evan Shelhamer . 2014 . Cudnn: Efficient primitives for deep learning. Arxiv Preprint Arxiv:1410.0759 (2014). Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer. 2014. Cudnn: Efficient primitives for deep learning. Arxiv Preprint Arxiv:1410.0759 (2014)."},{"key":"e_1_2_1_6_1","volume-title":"Collins and Pushmeet Kohli","author":"Maxwell","year":"2014","unstructured":"Maxwell D. Collins and Pushmeet Kohli . 2014 . Memory bounded deep convolutional networks. Arxiv Preprint Arxiv :1412.1442 (2014). Maxwell D. Collins and Pushmeet Kohli. 2014. Memory bounded deep convolutional networks. Arxiv Preprint Arxiv:1412.1442 (2014)."},{"key":"e_1_2_1_7_1","unstructured":"Matthieu Courbariaux Yoshua Bengio and Jean-Pierre David. 2015. BinaryConnect: Training deep neural networks with binary weights during propagations. Advances in Neural Information Processing Systems. 3105--3113.   Matthieu Courbariaux Yoshua Bengio and Jean-Pierre David. 2015. BinaryConnect: Training deep neural networks with binary weights during propagations. Advances in Neural Information Processing Systems. 3105--3113."},{"key":"e_1_2_1_8_1","volume-title":"Smith","author":"Gordon Neil J.","year":"1993","unstructured":"Neil J. Gordon , David J. Salmond , and Adrian F. M . Smith . 1993 . Novel approach to nonlinear\/non-gaussian bayesian state estimation. In IEEE Proceedings on F-Radar and Signal Processing 140. IET , 107--113. Neil J. Gordon, David J. Salmond, and Adrian F. M. Smith. 1993. Novel approach to nonlinear\/non-gaussian bayesian state estimation. In IEEE Proceedings on F-Radar and Signal Processing 140. IET, 107--113."},{"key":"e_1_2_1_9_1","volume-title":"Dally","author":"Han Song","year":"2015","unstructured":"Song Han , Huizi Mao , and William J . Dally . 2015 a. A deep neural network compression pipeline: Pruning, quantization, huffman encoding. Arxiv Preprint Arxiv :1510.00149 (2015). Song Han, Huizi Mao, and William J. Dally. 2015a. A deep neural network compression pipeline: Pruning, quantization, huffman encoding. Arxiv Preprint Arxiv:1510.00149 (2015)."},{"key":"e_1_2_1_10_1","unstructured":"Song Han Jeff Pool John Tran and William Dally. 2015b. Learning both weights and connections for efficient neural network. Advances in Neural Information Processing Systems. 1135--1143.   Song Han Jeff Pool John Tran and William Dally. 2015b. Learning both weights and connections for efficient neural network. Advances in Neural Information Processing Systems. 1135--1143."},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2012.2205597"},{"key":"e_1_2_1_12_1","volume-title":"Batch normalization: Accelerating deep network training by reducing internal covariate shift. Arxiv Preprint Arxiv:1502.03167","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy . 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. Arxiv Preprint Arxiv:1502.03167 ( 2015 ). Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. Arxiv Preprint Arxiv:1502.03167 (2015)."},{"key":"e_1_2_1_13_1","volume-title":"Learning multiple layers of features from tiny images. Technical report","author":"Krizhevsky A.","unstructured":"A. Krizhevsky . 2009. Learning multiple layers of features from tiny images. Technical report , University of Toronto . A. Krizhevsky. 2009. Learning multiple layers of features from tiny images. Technical report, University of Toronto."},{"key":"e_1_2_1_14_1","volume-title":"Hinton","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky , Ilya Sutskever , and Geoffrey E . Hinton . 2012 . Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems . 1097--1105. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems. 1097--1105."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/IROS.2005.1545119"},{"key":"e_1_2_1_16_1","volume-title":"Fast convnets using group-wise brain damage. Arxiv Preprint Arxiv:1506.02515","author":"Lebedev Vadim","year":"2015","unstructured":"Vadim Lebedev and Victor Lempitsky . 2015. Fast convnets using group-wise brain damage. Arxiv Preprint Arxiv:1506.02515 ( 2015 ). Vadim Lebedev and Victor Lempitsky. 2015. Fast convnets using group-wise brain damage. Arxiv Preprint Arxiv:1506.02515 (2015)."},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2013.12.031"},{"key":"e_1_2_1_19_1","volume-title":"Fast training of convolutional networks through FFTs. Arxiv Preprint Arxiv:1312.5851","author":"Mathieu Michael","year":"2013","unstructured":"Michael Mathieu , Mikael Henaff , and Yann LeCun . 2013. Fast training of convolutional networks through FFTs. Arxiv Preprint Arxiv:1312.5851 ( 2013 ). Michael Mathieu, Mikael Henaff, and Yann LeCun. 2013. Fast training of convolutional networks through FFTs. Arxiv Preprint Arxiv:1312.5851 (2013)."},{"key":"e_1_2_1_20_1","volume-title":"Proceedings of the Pacific Symposium on Biocomputing 14","author":"Nakamura Kazuyuki","year":"2009","unstructured":"Kazuyuki Nakamura , Ryo Yoshida , Masao Nagasaki , Satoru Miyano , and Tomoyuki Higuchi . 2009 . Parameter estimation of in silico biological pathways with particle filtering towards a petascale computing . In Proceedings of the Pacific Symposium on Biocomputing 14 . 227--238. Kazuyuki Nakamura, Ryo Yoshida, Masao Nagasaki, Satoru Miyano, and Tomoyuki Higuchi. 2009. Parameter estimation of in silico biological pathways with particle filtering towards a petascale computing. In Proceedings of the Pacific Symposium on Biocomputing 14. 227--238."},{"key":"e_1_2_1_21_1","volume-title":"NIPS Workshop on Deep Learning and Unsupervised Feature Learning","author":"Netzer Y.","year":"2011","unstructured":"Y. Netzer , T. Wang , A. Coates , A. Bissacco , B. Wu , and A. Y. Ng . 2011. Reading digits in natural images with unsupervised feature learning . In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011 . Y. Netzer, T. Wang, A. Coates, A. Bissacco, B. Wu, and A. Y. Ng. 2011. Reading digits in natural images with unsupervised feature learning. In NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1016\/S0262-8856(02)00129-4"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2015.2494536"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/72.248452"},{"key":"e_1_2_1_25_1","volume-title":"Proceedings of the 21st International Conference on Pattern Recognition (ICPR). IEEE, 3288--3291","author":"Sermanet Pierre","year":"2012","unstructured":"Pierre Sermanet , Soumith Chintala , and Yann LeCun . 2012 . Convolutional neural networks applied to house numbers digit classification . In Proceedings of the 21st International Conference on Pattern Recognition (ICPR). IEEE, 3288--3291 . Pierre Sermanet, Soumith Chintala, and Yann LeCun. 2012. Convolutional neural networks applied to house numbers digit classification. In Proceedings of the 21st International Conference on Pattern Recognition (ICPR). IEEE, 3288--3291."},{"key":"e_1_2_1_26_1","volume-title":"Very deep convolutional networks for large-scale image recognition. Arxiv Preprint Arxiv:1409.1556","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman . 2014. Very deep convolutional networks for large-scale image recognition. Arxiv Preprint Arxiv:1409.1556 ( 2014 ). Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. Arxiv Preprint Arxiv:1409.1556 (2014)."},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2670313"},{"key":"e_1_2_1_28_1","volume-title":"Keane","author":"Stepniewski Slawomir W.","year":"1997","unstructured":"Slawomir W. Stepniewski and Andy J . Keane . 1997 . Pruning backpropagation neural networks using modern stochastic optimisation techniques. Neural Computing 8 Applications 5, 2 (1997), 76--98. Slawomir W. Stepniewski and Andy J. Keane. 1997. Pruning backpropagation neural networks using modern stochastic optimisation techniques. Neural Computing 8 Applications 5, 2 (1997), 76--98."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/PDP.2010.43"},{"key":"e_1_2_1_30_1","volume-title":"Resiliency of deep neural networks under quantization. Arxiv Preprint Arxiv:1511.06488","author":"Sung Wonyong","year":"2015","unstructured":"Wonyong Sung , Sungho Shin , and Kyuyeon Hwang . 2015. Resiliency of deep neural networks under quantization. Arxiv Preprint Arxiv:1511.06488 ( 2015 ). Wonyong Sung, Sungho Shin, and Kyuyeon Hwang. 2015. Resiliency of deep neural networks under quantization. Arxiv Preprint Arxiv:1511.06488 (2015)."},{"key":"e_1_2_1_31_1","first-page":"2","article-title":"Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude","volume":"4","author":"Tieleman Tijmen","year":"2012","unstructured":"Tijmen Tieleman and Geoffrey Hinton . 2012 . Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude . COURSERA: Neural Networks for Machine Learning 4 (2012), 2 . Tijmen Tieleman and Geoffrey Hinton. 2012. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning 4 (2012), 2.","journal-title":"COURSERA: Neural Networks for Machine Learning"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.5555\/946247.946684"},{"key":"e_1_2_1_33_1","volume-title":"Proceedings of the 30th International Conference on Machine Learning (ICML-13)","author":"Wan Li","year":"2013","unstructured":"Li Wan , Matthew Zeiler , Sixin Zhang , Yann L. Cun , and Rob Fergus . 2013 . Regularization of neural networks using dropconnect . In Proceedings of the 30th International Conference on Machine Learning (ICML-13) . 1058--1066. Li Wan, Matthew Zeiler, Sixin Zhang, Yann L. Cun, and Rob Fergus. 2013. Regularization of neural networks using dropconnect. In Proceedings of the 30th International Conference on Machine Learning (ICML-13). 1058--1066."}],"container-title":["ACM Journal on Emerging Technologies in Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3005348","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3005348","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T04:23:50Z","timestamp":1750220630000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3005348"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2017,2,9]]},"references-count":33,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2017,7,31]]}},"alternative-id":["10.1145\/3005348"],"URL":"https:\/\/doi.org\/10.1145\/3005348","relation":{},"ISSN":["1550-4832","1550-4840"],"issn-type":[{"value":"1550-4832","type":"print"},{"value":"1550-4840","type":"electronic"}],"subject":[],"published":{"date-parts":[[2017,2,9]]},"assertion":[{"value":"2016-03-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2016-10-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2017-02-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}