{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,2]],"date-time":"2026-05-02T06:51:48Z","timestamp":1777704708135,"version":"3.51.4"},"reference-count":40,"publisher":"SAGE Publications","issue":"1","license":[{"start":{"date-parts":[[2020,5,6]],"date-time":"2020-05-06T00:00:00Z","timestamp":1588723200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"published-print":{"date-parts":[[2020,7,17]]},"abstract":"<jats:p>\u00a0The deployment of large-scale Convolutional Neural Networks (CNNs) in limited-power devices is hindered by their high computation cost and storage. In this paper, we propose a novel framework for CNNs to simultaneously achieve channel pruning and low-bit quantization by combining weight quantization with Sparse Group Lasso (SGL) regularization. We model this framework as a discretely constrained problem and solve it by Alternating Direction Method of Multipliers (ADMM). Different from previous approaches, the proposed method reduces not only model size but also computational operations. In experimental section, we evaluate the proposed framework on CIFAR datasets with several popular models such as VGG-7\/16\/19 and ResNet-18\/34\/50, which demonstrate that the proposed method can obtain low-bit networks and dramatically reduce redundant channels of the network with slight inference accuracy loss. Furthermore, we also visualize and analyze weight tensors, which showing the compact group-sparsity structure of them.<\/jats:p>","DOI":"10.3233\/jifs-191014","type":"journal-article","created":{"date-parts":[[2020,5,8]],"date-time":"2020-05-08T14:48:03Z","timestamp":1588949283000},"page":"221-232","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":2,"title":["A CNN channel pruning low-bit framework using weight quantization with sparse group lasso regularization"],"prefix":"10.1177","volume":"39","author":[{"given":"Xin","family":"Long","sequence":"first","affiliation":[{"name":"College of Systems Engineering, National University of Defense Technology, Changsha, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiangrong","family":"Zeng","sequence":"additional","affiliation":[{"name":"College of Systems Engineering, National University of Defense Technology, Changsha, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yan","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Systems Engineering, National University of Defense Technology, Changsha, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huaxin","family":"Xiao","sequence":"additional","affiliation":[{"name":"College of Systems Engineering, National University of Defense Technology, Changsha, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Maojun","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Systems Engineering, National University of Defense Technology, Changsha, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zongcheng","family":"Ben","sequence":"additional","affiliation":[{"name":"College of Computer, National University of Defense Technology, Changsha, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"179","published-online":{"date-parts":[[2020,5,6]]},"reference":[{"issue":"5","key":"e_1_3_2_2_2","first-page":"1097","article-title":"ImageNet Classification with Deep Convolutional Neural Networks","volume":"141","author":"Krizhevsky A.","year":"2012","unstructured":"KrizhevskyA., SutskeverI. and HintonG.E., ImageNet Classification with Deep Convolutional Neural Networks, neural information processing systems 141(5) (2012), 1097\u20131105.","journal-title":"neural information processing systems"},{"key":"e_1_3_2_3_2","first-page":"580","article-title":"Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation","author":"Girshick R.B.","year":"2014","unstructured":"GirshickR.B., DonahueJ., DarrellT. and MalikJ., Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation, Computer vision and pattern recognition (2014), 580\u2013587.","journal-title":"Computer vision and pattern recognition"},{"key":"e_1_3_2_4_2","first-page":"6499","article-title":"Efficient Video Object Segmentation via Network Modulation","author":"Yang L.","year":"2018","unstructured":"YangL., WangY., XiongX., YangJ. and KatsaggelosA.K., Efficient Video Object Segmentation via Network Modulation, Computer vision and pattern recognition (2018), 6499\u20136507.","journal-title":"Computer vision and pattern recognition"},{"key":"e_1_3_2_5_2","article-title":"Neural Machine Translation by Jointly Learning to Align and Translate","author":"Bahdanau D.","year":"2015","unstructured":"BahdanauD., ChoK. and BengioY., Neural Machine Translation by Jointly Learning to Align and Translate, International conference on learning representations (2015).","journal-title":"International conference on learning representations"},{"key":"e_1_3_2_6_2","first-page":"5406","article-title":"Deep Learning with Low Precision by Half-Wave Gaussian Quantization","author":"Cai Z.","year":"2017","unstructured":"CaiZ., HeX., SunJ. and VasconcelosN., Deep Learning with Low Precision by Half-Wave Gaussian Quantization, Computer vision and pattern recognition (2017), 5406\u20135414.","journal-title":"Computer vision and pattern recognition"},{"key":"e_1_3_2_7_2","first-page":"2755","article-title":"Learning Efficient Convolutional Networks through Network Slimming","author":"Liu Z.","year":"2017","unstructured":"LiuZ., LiJ., ShenZ., HuangG., YanS. and ZhangC., Learning Efficient Convolutional Networks through Network Slimming, International conference on computer vision (2017), 2755\u20132763.","journal-title":"International conference on computer vision"},{"key":"e_1_3_2_8_2","article-title":"Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding","author":"Han S.","year":"2016","unstructured":"HanS., MaoH. and DallyW.J., Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding, International conference on learning representations, (2016).","journal-title":"International conference on learning representations"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3007787.3001163"},{"key":"e_1_3_2_10_2","article-title":"Deep compression and EIE: Efficient inference engine on compressed deep neural network","author":"Han S.","year":"2016","unstructured":"HanS., LiuX., MaoH., PuJ., PedramA., HorowitzM. and DallyB., Deep compression and EIE: Efficient inference engine on compressed deep neural network, IEEE hot chips symposium, (2016).","journal-title":"IEEE hot chips symposium"},{"key":"e_1_3_2_11_2","article-title":"Sparse Convolutional Neural Networks","author":"Liu B.","year":"2015","unstructured":"LiuB., WangM., ForooshH., TappenM.F. and PenksyM., Sparse Convolutional Neural Networks, Computer vision and pattern recognition, (2015).","journal-title":"Computer vision and pattern recognition"},{"key":"e_1_3_2_12_2","article-title":"On Compressing Deep Models by Low Rank and Sparse Decomposition","author":"Yu X.","year":"2017","unstructured":"YuX., LiuT., WangX. and TaoD., On Compressing Deep Models by Low Rank and Sparse Decomposition, Computer vision and pattern recognition, (2017).","journal-title":"Computer vision and pattern recognition"},{"key":"e_1_3_2_13_2","first-page":"1269","article-title":"Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation","author":"Denton E.L.","year":"2014","unstructured":"DentonE.L., ZarembaW., BrunaJ., LecunY. and FergusR., Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation, Neural information processing systems (2014), 1269\u20131277.","journal-title":"Neural information processing systems"},{"key":"e_1_3_2_14_2","article-title":"Speeding up Convolutional Neural Networks with Low Rank Expansions","author":"Jaderberg M.","year":"2014","unstructured":"JaderbergM., VedaldiA. and ZissermanA., Speeding up Convolutional Neural Networks with Low Rank Expansions, British machine vision conference, 2014.","journal-title":"British machine vision conference"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2018.01.010"},{"key":"e_1_3_2_16_2","first-page":"3123","article-title":"BinaryConnect: training deep neural networks with binary weights during propagations","author":"Courbariaux M.","year":"2015","unstructured":"CourbariauxM., BengioY. and DavidJ., BinaryConnect: training deep neural networks with binary weights during propagations, Neural information processing systems (2015), 3123\u20133131.","journal-title":"Neural information processing systems"},{"key":"e_1_3_2_17_2","article-title":"Ternary Weight Networks","author":"Li F.","year":"2016","unstructured":"LiF. and LiuB., Ternary Weight Networks, arXiv: Computer Vision and Pattern Recognition, 2016.","journal-title":"arXiv: Computer Vision and Pattern Recognition"},{"key":"e_1_3_2_18_2","article-title":"Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1","author":"Courbariaux M.","year":"2016","unstructured":"CourbariauxM., HubaraI., SoudryD., ElyanivR. and BengioY., Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1, arXiv: Learning, 2016.","journal-title":"arXiv: Learning"},{"key":"e_1_3_2_19_2","first-page":"525","article-title":"XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks","author":"Rastegari M.","year":"2016","unstructured":"RastegariM., OrdonezV., RedmonJ. and FarhadiA., XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks, European conference on computer vision (2016), 525\u2013542.","journal-title":"European conference on computer vision"},{"key":"e_1_3_2_20_2","article-title":"Ternary Neural Networks with Fine-Grained Quantization","author":"Mellempudi N.","year":"2017","unstructured":"MellempudiN., KunduA., MudigereD., DasD., KaulB. and DubeyP., Ternary Neural Networks with Fine-Grained Quantization, arXiv: Learning, 2017.","journal-title":"arXiv: Learning"},{"key":"e_1_3_2_21_2","article-title":"DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients","author":"Zhou S.","year":"2016","unstructured":"ZhouS., NiZ., ZhouX., WenH., WuY. and ZouY., DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients, arXiv: Neural and Evolutionary Computing, 2016.","journal-title":"arXiv: Neural and Evolutionary Computing"},{"key":"e_1_3_2_22_2","article-title":"SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size","author":"Iandola F.N.","year":"2017","unstructured":"IandolaF.N., HanS., MoskewiczM.W., AshrafK., DallyW.J. and KeutzerK., SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size, arXiv: Computer Vision and Pattern Recognition, 2017.","journal-title":"arXiv: Computer Vision and Pattern Recognition"},{"key":"e_1_3_2_23_2","article-title":"MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications","author":"Howard A.G.","year":"2017","unstructured":"HowardA.G., ZhuM., ChenB., KalenichenkoD., WangW., WeyandT. and AdamH., MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications, arXiv: Computer Vision and Pattern Recognition, 2017.","journal-title":"arXiv: Computer Vision and Pattern Recognition"},{"key":"e_1_3_2_24_2","first-page":"4510","article-title":"MobileNetV2: Inverted Residuals and Linear Bottlenecks","author":"Sandler M.B.","year":"2018","unstructured":"SandlerM.B., HowardA.G., ZhuM., ZhmoginovA. and ChenL., MobileNetV2: Inverted Residuals and Linear Bottlenecks, Computer vision and pattern recognition (2018), 4510\u20134520.","journal-title":"Computer vision and pattern recognition"},{"key":"e_1_3_2_25_2","first-page":"6848","article-title":"ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices","author":"Zhang X.","year":"2018","unstructured":"ZhangX., ZhouX., LinM. and SunJ., ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices, Computer vision and pattern recognition (2018), 6848\u20136856.","journal-title":"Computer vision and pattern recognition"},{"key":"e_1_3_2_26_2","first-page":"122","article-title":"ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design","author":"Ma N.","year":"2018","unstructured":"MaN., ZhangX., ZhengH. and SunJ., ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design, European conference on computer vision (2018), 122\u2013138.","journal-title":"European conference on computer vision"},{"key":"e_1_3_2_27_2","article-title":"Designing Neural Network Architectures using Reinforcement Learning","author":"Baker B.","year":"2017","unstructured":"BakerB., GuptaO., NaikN. and RaskarR., Designing Neural Network Architectures using Reinforcement Learning, International conference on learning representations, 2017.","journal-title":"International conference on learning representations"},{"key":"e_1_3_2_28_2","article-title":"Neural Architecture Search with Reinforcement Learning","author":"Zoph B.","year":"2017","unstructured":"ZophB. and LeQ.V., Neural Architecture Search with Reinforcement Learning, International conference on learning representations, 2017.","journal-title":"International conference on learning representations"},{"key":"e_1_3_2_29_2","article-title":"Tree-Guided Group Lasso for Multi-Task Regression with Structured Sparsity","author":"Kim S.","year":"2010","unstructured":"KimS. and XingE.P., Tree-Guided Group Lasso for Multi-Task Regression with Structured Sparsity, International conference on machine learning, 2010.","journal-title":"International conference on machine learning"},{"key":"e_1_3_2_30_2","article-title":"Learning the Structure of Deep Convolutional Networks","author":"Feng J.","year":"2015","unstructured":"FengJ. and DarrellT., Learning the Structure of Deep Convolutional Networks, International conference on computer vision, 2015.","journal-title":"International conference on computer vision"},{"key":"e_1_3_2_31_2","first-page":"2074","article-title":"Learning Structured Sparsity in Deep Neural Networks","author":"Wen W.","year":"2016","unstructured":"WenW., WuC., WangY., ChenY. and LiH., Learning Structured Sparsity in Deep Neural Networks, Neural information processing systems (2016), 2074\u20132082.","journal-title":"Neural information processing systems"},{"key":"e_1_3_2_32_2","first-page":"5619","article-title":"A simple effective heuristic for embedded mixed-integer quadratic programming","author":"Takapoui R.","year":"2016","unstructured":"TakapouiR., MoehleN., BoydS.P. and BemporadA., A simple effective heuristic for embedded mixed-integer quadratic programming, Advances in computing and communications (2016), 5619\u20135625.","journal-title":"Advances in computing and communications"},{"key":"e_1_3_2_33_2","first-page":"3466","article-title":"Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMM","author":"Leng C.","year":"2018","unstructured":"LengC., DouZ., LiH., ZhuS. and JinR., Extremely Low Bit Neural Network: Squeeze the Last Bit Out with ADMM, National conference on artificial intelligence (2018), 3466\u20133473.","journal-title":"National conference on artificial intelligence"},{"key":"e_1_3_2_34_2","first-page":"191","article-title":"A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers","author":"Zhang T.","year":"2018","unstructured":"ZhangT., YeS., ZhangK., TangJ., WenW., FardadM. and WangY., A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers, European conference on computer vision (2018), 191\u2013207.","journal-title":"European conference on computer vision"},{"key":"e_1_3_2_35_2","article-title":"Pruning Convolutional Neural Networks for Resource Efficient Inference","author":"Molchanov P.","year":"2017","unstructured":"MolchanovP., TyreeS., KarrasT., AilaT. and KautzJ., Pruning Convolutional Neural Networks for Resource Efficient Inference, International conference on learning representations, 2017.","journal-title":"International conference on learning representations"},{"key":"e_1_3_2_36_2","article-title":"A note on the group lasso and a sparse group lasso","author":"Friedman J.H.","year":"2010","unstructured":"FriedmanJ.H., HastieT. and TibshiraniR., A note on the group lasso and a sparse group lasso, arXiv: Statistics Theory, 2010.","journal-title":"arXiv: Statistics Theory"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1080\/10618600.2012.681250"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2017.02.029"},{"key":"e_1_3_2_39_2","article-title":"Learning multiple layers of features from tiny images","author":"Krizhevsky A.","year":"2009","unstructured":"KrizhevskyA. and GeoffreyH., Learning multiple layers of features from tiny images, In Tech Report, 2009.","journal-title":"In Tech Report"},{"key":"e_1_3_2_40_2","first-page":"770","article-title":"Deep Residual Learning for Image Recognition","author":"He K.","year":"2016","unstructured":"HeK., ZhangX., RenS. and SunJ., Deep Residual Learning for Image Recognition, Computer vision and pattern recognition (2016), 770\u2013778.","journal-title":"Computer vision and pattern recognition"},{"key":"e_1_3_2_41_2","first-page":"1398","article-title":"Channel Pruning for Accelerating Very Deep Neural Networks","author":"He Y.","year":"2017","unstructured":"HeY., ZhangX. and SunJ., Channel Pruning for Accelerating Very Deep Neural Networks, International conference on computer vision (2017), 1398\u20131406.","journal-title":"International conference on computer vision"}],"container-title":["Journal of Intelligent &amp; Fuzzy Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JIFS-191014","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.3233\/JIFS-191014","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.3233\/JIFS-191014","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,29]],"date-time":"2026-04-29T09:41:50Z","timestamp":1777455710000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.3233\/JIFS-191014"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5,6]]},"references-count":40,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2020,7,17]]}},"alternative-id":["10.3233\/JIFS-191014"],"URL":"https:\/\/doi.org\/10.3233\/jifs-191014","relation":{},"ISSN":["1064-1246","1875-8967"],"issn-type":[{"value":"1064-1246","type":"print"},{"value":"1875-8967","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,5,6]]}}}