{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,25]],"date-time":"2026-04-25T15:12:07Z","timestamp":1777129927300,"version":"3.51.4"},"reference-count":52,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,4,4]],"date-time":"2024-04-04T00:00:00Z","timestamp":1712188800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,4,4]],"date-time":"2024-04-04T00:00:00Z","timestamp":1712188800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Process Lett"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The swift advancement of convolutional neural networks (CNNs) in numerous real-world utilizations urges an elevation in computational cost along with the size of the model. In this context, many researchers steered their focus to eradicate these specific issues by compressing the original CNN models by pruning weights and filters, respectively. As filter pruning has an upper hand over the weight pruning method because filter pruning methods don\u2019t impact sparse connectivity patterns. In this work, we suggested a Bayesian Convolutional Neural Network (BayesCNN) with Variational Inference, which prefaces probability distribution over weights. For the pruning task of Bayesian CNN, we utilized a combined version of L1-norm with capped L1-norm to help epitomize the amount of information that can be extracted through filter and control regularization. In this formation, we pruned unimportant filters directly without any test accuracy loss and achieved a slimmer model with comparative accuracy. The whole process of pruning is iterative and to validate the performance of our proposed work, we utilized several different CNN architectures on the standard classification dataset available. We have compared our results with non-Bayesian CNN models particularly, datasets such as CIFAR-10 on VGG-16, and pruned 75.8% parameters with float-point-operations (FLOPs) reduction of 51.3% without loss of accuracy and has achieved advancement in state-of-art.<\/jats:p>","DOI":"10.1007\/s11063-024-11593-1","type":"journal-article","created":{"date-parts":[[2024,4,4]],"date-time":"2024-04-04T13:01:49Z","timestamp":1712235709000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["Efficient Bayesian CNN Model Compression using Bayes by Backprop and L1-Norm Regularization"],"prefix":"10.1007","volume":"56","author":[{"given":"Ali Muhammad","family":"Shaikh","sequence":"first","affiliation":[]},{"given":"Yun-bo","family":"Zhao","sequence":"additional","affiliation":[]},{"given":"Aakash","family":"Kumar","sequence":"additional","affiliation":[]},{"given":"Munawar","family":"Ali","sequence":"additional","affiliation":[]},{"given":"Yu","family":"Kang","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2024,4,4]]},"reference":[{"issue":"7","key":"11593_CR1","doi-asserted-by":"publisher","first-page":"1949","DOI":"10.1007\/s00521-019-04491-4","volume":"32","author":"L Zhang","year":"2020","unstructured":"Zhang L, Sheng Z, Li Y, Sun Q, Zhao Y, Feng D (2020) Image object detection and semantic segmentation based on convolutional neural network. Neural Comput Appl 32(7):1949\u20131958. https:\/\/doi.org\/10.1007\/s00521-019-04491-4","journal-title":"Neural Comput Appl"},{"key":"11593_CR2","doi-asserted-by":"publisher","unstructured":"Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, pp 580\u2013587, https:\/\/doi.org\/10.1109\/CVPR.2014.81","DOI":"10.1109\/CVPR.2014.81"},{"key":"11593_CR3","doi-asserted-by":"publisher","unstructured":"He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit vol 2016, pp 770\u2013778, https:\/\/doi.org\/10.1109\/CVPR.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"issue":"1","key":"11593_CR4","doi-asserted-by":"publisher","first-page":"167","DOI":"10.32604\/cmc.2018.02356","volume":"57","author":"W Fang","year":"2018","unstructured":"Fang W, Zhang F, Sheng VS, Ding Y (2018) A method for improving CNN-based image recognition using DCGAN. Comput Mater Contin 57(1):167\u2013178. https:\/\/doi.org\/10.32604\/cmc.2018.02356","journal-title":"Comput Mater Contin"},{"key":"11593_CR5","doi-asserted-by":"publisher","unstructured":"Nguyen PQ, Do T, Nguyen-Thi AT, Ngo TD, Le DD, Nguyen TAH (2016) Clustering web video search results with convolutional neural networks. In: NICS 2016\u2014Proc 2016 3rd Natl Found Sci Technol Dev Conf Inf Comput Sci pp 135\u2013140, https:\/\/doi.org\/10.1109\/NICS.2016.7725638","DOI":"10.1109\/NICS.2016.7725638"},{"key":"11593_CR6","doi-asserted-by":"publisher","DOI":"10.18178\/ijmlc.2022.12.5.1111","author":"A Kumar","year":"2022","unstructured":"Kumar A et al (2022) Structure level pruning of efficient convolutional neural networks with sparse group LASSO. Int J Mach Learn Comput. https:\/\/doi.org\/10.18178\/ijmlc.2022.12.5.1111","journal-title":"Int J Mach Learn Comput"},{"key":"11593_CR7","unstructured":"Babak Hassibi DGS (2014) Second order derivatives for network pruning: optimal brain surgeon. pp 1\u20138, 2014, [Online]. Available: https:\/\/authors.library.caltech.edu\/54983\/3\/647-second-order-derivatives-for-network-pruning-optimal-brain-surgeon(1).pdf"},{"issue":"1","key":"11593_CR8","first-page":"598","volume":"2","author":"Y Le Cun","year":"1990","unstructured":"Le Cun Y, Denker JS, Solla S (1990) Optimal brain damage. Adv Neural Inf Process Syst 2(1):598\u2013605","journal-title":"Adv Neural Inf Process Syst"},{"key":"11593_CR9","doi-asserted-by":"publisher","DOI":"10.1201\/9781003162810-6","author":"A Goncharenko","year":"2022","unstructured":"Goncharenko A, Denisov A, Alyamkin S (2022) Fast adjustable threshold for uniform neural network quantization. Low-Power Comput Vis. https:\/\/doi.org\/10.1201\/9781003162810-6","journal-title":"Low-Power Comput Vis"},{"key":"11593_CR10","doi-asserted-by":"publisher","unstructured":"Choukroun Y, Kravchik E, Yang F, Kisilev P (2019) Low-bit quantization of neural networks for efficient inference. In: Proceedings\u20142019 international conference on computer vision workshop, ICCVW 2019, https:\/\/doi.org\/10.1109\/ICCVW.2019.00363","DOI":"10.1109\/ICCVW.2019.00363"},{"key":"11593_CR11","doi-asserted-by":"publisher","unstructured":"Yu R et al (2018) NISP: pruning networks using neuron importance score propagation. In: Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, pp 9194\u20139203, https:\/\/doi.org\/10.1109\/CVPR.2018.00958","DOI":"10.1109\/CVPR.2018.00958"},{"key":"11593_CR12","unstructured":"Li H, Samet H, Kadav A, Durdanovic I, Graf HP (2016) Pruning filters for efficient convnets. In: 5th Int Conf Learn Represent ICLR 2017\u2014Conf Track Proc, pp 1\u201313"},{"key":"11593_CR13","doi-asserted-by":"publisher","unstructured":"He Y, Ding Y, Liu P, Zhu L, Zhang H, Yang Y (2020) Learning filter pruning criteria for deep convolutional neural networks acceleration. In: Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit vol 2, pp 2006\u20132015, https:\/\/doi.org\/10.1109\/CVPR42600.2020.00208","DOI":"10.1109\/CVPR42600.2020.00208"},{"key":"11593_CR14","doi-asserted-by":"publisher","unstructured":"Han S, Pool J, Tran J, Dally WJ (2015) Learning both weights and connections for efficient. Neural Netw, pp 1\u20139, https:\/\/doi.org\/10.1016\/S0140-6736(95)92525-2","DOI":"10.1016\/S0140-6736(95)92525-2"},{"key":"11593_CR15","doi-asserted-by":"publisher","unstructured":"Carreira-Perpi\u00f1\u00e1n MA, Idelbayev Y (2018) Learning-compression\u2019 algorithms for neural net pruning. In: Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, pp 8532\u20138541, https:\/\/doi.org\/10.1109\/CVPR.2018.00890","DOI":"10.1109\/CVPR.2018.00890"},{"key":"11593_CR16","doi-asserted-by":"publisher","unstructured":"Liu B, Wang M, Foroosh H, Tappen M, Penksy M (2015) Sparse convolutional neural networks. In: Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit, vol 07\u201312-June, pp 806\u2013814, 2015, https:\/\/doi.org\/10.1109\/CVPR.2015.7298681","DOI":"10.1109\/CVPR.2015.7298681"},{"key":"11593_CR17","unstructured":"Guo Y, Yao A, Chen Y (2016) Dynamic network surgery for efficient DNNs. [Online]. Available: http:\/\/arxiv.org\/abs\/1608.04493"},{"key":"11593_CR18","unstructured":"Molchanov D, Ashukha A, Vetrov D (2017) Variational dropout sparsifies deep neural networks. 2017, [Online]. Available: http:\/\/arxiv.org\/abs\/1701.05369"},{"key":"11593_CR19","unstructured":"van Baalen M et al (2020) Bayesian bits: unifying quantization and pruning. Adv Neural Inf Process Syst, vol 2020, no. NeurIPS"},{"key":"11593_CR20","doi-asserted-by":"publisher","unstructured":"Wang W, Fu C, Guo J, Cai D, He X (2019) COP: customized deep model compression via regularized correlation-based filter-level pruning. In: IJCAI Int Jt Conf Artif Intell, vol 2019, pp 3785\u20133791, https:\/\/doi.org\/10.24963\/ijcai.2019\/525","DOI":"10.24963\/ijcai.2019\/525"},{"key":"11593_CR21","doi-asserted-by":"publisher","unstructured":"He Y, Kang G, Dong X, Fu Y, Yang Y (2018) Soft filter pruning for accelerating deep convolutional neural networks. In: IJCAI Int Jt Conf Artif Intell vol 2018, pp 2234\u20132240, https:\/\/doi.org\/10.24963\/ijcai.2018\/309","DOI":"10.24963\/ijcai.2018\/309"},{"key":"11593_CR22","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-020-01894-y","author":"A Kumar","year":"2021","unstructured":"Kumar A, Shaikh AM, Li Y, Bilal H, Yin B (2021) Pruning filters with L1-norm and capped L1-norm for CNN compression. Appl Intell. https:\/\/doi.org\/10.1007\/s10489-020-01894-y","journal-title":"Appl Intell"},{"key":"11593_CR23","unstructured":"Blundell C, Cornebise J, Kavukcuoglu K, Wierstra D (2015) Weight uncertainty in neural networks. In: 32nd International conference on machine learning, ICML 2015"},{"issue":"January","key":"11593_CR24","first-page":"1269","volume":"2","author":"E Denton","year":"2014","unstructured":"Denton E, Zaremba W, Bruna J, LeCun Y, Fergus R (2014) Exploiting linear structure within convolutional networks for efficient evaluation. Adv Neural Inf Process Syst 2(January):1269\u20131277","journal-title":"Adv Neural Inf Process Syst"},{"issue":"4","key":"11593_CR25","doi-asserted-by":"publisher","first-page":"769","DOI":"10.1016\/j.infsof.2008.09.005","volume":"51","author":"E Guerra","year":"2009","unstructured":"Guerra E, de Lara J, Malizia A, D\u00edaz P (2009) Supporting user-oriented analysis for multi-view domain-specific visual languages. Inf Softw Technol 51(4):769\u2013784. https:\/\/doi.org\/10.1016\/j.infsof.2008.09.005","journal-title":"Inf Softw Technol"},{"issue":"2","key":"11593_CR26","doi-asserted-by":"publisher","first-page":"29","DOI":"10.1109\/MCI.2022.3155327","volume":"17","author":"LV Jospin","year":"2020","unstructured":"Jospin LV, Buntine W, Boussaid F, Laga H, Bennamoun M (2020) Hands-on Bayesian neural networks\u2014a tutorial for deep learning users. IEEE Comput Intell Mag 17(2):29\u201348. https:\/\/doi.org\/10.1109\/MCI.2022.3155327","journal-title":"IEEE Comput Intell Mag"},{"key":"11593_CR27","doi-asserted-by":"publisher","DOI":"10.1109\/OJSP.2023.3337718","author":"J Beckers","year":"2023","unstructured":"Beckers J, Van Erp B, Zhao Z, Kondrashov K, De Vries B (2023) Principled pruning of bayesian neural networks through variational free energy minimization. IEEE Open J Signal Process. https:\/\/doi.org\/10.1109\/OJSP.2023.3337718","journal-title":"IEEE Open J Signal Process"},{"key":"11593_CR28","unstructured":"Wu A, Nowozin S, Meeds E, Turner RE, Miguel Hern\u00e1ndez-Lobato J, Gaunt AL, Deterministic variational inference for robust Bayesian neural networks"},{"key":"11593_CR29","unstructured":"Hau\u00dfmann M, Hamprecht FA, Kandemir M, Sampling-free variational inference of bayesian neural networks by variance backpropagation"},{"key":"11593_CR30","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4612-0745-0","volume-title":"Bayesian learning for neural networks","author":"RM Neal","year":"1996","unstructured":"Neal RM (1996) Bayesian learning for neural networks. Springer, New York, NY"},{"key":"11593_CR31","doi-asserted-by":"publisher","first-page":"4820","DOI":"10.1109\/CVPR.2016.521","volume":"2016","author":"J Wu","year":"2016","unstructured":"Wu J, Leng C, Wang Y, Hu Q, Cheng J (2016) Quantized convolutional neural networks for mobile devices. Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit 2016:4820\u20134828. https:\/\/doi.org\/10.1109\/CVPR.2016.521","journal-title":"Proc IEEE Comput Soc Conf Comput Vis Pattern Recognit"},{"issue":"4","key":"11593_CR32","doi-asserted-by":"publisher","first-page":"1","DOI":"10.3390\/informatics8040077","volume":"8","author":"A Alqahtani","year":"2021","unstructured":"Alqahtani A, Xie X, Jones MW (2021) Literature review of deep network compression. Informatics 8(4):1\u201312. https:\/\/doi.org\/10.3390\/informatics8040077","journal-title":"Informatics"},{"issue":"12","key":"11593_CR33","doi-asserted-by":"publisher","first-page":"3773","DOI":"10.1007\/s13042-022-01624-5","volume":"13","author":"A Kumar","year":"2022","unstructured":"Kumar A, Yin B, Shaikh AM, Ali M, Wei W (2022) CorrNet: pearson correlation based pruning for efficient convolutional neural networks. Int J Mach Learn Cybern 13(12):3773\u20133783. https:\/\/doi.org\/10.1007\/s13042-022-01624-5","journal-title":"Int J Mach Learn Cybern"},{"key":"11593_CR34","unstructured":"Han S, Mao H, Dally WJ (2015) Deep compression: compressing deep neural networks with pruning, trained quantization and Huffman coding. pp 1\u201314, 2015, abs\/1510.00149\/1510.00149"},{"key":"11593_CR35","unstructured":"Liu Z, Sun M, Zhou T, Huang G, Darrell T (2019) Rethinking the value of network pruning. In: 7th Int Conf Learn Represent ICLR 2019, pp 1\u201321, 2019"},{"key":"11593_CR36","unstructured":"Wen W, Wu C, Wang Y, Chen Y, Li H (2016) Learning structured sparsity in deep neural networks. 2016, [Online]. Available: http:\/\/arxiv.org\/abs\/1608.03665"},{"key":"11593_CR37","unstructured":"Frankle J, Dziugaite GK, Roy DM, Carbin M (2019) Stabilizing the lottery ticket hypothesis. [Online]. Available: http:\/\/arxiv.org\/abs\/1903.01611"},{"key":"11593_CR38","unstructured":"Chen W, Wilson JT, Tyree S, Weinberger KQ, Chen Y (2015) Compressing neural networks with the hashing trick. [Online]. Available: http:\/\/arxiv.org\/abs\/1504.04788"},{"key":"11593_CR39","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00447","author":"Y He","year":"2019","unstructured":"He Y, Liu P, Wang Z, Hu Z, Yang Y (2019) Filter pruning via geometric median for deep convolutional neural networks acceleration. Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn. https:\/\/doi.org\/10.1109\/CVPR.2019.00447","journal-title":"Proc IEEE Comput Soc Conf Comput Vis Pattern Recogn"},{"key":"11593_CR40","doi-asserted-by":"crossref","unstructured":"He Z, Qian Y, Wang Y, Wang B., Guan X, Gu Z, Zhou W (2022) Filter pruning via feature discrimination in deep neural networks. In: European conference on computer vision (pp 245\u2013261). Cham: Springer Nature Switzerland","DOI":"10.1007\/978-3-031-19803-8_15"},{"key":"11593_CR41","unstructured":"Shridhar K, Laumann F, Maurin AL, Olsen M, Liwicki M (2018) Bayesian convolutional neural networks with variational inference. arXiv:1806.05978 [cs.LG]"},{"key":"11593_CR42","doi-asserted-by":"publisher","DOI":"10.1214\/aoms\/1177729694","author":"S Kullback","year":"1951","unstructured":"Kullback S, Leibler RA (1951) On information and sufficiency. Ann Math Stat. https:\/\/doi.org\/10.1214\/aoms\/1177729694","journal-title":"Ann Math Stat"},{"key":"11593_CR43","doi-asserted-by":"publisher","DOI":"10.1109\/TIT.2005.850085","author":"JS Yedidia","year":"2005","unstructured":"Yedidia JS, Freeman WT, Weiss Y (2005) Constructing free-energy approximations and generalized belief propagation algorithms. IEEE Trans Inf Theory. https:\/\/doi.org\/10.1109\/TIT.2005.850085","journal-title":"IEEE Trans Inf Theory"},{"key":"11593_CR44","doi-asserted-by":"publisher","DOI":"10.1007\/978-94-011-5014-9_12","author":"RM Neal","year":"1998","unstructured":"Neal RM, Hinton GE (1998) A view of the Em algorithm that justifies incremental, sparse, and other variants. Learn Graph Models. https:\/\/doi.org\/10.1007\/978-94-011-5014-9_12","journal-title":"Learn Graph Models"},{"key":"11593_CR45","unstructured":"Kingma DP, Salimans T, Welling M (2015) Variational dropout and the local reparameterization trick. In: Advances in neural information processing systems"},{"key":"11593_CR46","unstructured":"Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: 3rd international conference on learning representations, ICLR 2015\u2014conference track proceedings"},{"key":"11593_CR47","unstructured":"Tuv E, Borisov A, Runger G, Torkkola K (2009) Feature selection with ensembles, artificial variables, and redundancy elimination. J Mach Learn Res"},{"key":"11593_CR48","doi-asserted-by":"publisher","first-page":"171924","DOI":"10.1109\/ACCESS.2020.3024992","volume":"8","author":"SA Aketi","year":"2020","unstructured":"Aketi SA, Roy S, Raghunathan A, Roy K (2020) Gradual channel pruning while training using feature relevance scores for convolutional neural networks. IEEE Access 8:171924\u2013171932. https:\/\/doi.org\/10.1109\/ACCESS.2020.3024992","journal-title":"IEEE Access"},{"key":"11593_CR49","doi-asserted-by":"crossref","unstructured":"Yan Z, Xing P, Wang Y, Tian Y (2020) Prune it yourself: automated pruning by multiple level sensitivity. In: 2020 IEEE Conference Multimedia Information Processing Retrievel pp 73\u201378, 2020","DOI":"10.1109\/MIPR49039.2020.00022"},{"key":"11593_CR50","doi-asserted-by":"publisher","first-page":"106816","DOI":"10.1016\/j.csda.2019.106816","volume":"142","author":"Y Kwon","year":"2020","unstructured":"Kwon Y, Won JH, Kim BJ, Paik MC (2020) Uncertainty quantification using Bayesian neural networks in classification: application to biomedical image segmentation. Comput Stat Data Anal 142:106816. https:\/\/doi.org\/10.1016\/j.csda.2019.106816","journal-title":"Comput Stat Data Anal"},{"key":"11593_CR51","unstructured":"Hafner D, Tran D, Lillicrap T, Irpan A, Davidson J (2018) Noise contrastive priors for functional uncertainty. 2018, [Online]. Available: http:\/\/arxiv.org\/abs\/1807.09289"},{"key":"11593_CR52","first-page":"2672","volume":"3","author":"IJ Goodfellow","year":"2024","unstructured":"Goodfellow IJ et al (2024) Generative adversarial networks. Sci Robot 3:2672\u20132680","journal-title":"Sci Robot"}],"container-title":["Neural Processing Letters"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11593-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11063-024-11593-1\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11063-024-11593-1.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,16]],"date-time":"2024-05-16T20:47:41Z","timestamp":1715892461000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11063-024-11593-1"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,4]]},"references-count":52,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,4]]}},"alternative-id":["11593"],"URL":"https:\/\/doi.org\/10.1007\/s11063-024-11593-1","relation":{},"ISSN":["1573-773X"],"issn-type":[{"value":"1573-773X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,4]]},"assertion":[{"value":"12 March 2024","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 April 2024","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors have no competing interests to declare that are relevant to the content of this article.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}},{"value":"I certify that there is no actual or potential conflict of interest about this article. This research does not involve human participants and\/or animals and also does not require informed consent.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethical approval"}}],"article-number":"140"}}