{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,8]],"date-time":"2026-04-08T08:54:49Z","timestamp":1775638489647,"version":"3.50.1"},"reference-count":86,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2022,3,4]],"date-time":"2022-03-04T00:00:00Z","timestamp":1646352000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Introduced in the late 1980s for generalization purposes, pruning has now become a staple for compressing deep neural networks. Despite many innovations in recent decades, pruning approaches still face core issues that hinder their performance or scalability. Drawing inspiration from early work in the field, and especially the use of weight decay to achieve sparsity, we introduce Selective Weight Decay (SWD), which carries out efficient, continuous pruning throughout training. Our approach, theoretically grounded on Lagrangian smoothing, is versatile and can be applied to multiple tasks, networks, and pruning structures. We show that SWD compares favorably to state-of-the-art approaches, in terms of performance-to-parameters ratio, on the CIFAR-10, Cora, and ImageNet ILSVRC2012 datasets.<\/jats:p>","DOI":"10.3390\/jimaging8030064","type":"journal-article","created":{"date-parts":[[2022,3,6]],"date-time":"2022-03-06T20:38:16Z","timestamp":1646599096000},"page":"64","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":22,"title":["Rethinking Weight Decay for Efficient Neural Network Pruning"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2347-2345","authenticated-orcid":false,"given":"Hugo","family":"Tessier","sequence":"first","affiliation":[{"name":"Stellantis, Centre Technique V\u00e9lizy, 78140 V\u00e9lizy-Villacoublay, France"},{"name":"IMT Atlantique, Lab-STICC, UMR CNRS 6285, 29238 Brest, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4353-4542","authenticated-orcid":false,"given":"Vincent","family":"Gripon","sequence":"additional","affiliation":[{"name":"IMT Atlantique, Lab-STICC, UMR CNRS 6285, 29238 Brest, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9973-843X","authenticated-orcid":false,"given":"Mathieu","family":"L\u00e9onardon","sequence":"additional","affiliation":[{"name":"IMT Atlantique, Lab-STICC, UMR CNRS 6285, 29238 Brest, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Matthieu","family":"Arzel","sequence":"additional","affiliation":[{"name":"IMT Atlantique, Lab-STICC, UMR CNRS 6285, 29238 Brest, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9816-1886","authenticated-orcid":false,"given":"Thomas","family":"Hannagan","sequence":"additional","affiliation":[{"name":"Stellantis, Centre Technique V\u00e9lizy, 78140 V\u00e9lizy-Villacoublay, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"David","family":"Bertrand","sequence":"additional","affiliation":[{"name":"Stellantis, Centre Technique V\u00e9lizy, 78140 V\u00e9lizy-Villacoublay, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,4]]},"reference":[{"key":"ref_1","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky","year":"2012","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Le Cun, Y., Haffner, P., Bottou, L., and Bengio, Y. (1999). Object recognition with gradient-based learning. Shape, Contour and Grouping in Computer Vision, Springer.","DOI":"10.1007\/3-540-46805-6_19"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_4","unstructured":"Ioffe, S., and Szegedy, C. (2015). Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. arXiv."},{"key":"ref_5","unstructured":"Lin, M., Chen, Q., and Yan, S. (2013). Network in network. arXiv."},{"key":"ref_6","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very deep convolutional networks for large-scale image recognition. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., and Rabinovich, A. (2015, January 7\u201312). Going deeper with convolutions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Xie, S., Girshick, R., Doll\u00e1r, P., Tu, Z., and He, K. (2017, January 21\u201326). Aggregated residual transformations for deep neural networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.634"},{"key":"ref_9","unstructured":"Boukli Hacene, G. (2019). Processing and Learning Deep Neural Networks on Chip. [Ph.D. Thesis, Ecole nationale sup\u00e9rieure Mines-T\u00e9l\u00e9com Atlantique Bretagne Pays de la Loire]."},{"key":"ref_10","first-page":"9","article-title":"Distilling the Knowledge in a Neural Network","volume":"1050","author":"Hinton","year":"2015","journal-title":"Stat"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Lassance, C., Bontonou, M., Hacene, G.B., Gripon, V., Tang, J., and Ortega, A. (2020, January 4\u20138). Deep Geometric Knowledge Distillation with Graphs. Proceedings of the ICASSP 2020\u20142020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9053986"},{"key":"ref_12","unstructured":"Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., and Garnett, R. (2015). BinaryConnect: Training Deep Neural Networks with binary weights during propagations. Advances in Neural Information Processing Systems 28, Curran Associates, Inc."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Rastegari, M., Ordonez, V., Redmon, J., and Farhadi, A. (2016). Xnor-net: Imagenet classification using binary convolutional neural networks. European Conference on Computer Vision, Springer.","DOI":"10.1007\/978-3-319-46493-0_32"},{"key":"ref_14","unstructured":"Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., and Weinberger, K.Q. (2014). Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation. Advances in Neural Information Processing Systems 27, Curran Associates, Inc."},{"key":"ref_15","unstructured":"Cortes, C., Lawrence, N.D., Lee, D.D., Sugiyama, M., and Garnett, R. (2015). Learning both Weights and Connections for Efficient Neural Network. Advances in Neural Information Processing Systems 28, Curran Associates, Inc."},{"key":"ref_16","unstructured":"Han, S., Mao, H., and Dally, W.J. (2015). Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S., and Zhang, C. (2017, January 22\u201329). Learning Efficient Convolutional Networks Through Network Slimming. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.298"},{"key":"ref_18","unstructured":"Li, H., Kadav, A., Durdanovic, I., Samet, H., and Graf, H.P. (2016). Pruning Filters for Efficient ConvNets. arXiv."},{"key":"ref_19","unstructured":"Liu, Z., Sun, M., Zhou, T., Huang, G., and Darrell, T. (May, January 30). Rethinking the Value of Network Pruning. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_20","unstructured":"Gale, T., Elsen, E., and Hooker, S. (2019). The State of Sparsity in Deep Neural Networks. arXiv."},{"key":"ref_21","unstructured":"Louizos, C., Welling, M., and Kingma, D.P. (May, January 30). Learning Sparse Neural Networks through L_0 Regularization. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_22","unstructured":"Moody, J.E., Hanson, S.J., and Lippmann, R.P. (1992). A Simple Weight Decay Can Improve Generalization. Advances in Neural Information Processing Systems 4, Morgan-Kaufmann."},{"key":"ref_23","unstructured":"Plaut, D.C., Nowlan, S.J., and Hinton, G.E. (1986). Experiments on Learning Back Propagation, Carnegie\u2013Mellon University. Technical Report CMU\u2013CS\u201386\u2013126."},{"key":"ref_24","unstructured":"Hanson, S.J., Cowan, J.D., and Giles, C.L. (1993). Second order derivatives for network pruning: Optimal Brain Surgeon. Advances in Neural Information Processing Systems 5, Morgan-Kaufmann."},{"key":"ref_25","unstructured":"Le Cun, Y., Denker, J.S., and Solla, S.A. (1990). Optimal Brain Damage. Advances in Neural Information Processing Systems 2, Morgan Kaufmann Publishers Inc."},{"key":"ref_26","unstructured":"Touretzky, D.S. (1989). Skeletonization: A Technique for Trimming the Fat from a Network via Relevance Assessment. Advances in Neural Information Processing Systems 1, Morgan-Kaufmann."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"740","DOI":"10.1109\/72.248452","article-title":"Pruning algorithms-a survey","volume":"4","author":"Reed","year":"1993","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_28","unstructured":"Blalock, D., Gonzalez Ortiz, J.J., Frankle, J., and Guttag, J. (2020). What is the State of Neural Network Pruning?. arXiv."},{"key":"ref_29","unstructured":"Anwar, S., and Sung, W. (2016). Compact Deep Convolutional Neural Networks With Coarse Pruning. arXiv."},{"key":"ref_30","unstructured":"Hu, H., Peng, R., Tai, Y.W., and Tang, C.K. (2016). Network Trimming: A Data-Driven Neuron Pruning Approach towards Efficient Deep Architectures. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Luo, J.H., Wu, J., and Lin, W. (2017, January 22\u201329). ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.541"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Srinivas, S., and Venkatesh Babu, R. (2015). Data-free parameter pruning for Deep Neural Networks. arXiv.","DOI":"10.5244\/C.29.31"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Yu, R., Li, A., Chen, C., Lai, J., Morariu, V.I., Han, X., Gao, M., Lin, C., and Davis, L.S. (2018, January 18\u201323). NISP: Pruning Networks Using Neuron Importance Score Propagation. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00958"},{"key":"ref_34","unstructured":"Cowan, J.D., Tesauro, G., and Alspector, J. (1994). Optimal Brain Surgeon: Extensions and performance comparisons. Advances in Neural Information Processing Systems 6, Morgan-Kaufmann."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"239","DOI":"10.1109\/72.80236","article-title":"A simple procedure for pruning back-propagation trained neural networks","volume":"1","author":"Karnin","year":"1990","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_36","unstructured":"Mozer, M.C., Jordan, M.I., and Petsche, T. (1997). Early Brain Damage. Advances in Neural Information Processing Systems 9, MIT Press."},{"key":"ref_37","unstructured":"Guyon, I., Luxburg, U.V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (2017). Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon. Advances in Neural Information Processing Systems 30, Curran Associates, Inc."},{"key":"ref_38","unstructured":"Molchanov, P., Tyree, S., Karras, T., Aila, T., and Kautz, J. (2017, January 24\u201326). Pruning convolutional neural networks for resource efficient inference. Proceedings of the 5th International Conference on Learning Representations, ICLR 2017-Conference Track Proceedings, Toulon, France."},{"key":"ref_39","unstructured":"Touretzky, D.S. (1989). A Back-Propagation Algorithm with Optimal Use of Hidden Units. Advances in Neural Information Processing Systems 1, Morgan-Kaufmann."},{"key":"ref_40","unstructured":"Touretzky, D.S. (1989). Comparing Biases for Minimal Network Construction with Back-Propagation. Advances in Neural Information Processing Systems 1, Morgan-Kaufmann."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"6600","DOI":"10.1103\/PhysRevA.39.6600","article-title":"Pruning versus clipping in neural networks","volume":"39","author":"Janowsky","year":"1989","journal-title":"Phys. Rev. A"},{"key":"ref_42","unstructured":"Segee, B.E., and Carter, M.J. (1991, January 8\u201314). Fault tolerance of pruned multilayer networks. Proceedings of the IJCNN-91-Seattle International Joint Conference on Neural Networks, Seattle, WA, USA."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep Learning","volume":"521","author":"Bengio","year":"2015","journal-title":"Nature"},{"key":"ref_44","unstructured":"Bellec, G., Kappel, D., Maass, W., and Legenstein, R. (2017). Deep Rewiring: Training very sparse deep networks. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1487","DOI":"10.1109\/TC.2019.2914438","article-title":"NeST: A Neural Network Synthesis Tool Based on a Grow-and-Prune Paradigm","volume":"68","author":"Dai","year":"2019","journal-title":"IEEE Trans. Comput."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"He, Y., Kang, G., Dong, X., Fu, Y., and Yang, Y. (2018, January 13\u201319). Soft filter pruning for accelerating deep convolutional neural networks. Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Sweden.","DOI":"10.24963\/ijcai.2018\/309"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Mocanu, D., Mocanu, E., Stone, P., Nguyen, P., Gibescu, M., and Liotta, A. (2018). Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science. Nat. Commun., 9.","DOI":"10.1038\/s41467-018-04316-3"},{"key":"ref_48","unstructured":"Dettmers, T., and Zettlemoyer, L. (2019). Sparse Networks from Scratch: Faster Training without Losing Performance. arXiv."},{"key":"ref_49","unstructured":"Evci, U., Gale, T., Menick, J., Castro, P.S., and Elsen, E. (2019). Rigging the Lottery: Making All Tickets Winners. arXiv."},{"key":"ref_50","unstructured":"Mostafa, H., and Wang, X. (2019). Parameter Efficient Training of Deep Convolutional Neural Networks by Dynamic Sparse Reparameterization. arXiv."},{"key":"ref_51","unstructured":"Frankle, J., Dziugaite, G.K., Roy, D.M., and Carbin, M. (2020). Pruning Neural Networks at Initialization: Why are We Missing the Mark?. arXiv."},{"key":"ref_52","unstructured":"Lee, N., Ajanthan, T., and Torr, P.H. (2019, January 6\u20139). SNIP: Single-shot network pruning based on connection sensitivity. Proceedings of the International Conference on Learning Representations, ICLR, New Orleans, LA, USA."},{"key":"ref_53","first-page":"6377","article-title":"Pruning neural networks without any data by iteratively conserving synaptic flow","volume":"33","author":"Tanaka","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_54","unstructured":"Wang, C., Zhang, G., and Grosse, R. (2019, January 6\u20139). Picking Winning Tickets Before Training by Preserving Gradient Flow. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_55","unstructured":"Frankle, J., and Carbin, M. (May, January 30). The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks. Proceedings of the International Conference on Learning Representations, Vancouver, BC, Canada."},{"key":"ref_56","unstructured":"Frankle, J., Dziugaite, G.K., Roy, D., and Carbin, M. (2020, January 13\u201318). Linear mode connectivity and the lottery ticket hypothesis. Proceedings of the International Conference on Machine Learning, PMLR, Virtual."},{"key":"ref_57","unstructured":"Frankle, J., Dziugaite, G.K., Roy, D.M., and Carbin, M. (2019). Stabilizing the lottery ticket hypothesis. arXiv."},{"key":"ref_58","unstructured":"Malach, E., Yehudai, G., Shalev-Schwartz, S., and Shamir, O. (2020, January 13\u201318). Proving the lottery ticket hypothesis: Pruning is all you need. Proceedings of the International Conference on Machine Learning, PMLR, Virtual."},{"key":"ref_59","first-page":"6","article-title":"One ticket to win them all: Generalizing lottery ticket initializations across datasets and optimizers","volume":"1050","author":"Morcos","year":"2019","journal-title":"Stat"},{"key":"ref_60","unstructured":"Zhou, H., Lan, J., Liu, R., and Yosinski, J. (2019). Deconstructing lottery tickets: Zeros, signs, and the supermask. arXiv."},{"key":"ref_61","unstructured":"Renda, A., Frankle, J., and Carbin, M. (2020). Comparing rewinding and fine-tuning in neural network pruning. arXiv."},{"key":"ref_62","unstructured":"Lee, D.D., Sugiyama, M., Luxburg, U.V., Guyon, I., and Garnett, R. (2016). Dynamic Network Surgery for Efficient DNNs. Advances in Neural Information Processing Systems 29, Curran Associates, Inc."},{"key":"ref_63","doi-asserted-by":"crossref","unstructured":"Srinivas, S., Subramanya, A., and Venkatesh Babu, R. (2017, January 21\u201326). Training Sparse Neural Networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, Honolulu, HI, USA.","DOI":"10.1109\/CVPRW.2017.61"},{"key":"ref_64","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9-Buc, F., Fox, E., and Garnett, R. (2019). AutoPrune: Automatic Network Pruning by Regularizing Auxiliary Parameters. Advances in Neural Information Processing Systems, Curran Associates, Inc."},{"key":"ref_65","unstructured":"Dai, B., Zhu, C., Guo, B., and Wipf, D. (2018, January 10\u201315). Compressing neural networks using the variational information bottleneck. Proceedings of the International Conference on Machine Learning, PMLR, Stockholm, Sweden."},{"key":"ref_66","unstructured":"Louizos, C., Ullrich, K., and Welling, M. (2017, January 4\u20139). Bayesian Compression for Deep Learning. Proceedings of the 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_67","unstructured":"Molchanov, D., Ashukha, A., and Vetrov, D. (2017, January 6\u201311). Variational dropout sparsifies deep neural networks. Proceedings of the International Conference on Machine Learning, PMLR, Sydney, Australia."},{"key":"ref_68","unstructured":"Neklyudov, K., Molchanov, D., Ashukha, A., and Vetrov, D. (2017, January 4\u20139). Structured Bayesian pruning via log-normal multiplicative noise. Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_69","first-page":"13","article-title":"Soft weight-sharing for neural network compression","volume":"1050","author":"Ullrich","year":"2017","journal-title":"Stat"},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"473","DOI":"10.1162\/neco.1992.4.4.473","article-title":"Simplifying neural networks by soft weight-sharing","volume":"4","author":"Nowlan","year":"1992","journal-title":"Neural Comput."},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Anwar, S., Hwang, K., and Sung, W. (2015). Structured Pruning of Deep Convolutional Neural Networks. ACM J. Emerg. Technol. Comput. Syst., 13.","DOI":"10.1145\/3005348"},{"key":"ref_72","doi-asserted-by":"crossref","unstructured":"He, Y., Zhang, X., and Sun, J. (2017, January 22\u201329). Channel Pruning for Accelerating Very Deep Neural Networks. Proceedings of the 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy.","DOI":"10.1109\/ICCV.2017.155"},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Huang, Q., Zhou, K., You, S., and Neumann, U. (2018, January 12\u201315). Learning to Prune Filters in Convolutional Neural Networks. Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), Lake Tahoe, NV, USA.","DOI":"10.1109\/WACV.2018.00083"},{"key":"ref_74","unstructured":"Yamamoto, K., and Maeno, K. (2019). PCAS: Pruning Channels with Attention Statistics for Deep Network Compression. arXiv."},{"key":"ref_75","unstructured":"Hacene, G.B., Lassance, C., Gripon, V., Courbariaux, M., and Bengio, Y. (2019). Attention based pruning for shift networks. arXiv."},{"key":"ref_76","doi-asserted-by":"crossref","first-page":"257","DOI":"10.1007\/s10589-008-9218-1","article-title":"Algorithm for nonlinear optimization problems with binary variables","volume":"47","author":"Murray","year":"2010","journal-title":"Comput. Optim. Appl."},{"key":"ref_77","unstructured":"Wallach, H., Larochelle, H., Beygelzimer, A., d\u2019Alch\u00e9 Buc, F., Fox, E., and Garnett, R. (2019). PyTorch: An Imperative Style, High-Performance Deep Learning Library. Advances in Neural Information Processing Systems 32, Curran Associates, Inc."},{"key":"ref_78","unstructured":"Liu, Z., Xu, J., Peng, X., and Xiong, R. (2018, January 3\u20138). Frequency-domain dynamic pruning for convolutional neural networks. Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_79","unstructured":"Zhu, M., and Gupta, S. (2017). To prune, or not to prune: Exploring the efficacy of pruning for model compression. arXiv."},{"key":"ref_80","doi-asserted-by":"crossref","unstructured":"Molchanov, P., Mallya, A., Tyree, S., Frosio, I., and Kautz, J. (2019, January 15\u201320). Importance estimation for neural network pruning. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01152"},{"key":"ref_81","unstructured":"Ye, J., Lu, X., Lin, Z., and Wang, J.Z. (2018). Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers. arXiv."},{"key":"ref_82","unstructured":"Krizhevsky, A. (2009). Learning Multiple Layers of Features from Tiny Images. [Master\u2019s Thesis, University of Toronto]."},{"key":"ref_83","unstructured":"Ma, X., Lin, S., Ye, S., He, Z., Zhang, L., Yuan, G., Tan, S.H., Li, Z., Fan, D., and Qian, X. (2021). Non-Structured DNN Weight Pruning\u2013Is It Beneficial in Any Platform?. IEEE Trans. Neural Netw. Learn. Syst., 1\u201315."},{"key":"ref_84","unstructured":"Kipf, T.N., and Welling, M. (2016). Semi-Supervised Classification with Graph Convolutional Networks. arXiv."},{"key":"ref_85","first-page":"93","article-title":"Collective classification in network data","volume":"29","author":"Sen","year":"2008","journal-title":"AI Mag."},{"key":"ref_86","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/3\/64\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:32:17Z","timestamp":1760135537000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/8\/3\/64"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,4]]},"references-count":86,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2022,3]]}},"alternative-id":["jimaging8030064"],"URL":"https:\/\/doi.org\/10.3390\/jimaging8030064","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,4]]}}}