{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,16]],"date-time":"2025-10-16T07:02:13Z","timestamp":1760598133964,"version":"build-2065373602"},"reference-count":43,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2022,1,13]],"date-time":"2022-01-13T00:00:00Z","timestamp":1642032000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61720106005"],"award-info":[{"award-number":["61720106005"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Symmetry"],"abstract":"<jats:p>In this paper, a novel smooth group L1\/2 (SGL1\/2) regularization method is proposed for pruning hidden nodes of the fully connected layer in convolution neural networks. Usually, the selection of nodes and weights is based on experience, and the convolution filter is symmetric in the convolution neural network. The main contribution of SGL1\/2 is to try to approximate the weights to 0 at the group level. Therefore, we will be able to prune the hidden node if the corresponding weights are all close to 0. Furthermore, the feasibility analysis of this new method is carried out under some reasonable assumptions due to the smooth function. The numerical results demonstrate the superiority of the SGL1\/2 method with respect to sparsity, without damaging the classification performance.<\/jats:p>","DOI":"10.3390\/sym14010154","type":"journal-article","created":{"date-parts":[[2022,1,14]],"date-time":"2022-01-14T03:14:56Z","timestamp":1642130096000},"page":"154","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Smooth Group L1\/2 Regularization for Pruning Convolutional Neural Networks"],"prefix":"10.3390","volume":"14","author":[{"given":"Yuan","family":"Bao","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, Dalian Maritime University, Dalian 116026, China"}]},{"given":"Zhaobin","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, Dalian Maritime University, Dalian 116026, China"}]},{"given":"Zhongxuan","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Software, Dalian University of Technology, Dalian 116620, China"}]},{"given":"Sibo","family":"Yang","sequence":"additional","affiliation":[{"name":"School of Science, Dalian Maritime University, Dalian 116026, China"}]}],"member":"1968","published-online":{"date-parts":[[2022,1,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Sharma, P., Singh, A., Singh, K.K., and Dhull, A. (2021). Vehicle identification using modified region based convolution network for intelligent transportation system. Multimed. Tools Appl., 1\u201325.","DOI":"10.1007\/s11042-020-10366-x"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"8","DOI":"10.1016\/j.patrec.2020.02.015","article-title":"Nom document digitalization by deep convolution neural networks","volume":"133","author":"Nguyen","year":"2020","journal-title":"Pattern Recognit. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Jogin, M., Madhulika, M.S., Divya, G.D., Meghana, R.K., and Apoorva, S. (2018, January 18\u201319). Feature Extraction using Convolution Neural Networks (CNN) and Deep Learning. Proceedings of the 2018 3rd IEEE International Conference on Recent Trends in Electronics, Information and Communication Technology (RTEICT), Bangalore, India.","DOI":"10.1109\/RTEICT42901.2018.9012507"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"2719","DOI":"10.1007\/s10586-017-1435-x","article-title":"Hand gesture recognition based on convolution neural network","volume":"22","author":"Li","year":"2019","journal-title":"Clust. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Brachmann, A., and Redies, C. (2016). Using convolutional neural network filters to measure left-right mirror symmetry in images. Symmetry, 8.","DOI":"10.3390\/sym8120144"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"909","DOI":"10.1007\/s00521-020-05288-6","article-title":"A new pose accuracy compensation method for parallel manipulators based on hybrid artificial neural network","volume":"33","author":"Yu","year":"2021","journal-title":"Neural Comput. Appl."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1016\/j.ins.2016.11.020","article-title":"Convergence analyses on sparse feedforward neural networks via group lasso regularization","volume":"381","author":"Wang","year":"2017","journal-title":"Inf. Sci."},{"key":"ref_8","unstructured":"Ng, A.Y. (2004, January 4\u20138). Feature selection, L1 vs. L2 regularization, and rotational invariance. Proceedings of the Twenty-First International Conference on Machine Learning, Banff, AB, Canada."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"1152","DOI":"10.1007\/s10489-020-01894-y","article-title":"Pruning filters with L1-norm and capped L1-norm for CNN compression","volume":"51","author":"Bilal","year":"2021","journal-title":"Appl. Intell."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"9479","DOI":"10.1007\/s00521-019-04460-x","article-title":"A new discriminative collaborative representation-based classification method via L2 regularizations","volume":"32","author":"Gou","year":"2020","journal-title":"Neural Comput. Appl."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"1013","DOI":"10.1109\/TNNLS.2012.2197412","article-title":"L1\/2 regularization: A thresholding representation theory and a fast solver","volume":"23","author":"Xu","year":"2012","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3969","DOI":"10.1007\/s11042-020-09738-0","article-title":"Early diagnosis model of Alzheimer\u2019s Disease based on sparse logistic regression","volume":"80","author":"Xiao","year":"2020","journal-title":"Multimed. Tools Appl."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2682","DOI":"10.1109\/TSP.2020.2985591","article-title":"Alternating Group Lasso for Block-Term Tensor Decomposition and Application to ECG Source Separation","volume":"68","author":"Goulart","year":"2020","journal-title":"IEEE Trans. Signal Process."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TIM.2020.3043940","article-title":"Collaborative Double Sparse Period-Group Lasso for Bearing Fault Diagnosis","volume":"70","author":"Diwu","year":"2020","journal-title":"IEEE Trans. Instrum. Meas."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/j.patrec.2019.12.020","article-title":"A group lasso based sparse KNN classifier","volume":"131","author":"Zheng","year":"2020","journal-title":"Pattern Recognit. Lett."},{"key":"ref_16","unstructured":"Friedman, J., Hastie, T., and Tibshirani, R. (2010). A note on the group lasso and a sparse group lasso. arXiv."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"9540","DOI":"10.1109\/ACCESS.2018.2890740","article-title":"Group L1\/2 regularization for pruning hidden layer nodes of feedforward neural networks","volume":"7","author":"Alemu","year":"2019","journal-title":"IEEE Access"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"72","DOI":"10.1016\/j.neunet.2013.11.006","article-title":"Batch gradient method with smoothing L1\/2 regularization for training of feedforward neural networks","volume":"50","author":"Wu","year":"2014","journal-title":"Neural Netw."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"333","DOI":"10.1016\/j.neucom.2014.09.031","article-title":"Convergence of batch gradient learning algorithm with smoothing L1\/2 regularization for Sigma\u2013Pi\u2013Sigma neural networks","volume":"151","author":"Liu","year":"2015","journal-title":"Neurocomputing"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.csl.2020.101159","article-title":"Gated dynamic convolutions with deep layer fusion for abstractive document summarization","volume":"66","author":"Kwon","year":"2021","journal-title":"Comput. Speech Lang."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"5201","DOI":"10.1109\/TIP.2019.2917234","article-title":"REMAP: Multi-layer entropy-guided pooling of dense CNN features for image retrieval","volume":"28","author":"Husain","year":"2019","journal-title":"IEEE Trans. Image Process."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Richter, O., and Wattenhofer, R. (2018, January 5\u20137). TreeConnect: A Sparse Alternative to Fully Connected Layers. Proceedings of the 2018 IEEE 30th International Conference on Tools with Artificial Intelligence (ICTAI), Volos, Greece.","DOI":"10.1109\/ICTAI.2018.00143"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1016\/j.neunet.2018.11.005","article-title":"A comparison of deep networks with ReLU activation function and linear spline-type methods","volume":"110","author":"Eckle","year":"2019","journal-title":"Neural Netw."},{"key":"ref_24","unstructured":"Guo, Z.Y., Shu, X., Liu, C.Y., and Lei, L.I. (2018). A Recognition Algorithm of Flower Based on Convolution Neural Network with ReLU Function. Comput. Technol. Dev., 05. Available online: http:\/\/en.cnki.com.cn\/Article_en\/CJFDTotal-WJFZ201805035.htm."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"385","DOI":"10.1007\/s00170-005-0089-7","article-title":"A study on using deviation function method to reshape a rack cutter","volume":"30","author":"Yang","year":"2006","journal-title":"Int. J. Adv. Manuf. Technol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1159","DOI":"10.1007\/s11432-010-0090-0","article-title":"L1\/2 regularization","volume":"53","author":"Xu","year":"2010","journal-title":"Sci. China Inf. Sci."},{"key":"ref_27","unstructured":"Haykin, S. (1998). Neural Networks: A Comprehensive Foundation, Prentice Hall. [3rd ed.]."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"182","DOI":"10.1109\/72.363438","article-title":"Gradient descent learning algorithm overview: A general dynamical systems perspective","volume":"6","author":"Baldi","year":"1995","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_29","unstructured":"Zhang, Z. (2016). Derivation of Backpropagation in Convolutional Neural Network (CNN), University of Tennessee."},{"key":"ref_30","first-page":"2539","article-title":"Sparsity of Hidden Layer Nodes Based on Bayesian Extreme Learning Machine","volume":"24","author":"Wu","year":"2017","journal-title":"Control Eng. China"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"962","DOI":"10.2991\/ijcis.11.1.73","article-title":"Sparsity-driven weighted ensemble classifier","volume":"11","author":"Nar","year":"2018","journal-title":"Int. J. Comput. Intell. Syst."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"607","DOI":"10.1038\/381607a0","article-title":"Emergence of simple-cell receptive field properties by learning a sparse code for natural images","volume":"381","author":"Olshausen","year":"1996","journal-title":"Nature"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1038\/226177a0","article-title":"Interaction effects in parafoveal letter recognition","volume":"226","author":"Bouma","year":"1970","journal-title":"Nature"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Carvalho, E.F., and Engel, P.M. (2013, January 19\u201324). Convolutional sparse feature descriptor for object recognition in cifar-10. Proceedings of the 2013 Brazilian Conference on Intelligent Systems, Fortaleza, Brazil.","DOI":"10.1109\/BRACIS.2013.30"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"113609","DOI":"10.1016\/j.cma.2020.113609","article-title":"The arithmetic optimization algorithm","volume":"376","author":"Abualigah","year":"2021","journal-title":"Comput. Methods Appl. Mech. Eng."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"1304","DOI":"10.1109\/TNNLS.2012.2199516","article-title":"Study on the impact of partition-induced dataset shift on k-fold cross-validation","volume":"23","author":"Herrera","year":"2012","journal-title":"IEEE Trans. Neural Networks Learn. Syst."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"503","DOI":"10.1093\/biomet\/76.3.503","article-title":"A comparative study of ordinary cross-validation, v-fold cross-validation and the repeated learning-testing methods","volume":"76","author":"Burman","year":"1989","journal-title":"Biometrika"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"244","DOI":"10.1016\/j.ecolmodel.2007.10.005","article-title":"Three way k-fold cross-validation of resource selection functions","volume":"212","author":"Wiens","year":"2008","journal-title":"Ecol. Model."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"1064","DOI":"10.1109\/TNN.2002.1031939","article-title":"Two highly efficient second-order algorithms for training feedforward networks","volume":"13","author":"Ampazis","year":"2002","journal-title":"IEEE Trans. Neural Netw."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zubic, S., Wahlroos, A., Altonen, J., Balcerek, P., and Dawidowski, P. (2016, January 7\u201310). Managing Post-fault Oscillation Phenomenon in Compensated MV-networks. Proceedings of the 13th IET International Conference on Developments in Power System Protection (DPSP 2016), Edinburgh, UK.","DOI":"10.1049\/cp.2016.0034"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"314","DOI":"10.1108\/IJESM-06-2017-0007","article-title":"Oscillation phenomenon and its mechanism of an energy-saving and emission-reduction system","volume":"12","author":"Yin","year":"2018","journal-title":"Int. J. Energy Sect. Manag."},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"183","DOI":"10.7153\/mia-02-16","article-title":"New estimation of the remainder in Taylor\u2019s formula using Gr\u00fcss\u2019 type inequalities and applications","volume":"2","author":"Dragomir","year":"1999","journal-title":"Math. Inequalities Appl."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"1630","DOI":"10.1016\/j.ins.2009.12.030","article-title":"A modified gradient-based neuro-fuzzy learning algorithm and its convergence","volume":"180","author":"Wu","year":"2010","journal-title":"Inf. Sci."}],"container-title":["Symmetry"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-8994\/14\/1\/154\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,13]],"date-time":"2025-10-13T13:27:14Z","timestamp":1760362034000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-8994\/14\/1\/154"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,1,13]]},"references-count":43,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2022,1]]}},"alternative-id":["sym14010154"],"URL":"https:\/\/doi.org\/10.3390\/sym14010154","relation":{},"ISSN":["2073-8994"],"issn-type":[{"type":"electronic","value":"2073-8994"}],"subject":[],"published":{"date-parts":[[2022,1,13]]}}}