{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T01:56:10Z","timestamp":1760147770788,"version":"build-2065373602"},"reference-count":18,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2023,3,2]],"date-time":"2023-03-02T00:00:00Z","timestamp":1677715200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Application of deep neural networks (DNN) in edge computing has emerged as a consequence of the need of real time and distributed response of different devices in a large number of scenarios. To this end, shredding these original structures is urgent due to the high number of parameters needed to represent them. As a consequence, the most representative components of different layers are kept in order to maintain the network\u2019s accuracy as close as possible to the entire network\u2019s ones. To do so, two different approaches have been developed in this work. First, the Sparse Low Rank Method (SLR) has been applied to two different Fully Connected (FC) layers to watch their effect on the final response, and the method has been applied to the latest of these layers as a duplicate. On the contrary, SLRProp has been proposed as a variant case, where the relevances of the previous FC layer\u2019s components were weighed as the sum of the products of each of these neurons\u2019 absolute values and the relevances of the neurons from the last FC layer that are connected with the neurons from the previous FC layer. Thus, the relationship of relevances across layer was considered. Experiments have been carried out in well-known architectures to conclude whether the relevances throughout layers have less effect on the final response of the network than the independent relevances intra-layer.<\/jats:p>","DOI":"10.3390\/s23052718","type":"journal-article","created":{"date-parts":[[2023,3,2]],"date-time":"2023-03-02T02:10:59Z","timestamp":1677723059000},"page":"2718","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["SLRProp: A Back-Propagation Variant of Sparse Low Rank Method for DNNs Reduction"],"prefix":"10.3390","volume":"23","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3787-8186","authenticated-orcid":false,"given":"Asier","family":"Garmendia-Orbegozo","sequence":"first","affiliation":[{"name":"Department of Applied Mathematics, University of the Basque Country UPV\/EHU, 20600 Eibar, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5047-1033","authenticated-orcid":false,"given":"Jose David","family":"Nu\u00f1ez-Gonzalez","sequence":"additional","affiliation":[{"name":"Department of Applied Mathematics, University of the Basque Country UPV\/EHU, 20600 Eibar, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3775-2906","authenticated-orcid":false,"given":"Miguel Angel","family":"Anton","sequence":"additional","affiliation":[{"name":"TECNALIA, Basque Research and Technology Alliance (BRTA), 20009 San Sebastian, Spain"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,2]]},"reference":[{"key":"ref_1","unstructured":"Touretzky, D. (1989). Advances in Neural Information Processing Systems, Morgan-Kaufmann."},{"key":"ref_2","unstructured":"Hassibi, B., Stork, D., and Wolff, G. (April, January 28). Optimal Brain Surgeon and general network pruning. Proceedings of the IEEE International Conference on Neural Networks, San Francisco, CA, USA."},{"key":"ref_3","unstructured":"Molchanov, P., Tyree, S., Karras, T., Aila, T., and Kautz, J. (2016). Pruning Convolutional Neural Networks for Resource Efficient Inference. arXiv."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"3129","DOI":"10.1007\/s13042-019-01004-6","article-title":"Transfer Channel Pruning for Compressing Deep Domain Adaptation Models","volume":"10","author":"Yu","year":"2019","journal-title":"Int. J. Mach. Learn. Cibern."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Muthukrishnan, R., and Rohini, R. (2016, January 24). LASSO: A feature selection technique in predictive modeling for machine learning. Proceedings of the 2016 IEEE International Conference on Advances in Computer Applications (ICACA), Coimbatore, India.","DOI":"10.1109\/ICACA.2016.7887916"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1111\/j.1467-9868.2005.00532.x","article-title":"Model selection and estimation in regression with grouped variables","volume":"68","author":"Yuan","year":"2006","journal-title":"J. R. Stat. Soc. Ser. B Stat. Methodol."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Girshick, R. (2015). Fast R-CNN. arXiv.","DOI":"10.1109\/ICCV.2015.169"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sainath, T.N., Kingsbury, B., Sindhwani, V., Arisoy, E., and Ramabhadran, B. (2013, January 26\u201331). Low-rank matrix factorization for Deep Neural Network training with high-dimensional output targets. Proceedings of the 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, Vancouver, BC, Canada.","DOI":"10.1109\/ICASSP.2013.6638949"},{"key":"ref_9","unstructured":"Xue, J., Li, J., and Gong, Y. (2016, January 8\u201312). Restructuring of deep neural network acoustic models with singular value decomposition. Proceedings of the Annual Conference of the International Speech Communication Association (INTERSPEECH 2016), San Francisco, CA, USA."},{"key":"ref_10","unstructured":"Kholiavchenko, M. (2018). Iterative Low-Rank Approximation for CNN Compression. arXiv."},{"key":"ref_11","unstructured":"(2017, January 21\u201326). On Compressing Deep Models by Low Rank and Sparse Decomposition. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.neucom.2020.02.035","article-title":"Sparse Low Rank Factorization for Deep Neural Network Compression","volume":"398","author":"Swaminathan","year":"2020","journal-title":"Neurocomputing"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yu, R., Li, A., Chen, C.F., Lai, J.H., Morariu, V.I., Han, X., Gao, M., Lin, C.Y., and Davis, L.S. (2017). NISP: Pruning Networks using Neuron Importance Score Propagation. arXiv.","DOI":"10.1109\/CVPR.2018.00958"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"107899","DOI":"10.1016\/j.patcog.2021.107899","article-title":"Pruning by explaining: A novel criterion for deep neural network pruning","volume":"115","author":"Yeom","year":"2021","journal-title":"Pattern Recognit."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"148","DOI":"10.1016\/j.neunet.2019.04.021","article-title":"Redundant feature pruning for accelerated inference in deep neural networks","volume":"118","author":"Ayinde","year":"2019","journal-title":"Neural Netw."},{"key":"ref_16","unstructured":"Han, S., Pool, J., Tran, J., and Dally, W.J. (2015). Learning both Weights and Connections for Efficient Neural Networks. arXiv."},{"key":"ref_17","unstructured":"Simonyan, K., and Zisserman, A. (2014). Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2278","DOI":"10.1109\/5.726791","article-title":"Gradient-Based Learning Applied to Document Recognition","volume":"86","author":"Lecun","year":"1998","journal-title":"Proc. IEEE"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2718\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T18:45:43Z","timestamp":1760121943000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/5\/2718"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,2]]},"references-count":18,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2023,3]]}},"alternative-id":["s23052718"],"URL":"https:\/\/doi.org\/10.3390\/s23052718","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,3,2]]}}}