{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T21:14:18Z","timestamp":1768338858657,"version":"3.49.0"},"reference-count":32,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T00:00:00Z","timestamp":1677196800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T00:00:00Z","timestamp":1677196800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003052","name":"Ministry of Trade, Industry and Energy","doi-asserted-by":"publisher","award":["20204010600090"],"award-info":[{"award-number":["20204010600090"]}],"id":[{"id":"10.13039\/501100003052","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Int J Comput Intell Syst"],"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Deep learning has shown excellent performance in numerous machine-learning tasks, but one practical obstacle in deep learning is that the amount of computation and required memory is huge. Model compression, especially in deep learning, is very useful because it saves memory and reduces storage size while maintaining model performance. Model compression in a layered network structure aims to reduce the number of edges by pruning weights that are deemed unnecessary during the calculation. However, existing weight pruning methods perform a layer-by-layer reduction, which requires a predefined removal-ratio constraint for each layer. Layer-by-layer removal ratios must be structurally specified depending on the task, causing a sharp increase in the training time due to a large number of tuning parameters. Thus, such a layer-by-layer strategy is hardly feasible for deep layered models. Our proposed method aims to perform weight pruning in a deep layered network, while producing similar performance, by setting a global removal ratio for the entire model without prior knowledge of the structural characteristics. Our experiments with the proposed method show reliable and high-quality performance, obviating layer-by-layer removal ratios. Furthermore, experiments with increasing layers yield a pattern in the pruned weights that could provide an insight into the layers\u2019 structural importance. The experiment with the LeNet-5 model using MNIST data results in a higher compression ratio of 98.8% for the proposed method, outperforming existing pruning algorithms. In the Resnet-56 experiment, the performance change according to removal ratios of 10\u201390% is investigated, and a higher removal ratio is achieved compared to other tested models. We also demonstrate the effectiveness of the proposed method with YOLOv4, a real-life object-detection model requiring substantial computation.<\/jats:p>","DOI":"10.1007\/s44196-023-00202-z","type":"journal-article","created":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T06:03:25Z","timestamp":1677218605000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Compression of Deep-Learning Models Through Global Weight Pruning Using Alternating Direction Method of Multipliers"],"prefix":"10.1007","volume":"16","author":[{"given":"Kichun","family":"Lee","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sunghun","family":"Hwangbo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Dongwook","family":"Yang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3962-3067","authenticated-orcid":false,"given":"Geonseok","family":"Lee","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2023,2,24]]},"reference":[{"issue":"1","key":"202_CR1","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1109\/TNNLS.2013.2251352","volume":"25","author":"D Brzezinski","year":"2013","unstructured":"Brzezinski, D., Stefanowski, J.: Reacting to different types of concept drift: the accuracy updated ensemble algorithm. IEEE Transac. Neural Netw. Learn. Syst. 25(1), 81\u201394 (2013)","journal-title":"IEEE Transac. Neural Netw. Learn. Syst."},{"key":"202_CR2","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1016\/j.neucom.2017.06.052","volume":"275","author":"H Guo","year":"2018","unstructured":"Guo, H., Liu, H., Li, R., Changan, W., Guo, Y., Mingliang, X.: Margin & diversity based ordering ensemble pruning. Neurocomputing 275, 237\u2013246 (2018)","journal-title":"Neurocomputing"},{"key":"202_CR3","doi-asserted-by":"publisher","DOI":"10.1016\/j.cie.2022.108022","volume":"19","author":"S Petchrompo","year":"2022","unstructured":"Petchrompo, S., Coit, D.W., Brintrup, A., Wannakrairot, A., Parlikad, A.K.: A review of Pareto pruning methods for multi-objective optimization. Computers Ind. Eng. 19, 108022 (2022)","journal-title":"Computers Ind. Eng."},{"issue":"3","key":"202_CR4","doi-asserted-by":"publisher","first-page":"e12661","DOI":"10.1111\/exsy.12661","volume":"38","author":"K Goel","year":"2021","unstructured":"Goel, K., Batra, S.: Two-level pruning based ensemble with abstained learners for concept drift in data streams. Expert. Syst. 38(3), e12661 (2021)","journal-title":"Expert. Syst."},{"issue":"4","key":"202_CR5","doi-asserted-by":"publisher","first-page":"485","DOI":"10.1109\/JPROC.2020.2976475","volume":"108","author":"L Deng","year":"2020","unstructured":"Deng, L., Li, G., Han, S., Shi, L., Xie, Y.: Model compression and hardware acceleration for neural networks: a comprehensive survey. Proc. IEEE 108(4), 485\u2013532 (2020)","journal-title":"Proc. IEEE"},{"key":"202_CR6","first-page":"184","volume-title":"Proceedings of the European conference on computer vision (ECCV)","author":"T Zhang","year":"2018","unstructured":"Zhang, T., Ye, S., Zhang, K., Tang, J., Wen, W., Fardad, M., Wang, Y.: A systematic dnn weight pruning framework using alternating direction method of multipliers. In: Proceedings of the European conference on computer vision (ECCV), pp. 184\u2013199. Springer (2018)"},{"key":"202_CR7","doi-asserted-by":"publisher","first-page":"614","DOI":"10.1016\/j.neunet.2021.09.015","volume":"144","author":"G Qingbei","year":"2021","unstructured":"Qingbei, G., Xiao-Jun, W., Josef, K., Zhiquan, F.: Weak sub-network pruning for strong and efficient neural networks. Neural Netw. 144, 614\u2013626 (2021)","journal-title":"Neural Netw."},{"key":"202_CR8","doi-asserted-by":"crossref","unstructured":"Zhuliang, Y., Shijie, C., Wencong, X., Chen, Z., Lanshun, N.: Balanced sparsity for efficient dnn inference on gpu. In: Proceedings of the AAAI conference on artificial intelligence. pp. 5676\u20135683 (2019)","DOI":"10.1609\/aaai.v33i01.33015676"},{"key":"202_CR9","unstructured":"Maohua, Z., Tao, Z., Zhenyu, G., Yuan, X.: Sparse tensor core: Algorithm and hardware co-design for vector-wise sparse neural networks on modern gpus. In: Proceedings of the 52nd Annual IEEE\/ACM International Symposium on Microarchitecture, pp 359\u2013371 (2019)"},{"key":"202_CR10","first-page":"31","volume-title":"Advances in neural information processing systems","author":"Y Ji","year":"2018","unstructured":"Ji, Y., Liang, L., Deng, L., Zhang, Y., Zhang, Y., Xie, Y.: Tetris: tile-matching the tremendous irregular sparsity. In: Advances in neural information processing systems, p. 31. MIT Press (2018)"},{"key":"202_CR11","doi-asserted-by":"publisher","first-page":"574","DOI":"10.1109\/TNNLS.2019.2906563","volume":"31","author":"S Lin","year":"2019","unstructured":"Lin, S., Ji, R., Li, Y., Deng, C., Li, X.: Toward compact convnets via structure-sparsity regularized filter pruning. IEEE Transac. Neural Netwo. Learning Syst. 31, 574\u2013588 (2019)","journal-title":"IEEE Transac. Neural Netwo. Learning Syst."},{"key":"202_CR12","first-page":"2","volume-title":"Advances in neural information processing systems","author":"Y LeCun","year":"1989","unstructured":"LeCun, Y., Denker, J., Solla, S.: Optimal brain damage. In: Advances in neural information processing systems, p. 2. MIT Press (1989)"},{"key":"202_CR13","volume-title":"Second order derivatives for network pruning: optimal brain surgeon","author":"B Hassibi","year":"1993","unstructured":"Hassibi, B., Stork, D.G.: Second order derivatives for network pruning: optimal brain surgeon. Morgan Kaufmann, Rome (1993)"},{"key":"202_CR14","unstructured":"Yiming, H., Siyang, S., Jianquan, L., Xingang, W., Qingyi, G.: A novel channel pruning method for deep neural network compression. arXiv preprint arXiv:1805.11394, (2018)"},{"key":"202_CR15","doi-asserted-by":"publisher","first-page":"58324","DOI":"10.1109\/ACCESS.2018.2874823","volume":"6","author":"L Liang","year":"2018","unstructured":"Liang, L., Deng, L., Zeng, Y., Xing, H., Ji, Y., Ma, X., Li, G., Xie, Y.: Crossbar-aware neural network pruning. IEEE Access 6, 58324\u201358337 (2018)","journal-title":"IEEE Access"},{"key":"202_CR16","unstructured":"Song, H., Huizi, M., William\u00a0J, D.: Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, (2015)"},{"key":"202_CR17","unstructured":"Song, H., Jeff, P., John, T., William\u00a0J, D.: Learning both weights and connections for efficient neural networks. arXiv preprint arXiv:1506.02626, (2015)"},{"issue":"10","key":"202_CR18","doi-asserted-by":"publisher","first-page":"1487","DOI":"10.1109\/TC.2019.2914438","volume":"68","author":"X Dai","year":"2019","unstructured":"Dai, X., Yin, H., Jha, N.K.: Nest: a neural network synthesis tool based on a grow-and-prune paradigm. IEEE Transac. Computers 68(10), 1487\u20131497 (2019)","journal-title":"IEEE Transac. Computers"},{"key":"202_CR19","unstructured":"Hao, L., Asim, K., Igor, D., Hanan, S., Hans\u00a0Peter G.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710, (2016)"},{"key":"202_CR20","first-page":"9194","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"R Yu","year":"2018","unstructured":"Yu, R., Li, A., Chen, C.-F., Lai, J.-H., Morariu, V.I., Han, X., Gao, M., Lin, C.-Y., Davis, L.S.: Nisp: pruning networks using neuron importance score propagation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9194\u20139203. IEEE (2018)"},{"key":"202_CR21","first-page":"1529","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"M Lin","year":"2020","unstructured":"Lin, M., Ji, R., Wang, Y., Zhang, Y., Zhang, B., Tian, Y., Shao, L.: Hrank: filter pruning using high-rank feature map. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 1529\u20131538. IEEE (2020)"},{"key":"202_CR22","first-page":"3977","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"T Li","year":"2019","unstructured":"Li, T., Wu, B., Yang, Y., Fan, Y., Zhang, Y., Liu, W.: Compressing convolutional neural networks via factorized convolutional filters. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 3977\u20133986. IEEE (2019)"},{"key":"202_CR23","unstructured":"Zhuangwei, Z., Mingkui, T., Bohan, Z., Jing, L., Yong, G., Qingyao, W., Junzhou, H., Jinhui Z.: Discrimination-aware channel pruning for deep neural networks. arXiv preprint arXiv:1810.11809, (2018)"},{"key":"202_CR24","volume-title":"Distributed optimization and statistical learning via the alternating direction method of multipliers","author":"S Boyd","year":"2011","unstructured":"Boyd, S., Parikh, N., Chu, E.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Now Publishers Inc (2011)"},{"key":"202_CR25","unstructured":"Tianyun, Z., Shaokai, Y., Kaiqi, Z., Xiaolong, M., Ning, L., Linfeng. Z., Jian, T., Kaisheng, M., Xue L., Makan F.: et\u00a0al. Structadmm: a systematic, high-efficiency framework of structured weight pruning for dnns. arXiv preprint arXiv:1807.11091, (2018)"},{"key":"202_CR26","first-page":"12538","volume-title":"Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition","author":"T-A Chen","year":"2022","unstructured":"Chen, T.-A., Yang, D.-N., Chen, M.-S.: AlignQ: alignment quantization with ADMM-based correlation preservation. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp. 12538\u201312547. IEEE (2022)"},{"key":"202_CR27","doi-asserted-by":"publisher","first-page":"2297","DOI":"10.1109\/TSP.2020.2984895","volume":"68","author":"C Kumar","year":"2020","unstructured":"Kumar, C., Rajawat, K.: Network dissensus via distributed ADMM. IEEE Transac. Signal Process. 68, 2297\u20132301 (2020)","journal-title":"IEEE Transac. Signal Process."},{"key":"202_CR28","first-page":"770","volume-title":"Proceedings of the IEEE conference on computer vision and pattern recognition","author":"H Kaiming","year":"2016","unstructured":"Kaiming, H., Xiangyu, Z., Shaoqing, R., Jian, S.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770\u2013778. IEEE (2016)"},{"key":"202_CR29","unstructured":"Karen, S., Andrew, Z.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, (2014)"},{"key":"202_CR30","unstructured":"Bochkovskiy, A., Wang, C.-Y., Mark Liao, H.-Y.: Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934, (2020)"},{"key":"202_CR31","doi-asserted-by":"publisher","first-page":"3645","DOI":"10.1109\/ICIP.2017.8296962","volume-title":"2017 IEEE international conference on image processing (ICIP)","author":"N Wojke","year":"2017","unstructured":"Wojke, N., Bewley, A., Paulus, D.: Simple online and realtime tracking with a deep association metric. In: 2017 IEEE international conference on image processing (ICIP), pp. 3645\u20133649. IEEE (2017)"},{"key":"202_CR32","first-page":"1440","volume-title":"Proceedings of the IEEE international conference on computer vision","author":"R Girshick","year":"2015","unstructured":"Girshick, R.: r-cnn fast. In: Proceedings of the IEEE international conference on computer vision, pp. 1440\u20131448. IEEE (2015)"}],"container-title":["International Journal of Computational Intelligence Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44196-023-00202-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s44196-023-00202-z\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s44196-023-00202-z.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,2,24]],"date-time":"2023-02-24T06:03:56Z","timestamp":1677218636000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s44196-023-00202-z"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,2,24]]},"references-count":32,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2023,12]]}},"alternative-id":["202"],"URL":"https:\/\/doi.org\/10.1007\/s44196-023-00202-z","relation":{},"ISSN":["1875-6883"],"issn-type":[{"value":"1875-6883","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,2,24]]},"assertion":[{"value":"24 October 2022","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 February 2023","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 February 2023","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"All the authors declare that there is no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of Interest"}},{"value":"All the authors agreed with the content of this paper.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics Approval and Consent to Participate"}},{"value":"All the authors reviewed the final manuscript and approved it for publication.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for Publication"}}],"article-number":"17"}}