{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T15:55:45Z","timestamp":1753890945208,"version":"3.41.2"},"reference-count":34,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T00:00:00Z","timestamp":1691107200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Big Data"],"abstract":"<jats:p>Convolutional neural networks have achieved remarkable success in computer vision research. However, to further improve their performance, network models have become increasingly complex and require more memory and computational resources. As a result, model compression has become an essential area of research in recent years. In this study, we focus on the best-case scenario for Huffman coding, which involves data with lower entropy. Building on this concept, we formulate a compression with a filter-wise difference minimization problem and propose a novel algorithm to solve it. Our approach involves filter-level pruning, followed by minimizing the difference between filters. Additionally, we perform filter permutation to further enhance compression. Our proposed algorithm achieves a compression rate of 94\u00d7 on Lenet-5 and 50\u00d7 on VGG16. The results demonstrate the effectiveness of our method in significantly reducing the size of deep neural networks while maintaining a high level of accuracy. We believe that our approach holds great promise in advancing the field of model compression and can benefit various applications that require efficient neural network models. Overall, this study provides important insights and contributions toward addressing the challenges of model compression in deep neural networks.<\/jats:p>","DOI":"10.3389\/fdata.2023.1200382","type":"journal-article","created":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T12:17:03Z","timestamp":1691151423000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Kernel-wise difference minimization for convolutional neural network compression in metaverse"],"prefix":"10.3389","volume":"6","author":[{"given":"Yi-Ting","family":"Chang","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1965","published-online":{"date-parts":[[2023,8,4]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"1552","DOI":"10.1109\/TCSS.2022.3152179","article-title":"Learning to extract expert teams in social networks","volume":"9","author":"Chang","year":"2022","journal-title":"IEEE Trans. Comput. Soc. Syst."},{"key":"B2","first-page":"2285","article-title":"\u201cCompressing neural networks with the hashing trick,\u201d","volume-title":"International Conference on Machine Learning","author":"Chen","year":"2015"},{"key":"B3","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1602.02830","article-title":"Binarized neural networks: training deep neural networks with weights and activations constrained to +1 or -1","author":"Courbariaux","year":"2016","journal-title":"arXiv preprint arXiv:1602.02830"},{"key":"B4","first-page":"1737","article-title":"\u201cDeep learning with limited numerical precision,\u201d","volume-title":"International Conference on Machine Learning","author":"Gupta","year":"2015"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1510.00149","article-title":"Deep compression: compressing deep neural networks with pruning, trained quantization and huffman coding","author":"Han","year":"2015","journal-title":"arXiv preprint arXiv:1510.00149"},{"key":"B6","first-page":"770","article-title":"\u201cDeep residual learning for image recognition,\u201d","author":"He","year":"2016","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B7","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1503.02531","article-title":"Distilling the knowledge in a neural network","author":"Hinton","year":"2015","journal-title":"arXiv preprint arXiv:1503.02531"},{"key":"B8","doi-asserted-by":"publisher","first-page":"713","DOI":"10.1109\/TCSS.2018.2859580","article-title":"On automatic formation of effective therapy groups in social networks","volume":"5","author":"Hsu","year":"2018","journal-title":"IEEE Trans. Comput. Soc. Syst."},{"key":"B9","first-page":"1198","article-title":"\u201cOn minimizing diagonal block-wise differences for neural network compression,\u201d","volume-title":"ECAI 2020","author":"Hsu","year":"2020"},{"key":"B10","doi-asserted-by":"crossref","first-page":"709","DOI":"10.1109\/WACV.2018.00083","article-title":"\u201cLearning to prune filters in convolutional neural networks,\u201d","volume-title":"2018 IEEE Winter Conference on Applications of Computer Vision (WACV)","author":"Huang","year":"2018"},{"key":"B11","first-page":"1","article-title":"\u201cSynchronous weight quantization-compression for low-bit quantized neural network,\u201d","author":"Jiao","year":"2021","journal-title":"2021 International Joint Conference on Neural Networks (IJCNN)"},{"key":"B12","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1511.06530","article-title":"Compression of deep convolutional neural networks for fast and low power mobile applications","author":"Kim","year":"2015","journal-title":"arXiv preprint arXiv:1511.06530"},{"key":"B13","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"B14","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1412.6553","article-title":"Speeding-up convolutional neural networks using fine-tuned cp-decomposition","author":"Lebedev","year":"2014","journal-title":"arXiv preprint arXiv:1412.6553"},{"key":"B15","first-page":"53","article-title":"\u201cComparison of learning algorithms for handwritten digit recognition,\u201d","volume-title":"International Conference on Artificial Neural Networks","author":"LeCun","year":"1995"},{"key":"B16","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1608.08710","article-title":"Pruning filters for efficient convnets","author":"Li","year":"2016","journal-title":"arXiv preprint arXiv:1608.08710"},{"key":"B17","doi-asserted-by":"crossref","first-page":"498","DOI":"10.1287\/opre.21.2.498","article-title":"An effective heuristic algorithm for the traveling-salesman problem","volume":"21","author":"Lin","year":"1973","journal-title":"Operat. Res."},{"key":"B18","first-page":"5058","article-title":"\u201cThinET: a filter level pruning method for deep neural network compression,\u201d","author":"Luo","year":"2017","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"B19","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1412.6550","article-title":"FitNets: hints for thin deep nets","author":"Romero","year":"2014","journal-title":"arXiv preprint arXiv:1412.6550"},{"key":"B20","doi-asserted-by":"crossref","DOI":"10.1609\/aaai.v32i1.11310","article-title":"\u201cOn organizing online soirees with live multi-streaming,\u201d","author":"Shen","year":"2018","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B21","doi-asserted-by":"crossref","first-page":"415","DOI":"10.1145\/3097983.3097995","article-title":"\u201cOn finding socially tenuous groups for online social networks,\u201d","author":"Shen","year":"2017","journal-title":"Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1478","DOI":"10.1109\/TMC.2011.173","article-title":"Collaborative and distributed search system with mobile devices","volume":"11","author":"Shen","year":"2011","journal-title":"IEEE Trans. Mobile Comput."},{"key":"B23","doi-asserted-by":"publisher","first-page":"196","DOI":"10.1109\/TKDE.2015.2468726","article-title":"Socio-spatial group queries for impromptu activity planning","volume":"28","author":"Shen","year":"2015","journal-title":"IEEE Trans. Knowledge Data Eng."},{"key":"B24","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1409.1556","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2014","journal-title":"arXiv preprint arXiv:14c09.1556"},{"key":"B25","first-page":"1","article-title":"\u201cGoing deeper with convolutions,\u201d","author":"Szegedy","year":"2015","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B26","doi-asserted-by":"crossref","first-page":"916","DOI":"10.1109\/ICIP46576.2022.9897466","article-title":"\u201cOn quantization of image classification neural networks for compression without retraining,\u201d","author":"Tonin","year":"2022","journal-title":"2022 IEEE International Conference on Image Processing (ICIP)"},{"key":"B27","first-page":"7873","article-title":"\u201cClip-Q: deep network compression learning by in-parallel pruning-quantization,\u201d","author":"Tung","year":"2018","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"},{"key":"B28","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1145\/3020078.3021744","article-title":"\u201cFINN: a framework for fast, scalable binarized neural network inference,\u201d","author":"Umuroglu","year":"2017","journal-title":"Proceedings of the 2017 ACM\/SIGDA International Symposium on Field-Programmable Gate Arrays"},{"key":"B29","article-title":"\u201cLearning structured sparsity in deep neural networks,\u201d","volume-title":"Advances in Neural Information Processing Systems, Vol. 29","author":"Wen","year":"2016"},{"key":"B30","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1109\/LSP.2023.3264558","article-title":"Hybrid post-training quantization for super-resolution neural network compression","volume":"30","author":"Xu","year":"2023","journal-title":"IEEE Signal Process. Lett."},{"key":"B31","doi-asserted-by":"crossref","first-page":"2191","DOI":"10.1145\/3534678.3539437","article-title":"\u201cEnhancing machine learning approaches for graph optimization problems with diversifying graph augmentation,\u201d","author":"Yang","year":"2022","journal-title":"Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining"},{"key":"B32","doi-asserted-by":"publisher","first-page":"5429","DOI":"10.1109\/TKDE.2021.3057361","article-title":"Learning to solve task-optimized group search for social internet of things","volume":"34","author":"Yang","year":"2021","journal-title":"IEEE Trans. Knowledge Data Eng."},{"key":"B33","doi-asserted-by":"crossref","first-page":"949","DOI":"10.1145\/2339530.2339679","article-title":"\u201cOn socio-spatial group query for location-based social networks,\u201d","author":"Yang","year":"2012","journal-title":"Proceedings of the 18th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining"},{"key":"B34","first-page":"4133","article-title":"\u201cA gift from knowledge distillation: fast optimization, network minimization and transfer learning,\u201d","author":"Yim","year":"2017","journal-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition"}],"container-title":["Frontiers in Big Data"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2023.1200382\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,8,4]],"date-time":"2023-08-04T12:17:33Z","timestamp":1691151453000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdata.2023.1200382\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,8,4]]},"references-count":34,"alternative-id":["10.3389\/fdata.2023.1200382"],"URL":"https:\/\/doi.org\/10.3389\/fdata.2023.1200382","relation":{},"ISSN":["2624-909X"],"issn-type":[{"type":"electronic","value":"2624-909X"}],"subject":[],"published":{"date-parts":[[2023,8,4]]},"article-number":"1200382"}}