{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,2]],"date-time":"2025-07-02T04:14:35Z","timestamp":1751429675643,"version":"3.41.0"},"reference-count":59,"publisher":"Association for Computing Machinery (ACM)","issue":"6","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62176027"],"award-info":[{"award-number":["62176027"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Chongqing Talent","award":["cstc2024ycjh-bgzxm0082"],"award-info":[{"award-number":["cstc2024ycjh-bgzxm0082"]}]},{"name":"Central University Operating Expenses","award":["2024CDJGF-044"],"award-info":[{"award-number":["2024CDJGF-044"]}]},{"name":"Chongqing new YC","award":["CSTB2024YCJH-KYXM0126"],"award-info":[{"award-number":["CSTB2024YCJH-KYXM0126"]}]},{"name":"Key Project of Science and Technology Research Program of Chongqing Education Commission of China","award":["KJZD-K202402501"],"award-info":[{"award-number":["KJZD-K202402501"]}]},{"DOI":"10.13039\/501100001809","name":"National Nature Science Foundation of China","doi-asserted-by":"crossref","award":["62262006"],"award-info":[{"award-number":["62262006"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>\n            Fully connected (FC) layers play a significant role in deep neural networks (DNNs) models. Owing to the complexity of its parameters, an FC layer has sufficient capacity to manage high-dimensional tasks, so a large amount of memory and powerful computing capabilities become essential requirements. However, the large number of parameters in an FC layer greatly limits the practical application of this model. To address this problem, we apply matrix optimization to an FC layer. First, an added penalty term properly maintains the sparsity of the imposed weights. Second, a rank constraint is applied to the two components of the factorized weight matrix. Our compression algorithm can effectively reduce the number of required network parameters, which not only reduces the computational complexity of the network but also results in better generalizability on a test dataset. Finally, the effectiveness of the proposed method is verified in two different computer vision task domains. Experiments show that our sparse reduced-rank method achieves a better compression ratio with a lower accuracy loss relative to the competing approaches. The code is available at\n            <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"uri\" xlink:href=\"https:\/\/github.com\/cheer79\/Compress_FC\">https:\/\/github.com\/cheer79\/Compress_FC<\/jats:ext-link>\n            .\n          <\/jats:p>","DOI":"10.1145\/3727982","type":"journal-article","created":{"date-parts":[[2025,4,8]],"date-time":"2025-04-08T18:24:30Z","timestamp":1744136670000},"page":"1-23","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Sparse Reduced-Rank Fully Connected Layers with Its Applications in Detection and Classification"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-4606-1582","authenticated-orcid":false,"given":"Linlin","family":"Fan","sequence":"first","affiliation":[{"name":"School of Computer Science, Chongqing University, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1874-3641","authenticated-orcid":false,"given":"Mingliang","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Computer Science, Chongqing University, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3761-1759","authenticated-orcid":false,"given":"Xuekai","family":"Wei","sequence":"additional","affiliation":[{"name":"School of Computer Science, Chongqing University, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8820-8388","authenticated-orcid":false,"given":"Yong","family":"Feng","sequence":"additional","affiliation":[{"name":"School of Computer Science, Chongqing University, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9439-4623","authenticated-orcid":false,"given":"Tao","family":"Xiang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Chongqing University, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1955-6626","authenticated-orcid":false,"given":"Bin","family":"Fang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Chongqing University, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1588-1387","authenticated-orcid":false,"given":"Zhaowei","family":"Shang","sequence":"additional","affiliation":[{"name":"School of Computer Science, Chongqing University, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-1206-1632","authenticated-orcid":false,"given":"Fan","family":"Jia","sequence":"additional","affiliation":[{"name":"Guangdong Opel Mobile Communications Company, Ltd., OPPO, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5261-8580","authenticated-orcid":false,"given":"Xu","family":"Zhuang","sequence":"additional","affiliation":[{"name":"Guangdong Opel Mobile Communications Company, Ltd., OPPO, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9830-3955","authenticated-orcid":false,"given":"Huayan","family":"Pu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Mechanical Transmission, Chongqing University, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4475-2338","authenticated-orcid":false,"given":"Jun","family":"Luo","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Mechanical Transmission, Chongqing University, Chongqing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2025,7]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/3106156"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-022-01726-1"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447431"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3355393"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01481-9"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458825"},{"key":"e_1_3_1_8_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 26","author":"Denil Misha","year":"2013","unstructured":"Misha Denil, Babak Shakibi, Laurent Dinh, Marc\u2019Aurelio Ranzato, and Nando De Freitas. 2013. Predicting parameters in deep learning. In Proceedings of the Advances in Neural Information Processing Systems 26."},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2020.2976475"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2023.110229"},{"key":"e_1_3_1_11_2","first-page":"229","volume-title":"Proceedings of the 27th ACM International Conference on Multimedia","author":"Ge Shiming","year":"2019","unstructured":"Shiming Ge, Shengwei Zhao, Xindi Gao, and Jia Li. 2019. Fewer-shots and lower-resolutions: Towards ultrafast face recognition in the wild. In Proceedings of the 27th ACM International Conference on Multimedia, 229\u2013237."},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSTARS.2020.3041783"},{"key":"e_1_3_1_13_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 27","author":"Denton Emily L.","year":"2014","unstructured":"Emily L. Denton, Wojciech Zaremba, Joan Bruna, Yann LeCun, and Rob Fergus. 2014. Exploiting linear structure within convolutional networks for efficient evaluation. In Proceedings of the Advances in Neural Information Processing Systems 27."},{"key":"e_1_3_1_14_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Hinton Geoffrey E.","year":"2018","unstructured":"Geoffrey E. Hinton, Sara Sabour, and Nicholas Frosst. 2018. Matrix capsules with EM routing. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01237-3_19"},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN52387.2021.9533546"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2947846"},{"key":"e_1_3_1_18_2","first-page":"667","volume-title":"Proceedings of the 2017 IEEE International Conference on Multimedia and Expo (ICME)","author":"Ge Shiming","year":"2017","unstructured":"Shiming Ge, Zhao Luo, Shengwei Zhao, Xin Jin, and Xiao-Yu Zhang. 2017. Compressing deep neural networks for efficient visual inference. In Proceedings of the 2017 IEEE International Conference on Multimedia and Expo (ICME). IEEE, 667\u2013672."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.15"},{"issue":"2","key":"e_1_3_1_20_2","doi-asserted-by":"crossref","first-page":"287","DOI":"10.1109\/TETCI.2021.3058672","article-title":"Learning sparse neural networks using non-convex regularization","volume":"6","author":"Pandit Mohammad Khalid","year":"2021","unstructured":"Mohammad Khalid Pandit, Roohie Naaz, and Mohammad Ahsan Chishti. 2021. Learning sparse neural networks using non-convex regularization. IEEE Transactions on Emerging Topics in Computational Intelligence 6, 2 (2021), 287\u2013299.","journal-title":"IEEE Transactions on Emerging Topics in Computational Intelligence"},{"issue":"4","key":"e_1_3_1_21_2","doi-asserted-by":"crossref","first-page":"2374","DOI":"10.1109\/TCSVT.2023.3310042","article-title":"Low-resolution object recognition with cross-resolution relational contrastive distillation","volume":"34","author":"Zhang Kangkai","year":"2023","unstructured":"Kangkai Zhang, Shiming Ge, Ruixin Shi, and Dan Zeng. 2023. Low-resolution object recognition with cross-resolution relational contrastive distillation. IEEE Transactions on Circuits and Systems for Video Technology 34, 4 (2023), 2374\u20132384.","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"e_1_3_1_22_2","first-page":"4289","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Masana Marc","year":"2017","unstructured":"Marc Masana, Joost van de Weijer, Luis Herranz, Andrew D. Bagdanov, and Jose M. Alvarez. 2017. Domain-adaptive deep network compression. In Proceedings of the IEEE International Conference on Computer Vision, 4289\u20134297."},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447878"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01543-y"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.21437\/Interspeech.2013-552"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01237-3_12"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2020.02.035"},{"key":"e_1_3_1_28_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 28","author":"Ren Shaoqing","year":"2015","unstructured":"Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster R-CNN: Towards real-time object detection with region proposal networks. In Proceedings of the Advances in Neural Information Processing Systems 28."},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/3065386"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"e_1_3_1_31_2","unstructured":"Catherine Wah Steve Branson Peter Welinder Pietro Perona and Serge J. Belongie. 2011. The Caltech-UCSD Birds-200-2011 Dataset. Retrieved from https:\/\/api.semanticscholar.org\/CorpusID:16119123"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICVGIP.2008.47"},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2011.6126386"},{"key":"e_1_3_1_34_2","unstructured":"Song Han Huizi Mao and William J. Dally. 2016. Deep compression: Compressing deep neural networks with pruning trained quantization and Huffman coding. arXiv:1510.00149. Retrieved from https:\/\/arxiv.org\/abs\/1510.00149"},{"key":"e_1_3_1_35_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 5","author":"Hassibi Babak","year":"1992","unstructured":"Babak Hassibi and David Stork. 1992. Second order derivatives for network pruning: Optimal brain surgeon. In Proceedings of the Advances in Neural Information Processing Systems 5."},{"key":"e_1_3_1_36_2","article-title":"Optimal brain damage","author":"LeCun Yann","year":"1989","unstructured":"Yann LeCun, John Denker, and Sara Solla. 1989. Optimal brain damage. In Advances in neural information processing systems 2.","journal-title":"Advances in neural information processing systems 2"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.01152"},{"key":"e_1_3_1_38_2","first-page":"10936","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 33","author":"Tang Yehui","year":"2020","unstructured":"Yehui Tang, Yunhe Wang, Yixing Xu, Dacheng Tao, Chunjing Xu, Chao Xu, and Chang Xu. 2020. SCOP: Scientific control for reliable neural network pruning. In Proceedings of the Advances in Neural Information Processing Systems 33, 10936\u201310947."},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.298"},{"key":"e_1_3_1_40_2","first-page":"322","volume-title":"Proceedings of the European Conference on Computer Vision","author":"Ye Xucheng","year":"2020","unstructured":"Xucheng Ye, Pengcheng Dai, Junyu Luo, Xin Guo, Yingjie Qi, Jianlei Yang, and Yiran Chen. 2020. Accelerating CNN training by pruning activation gradients. In Proceedings of the European Conference on Computer Vision. Springer, 322\u2013338."},{"key":"e_1_3_1_41_2","first-page":"806","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Liu Baoyuan","year":"2015","unstructured":"Baoyuan Liu, Min Wang, Hassan Foroosh, Marshall Tappen, and Marianna Pensky. 2015. Sparse convolutional neural networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 806\u2013814."},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.78"},{"key":"e_1_3_1_43_2","first-page":"9329","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Wang Wenqi","year":"2018","unstructured":"Wenqi Wang, Yifan Sun, Brian Eriksson, Wenlin Wang, and Vaneet Aggarwal. 2018. Wide compression: Tensor ring nets. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 9329\u20139338."},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/1150402.1150464"},{"key":"e_1_3_1_45_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 31","author":"Lan Xu","year":"2018","unstructured":"Xu Lan, Xiatian Zhu, and Shaogang Gong. 2018. Knowledge distillation by on-the-fly native ensemble. In Proceedings of the Advances in Neural Information Processing Systems 31."},{"issue":"4","key":"e_1_3_1_46_2","first-page":"2051","article-title":"Low-resolution face recognition in the wild via selective knowledge distillation","volume":"28","author":"Ge Shiming","year":"2018","unstructured":"Shiming Ge, Shengwei Zhao, Chenyu Li, and Jia Li. 2018. Low-resolution face recognition in the wild via selective knowledge distillation. IEEE Transactions on Image Processing 28, 4 (2018), 2051\u20132062.","journal-title":"IEEE Transactions on Image Processing"},{"key":"e_1_3_1_47_2","volume-title":"Proceedings of the 32nd AAAI Conference on Artificial Intelligence","author":"Chen Yuntao","year":"2018","unstructured":"Yuntao Chen, Naiyan Wang, and Zhaoxiang Zhang. 2018. DarkRank: Accelerating deep metric learning via cross sample similarities transfer. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence."},{"key":"e_1_3_1_48_2","first-page":"11547","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","author":"Xu Runxin","year":"2022","unstructured":"Runxin Xu, Fuli Luo, Chengyu Wang, Baobao Chang, Jun Huang, Songfang Huang, and Fei Huang. 2022. From dense to sparse: Contrastive pruning for better pre-trained language model compression. In Proceedings of the AAAI Conference on Artificial Intelligence, 11547\u201311555."},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.asoc.2022.109331"},{"key":"e_1_3_1_50_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1467-9868.2005.00532.x"},{"key":"e_1_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.2012.734178"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1214\/07-AOAS131"},{"key":"e_1_3_1_53_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-009-0275-4"},{"issue":"5","key":"e_1_3_1_54_2","first-page":"2","article-title":"The pascal visual object classes challenge 2012 (voc2012) development kit","volume":"8","author":"Everingham Mark","year":"2011","unstructured":"Mark Everingham and John Winn. 2011. The pascal visual object classes challenge 2012 (voc2012) development kit. Pattern Analysis, Statistical Modelling and Computational Learning 8, 5 (2011), 2\u20135.","journal-title":"Pattern Analysis, Statistical Modelling and Computational Learning"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_1_56_2","unstructured":"Sourav Banerjee. 2021. Animal Image Dataset. Retrieved from https:\/\/www.kaggle.com\/datasets\/iamsouravbanerjee\/animal-image-dataset-90-different-animals"},{"key":"e_1_3_1_57_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 32","author":"Xu Yilun","year":"2019","unstructured":"Yilun Xu, Peng Cao, Yuqing Kong, and Yizhou Wang. 2019. L_dMI: A novel information-theoretic loss function for training deep nets robust to label noise. In Proceedings of the Advances in Neural Information Processing Systems 32."},{"issue":"3","key":"e_1_3_1_58_2","doi-asserted-by":"crossref","first-page":"149","DOI":"10.1049\/cdt2.12060","article-title":"Compressing fully connected layers of deep neural networks using permuted features","volume":"17","author":"Dara Nagaraju","year":"2023","unstructured":"Nagaraju Dara and Chandrachoodan Nitin. 2023. Compressing fully connected layers of deep neural networks using permuted features. IET Computers & Digital Techniques 17, 3\u20134 (2023), 149\u2013161.","journal-title":"IET Computers & Digital Techniques"},{"key":"e_1_3_1_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.81"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.169"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3727982","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,1]],"date-time":"2025-07-01T14:48:26Z","timestamp":1751381306000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3727982"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,30]]},"references-count":59,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3727982"],"URL":"https:\/\/doi.org\/10.1145\/3727982","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2025,6,30]]},"assertion":[{"value":"2023-12-03","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-03-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-01","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}