{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,1,4]],"date-time":"2025-01-04T05:38:00Z","timestamp":1735969080038,"version":"3.32.0"},"reference-count":60,"publisher":"Institute of Electronics, Information and Communications Engineers (IEICE)","issue":"1","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IEICE Trans. Inf. &amp; Syst."],"published-print":{"date-parts":[[2025,1,1]]},"DOI":"10.1587\/transinf.2024pap0002","type":"journal-article","created":{"date-parts":[[2024,7,10]],"date-time":"2024-07-10T22:12:20Z","timestamp":1720649540000},"page":"23-36","source":"Crossref","is-referenced-by-count":0,"title":["Towards Superior Pruning Performance in Federated Learning with Discriminative Data"],"prefix":"10.1587","volume":"E108.D","author":[{"given":"Yinan","family":"YANG","sequence":"first","affiliation":[{"name":"Graduate School of Information Science and Engineering, Ritsumeikan University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"532","reference":[{"key":"1","doi-asserted-by":"publisher","unstructured":"[1] S. Wang, C. Li, R. Wang, Z. Liu, M. Wang, H. Tan, Y. Wu, X. Liu, H. Sun, R. Yang, X. Liu, J. Chen, H. Zhou, I. Ben Ayed, and H. Zheng, \u201cAnnotation-efficient deep learning for automatic medical image segmentation,\u201d Nature communications, vol.12, no.1, p.5915, 2021. 10.1038\/s41467-021-26216-9","DOI":"10.1038\/s41467-021-26216-9"},{"key":"2","doi-asserted-by":"crossref","unstructured":"[2] J. Wang, S. Guo, X. Xie, and H. Qi, \u201cProtect privacy from gradient leakage attack in federated learning,\u201d IEEE INFOCOM 2022 - IEEE Conf. Computer Communications, vol.00, p.580-589, 2022. 10.1109\/infocom48880.2022.9796841","DOI":"10.1109\/INFOCOM48880.2022.9796841"},{"key":"3","doi-asserted-by":"publisher","unstructured":"[3] D.C. Nguyen, Q.-V. Pham, P.N. Pathirana, M. Ding, A. Seneviratne, Z. Lin, O. Dobre, and W.-J. Hwang, \u201cFederated learning for smart healthcare: A survey,\u201d ACM Computing Surveys (CSUR), vol.55, no.3, pp.1-37, 2022. 10.1145\/3501296","DOI":"10.1145\/3501296"},{"key":"4","unstructured":"[4] R.C. Geyer, T. Klein, and M. Nabi, \u201cDifferentially private federated learning: A client level perspective,\u201d 2018."},{"key":"5","unstructured":"[5] B. McMahan, E. Moore, D. Ramage, S. Hampson, and B.A. y Arcas, \u201cCommunication-efficient learning of deep networks from decentralized data,\u201d Artificial intelligence and statistics, pp.1273-1282, PMLR, 2017."},{"key":"6","doi-asserted-by":"publisher","unstructured":"[6] V. Mothukuri, R.M. Parizi, S. Pouriyeh, Y. Huang, A. Dehghantanha, and G. Srivastava, \u201cA survey on security and privacy of federated learning,\u201d Future Generation Computer Systems, vol.115, pp.619-640, 2021. 10.1016\/j.future.2020.10.007","DOI":"10.1016\/j.future.2020.10.007"},{"key":"7","doi-asserted-by":"publisher","unstructured":"[7] N. Rieke, J. Hancox, W. Li, F. Milletar\u00ec, H.R. Roth, S. Albarqouni, S. Bakas, M.N. Galtier, B.A. Landman, K. Maier-Hein, S. Ourselin, M. Sheller, R.M. Summers, A. Trask, D. Xu, M. Baust, and M.J. Cardoso, \u201cThe future of digital health with federated learning,\u201d NPJ digital medicine, vol.3, no.1, p.119, 2020. 10.1038\/s41746-020-00323-1","DOI":"10.1038\/s41746-020-00323-1"},{"key":"8","unstructured":"[8] N. Onoszko, G. Karlsson, O. Mogren, and E.L. Zec, \u201cDecentralized federated learning of deep neural networks on non-iid data,\u201d 2021."},{"key":"9","unstructured":"[9] N. Shibahara, M. Koibuchi, and H. Matsutani, \u201cPerformance improvement of federated learning server using smart nic,\u201d Proc. 11th International Symposium on Computing and Networking (CANDAR\u2019 23) Workshops, The 6th Sustainable Computing Systems Workshop (SUSCW\u2019 23), p.xxx-xxx, 2023."},{"key":"10","unstructured":"[10] S. Han, J. Pool, J. Tran, and W. Dally, \u201cLearning both weights and connections for efficient neural network,\u201d Advances in neural information processing systems, vol.28, 2015."},{"key":"11","doi-asserted-by":"crossref","unstructured":"[11] H. Wang, C. Qin, Y. Bai, Y. Zhang, and Y. Fu, \u201cRecent advances on neural network pruning at initialization,\u201d Proc. Thirty-First Int. Joint Conf. Artificial Intelligence, IJCAI-22, ed. L.D. Raedt, pp.5638-5645, Int. Joint Conf. Artificial Intelligence Organization, 7 2022. Survey Track. 10.24963\/ijcai.2022\/786","DOI":"10.24963\/ijcai.2022\/786"},{"key":"12","doi-asserted-by":"publisher","unstructured":"[12] W. Xu, W. Fang, Y. Ding, M. Zou, and N. Xiong, \u201cAccelerating federated learning for iot in big data analytics with pruning, quantization and selective updating,\u201d IEEE Access, vol.9, pp.38457-38466, 2021. 10.1109\/access.2021.3063291","DOI":"10.1109\/ACCESS.2021.3063291"},{"key":"13","unstructured":"[13] Y. Jiang, S. Wang, V. Valls, B.J. Ko, W.H. Lee, K.K. Leung, L. Tassiulas, and Y. Jiang, \u201cModel Pruning Enables Efficient Federated Learning on Edge Devices,\u201d IEEE Trans. Neural Networks and Learning Systems, vol.PP, no.99, pp.1-13, 2022."},{"key":"14","unstructured":"[14] M.T. Munir, M.M. Saeed, M. Ali, Z.A. Qazi, and I.A. Qazi, \u201cFedprune: Towards inclusive federated learning,\u201d 2021."},{"key":"15","unstructured":"[15] A. Li, J. Sun, B. Wang, L. Duan, S. Li, Y. Chen, and H. Li, \u201cLotteryfl: Empower edge intelligence with personalized and communication-efficient federated learning,\u201d 2021 IEEE\/ACM Symposium on Edge Computing (SEC), pp.68-79, IEEE, 2021."},{"key":"16","doi-asserted-by":"crossref","unstructured":"[16] H. Huang, L. Zhang, C. Sun, R. Fang, X. Yuan, and D. Wu, \u201cDistributed Pruning Towards Tiny Neural Networks in Federated Learning,\u201d 2023 IEEE 43rd Int. Conf. Distributed Computing Systems (ICDCS), vol.00, pp.190-201, 2023. 10.1109\/icdcs57875.2023.00036","DOI":"10.1109\/ICDCS57875.2023.00036"},{"key":"17","unstructured":"[17] Y. Yang, Y. Wang, Y. Ji, H. Qi, and J. Kato, \u201cOne-shot network pruning at initialization with discriminative image patches,\u201d 33rd British Machine Vision Conference 2022, BMVC 2022, London, UK, November 21-24, 2022, BMVA Press, 2022."},{"key":"18","unstructured":"[18] J. Frankle, G.K. Dziugaite, D. Roy, and M. Carbin, \u201cPruning neural networks at initialization: Why are we missing the mark?,\u201d Int. Conf. Learning Representations, 2021."},{"key":"19","doi-asserted-by":"publisher","unstructured":"[19] A. Imteaj, U. Thakker, S. Wang, J. Li, and M.H. Amini, \u201cA survey on federated learning for resource-constrained iot devices,\u201d IEEE Internet of Things Journal, vol.9, no.1, pp.1-24, 2022. 10.1109\/jiot.2021.3095077","DOI":"10.1109\/JIOT.2021.3095077"},{"key":"20","unstructured":"[20] A. Ghorbani, J. Wexler, J.Y. Zou, and B. Kim, \u201cTowards automatic concept-based explanations,\u201d Advances in neural information processing systems, vol.32, 2019."},{"key":"21","unstructured":"[21] N. Lee, T. Ajanthan, and P. Torr, \u201cSNIP: Single-shot Network Pruning based on Connection Sensitivity,\u201d Int. Conf. Learning Representations, 2019."},{"key":"22","unstructured":"[22] M.C. Mozer and P. Smolensky, \u201cSkeletonization: A technique for trimming the fat from a network via relevance assessment,\u201d Advances in neural information processing systems, pp.107-115, 1989."},{"key":"23","unstructured":"[23] P. de Jorge, A. Sanyal, H. Behl, P. Torr, G. Rogez, and P.K. Dokania, \u201cProgressive skeletonization: Trimming more fat from a network at initialization,\u201d Int. Conf. Learning Representations, 2021."},{"key":"24","unstructured":"[24] A. Krizhevsky, G. Hinton, <i>et al.<\/i>, \u201cLearning multiple layers of features from tiny images,\u201d Master\u2019s thesis, University of Toronto, Toronto, Canada, 2009."},{"key":"25","unstructured":"[25] S. Caldas, S.M.K. Duddu, P. Wu, T. Li, J. Kone\u010dn\u00fd, H.B. McMahan, V. Smith, and A. Talwalkar, \u201cLeaf: A benchmark for federated settings,\u201d 2019."},{"key":"26","doi-asserted-by":"crossref","unstructured":"[26] Z. Liu, P. Luo, X. Wang, and X. Tang, \u201cDeep learning face attributes in the wild,\u201d Proc. IEEE Int. Conf. Computer Vision (ICCV), December 2015.","DOI":"10.1109\/ICCV.2015.425"},{"key":"27","doi-asserted-by":"crossref","unstructured":"[27] J. Deng, W. Dong, R. Socher, L.J. Li, K. Li, and L. Fei-Fei, \u201cImagenet: A large-scale hierarchical image database,\u201d 2009 IEEE Conf. Computer Vision and Pattern Recognition, pp.248-255, 2009. 10.1109\/cvpr.2009.5206848","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"28","unstructured":"[28] A. Krizhevsky, I. Sutskever, and G.E. Hinton, \u201cImagenet classification with deep convolutional neural networks,\u201d Advances in neural information processing systems, vol.25, 2012."},{"key":"29","unstructured":"[29] K. Simonyan and A. Zisserman, \u201cVery deep convolutional networks for large-scale image recognition,\u201d 3rd Int. Conf. Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, ed. Y. Bengio and Y. LeCun, 2015."},{"key":"30","doi-asserted-by":"crossref","unstructured":"[30] K. He, X. Zhang, S. Ren, and J. Sun, \u201cDeep residual learning for image recognition,\u201d 2016 IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp.770-778, 2016. 10.1109\/cvpr.2016.90","DOI":"10.1109\/CVPR.2016.90"},{"key":"31","unstructured":"[31] Y. Zhao, M. Li, L. Lai, N. Suda, D. Civin, and V. Chandra, \u201cFederated learning with non-iid data,\u201d CoRR, vol.abs\/1806.00582, 2018."},{"key":"32","unstructured":"[32] K. Bonawitz, H. Eichner, W. Grieskamp, D. Huba, A. Ingerman, V. Ivanov, C. Kiddon, J. Kone\u010dn\u00fd, S. Mazzocchi, B. McMahan, T. Van Overveldt, D. Petrou, D. Ramage, and J. Roselander, \u201cTowards federated learning at scale: System design,\u201d Proc. Machine Learning and Systems, ed. A. Talwalkar, V. Smith, and M. Zaharia, pp.374-388, 2019."},{"key":"33","doi-asserted-by":"crossref","unstructured":"[33] I.S. Dhillon and D.S. Modha, \u201cA data-clustering algorithm on distributed memory multiprocessors,\u201d in Large-scale parallel data mining, pp.245-260, Springer, 2002. 10.1007\/3-540-46502-2_13","DOI":"10.1007\/3-540-46502-2_13"},{"key":"34","doi-asserted-by":"crossref","unstructured":"[34] R. Reed, \u201cPruning algorithms-a survey,\u201d IEEE Trans. Neural Networks, vol.4, no.5, p.740-747, 1993. 10.1109\/72.248452","DOI":"10.1109\/72.248452"},{"key":"35","unstructured":"[35] Y. LeCun, J. Denker, and S. Solla, \u201cOptimal brain damage,\u201d Advances in Neural Information Processing Systems, ed. D. Touretzky, Morgan-Kaufmann, 1989."},{"key":"36","unstructured":"[36] J. Frankle and M. Carbin, \u201cThe lottery ticket hypothesis: Finding sparse, trainable neural networks,\u201d Int. Conf. Learning Representations, 2019."},{"key":"37","doi-asserted-by":"crossref","unstructured":"[37] T. Chen, J. Frankle, S. Chang, S. Liu, Y. Zhang, M. Carbin, and Z. Wang, \u201cThe lottery tickets hypothesis for supervised and self-supervised pre-training in computer vision models,\u201d Proc. IEEE\/CVF Conf. Computer Vision and Pattern Recognition (CVPR), pp.16306-16316, June 2021.","DOI":"10.1109\/CVPR46437.2021.01604"},{"key":"38","unstructured":"[38] C. Wang, G. Zhang, and R. Grosse, \u201cPicking winning tickets before training by preserving gradient flow,\u201d Int. Conf. Learning Representations, 2020."},{"key":"39","unstructured":"[39] H. Tanaka, D. Kunin, D.L. Yamins, and S. Ganguli, \u201cPruning neural networks without any data by iteratively conserving synaptic flow,\u201d Advances in neural information processing systems, vol.33, pp.6377-6389, 2020."},{"key":"40","doi-asserted-by":"publisher","unstructured":"[40] S. Bibikar, H. Vikalo, Z. Wang, and X. Chen, \u201cFederated dynamic sparse training: Computing less, communicating less, yet learning better,\u201d Proc. AAAI Conf. Artificial Intelligence, vol.36, no.6, pp.6080-6088, Jun. 2022. 10.1609\/aaai.v36i6.20555","DOI":"10.1609\/aaai.v36i6.20555"},{"key":"41","unstructured":"[41] D. Stripelis, U. Gupta, G.V. Steeg, and J.L. Ambite, \u201cFederated progressive sparsification (purge-merge-tune)+,\u201d Workshop on Federated Learning: Recent Advances and New Challenges (in Conjunction with NeurIPS 2022), 2022."},{"key":"42","unstructured":"[42] X. Li, K. Huang, W. Yang, S. Wang, and Z. Zhang, \u201cOn the convergence of fedavg on non-iid data,\u201d Int. Conf. Learning Representations, 2020."},{"key":"43","doi-asserted-by":"publisher","unstructured":"[43] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. S\u00fcsstrunk, \u201cSlic superpixels compared to state-of-the-art superpixel methods,\u201d IEEE Trans. Pattern Analysis and Machine Intelligence, vol.34, no.11, pp.2274-2282, 2012. 10.1109\/tpami.2012.120","DOI":"10.1109\/TPAMI.2012.120"},{"key":"44","unstructured":"[44] B. Kim, M. Wattenberg, J. Gilmer, C. Cai, J. Wexler, F. Viegas, and R. sayres, \u201cInterpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV),\u201d Proc. 35th Int. Conf. Machine Learning, ed. J. Dy and A. Krause, Proc. Machine Learning Research, vol.80, pp.2668-2677, PMLR, 10-15 Jul 2018."},{"key":"45","unstructured":"[45] J. Su, Y. Chen, T. Cai, T. Wu, R. Gao, L. Wang, and J.D. Lee, \u201cSanity-checking pruning methods: Random tickets can win the jackpot,\u201d Advances in neural information processing systems, vol.33, pp.20390-20401, 2020."},{"key":"46","unstructured":"[46] S. Hayou, J.F. Ton, A. Doucet, and Y.W. Teh, \u201cRobust pruning at initialization,\u201d Int. Conf. Learning Representations, 2021."},{"key":"47","unstructured":"[47] H. Tanaka, D. Kunin, D.L. Yamins, and S. Ganguli, \u201cPruning neural networks without any data by iteratively conserving synaptic flow,\u201d Advances in neural information processing systems, vol.33, pp.6377-6389, 2020."},{"key":"48","unstructured":"[48] R. Tang, A. Adhikari, and J. Lin, \u201cFlops as a direct optimization objective for learning sparse neural networks,\u201d 2018."},{"key":"49","unstructured":"[49] Y. Zhou, X. Lin, X. Zhang, M. Wang, G. Jiang, H. Lu, Y. Wu, K. Zhang, Z. Yang, K. Wang, Y. Sui, F. Jia, Z. Tang, Y. Zhao, H. Zhang, T. Yang, W. Chen, Y. Mao, Y. Li, D. Bao, Y. Li, H. Liao, T. Liu, J. Liu, J. Guo, X. Zhao, Y. WEI, H. Qian, Q. Liu, X. Wang, W. Kin, Chan, C. Li, Y. Li, S. Yang, J. Yan, C. Mou, S. Han, W. Jin, G. Zhang, and X. Zeng, \u201cOn the opportunities of green computing: A survey,\u201d 2023."},{"key":"50","unstructured":"[50] L. Zhu, \u201cThop: Pytorch-opcounter.\u201d https:\/\/github.com\/Lyken17\/pytorch-OpCounter."},{"key":"51","doi-asserted-by":"publisher","unstructured":"[51] X. Wang, G. Li, A. Plaza, and Y. He, \u201cRevisiting slic: Fast superpixel segmentation of marine sar images using density features,\u201d IEEE Trans. Geoscience and Remote Sensing, vol.60, p.1-18, 2022. 10.1109\/tgrs.2022.3142068","DOI":"10.1109\/TGRS.2022.3142068"},{"key":"52","unstructured":"[52] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, \u201cScikit-learn: Machine learning in Python,\u201d Journal of Machine Learning Research, vol.12, pp.2825-2830, 2011."},{"key":"53","doi-asserted-by":"crossref","unstructured":"[53] J. Sevilla, L. Heim, A. Ho, T. Besiroglu, M. Hobbhahn, and P. Villalobos, \u201cCompute trends across three eras of machine learning,\u201d 2022 International Joint Conf. Neural Networks (IJCNN), pp.1-8, 2022. 10.1109\/ijcnn55064.2022.9891914","DOI":"10.1109\/IJCNN55064.2022.9891914"},{"key":"54","doi-asserted-by":"publisher","unstructured":"[54] H. Dong, B. Yu, W. Wu, and C. He, \u201cEnhanced lightweight end-to-end semantic segmentation for high-resolution remote sensing images,\u201d IEEE Access, vol.10, pp.70947-70954, 2022. 10.1109\/access.2022.3182370","DOI":"10.1109\/ACCESS.2022.3182370"},{"key":"55","doi-asserted-by":"crossref","unstructured":"[55] T. Kanungo, D.M. Mount, N.S. Netanyahu, C.D. Piatko, R. Silverman, and A.Y. Wu, \u201cAn efficient k-means clustering algorithm: analysis and implementation,\u201d IEEE Trans. Pattern Analysis and Machine Intelligence, vol.24, no.7, pp.881-892, 2002. 10.1109\/tpami.2002.1017616","DOI":"10.1109\/TPAMI.2002.1017616"},{"key":"56","doi-asserted-by":"crossref","unstructured":"[56] A. Sayar and F.T.Y. Vural, \u201cImage annotation with semi-supervised clustering,\u201d 2009 24th International Symposium on Computer and Information Sciences, pp.12-17, IEEE, 2009. 10.1109\/iscis.2009.5291929","DOI":"10.1109\/ISCIS.2009.5291929"},{"key":"57","doi-asserted-by":"publisher","unstructured":"[57] J. Tian, L. Zhu, S. Zhang, and L. Liu, \u201cImprovement and parallelism of k-means clustering algorithm,\u201d Tsinghua Science and Technology, vol.10, no.3, pp.277-281, 2005. 10.1016\/s1007-0214(05)70069-9","DOI":"10.1016\/S1007-0214(05)70069-9"},{"key":"58","doi-asserted-by":"publisher","unstructured":"[58] H. Sabit, A. Al-Anbuky, and H. Gholamhosseini, \u201cData stream mining for wireless sensor networks environment: energy efficient fuzzy clustering algorithm,\u201d International Journal of Autonomous and Adaptive Communications Systems, vol.4, no.4, pp.383-397, 2011. 10.1504\/ijaacs.2011.043478","DOI":"10.1504\/IJAACS.2011.043478"},{"key":"59","unstructured":"[59] H. Sabit, Distributed incremental data stream mining for wireless sensor network, Ph.D. thesis, Auckland University of Technology, 2012."},{"key":"60","doi-asserted-by":"publisher","unstructured":"[60] T. Kucukyilmaz, U. of Turkish Aeronautical Association, <i>et al.<\/i>, \u201cParallel k-means algorithm for shared memory multiprocessors,\u201d Journal of computer and communications, vol.2, no.11, p.15, 2014. 10.4236\/jcc.2014.211002","DOI":"10.4236\/jcc.2014.211002"}],"container-title":["IEICE Transactions on Information and Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.jstage.jst.go.jp\/article\/transinf\/E108.D\/1\/E108.D_2024PAP0002\/_pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,4]],"date-time":"2025-01-04T03:19:45Z","timestamp":1735960785000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.jstage.jst.go.jp\/article\/transinf\/E108.D\/1\/E108.D_2024PAP0002\/_article"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,1]]},"references-count":60,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2025]]}},"URL":"https:\/\/doi.org\/10.1587\/transinf.2024pap0002","relation":{},"ISSN":["0916-8532","1745-1361"],"issn-type":[{"type":"print","value":"0916-8532"},{"type":"electronic","value":"1745-1361"}],"subject":[],"published":{"date-parts":[[2025,1,1]]},"article-number":"2024PAP0002"}}