{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T16:14:52Z","timestamp":1775578492084,"version":"3.50.1"},"reference-count":92,"publisher":"Association for Computing Machinery (ACM)","issue":"5s","license":[{"start":{"date-parts":[[2023,9,9]],"date-time":"2023-09-09T00:00:00Z","timestamp":1694217600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Union through European Social Fund in the frames of the \u201cInformation and Communication Technologies (ICT) program\u201d"},{"name":"Swedish Innovation Agency VINNOVA project \u201cAutoDeep\u201d, \u201cSafeDeep\u201d, and \u201cKKS DPAC\u201d"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Embed. Comput. Syst."],"published-print":{"date-parts":[[2023,10,31]]},"abstract":"<jats:p>The deployment of Deep Neural Networks (DNNs) on edge devices is hindered by the substantial gap between performance requirements and available computational power. While recent research has made significant strides in developing pruning methods to build a sparse network for reducing the computing overhead of DNNs, there remains considerable accuracy loss, especially at high pruning ratios. We find that the architectures designed for dense networks by differentiable architecture search methods are ineffective when pruning mechanisms are applied to them. The main reason is that the current methods do not support sparse architectures in their search space and use a search objective that is made for dense networks and does not focus on sparsity.<\/jats:p>\n          <jats:p>\n            This paper proposes a new method to search for sparsity-friendly neural architectures. It is done by adding two new sparse operations to the search space and modifying the search objective. We propose two novel parametric\n            <jats:monospace>SparseConv<\/jats:monospace>\n            and\n            <jats:monospace>SparseLinear<\/jats:monospace>\n            operations in order to expand the search space to include sparse operations. In particular, these operations make a flexible search space due to using sparse parametric versions of linear and convolution operations. The proposed search objective lets us train the architecture based on the sparsity of the search space operations. Quantitative analyses demonstrate that architectures found through DASS outperform those used in the state-of-the-art sparse networks on the CIFAR-10 and ImageNet datasets. In terms of performance and hardware effectiveness, DASS increases the accuracy of the sparse version of MobileNet-v2 from 73.44% to 81.35% (+7.91% improvement) with a 3.87\u00d7 faster inference time.\n          <\/jats:p>","DOI":"10.1145\/3609385","type":"journal-article","created":{"date-parts":[[2023,9,9]],"date-time":"2023-09-09T13:33:18Z","timestamp":1694266398000},"page":"1-21","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":13,"title":["DASS: Differentiable Architecture Search for Sparse Neural Networks"],"prefix":"10.1145","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5710-1206","authenticated-orcid":false,"given":"Hamid","family":"Mousavi","sequence":"first","affiliation":[{"name":"School of Innovation, Design and Engineering, M\u00e4lardalen University, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9704-7117","authenticated-orcid":false,"given":"Mohammad","family":"Loni","sequence":"additional","affiliation":[{"name":"School of Innovation, Design and Engineering, M\u00e4lardalen University, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5078-0194","authenticated-orcid":false,"given":"Mina","family":"Alibeigi","sequence":"additional","affiliation":[{"name":"Zenseact AB, Lindholmspiren 2, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6289-1521","authenticated-orcid":false,"given":"Masoud","family":"Daneshtalab","sequence":"additional","affiliation":[{"name":"School of Innovation, Design and Engineering, M\u00e4lardalen University,Sweden and Computer systems, Tallinn University of Technology, Estonia"}]}],"member":"320","published-online":{"date-parts":[[2023,9,9]]},"reference":[{"key":"e_1_3_1_2_2","article-title":"Faster r-cnn: Towards real-time object detection with region proposal networks","volume":"28","author":"Ren Shaoqing","year":"2015","unstructured":"Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. 2015. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in Neural Information Processing Systems 28 (2015).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_4_2","article-title":"Deep learning for computer vision: A brief review","volume":"2018","author":"Voulodimos Athanasios","year":"2018","unstructured":"Athanasios Voulodimos, Nikolaos Doulamis, Anastasios Doulamis, and Eftychios Protopapadakis. 2018. Deep learning for computer vision: A brief review. Computational Intelligence and Neuroscience 2018 (2018).","journal-title":"Computational Intelligence and Neuroscience"},{"key":"e_1_3_1_5_2","article-title":"How convolutional neural network see the world-A survey of convolutional neural network visualization methods","author":"Qin Zhuwei","year":"2018","unstructured":"Zhuwei Qin, Fuxun Yu, Chenchen Liu, and Xiang Chen. 2018. How convolutional neural network see the world-A survey of convolutional neural network visualization methods. arXiv preprint arXiv:1804.11191 (2018).","journal-title":"arXiv preprint arXiv:1804.11191"},{"key":"e_1_3_1_6_2","article-title":"AI and memory wall","author":"Gholami Amir","year":"2021","unstructured":"Amir Gholami, Zhewei Yao, Sehoon Kim, Michael Mahoney, and Kurt Keutzer. 2021. AI and memory wall. RiseLab Medium Post (2021).","journal-title":"RiseLab Medium Post"},{"key":"e_1_3_1_7_2","article-title":"FastStereoNet: A fast neural architecture search for improving the inference of disparity estimation on resource-limited platforms","author":"Loni Mohammad","year":"2021","unstructured":"Mohammad Loni, Ali Zoljodi, Amin Majd, Byung Hoon Ahn, Masoud Daneshtalab, Mikael Sj\u00f6din, and Hadi Esmaeilzadeh. 2021. FastStereoNet: A fast neural architecture search for improving the inference of disparity estimation on resource-limited platforms. IEEE Transactions on Systems, Man, and Cybernetics: Systems (2021).","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics: Systems"},{"key":"e_1_3_1_8_2","article-title":"On-device training under 256kb memory","author":"Lin Ji","year":"2022","unstructured":"Ji Lin, Ligeng Zhu, Wei-Ming Chen, Wei-Chen Wang, Chuang Gan, and Song Han. 2022. On-device training under 256kb memory. arXiv preprint arXiv:2206.15472 (2022).","journal-title":"arXiv preprint arXiv:2206.15472"},{"key":"e_1_3_1_9_2","article-title":"Mcunet: Tiny deep learning on iot devices","author":"Lin Ji","year":"2020","unstructured":"Ji Lin, Wei-Ming Chen, Yujun Lin, John Cohn, Chuang Gan, and Song Han. 2020. Mcunet: Tiny deep learning on iot devices. arXiv preprint arXiv:2007.10319 (2020).","journal-title":"arXiv preprint arXiv:2007.10319"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.micpro.2020.102989"},{"key":"e_1_3_1_11_2","first-page":"19655","article-title":"Hydra: Pruning adversarially robust neural networks","volume":"33","author":"Sehwag Vikash","year":"2020","unstructured":"Vikash Sehwag, Shiqi Wang, Prateek Mittal, and Suman Jana. 2020. Hydra: Pruning adversarially robust neural networks. Advances in Neural Information Processing Systems 33 (2020), 19655\u201319666.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2021.07.045"},{"key":"e_1_3_1_13_2","article-title":"Training deep neural networks with joint quantization and pruning of weights and activations","author":"Zhang Xinyu","year":"2021","unstructured":"Xinyu Zhang, Ian Colbert, Ken Kreutz-Delgado, and Srinjoy Das. 2021. Training deep neural networks with joint quantization and pruning of weights and activations. arXiv preprint arXiv:2110.08271 (2021).","journal-title":"arXiv preprint arXiv:2110.08271"},{"key":"e_1_3_1_14_2","article-title":"Pruning deep neural networks from a sparsity perspective","author":"Diao Enmao","year":"2023","unstructured":"Enmao Diao, Ganghua Wang, Jiawei Zhan, Yuhong Yang, Jie Ding, and Vahid Tarokh. 2023. Pruning deep neural networks from a sparsity perspective. arXiv preprint arXiv:2302.05601 (2023).","journal-title":"arXiv preprint arXiv:2302.05601"},{"key":"e_1_3_1_15_2","volume-title":"Design, Automation & Test in Europe Conference & Exhibition DATE\u201922, 14 March 2022, Antwerp, Belgium","author":"Loni Mohammad","year":"2022","unstructured":"Mohammad Loni, Hamid Mousavi, Mohammad Riazati, Masoud Daneshtalab, and Mikael Sj\u00f6din. 2022. TAS: Ternarized neural architecture search for resource-constrained edge devices. In Design, Automation & Test in Europe Conference & Exhibition DATE\u201922, 14 March 2022, Antwerp, Belgium. IEEE. http:\/\/www.es.mdh.se\/publications\/6351-"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1007\/978-3-030-58592-1_19","volume-title":"Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK, August 23\u201328, 2020, Proceedings, Part XXIII 16","author":"Bulat Adrian","year":"2020","unstructured":"Adrian Bulat, Brais Martinez, and Georgios Tzimiropoulos. 2020. Bats: Binary architecture search. In Computer Vision\u2013ECCV 2020: 16th European Conference, Glasgow, UK, August 23\u201328, 2020, Proceedings, Part XXIII 16. Springer, 309\u2013325."},{"key":"e_1_3_1_17_2","first-page":"575","volume-title":"European Conference on Computer Vision","author":"Kim Dahyun","year":"2020","unstructured":"Dahyun Kim, Kunal Pratap Singh, and Jonghyun Choi. 2020. Learning architectures for binary networks. In European Conference on Computer Vision. Springer, 575\u2013591."},{"key":"e_1_3_1_18_2","unstructured":"G. Hinton O. Vinyals and J. Dean. 2015. Distilling the Knowledge in a Neural Network (cite arxiv:1503.02531Comment: NIPS 2014 Deep Learning Workshop)."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01453-z"},{"key":"e_1_3_1_20_2","article-title":"Speeding up convolutional neural networks with low rank expansions","author":"Jaderberg Max","year":"2014","unstructured":"Max Jaderberg, Andrea Vedaldi, and Andrew Zisserman. 2014. Speeding up convolutional neural networks with low rank expansions. arXiv preprint arXiv:1405.3866 (2014).","journal-title":"arXiv preprint arXiv:1405.3866"},{"key":"e_1_3_1_21_2","article-title":"Learned threshold pruning","author":"Azarian Kambiz","year":"2020","unstructured":"Kambiz Azarian, Yash Bhalgat, Jinwon Lee, and Tijmen Blankevoort. 2020. Learned threshold pruning. arXiv preprint arXiv:2003.00075 (2020).","journal-title":"arXiv preprint arXiv:2003.00075"},{"key":"e_1_3_1_22_2","article-title":"Data-efficient structured pruning via submodular optimization","author":"Halabi Marwa El","year":"2022","unstructured":"Marwa El Halabi, Suraj Srinivas, and Simon Lacoste-Julien. 2022. Data-efficient structured pruning via submodular optimization. arXiv preprint arXiv:2203.04940 (2022).","journal-title":"arXiv preprint arXiv:2203.04940"},{"key":"e_1_3_1_23_2","article-title":"Soft filter pruning for accelerating deep convolutional neural networks","author":"He Yang","year":"2018","unstructured":"Yang He, Guoliang Kang, Xuanyi Dong, Yanwei Fu, and Yi Yang. 2018. Soft filter pruning for accelerating deep convolutional neural networks. arXiv preprint arXiv:1808.06866 (2018).","journal-title":"arXiv preprint arXiv:1808.06866"},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00447"},{"key":"e_1_3_1_25_2","first-page":"2009","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"He Yang","year":"2020","unstructured":"Yang He, Yuhang Ding, Ping Liu, Linchao Zhu, Hanwang Zhang, and Yi Yang. 2020. Learning filter pruning criteria for deep convolutional neural networks acceleration. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 2009\u20132018."},{"key":"e_1_3_1_26_2","volume-title":"NeurIPS","author":"Zhuang Tao","year":"2020","unstructured":"Tao Zhuang, Zhixuan Zhang, Yuheng Huang, Xiaoyi Zeng, Kai Shuang, and Xiang Li. 2020. Neuron-level structured pruning using polarization regularizer. In NeurIPS."},{"key":"e_1_3_1_27_2","article-title":"Advancing model pruning via Bi-level optimization","author":"Zhang Yihua","year":"2022","unstructured":"Yihua Zhang, Yuguang Yao, Parikshit Ram, Pu Zhao, Tianlong Chen, Mingyi Hong, Yanzhi Wang, and Sijia Liu. 2022. Advancing model pruning via Bi-level optimization. arXiv preprint arXiv:2210.04092 (2022).","journal-title":"arXiv preprint arXiv:2210.04092"},{"key":"e_1_3_1_28_2","article-title":"Darts: Differentiable architecture search","author":"Liu Hanxiao","year":"2018","unstructured":"Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018. Darts: Differentiable architecture search. arXiv preprint arXiv:1806.09055 (2018).","journal-title":"arXiv preprint arXiv:1806.09055"},{"key":"e_1_3_1_29_2","first-page":"10874","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Ye Peng","year":"2022","unstructured":"Peng Ye, Baopu Li, Yikang Li, Tao Chen, Jiayuan Fan, and Wanli Ouyang. 2022. b-darts: Beta-decay regularization for differentiable architecture search. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 10874\u201310883."},{"key":"e_1_3_1_30_2","first-page":"10864","volume-title":"2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201922)","author":"Ye Peng","year":"2022","unstructured":"Peng Ye, Baopu Li, Yikang Li, Tao Chen, Jiayuan Fan, and Wanli Ouyang. 2022. beta-DARTS: Beta-decay regularization for differentiable architecture search. In 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR\u201922). IEEE, 10864\u201310873."},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00474"},{"key":"e_1_3_1_32_2","first-page":"6105","volume-title":"International Conference on Machine Learning","author":"Tan Mingxing","year":"2019","unstructured":"Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning. PMLR, 6105\u20136114."},{"key":"e_1_3_1_33_2","first-page":"8697","volume-title":"International Conference on Machine Learning","author":"Shen Zhengyang","year":"2020","unstructured":"Zhengyang Shen, Lingshen He, Zhouchen Lin, and Jinwen Ma. 2020. Pdo-econvs: Partial differential operator based equivariant convolutions. In International Conference on Machine Learning. PMLR, 8697\u20138706."},{"key":"e_1_3_1_34_2","article-title":"Flexconv: Continuous kernel convolutions with differentiable kernel sizes","author":"Romero David W.","year":"2021","unstructured":"David W. Romero, Robert-Jan Bruintjes, Jakub M. Tomczak, Erik J. Bekkers, Mark Hoogendoorn, and Jan C. van Gemert. 2021. Flexconv: Continuous kernel convolutions with differentiable kernel sizes. arXiv preprint arXiv:2110.08059 (2021).","journal-title":"arXiv preprint arXiv:2110.08059"},{"key":"e_1_3_1_35_2","article-title":"Momentum residual neural networks","author":"Sander Michael E.","year":"2021","unstructured":"Michael E. Sander, Pierre Ablin, Mathieu Blondel, and Gabriel Peyr\u00e9. 2021. Momentum residual neural networks. arXiv preprint arXiv:2102.07870 (2021).","journal-title":"arXiv preprint arXiv:2102.07870"},{"key":"e_1_3_1_36_2","article-title":"Drnas: Dirichlet neural architecture search","author":"Chen Xiangning","year":"2020","unstructured":"Xiangning Chen, Ruochen Wang, Minhao Cheng, Xiaocheng Tang, and Cho-Jui Hsieh. 2020. Drnas: Dirichlet neural architecture search. arXiv preprint arXiv:2006.10355 (2020).","journal-title":"arXiv preprint arXiv:2006.10355"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33014780"},{"key":"e_1_3_1_38_2","first-page":"208","volume-title":"International Conference on Artificial Neural Networks","author":"Loni Mohammad","year":"2019","unstructured":"Mohammad Loni, Ali Zoljodi, Sima Sinaei, Masoud Daneshtalab, and Mikael Sj\u00f6din. 2019. Neuropower: Designing energy efficient convolutional neural network architecture for embedded systems. In International Conference on Artificial Neural Networks. Springer, 208\u2013222."},{"key":"e_1_3_1_39_2","article-title":"A survey on evolutionary neural architecture search","author":"Liu Yuqiao","year":"2021","unstructured":"Yuqiao Liu, Yanan Sun, Bing Xue, Mengjie Zhang, Gary G. Yen, and Kay Chen Tan. 2021. A survey on evolutionary neural architecture search. IEEE Transactions on Neural Networks and Learning Systems (2021).","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00907"},{"key":"e_1_3_1_41_2","article-title":"Neural architecture search with reinforcement learning","author":"Zoph Barret","year":"2016","unstructured":"Barret Zoph and Quoc V. Le. 2016. Neural architecture search with reinforcement learning. arXiv preprint arXiv:1611.01578 (2016).","journal-title":"arXiv preprint arXiv:1611.01578"},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.imavis.2019.06.005"},{"key":"e_1_3_1_43_2","article-title":"Smash: One-shot model architecture search through hypernetworks","author":"Brock Andrew","year":"2017","unstructured":"Andrew Brock, Theodore Lim, James M. Ritchie, and Nick Weston. 2017. Smash: One-shot model architecture search through hypernetworks. arXiv preprint arXiv:1708.05344 (2017).","journal-title":"arXiv preprint arXiv:1708.05344"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58517-4_32"},{"key":"e_1_3_1_45_2","first-page":"550","volume-title":"International Conference on Machine Learning","author":"Bender Gabriel","year":"2018","unstructured":"Gabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, and Quoc Le. 2018. Understanding and simplifying one-shot architecture search. In International Conference on Machine Learning. PMLR, 550\u2013559."},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447582"},{"key":"e_1_3_1_47_2","article-title":"sharpdarts: Faster and more accurate differentiable architecture search","author":"Hundt Andrew","year":"2019","unstructured":"Andrew Hundt, Varun Jain, and Gregory D. Hager. 2019. sharpdarts: Faster and more accurate differentiable architecture search. arXiv preprint arXiv:1903.09900 (2019).","journal-title":"arXiv preprint arXiv:1903.09900"},{"key":"e_1_3_1_48_2","doi-asserted-by":"crossref","first-page":"9666","DOI":"10.1109\/ICPR48806.2021.9412285","volume-title":"2020 25th International Conference on Pattern Recognition (ICPR\u201921)","author":"Siddiqui Shahid","year":"2021","unstructured":"Shahid Siddiqui, Christos Kyrkou, and Theocharis Theocharides. 2021. Operation and topology aware fast differentiable architecture search. In 2020 25th International Conference on Pattern Recognition (ICPR\u201921). IEEE, 9666\u20139673."},{"key":"e_1_3_1_49_2","article-title":"Rethinking architecture selection in differentiable NAS","author":"Wang Ruochen","year":"2021","unstructured":"Ruochen Wang, Minhao Cheng, Xiangning Chen, Xiaocheng Tang, and Cho-Jui Hsieh. 2021. Rethinking architecture selection in differentiable NAS. arXiv preprint arXiv:2108.04392 (2021).","journal-title":"arXiv preprint arXiv:2108.04392"},{"key":"e_1_3_1_50_2","article-title":"Understanding and robustifying differentiable architecture search","author":"Zela Arber","year":"2019","unstructured":"Arber Zela, Thomas Elsken, Tonmoy Saikia, Yassine Marrakchi, Thomas Brox, and Frank Hutter. 2019. Understanding and robustifying differentiable architecture search. arXiv preprint arXiv:1909.09656 (2019).","journal-title":"arXiv preprint arXiv:1909.09656"},{"key":"e_1_3_1_51_2","article-title":"Effective, efficient and robust neural architecture search","author":"Yue Zhixiong","year":"2020","unstructured":"Zhixiong Yue, Baijiong Lin, Xiaonan Huang, and Yu Zhang. 2020. Effective, efficient and robust neural architecture search. arXiv preprint arXiv:2011.09820 (2020).","journal-title":"arXiv preprint arXiv:2011.09820"},{"key":"e_1_3_1_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00613"},{"key":"e_1_3_1_53_2","article-title":"Rc-darts: Resource constrained differentiable architecture search","author":"Jin Xiaojie","year":"2019","unstructured":"Xiaojie Jin, Jiang Wang, Joshua Slocum, Ming-Hsuan Yang, Shengyang Dai, Shuicheng Yan, and Jiashi Feng. 2019. Rc-darts: Resource constrained differentiable architecture search. arXiv preprint arXiv:1912.12814 (2019).","journal-title":"arXiv preprint arXiv:1912.12814"},{"key":"e_1_3_1_54_2","article-title":"HELP: Hardware-adaptive efficient latency predictor for NAS via meta-learning","author":"Lee Hayeon","year":"2021","unstructured":"Hayeon Lee, Sewoong Lee, Song Chong, and Sung Ju Hwang. 2021. HELP: Hardware-adaptive efficient latency predictor for NAS via meta-learning. arXiv preprint arXiv:2106.08630 (2021).","journal-title":"arXiv preprint arXiv:2106.08630"},{"key":"e_1_3_1_55_2","article-title":"Once-for-all: Train one network and specialize it for efficient deployment","author":"Cai Han","year":"2019","unstructured":"Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. 2019. Once-for-all: Train one network and specialize it for efficient deployment. arXiv preprint arXiv:1908.09791 (2019).","journal-title":"arXiv preprint arXiv:1908.09791"},{"key":"e_1_3_1_56_2","first-page":"915","volume-title":"2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA\u201919)","author":"Laube Kevin Alexander","year":"2019","unstructured":"Kevin Alexander Laube and Andreas Zell. 2019. Prune and replace nas. In 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA\u201919). IEEE, 915\u2013921."},{"key":"e_1_3_1_57_2","first-page":"493","volume-title":"International Conference on Artificial Intelligence and Statistics","author":"Noy Asaf","year":"2020","unstructured":"Asaf Noy, Niv Nayman, Tal Ridnik, Nadav Zamir, Sivan Doveh, Itamar Friedman, Raja Giryes, and Lihi Zelnik. 2020. Asap: Architecture search, anneal and prune. In International Conference on Artificial Intelligence and Statistics. PMLR, 493\u2013503."},{"key":"e_1_3_1_58_2","first-page":"2326","volume-title":"Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence","author":"Hong Weijun","year":"2021","unstructured":"Weijun Hong, Guilin Li, Weinan Zhang, Ruiming Tang, Yunhe Wang, Zhenguo Li, and Yong Yu. 2021. Dropnas: Grouped operation dropout for differentiable architecture search. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence. 2326\u20132332."},{"key":"e_1_3_1_59_2","article-title":"NAP: Neural architecture search with pruning","author":"Ding Yadong","year":"2022","unstructured":"Yadong Ding, Yu Wu, Chengyue Huang, Siliang Tang, Fei Wu, Yi Yang, Wenwu Zhu, and Yueting Zhuang. 2022. NAP: Neural architecture search with pruning. Neurocomputing (2022).","journal-title":"Neurocomputing"},{"key":"e_1_3_1_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3532213.3532225"},{"key":"e_1_3_1_61_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ins.2021.09.041"},{"key":"e_1_3_1_62_2","first-page":"3977","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Li Tuanhui","year":"2019","unstructured":"Tuanhui Li, Baoyuan Wu, Yujiu Yang, Yanbo Fan, Yong Zhang, and Wei Liu. 2019. Compressing convolutional neural networks via factorized convolutional filters. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 3977\u20133986."},{"key":"e_1_3_1_63_2","article-title":"Dais: Automatic channel pruning via differentiable annealing indicator search","author":"Guan Yushuo","year":"2022","unstructured":"Yushuo Guan, Ning Liu, Pengyu Zhao, Zhengping Che, Kaigui Bian, Yanzhi Wang, and Jian Tang. 2022. Dais: Automatic channel pruning via differentiable annealing indicator search. IEEE Transactions on Neural Networks and Learning Systems (2022).","journal-title":"IEEE Transactions on Neural Networks and Learning Systems"},{"key":"e_1_3_1_64_2","article-title":"Rethinking the value of network pruning","author":"Liu Zhuang","year":"2018","unstructured":"Zhuang Liu, Mingjie Sun, Tinghui Zhou, Gao Huang, and Trevor Darrell. 2018. Rethinking the value of network pruning. arXiv preprint arXiv:1810.05270 (2018).","journal-title":"arXiv preprint arXiv:1810.05270"},{"key":"e_1_3_1_65_2","article-title":"Learning both weights and connections for efficient neural networks","author":"Han Song","year":"2015","unstructured":"Song Han, Jeff Pool, John Tran, and William J. Dally. 2015. Learning both weights and connections for efficient neural networks. arXiv preprint arXiv:1506.02626 (2015).","journal-title":"arXiv preprint arXiv:1506.02626"},{"key":"e_1_3_1_66_2","article-title":"Pruning filters for efficient convnets","author":"Li Hao","year":"2016","unstructured":"Hao Li, Asim Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. 2016. Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016).","journal-title":"arXiv preprint arXiv:1608.08710"},{"key":"e_1_3_1_67_2","article-title":"The lottery ticket hypothesis: Finding sparse, trainable neural networks","author":"Frankle Jonathan","year":"2018","unstructured":"Jonathan Frankle and Michael Carbin. 2018. The lottery ticket hypothesis: Finding sparse, trainable neural networks. arXiv preprint arXiv:1803.03635 (2018).","journal-title":"arXiv preprint arXiv:1803.03635"},{"key":"e_1_3_1_68_2","first-page":"598","volume-title":"Advances in Neural Information Processing Systems","author":"LeCun Yann","year":"1990","unstructured":"Yann LeCun, John S. Denker, and Sara A. Solla. 1990. Optimal brain damage. In Advances in Neural Information Processing Systems. 598\u2013605."},{"key":"e_1_3_1_69_2","volume-title":"Second Order Derivatives for Network Pruning: Optimal Brain Surgeon","author":"Hassibi Babak","year":"1993","unstructured":"Babak Hassibi and David G. Stork. 1993. Second Order Derivatives for Network Pruning: Optimal Brain Surgeon. Morgan Kaufmann."},{"key":"e_1_3_1_70_2","article-title":"Deconstructing lottery tickets: Zeros, signs, and the supermask","author":"Zhou Hattie","year":"2019","unstructured":"Hattie Zhou, Janice Lan, Rosanne Liu, and Jason Yosinski. 2019. Deconstructing lottery tickets: Zeros, signs, and the supermask. arXiv preprint arXiv:1905.01067 (2019).","journal-title":"arXiv preprint arXiv:1905.01067"},{"key":"e_1_3_1_71_2","first-page":"111","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Ye Shaokai","year":"2019","unstructured":"Shaokai Ye, Kaidi Xu, Sijia Liu, Hao Cheng, Jan-Henrik Lambrechts, Huan Zhang, Aojun Zhou, Kaisheng Ma, Yanzhi Wang, and Xue Lin. 2019. Adversarial robustness vs. model compression, or both?. In Proceedings of the IEEE\/CVF International Conference on Computer Vision. 111\u2013120."},{"key":"e_1_3_1_72_2","first-page":"1285","article-title":"Model compression with adversarial robustness: A unified optimization framework","volume":"32","author":"Gui Shupeng","year":"2019","unstructured":"Shupeng Gui, Haotao N. Wang, Haichuan Yang, Chen Yu, Zhangyang Wang, and Ji Liu. 2019. Model compression with adversarial robustness: A unified optimization framework. Advances in Neural Information Processing Systems 32 (2019), 1285\u20131296.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_73_2","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Kusupati Aditya","year":"2020","unstructured":"Aditya Kusupati, Vivek Ramanujan, Raghav Somani, Mitchell Wortsman, Prateek Jain, Sham Kakade, and Ali Farhadi. 2020. Soft threshold weight reparameterization for learnable sparsity. In Proceedings of the International Conference on Machine Learning."},{"key":"e_1_3_1_74_2","article-title":"On the existence of universal lottery tickets","author":"Burkholz Rebekka","year":"2021","unstructured":"Rebekka Burkholz, Nilanjana Laha, Rajarshi Mukherjee, and Alkis Gotovos. 2021. On the existence of universal lottery tickets. arXiv preprint arXiv:2111.11146 (2021).","journal-title":"arXiv preprint arXiv:2111.11146"},{"key":"e_1_3_1_75_2","first-page":"3747","volume-title":"International Conference on Machine Learning","author":"Chen Tianlong","year":"2022","unstructured":"Tianlong Chen, Zhenyu Zhang, Sijia Liu, Yang Zhang, Shiyu Chang, and Zhangyang Wang. 2022. Data-efficient double-win lottery tickets from robust pre-training. In International Conference on Machine Learning. PMLR, 3747\u20133759."},{"key":"e_1_3_1_76_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10710-017-9314-z"},{"key":"e_1_3_1_77_2","first-page":"508","article-title":"The Kendall rank correlation coefficient","author":"Abdi Herv\u00e9","year":"2007","unstructured":"Herv\u00e9 Abdi. 2007. The Kendall rank correlation coefficient. Encyclopedia of Measurement and Statistics. Sage, Thousand Oaks, CA (2007), 508\u2013510.","journal-title":"Encyclopedia of Measurement and Statistics. Sage, Thousand Oaks, CA"},{"key":"e_1_3_1_78_2","unstructured":"Alex Krizhevsky and Geoffrey Hinton. 2009. Learning multiple layers of features from tiny images. (2009) 7."},{"key":"e_1_3_1_79_2","first-page":"1097","article-title":"Imagenet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25 (2012), 1097\u20131105.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_80_2","article-title":"Sgdr: Stochastic gradient descent with warm restarts","author":"Loshchilov Ilya","year":"2016","unstructured":"Ilya Loshchilov and Frank Hutter. 2016. Sgdr: Stochastic gradient descent with warm restarts. arXiv preprint arXiv:1608.03983 (2016).","journal-title":"arXiv preprint arXiv:1608.03983"},{"key":"e_1_3_1_81_2","first-page":"7105","volume-title":"International Conference on Machine Learning","author":"Ying Chris","year":"2019","unstructured":"Chris Ying, Aaron Klein, Eric Christiansen, Esteban Real, Kevin Murphy, and Frank Hutter. 2019. Nas-bench-101: Towards reproducible neural architecture search. In International Conference on Machine Learning. PMLR, 7105\u20137114."},{"key":"e_1_3_1_82_2","article-title":"Nas-bench-201: Extending the scope of reproducible neural architecture search","author":"Dong Xuanyi","year":"2020","unstructured":"Xuanyi Dong and Yi Yang. 2020. Nas-bench-201: Extending the scope of reproducible neural architecture search. arXiv preprint arXiv:2001.00326 (2020).","journal-title":"arXiv preprint arXiv:2001.00326"},{"key":"e_1_3_1_83_2","article-title":"Quantifying the carbon emissions of machine learning","author":"Lacoste Alexandre","year":"2019","unstructured":"Alexandre Lacoste, Alexandra Luccioni, Victor Schmidt, and Thomas Dandres. 2019. Quantifying the carbon emissions of machine learning. arXiv preprint arXiv:1910.09700 (2019).","journal-title":"arXiv preprint arXiv:1910.09700"},{"key":"e_1_3_1_84_2","doi-asserted-by":"publisher","DOI":"10.1145\/3458864.3467882"},{"key":"e_1_3_1_85_2","first-page":"1620","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Li Guohao","year":"2020","unstructured":"Guohao Li, Guocheng Qian, Itzel C. Delgadillo, Matthias Muller, Ali Thabet, and Bernard Ghanem. 2020. Sgas: Sequential greedy architecture search. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 1620\u20131630."},{"key":"e_1_3_1_86_2","first-page":"1554","volume-title":"International Conference on Machine Learning","author":"Chen Xiangning","year":"2020","unstructured":"Xiangning Chen and Cho-Jui Hsieh. 2020. Stabilizing differentiable architecture search via perturbation-based regularization. In International Conference on Machine Learning. PMLR, 1554\u20131565."},{"key":"e_1_3_1_87_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2877890"},{"key":"e_1_3_1_88_2","article-title":"Network pruning via transformable architecture search","volume":"32","author":"Dong Xuanyi","year":"2019","unstructured":"Xuanyi Dong and Yi Yang. 2019. Network pruning via transformable architecture search. Advances in Neural Information Processing Systems 32 (2019).","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_89_2","article-title":"To prune, or not to prune: Exploring the efficacy of pruning for model compression","author":"Zhu Michael","year":"2017","unstructured":"Michael Zhu and Suyog Gupta. 2017. To prune, or not to prune: Exploring the efficacy of pruning for model compression. arXiv preprint arXiv:1710.01878 (2017).","journal-title":"arXiv preprint arXiv:1710.01878"},{"key":"e_1_3_1_90_2","first-page":"1","volume-title":"2020 International SAUPEC\/RobMech\/PRASA Conference","author":"Paupamah Kimessha","year":"2020","unstructured":"Kimessha Paupamah, Steven James, and Richard Klein. 2020. Quantisation and pruning for neural network compression and regularisation. In 2020 International SAUPEC\/RobMech\/PRASA Conference. IEEE, 1\u20136."},{"key":"e_1_3_1_91_2","doi-asserted-by":"crossref","first-page":"100907","DOI":"10.1016\/j.softx.2021.100907","article-title":"Simplify: A Python library for optimizing pruned neural networks","volume":"17","author":"Bragagnolo Andrea","year":"2022","unstructured":"Andrea Bragagnolo and Carlo Alberto Barbano. 2022. Simplify: A Python library for optimizing pruned neural networks. SoftwareX 17 (2022), 100907.","journal-title":"SoftwareX"},{"issue":"11","key":"e_1_3_1_92_2","article-title":"Visualizing data using t-SNE.","volume":"9","author":"Maaten Laurens Van der","year":"2008","unstructured":"Laurens Van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research 9, 11 (2008).","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_1_93_2","first-page":"364","volume-title":"International Conference on Rough Sets and Knowledge Technology","author":"Yu Dingjun","year":"2014","unstructured":"Dingjun Yu, Hanli Wang, Peiqiu Chen, and Zhihua Wei. 2014. Mixed pooling for convolutional neural networks. In International Conference on Rough Sets and Knowledge Technology. Springer, 364\u2013375."}],"container-title":["ACM Transactions on Embedded Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3609385","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3609385","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:46:23Z","timestamp":1750178783000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3609385"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,9]]},"references-count":92,"journal-issue":{"issue":"5s","published-print":{"date-parts":[[2023,10,31]]}},"alternative-id":["10.1145\/3609385"],"URL":"https:\/\/doi.org\/10.1145\/3609385","relation":{},"ISSN":["1539-9087","1558-3465"],"issn-type":[{"value":"1539-9087","type":"print"},{"value":"1558-3465","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,9]]},"assertion":[{"value":"2023-03-23","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-07-10","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}