{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,24]],"date-time":"2026-02-24T17:05:52Z","timestamp":1771952752808,"version":"3.50.1"},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2021,6,30]],"date-time":"2021-06-30T00:00:00Z","timestamp":1625011200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"NSF","award":["CCF-1725456"],"award-info":[{"award-number":["CCF-1725456"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["J. Emerg. Technol. Comput. Syst."],"published-print":{"date-parts":[[2021,10,31]]},"abstract":"<jats:p>\n            When deploying deep neural networks in embedded systems, it is crucial to decrease the model size and computational complexity for improving the execution speed and efficiency. In addition to conventional compression techniques, e.g., weight pruning and quantization, removing unimportant activations can also dramatically reduce the amount of data communication and the computation cost. Unlike weight parameters, the pattern of activations is directly related to input data and thereby changes dynamically. To regulate the dynamic activation sparsity (DAS), in this work, we propose a generic low-cost approach based on winners-take-all (WTA) dropout technique. The network enhanced by the proposed WTA dropout, namely\n            <jats:italic>DASNet<\/jats:italic>\n            , features structured activation sparsity with an improved sparsity level. Compared to the static feature map pruning methods, DASNets provide better computation cost reduction. The WTA dropout technique can be easily applied in deep neural networks without incurring additional training variables. More importantly, DASNet can be seamlessly integrated with other compression techniques, such as weight pruning and quantization, without compromising accuracy. Our experiments on various networks and datasets present significant runtime speedups with negligible accuracy losses.\n          <\/jats:p>","DOI":"10.1145\/3447776","type":"journal-article","created":{"date-parts":[[2021,6,30]],"date-time":"2021-06-30T19:37:47Z","timestamp":1625081867000},"page":"1-16","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Dynamic Regularization on Activation Sparsity for Neural Network Efficiency Improvement"],"prefix":"10.1145","volume":"17","author":[{"given":"Qing","family":"Yang","sequence":"first","affiliation":[{"name":"Department of Electrical and Computer Engineering,Duke University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jiachen","family":"Mao","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering,Duke University"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zuoguan","family":"Wang","sequence":"additional","affiliation":[{"name":"Black Sesame Technologies"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"\u201cHelen\u201d Li","family":"Hai","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Duke University"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,6,30]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/3007787.3001138"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/2999792.2999956"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.354"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/1008861.1008865"},{"key":"e_1_2_1_5_1","volume-title":"Mxnet: A flexible and efficient machine learning library for heterogeneous distributed systems. arXiv:1512.01274.","author":"Chen Tianqi","year":"2015"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/JSSC.2016.2616357"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/2969442.2969588"},{"key":"e_1_2_1_8_1","volume-title":"International Conference on Machine Learning. 1135\u20131144","author":"Dai Bin","year":"2018"},{"key":"e_1_2_1_9_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Gao Xitong","year":"2019"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3007787.3001163"},{"key":"e_1_2_1_11_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Han Song"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/ISSCC.2014.6757323"},{"key":"e_1_2_1_14_1","volume-title":"Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861.","author":"Howard Andrew G.","year":"2017"},{"key":"e_1_2_1_15_1","unstructured":"Hengyuan Hu Rui Peng Yu-Wing Tai and Chi-Keung Tang. 2016. Network trimming: A data-driven neuron pruning approach towards efficient deep architectures. arXiv:1607.03250. Retrieved from https:\/\/arxiv.org\/abs\/1607.03250.  Hengyuan Hu Rui Peng Yu-Wing Tai and Chi-Keung Tang. 2016. Network trimming: A data-driven neuron pruning approach towards efficient deep architectures. arXiv:1607.03250. Retrieved from https:\/\/arxiv.org\/abs\/1607.03250."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.5555\/3454287.3454456"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01270-0_19"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.5555\/3157382.3157557"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3140659.3080246"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/MDAT.2017.2741463"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.5555\/2999134.2999257"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.298"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/2969442.2969552"},{"key":"e_1_2_1_24_1","volume-title":"Proceedings of the International Conference on Learning Representations.","author":"Molchanov Pavlo","year":"2017"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46493-0_32"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3007787.3001165"},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the IEEE International Symposium on High Performance Computer Architecture. 78\u201391","author":"Rhu Minsoo"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3287624.3287663"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3097983.3098035"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.5555\/2627435.2670313"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.5555\/3157096.3157329"},{"key":"e_1_2_1_32_1","volume-title":"Proceedings of the 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA\u201919)","author":"Yang Q."},{"key":"e_1_2_1_33_1","unstructured":"Jianbo Ye Xin Lu Zhe Lin and James Z Wang. 2018. Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers. arXiv:1802.00124. Retrieved from https:\/\/arxiv.org\/abs\/1802.00124.  Jianbo Ye Xin Lu Zhe Lin and James Z Wang. 2018. Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers. arXiv:1802.00124. Retrieved from https:\/\/arxiv.org\/abs\/1802.00124."},{"key":"e_1_2_1_34_1","volume-title":"Proceedings of the European Conference on Computer Vision. 818\u2013833","author":"Matthew"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2015.2502579"}],"container-title":["ACM Journal on Emerging Technologies in Computing Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447776","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447776","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447776","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:41:10Z","timestamp":1750200070000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447776"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,30]]},"references-count":35,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,10,31]]}},"alternative-id":["10.1145\/3447776"],"URL":"https:\/\/doi.org\/10.1145\/3447776","relation":{},"ISSN":["1550-4832","1550-4840"],"issn-type":[{"value":"1550-4832","type":"print"},{"value":"1550-4840","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,30]]},"assertion":[{"value":"2020-04-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-01-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2021-06-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}