{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,29]],"date-time":"2026-09-29T13:03:41Z","timestamp":1790687021467,"version":"4.1.0"},"publisher-location":"New York, NY, USA","reference-count":56,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,8,14]],"date-time":"2021-08-14T00:00:00Z","timestamp":1628899200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Turing AI Institute of Nanjing"},{"name":"Xi'an Institute for Interdisciplinary Information Core Technology"},{"name":"Zhongguancun Haihua Institute for Frontier Information Technology"},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["61822203, 61772297, 61632016"],"award-info":[{"award-number":["61822203, 61772297, 61632016"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,8,14]]},"DOI":"10.1145\/3447548.3467262","type":"proceedings-article","created":{"date-parts":[[2021,8,13]],"date-time":"2021-08-13T18:33:08Z","timestamp":1628879588000},"page":"1933-1943","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":45,"title":["NAS-BERT"],"prefix":"10.1145","author":[{"given":"Jin","family":"Xu","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xu","family":"Tan","sequence":"additional","affiliation":[{"name":"Microsoft Research Asia, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Renqian","family":"Luo","sequence":"additional","affiliation":[{"name":"University of Science and Technology of China, Hefei, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kaitao","family":"Song","sequence":"additional","affiliation":[{"name":"Nanjing University of Science and Technology, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jian","family":"Li","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tao","family":"Qin","sequence":"additional","affiliation":[{"name":"Microsoft Research Asia, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tie-Yan","family":"Liu","sequence":"additional","affiliation":[{"name":"Microsoft Research Asia, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2021,8,14]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"International Conference on Machine Learning. 550--559","author":"Bender Gabriel","year":"2018","unstructured":"Gabriel Bender , Pieter-Jan Kindermans , Barret Zoph , Vijay Vasudevan , and Quoc Le . 2018 . Understanding and simplifying one-shot architecture search . In International Conference on Machine Learning. 550--559 . Gabriel Bender, Pieter-Jan Kindermans, Barret Zoph, Vijay Vasudevan, and Quoc Le. 2018. Understanding and simplifying one-shot architecture search. In International Conference on Machine Learning. 550--559."},{"key":"e_1_3_2_2_2_1","volume-title":"International Conference on Learning Representations.","author":"Cai Han","year":"2019","unstructured":"Han Cai , Chuang Gan , Tianzhe Wang , Zhekai Zhang , and Song Han . 2019 . Oncefor- All: Train One Network and Specialize it for Efficient Deployment . In International Conference on Learning Representations. Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. 2019. Oncefor- All: Train One Network and Specialize it for Efficient Deployment. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_3_1","volume-title":"ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. In International Conference on Learning Representations.","author":"Cai Han","year":"2018","unstructured":"Han Cai , Ligeng Zhu , and Song Han . 2018 . ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. In International Conference on Learning Representations. Han Cai, Ligeng Zhu, and Song Han. 2018. ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S17-2001"},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2020\/341"},{"key":"e_1_3_2_2_6_1","unstructured":"Zihan Chen Hongbo Zhang Xiaoji Zhang and Leqi Zhao. 2018. Quora question pairs.  Zihan Chen Hongbo Zhang Xiaoji Zhang and Leqi Zhao. 2018. Quora question pairs."},{"key":"e_1_3_2_2_7_1","volume-title":"Fairnas: Rethinking evaluation fairness of weight sharing neural architecture search. arXiv preprint arXiv:1907.01845","author":"Chu Xiangxiang","year":"2019","unstructured":"Xiangxiang Chu , Bo Zhang , Ruijun Xu , and Jixiang Li . 2019 . Fairnas: Rethinking evaluation fairness of weight sharing neural architecture search. arXiv preprint arXiv:1907.01845 (2019). Xiangxiang Chu, Bo Zhang, Ruijun Xu, and Jixiang Li. 2019. Fairnas: Rethinking evaluation fairness of weight sharing neural architecture search. arXiv preprint arXiv:1907.01845 (2019)."},{"key":"e_1_3_2_2_8_1","volume-title":"International Conference on Learning Representations.","author":"Clark Kevin","year":"2019","unstructured":"Kevin Clark , Minh-Thang Luong , Quoc V Le , and Christopher D Manning . 2019 . ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators . In International Conference on Learning Representations. Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2019. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"crossref","unstructured":"Ido Dagan Oren Glickman and Bernardo Magnini. 2006. The PASCAL Recognising Textual Entailment Challenge. In Machine Learning Challenges. Evaluating Predictive Uncertainty Visual Object Classification and Recognising Tectual Entailment. 177--190.  Ido Dagan Oren Glickman and Bernardo Magnini. 2006. The PASCAL Recognising Textual Entailment Challenge. In Machine Learning Challenges. Evaluating Predictive Uncertainty Visual Object Classification and Recognising Tectual Entailment. 177--190.","DOI":"10.1007\/11736790_9"},{"key":"e_1_3_2_2_10_1","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","volume":"1","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, 4171--4186. https:\/\/doi.org\/10. 18653\/v1\/N19--1423 10.18653\/v1 Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, 4171--4186. https:\/\/doi.org\/10.18653\/v1\/N19--1423"},{"key":"e_1_3_2_2_11_1","volume-title":"Proceedings of the Third International Workshop on Paraphrasing (IWP2005)","author":"William","unstructured":"William B. Dolan and Chris Brockett. 2005. Automatically Constructing a Corpus of Sentential Paraphrases . In Proceedings of the Third International Workshop on Paraphrasing (IWP2005) . William B. Dolan and Chris Brockett. 2005. Automatically Constructing a Corpus of Sentential Paraphrases. In Proceedings of the Third International Workshop on Paraphrasing (IWP2005)."},{"key":"e_1_3_2_2_12_1","volume-title":"Jan Hendrik Metzen, and Frank Hutter","author":"Elsken Thomas","year":"2018","unstructured":"Thomas Elsken , Jan Hendrik Metzen, and Frank Hutter . 2018 . Neural architecture search: A survey. arXiv preprint arXiv:1808.05377 (2018). Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. 2018. Neural architecture search: A survey. arXiv preprint arXiv:1808.05377 (2018)."},{"key":"e_1_3_2_2_13_1","volume-title":"Reducing Transformer Depth on Demand with Structured Dropout. In International Conference on Learning Representations.","author":"Fan Angela","year":"2019","unstructured":"Angela Fan , Edouard Grave , and Armand Joulin . 2019 . Reducing Transformer Depth on Demand with Structured Dropout. In International Conference on Learning Representations. Angela Fan, Edouard Grave, and Armand Joulin. 2019. Reducing Transformer Depth on Demand with Structured Dropout. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.repl4nlp-1.18"},{"key":"e_1_3_2_2_15_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58517-4_32"},{"key":"e_1_3_2_2_16_1","volume-title":"DynaBERT: Dynamic BERT with Adaptive Width and Depth. Advances in Neural Information Processing Systems 33","author":"Hou Lu","year":"2020","unstructured":"Lu Hou , Zhiqi Huang , Lifeng Shang , Xin Jiang , Xiao Chen , and Qun Liu . 2020. DynaBERT: Dynamic BERT with Adaptive Width and Depth. Advances in Neural Information Processing Systems 33 ( 2020 ). Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, and Qun Liu. 2020. DynaBERT: Dynamic BERT with Adaptive Width and Depth. Advances in Neural Information Processing Systems 33 (2020)."},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00140"},{"key":"e_1_3_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.372"},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330648"},{"key":"e_1_3_2_2_20_1","volume-title":"Depthwise Separable Convolutions for Neural Machine Translation. In International Conference on Learning Representations.","author":"Kaiser Lukasz","year":"2018","unstructured":"Lukasz Kaiser , Aidan N Gomez , and Francois Chollet . 2018 . Depthwise Separable Convolutions for Neural Machine Translation. In International Conference on Learning Representations. Lukasz Kaiser, Aidan N Gomez, and Francois Chollet. 2018. Depthwise Separable Convolutions for Neural Machine Translation. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00521-019-04603-0"},{"key":"e_1_3_2_2_22_1","volume-title":"Adam: A Method for Stochastic Optimization. In ICLR (Poster).","author":"Kingma Diederik P","year":"2015","unstructured":"Diederik P Kingma and Jimmy Ba . 2015 . Adam: A Method for Stochastic Optimization. In ICLR (Poster). Diederik P Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In ICLR (Poster)."},{"key":"e_1_3_2_2_23_1","unstructured":"Guillaume Lample Alexandre Sablayrolles Marc'Aurelio Ranzato Ludovic Denoyer and Herv\u00e9 J\u00e9gou. 2019. Large memory layers with product keys. In Advances in Neural Information Processing Systems. 8548--8559.  Guillaume Lample Alexandre Sablayrolles Marc'Aurelio Ranzato Ludovic Denoyer and Herv\u00e9 J\u00e9gou. 2019. Large memory layers with product keys. In Advances in Neural Information Processing Systems. 8548--8559."},{"key":"e_1_3_2_2_24_1","volume-title":"ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations.","author":"Lan Zhenzhong","year":"2019","unstructured":"Zhenzhong Lan , Mingda Chen , Sebastian Goodman , Kevin Gimpel , Piyush Sharma , and Radu Soricut . 2019 . ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00206"},{"key":"e_1_3_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403122"},{"key":"e_1_3_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01385"},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3394486.3403153"},{"key":"e_1_3_2_2_29_1","volume-title":"DARTS: Differentiable Architecture Search. In International Conference on Learning Representations.","author":"Liu Hanxiao","year":"2018","unstructured":"Hanxiao Liu , Karen Simonyan , and Yiming Yang . 2018 . DARTS: Differentiable Architecture Search. In International Conference on Learning Representations. Hanxiao Liu, Karen Simonyan, and Yiming Yang. 2018. DARTS: Differentiable Architecture Search. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_30_1","volume-title":"Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu , Myle Ott , Naman Goyal , Jingfei Du , Mandar Joshi , Danqi Chen , Omer Levy , Mike Lewis , Luke Zettlemoyer , and Veselin Stoyanov . 2019 . Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019). Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)."},{"key":"e_1_3_2_2_31_1","unstructured":"Renqian Luo Tao Qin and Enhong Chen. 2019. Balanced One-shot Neural Architecture Optimization. arXiv:1909.10815 [cs.LG]  Renqian Luo Tao Qin and Enhong Chen. 2019. Balanced One-shot Neural Architecture Optimization. arXiv:1909.10815 [cs.LG]"},{"key":"e_1_3_2_2_32_1","volume-title":"Neural architecture search with gbdt. arXiv preprint arXiv:2007.04785","author":"Luo Renqian","year":"2020","unstructured":"Renqian Luo , Xu Tan , Rui Wang , Tao Qin , Enhong Chen , and Tie-Yan Liu . 2020. Neural architecture search with gbdt. arXiv preprint arXiv:2007.04785 ( 2020 ). Renqian Luo, Xu Tan, Rui Wang, Tao Qin, Enhong Chen, and Tie-Yan Liu. 2020. Neural architecture search with gbdt. arXiv preprint arXiv:2007.04785 (2020)."},{"key":"e_1_3_2_2_33_1","volume-title":"Pruning a bert-based question answering model. arXiv preprint arXiv:1910.06360","author":"McCarley J Scott","year":"2019","unstructured":"J Scott McCarley . 2019. Pruning a bert-based question answering model. arXiv preprint arXiv:1910.06360 ( 2019 ). J Scott McCarley. 2019. Pruning a bert-based question answering model. arXiv preprint arXiv:1910.06360 (2019)."},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.01044"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-2124"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"crossref","unstructured":"Pranav Rajpurkar Jian Zhang Konstantin Lopyrev and Percy Liang. 2016. SQuAD: 100 000+ Questions for Machine Comprehension of Text. In EMNLP. 2383--2392.  Pranav Rajpurkar Jian Zhang Konstantin Lopyrev and Percy Liang. 2016. SQuAD: 100 000+ Questions for Machine Comprehension of Text. In EMNLP. 2383--2392.","DOI":"10.18653\/v1\/D16-1264"},{"key":"e_1_3_2_2_37_1","volume-title":"a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108","author":"Sanh Victor","year":"2019","unstructured":"Victor Sanh , Lysandre Debut , Julien Chaumond , and Thomas Wolf . 2019. DistilBERT , a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 ( 2019 ). Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)."},{"key":"e_1_3_2_2_38_1","doi-asserted-by":"crossref","unstructured":"Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Zhewei Yao Amir Gholami Michael W Mahoney and Kurt Keutzer. 2020. Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT.. In AAAI. 8815--8821.  Sheng Shen Zhen Dong Jiayu Ye Linjian Ma Zhewei Yao Amir Gholami Michael W Mahoney and Kurt Keutzer. 2020. Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT.. In AAAI. 8815--8821.","DOI":"10.1609\/aaai.v34i05.6409"},{"key":"e_1_3_2_2_39_1","unstructured":"Richard Socher Alex Perelygin JeanWu Jason Chuang Christopher D. Manning Andrew Ng and Christopher Potts. 2013. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In EMNLP. 1631--1642.  Richard Socher Alex Perelygin JeanWu Jason Chuang Christopher D. Manning Andrew Ng and Christopher Potts. 2013. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In EMNLP. 1631--1642."},{"key":"e_1_3_2_2_40_1","volume-title":"LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning. arXiv preprint arXiv:2004.12817","author":"Song Kaitao","year":"2020","unstructured":"Kaitao Song , Hao Sun , Xu Tan , Tao Qin , Jianfeng Lu , Hongzhi Liu , and Tie-Yan Liu . 2020. LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning. arXiv preprint arXiv:2004.12817 ( 2020 ). Kaitao Song, Hao Sun, Xu Tan, Tao Qin, Jianfeng Lu, Hongzhi Liu, and Tie-Yan Liu. 2020. LightPAFF: A Two-Stage Distillation Framework for Pre-training and Fine-tuning. arXiv preprint arXiv:2004.12817 (2020)."},{"key":"e_1_3_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1441"},{"key":"e_1_3_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.195"},{"key":"e_1_3_2_2_43_1","volume-title":"Hyung Won Chung, and Jason Riesa","author":"Tsai Henry","year":"2020","unstructured":"Henry Tsai , Jayden Ooi , Chun-Sung Ferng , Hyung Won Chung, and Jason Riesa . 2020 . Finding Fast Transformers: One- Shot Neural Architecture Search by Component Composition . arXiv preprint arXiv:2008.06808 (2020). Henry Tsai, Jayden Ooi, Chun-Sung Ferng, Hyung Won Chung, and Jason Riesa. 2020. Finding Fast Transformers: One-Shot Neural Architecture Search by Component Composition. arXiv preprint arXiv:2008.06808 (2020)."},{"key":"e_1_3_2_2_44_1","volume-title":"Well-read students learn better: On the importance of pre-training compact models. arXiv preprint arXiv:1908.08962","author":"Turc Iulia","year":"2019","unstructured":"Iulia Turc , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019. Well-read students learn better: On the importance of pre-training compact models. arXiv preprint arXiv:1908.08962 ( 2019 ). Iulia Turc, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Well-read students learn better: On the importance of pre-training compact models. arXiv preprint arXiv:1908.08962 (2019)."},{"key":"e_1_3_2_2_45_1","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Lukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.  Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N Gomez Lukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008."},{"key":"e_1_3_2_2_46_1","volume-title":"GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In International Conference on Learning Representations.","author":"Singh Amanpreet","year":"2018","unstructured":"AlexWang, Amanpreet Singh , Julian Michael , Felix Hill , Omer Levy , and Samuel R Bowman . 2018 . GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In International Conference on Learning Representations. AlexWang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. 2018. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.686"},{"key":"e_1_3_2_2_48_1","volume-title":"Sample-efficient neural architecture search by learning action space. arXiv preprint arXiv:1906.06832","author":"Wang Linnan","year":"2019","unstructured":"Linnan Wang , Saining Xie , Teng Li , Rodrigo Fonseca , and Yuandong Tian . 2019. Sample-efficient neural architecture search by learning action space. arXiv preprint arXiv:1906.06832 ( 2019 ). Linnan Wang, Saining Xie, Teng Li, Rodrigo Fonseca, and Yuandong Tian. 2019. Sample-efficient neural architecture search by learning action space. arXiv preprint arXiv:1906.06832 (2019)."},{"key":"e_1_3_2_2_49_1","volume-title":"Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems","author":"Wang Wenhui","year":"2020","unstructured":"Wenhui Wang , Furu Wei , Li Dong , Hangbo Bao , Nan Yang , and Ming Zhou . 2020. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers . In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020 , NeurIPS 2020, December 6-12, 2020, virtual, Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds .). https:\/\/proceedings. neurips.cc\/paper\/2020\/hash\/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, Hugo Larochelle, Marc'Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). https:\/\/proceedings. neurips.cc\/paper\/2020\/hash\/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html"},{"key":"e_1_3_2_2_50_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00290"},{"key":"e_1_3_2_2_51_1","doi-asserted-by":"crossref","unstructured":"Adina Williams Nikita Nangia and Samuel Bowman. 2018. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. In NAACL. 1112--1122.  Adina Williams Nikita Nangia and Samuel Bowman. 2018. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. In NAACL. 1112--1122.","DOI":"10.18653\/v1\/N18-1101"},{"key":"e_1_3_2_2_52_1","volume-title":"Pay Less Attention with Lightweight and Dynamic Convolutions. In International Conference on Learning Representations.","author":"Wu Felix","year":"2018","unstructured":"Felix Wu , Angela Fan , Alexei Baevski , Yann Dauphin , and Michael Auli . 2018 . Pay Less Attention with Lightweight and Dynamic Convolutions. In International Conference on Learning Representations. Felix Wu, Angela Fan, Alexei Baevski, Yann Dauphin, and Michael Auli. 2018. Pay Less Attention with Lightweight and Dynamic Convolutions. In International Conference on Learning Representations."},{"key":"e_1_3_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.633"},{"key":"e_1_3_2_2_54_1","volume-title":"Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5753--5763.","author":"Yang Zhilin","year":"2019","unstructured":"Zhilin Yang , Zihang Dai , Yiming Yang , Jaime Carbonell , Russ R Salakhutdinov , and Quoc V Le . 2019 . Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5753--5763. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: Generalized autoregressive pretraining for language understanding. In Advances in neural information processing systems. 5753--5763."},{"key":"e_1_3_2_2_55_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58571-6_41"},{"key":"e_1_3_2_2_56_1","volume-title":"Q8bert: Quantized 8bit bert. arXiv preprint arXiv:1910.06188","author":"Zafrir Ofir","year":"2019","unstructured":"Ofir Zafrir , Guy Boudoukh , Peter Izsak , and Moshe Wasserblat . 2019. Q8bert: Quantized 8bit bert. arXiv preprint arXiv:1910.06188 ( 2019 ). Ofir Zafrir, Guy Boudoukh, Peter Izsak, and Moshe Wasserblat. 2019. Q8bert: Quantized 8bit bert. arXiv preprint arXiv:1910.06188 (2019)."}],"event":{"name":"KDD '21: The 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Virtual Event Singapore","acronym":"KDD '21","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447548.3467262","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3447548.3467262","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:18:28Z","timestamp":1750191508000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3447548.3467262"}},"subtitle":["Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Search"],"short-title":[],"issued":{"date-parts":[[2021,8,14]]},"references-count":56,"alternative-id":["10.1145\/3447548.3467262","10.1145\/3447548"],"URL":"https:\/\/doi.org\/10.1145\/3447548.3467262","relation":{},"subject":[],"published":{"date-parts":[[2021,8,14]]},"assertion":[{"value":"2021-08-14","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}