{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,14]],"date-time":"2026-11-14T10:18:32Z","timestamp":1794651512656,"version":"4.9.1"},"reference-count":86,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,10,18]],"date-time":"2024-10-18T00:00:00Z","timestamp":1729209600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62376027"],"award-info":[{"award-number":["62376027"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Beijing Municipal Natural Science Foundation","award":["4222036 and IS23061"],"award-info":[{"award-number":["4222036 and IS23061"]}]},{"name":"Shenzhen Science and Technology Program","award":["JCYJ20220818103001002"],"award-info":[{"award-number":["JCYJ20220818103001002"]}]},{"name":"Shenzhen Doctoral Startup Funding","award":["RCBS20221008093330065"],"award-info":[{"award-number":["RCBS20221008093330065"]}]},{"name":"Tianyuan Fund for Mathematics of National Natural Science Foundation of China","award":["12326608"],"award-info":[{"award-number":["12326608"]}]},{"name":"Shenzhen Key Laboratory of Cross-Modal Cognitive Computing","award":["ZDSYS20230626091302006"],"award-info":[{"award-number":["ZDSYS20230626091302006"]}]},{"name":"Shenzhen Stability Science Program 2023"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2024,11,30]]},"abstract":"<jats:p>\n                    Large-scale pretrained language models have achieved compelling performance in a wide range of language understanding and information retrieval tasks. While their large scales ensure capacity, they also hinder deployment. Knowledge distillation offers an opportunity to compress a large language model to a small one, in order to reach a reasonable latency-performance tradeoff. However, for scenarios where the number of requests (e.g., queries submitted to a search engine) is highly variant, the static tradeoff attained by the compressed language model might not always fit. Once a model is assigned with a static tradeoff, it could be inadequate in that the latency is too high when the number of requests is large, or the performance is too low when the number of requests is small. To this end, we propose an elastic language model (\n                    <jats:sc>ElasticLM<\/jats:sc>\n                    ) that elastically adjusts the tradeoff according to the request stream. The basic idea is to introduce a compute elasticity to the compressed language model, so that the tradeoff could vary on-the-fly along a scalable and controllable compute. Specifically, we impose an elastic structure to equip\n                    <jats:sc>ElasticLM<\/jats:sc>\n                    with compute elasticity and design an elastic optimization method to learn\n                    <jats:sc>ElasticLM<\/jats:sc>\n                    under compute elasticity. To serve\n                    <jats:sc>ElasticLM<\/jats:sc>\n                    , we apply an elastic schedule. Considering the specificity of information retrieval, we adapt\n                    <jats:sc>ElasticLM<\/jats:sc>\n                    to dense retrieval and reranking, and present an\n                    <jats:sc>ElasticDenser<\/jats:sc>\n                    and an\n                    <jats:sc>ElasticRanker,<\/jats:sc>\n                    respectively. Offline evaluation is conducted on a language understanding benchmark GLUE, and several information retrieval tasks including Natural Question, Trivia QA and MS MARCO. The results show that\n                    <jats:sc>ElasticLM<\/jats:sc>\n                    along with\n                    <jats:sc>ElasticDenser<\/jats:sc>\n                    and\n                    <jats:sc>ElasticRanker<\/jats:sc>\n                    can perform correctly and competitively compared with an array of static baselines. Furthermore, an online simulation with concurrency is also carried out. The results demonstrate that\n                    <jats:sc>ElasticLM<\/jats:sc>\n                    can provide elastic tradeoffs with respect to varying request stream.\n                  <\/jats:p>","DOI":"10.1145\/3677375","type":"journal-article","created":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T12:38:00Z","timestamp":1720787880000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["On Elastic Language Models"],"prefix":"10.1145","volume":"42","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4395-9943","authenticated-orcid":false,"given":"Chen","family":"Zhang","sequence":"first","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1501-9914","authenticated-orcid":false,"given":"Benyou","family":"Wang","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Shenzhen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8660-3608","authenticated-orcid":false,"given":"Dawei","family":"Song","sequence":"additional","affiliation":[{"name":"Beijing Institute of Technology, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,10,18]]},"reference":[{"key":"e_1_3_2_2_2","first-page":"1","volume-title":"Proceedings of the 4th Text Analysis Conference (TAC \u201911)","author":"Bentivogli Luisa","year":"2011","unstructured":"Luisa Bentivogli, Peter Clark, Ido Dagan, and Danilo Giampiccolo. 2011. The Seventh PASCAL Recognizing Textual Entailment Challenge. In Proceedings of the 4th Text Analysis Conference (TAC \u201911). 1\u201316. Retrieved from https:\/\/tac.nist.gov\/publications\/2011\/additional.papers\/RTE7_overview.proceedings.pdf"},{"key":"e_1_3_2_3_2","volume-title":"Proceedings of the 8th International Conference on Learning Representations (ICLR \u201920)","author":"Cai Han","year":"2020","unstructured":"Han Cai, Chuang Gan, Tianzhe Wang, Zhekai Zhang, and Song Han. 2020. Once-for-All: Train One Network and Specialize it for Efficient Deployment. In Proceedings of the 8th International Conference on Learning Representations (ICLR \u201920). OpenReview.net, 1\u201315. Retrieved from https:\/\/openreview.net\/forum?id=HylxE1HKwS"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/S17-2001"},{"key":"e_1_3_2_5_2","first-page":"1","volume-title":"Proceedings of the 8th International Conference on Learning Representations (ICLR \u201920)","author":"Chang Wei-Cheng","year":"2020","unstructured":"Wei-Cheng Chang, Felix X. Yu, Yin-Wen Chang, Yiming Yang, and Sanjiv Kumar. 2020. Pre-Training Tasks for Embedding-Based Large-Scale Retrieval. In Proceedings of the 8th International Conference on Learning Representations (ICLR \u201920). OpenReview.net, 1\u201312. Retrieved from https:\/\/openreview.net\/forum?id=rkg-mA4FDr"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531943"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-72240-1_21"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401204"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/n19-1423"},{"key":"e_1_3_2_10_2","first-page":"1","volume-title":"Proceedings of the 3rd International Workshop on Paraphrasing (IWP@IJCNLP \u201905)","author":"Dolan William B.","year":"2005","unstructured":"William B. Dolan and Chris Brockett. 2005. Automatically Constructing a Corpus of Sentential Paraphrases. In Proceedings of the 3rd International Workshop on Paraphrasing (IWP@IJCNLP \u201905). 1\u20138. Retrieved from https:\/\/aclanthology.org\/I05-5002\/"},{"key":"e_1_3_2_11_2","first-page":"1","volume-title":"Proceedings of the 8th International Conference on Learning Representations (ICLR \u201920)","author":"Fan Angela","year":"2020","unstructured":"Angela Fan, Edouard Grave, and Armand Joulin. 2020. Reducing Transformer Depth on Demand with Structured Dropout. In Proceedings of the 8th International Conference on Learning Representations (ICLR \u201920). OpenReview.net, 1\u201316. Retrieved from https:\/\/openreview.net\/forum?id=SylO2yStDr"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1561\/1500000100"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463098"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.75"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.203"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.342"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3409256.3409838"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.241"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-72240-1_26"},{"key":"e_1_3_2_20_2","first-page":"1135","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015","author":"Han Song","year":"2015","unstructured":"Song Han, Jeff Pool, John Tran, and William J. Dally. 2015. Learning Both Weights and Connections for Efficient Neural Network. In Proceedings of the Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, Corinna Cortes, Neil D. Lawrence, Daniel D. Lee, Masashi Sugiyama, and Roman Garnett (Eds.). 1135\u20131143. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2015\/hash\/ae0eb3eed39d2bcef4622b2499a05fe6-Abstract.html"},{"key":"e_1_3_2_21_2","unstructured":"Geoffrey E. Hinton Oriol Vinyals and Jeffrey Dean. 2015. Distilling the Knowledge in a Neural Network. arXiv:1503.02531. Retrieved from http:\/\/arxiv.org\/abs\/1503.02531"},{"key":"e_1_3_2_22_2","first-page":"9782","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS \u201920)","author":"Hou Lu","year":"2020","unstructured":"Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, and Qun Liu. 2020. DynaBERT: Dynamic BERT with Adaptive Width and Depth. In Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS \u201920), Hugo Larochelle, Marc\u2019Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). 9782\u20139793. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/6f5216f8d89b086c18298e043bfe48ed-Abstract.html"},{"key":"e_1_3_2_23_2","first-page":"1","volume-title":"Proceedings of the 8th International Conference on Learning Representations (ICLR \u201920)","author":"Humeau Samuel","year":"2020","unstructured":"Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2020. Poly-Encoders: Architectures and Pre-Training Strategies for Fast and Accurate Multi-sentence Scoring. In Proceedings of the 8th International Conference on Learning Representations (ICLR \u201920). OpenReview.net, 1\u201314. Retrieved from https:\/\/openreview.net\/forum?id=SkxgnnNFvH"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.372"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1147"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.550"},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401075"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.508"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00276"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/p19-1612"},{"key":"e_1_3_2_31_2","first-page":"552","volume-title":"Principles of Knowledge Representation and Reasoning: Proceedings of the 30th International Conference (KR \u201912)","author":"Levesque Hector J.","year":"2012","unstructured":"Hector J. Levesque, Ernest Davis, and Leora Morgenstern. 2012. The Winograd Schema Challenge. In Principles of Knowledge Representation and Reasoning: Proceedings of the 30th International Conference (KR \u201912). Institute of Electrical and Electronics Engineers Inc., 552\u2013561. Retrieved from http:\/\/www.aaai.org\/ocs\/index.php\/KR\/KR12\/paper\/view\/4492"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.2200\/S01123ED1V01Y202108HLT053"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1145\/3543507.3583421"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.537"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.naacl-main.240"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.220"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00369"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2022.EMNLP-MAIN.758"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531772"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3437963.3441777"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462869"},{"key":"e_1_3_2_42_2","unstructured":"Xinyu Ma Ruqing Zhang Jiafeng Guo Yixing Fan and Xueqi Cheng. 2022b. A Contrastive Pre-training Approach to Learn Discriminative Autoencoder for Dense Retrieval. arXiv:2208.09846. Retrieved from https:\/\/arxiv.org\/abs\/2208.09846"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3459637.3482286"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401093"},{"key":"e_1_3_2_45_2","first-page":"14014","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems (NeurIPS \u201919)","author":"Michel Paul","year":"2019","unstructured":"Paul Michel, Omer Levy, and Graham Neubig. 2019. Are Sixteen Heads Really Better than One? In Proceedings of the Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems (NeurIPS \u201919), Hanna M. Wallach, Hugo Larochelle, Alina Beygelzimer, Florence d\u2019Alch\u00e9-Buc, Emily B. Fox, and Roman Garnett (Eds.). 14014\u201314024. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2019\/hash\/2c601ad9d2ff9bc8b282670cdd54f69f-Abstract.html"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i04.5963"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3038912.3052579"},{"key":"e_1_3_2_48_2","first-page":"1","volume-title":"Proceedings of the 5th International Conference on Learning Representations (ICLR \u201917), Conference Track Proceedings","author":"Molchanov Pavlo","year":"2017","unstructured":"Pavlo Molchanov, Stephen Tyree, Tero Karras, Timo Aila, and Jan Kautz. 2017. Pruning Convolutional Neural Networks for Resource Efficient Inference. In Proceedings of the 5th International Conference on Learning Representations (ICLR \u201917), Conference Track Proceedings. OpenReview.net., 1\u201317. Retrieved from https:\/\/openreview.net\/forum?id=SJGCiw5gl"},{"key":"e_1_3_2_49_2","first-page":"1","volume-title":"Proceedings of the Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches 2016 Co-Located with the 30th Annual Conference on Neural Information Processing Systems (NIPS \u201916)","volume":"1773","author":"Nguyen Tri","year":"2016","unstructured":"Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A Human Generated MAchine Reading COmprehension Dataset. In Proceedings of the Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches 2016 Co-Located with the 30th Annual Conference on Neural Information Processing Systems (NIPS \u201916), (CEUR Workshop Proceedings, Vol. 1773), Tarek Richard Besold, Antoine Bordes, Artur S. d\u2019Avila Garcez, and Greg Wayne (Eds.). CEUR-WS.org, 1\u201310. Retrieved from http:\/\/ceur-ws.org\/Vol-1773\/CoCoNIPS_2016_paper9.pdf"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2022.EMNLP-MAIN.669"},{"key":"e_1_3_2_51_2","unstructured":"Rodrigo Frassetto Nogueira and Kyunghyun Cho. 2019. Passage Re-Ranking with BERT. arXiv:1901.04085. Retrieved from http:\/\/arxiv.org\/abs\/1901.04085"},{"key":"e_1_3_2_52_2","unstructured":"Rodrigo Frassetto Nogueira and Jimmy Lin. 2019. From doc2query to docTTTTTquery. Retrieved from https:\/\/cs.uwaterloo.ca\/jimmylin\/publications\/Nogueira_Lin_2019_docTTTTTquery-v2.pdf"},{"key":"e_1_3_2_53_2","unstructured":"Rodrigo Frassetto Nogueira Wei Yang Jimmy Lin and Kyunghyun Cho. 2019. Document Expansion by Query Prediction. arXiv:1904.08375. Retrieved from http:\/\/arxiv.org\/abs\/1904.08375"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.466"},{"key":"e_1_3_2_55_2","first-page":"140:1","article-title":"Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research 21 (2020), 140:1\u2013140:67. Retrieved from http:\/\/jmlr.org\/papers\/v21\/20-074.html","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/d16-1264"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.224"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1561\/1500000019"},{"key":"e_1_3_2_59_2","first-page":"1631","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP \u201913)","author":"Socher Richard","year":"2013","unstructured":"Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Y. Ng, and Christopher Potts. 2013. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP \u201913), A Meeting of SIGDAT, a Special Interest Group of the ACL. 1631\u20131642. Retrieved from https:\/\/aclanthology.org\/D13-1170\/"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1441"},{"key":"e_1_3_2_61_2","unstructured":"Wonyong Sung Sungho Shin and Kyuyeon Hwang. 2015. Resiliency of Deep Neural Networks under Quantization. arXiv:1511.06488. Retrieved from http:\/\/arxiv.org\/abs\/1511.06488"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401296"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acl-long.392"},{"key":"e_1_3_2_64_2","unstructured":"Iulia Turc Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2019. Well-Read Students Learn Better: The Impact of Student Initialization on Knowledge Distillation. arXiv:1908.08962. Retrieved from http:\/\/arxiv.org\/abs\/1908.08962"},{"key":"e_1_3_2_65_2","first-page":"5998","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 5998\u20136008. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html"},{"key":"e_1_3_2_66_2","volume-title":"Proceedings of the 7th International Conference on Learning Representations (ICLR \u201919)","author":"Wang Alex","year":"2019","unstructured":"Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In Proceedings of the 7th International Conference on Learning Representations (ICLR \u201919). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=rJ4km2R5t7"},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.686"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.findings-acl.188"},{"key":"e_1_3_2_69_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS \u201920)","author":"Wang Wenhui","year":"2020","unstructured":"Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020a. MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers. In Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS \u201920), Hugo Larochelle, Marc\u2019Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html"},{"key":"e_1_3_2_70_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00290"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/n18-1101"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.107"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.sustainlp-1.11"},{"key":"e_1_3_2_74_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.204"},{"key":"e_1_3_2_75_2","volume-title":"Proceedings of the 9th International Conference on Learning Representations (ICLR \u201921)","author":"Xiong Lee","year":"2021","unstructured":"Lee Xiong, Chenyan Xiong, Ye Li, Kwok-Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021. Approximate Nearest Neighbor Negative Contrastive Learning for Dense Text Retrieval. In Proceedings of the 9th International Conference on Learning Representations (ICLR \u201921). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=zeFrfgyZln"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2022.EMNLP-MAIN.258"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-17120-8_12"},{"key":"e_1_3_2_78_2","volume-title":"Proceedings of the 7th International Conference on Learning Representations (ICLR \u201919)","author":"Yu Jiahui","year":"2019","unstructured":"Jiahui Yu, Linjie Yang, Ning Xu, Jianchao Yang, and Thomas S. Huang. 2019. Slimmable Neural Networks. In Proceedings of the 7th International Conference on Learning Representations (ICLR \u201919). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=H1gMCsAqY7"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462880"},{"key":"e_1_3_2_80_2","unstructured":"Jingtao Zhan Jiaxin Mao Yiqun Liu Min Zhang and Shaoping Ma. 2020. RepBERT: Contextualized Text Embeddings for First-Stage Retrieval. arXiv:2006.15498. Retrieved from https:\/\/arxiv.org\/abs\/2006.15498"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","unstructured":"Chen Zhang Dawei Song Zheyu Ye and Yan Gao. 2023a. Towards the Law of Capacity Gap in Distilling Language Models. arXiv:2311.07052. Retrieved from 10.48550\/ARXIV.2311.07052","DOI":"10.48550\/ARXIV.2311.07052"},{"key":"e_1_3_2_82_2","first-page":"15629","volume-title":"Proceedings of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC\/COLING \u201924)","author":"Zhang Chen","year":"2024","unstructured":"Chen Zhang, Yang Yang, Qiuchi Li, Jingang Wang, and Dawei Song. 2024a. Task-Agnostic Distillation of Encoder-Decoder Language Models. In Proceedings of the Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC\/COLING \u201924), Nicoletta Calzolari, Min-Yen Kan, V\u00e9ronique Hoste, Alessandro Lenci, Sakriani Sakti, and Nianwen Xue (Eds.). ELRA and ICCL, 15629\u201315639. Retrieved from https:\/\/aclanthology.org\/2024.lrec-main.1359"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2023.ACL-LONG.249"},{"key":"e_1_3_2_84_2","first-page":"1378","volume-title":"Proceedings of the Findings of the Association for Computational Linguistics (EACL \u201924)","author":"Zhang Chen","year":"2024","unstructured":"Chen Zhang, Yang Yang, Qifan Wang, Jiahao Liu, Jingang Wang, Wei Wu, and Dawei Song. 2024b. Minimal Distillation Schedule for Extreme Language Model Compression. In Proceedings of the Findings of the Association for Computational Linguistics (EACL \u201924), Yvette Graham and Matthew Purver (Eds.). Association for Computational Linguistics, 1378\u20131394. Retrieved from https:\/\/aclanthology.org\/2024.findings-eacl.93"},{"key":"e_1_3_2_85_2","first-page":"18330","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS \u201920)","author":"Zhou Wangchunshu","year":"2020","unstructured":"Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian J. McAuley, Ke Xu, and Furu Wei. 2020. BERT Loses Patience: Fast and Robust Inference with Early Exit. In Proceedings of the Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems (NeurIPS \u201920), Hugo Larochelle, Marc\u2019Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (Eds.). 18330\u201318341. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2020\/hash\/d4dd111a4fd973394238aca5c05bebe3-Abstract.html"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.11"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.naacl-main.116"}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3677375","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3677375","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T20:04:22Z","timestamp":1750277062000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3677375"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,18]]},"references-count":86,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,11,30]]}},"alternative-id":["10.1145\/3677375"],"URL":"https:\/\/doi.org\/10.1145\/3677375","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,18]]},"assertion":[{"value":"2023-04-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-27","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}