{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,27]],"date-time":"2026-07-27T14:13:18Z","timestamp":1785161598326,"version":"3.55.0"},"reference-count":168,"publisher":"Association for Computing Machinery (ACM)","issue":"14","license":[{"start":{"date-parts":[[2026,7,27]],"date-time":"2026-07-27T00:00:00Z","timestamp":1785110400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/legalcode"}],"funder":[{"name":"Institute of Information & communications Technology Planning & Evaluatio"},{"name":"Flexible and Efficient Model Compression Method for Various Applications and Environments","award":["2020-0-00894"],"award-info":[{"award-number":["2020-0-00894"]}]},{"name":"Artificial Intelligence Graduate School Program","award":["2021-0-01343"],"award-info":[{"award-number":["2021-0-01343"]}]},{"name":"AI star Fellowship Support Progra","award":["RS-2025-25442338"],"award-info":[{"award-number":["RS-2025-25442338"]}]},{"name":"Global AI Frontier Lab","award":["RS-2024-00509257"],"award-info":[{"award-number":["RS-2024-00509257"]}]},{"name":"The Institute of Engineering Research at Seoul National University"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2026,10,31]]},"abstract":"<jats:p>How can we compress language models without sacrificing accuracy? The number of compression algorithms for language models is rapidly growing to leverage the remarkable advances of recent language models without side effects induced by their gigantic size, including increased carbon emissions, high latency, and restricted usage on resource-constrained mobile devices. While numerous compression algorithms have shown remarkable progress in compressing language models, it ironically becomes challenging to capture emerging trends and identify the fundamental concepts underlying them due to the excessive number of algorithms. In this article, we survey and summarize diverse compression algorithms including pruning, quantization, knowledge distillation, low-rank approximation, and dynamic inference. We not only summarize the overall trend of diverse compression algorithms but also select representative algorithms and provide in-depth analyses of them. Finally, we introduce and discuss promising future research directions.<\/jats:p>","DOI":"10.1145\/3819816","type":"journal-article","created":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T20:42:07Z","timestamp":1781815327000},"page":"1-35","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["A Comprehensive Survey of Compression Algorithms for Language Models"],"prefix":"10.1145","volume":"58","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-5315-7142","authenticated-orcid":false,"given":"Seungcheol","family":"Park","sequence":"first","affiliation":[{"name":"Computer Science and Engineering, Seoul National University","place":["Gwanak-gu, Korea (the Republic of)"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7214-5554","authenticated-orcid":false,"given":"Jaehyeon","family":"Choi","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, Seoul National University","place":["Gwanak-gu, Korea (the Republic of)"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-1884-6199","authenticated-orcid":false,"given":"Sojin","family":"Lee","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, Seoul National University","place":["Gwanak-gu, Korea (the Republic of)"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8774-6950","authenticated-orcid":false,"given":"U","family":"Kang","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, Seoul National University","place":["Gwanak-gu, Korea (the Republic of)"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,7,27]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"Abien Fred Agarap. 2018. Deep Learning using Rectified Linear Units (ReLU). arXiv:1803.08375 (2018)."},{"key":"e_1_3_1_3_2","volume-title":"AAAI","author":"Aguilar Gustavo","year":"2020","unstructured":"Gustavo Aguilar, Yuan Ling, Yu Zhang, Benjamin Yao, Xing Fan, and Chenlei Guo. 2020. Knowledge Distillation from Internal Representations. In AAAI."},{"key":"e_1_3_1_4_2","unstructured":"Lei Jimmy Ba Jamie Ryan Kiros and Geoffrey E. Hinton. 2016. Layer Normalization. arXiv:1607.06450 (2016)."},{"key":"e_1_3_1_5_2","volume-title":"NeurIPS","author":"Bai Haoli","year":"2022","unstructured":"Haoli Bai, Lu Hou, Lifeng Shang, Xin Jiang, Irwin King, and Michael R. Lyu. 2022. Towards efficient post-training quantization of pre-trained language models. In NeurIPS."},{"key":"e_1_3_1_6_2","volume-title":"IJCNLP","author":"Bai Haoli","year":"2021","unstructured":"Haoli Bai, Wei Zhang, Lu Hou, Lifeng Shang, Jin Jin, Xin Jiang, Qun Liu, Michael R. Lyu, and Irwin King. 2021. BinaryBERT: Pushing the limit of bert quantization. In IJCNLP."},{"key":"e_1_3_1_7_2","unstructured":"Kayhan Behdin Ayan Acharya Aman Gupta Sathiya Keerthi Selvaraj and Rahul Mazumder. 2023. QuantEase: Optimization-based quantization for language models - An efficient and intuitive algorithm. arXiv:2309.01885 (2023)."},{"key":"e_1_3_1_8_2","unstructured":"Yoshua Bengio Nicholas L\u00e9onard and Aaron C. Courville. 2013. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv:1308.3432 (2013)."},{"key":"e_1_3_1_9_2","unstructured":"Aishwarya Bhandare Vamsi Sripathi Deepthi Karkada Vivek Menon Sun Choi Kushal Datta and Vikram A. Saletore. 2019. Efficient 8-bit quantization of transformer neural machine language translation Model. arXiv:1906.00532 (2019)."},{"key":"e_1_3_1_10_2","volume-title":"NeurIPS","author":"Brown Tom B.","year":"2020","unstructured":"Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell et\u00a0al. 2020. Language models are few-shot learners. In NeurIPS."},{"key":"e_1_3_1_11_2","doi-asserted-by":"crossref","unstructured":"Daniel M. Cer Mona T. Diab Eneko Agirre I\u00f1igo Lopez-Gazpio and Lucia Specia. 2017. SemEval-2017 task 1: Semantic textual similarity - multilingual and cross-lingual focused evaluation. arXiv:1708.00055 (2017).","DOI":"10.18653\/v1\/S17-2001"},{"key":"e_1_3_1_12_2","unstructured":"Yuji Chai John Gkountouras Glenn G. Ko David Brooks and Gu-Yeon Wei. 2023. INT2.1: Towards fine-tunable quantized large language models with error correction through low-rank adaptation. arXiv:2306.08162 (2023)."},{"key":"e_1_3_1_13_2","volume-title":"NeurIPS","author":"Chee Jerry","year":"2023","unstructured":"Jerry Chee, Yaohui Cai, Volodymyr Kuleshov, and Christopher De Sa. 2023. QuIP: 2-bit quantization of large language models with guarantees. In NeurIPS."},{"key":"e_1_3_1_14_2","volume-title":"ICLR","author":"Chen Jiaao","year":"2023","unstructured":"Jiaao Chen, Aston Zhang, Xingjian Shi, Mu Li, Alex Smola, and Diyi Yang. 2023. Parameter-efficient fine-tuning design spaces. In ICLR."},{"key":"e_1_3_1_15_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Pond\u00e9 de Oliveira Pinto Jared Kaplan Harrison Edwards Yuri Burda Nicholas Joseph Greg Brockman et\u00a0al. 2021. Evaluating large language models trained on code. arXiv:2107.03374 (2021)."},{"key":"e_1_3_1_16_2","volume-title":"NeurIPS","author":"Chen Patrick H.","year":"2021","unstructured":"Patrick H. Chen, Hsiang-Fu Yu, Inderjit S. Dhillon, and Cho-Jui Hsieh. 2021. DRONE: Data-aware low-rank compression for large nlp models. In NeurIPS."},{"key":"e_1_3_1_17_2","volume-title":"ICML","author":"Chen Ting","year":"2020","unstructured":"Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey E. Hinton. 2020. A simple framework for contrastive learning of visual representations. In ICML."},{"key":"e_1_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1053"},{"issue":"7","key":"e_1_3_1_19_2","first-page":"5113","article-title":"A comprehensive survey on model compression and acceleration","volume":"53","author":"Choudhary Tejalal","year":"2020","unstructured":"Tejalal Choudhary, Vipul Kumar Mishra, Anurag Goswami, and Jagannathan Sarangapani. 2020. A comprehensive survey on model compression and acceleration. AIR 53, 7 (2020), 5113\u20135155.","journal-title":"AIR"},{"key":"e_1_3_1_20_2","volume-title":"ICLR","author":"Clark Kevin","year":"2020","unstructured":"Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. ELECTRA: Pre-training text encoders as discriminators rather than generators. In ICLR."},{"key":"e_1_3_1_21_2","unstructured":"Karl Cobbe Vineet Kosaraju Mohammad Bavarian Mark Chen Heewoo Jun Lukasz Kaiser Matthias Plappert Jerry Tworek Jacob Hilton Reiichiro Nakano Christopher Hesse and John Schulman. 2021. Training verifiers to solve math word problems. arXiv:2110.14168 (2021)."},{"key":"e_1_3_1_22_2","unstructured":"Jean-Baptiste Cordonnier Andreas Loukas and Martin Jaggi. 2020. Multi-head attention: Collaborate instead of concatenate. arXiv:2006.16362 (2020)."},{"key":"e_1_3_1_23_2","volume-title":"EMNLP","author":"Dalvi Fahim","year":"2020","unstructured":"Fahim Dalvi, Hassan Sajjad, Nadir Durrani, and Yonatan Belinkov. 2020. Analyzing redundancy in pretrained transformer models. In EMNLP."},{"key":"e_1_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/JPROC.2020.2976475"},{"key":"e_1_3_1_25_2","volume-title":"NeurIPS","author":"Dettmers Tim","year":"2022","unstructured":"Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. 2022. GPT3.int8(): 8-bit matrix multiplication for transformers at scale. In NeurIPS."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.52202\/075280-0441"},{"key":"e_1_3_1_27_2","unstructured":"Tim Dettmers Ruslan Svirschevski Vage Egiazarian Denis Kuznedelev Elias Frantar Saleh Ashkboos Alexander Borzunov Torsten Hoefler and Dan Alistarh. 2023. SpQR: A sparse-quantized representation for near-lossless llm weight compression. arXiv:2306.03078 (2023)."},{"key":"e_1_3_1_28_2","volume-title":"NAACL-HLT","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT."},{"key":"e_1_3_1_29_2","volume-title":"IWP","author":"Dolan William B.","year":"2005","unstructured":"William B. Dolan and Chris Brockett. 2005. Automatically constructing a corpus of sentential paraphrases. In IWP."},{"key":"e_1_3_1_30_2","volume-title":"ICLR","author":"Frankle Jonathan","year":"2019","unstructured":"Jonathan Frankle and Michael Carbin. 2019. The lottery ticket hypothesis: Finding sparse, trainable neural networks. In ICLR."},{"key":"e_1_3_1_31_2","volume-title":"ICML","author":"Frantar Elias","year":"2023","unstructured":"Elias Frantar and Dan Alistarh. 2023. SparseGPT: Massive language models can be accurately pruned in one-shot. In ICML."},{"key":"e_1_3_1_32_2","unstructured":"Elias Frantar Saleh Ashkboos Torsten Hoefler and Dan Alistarh. 2022. GPTQ: Accurate post-training quantization for generative pre-trained transformers. arXiv:2210.17323."},{"key":"e_1_3_1_33_2","volume-title":"ICLR","author":"Frantar Elias","year":"2023","unstructured":"Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. 2023. OPTQ: Accurate quantization for generative pre-trained transformers. In ICLR."},{"key":"e_1_3_1_34_2","volume-title":"AAAI","author":"Fu Hao","year":"2021","unstructured":"Hao Fu, Shaojun Zhou, Qihong Yang, Junjie Tang, Guiquan Liu, Kaikui Liu, and Xiaolong Li. 2021. LRC-BERT: Latent-representation contrastive knowledge distillation for natural language understanding. In AAAI."},{"key":"e_1_3_1_35_2","doi-asserted-by":"crossref","first-page":"1061","DOI":"10.1162\/tacl_a_00413","article-title":"Compressing large-scale transformer-based models: A case study on bert","volume":"9","author":"Ganesh Prakhar","year":"2021","unstructured":"Prakhar Ganesh, Yao Chen, Xin Lou, Mohammad Ali Khan, Yin Yang, Hassan Sajjad, Preslav Nakov, Deming Chen, and Marianne Winslett. 2021. Compressing large-scale transformer-based models: A case study on bert. TACL 9 (2021), 1061\u20131080.","journal-title":"TACL"},{"key":"e_1_3_1_36_2","doi-asserted-by":"crossref","unstructured":"Amir Gholami Sehoon Kim Zhen Dong Zhewei Yao Michael W. Mahoney and Kurt Keutzer. 2021. A survey of quantization methods for efficient neural network inference. arXiv:2103.13630.","DOI":"10.1201\/9781003162810-13"},{"key":"e_1_3_1_37_2","volume-title":"ACL 2023","author":"Gong Zhuocheng","year":"2023","unstructured":"Zhuocheng Gong, Jiahao Liu, Qifan Wang, Yang Yang, Jingang Wang, Wei Wu, Yunsen Xian, Dongyan Zhao, and Rui Yan. 2023. PreQuant: A task-agnostic quantization approach for pre-trained language models. In ACL 2023."},{"key":"e_1_3_1_38_2","series-title":"PMLR","volume-title":"ICML","author":"Goyal Saurabh","year":"2020","unstructured":"Saurabh Goyal, Anamitra Roy Choudhury, Saurabh Raje, Venkatesan T. Chakaravarthy, Yogish Sabharwal, and Ashish Verma. 2020. PoWER-BERT: Accelerating bert inference via progressive word-vector elimination. In ICML(PMLR, Vol. 119)."},{"key":"e_1_3_1_39_2","volume-title":"ICLR","author":"Gu Yuxian","year":"2024","unstructured":"Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. MiniLLM: Knowledge distillation of large language models. In ICLR."},{"key":"e_1_3_1_40_2","volume-title":"ICML","author":"Guo Jialong","year":"2025","unstructured":"Jialong Guo, Xinghao Chen, Yehui Tang, and Yunhe Wang. 2025. SlimLLM: Accurate structured pruning for large language models. In ICML."},{"key":"e_1_3_1_41_2","doi-asserted-by":"publisher","DOI":"10.1145\/3487045"},{"key":"e_1_3_1_42_2","volume-title":"NAACL","author":"Haidar Md. Akmal","year":"2022","unstructured":"Md. Akmal Haidar, Nithin Anchuri, Mehdi Rezagholizadeh, Abbas Ghaddar, Philippe Langlais, and Pascal Poupart. 2022. RAIL-KD: RAndom intermediate layer mapping for knowledge distillation. In NAACL."},{"key":"e_1_3_1_43_2","volume-title":"NIPS","author":"Han Song","year":"2015","unstructured":"Song Han, Jeff Pool, John Tran, and William J. Dally. 2015. Learning both weights and connections for efficient neural networks. In NIPS."},{"key":"e_1_3_1_44_2","volume-title":"NeurIPS","author":"Hassibi Babak","year":"1992","unstructured":"Babak Hassibi and David G. Stork. 1992. Second order derivatives for network pruning: Optimal brain surgeon. In NeurIPS."},{"key":"e_1_3_1_45_2","volume-title":"NeurIPS Datasets and Benchmarks","author":"Hendrycks Dan","year":"2021","unstructured":"Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring mathematical problem solving with the math dataset. In NeurIPS Datasets and Benchmarks."},{"key":"e_1_3_1_46_2","unstructured":"Dan Hendrycks and Kevin Gimpel. 2016. Bridging nonlinearities and stochastic regularizers with gaussian error linear units. arXiv:1606.08415 (2016)."},{"issue":"241","key":"e_1_3_1_47_2","first-page":"1","article-title":"Sparsity in deep learning: pruning and growth for efficient inference and training in neural networks","volume":"22","author":"Hoefler Torsten","year":"2021","unstructured":"Torsten Hoefler, Dan Alistarh, Tal Ben-Nun, Nikoli Dryden, and Alexandra Peste. 2021. Sparsity in deep learning: pruning and growth for efficient inference and training in neural networks. JMLR 22, 241 (2021), 1\u2013124.","journal-title":"JMLR"},{"key":"e_1_3_1_48_2","volume-title":"NeurIPS","author":"Hou Lu","year":"2020","unstructured":"Lu Hou, Zhiqi Huang, Lifeng Shang, Xin Jiang, Xiao Chen, and Qun Liu. 2020. DynaBERT: Dynamic bert with adaptive width and depth. In NeurIPS."},{"key":"e_1_3_1_49_2","volume-title":"ICML","author":"Houlsby Neil","year":"2019","unstructured":"Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. 2019. Parameter-efficient transfer learning for nlp. In ICML."},{"key":"e_1_3_1_50_2","volume-title":"ICLR","author":"Hsu Yen-Chang","year":"2022","unstructured":"Yen-Chang Hsu, Ting Hua, Sungen Chang, Qian Lou, Yilin Shen, and Hongxia Jin. 2022. Language model compression with weighted low-rank factorization. In ICLR."},{"key":"e_1_3_1_51_2","volume-title":"ICLR","author":"Hu Edward J.","year":"2022","unstructured":"Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. LoRA: Low-rank adaptation of large language models. In ICLR."},{"key":"e_1_3_1_52_2","volume-title":"ICML","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In ICML."},{"key":"e_1_3_1_53_2","volume-title":"CVPR","author":"Jeon Yongkweon","year":"2022","unstructured":"Yongkweon Jeon, Chungman Lee, Eulrang Cho, and Yeonju Ro. 2022. Mr.BiQ: Post-training non-uniform quantization based on minimizing the reconstruction error. In CVPR."},{"key":"e_1_3_1_54_2","unstructured":"Ananya Harsh Jha Dirk Groeneveld Emma Strubell and Iz Beltagy. 2023. How to train your (compressed) large language model. arXiv:2305.14864v2 (2023)."},{"key":"e_1_3_1_55_2","volume-title":"IJCNLP","author":"Ji Tianchu","year":"2021","unstructured":"Tianchu Ji, Shraddhan Jain, Michael Ferdman, Peter A. Milder, H. Andrew Schwartz, and Niranjan Balasubramanian. 2021. On the distribution, sparsity, and inference-time quantization of attention values in transformers. In IJCNLP."},{"key":"e_1_3_1_56_2","volume-title":"EMNLP (ACL)","author":"Jiao Xiaoqi","year":"2020","unstructured":"Xiaoqi Jiao, Yichun Yin, Lifeng Shang, Xin Jiang, Xiao Chen, Linlin Li, Fang Wang, and Qun Liu. 2020. TinyBERT: Distilling bert for natural language understanding. In EMNLP (ACL)."},{"key":"e_1_3_1_57_2","unstructured":"Jing Jin Cai Liang Tiancheng Wu Liqin Zou and Zhiliang Gan. 2021. KDLSQ-BERT: A quantized bert combining knowledge distillation with learned step size quantization. arXiv:2101.05938 (2021)."},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P17-1147"},{"key":"e_1_3_1_59_2","volume-title":"NeurIPS","author":"Kim Jeonghoon","year":"2023","unstructured":"Jeonghoon Kim, Jung Hyun Lee, Sungdong Kim, Joonsuk Park, Kang Min Yoo, Se Jung Kwon, and Dongsoo Lee. 2023. Memory-efficient fine-tuning of compressed large language models via sub-4-bit integer quantization. In NeurIPS."},{"key":"e_1_3_1_60_2","volume-title":"EMNLP","author":"Kim Junho","year":"2022","unstructured":"Junho Kim, Jun-Hyung Park, Mingyu Lee, Wing-Lam Mok, Joon-Young Choi, and SangKeun Lee. 2022. Tutoring helps students learn better: Improving knowledge distillation for bert with tutor network. In EMNLP."},{"key":"e_1_3_1_61_2","volume-title":"ICLR","author":"Kim Minjun","year":"2026","unstructured":"Minjun Kim, Jaehyeon Choi, Hyunwoo Yang, Jongjin Kim, Jinho Song, and U Kang. 2026. Prune-then-quantize or quantize-then-prune? Understanding the impact of compression order in joint model compression. In ICLR."},{"key":"e_1_3_1_62_2","series-title":"PMLR","volume-title":"ICML","author":"Kim Sehoon","year":"2021","unstructured":"Sehoon Kim, Amir Gholami, Zhewei Yao, Michael W. Mahoney, and Kurt Keutzer. 2021. I-BERT: Integer-only bert quantization. In ICML(PMLR, Vol. 139)."},{"key":"e_1_3_1_63_2","volume-title":"ICML","author":"Kim Sehoon","year":"2024","unstructured":"Sehoon Kim, Coleman Hooper, Amir Gholami, Zhen Dong, Xiuyu Li, Sheng Shen, Michael W. Mahoney, and Kurt Keutzer. 2024. SqueezeLLM: Dense-and-sparse quantization. In ICML."},{"key":"e_1_3_1_64_2","unstructured":"Sehoon Kim Coleman Hooper Thanakul Wattanawong Minwoo Kang Ruohan Yan Hasan Genc Grace Dinh Qijing Huang Kurt Keutzer Michael W. Mahoney Yakun Sophia Shao and Amir Gholami. 2023. Full stack optimization of transformer inference: A survey. arXiv:2302.14017."},{"key":"e_1_3_1_65_2","volume-title":"KDD","author":"Kim Sehoon","year":"2022","unstructured":"Sehoon Kim, Sheng Shen, David Thorsley, Amir Gholami, Woosuk Kwon, Joseph Hassoun, and Kurt Keutzer. 2022. Learned token pruning for transformers. In KDD."},{"key":"e_1_3_1_66_2","unstructured":"Young Jin Kim Rawn Henry Raffy Fahim and Hany Hassan Awadalla. 2023. FineQuant: Unlocking efficiency with fine-grained weight-only quantization for llms. arXiv:2308.09723."},{"key":"e_1_3_1_67_2","volume-title":"ICML","author":"Ko Jongwoo","year":"2024","unstructured":"Jongwoo Ko, Sungnyun Kim, Tianyi Chen, and Se-Young Yun. 2024. DistiLLM: Towards streamlined distillation for large language models. In ICML."},{"key":"e_1_3_1_68_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.279"},{"key":"e_1_3_1_69_2","volume-title":"NeurIPS","author":"Kurtic Eldar","year":"2023","unstructured":"Eldar Kurtic, Elias Frantar, and Dan Alistarh. 2023. ZipLM: Hardware-aware structured pruning of language models. In NeurIPS."},{"key":"e_1_3_1_70_2","volume-title":"NeurIPS","author":"Kuzmin Andrey","year":"2022","unstructured":"Andrey Kuzmin, Mart van Baalen, Yuwei Ren, Markus Nagel, Jorn Peters, and Tijmen Blankevoort. 2022. FP8 quantization: The power of the exponent. In NeurIPS."},{"key":"e_1_3_1_71_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00276"},{"key":"e_1_3_1_72_2","volume-title":"EMNLP","author":"Kwon Se Jung","year":"2022","unstructured":"Se Jung Kwon, Jeonghoon Kim, Jeongin Bae, Kang Min Yoo, Jin-Hwa Kim, Baeseong Park, Byeongwook Kim, Jung-Woo Ha, Nako Sung, and Dongsoo Lee. 2022. AlphaTuning: Quantization-aware parameter-efficient adaptation of large-scale pre-trained language models. In EMNLP."},{"key":"e_1_3_1_73_2","volume-title":"NeurIPS","author":"Kwon Woosuk","year":"2022","unstructured":"Woosuk Kwon, Sehoon Kim, Michael W. Mahoney, Joseph Hassoun, Kurt Keutzer, and Amir Gholami. 2022. A fast post-training pruning framework for transformers. In NeurIPS."},{"key":"e_1_3_1_74_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.829"},{"key":"e_1_3_1_75_2","volume-title":"ICLR","author":"Lan Zhenzhong","year":"2020","unstructured":"Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite bert for self-supervised learning of language representations. In ICLR."},{"key":"e_1_3_1_76_2","volume-title":"NeurIPS","author":"LeCun Yann","year":"1989","unstructured":"Yann LeCun, John S. Denker, and Sara A. Solla. 1989. Optimal brain damage. In NeurIPS."},{"key":"e_1_3_1_77_2","volume-title":"CVPR","author":"Lee Byung-Kwan","year":"2022","unstructured":"Byung-Kwan Lee, Junho Kim, and Yong Man Ro. 2022. Masking adversarial damage: Finding adversarial saliency for robust and sparse network. In CVPR."},{"key":"e_1_3_1_78_2","volume-title":"AAAI","author":"Lee Changhun","year":"2024","unstructured":"Changhun Lee, Jungyu Jin, Taesu Kim, Hyungjun Kim, and Eunhyeok Park. 2024. OWQ: Lessons learned from activation outliers for weight quantization in large language models. In AAAI."},{"key":"e_1_3_1_79_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0253241"},{"key":"e_1_3_1_80_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.242"},{"key":"e_1_3_1_81_2","volume-title":"AAAI","author":"Li Liang","year":"2024","unstructured":"Liang Li, Qingyuan Li, Bo Zhang, and Xiangxiang Chu. 2024. Norm tweaking: High-performance low-bit quantization of large language models. In AAAI."},{"key":"e_1_3_1_82_2","volume-title":"ACL\/IJCNLP","author":"Li Xiang Lisa","year":"2021","unstructured":"Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In ACL\/IJCNLP."},{"key":"e_1_3_1_83_2","unstructured":"Yixiao Li Yifan Yu Chen Liang Pengcheng He Nikos Karampatziakis Weizhu Chen and Tuo Zhao. 2023. LoftQ: LoRA-fine-tuning-aware quantization for large language models. arXiv:2310.08659."},{"key":"e_1_3_1_84_2","volume-title":"ICML","author":"Liang Chen","year":"2023","unstructured":"Chen Liang, Simiao Zuo, Qingru Zhang, Pengcheng He, Weizhu Chen, and Tuo Zhao. 2023. Less is more: Task-aware layer-wise distillation for language model compression. In ICML."},{"key":"e_1_3_1_85_2","volume-title":"ICLR","author":"Liang Kevin J.","year":"2021","unstructured":"Kevin J. Liang, Weituo Hao, Dinghan Shen, Yufan Zhou, Weizhu Chen, Changyou Chen, and Lawrence Carin. 2021. MixKD: Towards efficient distillation of large-scale language models. In ICLR."},{"key":"e_1_3_1_86_2","volume-title":"NAACL-HLT","author":"Liao Kaiyuan","year":"2021","unstructured":"Kaiyuan Liao, Yi Zhang, Xuancheng Ren, Qi Su, Xu Sun, and Bin He. 2021. A global past-future early exit method for accelerating inference of pre-trained language models. In NAACL-HLT."},{"key":"e_1_3_1_87_2","volume-title":"MLSys","author":"Lin Ji","year":"2024","unstructured":"Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Xingyu Dang, and Song Han. 2024. AWQ: Activation-aware weight quantization for llm compression and acceleration. In MLSys."},{"key":"e_1_3_1_88_2","volume-title":"EMNLP","author":"Lin Zi","year":"2020","unstructured":"Zi Lin, Jeremiah Z. Liu, Zi Yang, Nan Hua, and Dan Roth. 2020. Pruning redundant mappings in transformer models via spectral-normalized identity prior. In EMNLP."},{"key":"e_1_3_1_89_2","volume-title":"ICLR","author":"Liu James","year":"2025","unstructured":"James Liu, Pragaash Ponnusamy, Tianle Cai, Han Guo, Yoon Kim, and Ben Athiwaratkun. 2025. Training-free activation sparsity in large language models. In ICLR."},{"key":"e_1_3_1_90_2","volume-title":"NeurIPS","author":"Liu Jiaheng","year":"2024","unstructured":"Jiaheng Liu, Chenchen Zhang, Jinyang Guo, Yuanxing Zhang, Haoran Que, Ken Deng, Zhiqi Bai, Jie Liu, Ge Zhang, Jiakai Wang, Yanan Wu, Congnan Liu, Jiamang Wang, Lin Qu, Wenbo Su, and Bo Zheng. 2024. DDK: Distilling domain knowledge for efficient large language models. In NeurIPS."},{"key":"e_1_3_1_91_2","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized bert pretraining approach. arXiv:1907.11692."},{"key":"e_1_3_1_92_2","volume-title":"CVPR","author":"Liu Zechun","year":"2022","unstructured":"Zechun Liu, Kwang-Ting Cheng, Dong Huang, Eric P. Xing, and Zhiqiang Shen. 2022. Nonuniform-to-uniform quantization: Towards accurate quantization via generalized straight-through estimation. In CVPR."},{"key":"e_1_3_1_93_2","volume-title":"NeurIPS","author":"Liu Zechun","year":"2022","unstructured":"Zechun Liu, Barlas Oguz, Aasish Pappu, Lin Xiao, Scott Yih, Meng Li, Raghuraman Krishnamoorthi, and Yashar Mehdad. 2022. BiT: Robustly binarized multi-distilled transformer. In NeurIPS."},{"key":"e_1_3_1_94_2","volume-title":"Findings of ACL","author":"Liu Zechun","year":"2024","unstructured":"Zechun Liu, Barlas Oguz, Changsheng Zhao, Ernie Chang, Pierre Stock, Yashar Mehdad, Yangyang Shi, Raghuraman Krishnamoorthi, and Vikas Chandra. 2024. LLM-QAT: Data-free quantization aware training for large language models. In Findings of ACL."},{"key":"e_1_3_1_95_2","volume-title":"ICML","author":"Liu Zichang","year":"2023","unstructured":"Zichang Liu, Jue Wang, Tri Dao, Tianyi Zhou, Binhang Yuan, Zhao Song, Anshumali Shrivastava, Ce Zhang, Yuandong Tian, Christopher R\u00e9, and Beidi Chen. 2023. Deja Vu: Contextual sparsity for efficient llms at inference time. In ICML."},{"key":"e_1_3_1_96_2","volume-title":"ICLR","author":"Louizos Christos","year":"2018","unstructured":"Christos Louizos, Max Welling, and Diederik P. Kingma. 2018. Learning sparse neural networks through l \\({}_{\\mbox{0}}\\) regularization. In ICLR."},{"key":"e_1_3_1_97_2","volume-title":"NeurIPS","author":"Ma Xinyin","year":"2023","unstructured":"Xinyin Ma, Gongfan Fang, and Xinchao Wang. 2023. LLM-pruner: On the structural pruning of large language models. In NeurIPS."},{"key":"e_1_3_1_98_2","volume-title":"ACL","author":"Mao Yuning","year":"2022","unstructured":"Yuning Mao, Lambert Mathias, Rui Hou, Amjad Almahairi, Hao Ma, Jiawei Han, Scott Yih, and Madian Khabsa. 2022. UniPELT: A unified framework for parameter-efficient language model tuning. In ACL."},{"key":"e_1_3_1_99_2","volume-title":"NeurIPS","author":"Michel Paul","year":"2019","unstructured":"Paul Michel, Omer Levy, and Graham Neubig. 2019. Are sixteen heads really better than one?. In NeurIPS."},{"key":"e_1_3_1_100_2","unstructured":"Paulius Micikevicius Dusan Stosic Neil Burgess Marius Cornea Pradeep Dubey Richard Grisenthwaite Sangwon Ha Alexander Heinecke Patrick Judd John Kamalu et\u00a0al. 2022. FP8 formats for deep learning. arXiv:2209.05433."},{"key":"e_1_3_1_101_2","volume-title":"ICLR","author":"Mirzadeh Seyed-Iman","year":"2024","unstructured":"Seyed-Iman Mirzadeh, Keivan Alizadeh-Vahid, Sachin Mehta, Carlo C. del Mundo, Oncel Tuzel, Golnoosh Samei, Mohammad Rastegari, and Mehrdad Farajtabar. 2024. ReLU strikes back: Exploiting activation sparsity in large language models. In ICLR."},{"key":"e_1_3_1_102_2","volume-title":"IJCNLP","author":"Noach Matan Ben","year":"2020","unstructured":"Matan Ben Noach and Yoav Goldberg. 2020. Compressing pre-trained language models by matrix decomposition. In IJCNLP."},{"key":"e_1_3_1_103_2","series-title":"PMLR","volume-title":"ICML","author":"Nova Azade","year":"2023","unstructured":"Azade Nova, Hanjun Dai, and Dale Schuurmans. 2023. Gradient-free structured pruning with unlabeled data. In ICML(PMLR, Vol. 202)."},{"key":"e_1_3_1_104_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1244"},{"key":"e_1_3_1_105_2","volume-title":"ACL","author":"Park Seungcheol","year":"2025","unstructured":"Seungcheol Park, Jeongin Bae, Beomseok Kwon, Minjun Kim, Byeongwook Kim, Se Jung Kwon, U Kang, and Dongsoo Lee. 2025. Unifying uniform and binary-coding quantization for accurate compression of large language models. In ACL."},{"key":"e_1_3_1_106_2","volume-title":"ICLR","author":"Park Seungcheol","year":"2024","unstructured":"Seungcheol Park, Hojun Choi, and U Kang. 2024. Accurate retraining-free pruning for pretrained encoder-based language models. In ICLR."},{"key":"e_1_3_1_107_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2025\/913"},{"key":"e_1_3_1_108_2","volume-title":"AAAI","author":"Passban Peyman","year":"2021","unstructured":"Peyman Passban, Yimeng Wu, Mehdi Rezagholizadeh, and Qun Liu. 2021. ALP-KD: Attention-based layer projection for knowledge distillation. In AAAI."},{"key":"e_1_3_1_109_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0265621"},{"key":"e_1_3_1_110_2","volume-title":"ICLR","author":"Qin Haotong","year":"2022","unstructured":"Haotong Qin, Yifu Ding, Mingyuan Zhang, Qinghua Yan, Aishan Liu, Qingqing Dang, Ziwei Liu, and Xianglong Liu. 2022. BiBERT: Accurate fully binarized bert. In ICLR."},{"issue":"140","key":"e_1_3_1_111_2","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. JMLR 21, 140 (2020), 1\u201367.","journal-title":"JMLR"},{"key":"e_1_3_1_112_2","volume-title":"ACL","author":"Rajpurkar Pranav","year":"2018","unstructured":"Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don\u2019t know: Unanswerable questions for squad. In ACL."},{"key":"e_1_3_1_113_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1264"},{"key":"e_1_3_1_114_2","unstructured":"David Raposo Samuel Ritter Blake A. Richards Timothy P. Lillicrap Peter Conway Humphreys and Adam Santoro. 2024. Mixture-of-depths: Dynamically allocating compute in transformer-based language models. arXiv:2404.02258."},{"key":"e_1_3_1_115_2","first-page":"101429","article-title":"On the effect of dropping layers of pre-trained transformer models","volume":"77","author":"Sajjad Hassan","year":"2023","unstructured":"Hassan Sajjad, Fahim Dalvi, Nadir Durrani, and Preslav Nakov. 2023. On the effect of dropping layers of pre-trained transformer models. CSL 77 (2023), 101429.","journal-title":"CSL"},{"key":"e_1_3_1_116_2","unstructured":"Victor Sanh Lysandre Debut Julien Chaumond and Thomas Wolf. 2019. DistilBERT a distilled version of bert: smaller faster cheaper and lighter. arXiv:1910.01108."},{"key":"e_1_3_1_117_2","volume-title":"NeurIPS","author":"Sanh Victor","year":"2020","unstructured":"Victor Sanh, Thomas Wolf, and Alexander M. Rush. 2020. Movement pruning: Adaptive sparsity by fine-tuning. In NeurIPS."},{"key":"e_1_3_1_118_2","volume-title":"NeurIPS","author":"Savarese Pedro","year":"2020","unstructured":"Pedro Savarese, Hugo Silva, and Michael Maire. 2020. Winning the lottery with continuous sparsification. In NeurIPS."},{"key":"e_1_3_1_119_2","unstructured":"Teven Le Scao Angela Fan Christopher Akiki Ellie Pavlick Suzana Ilic Daniel Hesslow Roman Castagn\u00e9 Alexandra Sasha Luccioni Fran\u00e7ois Yvon Matthias Gall\u00e9 et\u00a0al. 2022. BLOOM: A 176B-parameter open-access multilingual language model. arXiv:2211.05100."},{"key":"e_1_3_1_120_2","volume-title":"NeurIPS","author":"Schuster Tal","year":"2022","unstructured":"Tal Schuster, Adam Fisch, Jai Gupta, Mostafa Dehghani, Dara Bahri, Vinh Tran, Yi Tay, and Donald Metzler. 2022. Confident adaptive language modeling. In NeurIPS."},{"key":"e_1_3_1_121_2","volume-title":"ICLR","author":"Shao Wenqi","year":"2024","unstructured":"Wenqi Shao, Mengzhao Chen, Zhaoyang Zhang, Peng Xu, Lirui Zhao, Zhiqian Li, Kaipeng Zhang, Peng Gao, Yu Qiao, and Ping Luo. 2024. OmniQuant: Omnidirectionally calibrated quantization for large language models. In ICLR."},{"key":"e_1_3_1_122_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6409"},{"key":"e_1_3_1_123_2","volume-title":"DATE","author":"Shin Jiho","year":"2025","unstructured":"Jiho Shin, Hoeseok Yang, and Youngmin Yi. 2025. SparseInfer: Training-free prediction of activation sparsity for fast llm inference. In DATE."},{"key":"e_1_3_1_124_2","volume-title":"COLING","author":"Song Chenyang","year":"2025","unstructured":"Chenyang Song, Xu Han, Zhengyan Zhang, Shengding Hu, Xiyu Shi, Kuai Li, Chen Chen, Zhiyuan Liu, Guangli Li, Tao Yang, and Maosong Sun. 2025. ProSparse: Introducing and enhancing intrinsic activation sparsity within large language models. In COLING."},{"key":"e_1_3_1_125_2","volume-title":"ICLR","author":"Stock Pierre","year":"2021","unstructured":"Pierre Stock, Angela Fan, Benjamin Graham, Edouard Grave, R\u00e9mi Gribonval, Herv\u00e9 J\u00e9gou, and Armand Joulin. 2021. Training with quantization noise for extreme model compression. In ICLR."},{"key":"e_1_3_1_126_2","volume-title":"ICLR","author":"Sun Mingjie","year":"2024","unstructured":"Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. 2024. A simple and effective pruning approach for large language models. In ICLR."},{"key":"e_1_3_1_127_2","volume-title":"IJCNLP","author":"Sun Siqi","year":"2019","unstructured":"Siqi Sun, Yu Cheng, Zhe Gan, and Jingjing Liu. 2019. Patient knowledge distillation for bert model compression. In IJCNLP."},{"key":"e_1_3_1_128_2","unstructured":"Hanlin Tang Xipeng Zhang Kai Liu Jianchen Zhu and Zhanhui Kang. 2022. MKQ-BERT: Quantized bert with 4-bits weights and activations. arXiv:2203.13483."},{"key":"e_1_3_1_129_2","volume-title":"ACL","author":"Tao Chaofan","year":"2023","unstructured":"Chaofan Tao, Lu Hou, Haoli Bai, Jiansheng Wei, Xin Jiang, Qun Liu, Ping Luo, and Ngai Wong. 2023. Structured pruning for efficient generative pre-trained language models. In ACL."},{"key":"e_1_3_1_130_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.331"},{"key":"e_1_3_1_131_2","unstructured":"Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux Timoth\u00e9e Lacroix Baptiste Rozi\u00e8re Naman Goyal Eric Hambro Faisal Azhar et\u00a0al. 2023. LLaMA: Open and efficient foundation language models. arXiv:2302.13971."},{"key":"e_1_3_1_132_2","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale et\u00a0al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288."},{"key":"e_1_3_1_133_2","volume-title":"EACL","author":"Valipour Mojtaba","year":"2023","unstructured":"Mojtaba Valipour, Mehdi Rezagholizadeh, Ivan Kobyzev, and Ali Ghodsi. 2023. DyLoRA: Parameter-efficient tuning of pre-trained models using dynamic search-free low-rank adaptation. In EACL."},{"key":"e_1_3_1_134_2","volume-title":"NeurIPS","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NeurIPS."},{"key":"e_1_3_1_135_2","volume-title":"ICLR","author":"Wang Alex","year":"2019","unstructured":"Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R. Bowman. 2019. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In ICLR."},{"key":"e_1_3_1_136_2","volume-title":"ICLR","author":"Wang Benyou","year":"2022","unstructured":"Benyou Wang, Yuxin Ren, Lifeng Shang, Xin Jiang, and Qun Liu. 2022. Exploring extreme parameter compression for pre-trained language models. In ICLR."},{"key":"e_1_3_1_137_2","unstructured":"Sinong Wang Belinda Z. Li Madian Khabsa Han Fang and Hao Ma. 2020. Linformer: Self-attention with linear complexity. arXiv:2006.04768."},{"key":"e_1_3_1_138_2","volume-title":"IJCNLP (ACL)","author":"Wang Wenhui","year":"2021","unstructured":"Wenhui Wang, Hangbo Bao, Shaohan Huang, Li Dong, and Furu Wei. 2021. MiniLMv2: Multi-head self-attention relation distillation for compressing pretrained transformers. In IJCNLP (ACL)."},{"key":"e_1_3_1_139_2","volume-title":"NeurIPS","author":"Wang Wenhui","year":"2020","unstructured":"Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. MiniLM: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. In NeurIPS."},{"key":"e_1_3_1_140_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.496"},{"key":"e_1_3_1_141_2","volume-title":"NeurIPS","author":"Wei Xiuying","year":"2022","unstructured":"Xiuying Wei, Yunchen Zhang, Xiangguo Zhang, Ruihao Gong, Shanghang Zhang, Qi Zhang, Fengwei Yu, and Xianglong Liu. 2022. Outlier suppression: Pushing the limit of low-bit transformer language models. In NeurIPS."},{"key":"e_1_3_1_142_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1101"},{"key":"e_1_3_1_143_2","volume-title":"IJCNLP (ACL)","author":"Wu Chuhan","year":"2021","unstructured":"Chuhan Wu, Fangzhao Wu, and Yongfeng Huang. 2021. One teacher is enough? pre-trained language model distillation from multiple teachers. In IJCNLP (ACL)."},{"key":"e_1_3_1_144_2","unstructured":"Hao Wu Patrick Judd Xiaojie Zhang Mikhail Isaev and Paulius Micikevicius. 2020. Integer quantization for deep learning inference: Principles and empirical evaluation. arXiv:2004.09602."},{"key":"e_1_3_1_145_2","volume-title":"ICML (PMLR)","author":"Wu Xiaoxia","year":"2023","unstructured":"Xiaoxia Wu, Cheng Li, Reza Yazdani Aminabadi, Zhewei Yao, and Yuxiong He. 2023. Understanding int4 quantization for transformer models: Latency speedup, composability, and failure cases. In ICML (PMLR)."},{"key":"e_1_3_1_146_2","unstructured":"Xiaoxia Wu Zhewei Yao and Yuxiong He. 2023. ZeroQuant-fp: A leap forward in llms post-training w4a8 quantization using floating-point formats. arXiv:2307.09782."},{"key":"e_1_3_1_147_2","volume-title":"EMNLP","author":"Wu Yimeng","year":"2020","unstructured":"Yimeng Wu, Peyman Passban, Mehdi Rezagholizadeh, and Qun Liu. 2020. Why skip if you can combine: A simple knowledge distillation technique for intermediate layers. In EMNLP."},{"key":"e_1_3_1_148_2","volume-title":"EMNLP","author":"Wu Yimeng","year":"2021","unstructured":"Yimeng Wu, Mehdi Rezagholizadeh, Abbas Ghaddar, Md. Akmal Haidar, and Ali Ghodsi. 2021. Universal-kd: Attention-based output-grounded intermediate layer knowledge distillation. In EMNLP."},{"key":"e_1_3_1_149_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.107"},{"key":"e_1_3_1_150_2","series-title":"PMLR","volume-title":"ICML","author":"Xiao Guangxuan","year":"2023","unstructured":"Guangxuan Xiao, Ji Lin, Micka\u00ebl Seznec, Hao Wu, Julien Demouth, and Song Han. 2023. SmoothQuant: Accurate and efficient post-training quantization for large language models. In ICML(PMLR, Vol. 202)."},{"key":"e_1_3_1_151_2","volume-title":"AAAI","author":"Xu Runxin","year":"2022","unstructured":"Runxin Xu, Fuli Luo, Chengyu Wang, Baobao Chang, Jun Huang, Songfang Huang, and Fei Huang. 2022. From dense to sparse: Contrastive pruning for better pre-trained language model compression. In AAAI."},{"key":"e_1_3_1_152_2","unstructured":"June Yong Yang Byeongwook Kim Jeongin Bae Beomseok Kwon Gunho Park Eunho Yang Se Jung Kwon and Dongsoo Lee. 2024. No token left behind: reliable kv cache compression via importance-aware mixed precision quantization. arXiv:2402.18096."},{"key":"e_1_3_1_153_2","volume-title":"NeurIPS","author":"Yao Zhewei","year":"2022","unstructured":"Zhewei Yao, Reza Yazdani Aminabadi, Minjia Zhang, Xiaoxia Wu, Conglong Li, and Yuxiong He. 2022. ZeroQuant: Efficient and affordable post-training quantization for large-scale transformers. In NeurIPS."},{"key":"e_1_3_1_154_2","unstructured":"Zhewei Yao Xiaoxia Wu Cheng Li Stephen Youn and Yuxiong He. 2023. ZeroQuant-v2: Exploring post-training quantization in llms from comprehensive study to low rank compensation. arXiv:2303.08302."},{"key":"e_1_3_1_155_2","unstructured":"Zhewei Yao Xiaoxia Wu Linjian Ma Sheng Shen Kurt Keutzer Michael W. Mahoney and Yuxiong He. 2022. LEAP: Learnable pruning for transformer-based models. arXiv:2105.14636."},{"key":"e_1_3_1_156_2","volume-title":"ICML","author":"Yin Lu","year":"2024","unstructured":"Lu Yin, You Wu, Zhenyu Zhang, Cheng-Yu Hsieh, Yaqing Wang, Yiling Jia, Mykola Pechenizkiy, Yi Liang, Zhangyang Wang, and Shiwei Liu. 2024. Outlier weighed layerwise sparsity (owl): A missing secret sauce for pruning llms to high sparsity. In ICML."},{"key":"e_1_3_1_157_2","unstructured":"Zhihang Yuan Lin Niu Jiawei Liu Wenyu Liu Xinggang Wang Yuzhang Shang Guangyu Sun Qiang Wu Jiaxiang Wu and Bingzhe Wu. 2023. RPTQ: Reorder-based post-training quantization for large language models. arXiv:2304.01089."},{"key":"e_1_3_1_158_2","volume-title":"ACL","author":"Yun Jeongin","year":"2026","unstructured":"Jeongin Yun, Jaeri Lee, Jongjin Kim, Minjun Kim, Jinho Song, and U Kang. 2026. SharVeT: Similarity-aware parameter sharing with vector-based tuning for efficient llm compression. In ACL."},{"key":"e_1_3_1_159_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2025.3593987"},{"key":"e_1_3_1_160_2","volume-title":"ICLR","author":"Yvinec Edouard","year":"2023","unstructured":"Edouard Yvinec, Arnaud Dapogny, Matthieu Cord, and Kevin Bailly. 2023. PowerQuant: Automorphism search for non-uniform quantization. In ICLR."},{"key":"e_1_3_1_161_2","volume-title":"MICRO","author":"Zadeh Ali Hadi","year":"2020","unstructured":"Ali Hadi Zadeh, Isak Edo, Omar Mohamed Awad, and Andreas Moshovos. 2020. GOBO: Quantizing attention-based nlp models for low latency and energy efficient inference. In MICRO."},{"key":"e_1_3_1_162_2","volume-title":"NeurIPS","author":"Zafrir Ofir","year":"2019","unstructured":"Ofir Zafrir, Guy Boudoukh, Peter Izsak, and Moshe Wasserblat. 2019. Q8BERT: Quantized 8bit bert. In NeurIPS."},{"key":"e_1_3_1_163_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-short.1"},{"key":"e_1_3_1_164_2","volume-title":"Findings of ACL","author":"Zhang Mingyang","year":"2024","unstructured":"Mingyang Zhang, Hao Chen, Chunhua Shen, Zhen Yang, Linlin Ou, Xinyi Yu, and Bohan Zhuang. 2024. LoRAPrune: Structured Pruning meets low-rank parameter-efficient fine-tuning. In Findings of ACL."},{"key":"e_1_3_1_165_2","volume-title":"ICLR","author":"Zhang Qingru","year":"2023","unstructured":"Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao. 2023. Adaptive budget allocation for parameter-efficient fine-tuning. In ICLR."},{"key":"e_1_3_1_166_2","unstructured":"Susan Zhang Stephen Roller Naman Goyal Mikel Artetxe Moya Chen Shuohui Chen Christopher Dewan Mona T. Diab Xian Li Xi Victoria Lin et\u00a0al. 2022. OPT: Open pre-trained transformer language models. arXiv:2205.01068."},{"key":"e_1_3_1_167_2","volume-title":"EMNLP","author":"Zhang Wei","year":"2020","unstructured":"Wei Zhang, Lu Hou, Yichun Yin, Lifeng Shang, Xiao Chen, Xin Jiang, and Qun Liu. 2020. TernaryBERT: Distillation-aware ultra-low bit bert. In EMNLP."},{"key":"e_1_3_1_168_2","volume-title":"ACL","author":"Zhao Jiaqi","year":"2025","unstructured":"Jiaqi Zhao, Miao Zhang, Ming Wang, Yuzhang Shang, Kaihao Zhang, Weili Guan, Yaowei Wang, and Min Zhang. 2025. PTQ1.61: Push the real limit of extremely low-bit post-training quantization methods for large language models. In ACL."},{"key":"e_1_3_1_169_2","volume-title":"NLPCC","author":"Zhao Zihan","year":"2020","unstructured":"Zihan Zhao, Yuncong Liu, Lu Chen, Qi Liu, Rao Ma, and Kai Yu. 2020. An investigation on different underlying quantization schemes for pre-trained language models. In NLPCC."}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3819816","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,27]],"date-time":"2026-07-27T13:23:32Z","timestamp":1785158612000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3819816"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,27]]},"references-count":168,"journal-issue":{"issue":"14","published-print":{"date-parts":[[2026,10,31]]}},"alternative-id":["10.1145\/3819816"],"URL":"https:\/\/doi.org\/10.1145\/3819816","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,27]]},"assertion":[{"value":"2024-01-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-07-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}