{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,11]],"date-time":"2026-08-11T16:22:27Z","timestamp":1786465347684,"version":"build-2736575974"},"publisher-location":"New York, NY, USA","reference-count":84,"publisher":"ACM","license":[{"start":{"date-parts":[[2025,3,10]],"date-time":"2025-03-10T00:00:00Z","timestamp":1741564800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,3,10]]},"DOI":"10.1145\/3701551.3703577","type":"proceedings-article","created":{"date-parts":[[2025,2,26]],"date-time":"2025-02-26T12:30:16Z","timestamp":1740573016000},"page":"251-260","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Beyond Answers: Transferring Reasoning Capabilities to Smaller LLMs Using Multi-Teacher Knowledge Distillation"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2795-6080","authenticated-orcid":false,"given":"Yijun","family":"Tian","sequence":"first","affiliation":[{"name":"University of Notre Dame, Notre Dame, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-9683-3092","authenticated-orcid":false,"given":"Yikun","family":"Han","sequence":"additional","affiliation":[{"name":"University of Michigan, Ann Arbor, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9713-8000","authenticated-orcid":false,"given":"Xiusi","family":"Chen","sequence":"additional","affiliation":[{"name":"University of California, Los Angeles, Los Angeles, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8180-2886","authenticated-orcid":false,"given":"Wei","family":"Wang","sequence":"additional","affiliation":[{"name":"University of California, Los Angeles, Los Angeles, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3932-5956","authenticated-orcid":false,"given":"Nitesh V.","family":"Chawla","sequence":"additional","affiliation":[{"name":"University of Notre Dame, Notre Dame, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,3,10]]},"reference":[{"key":"e_1_3_2_1_1_1","volume-title":"Generalized Knowledge Distillation for Auto-regressive Language Models. In The Twelfth International Conference on Learning Representations.","author":"Agarwal Rishabh","year":"2024","unstructured":"Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. 2024. Generalized Knowledge Distillation for Auto-regressive Language Models. In The Twelfth International Conference on Learning Representations."},{"key":"e_1_3_2_1_2_1","unstructured":"Rohan Anil Andrew M Dai Orhan Firat Melvin Johnson Dmitry Lepikhin Alexandre Passos Siamak Shakeri Emanuel Taropa Paige Bailey Zhifeng Chen et al. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403 (2023)."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"crossref","unstructured":"Yejin Bang Samuel Cahyawijaya Nayeon Lee Wenliang Dai Dan Su Bryan Wilie Holy Lovenia Ziwei Ji Tiezheng Yu Willy Chung Quyet V. Do Yan Xu and Pascale Fung. 2023. A Multitask Multilingual Multimodal Evaluation of ChatGPT on Reasoning Hallucination and Interactivity. In ACL.","DOI":"10.18653\/v1\/2023.ijcnlp-main.45"},{"key":"e_1_3_2_1_4_1","volume-title":"Jianfeng Gao, and Yejin Choi.","author":"Bisk Yonatan","year":"2020","unstructured":"Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. 2020. PIQA: Reasoning about Physical Commonsense in Natural Language. In AAAI."},{"key":"e_1_3_2_1_5_1","unstructured":"Tom B Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. In NeurIPS."},{"key":"e_1_3_2_1_6_1","unstructured":"Oana-Maria Camburu Tim Rockt\u00e4schel Thomas Lukasiewicz and Phil Blunsom. 2018. e-SNLI: Natural Language Inference with Natural Language Explanations. In NeurIPS."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Hongzhan Chen Siyue Wu Xiaojun Quan Rui Wang Ming Yan and Ji Zhang. 2023. MCC-KD: Multi-CoT Consistent Knowledge Distillation. In EMNLP Findings.","DOI":"10.18653\/v1\/2023.findings-emnlp.454"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"crossref","unstructured":"Xiusi Chen Jyun-Yu Jiang Wei-Cheng Chang Cho-Jui Hsieh Hsiang-Fu Yu and Wei Wang. 2024. MinPrompt: Graph-based Minimal Prompt Data Augmentation for Few-shot Question Answering. In ACL.","DOI":"10.18653\/v1\/2024.acl-long.16"},{"key":"e_1_3_2_1_9_1","unstructured":"Jang Hyun Cho and Bharath Hariharan. 2019. On the efficacy of knowledge distillation. In ICCV."},{"key":"e_1_3_2_1_10_1","volume-title":"Charles Sutton, Sebastian Gehrmann, et al.","author":"Chowdhery Aakanksha","year":"2023","unstructured":"Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. PALM: Scaling Language Modeling with Pathways. Journal of Machine Learning Research (2023)."},{"key":"e_1_3_2_1_11_1","unstructured":"Hyung Won Chung Le Hou Shayne Longpre Barret Zoph Yi Tay William Fedus Yunxuan Li Xuezhi Wang Mostafa Dehghani Siddhartha Brahma et al. 2022. Scaling Instruction-Finetuned Language Models. arXiv preprint arXiv:2210.11416 (2022)."},{"key":"e_1_3_2_1_12_1","volume-title":"Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge. arXiv preprint arXiv:1803.05457","author":"Clark Peter","year":"2018","unstructured":"Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge. arXiv preprint arXiv:1803.05457 (2018)."},{"key":"e_1_3_2_1_13_1","unstructured":"Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle Aiesha Letman Akhil Mathur Alan Schelten Amy Yang Angela Fan et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)."},{"key":"e_1_3_2_1_14_1","volume-title":"Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model. arXiv preprint arXiv:2210.02498","author":"Eisenstein Jacob","year":"2022","unstructured":"Jacob Eisenstein, Daniel Andor, Bernd Bohnet, Michael Collins, and David Mimno. 2022. Honest Students from Untrusted Teachers: Learning an Interpretable Question-Answering Pipeline from a Pretrained Language Model. arXiv preprint arXiv:2210.02498 (2022)."},{"key":"e_1_3_2_1_15_1","unstructured":"Yao Fu Hao Peng Litu Ou Ashish Sabharwal and Tushar Khot. 2023. Specializing Smaller Language Models towards Multi-Step Reasoning. In ICML."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-021-01453-z"},{"key":"e_1_3_2_1_17_1","volume-title":"The Twelfth International Conference on Learning Representations.","author":"Gu Yuxian","year":"2024","unstructured":"Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. MiniLLM: Knowledge distillation of large language models. In The Twelfth International Conference on Learning Representations."},{"key":"e_1_3_2_1_18_1","volume-title":"Ppt: Pre-trained prompt tuning for few-shot learning. arXiv preprint arXiv:2109.04332","author":"Gu Yuxian","year":"2021","unstructured":"Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. 2021. Ppt: Pre-trained prompt tuning for few-shot learning. arXiv preprint arXiv:2109.04332 (2021)."},{"key":"e_1_3_2_1_19_1","unstructured":"Zhichun Guo Chunhui Zhang Yujie Fan Yijun Tian Chuxu Zhang and Nitesh V Chawla. 2023. Boosting graph neural networks via adaptive knowledge distillation. In AAAI."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"crossref","unstructured":"Braden Hancock Antoine Bordes Pierre-Emmanuel Mazare and Jason Weston. 2019. Learning from Dialogue after Deployment: Feed Yourself Chatbot!. In ACL.","DOI":"10.18653\/v1\/P19-1358"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.lnls-1.4"},{"key":"e_1_3_2_1_22_1","unstructured":"Geoffrey Hinton Oriol Vinyals Jeff Dean et al. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531 (2015)."},{"key":"e_1_3_2_1_23_1","volume-title":"Large Language Models Are Reasoning Teachers. arXiv preprint arXiv:2212.10071","author":"Ho Namgyu","year":"2022","unstructured":"Namgyu Ho, Laura Schmid, and Se-Young Yun. 2022. Large Language Models Are Reasoning Teachers. arXiv preprint arXiv:2212.10071 (2022)."},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"crossref","unstructured":"Cheng-Yu Hsieh Chun-Liang Li Chih-kuan Yeh Hootan Nakhost Yasuhisa Fujii Alex Ratner Ranjay Krishna Chen-Yu Lee and Tomas Pfister. 2023. Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizes. In ACL Findings.","DOI":"10.18653\/v1\/2023.findings-acl.507"},{"key":"e_1_3_2_1_25_1","unstructured":"Edward J Hu Yelong Shen Phillip Wallis Zeyuan Allen-Zhu Yuanzhi Li Shean Wang Lu Wang and Weizhu Chen. 2022. LoRA: Low-Rank Adaptation of Large Language Models. In ICLR."},{"key":"e_1_3_2_1_26_1","volume-title":"Towards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403","author":"Huang Jie","year":"2022","unstructured":"Jie Huang and Kevin Chen-Chuan Chang. 2022. Towards reasoning in large language models: A survey. arXiv preprint arXiv:2212.10403 (2022)."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.67"},{"key":"e_1_3_2_1_28_1","volume-title":"Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han.","author":"Huang Jiaxin","year":"2022","unstructured":"Jiaxin Huang, Shixiang Shane Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. 2022. Large language models can self-improve. arXiv preprint arXiv:2210.11610 (2022)."},{"key":"e_1_3_2_1_29_1","volume-title":"Survey of Hallucination in Natural Language Generation. Comput. Surveys","author":"Ji Z","year":"2023","unstructured":"Z Ji, N Lee, R Frieske, T Yu, D Su, Y Xu, E Ishii, Y J Bang, A Madotto, and P Fung. 2023. Survey of Hallucination in Natural Language Generation. Comput. Surveys (2023)."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"crossref","unstructured":"Qiao Jin Bhuwan Dhingra Zhengping Liu William Cohen and Xinghua Lu. 2019. PubMedQA: A Dataset for Biomedical Research Question Answering. In EMNLP.","DOI":"10.18653\/v1\/D19-1259"},{"key":"e_1_3_2_1_31_1","unstructured":"J Kaplan S McCandlish T Henighan et al. 2020. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361 (2020)."},{"key":"e_1_3_2_1_32_1","volume-title":"DistiLLM: Towards Streamlined Distillation for Large Language Models. In Forty-first International Conference on Machine Learning.","author":"Ko Jongwoo","unstructured":"Jongwoo Ko, Sungnyun Kim, Tianyi Chen, and Se-Young Yun. [n.,d.]. DistiLLM: Towards Streamlined Distillation for Large Language Models. In Forty-first International Conference on Machine Learning."},{"key":"e_1_3_2_1_33_1","unstructured":"Takeshi Kojima Shixiang (Shane) Gu Machel Reid Yutaka Matsuo and Yusuke Iwasawa. 2022. Large Language Models are Zero-Shot Reasoners. In NeurIPS."},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"crossref","unstructured":"B Lester R Al-Rfou and N Constant. 2021. The Power of Scale for Parameter-Efficient Prompt Tuning. In EMNLP.","DOI":"10.18653\/v1\/2021.emnlp-main.243"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"crossref","unstructured":"Liunian Harold Li et al. 2023. Symbolic Chain-of-Thought Distillation: Small Models Can Also \"Think\" Step-by-Step. arXiv preprint arXiv:2306.14050 (2023).","DOI":"10.18653\/v1\/2023.acl-long.150"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"crossref","unstructured":"X L Li and P Liang. 2021. Prefix-Tuning: Optimizing Continuous Prompts for Generation. In ACL.","DOI":"10.18653\/v1\/2021.acl-long.353"},{"key":"e_1_3_2_1_37_1","unstructured":"Bill Yuchen Lin Ziyi Wu Yichi Yang Dong-Ho Lee and Xiang Ren. 2021. RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge. In ACL Findings."},{"key":"e_1_3_2_1_38_1","volume-title":"Mind's Mirror: Distilling Self-Evaluation Capability and Comprehensive Thinking from Large Language Models. arXiv preprint arXiv:2311.09214","author":"Liu Weize","year":"2023","unstructured":"Weize Liu, Guocong Li, Kai Zhang, Bang Du, Qiyuan Chen, Xuming Hu, Hongxia Xu, Jintai Chen, and Jian Wu. 2023. Mind's Mirror: Distilling Self-Evaluation Capability and Comprehensive Thinking from Large Language Models. arXiv preprint arXiv:2311.09214 (2023)."},{"key":"e_1_3_2_1_39_1","volume-title":"Zhengxiao Du, Zhilin Yang, and Jie Tang.","author":"Liu Xiao","year":"2021","unstructured":"Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2021. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602 (2021)."},{"key":"e_1_3_2_1_40_1","volume-title":"Towards Safer Large Language Models through Machine Unlearning. arXiv preprint arXiv:2402.10058","author":"Liu Zheyuan","year":"2024","unstructured":"Zheyuan Liu, Guangyao Dou, Zhaoxuan Tan, Yijun Tian, and Meng Jiang. 2024a. Towards Safer Large Language Models through Machine Unlearning. arXiv preprint arXiv:2402.10058 (2024)."},{"key":"e_1_3_2_1_41_1","unstructured":"Zheyuan Liu Xiaoxin He Yijun Tian and Nitesh V Chawla. 2024b. Can we soft prompt LLMs for graph learning tasks?. In WWW."},{"key":"e_1_3_2_1_42_1","unstructured":"P Lu S Mishra T Xia L Qiu K-W Chang S-C Zhu O Tafjord P Clark and A Kalyan. 2022. Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering. In NeurIPS."},{"key":"e_1_3_2_1_43_1","volume-title":"Teaching Small Language Models to Reason. arXiv preprint arXiv:2212.08410","author":"Magister Lucie Charlotte","year":"2022","unstructured":"Lucie Charlotte Magister, Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. 2022. Teaching Small Language Models to Reason. arXiv preprint arXiv:2212.08410 (2022)."},{"key":"e_1_3_2_1_44_1","doi-asserted-by":"crossref","unstructured":"Todor Mihaylov Peter Clark Tushar Khot and Ashish Sabharwal. 2018. Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering. In EMNLP.","DOI":"10.18653\/v1\/D18-1260"},{"key":"e_1_3_2_1_45_1","volume-title":"arXiv preprint arXiv:2004.14546","author":"Narang Sharan","year":"2020","unstructured":"Sharan Narang, Colin Raffel, Katherine Lee, Adam Roberts, Noah Fiedel, and Karishma Malkan. 2020. WT5?! Training Text-to-Text Models to Explain Their Predictions. arXiv preprint arXiv:2004.14546 (2020)."},{"key":"e_1_3_2_1_46_1","volume-title":"Michael Collins, Zachary C Lipton, Graham Neubig, and William W Cohen.","author":"Pruthi Danish","year":"2022","unstructured":"Danish Pruthi, Rachit Bansal, Bhuwan Dhingra, Livio Baldini Soares, Michael Collins, Zachary C Lipton, Graham Neubig, and William W Cohen. 2022. Evaluating Explanations: How Much Do Explanations from the Teacher Aid Students?. In ACL."},{"key":"e_1_3_2_1_47_1","unstructured":"C Raffel N Shazeer A Roberts et al. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. Journal of Machine Learning Research (2020)."},{"key":"e_1_3_2_1_48_1","unstructured":"Nazneen Fatema Rajani Bryan McCann Caiming Xiong and Richard Socher. 2019. Explain Yourself! Leveraging Language Models for Commonsense Reasoning. In ACL."},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"crossref","unstructured":"Kavel Rao Liwei Jiang Valentina Pyatkin Yuling Gu Niket Tandon Nouha Dziri Faeze Brahman and Yejin Choi. 2023. What Makes it Ok to Set a Fire? Iterative Self-distillation of Contexts and Rationales for Disambiguating Defeasible Social and Moral Situations. In EMNLP Findings.","DOI":"10.18653\/v1\/2023.findings-emnlp.812"},{"key":"e_1_3_2_1_50_1","volume-title":"Right for the Right Reasons: Training Differentiable Models by Constraining Their Explanations. arXiv preprint arXiv:1703.03717","author":"Ross Andrew Slavin","year":"2017","unstructured":"Andrew Slavin Ross, Michael C Hughes, and Finale Doshi-Velez. 2017. Right for the Right Reasons: Training Differentiable Models by Constraining Their Explanations. arXiv preprint arXiv:1703.03717 (2017)."},{"key":"e_1_3_2_1_51_1","volume-title":"a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108","author":"Sanh Victor","year":"2019","unstructured":"Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 (2019)."},{"key":"e_1_3_2_1_52_1","volume-title":"Mededit: Model Editing for Medical Question Answering with External Knowledge Bases. arXiv preprint arXiv:2309.16035","author":"Shi Yucheng","year":"2023","unstructured":"Yucheng Shi, Shaochen Xu, et al. 2023. Mededit: Model Editing for Medical Question Answering with External Knowledge Bases. arXiv preprint arXiv:2309.16035 (2023)."},{"key":"e_1_3_2_1_53_1","volume-title":"Julie Bernauer, Xia Song, Mohammad Shoeybi, Yuxiong He, Michael Houston, Saurabh Tiwary, and Bryan Catanzaro.","author":"Smith Shaden","year":"2022","unstructured":"Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, Elton Zhang, Rewon Child, Reza Yazdani Aminabadi, Julie Bernauer, Xia Song, Mohammad Shoeybi, Yuxiong He, Michael Houston, Saurabh Tiwary, and Bryan Catanzaro. 2022. Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, a Large-Scale Generative Language Model. arXiv preprint arXiv:2201.11990 (2022)."},{"key":"e_1_3_2_1_54_1","volume-title":"Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning. arXiv preprint arXiv:2402.04401","author":"Tan Zhaoxuan","year":"2024","unstructured":"Zhaoxuan Tan, Qingkai Zeng, Yijun Tian, Zheyuan Liu, Bing Yin, and Meng Jiang. 2024. Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning. arXiv preprint arXiv:2402.04401 (2024)."},{"key":"e_1_3_2_1_55_1","volume-title":"Stanford Alpaca: An Instruction-following LLaMA model. GitHub repository","author":"Taori Rohan","year":"2023","unstructured":"Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B Hashimoto. 2023. Stanford Alpaca: An Instruction-following LLaMA model. GitHub repository (2023)."},{"key":"e_1_3_2_1_56_1","volume-title":"Cassidy Hardin, Surya Bhupatiraju, L\u00e9onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram\u00e9, et al.","author":"Team Gemma","year":"2024","unstructured":"Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, L\u00e9onard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ram\u00e9, et al. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118 (2024)."},{"key":"e_1_3_2_1_57_1","volume-title":"Knowledge Distillation on Graphs: A Survey. arXiv preprint arXiv:2302.00219","author":"Tian Yijun","year":"2023","unstructured":"Yijun Tian, Shichao Pei, Xiangliang Zhang, Chuxu Zhang, and Nitesh V Chawla. 2023a. Knowledge Distillation on Graphs: A Survey. arXiv preprint arXiv:2302.00219 (2023)."},{"key":"e_1_3_2_1_58_1","doi-asserted-by":"crossref","unstructured":"Yijun Tian Huan Song Zichen Wang Haozhu Wang Ziqing Hu Fang Wang Nitesh V Chawla and Panpan Xu. 2024. Graph neural prompting with large language models. In AAAI.","DOI":"10.1609\/aaai.v38i17.29875"},{"key":"e_1_3_2_1_59_1","unstructured":"Yijun Tian Chuxu Zhang Zhichun Guo Xiangliang Zhang and Nitesh V Chawla. 2023b. Learning MLPs on Graphs: A Unified View of Effectiveness Robustness and Efficiency. In ICLR."},{"key":"e_1_3_2_1_60_1","volume-title":"Llama: Open and Efficient Foundation Language Models. arXiv preprint arXiv:2302.13971","author":"Touvron Hugo","year":"2023","unstructured":"Hugo Touvron, Thibaut Lavril, et al. 2023a. Llama: Open and Efficient Foundation Language Models. arXiv preprint arXiv:2302.13971 (2023)."},{"key":"e_1_3_2_1_61_1","unstructured":"Hugo Touvron Louis Martin et al. 2023b. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv preprint arXiv:2307.09288 (2023)."},{"key":"e_1_3_2_1_62_1","doi-asserted-by":"crossref","unstructured":"George Tsatsaronis Georgios Balikas Prodromos Malakasiotis et al. 2015. An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC Bioinformatics (2015).","DOI":"10.1186\/s12859-015-0564-6"},{"key":"e_1_3_2_1_63_1","volume-title":"Advances in Neural Information Processing Systems","volume":"36","author":"Turpin Miles","year":"2024","unstructured":"Miles Turpin, Julian Michael, Ethan Perez, and Samuel Bowman. 2024. Language models don't always say what they think: unfaithful explanations in chain-of-thought prompting. Advances in Neural Information Processing Systems, Vol. 36 (2024)."},{"key":"e_1_3_2_1_64_1","unstructured":"Zhongwei Wan Xin Wang Che Liu Samiul Alam Yu Zheng Zhongnan Qu Shen Yan Yi Zhu Quanlu Zhang Mosharaf Chowdhury et al. 2023. Efficient large language models: A survey. arXiv preprint arXiv:2312.03863 Vol. 1 (2023)."},{"key":"e_1_3_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549113"},{"key":"e_1_3_2_1_66_1","volume-title":"Roselora: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning. arXiv preprint arXiv:2406.10777","author":"Wang Haoyu","year":"2024","unstructured":"Haoyu Wang, Tianci Liu, Ruirui Li, Monica Cheng, Tuo Zhao, and Jing Gao. 2024. Roselora: Row and column-wise sparse low-rank adaptation of pre-trained language model for knowledge editing and fine-tuning. arXiv preprint arXiv:2406.10777 (2024)."},{"key":"e_1_3_2_1_67_1","doi-asserted-by":"crossref","unstructured":"Haoyu Wang Yaqing Wang Tianci Liu Tuo Zhao and Jing Gao. 2023. HadSkip: Homotopic and Adaptive Layer Skipping of Pre-trained Language Models for Efficient Inference. In EMNLP Findings.","DOI":"10.18653\/v1\/2023.findings-emnlp.283"},{"key":"e_1_3_2_1_68_1","volume-title":"PINTO: Faithful Language Reasoning Using Prompt-Generated Rationales. In ICLR.","author":"Wang Peifeng","year":"2022","unstructured":"Peifeng Wang, Aaron Chan, Filip Ilievski, Muhao Chen, and Xiang Ren. 2022a. PINTO: Faithful Language Reasoning Using Prompt-Generated Rationales. In ICLR."},{"key":"e_1_3_2_1_69_1","doi-asserted-by":"crossref","unstructured":"Shuohang Wang Yang Liu Yichong Xu Chenguang Zhu and Michael Zeng. 2021. Want To Reduce Labeling Cost? GPT-3 Can Help. In EMNLP Findings.","DOI":"10.18653\/v1\/2021.findings-emnlp.354"},{"key":"e_1_3_2_1_70_1","unstructured":"J Wei Y Tay R Bommasani et al. 2022a. Emergent Abilities of Large Language Models. arXiv preprint arXiv:2206.07682 (2022)."},{"key":"e_1_3_2_1_71_1","unstructured":"Jason Wei Xuezhi Wang et al. 2022b. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. In NeurIPS."},{"key":"e_1_3_2_1_72_1","doi-asserted-by":"crossref","unstructured":"W Wei X Ren J Tang Q Wang L Su S Cheng J Wang D Yin and C Huang. 2024. LLMRec: Large Language Models with Graph Augmentation for Recommendation. In WSDM.","DOI":"10.1145\/3616855.3635853"},{"key":"e_1_3_2_1_73_1","doi-asserted-by":"crossref","unstructured":"Sarah Wiegreffe Ana Marasovic and Noah A Smith. 2021. Measuring Association Between Labels and Free-Text Rationales. In EMNLP.","DOI":"10.18653\/v1\/2021.emnlp-main.804"},{"key":"e_1_3_2_1_74_1","volume-title":"Workshop, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili\u0107, Daniel Hesslow, Roman Castagn\u00e9, Alexandra Sasha Luccioni, Fran\u00e7ois Yvon, et al. 2022","year":"2022","unstructured":"BigScience Workshop, Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ili\u0107, Daniel Hesslow, Roman Castagn\u00e9, Alexandra Sasha Luccioni, Fran\u00e7ois Yvon, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100 (2022)."},{"key":"e_1_3_2_1_75_1","unstructured":"Likang Wu Zhi Zheng Zhaopeng Qiu Hao Wang Hongchao Gu Tingjia Shen Chuan Qin Chen Zhu Hengshu Zhu Qi Liu et al. 2023. A Survey on Large Language Models for Recommendation. arXiv preprint arXiv:2305.19860 (2023)."},{"key":"e_1_3_2_1_76_1","volume-title":"A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116","author":"Xu Xiaohan","year":"2024","unstructured":"Xiaohan Xu, Ming Li, Chongyang Tao, Tao Shen, Reynold Cheng, Jinyang Li, Can Xu, Dacheng Tao, and Tianyi Zhou. 2024. A survey on knowledge distillation of large language models. arXiv preprint arXiv:2402.13116 (2024)."},{"key":"e_1_3_2_1_77_1","volume-title":"Using \u201dAnnotator Rationales","author":"Zaidan Omar","unstructured":"Omar Zaidan, Jason Eisner, and Christine Piatko. 2007. Using \u201dAnnotator Rationales\u201d to Improve Machine Learning for Text Categorization. In NAACL."},{"key":"e_1_3_2_1_78_1","doi-asserted-by":"crossref","unstructured":"Eric Zelikman Wanjing Ma Jasmine Tran Diyi Yang Jason Yeatman and Nick Haber. 2023. Generating and Evaluating Tests for K-12 Students with Language Model Simulations: A Case Study on Sentence Reading Efficiency. In EMNLP.","DOI":"10.18653\/v1\/2023.emnlp-main.135"},{"key":"e_1_3_2_1_79_1","volume-title":"Star: Bootstrapping Reasoning with Reasoning. In NeurIPS.","author":"Zelikman Eric","year":"2022","unstructured":"Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. 2022. Star: Bootstrapping Reasoning with Reasoning. In NeurIPS."},{"key":"e_1_3_2_1_80_1","doi-asserted-by":"crossref","unstructured":"Ye Zhang Iain Marshall and Byron C Wallace. 2016. Rationale-Augmented Convolutional Neural Networks for Text Classification. In EMNLP.","DOI":"10.18653\/v1\/D16-1076"},{"key":"e_1_3_2_1_81_1","author":"Zhang Zhuosheng","unstructured":"Zhuosheng Zhang, Aston Zhang, Mu Li, George Karypis, Alex Smola, et al. [n.,d.]. Multimodal Chain-of-Thought Reasoning in Language Models. Transactions on Machine Learning Research ( [n.,d.]).","journal-title":"[n.,d.]. Multimodal Chain-of-Thought Reasoning in Language Models. Transactions on Machine Learning Research ( [n.,d.])."},{"key":"e_1_3_2_1_82_1","volume-title":"A Survey of Large Language Models. arXiv preprint arXiv:2303.18223","author":"Zhao Wayne Xin","year":"2023","unstructured":"Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian-Yun Nie, and Ji-Rong Wen. 2023. A Survey of Large Language Models. arXiv preprint arXiv:2303.18223 (2023)."},{"key":"e_1_3_2_1_83_1","unstructured":"Lianmin Zheng Wei-Lin Chiang et al. 2023. Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. arXiv preprint arXiv:2306.05685 (2023)."},{"key":"e_1_3_2_1_84_1","volume-title":"Retrieving and Reading: A Comprehensive Survey on Open-Domain Question Answering. arXiv preprint arXiv:2101.00774","author":"Zhu Fengbin","year":"2021","unstructured":"Fengbin Zhu, Wenqiang Lei, Chao Wang, Jianming Zheng, Soujanya Poria, and Tat-S Chua. 2021. Retrieving and Reading: A Comprehensive Survey on Open-Domain Question Answering. arXiv preprint arXiv:2101.00774 (2021)."}],"event":{"name":"WSDM '25: The Eighteenth ACM International Conference on Web Search and Data Mining","location":"Hannover Germany","acronym":"WSDM '25","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data","SIGIR ACM Special Interest Group on Information Retrieval"]},"container-title":["Proceedings of the Eighteenth ACM International Conference on Web Search and Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3701551.3703577","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3701551.3703577","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,8,21]],"date-time":"2025-08-21T09:13:30Z","timestamp":1755767610000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3701551.3703577"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,10]]},"references-count":84,"alternative-id":["10.1145\/3701551.3703577","10.1145\/3701551"],"URL":"https:\/\/doi.org\/10.1145\/3701551.3703577","relation":{},"subject":[],"published":{"date-parts":[[2025,3,10]]},"assertion":[{"value":"2025-03-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}