{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T10:40:16Z","timestamp":1790937616953,"version":"4.1.0"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"6","funder":[{"name":"Research Grants Council Grant for Theme-based Research Scheme Project","award":["T43-513\/23-N"],"award-info":[{"award-number":["T43-513\/23-N"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation","doi-asserted-by":"crossref","award":["62472126"],"award-info":[{"award-number":["62472126"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100003453","name":"Natural Science Foundation of Guangdong Province","doi-asserted-by":"crossref","award":["2023A1515011959"],"award-info":[{"award-number":["2023A1515011959"]}],"id":[{"id":"10.13039\/501100003453","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>Code retrieval aims to provide users with desired code snippets based on users\u2019 natural language queries. With the development of deep learning technologies, adopting pre-trained models for this task has become mainstream. Considering the retrieval efficiency, most of the previous approaches adopt a dual-encoder for this task, which encodes the description and code snippet into representation vectors, respectively. However, the model structure of the dual-encoder tends to limit the model\u2019s performance, since it lacks the interaction between the code snippet and description at the bottom layer of the model during training. To improve the model\u2019s effectiveness while preserving its efficiency, we propose a framework, which adopts self-adaptive model distillation for efficient code retrieval (SPENCER). SPENCER first adopts the dual-encoder to narrow the search space and then adopts the cross-encoder to improve accuracy. To improve the efficiency of SPENCER, we propose a novel model distillation technique, which can greatly reduce the inference time of the dual-encoder while maintaining the overall performance. We also propose a teaching assistant selection strategy for our model distillation, which can adaptively select the suitable teaching assistant models for different pre-trained models during the model distillation to ensure the model performance. Extensive experiments demonstrate that the combination of dual-encoder and cross-encoder improves overall performance compared to solely dual-encoder-based models for code retrieval. Besides, our model distillation technique retains over 98% of the overall performance while reducing the inference time of the dual-encoder by 70%.<\/jats:p>","DOI":"10.1145\/3765748","type":"journal-article","created":{"date-parts":[[2025,9,4]],"date-time":"2025-09-04T15:36:30Z","timestamp":1757000190000},"page":"1-30","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["SPENCER: Self-Adaptive Model Distillation for Efficient Code\u00a0Retrieval"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3503-8845","authenticated-orcid":false,"given":"Wenchao","family":"Gu","sequence":"first","affiliation":[{"name":"Computer Science and Engineering, The Chinese University of Hong Kong, Hong\u00a0Kong,\u00a0Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-1600-4378","authenticated-orcid":false,"given":"Zongyi","family":"Lyu","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology, Hong Kong, Hong Kong"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7761-7269","authenticated-orcid":false,"given":"Yanlin","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Software Engineering, Sun Yat-Sen University, Zhuhai, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3063-9425","authenticated-orcid":false,"given":"Hongyu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Chongqing University, Chongqing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4774-2434","authenticated-orcid":false,"given":"Cuiyun","family":"Gao","sequence":"additional","affiliation":[{"name":"Harbin Institute of Technology, Shenzhen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3666-5798","authenticated-orcid":false,"given":"Michael R.","family":"Lyu","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,5,13]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.211"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00051"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1145\/1518701.1518944"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3462840"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3340458"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2023.111856"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/n19-1423"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3641542"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2021.106542"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"e_1_3_2_12_2","first-page":"1602","volume-title":"Proceedings of the 35th International Conference on Machine Learning (ICML \u201918)Proceedings of Machine Learning Research","author":"Furlanello Tommaso","year":"2018","unstructured":"Tommaso Furlanello, Zachary Chase Lipton, Michael Tschannen, Laurent Itti, and Anima Anandkumar. 2018. Born-again neural networks. In Proceedings of the 35th International Conference on Machine Learning (ICML \u201918). Jennifer G. Dy and Andreas Krause (Eds.), Proceedings of Machine Learning Research, Vol. 80, PMLR, 1602\u20131611. Retrieved from http:\/\/proceedings.mlr.press\/v80\/furlanello18a.html"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.552"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","unstructured":"Andreas Schaad Stefan G\u00f6tz and Dominik Binder. 2025. You still have to study on the security of LLM generated code. In ICT Systems Security and Privacy Protection. Lili Nemec Zlatolas Kai Rannenberg Tatjana Welzer and Joaquin Garcia-Alfaro (Eds.) Vol. 746 Springer 111\u2013124. DOI: 10.1007\/978-3-031-92886-4_8","DOI":"10.1007\/978-3-031-92886-4_8"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.neunet.2021.04.019"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180167"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.499"},{"key":"e_1_3_2_18_2","volume-title":"Proceedings of the 9th International Conference on Learning Representations (ICLR \u201921)","author":"Guo Daya","year":"2021","unstructured":"Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie Liu, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, et al. 2021. GraphCodeBERT: Pre-training code representations with data flow. In Proceedings of the 9th International Conference on Learning Representations (ICLR \u201921). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=jLoC4ez43PZ"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.758"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"e_1_3_2_21_2","unstructured":"Geert Heyman and Tom Van Cutsem. 2020. Neural code search revisited: Enhancing code snippet retrieval through natural language intent. arXiv:2008.12193. Retrieved from https:\/\/arxiv.org\/abs\/2008.12193"},{"key":"e_1_3_2_22_2","unstructured":"Geoffrey E. Hinton Oriol Vinyals and Jeffrey Dean. 2015. Distilling the knowledge in a neural network. arXiv:1503.02531. Retrieved from http:\/\/arxiv.org\/abs\/1503.02531"},{"key":"e_1_3_2_23_2","unstructured":"Hamel Husain Ho-Hsiang Wu Tiferet Gazit Miltiadis Allamanis and Marc Brockschmidt. 2019. CodeSearchNet Challenge: Evaluating the state of semantic code search. arXiv:1909.09436. Retrieved from http:\/\/arxiv.org\/abs\/1909.09436"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.482"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/TBDATA.2019.2921572"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401075"},{"key":"e_1_3_2_27_2","volume-title":"Proceedings of the 3rd International Conference on Learning Representations (ICLR \u201915)","author":"Kingma Diederik P.","year":"2015","unstructured":"Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR \u201915). Yoshua Bengio and Yann LeCun (Eds.). Retrieved from https:\/\/dblp.org\/rec\/journals\/corr\/KingmaB14.bib"},{"key":"e_1_3_2_28_2","first-page":"1371","volume-title":"Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL \u201924)Vol. 1, Long Papers,","author":"Li Haochen","year":"2024","unstructured":"Haochen Li, Xin Zhou, and Zhiqi Shen. 2024. Rewriting the code: A simple method for large language model augmented code search. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL \u201924). Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.), Vol. 1, Long Papers, Association for Computational Linguistics, 1371\u20131389. Retrieved from https:\/\/aclanthology.org\/2024.acl-long.75"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01153"},{"key":"e_1_3_2_30_2","unstructured":"Ming Liang Xiaoheng Xie Gehao Zhang Xunjin Zheng Peng Di Wei Jiang Hongwei Chen Chengpeng Wang and Gang Fan. 2024. REPOFUSE: Repository-level code completion with fused dual context. arXiv:2402.14323. Retrieved from https:\/\/arxiv.org\/abs\/2402.14323"},{"key":"e_1_3_2_31_2","unstructured":"Fang Liu Yang Liu Lin Shi Houkun Huang Ruifeng Wang Zhen Yang and Li Zhang. 2024. Exploring and evaluating hallucinations in LLM-powered code generation. arXiv:2404.00971. Retrieved from https:\/\/arxiv.org\/abs\/2404.00971"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00207"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2022.3233901"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2015.42"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER56733.2023.00021"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/1985793.1985809"},{"key":"e_1_3_2_37_2","unstructured":"Bhaskar Mitra and Nick Craswell. 2017. Neural models for information retrieval. arXiv:1705.01509. Retrieved from http:\/\/arxiv.org\/abs\/1705.01509"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510096"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2023.EACL-MAIN.262"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00409"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP46214.2022.9833571"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3576915.3623157"},{"key":"e_1_3_2_43_2","unstructured":"Mirza Masfiqur Rahman and Ashish Kundu. 2024. Code hallucination. arXiv:2407.04831. Retrieved from https:\/\/arxiv.org\/abs\/2407.04831"},{"key":"e_1_3_2_44_2","volume-title":"Proceedings of the 3rd International Conference on Learning Representations (ICLR \u201915)","author":"Romero Adriana","year":"2015","unstructured":"Adriana Romero, Nicolas Ballas, Samira Ebrahimi Kahou, Antoine Chassang, Carlo Gatta, and Yoshua Bengio. 2015. FitNets: Hints for thin deep nets. In Proceedings of the 3rd International Conference on Learning Representations (ICLR \u201915). Yoshua Bengio and Yann LeCun (Eds.). Retrieved from https:\/\/dblp.org\/rec\/journals\/corr\/RomeroBKCGB14.bib"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3211346.3211353"},{"key":"e_1_3_2_46_2","first-page":"2205","volume-title":"Proceedings of the 32nd USENIX Security Symposium, (USENIX Security \u201923)","author":"Sandoval Gustavo","year":"2023","unstructured":"Gustavo Sandoval, Hammond Pearce, Teo Nys, Ramesh Karri, Siddharth Garg, and Brendan Dolan-Gavitt. 2023. Lost at C: A user study on the security implications of large language model code assistants. In Proceedings of the 32nd USENIX Security Symposium, (USENIX Security \u201923). Joseph A. Calandrino and Carmela Troncoso (Eds.), USENIX Association, 2205\u20132222. Retrieved from https:\/\/www.usenix.org\/conference\/usenixsecurity23\/presentation\/sandoval"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00185"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556964"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC58990.2023.00043"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.24963\/IJCAI.2019\/477"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387904.3389269"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3645071"},{"key":"e_1_3_2_53_2","first-page":"1849","volume-title":"Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016","author":"Sohn Kihyuk","year":"2016","unstructured":"Kihyuk Sohn. 2016. Improved deep metric learning with multi-class N-pair loss objective. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016. Daniel D. Lee, Masashi Sugiyama, Ulrike von Luxburg, Isabelle Guyon, and Roman Garnett (Eds.), 1849\u20131857. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2016\/hash\/6b180037abbebea991d8b1232f8a8ca9-Abstract.html"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00926"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3219819.3220021"},{"key":"e_1_3_2_56_2","unstructured":"Raphael Tang Yao Lu Linqing Liu Lili Mou Olga Vechtomova and Jimmy Lin. 2019. Distilling task-specific knowledge from BERT into simple neural networks. arXiv:1903.12136. Retrieved from http:\/\/arxiv.org\/abs\/1903.12136"},{"key":"e_1_3_2_57_2","unstructured":"Yuchen Tian Weixiang Yan Qian Yang Qian Chen Wen Wang Ziyang Luo and Lei Ma. 2024. CodeHalu: Code hallucinations in LLMs driven by execution-based verification. arXiv:2405.00253. Retrieved from https:\/\/arxiv.org\/abs\/2405.00253"},{"key":"e_1_3_2_58_2","unstructured":"A\u00e4ron van den Oord Yazhe Li and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv:1807.03748. Retrieved from http:\/\/arxiv.org\/abs\/1807.03748"},{"key":"e_1_3_2_59_2","unstructured":"Xin Wang Yasheng Wang Fei Mi Pingyi Zhou Yao Wan Xiao Liu Li Li Hao Wu Jin Liu and Xin Jiang. 2021. SynCoBERT: Syntax-guided multi-modal contrastive pre-training for code representation. arXiv:2108.04556. Retrieved from https:\/\/arxiv.org\/abs\/2108.04556"},{"key":"e_1_3_2_60_2","unstructured":"Yanlin Wang Daya Guo Jiachi Chen Ruikai Zhang Yuchi Ma and Zibin Zheng. 2024. RLCoder: Reinforcement learning for repository-level code completion. arXiv:2407.19487. Retrieved from https:\/\/arxiv.org\/abs\/2407.19487"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.685"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00393"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3308558.3313632"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.18653\/V1\/2023.EMNLP-MAIN.151"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00454"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3765748","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T17:26:27Z","timestamp":1778693187000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3765748"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,13]]},"references-count":64,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3765748"],"URL":"https:\/\/doi.org\/10.1145\/3765748","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,13]]},"assertion":[{"value":"2024-11-19","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-07-29","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}