{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,11]],"date-time":"2026-02-11T13:36:47Z","timestamp":1770817007988,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":58,"publisher":"ACM","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,6,20]]},"DOI":"10.1145\/3755881.3755925","type":"proceedings-article","created":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T11:46:17Z","timestamp":1761565577000},"page":"461-473","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["AdaptiveLLM: A Framework for Selecting Optimal Cost-Efficient LLM for Code-Generation Based on CoT Length"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-2254-9198","authenticated-orcid":false,"given":"Junhang","family":"Cheng","sequence":"first","affiliation":[{"name":"State Key Laboratory of Complex & Critical Software Environment, School of Computer Science and Engineering, Beihang University, Beijing, China, Beihang University, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3905-8133","authenticated-orcid":false,"given":"Fang","family":"Liu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Complex & Critical Software Environment, School of Computer Science and Engineering, Beihang University, Beijing, China, Beihang University, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-0195-0684","authenticated-orcid":false,"given":"Chengru","family":"Wu","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Complex & Critical Software Environment, School of Computer Science and Engineering, Beihang University, Beijing, China, Beihang University, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2258-5893","authenticated-orcid":false,"given":"Li","family":"Zhang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Complex & Critical Software Environment, School of Computer Science and Engineering, Beihang University, Beijing, China, Beihang University, Beijing, China"}]}],"member":"320","published-online":{"date-parts":[[2025,10,27]]},"reference":[{"key":"e_1_3_3_1_2_2","unstructured":"Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad Ilge Akkaya Florencia\u00a0Leoni Aleman Diogo Almeida Janko Altenschmidt Sam Altman Shyamal Anadkat et\u00a0al. 2023. Gpt-4 technical report. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2303.08774 (2023)."},{"key":"e_1_3_3_1_3_2","unstructured":"Anysphere. 2009. Codeforces. https:\/\/codeforces.com\/"},{"key":"e_1_3_3_1_4_2","unstructured":"Anysphere. 2015. LeetCode. https:\/\/leetcode.com\/"},{"key":"e_1_3_3_1_5_2","unstructured":"Anysphere. 2023. Cursor. https:\/\/www.cursor.com\/"},{"key":"e_1_3_3_1_6_2","unstructured":"Zeliha\u00a0Ergul Aydin and Zehra\u00a0Kamisli Ozturk. 2021. Performance analysis of XGBoost classifier with missing data. Manchester Journal of Artificial Intelligence and Applied Sciences (MJAIAS) 2 02 (2021) 2021."},{"key":"e_1_3_3_1_7_2","unstructured":"Henry Bae Aghyad Deeb Alex Fleury and Kehang Zhu. 2023. Complexitynet: Increasing llm inference efficiency by learning task complexity. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2312.11511 (2023)."},{"key":"e_1_3_3_1_8_2","unstructured":"Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared\u00a0D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et\u00a0al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020) 1877\u20131901."},{"key":"e_1_3_3_1_9_2","doi-asserted-by":"crossref","unstructured":"G\u00a0Ann Campbell. 2018. Cognitive Complexity-A new way of measuring understandability. SonarSource SA 10 (2018).","DOI":"10.1145\/3194164.3194186"},{"key":"e_1_3_3_1_10_2","unstructured":"Federico Cassano John Gouwar Daniel Nguyen Sydney Nguyen Luna Phipps-Costin Donald Pinckney Ming-Ho Yee Yangtian Zi Carolyn\u00a0Jane Anderson Molly\u00a0Q Feldman et\u00a0al. 2022. Multipl-e: A scalable and extensible approach to benchmarking neural code generation. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2208.08227 (2022)."},{"key":"e_1_3_3_1_11_2","unstructured":"Lingjiao Chen Matei Zaharia and James Zou. 2023. Frugalgpt: How to use large language models while reducing cost and improving performance. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2305.05176 (2023)."},{"key":"e_1_3_3_1_12_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde De\u00a0Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et\u00a0al. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2107.03374 (2021)."},{"key":"e_1_3_3_1_13_2","unstructured":"Shuhao Chen Weisen Jiang Baijiong Lin James Kwok and Yu Zhang. 2024. RouterDC: Query-based router by dual contrastive learning for assembling large language models. Advances in Neural Information Processing Systems 37 (2024) 66305\u201366328."},{"key":"e_1_3_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939785"},{"key":"e_1_3_3_1_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/BigComp.2018.00044"},{"key":"e_1_3_3_1_16_2","doi-asserted-by":"crossref","unstructured":"Christof Ebert James Cain Giuliano Antoniol Steve Counsell and Phillip Laplante. 2016. Cyclomatic complexity. IEEE software 33 6 (2016) 27\u201329.","DOI":"10.1109\/MS.2016.147"},{"key":"e_1_3_3_1_17_2","doi-asserted-by":"crossref","unstructured":"Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng Ming Gong Linjun Shou Bing Qin Ting Liu Daxin Jiang et\u00a0al. 2020. Codebert: A pre-trained model for programming and natural languages. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2002.08155 (2020).","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"e_1_3_3_1_18_2","unstructured":"Github. 2021. Github Copilot. https:\/\/github.com\/features\/copilot"},{"key":"e_1_3_3_1_19_2","unstructured":"Aaron Grattafiori Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle Aiesha Letman Akhil Mathur Alan Schelten Alex Vaughan et\u00a0al. 2024. The llama 3 herd of models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2407.21783 (2024)."},{"key":"e_1_3_3_1_20_2","unstructured":"Daya Guo Dejian Yang Haowei Zhang Junxiao Song Ruoyu Zhang Runxin Xu Qihao Zhu Shirong Ma Peiyi Wang Xiao Bi et\u00a0al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2501.12948 (2025)."},{"key":"e_1_3_3_1_21_2","unstructured":"Daya Guo Qihao Zhu Dejian Yang Zhenda Xie Kai Dong Wentao Zhang Guanting Chen Xiao Bi Yu Wu YK Li et\u00a0al. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming\u2013The Rise of Code Intelligence. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2401.14196 (2024)."},{"key":"e_1_3_3_1_22_2","unstructured":"Neha Gupta Harikrishna Narasimhan Wittawat Jitkrittum Ankit\u00a0Singh Rawat Aditya\u00a0Krishna Menon and Sanjiv Kumar. 2024. Language model cascades: Token-level uncertainty and beyond. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2404.10136 (2024)."},{"key":"e_1_3_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICOEI.2017.8300883"},{"key":"e_1_3_3_1_24_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-24261-3_7"},{"key":"e_1_3_3_1_25_2","unstructured":"Binyuan Hui Jian Yang Zeyu Cui Jiaxi Yang Dayiheng Liu Lei Zhang Tianyu Liu Jiajun Zhang Bowen Yu Keming Lu et\u00a0al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2409.12186 (2024)."},{"key":"e_1_3_3_1_26_2","doi-asserted-by":"crossref","unstructured":"Soyeong Jeong Jinheon Baek Sukmin Cho Sung\u00a0Ju Hwang and Jong\u00a0C Park. 2024. Adaptive-rag: Learning to adapt retrieval-augmented large language models through question complexity. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2403.14403 (2024).","DOI":"10.18653\/v1\/2024.naacl-long.389"},{"key":"e_1_3_3_1_27_2","doi-asserted-by":"crossref","unstructured":"Dongfu Jiang Xiang Ren and Bill\u00a0Yuchen Lin. 2023. Llm-blender: Ensembling large language models with pairwise ranking and generative fusion. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2306.02561 (2023).","DOI":"10.18653\/v1\/2023.acl-long.792"},{"key":"e_1_3_3_1_28_2","unstructured":"Carlos\u00a0E Jimenez John Yang Alexander Wettig Shunyu Yao Kexin Pei Ofir Press and Karthik Narasimhan. 2023. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2310.06770 (2023)."},{"key":"e_1_3_3_1_29_2","unstructured":"Bonan Kou Shengmai Chen Zhijie Wang Lei Ma and Tianyi Zhang. 2023. Is model attention aligned with human attention? an empirical study on large language models for code generation. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2306.01220 (2023)."},{"key":"e_1_3_3_1_30_2","doi-asserted-by":"crossref","unstructured":"Yujia Li David Choi Junyoung Chung Nate Kushman Julian Schrittwieser R\u00e9mi Leblond Tom Eccles James Keeling Felix Gimeno Agustin Dal\u00a0Lago et\u00a0al. 2022. Competition-level code generation with alphacode. Science 378 6624 (2022) 1092\u20131097.","DOI":"10.1126\/science.abq1158"},{"key":"e_1_3_3_1_31_2","unstructured":"Aixin Liu Bei Feng Bin Wang Bingxuan Wang Bo Liu Chenggang Zhao Chengqi Dengr Chong Ruan Damai Dai Daya Guo et\u00a0al. 2024. Deepseek-v2: A strong economical and efficient mixture-of-experts language model. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2405.04434 (2024)."},{"key":"e_1_3_3_1_32_2","unstructured":"Aixin Liu Bei Feng Bing Xue Bingxuan Wang Bochao Wu Chengda Lu Chenggang Zhao Chengqi Deng Chenyu Zhang Chong Ruan et\u00a0al. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2412.19437 (2024)."},{"key":"e_1_3_3_1_33_2","unstructured":"Jiawei Liu Chunqiu\u00a0Steven Xia Yuyao Wang and Lingming Zhang. 2023. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in Neural Information Processing Systems 36 (2023) 21558\u201321572."},{"key":"e_1_3_3_1_34_2","unstructured":"Siyao Liu He Zhu Jerry Liu Shulin Xin Aoyan Li Rui Long Li Chen Jack Yang Jinxiang Xia ZY Peng et\u00a0al. 2024. Fullstack bench: Evaluating llms as full stack coder. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2412.00535 (2024)."},{"key":"e_1_3_3_1_35_2","unstructured":"Anton Lozhkov Raymond Li Loubna\u00a0Ben Allal Federico Cassano Joel Lamy-Poirier Nouamane Tazi Ao Tang Dmytro Pykhtar Jiawei Liu Yuxiang Wei et\u00a0al. 2024. Starcoder 2 and the stack v2: The next generation. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2402.19173 (2024)."},{"key":"e_1_3_3_1_36_2","unstructured":"Haipeng Luo Qingfeng Sun Can Xu Pu Zhao Jianguang Lou Chongyang Tao Xiubo Geng Qingwei Lin Shifeng Chen and Dongmei Zhang. 2023. Wizardmath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2308.09583 (2023)."},{"key":"e_1_3_3_1_37_2","doi-asserted-by":"crossref","unstructured":"Alex Mallen Akari Asai Victor Zhong Rajarshi Das Daniel Khashabi and Hannaneh Hajishirzi. 2022. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2212.10511 (2022).","DOI":"10.18653\/v1\/2023.acl-long.546"},{"key":"e_1_3_3_1_38_2","volume-title":"NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following","author":"Muennighoff Niklas","year":"2023","unstructured":"Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry\u00a0Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro Von\u00a0Werra, and Shayne Longpre. 2023. Octopack: Instruction tuning code large language models. In NeurIPS 2023 Workshop on Instruction Tuning and Instruction Following."},{"key":"e_1_3_3_1_39_2","unstructured":"Aaron van\u00a0den Oord Yazhe Li and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/1807.03748 (2018)."},{"key":"e_1_3_3_1_40_2","unstructured":"OpenAI. 2024. OpenAI o1. https:\/\/openai.com\/o1\/"},{"key":"e_1_3_3_1_41_2","unstructured":"OpenAI. 2025. OpenAI o3mini. https:\/\/openai.com\/index\/openai-o3-mini\/"},{"key":"e_1_3_3_1_42_2","first-page":"26619","volume-title":"International Conference on Machine Learning","author":"Orlanski Gabriel","year":"2023","unstructured":"Gabriel Orlanski, Kefan Xiao, Xavier Garcia, Jeffrey Hui, Joshua Howland, Jonathan Malmaud, Jacob Austin, Rishabh Singh, and Michele Catasta. 2023. Measuring the impact of programming language distribution. In International Conference on Machine Learning. PMLR, 26619\u201326645."},{"key":"e_1_3_3_1_43_2","doi-asserted-by":"crossref","unstructured":"Shuyin Ouyang Jie\u00a0M Zhang Mark Harman and Meng Wang. 2025. An empirical study of the non-determinism of chatgpt in code generation. ACM Transactions on Software Engineering and Methodology 34 2 (2025) 1\u201328.","DOI":"10.1145\/3697010"},{"key":"e_1_3_3_1_44_2","unstructured":"Baptiste Roziere Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat Xiaoqing\u00a0Ellen Tan Yossi Adi Jingyu Liu Romain Sauvestre Tal Remez et\u00a0al. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2308.12950 (2023)."},{"key":"e_1_3_3_1_45_2","unstructured":"Gemini Team Petko Georgiev Ving\u00a0Ian Lei Ryan Burnell Libin Bai Anmol Gulati Garrett Tanzer Damien Vincent Zhufeng Pan Shibo Wang et\u00a0al. 2024. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2403.05530 (2024)."},{"key":"e_1_3_3_1_46_2","unstructured":"Gemma Team Thomas Mesnard Cassidy Hardin Robert Dadashi Surya Bhupatiraju Shreya Pathak Laurent Sifre Morgane Rivi\u00e8re Mihir\u00a0Sanjay Kale Juliette Love et\u00a0al. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2403.08295 (2024)."},{"key":"e_1_3_3_1_47_2","unstructured":"Kimi Team Angang Du Bofei Gao Bowei Xing Changjiu Jiang Cheng Chen Cheng Li Chenjun Xiao Chenzhuang Du Chonghua Liao et\u00a0al. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2501.12599 (2025)."},{"key":"e_1_3_3_1_48_2","unstructured":"Mistral\u00a0AI team. 2024. Codestral. https:\/\/mistral.ai\/news\/codestral"},{"key":"e_1_3_3_1_49_2","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale et\u00a0al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2307.09288 (2023)."},{"key":"e_1_3_3_1_50_2","unstructured":"Chung-Yu Wang Alireza DaghighFarsoodeh and Hung\u00a0Viet Pham. 2024. Selection of Prompt Engineering Techniques for Code Generation through Predicting Code Complexity. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2409.16416 (2024)."},{"key":"e_1_3_3_1_51_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510159"},{"key":"e_1_3_3_1_52_2","unstructured":"Kurt\u00a0D Welker. 2001. The software maintainability index revisited. CrossTalk 14 (2001) 18\u201321."},{"key":"e_1_3_3_1_53_2","unstructured":"An Yang Baosong Yang Beichen Zhang Binyuan Hui Bo Zheng Bowen Yu Chengyuan Li Dayiheng Liu Fei Huang Haoran Wei et\u00a0al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2412.15115 (2024)."},{"key":"e_1_3_3_1_54_2","unstructured":"An Yang Beichen Zhang Binyuan Hui Bofei Gao Bowen Yu Chengpeng Li Dayiheng Liu Jianhong Tu Jingren Zhou Junyang Lin et\u00a0al. 2024. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2409.12122 (2024)."},{"key":"e_1_3_3_1_55_2","unstructured":"Alex Young Bei Chen Chao Li Chengen Huang Ge Zhang Guanwei Zhang Guoyin Wang Heng Li Jiangcheng Zhu Jianqun Chen et\u00a0al. 2024. Yi: Open foundation models by 01. ai. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2403.04652 (2024)."},{"key":"e_1_3_3_1_56_2","unstructured":"Longhui Yu Weisen Jiang Han Shi Jincheng Yu Zhengying Liu Yu Zhang James\u00a0T Kwok Zhenguo Li Adrian Weller and Weiyang Liu. 2023. Metamath: Bootstrap your own mathematical questions for large language models. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2309.12284 (2023)."},{"key":"e_1_3_3_1_57_2","unstructured":"Murong Yue Jie Zhao Min Zhang Liang Du and Ziyu Yao. 2023. Large language model cascades with mixture of thoughts representations for cost-efficient reasoning. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2310.03094 (2023)."},{"key":"e_1_3_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3580305.3599790"},{"key":"e_1_3_3_1_59_2","unstructured":"Qihao Zhu Daya Guo Zhihong Shao Dejian Yang Peiyi Wang Runxin Xu Y Wu Yukun Li Huazuo Gao Shirong Ma et\u00a0al. 2024. Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2406.11931 (2024)."}],"event":{"name":"Internetware 2025: the 16th International Conference on Internetware","location":"Trondheim Norway","acronym":"Internetware 2025","sponsor":["SIGSOFT ACM Special Interest Group on Artificial Intelligence"]},"container-title":["Proceedings of the 16th International Conference on Internetware"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3755881.3755925","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,27]],"date-time":"2025-10-27T11:48:03Z","timestamp":1761565683000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3755881.3755925"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,20]]},"references-count":58,"alternative-id":["10.1145\/3755881.3755925","10.1145\/3755881"],"URL":"https:\/\/doi.org\/10.1145\/3755881.3755925","relation":{},"subject":[],"published":{"date-parts":[[2025,6,20]]},"assertion":[{"value":"2025-10-27","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}