{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,1]],"date-time":"2026-07-01T04:01:40Z","timestamp":1782878500659,"version":"3.54.5"},"reference-count":58,"publisher":"Association for Computing Machinery (ACM)","issue":"2","funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62372232"],"award-info":[{"award-number":["62372232"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Short-term Visiting Program of Nanjing University of Aeronautics and Astronautics for Ph.D. Students Abroad","award":["240602DF16"],"award-info":[{"award-number":["240602DF16"]}]},{"name":"High Performance Computing Platform of Nanjing University of Aeronautics and Astronautics, and the Collaborative Innovation Center of Novel Software Technology and Industrialization"},{"name":"State Key Laboratory of Novel Software Technology, Nanjing University","award":["KFKT2022A03, KFKT2023A04"],"award-info":[{"award-number":["KFKT2022A03, KFKT2023A04"]}]},{"name":"A*STAR under its 2nd CSIRO and A*STAR: Research-Industry (2+2) Partnership Program","award":["R24I5IR047"],"award-info":[{"award-number":["R24I5IR047"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2026,2,28]]},"abstract":"<jats:p>The widespread use of Large Language Models (LLMs) in software engineering has intensified the need for improved model and resource efficiency. In particular, for neural code generation, LLMs are used to translate function\/method signature and DocString to executable code. DocStrings, which capture user requirements for the code and are typically used as the prompt for LLMs, often contain redundant information. Recent advancements in prompt compression have shown promising results in Natural Language Processing (NLP), but their applicability to code generation remains uncertain. Our empirical study shows that the state-of-the-art prompt compression methods achieve only about 10% reduction, as further reductions would cause significant performance degradation. In our study, we propose a novel compression method, ShortenDoc, dedicated to DocString compression for code generation. Our experiments on six code generation datasets, five open source LLMs (1B to 10B parameters), and one closed-source LLM GPT-4o confirm that ShortenDoc achieves 25\u201340% compression while preserving the quality of generated code, outperforming other baseline methods at similar compression levels. The benefit of this method is to improve efficiency and reduce the token processing cost while maintaining the quality of the generated code, especially when calling third-party APIs.<\/jats:p>","DOI":"10.1145\/3735636","type":"journal-article","created":{"date-parts":[[2025,5,14]],"date-time":"2025-05-14T12:24:27Z","timestamp":1747225467000},"page":"1-31","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":4,"title":["Less Is More: DocString Compression in Code Generation"],"prefix":"10.1145","volume":"35","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3374-6680","authenticated-orcid":false,"given":"Guang","family":"Yang","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology\/College of Artificial Intelligence\/College of Software, Nanjing University of Aeronautics and Astronautics, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3723-7584","authenticated-orcid":false,"given":"Yu","family":"Zhou","sequence":"additional","affiliation":[{"name":"Nanjing University of Aeronautics and Astronautics, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-9418-5753","authenticated-orcid":false,"given":"Wei","family":"Cheng","sequence":"additional","affiliation":[{"name":"Nanjing University of Aeronautics and Astronautics, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-6271-746X","authenticated-orcid":false,"given":"Xiangyu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Nanjing University of Aeronautics and Astronautics, Nanjing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1180-3891","authenticated-orcid":false,"given":"Xiang","family":"Chen","sequence":"additional","affiliation":[{"name":"Nantong University, Nantong, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5760-5188","authenticated-orcid":false,"given":"Terry Yue","family":"Zhuo","sequence":"additional","affiliation":[{"name":"Monash University, Clayton, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5224-1485","authenticated-orcid":false,"given":"Ke","family":"Liu","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4558-0622","authenticated-orcid":false,"given":"Xin","family":"Zhou","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4367-7201","authenticated-orcid":false,"given":"David","family":"Lo","sequence":"additional","affiliation":[{"name":"Singapore Management University, Singapore, Singapore"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5993-1665","authenticated-orcid":false,"given":"Taolue","family":"Chen","sequence":"additional","affiliation":[{"name":"Birkbeck University of London, London, United Kingdom of Great Britain and Northern Ireland"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,1,21]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray and Kai-Wei Chang. 2021. Unified pre-training for program understanding and generation. arXiv:2103.06333. Retrieved from https:\/\/arxiv.org\/abs\/2103.06333"},{"key":"e_1_3_2_3_2","unstructured":"Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski David Dohan Ellen Jiang Carrie Cai Michael Terry Quoc Le et al. 2021. Program synthesis with large language models. arXiv:2108.07732. Retrieved from https:\/\/arxiv.org\/abs\/2108.07732"},{"key":"e_1_3_2_4_2","unstructured":"Jinze Bai Shuai Bai Yunfei Chu Zeyu Cui Kai Dang Xiaodong Deng Yang Fan Wenbin Ge Yu Han Fei Huang et al. 2023. Qwen technical report. arXiv:2309.16609. Retrieved from https:\/\/arxiv.org\/abs\/2309.16609"},{"key":"e_1_3_2_5_2","unstructured":"Tom B. Brown. 2020. Language models are few-shot learners. arXiv:2005.14165. Retrieved from https:\/\/arxiv.org\/abs\/2005.14165"},{"key":"e_1_3_2_6_2","article-title":"Code alpaca: An instruction-following Llama model for code generation","author":"Chaudhary Sahil","year":"2023","unstructured":"Sahil Chaudhary. 2023. Code alpaca: An instruction-following Llama model for code generation. GitHub Repository. Retrieved from https:\/\/github.com\/sahil280114\/codealpaca","journal-title":"GitHub Repository"},{"key":"e_1_3_2_7_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde De Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et al. 2021. Evaluating large language models trained on code. arXiv:2107.03374. Retrieved from https:\/\/arxiv.org\/abs\/2107.03374"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/3663529.3663801"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.eacl-srw.24"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3652156"},{"key":"e_1_3_2_11_2","unstructured":"Abhimanyu Dubey Abhinav Jauhri Abhinav Pandey Abhishek Kadian Ahmad Al-Dahle Aiesha Letman Akhil Mathur Alan Schelten Amy Yang Angela Fan et al. 2024. The Llama 3 herd of models. arXiv:2407.21783. Retrieved from https:\/\/arxiv.org\/abs\/2407.21783"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3664646.3664765"},{"key":"e_1_3_2_13_2","unstructured":"Flab-Pruner. 2024. Flab-Pruner: Towards Greener yet Powerful Code Intelligence via Structural Pruning. Retrieved from https:\/\/github.com\/Flab-Pruner\/Flab-Pruner"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0031619"},{"key":"e_1_3_2_15_2","unstructured":"Daya Guo Qihao Zhu Dejian Yang Zhenda Xie Kai Dong Wentao Zhang Guanting Chen Xiao Bi Y. Wu Y. K. Li et al. 2024. DeepSeek-Coder: When the large language model meets programming\u2013The rise of code intelligence. arXiv:2401.14196. Retrieved from https:\/\/arxiv.org\/abs\/2401.14196"},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.825"},{"key":"e_1_3_2_17_2","unstructured":"Huiqiang Jiang Qianhui Wu Xufang Luo Dongsheng Li Chin-Yew Lin Yuqing Yang and Lili Qiu. 2023. LongLLMLingua: Accelerating and enhancing LLMs in long context scenarios via prompt compression. arXiv:2310.06839. Retrieved from https:\/\/arxiv.org\/abs\/2310.06839"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3600006.3613165"},{"issue":"8","key":"e_1_3_2_19_2","first-page":"1","article-title":"AceCoder: An effective prompting technique specialized in code generation","volume":"33","author":"Li Jia","year":"2024","unstructured":"Jia Li, Yunfei Zhao, Yongmin Li, Ge Li, and Zhi Jin. 2024. AceCoder: An effective prompting technique specialized in code generation. ACM Transactions on Software Engineering and Methodology 33, 8 (2024), 1\u201326.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_20_2","unstructured":"Raymond Li Loubna Ben Allal Yangtian Zi Niklas Muennighoff Denis Kocetkov Chenghao Mou Marc Marone Christopher Akiki Jia Li Jenny Chim et al. 2023. Starcoder: May the source be with you! arXiv:2305.06161. Retrieved from https:\/\/arxiv.org\/abs\/2305.06161"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.abq1158"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.391"},{"key":"e_1_3_2_23_2","unstructured":"Wang Ling Edward Grefenstette Karl Moritz Hermann Tom\u00e1\u0161 Ko\u010disk\u1ef3 Andrew Senior Fumin Wang and Phil Blunsom. 2016. Latent predictor networks for code generation. arXiv:1603.06744. Retrieved from https:\/\/arxiv.org\/abs\/1603.06744"},{"key":"e_1_3_2_24_2","article-title":"Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation","author":"Liu Jiawei","year":"2024","unstructured":"Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2024. Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 36.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_25_2","volume-title":"Proceedings of the 35th Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)","author":"Lu Shuai","year":"2021","unstructured":"Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. 2021. CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. In Proceedings of the 35th Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)."},{"key":"e_1_3_2_26_2","unstructured":"Ziyang Luo Can Xu Pu Zhao Qingfeng Sun Xiubo Geng Wenxiang Hu Chongyang Tao Jing Ma Qingwei Lin and Daxin Jiang. 2023. Wizardcoder: Empowering code large language models with Evol-Instruct. arXiv:2306.08568. Retrieved from https:\/\/arxiv.org\/abs\/2306.08568"},{"key":"e_1_3_2_27_2","first-page":"314","volume-title":"Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 2: Short Papers)","author":"Valerio Miceli-Barone Antonio","year":"2017","unstructured":"Antonio Valerio Miceli-Barone and Rico Sennrich. 2017. A parallel corpus of Python functions and documentation strings for automated code documentation and code generation. In Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), 314\u2013319."},{"key":"e_1_3_2_28_2","doi-asserted-by":"crossref","unstructured":"Ali Modarressi Hosein Mohebbi and Mohammad Taher Pilehvar. 2022. AdapLeR: Speeding up inference by adaptive length reduction. arXiv:2203.08991. Retrieved from https:\/\/arxiv.org\/abs\/2203.08991","DOI":"10.18653\/v1\/2022.acl-long.1"},{"key":"e_1_3_2_29_2","unstructured":"Lili Mou Rui Men Ge Li Lu Zhang and Zhi Jin. 2015. On end-to-end program generation from user intention by deep neural networks. arXiv:1510.07211. Retrieved from https:\/\/arxiv.org\/abs\/1510.07211"},{"key":"e_1_3_2_30_2","article-title":"Learning to compress prompts with gist tokens","author":"Mu Jesse","year":"2024","unstructured":"Jesse Mu, Xiang Li, and Noah Goodman. 2024. Learning to compress prompts with gist tokens. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 36.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639187"},{"key":"e_1_3_2_32_2","unstructured":"Arvind Neelakantan Tao Xu Raul Puri Alec Radford Jesse Michael Han Jerry Tworek Qiming Yuan Nikolas Tezak Jong Wook Kim Chris Hallacy et al. 2022. Text and code embeddings by contrastive pre-training. arXiv:2201.10005. Retrieved from https:\/\/arxiv.org\/abs\/2201.10005"},{"key":"e_1_3_2_33_2","unstructured":"Erik Nijkamp Bo Pang Hiroaki Hayashi Lifu Tu Huan Wang Yingbo Zhou Silvio Savarese and Caiming Xiong. 2022. CodeGen: An open large language model for code with multi-turn program synthesis. arXiv:2203.13474. Retrieved from https:\/\/arxiv.org\/abs\/2203.13474"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650105.3652295"},{"key":"e_1_3_2_35_2","unstructured":"OpenAI. 2022. ChatGPT. Retrieved from https:\/\/openai.com\/blog\/chatgpt"},{"key":"e_1_3_2_36_2","first-page":"27730","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang Long","year":"2022","unstructured":"Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 35, 27730\u201327744.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-acl.57"},{"key":"e_1_3_2_38_2","unstructured":"Bibek Poudel Adam Cook Sekou Traore and Shelah Ameli. 2024. DocuMint: Docstring generation for Python using small language models. arXiv:2405.10243. Retrieved from https:\/\/arxiv.org\/abs\/2405.10243"},{"key":"e_1_3_2_39_2","unstructured":"Alec Radford Karthik Narasimhan Tim Salimans and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. OpenAI Blog."},{"issue":"8","key":"e_1_3_2_40_2","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"key":"e_1_3_2_41_2","article-title":"Free-Marginal Multirater Kappa (Multirater K [Free]): An Alternative to Fleiss\u2019 Fixed-Marginal Multirater Kappa","author":"Randolph Justus J.","year":"2005","unstructured":"Justus J. Randolph. 2005. Free-Marginal Multirater Kappa (Multirater K [Free]): An Alternative to Fleiss\u2019 Fixed-Marginal Multirater Kappa. In Presented at the Joensuu Learning and Instruction Symposium.","journal-title":"Presented at the Joensuu Learning and Instruction Symposium"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643795.3648379"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1145\/3639475.3640097"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639120"},{"issue":"1","key":"e_1_3_2_45_2","first-page":"1","article-title":"Don\u2019t complete it! Preventing unhelpful code completion for productive and sustainable neural code completion systems","volume":"34","author":"Sun Zhensu","year":"2024","unstructured":"Zhensu Sun, Xiaoning Du, Fu Song, Shangwen Wang, Mingze Ni, Li Li, and David Lo. 2024. Don\u2019t complete it! Preventing unhelpful code completion for productive and sustainable neural code completion systems. ACM Transactions on Software Engineering and Methodology 34, 1 (2024), 1\u201322.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3650212.3680347"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6430"},{"key":"e_1_3_2_48_2","first-page":"5998","volume-title":"Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017. Isabelle Guyon, Ulrike Von Luxburg","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017. Isabelle Guyon, Ulrike Von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.), 5998\u20136008. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.773"},{"key":"e_1_3_2_50_2","doi-asserted-by":"crossref","unstructured":"Yue Wang Hung Le Akhilesh Deepak Gotmare Nghi D. Q. Bui Junnan Li and Steven C. H. Hoi.2023. Codet5+: Open code large language models for code understanding and generation. arXiv:2305.07922. Retrieved from https:\/\/arxiv.org\/abs\/2305.07922","DOI":"10.18653\/v1\/2023.emnlp-main.68"},{"key":"e_1_3_2_51_2","doi-asserted-by":"crossref","unstructured":"Yue Wang Weishi Wang Shafiq Joty and Steven C. H. Hoi. 2021. Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. arXiv:2109.00859. Retrieved from https:\/\/arxiv.org\/abs\/2109.00859","DOI":"10.18653\/v1\/2021.emnlp-main.685"},{"key":"e_1_3_2_52_2","unstructured":"Jason Wei Maarten Bosma Vincent Y. Zhao Kelvin Guu Adams Wei Yu Brian Lester Nan Du Andrew M. Dai and Quoc V. Le. 2021. Finetuned language models are zero-shot learners. arXiv:2109.01652. Retrieved from https:\/\/arxiv.org\/abs\/2109.01652"},{"key":"e_1_3_2_53_2","doi-asserted-by":"crossref","unstructured":"David Wingate Mohammad Shoeybi and Taylor Sorensen. 2022. Prompt compression and contrastive conditioning for controllability and toxicity reduction in language models. arXiv:2210.03162. Retrieved from https:\/\/arxiv.org\/abs\/2210.03162","DOI":"10.18653\/v1\/2022.findings-emnlp.412"},{"key":"e_1_3_2_54_2","unstructured":"Chunqiu Steven Xia Yinlin Deng and Lingming Zhang. 2024. Top leaderboard ranking= top coding proficiency always? EvoEval: Evolving coding benchmarks via LLM. arXiv:2403.19114. Retrieved from https:\/\/arxiv.org\/abs\/2403.19114"},{"issue":"3","key":"e_1_3_2_55_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3630010","article-title":"How important are good method names in neural code generation? A model robustness perspective","volume":"33","author":"Yang Guang","year":"2024","unstructured":"Guang Yang, Yu Zhou, Wenhua Yang, Tao Yue, Xiang Chen, and Taolue Chen. 2024. How important are good method names in neural code generation? A model robustness perspective. ACM Transactions on Software Engineering and Methodology 33, 3 (2024), 1\u201335.","journal-title":"ACM Transactions on Software Engineering and Methodology"},{"key":"e_1_3_2_56_2","unstructured":"Zhou Yang Zhensu Sun Terry Zhuo Yue Premkumar Devanbu and David Lo. 2024. Robustness security privacy explainability efficiency and usability of large language models for code. arXiv:2403.07506. Retrieved from https:\/\/arxiv.org\/abs\/2403.07506"},{"key":"e_1_3_2_57_2","unstructured":"Pengcheng Yin and Graham Neubig. 2017. A syntactic neural model for general-purpose code generation. arXiv:1704.01696. Retrieved from https:\/\/arxiv.org\/abs\/1704.01696"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1145\/3580305.3599790"},{"key":"e_1_3_2_59_2","unstructured":"Terry Yue Zhuo Minh Chien Vu Jenny Chim Han Hu Wenhao Yu Ratnadira Widyasari Imam Nur Bani Yusuf Haolan Zhan Junda He Indraneil Paul et al. 2024. BigCodeBench: Benchmarking code generation with diverse function calls and complex instructions. arXiv:2406.15877. Retrieved from https:\/\/arxiv.org\/abs\/2406.15877"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3735636","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T16:31:52Z","timestamp":1769013112000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3735636"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,21]]},"references-count":58,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,2,28]]}},"alternative-id":["10.1145\/3735636"],"URL":"https:\/\/doi.org\/10.1145\/3735636","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,21]]},"assertion":[{"value":"2024-10-30","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-06","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-21","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}