{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,16]],"date-time":"2026-10-16T08:39:36Z","timestamp":1792139976128,"version":"4.4.0"},"reference-count":161,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2025,4,28]],"date-time":"2025-04-28T00:00:00Z","timestamp":1745798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Union-NextGenerationEU through the Italian Ministry of University and Research","award":["2022B3BP5S"],"award-info":[{"award-number":["2022B3BP5S"]}]},{"name":"QualAI: Continuous Quality Improvement of AI-Based Systems","award":["2022B3BP5S"],"award-info":[{"award-number":["2022B3BP5S"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,5,31]]},"abstract":"<jats:p>Large Language Models (LLMs) are increasingly becoming fundamental in supporting software developers in coding tasks. The massive datasets used for training LLMs are often collected automatically, leading to the introduction of data smells. Previous work addressed this issue by using quality filters to handle some specific smells. Still, the literature lacks a systematic catalog of the data smells for coding tasks currently known. This article presents a Systematic Literature Review (SLR) focused on articles that introduce LLMs for coding tasks. We first extracted the quality filters adopted for training and testing such LLMs, inferred the root problem behind their adoption (data smells for coding tasks), and defined a taxonomy of such smells. Our results highlight discrepancies in the adoption of quality filters between pre-training and fine-tuning stages and across different coding tasks, shedding light on areas for improvement in LLM-based software development support.<\/jats:p>","DOI":"10.1145\/3707457","type":"journal-article","created":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T14:56:41Z","timestamp":1733842601000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":8,"title":["A Catalog of Data Smells for Coding Tasks"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-5840-9777","authenticated-orcid":false,"given":"Antonio","family":"Vitale","sequence":"first","affiliation":[{"name":"Automatica e Informatica, Politecnico di Torino, Torino, Italy and Department of Biosciences and Territory, University of Molise, Pesche, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7995-8582","authenticated-orcid":false,"given":"Rocco","family":"Oliveto","sequence":"additional","affiliation":[{"name":"Department of Biosciences and Territory, University of Molise, Pesche, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1764-9685","authenticated-orcid":false,"given":"Simone","family":"Scalabrino","sequence":"additional","affiliation":[{"name":"Department of Biosciences and Territory, University of Molise, Pesche, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,4,28]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"ACM Digital Library. 2024. Retrieved from https:\/\/dl.acm.org\/"},{"key":"e_1_3_2_3_2","unstructured":"FastText. 2022. Retrieved from https:\/\/fasttext.cc\/docs\/en\/language-identification.html"},{"key":"e_1_3_2_4_2","unstructured":"GitHub Code Dataset. 2022. Retrieved from https:\/\/huggingface.co\/datasets\/codeparrot\/github-code"},{"key":"e_1_3_2_5_2","unstructured":"IEEE Xplore Digital Library. 2024. Retrieved from https:\/\/ieeexplore.ieee.org\/"},{"key":"e_1_3_2_6_2","unstructured":"Langid.py: Standalone Language Identification (LangID) Tool. 2011. Retrieved from https:\/\/github.com\/saffsd\/langid.py"},{"key":"e_1_3_2_7_2","unstructured":"pycld3: Python3 Bindings for the Compact Language Detector v3 (CLD3). 2004. Retrieved from https:\/\/github.com\/bsolomon1124\/pycld3"},{"key":"e_1_3_2_8_2","unstructured":"Scopus. 2024. Retrieved from https:\/\/www.scopus.com\/"},{"key":"e_1_3_2_9_2","unstructured":"Springer Link Online Library. 2024. Retrieved from https:\/\/link.springer.com\/"},{"key":"e_1_3_2_10_2","unstructured":"Josh Achiam Steven Adler Sandhini Agarwal Lama Ahmad Ilge Akkaya Florencia Leoni Aleman Diogo Almeida Janko Altenschmidt Sam Altman Shyamal Anadkat et al. 2023. Gpt-4 technical report. arXiv:2303.08774. Retrieved from https:\/\/arxiv.org\/abs\/2303.08774"},{"key":"e_1_3_2_11_2","doi-asserted-by":"crossref","unstructured":"Qurat Ul Ain Wasi Haider Butt Muhammad Waseem Anwar Farooque Azam and Bilal Maqbool. 2019. A systematic review on code clone detection. IEEE Access 7 (2019) 86121\u201386144.","DOI":"10.1109\/ACCESS.2019.2918202"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1145\/3359591.3359735"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC52881.2021.00032"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/2901739.2901742"},{"key":"e_1_3_2_15_2","unstructured":"Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D. Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems 1877\u20131901."},{"key":"e_1_3_2_16_2","unstructured":"Nicholas Carlini Florian Tramer Eric Wallace Matthew Jagielski Ariel Herbert-Voss Katherine Lee Adam Roberts Tom Brown Dawn Song Ulfar Erlingsson et al. 2021. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security \u201921) 2633\u20132650."},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","unstructured":"Roland Croft Yongzheng Xie and Muhammad Ali Babar. 2022. Data preparation for software vulnerability prediction: A systematic literature review. IEEE Transactions on Software Engineering 49 3 (2022) 1044\u20131063.","DOI":"10.1109\/TSE.2022.3171202"},{"key":"e_1_3_2_18_2","unstructured":"Anh T. V. Dau Nghi D. Q. Bui and Jin L. C. Guo. 2023. Bootstrapping code-text pretrained language model to detect inconsistency between code and comment. arXiv:2306.06347. Retrieved from https:\/\/arxiv.org\/abs\/2306.06347"},{"key":"e_1_3_2_19_2","unstructured":"Jacob Devlin Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805. Retrieved from https:\/\/arxiv.org\/abs\/1810.04805"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","unstructured":"Zhangyin Feng Daya Guo Duyu Tang Nan Duan Xiaocheng Feng Ming Gong Linjun Shou Bing Qin Ting Liu Daxin Jiang et al. 2020. CodeBERT: A pre-trained model for programming and natural languages. arXiv:2002.08155. Retrieved from https:\/\/arxiv.org\/abs\/2002.08155","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1145\/3522664.3528590"},{"key":"e_1_3_2_22_2","unstructured":"Daya Guo Shuo Ren Shuai Lu Zhangyin Feng Duyu Tang Shujie Liu Long Zhou Nan Duan Alexey Svyatkovskiy Shengyu Fu et al. 2020. GraphCodeBERT: Pre-training code representations with data flow. arXiv:2009.08366. Retrieved from https:\/\/arxiv.org\/abs\/2009.08366"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1002\/9781118646106"},{"key":"e_1_3_2_24_2","doi-asserted-by":"crossref","unstructured":"Abram Hindle Earl T. Barr Mark Gabel Zhendong Su and Premkumar Devanbu. 2016. On the naturalness of software. Communications of the ACM 59 5 (2016) 122\u2013131.","DOI":"10.1145\/2902362"},{"key":"e_1_3_2_25_2","unstructured":"Geoffrey Hinton. 2015. Distilling the knowledge in a neural network. arXiv:1503.02531. Retrieved from https:\/\/arxiv.org\/abs\/1503.02531"},{"key":"e_1_3_2_26_2","unstructured":"Xinyi Hou Yanjie Zhao Yue Liu Zhou Yang Kailong Wang Li Li Xiapu Luo David Lo John Grundy and Haoyu Wang. 2023. Large language models for software engineering: A systematic literature review. arXiv:2308.10620. Retrieved from https:\/\/arxiv.org\/abs\/2308.10620"},{"key":"e_1_3_2_27_2","unstructured":"Hamel Husain Ho-Hsiang Wu Tiferet Gazit Miltiadis Allamanis and Marc Brockschmidt. 2019. CodeSearchNet challenge: Evaluating the state of semantic code search. arXiv:1909.09436. Retrieved from https:\/\/arxiv.org\/abs\/1909.09436"},{"key":"e_1_3_2_28_2","doi-asserted-by":"crossref","unstructured":"Nathalie Japkowicz and Shaju Stephen. 2002. The class imbalance problem: A systematic study. Intelligent Data Analysis 6 5 (2002) 429\u2013449.","DOI":"10.3233\/IDA-2002-6504"},{"key":"e_1_3_2_29_2","doi-asserted-by":"crossref","unstructured":"Justin M. Johnson and Taghi M. Khoshgoftaar. 2019. Survey on deep learning with class imbalance. Journal of Big Data 6 1 (2019) 1\u201354.","DOI":"10.1186\/s40537-019-0192-5"},{"key":"e_1_3_2_30_2","unstructured":"Armand Joulin. 2016. FastText.zip: Compressing text classification models. arXiv:1612.03651. Retrieved from https:\/\/arxiv.org\/abs\/1612.03651"},{"key":"e_1_3_2_31_2","unstructured":"Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing systematic literature reviews in software engineering."},{"key":"e_1_3_2_32_2","unstructured":"Denis Kocetkov Raymond Li Loubna Ben Allal Jia Li Chenghao Mou Carlos Mu\u00f1oz Ferrandis Yacine Jernite Margaret Mitchell Sean Hughes Thomas Wolf et al. 2022. The Stack: 3 TB of permissively licensed source code. arXiv:2211.15533. Retrieved from https:\/\/arxiv.org\/abs\/2211.15533"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature14539"},{"key":"e_1_3_2_34_2","unstructured":"Patrick Lewis Ethan Perez Aleksandra Piktus Fabio Petroni Vladimir Karpukhin Naman Goyal Heinrich K\u00fcttler Mike Lewis Wen-tau Yih Tim Rockt\u00e4schel et al. 2020. Retrieval-augmented generation for knowledge-intensive NLP tasks. In Advances in Neural Information Processing Systems 9459\u20139474."},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","unstructured":"T. Lin. 2017. Focal loss for dense object detection. arXiv:1708.02002. Retrieved from https:\/\/arxiv.org\/abs\/1708.02002","DOI":"10.1109\/ICCV.2017.324"},{"key":"e_1_3_2_36_2","unstructured":"Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy Ambrosio Blanco Colin Clement Dawn Drain Daxin Jiang Duyu Tang et al. 2021. CodeXGLUE: A machine learning benchmark dataset for code understanding and generation. arXiv:2102.04664. Retrieved from https:\/\/arxiv.org\/abs\/2102.04664"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00041"},{"key":"e_1_3_2_38_2","first-page":"311","volume-title":"40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In 40th Annual Meeting of the Association for Computational Linguistics, 311\u2013318."},{"key":"e_1_3_2_39_2","unstructured":"Fanchao Qi Yangyi Chen Mukai Li Yuan Yao Zhiyuan Liu and Maosong Sun. 2020. Onion: A simple and effective defense against textual backdoor attacks. arXiv:2011.10369. Retrieved from https:\/\/arxiv.org\/abs\/2011.10369"},{"key":"e_1_3_2_40_2","unstructured":"Alec Radford Karthik Narasimhan Tim Salimans and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. (2018)."},{"key":"e_1_3_2_41_2","unstructured":"Colin Raffel Noam Shazeer Adam Roberts Katherine Lee Sharan Narang Michael Matena Yanqi Zhou Wei Li and Peter J. Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research 21 1 (2020) 5485\u20135551."},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPR56361.2022.9956690"},{"key":"e_1_3_2_43_2","doi-asserted-by":"crossref","unstructured":"Gilberto Recupito Raimondo Rapacciuolo Dario Di Nucci and Fabio Palomba. 2024. Unmasking data secrets: An empirical investigation into data smells and their impact on data quality. In 3rd International Conference on AI Engineering - Software Engineering for AI 53\u201363.","DOI":"10.1145\/3644815.3644960"},{"key":"e_1_3_2_44_2","doi-asserted-by":"crossref","unstructured":"Neha Saini Sukhdip Singh and Suman. 2018. Code clones: Detection and management. Procedia Computer Science 132 (2018) 718\u2013727.","DOI":"10.1016\/j.procs.2018.05.080"},{"key":"e_1_3_2_45_2","unstructured":"Roei Schuster Congzheng Song Eran Tromer and Vitaly Shmatikov. 2021. You autocomplete me: Poisoning vulnerabilities in neural code completion. In 30th USENIX Security Symposium (USENIX Security \u201921) 1559\u20131575."},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510160"},{"key":"e_1_3_2_47_2","unstructured":"Brandon Tran Jerry Li and Aleksander Madry. 2018. Spectral signatures in backdoor attacks. In Advances in Neural Information Processing Systems 8011\u20138021."},{"key":"e_1_3_2_48_2","doi-asserted-by":"crossref","unstructured":"Michele Tufano Cody Watson Gabriele Bavota Massimiliano Di Penta Martin White and Denys Poshyvanyk. 2019. An empirical study on learning bug-fixing patches in the wild via neural machine translation. ACM Transactions on Software Engineering and Methodology 28 4 (2019) 1\u201329.","DOI":"10.1145\/3340544"},{"key":"e_1_3_2_49_2","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez \u0141ukasz Kaiser and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 6000\u20136010."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","unstructured":"Antonio Vitale Rocco Oliveto and Simone Scalabrino. 2024. Replication package of \u201ca catalog of data smells for coding tasks.\u201d (2024). DOI: 10.6084\/m9.figshare.25898650","DOI":"10.6084\/m9.figshare.25898650"},{"key":"e_1_3_2_51_2","doi-asserted-by":"crossref","unstructured":"Yue Wang Hung Le Akhilesh Deepak Gotmare Nghi D. Q. Bui Junnan Li and Steven C. H. Hoi. 2023. CodeT5+: Open code large language models for code understanding and generation. arXiv:2305.07922. Retrieved from https:\/\/arxiv.org\/abs\/2305.07922","DOI":"10.18653\/v1\/2023.emnlp-main.68"},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","unstructured":"Yue Wang Weishi Wang Shafiq Joty and Steven C. H. Hoi. 2021. CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. arXiv:2109.00859. Retrieved from https:\/\/arxiv.org\/abs\/2109.00859","DOI":"10.18653\/v1\/2021.emnlp-main.685"},{"key":"e_1_3_2_53_2","unstructured":"Jason Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter Fei Xia Ed Chi Quoc V. Le and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 24824\u201324837."},{"key":"e_1_3_2_54_2","unstructured":"Yan Xiao Xinyue Zuo Lei Xue Kailong Wang Jin Song Dong and Ivan Beschastnikh. 2023. Empirical study on transformer-based techniques for software engineering. arXiv:2310.00399. Retrieved from https:\/\/arxiv.org\/abs\/2310.00399"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3558959"},{"key":"e_1_3_2_56_2","doi-asserted-by":"crossref","unstructured":"Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray and Kai-Wei Chang. 2021. Unified pre-training for program understanding and generation. arXiv:2103.06333. Retrieved from https:\/\/arxiv.org\/abs\/2103.06333","DOI":"10.18653\/v1\/2021.naacl-main.211"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510049"},{"key":"e_1_3_2_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER56733.2023.00033"},{"key":"e_1_3_2_59_2","first-page":"780","volume-title":"International Conference on Machine Learning","author":"Berabi Berkay","year":"2021","unstructured":"Berkay Berabi, Jingxuan He, Veselin Raychev, and Martin Vechev. 2021. Tfix: Learning to fix coding errors with a text-to-text transformer. In International Conference on Machine Learning. PMLR, 780\u2013791."},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678559"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643991.3644923"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3527917"},{"key":"e_1_3_2_63_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde De Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et al. 2021. Evaluating large language models trained on code. arXiv:2107.03374. Retrieved from https:\/\/arxiv.org\/abs\/2107.03374"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3423769"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639184"},{"key":"e_1_3_2_66_2","unstructured":"Fenia Christopoulou Gerasimos Lampouras Milan Gritta Guchun Zhang Yinpeng Guo Zhongqi Li Qi Zhang Meng Xiao Bo Shen Lin Li et al. 2022. Pangu-Coder: Program synthesis with function-level language modeling. arXiv:2207.11280. Retrieved from https:\/\/arxiv.org\/abs\/2207.11280"},{"issue":"12","key":"e_1_3_2_67_2","first-page":"4818","article-title":"An empirical study on the usage of transformer models for code completion","volume":"48","author":"Ciniselli Matteo","year":"2021","unstructured":"Matteo Ciniselli, Nathan Cooper, Luca Pascarella, Antonio Mastropaolo, Emad Aghajani, Denys Poshyvanyk, Massimiliano Di Penta, and Gabriele Bavota. 2021. An empirical study on the usage of transformer models for code completion. IEEE Transactions on Software Engineering 48, 12 (2021), 4818\u20134837.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643916.3644411"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524842.3528440"},{"key":"e_1_3_2_70_2","doi-asserted-by":"crossref","unstructured":"Colin B. Clement Dawn Drain Jonathan Timcheck Alexey Svyatkovskiy and Neel Sundaresan. 2020. PyMT5: Multi-mode translation of natural language and Python code with transformers. arXiv:2010.03150. Retrieved from https:\/\/arxiv.org\/abs\/2010.03150","DOI":"10.18653\/v1\/2020.emnlp-main.728"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643916.3644416"},{"key":"e_1_3_2_72_2","doi-asserted-by":"crossref","unstructured":"Xi Ding Rui Peng Xiangping Chen Yuan Huang Jing Bian and Zibin Zheng. 2024. Do code summarization models process too much information? Function signature may be all that is needed. ACM Transactions on Software Engineering and Methodology 33 6 (2024) 1\u201335.","DOI":"10.1145\/3652156"},{"key":"e_1_3_2_73_2","unstructured":"Yangruibo Ding Zijian Wang Wasi Uddin Ahmad Murali Krishna Ramanathan Ramesh Nallapati Parminder Bhatia Dan Roth and Bing Xiang. 2022. CoCoMIC: Code completion by jointly modeling in-file and cross-file context. arXiv:2212.10007. Retrieved from https:\/\/arxiv.org\/abs\/2212.10007"},{"key":"e_1_3_2_74_2","doi-asserted-by":"crossref","unstructured":"Zishuo Ding Yiming Tang Xiaoyu Cheng Heng Li and Weiyi Shang. 2023. LoGenText-plus: Improving neural machine translation based logging texts generation with syntactic templates. ACM Transactions on Software Engineering and Methodology 33 2 (2023) 1\u201345.","DOI":"10.1145\/3624740"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460945.3464951"},{"key":"e_1_3_2_76_2","doi-asserted-by":"crossref","unstructured":"Luke Dramko Jeremy Lacomis Pengcheng Yin Ed Schwartz Miltiadis Allamanis Graham Neubig Bogdan Vasilescu and Claire Le Goues. 2023. DIRE and its data: Neural decompiled variable renamings with respect to software class. ACM Transactions on Software Engineering and Methodology 32 2 (2023) 1\u201334.","DOI":"10.1145\/3546946"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3422274"},{"key":"e_1_3_2_78_2","unstructured":"Daniel Fried Armen Aghajanyan Jessy Lin Sida Wang Eric Wallace Freda Shi Ruiqi Zhong Wen-tau Yih Luke Zettlemoyer and Mike Lewis. 2022. Incoder: A generative model for code infilling and synthesis. arXiv:2204.05999. Retrieved from https:\/\/arxiv.org\/abs\/2204.05999"},{"key":"e_1_3_2_79_2","doi-asserted-by":"crossref","unstructured":"Michael Fu Van Nguyen Chakkrit Tantithamthavorn Dinh Phung and Trung Le. 2024. Vision transformer inspired automated vulnerability repair. ACM Transactions on Software Engineering and Methodology 33 3 (2024) 1\u201329.","DOI":"10.1145\/3632746"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE48619.2023.00164"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639216"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2024.112083"},{"key":"e_1_3_2_83_2","unstructured":"Daya Guo Qihao Zhu Dejian Yang Zhenda Xie Kai Dong Wentao Zhang Guanting Chen Xiao Bi Yu Wu Y. K. Li et al. 2024. DeepSeek-Coder: When the large language model meets programming\u2013The rise of code intelligence. arXiv:2401.14196. Retrieved from https:\/\/arxiv.org\/abs\/2401.14196"},{"key":"e_1_3_2_84_2","doi-asserted-by":"crossref","unstructured":"Masum Hasan Tanveer Muttaqueen Abdullah Al Ishtiaq Kazi Sajeed Mehrab Md Mahim Anjum Haque Tahmid Hasan Wasi Uddin Ahmad Anindya Iqbal and Rifat Shahriyar. 2021. CoDesc: A large code-description parallel dataset. arXiv:2105.14220. Retrieved from https:\/\/arxiv.org\/abs\/2105.14220","DOI":"10.18653\/v1\/2021.findings-acl.18"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/3660785"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1145\/3660773"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510172"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639138"},{"key":"e_1_3_2_89_2","doi-asserted-by":"crossref","unstructured":"Kevin Jesse Premkumar T. Devanbu and Anand Sawant. 2022. Learning to predict user-defined types. IEEE Transactions on Software Engineering 49 4 (2022) 1508\u20131522.","DOI":"10.1109\/TSE.2022.3178945"},{"key":"e_1_3_2_90_2","unstructured":"Marie-Anne Lachaux Baptiste Roziere Lowik Chanussot and Guillaume Lample. 2020. Unsupervised translation of programming languages. arXiv:2006.03511. Retrieved from https:\/\/arxiv.org\/abs\/2006.03511"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME52107.2021.00024"},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1145\/3630008"},{"key":"e_1_3_2_93_2","unstructured":"Raymond Li Loubna Ben Allal Yangtian Zi Niklas Muennighoff Denis Kocetkov Chenghao Mou Marc Marone Christopher Akiki Jia Li Jenny Chim et al. 2023. StarCoder: May the source be with you! arXiv:2305.06161. Retrieved from https:\/\/arxiv.org\/abs\/2305.06161"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3560422"},{"key":"e_1_3_2_95_2","doi-asserted-by":"crossref","unstructured":"Zhihao Li Chuanyi Li Ze Tang Wanhong Huang Jidong Ge Bin Luo Vincent Ng Ting Wang Yucheng Hu and Xiaopeng Zhang. 2023. PTM-APIRec: Leveraging pre-trained models of source code in API recommendation. ACM Transactions on Software Engineering and Methodology 33 (3) (2023) 1\u201330.","DOI":"10.1145\/3632745"},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549081"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639091"},{"key":"e_1_3_2_98_2","doi-asserted-by":"crossref","unstructured":"Bo Lin Shangwen Wang Zhongxin Liu Yepang Liu Xin Xia and Xiaoguang Mao. 2023. CCT5: A code-change-oriented pre-trained model. arXiv:2305.10785. Retrieved from https:\/\/arxiv.org\/abs\/2305.10785","DOI":"10.1145\/3611643.3616339"},{"key":"e_1_3_2_99_2","doi-asserted-by":"crossref","unstructured":"Fang Liu Zhiyi Fu Ge Li Zhi Jin Hui Liu Yiyang Hao and Li Zhang. 2024. Non-autoregressive line-level code completion. ACM Transactions on Software Engineering and Methodology 33 5 (2024) 1\u201334.","DOI":"10.1145\/3649594"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510154"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00159"},{"key":"e_1_3_2_102_2","doi-asserted-by":"crossref","unstructured":"Yue Liu Chakkrit Tantithamthavorn Yonghui Liu and Li Li. 2024. On the reliability and explainability of language models for program generation. ACM Transactions on Software Engineering and Methodology 33 5 (2024) 1\u201326.","DOI":"10.1145\/3641540"},{"key":"e_1_3_2_103_2","doi-asserted-by":"crossref","unstructured":"Yue Liu Chakkrit Tantithamthavorn Yonghui Liu Patanamon Thongtanunam and Li Li. 2022. Automatically recommend code updates: Are we there yet? ACM Transactions on Software Engineering and Methodology 33 8 (2022) 1\u201327.","DOI":"10.1145\/3678167"},{"key":"e_1_3_2_104_2","unstructured":"Ziyang Luo Can Xu Pu Zhao Qingfeng Sun Xiubo Geng Wenxiang Hu Chongyang Tao Jing Ma Qingwei Lin and Daxin Jiang. 2023. WizardCoder: Empowering code large language models with Evol-Instruct. arXiv:2306.08568. Retrieved from https:\/\/arxiv.org\/abs\/2306.08568"},{"key":"e_1_3_2_105_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME52107.2021.00021"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-022-10274-8"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643916.3644400"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00103"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2023.111947"},{"key":"e_1_3_2_110_2","doi-asserted-by":"crossref","unstructured":"Antonio Mastropaolo Vittoria Nardone Gabriele Bavota and Massimiliano Di Penta. 2024. How the training procedure impacts the performance of deep learning-based vulnerability patching. arXiv:2404.17896. Retrieved from https:\/\/arxiv.org\/abs\/2404.17896","DOI":"10.1145\/3661167.3661200"},{"key":"e_1_3_2_111_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3511561"},{"key":"e_1_3_2_112_2","unstructured":"Niklas Muennighoff Qian Liu Armel Zebaze Qinkai Zheng Binyuan Hui Terry Yue Zhuo Swayam Singh Xiangru Tang Leandro Von Werra and Shayne Longpre. 2023. OctoPack: Instruction tuning code large language models. arXiv:2308.07124. Retrieved from https:\/\/arxiv.org\/abs\/2308.07124"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643774"},{"key":"e_1_3_2_114_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3428324"},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639116"},{"key":"e_1_3_2_116_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643916.3644401"},{"key":"e_1_3_2_117_2","doi-asserted-by":"crossref","unstructured":"Md Rizwan Parvez Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray and Kai-Wei Chang. 2021. Retrieval augmented code generation and summarization. arXiv:2108.11601. Retrieved from https:\/\/arxiv.org\/abs\/2108.11601","DOI":"10.18653\/v1\/2021.findings-emnlp.232"},{"key":"e_1_3_2_118_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639086"},{"key":"e_1_3_2_119_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00193"},{"key":"e_1_3_2_120_2","unstructured":"Baptiste Roziere Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat Xiaoqing Ellen Tan Yossi Adi Jingyu Liu Romain Sauvestre Tal Remez et al. 2023. Code Llama: Open foundation models for code. arXiv:2308.12950. Retrieved from https:\/\/arxiv.org\/abs\/2308.12950"},{"key":"e_1_3_2_121_2","unstructured":"Baptiste Roziere Jie M. Zhang Francois Charton Mark Harman Gabriel Synnaeve and Guillaume Lample. 2021. Leveraging automated unit tests for unsupervised code translation. arXiv:2110.06773. Retrieved from https:\/\/arxiv.org\/abs\/2110.06773"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC52881.2021.00049"},{"key":"e_1_3_2_123_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3558965"},{"key":"e_1_3_2_124_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510060"},{"key":"e_1_3_2_125_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549145"},{"key":"e_1_3_2_126_2","unstructured":"Andr\u00e9 Silva Sen Fang and Martin Monperrus. 2023. RepairLLaMA: Efficient representations and fine-tuned adapters for program repair. arXiv:2312.15698. Retrieved from https:\/\/arxiv.org\/abs\/2312.15698"},{"key":"e_1_3_2_127_2","doi-asserted-by":"crossref","unstructured":"Weisong Sun Chunrong Fang Yuchen Chen Quanjun Zhang Guanhong Tao Yudu You Tingxu Han Yifei Ge Yuling Hu Bin Luo et al. 2024. An extractive-and-abstractive framework for source code summarization. ACM Transactions on Software Engineering and Methodology 33 3 (2024) 1\u201339.","DOI":"10.1145\/3632742"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3417058"},{"key":"e_1_3_2_129_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524481.3527220"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510621"},{"key":"e_1_3_2_131_2","doi-asserted-by":"crossref","unstructured":"Rosalia Tufano Luca Pascarella and Gabriele Bavota. 2023. Automating code-related tasks through transformers: The impact of pre-training. arXiv:2302.04048. Retrieved from https:\/\/arxiv.org\/abs\/2302.04048","DOI":"10.1109\/ICSE48619.2023.00203"},{"key":"e_1_3_2_132_2","doi-asserted-by":"crossref","unstructured":"Tim van Dam Maliheh Izadi and Arie van Deursen. 2023. Enriching source code with contextual data for code completion models: An empirical study. arXiv:2304.12269. Retrieved from https:\/\/arxiv.org\/abs\/2304.12269","DOI":"10.1109\/MSR59073.2023.00035"},{"key":"e_1_3_2_133_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE56229.2023.00112"},{"key":"e_1_3_2_134_2","doi-asserted-by":"crossref","unstructured":"Deze Wang Boxing Chen Shanshan Li Wei Luo Shaoliang Peng Wei Dong and Xiangke Liao. 2023. One adapter for all programming languages? Adapter tuning for code search and summarization. arXiv:2303.15822. Retrieved from https:\/\/arxiv.org\/abs\/2303.15822","DOI":"10.1109\/ICSE48619.2023.00013"},{"key":"e_1_3_2_135_2","doi-asserted-by":"crossref","unstructured":"Xu Wang Hongwei Yu Xiangxin Meng Hongliang Cao Hongyu Zhang Hailong Sun Xudong Liu and Chunming Hu. 2024. MTL-TRANSFER: Leveraging multi-task learning and transferred knowledge for improving fault localization and program repair. ACM Transactions on Software Engineering and Methodology 33 6 (2024) 1\u201331.","DOI":"10.1145\/3654441"},{"key":"e_1_3_2_136_2","doi-asserted-by":"crossref","unstructured":"Yanlin Wang Yanxian Huang Daya Guo Hongyu Zhang and Zibin Zheng. 2024. SparseCoder: Identifier-aware sparse transformer for file-level code summarization. arXiv:2401.14727. Retrieved from https:\/\/arxiv.org\/abs\/2401.14727","DOI":"10.1109\/SANER60148.2024.00068"},{"key":"e_1_3_2_137_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643753"},{"key":"e_1_3_2_138_2","unstructured":"Yuxiang Wei Zhe Wang Jiawei Liu Yifeng Ding and Lingming Zhang. 2023. Magicoder: Source code is all you need. arXiv:2312.02120. Retrieved from https:\/\/arxiv.org\/abs\/2312.02120"},{"key":"e_1_3_2_139_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556909"},{"key":"e_1_3_2_140_2","doi-asserted-by":"publisher","DOI":"10.1145\/3520312.3534862"},{"key":"e_1_3_2_141_2","unstructured":"Frank F. Xu Zhengbao Jiang Pengcheng Yin Bogdan Vasilescu and Graham Neubig. 2020. Incorporating external knowledge through pre-training for natural language to code generation. arXiv:2004.09015. Retrieved from https:\/\/arxiv.org\/abs\/2004.09015"},{"key":"e_1_3_2_142_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3440503"},{"key":"e_1_3_2_143_2","unstructured":"Guang Yang Yu Zhou Wenhua Yang Tao Yue Xiang Chen and Taolue Chen. 2022. How important are good method names in neural code generation? A model robustness perspective. arXiv:2211.15844. Retrieved from https:\/\/arxiv.org\/abs\/2211.15844"},{"key":"e_1_3_2_144_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3361661"},{"key":"e_1_3_2_145_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556926"},{"key":"e_1_3_2_146_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510222"},{"key":"e_1_3_2_147_2","unstructured":"Daoguang Zan Bei Chen Zeqi Lin Bei Guan Yongji Wang and Jian-Guang Lou. 2022. When language model meets private library. arXiv:2210.17236. Retrieved from https:\/\/arxiv.org\/abs\/2210.17236"},{"key":"e_1_3_2_148_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643745"},{"key":"e_1_3_2_149_2","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616350"},{"key":"e_1_3_2_150_2","doi-asserted-by":"crossref","unstructured":"Xiaowei Zhang Weiqin Zou Lin Chen Yanhui Li and Yuming Zhou. 2022. Towards the analysis and completion of syntactic structure ellipsis for inline comments. IEEE Transactions on Software Engineering 49 4 (2022) 2285\u20132302.","DOI":"10.1109\/TSE.2022.3216279"},{"key":"e_1_3_2_151_2","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549094"},{"key":"e_1_3_2_152_2","doi-asserted-by":"publisher","DOI":"10.1145\/3580305.3599790"},{"key":"e_1_3_2_153_2","unstructured":"Shuyan Zhou Uri Alon Frank F. Xu Zhiruo Wang Zhengbao Jiang and Graham Neubig. 2022. DocPrompting: Generating code by retrieving the docs. arXiv:2207.05987. Retrieved from https:\/\/arxiv.org\/abs\/2207.05987"},{"key":"e_1_3_2_154_2","doi-asserted-by":"publisher","DOI":"10.1145\/3660808"},{"key":"e_1_3_2_155_2","unstructured":"Xin Zhou Kisub Kim Bowen Xu DongGyun Han Junda He and David Lo. 2023. Generation-based code review automation: How far are we? arXiv:2303.07221. Retrieved from https:\/\/arxiv.org\/abs\/2303.07221"},{"key":"e_1_3_2_156_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639222"},{"key":"e_1_3_2_157_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3439562"},{"key":"e_1_3_2_158_2","first-page":"40","volume-title":"2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE)","author":"Zhout Xin","year":"2023","unstructured":"Xin Zhout, Kisub Kim, Bowen Xu, Jiakun Liu, DongGyun Han, and David Lo. 2023. The devil is in the tails: How long-tailed code distributions impact large language models. In 2023 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE). IEEE, 40\u201352."},{"key":"e_1_3_2_159_2","unstructured":"Ming Zhu Aneesh Jain Karthik Suresh Roshan Ravindran Sindhu Tipirneni and Chandan K. Reddy. 2022. XLCoST: A benchmark dataset for cross-lingual code intelligence. arXiv:2206.08474. Retrieved from https:\/\/arxiv.org\/abs\/2206.08474"},{"key":"e_1_3_2_160_2","doi-asserted-by":"crossref","unstructured":"Tingwei Zhu Zhong Li Minxue Pan Chaoxuan Shi Tian Zhang Yu Pei and Xuandong Li. 2023. Deep is better? An empirical comparison of information retrieval and deep learning approaches to code summarization. ACM Transactions on Software Engineering and Methodology 33 3 (2023) 1\u201337.","DOI":"10.1145\/3631975"},{"key":"e_1_3_2_161_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643916.3644421"},{"key":"e_1_3_2_162_2","doi-asserted-by":"crossref","unstructured":"Armin Zirak and Hadi Hemmati. 2024. Improving automated program repair with domain adaptation. ACM Transactions on Software Engineering and Methodology 33 3 (2024) 1\u201343.","DOI":"10.1145\/3631972"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3707457","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3707457","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:17:38Z","timestamp":1750295858000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3707457"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,28]]},"references-count":161,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,5,31]]}},"alternative-id":["10.1145\/3707457"],"URL":"https:\/\/doi.org\/10.1145\/3707457","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,28]]},"assertion":[{"value":"2024-05-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-22","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}