{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T20:58:53Z","timestamp":1774731533798,"version":"3.50.1"},"reference-count":96,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:00:00Z","timestamp":1720742400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2024,7,12]]},"abstract":"<jats:p>Software engineering activities such as package migration, fixing error reports from static analysis or testing, and adding type annotations or other specifications to a codebase, involve pervasively editing the entire repository of code. We formulate these activities as repository-level coding tasks.<\/jats:p>\n                  <jats:p>\n                    Recent tools like GitHub Copilot, which are powered by Large Language Models (LLMs), have succeeded in offering high-quality solutions to localized coding problems. Repository-level coding tasks are more involved and cannot be solved directly using LLMs, since code within a repository is inter-dependent and the entire repository may be too large to fit into the prompt. We frame repository-level coding as a planning problem and present a task-agnostic, neuro-symbolic framework called\n                    <jats:monospace>CodePlan<\/jats:monospace>\n                    to solve it.\n                    <jats:monospace>CodePlan<\/jats:monospace>\n                    synthesizes a multi-step chain-of-edits (plan), where each step results in a call to an LLM on a code location with context derived from the entire repository, previous code changes and task-specific instructions.\n                    <jats:monospace>CodePlan<\/jats:monospace>\n                    is based on a novel combination of an incremental dependency analysis, a change may-impact analysis and an adaptive planning algorithm (symbolic components) with the neural LLMs.\n                  <\/jats:p>\n                  <jats:p>\n                    We evaluate the effectiveness of\n                    <jats:monospace>CodePlan<\/jats:monospace>\n                    on two repository-level tasks: package migration (C#) and temporal code edits (Python). Each task is evaluated on multiple code repositories, each of which requires inter-dependent changes to many files (between 2-97 files). Coding tasks of this level of complexity have not been automated using LLMs before. Our results show that\n                    <jats:monospace>CodePlan<\/jats:monospace>\n                    has better match with the ground truth compared to baselines.\n                    <jats:monospace>CodePlan<\/jats:monospace>\n                    is able to get 5\/7 repositories to pass the validity checks (i.e., to build without errors and make correct code edits) whereas the baselines (without planning but with the same type of contextual information as CodePlan) cannot get any of the repositories to pass them. We provide our (nonproprietary) data, evaluation scripts and supplementary material at https:\/\/github.com\/microsoft\/codeplan.\n                  <\/jats:p>","DOI":"10.1145\/3643757","type":"journal-article","created":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T10:22:09Z","timestamp":1720779729000},"page":"675-698","source":"Crossref","is-referenced-by-count":58,"title":["CodePlan: Repository-Level Coding using LLMs and Planning"],"prefix":"10.1145","volume":"1","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-9191-3748","authenticated-orcid":false,"given":"Ramakrishna","family":"Bairi","sequence":"first","affiliation":[{"name":"Microsoft Research, Bangalore, India"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-2132-0458","authenticated-orcid":false,"given":"Atharv","family":"Sonwane","sequence":"additional","affiliation":[{"name":"Microsoft Research, Bangalore, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8701-6977","authenticated-orcid":false,"given":"Aditya","family":"Kanade","sequence":"additional","affiliation":[{"name":"Microsoft Research, Bangalore, India"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-4206-5390","authenticated-orcid":false,"given":"Vageesh D.","family":"C.","sequence":"additional","affiliation":[{"name":"Microsoft Research, Bangalore, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7377-7599","authenticated-orcid":false,"given":"Arun","family":"Iyer","sequence":"additional","affiliation":[{"name":"Microsoft Research, Bangalore, India"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-6675-3219","authenticated-orcid":false,"given":"Suresh","family":"Parthasarathy","sequence":"additional","affiliation":[{"name":"Microsoft Research, Bangalore, India"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1400-7065","authenticated-orcid":false,"given":"Sriram","family":"Rajamani","sequence":"additional","affiliation":[{"name":"Microsoft Research, Bangalore, India"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-8063-9861","authenticated-orcid":false,"given":"B.","family":"Ashok","sequence":"additional","affiliation":[{"name":"Microsoft Research, Bangalore, India"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-7108-5816","authenticated-orcid":false,"given":"Shashank","family":"Shet","sequence":"additional","affiliation":[{"name":"Microsoft Research, Bangalore, India"}]}],"member":"320","published-online":{"date-parts":[[2024,7,12]]},"reference":[{"key":"e_1_3_1_2_2","unstructured":"2020. Reactive Streams TCK. https:\/\/github.com\/reactive-streams\/reactive-streams-dotnet\/tree\/master\/src\/tck."},{"key":"e_1_3_1_3_2","unstructured":"2022. das-qna-api. https:\/\/github.com\/SkillsFundingAgency\/das-qna-api."},{"key":"e_1_3_1_4_2","unstructured":"2023. Amazon Code Whisperer - AI Code Generator. https:\/\/aws.amazon.com\/codewhisperer\/."},{"key":"e_1_3_1_5_2","unstructured":"2023. audiocraft. https:\/\/github.com\/facebookresearch\/audiocraft."},{"key":"e_1_3_1_6_2","unstructured":"2023. git-diff. https:\/\/git-scm.com\/docs\/git-diff."},{"key":"e_1_3_1_7_2","unstructured":"2023. GitHub Copilot chat for Visual Studio 2022. https:\/\/devblogs.microsoft.com\/visualstudio\/github-copilot-chat-for-visual-studio-2022\/."},{"key":"e_1_3_1_8_2","unstructured":"2023. GitHub Copilot: Your AI pair programmer. https:\/\/github.com\/features\/copilot."},{"key":"e_1_3_1_9_2","unstructured":"2023. GPT-4 32K. https:\/\/platform.openai.com\/docs\/models\/gpt-4."},{"key":"e_1_3_1_10_2","unstructured":"2023. JARVIS. https:\/\/github.com\/microsoft\/JARVIS."},{"key":"e_1_3_1_11_2","unstructured":"2023. Jedi. https:\/\/github.com\/davidhalter\/jedi."},{"key":"e_1_3_1_12_2","unstructured":"2023. MS-Build. https:\/\/learn.microsoft.com\/en-us\/visualstudio\/msbuild\/msbuild."},{"key":"e_1_3_1_13_2","unstructured":"2023. Pyright. https:\/\/github.com\/microsoft\/pyright."},{"key":"e_1_3_1_14_2","unstructured":"2023. Replit. https:\/\/replit.com\/."},{"key":"e_1_3_1_15_2","unstructured":"2023. whisper. https:\/\/github.com\/openai\/whisper."},{"key":"e_1_3_1_16_2","unstructured":"Lakshya A Agrawal Aditya Kanade Navin Goyal Shuvendu K. Lahiri and Sriram K. Rajamani. 2023. Guiding Language Models of Code with Global Context using Monitors. arXiv:2306.10763 [cs.CL]."},{"key":"e_1_3_1_17_2","doi-asserted-by":"crossref","unstructured":"Wasi Uddin Ahmad Saikat Chakraborty Baishakhi Ray and Kai-Wei Chang. 2021. Unified Pre-training for Program Understanding and Generation. arXiv:2103.06333 [cs.CL].","DOI":"10.18653\/v1\/2021.naacl-main.211"},{"key":"e_1_3_1_18_2","doi-asserted-by":"crossref","unstructured":"Toufique Ahmed and Premkumar Devanbu. 2023. Better patching using LLM prompting via Self-Consistency. arXiv:2306.00108 [cs.SE].","DOI":"10.1109\/ASE56229.2023.00065"},{"key":"e_1_3_1_19_2","unstructured":"Alfred V Aho Ravi Sethi Jeffrey D Ullman et al. 2007. Compilers: principles techniques and tools. Vol. 2. Addison-wesley Reading."},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2661674"},{"key":"e_1_3_1_21_2","first-page":"148","article-title":"Generic patch inference","volume":"17","author":"Andersen Jesper","year":"2010","unstructured":"Jesper Andersen and Julia L. Lawall. 2010. Generic patch inference. Automated software engineering 17 (2010), 119\u2013148.","journal-title":"Automated software engineering"},{"key":"e_1_3_1_22_2","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1109\/ASE.2004.1342719","article-title":"A differencing algorithm for object-oriented programs","author":"Apiwattanapong Taweesup","year":"2004","unstructured":"Taweesup Apiwattanapong, Alessandro Orso, and Mary Jean Harrold. 2004. A differencing algorithm for object-oriented programs. In Proceedings. 19th International Conference on Automated Software Engineering, 2004. IEEE, 2-13.","journal-title":"Proceedings. 19th International Conference on Automated Software Engineering, 2004."},{"key":"e_1_3_1_23_2","first-page":"1","article-title":"An introduction to software change impact analysis","author":"Arnold RS","year":"1996","unstructured":"RS Arnold and SA Bohner. 1996. An introduction to software change impact analysis. Software Change Impact Analysis (1996), 1\u201326.","journal-title":"Software Change Impact Analysis"},{"key":"e_1_3_1_24_2","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1145\/2568225.2568243","article-title":"Reviser: efficiently updating IDE-\/IFDS-based data-flow analyses in response to incremental program changes","author":"Arzt Steven","year":"2014","unstructured":"Steven Arzt and Eric Bodden. 2014. Reviser: efficiently updating IDE-\/IFDS-based data-flow analyses in response to incremental program changes. In Proceedings of the 36th International Conference on Software Engineering. 288\u2013298.","journal-title":"Proceedings of the 36th International Conference on Software Engineering"},{"key":"e_1_3_1_25_2","unstructured":"Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski David Dohan Ellen Jiang Carrie Cai Michael Terry Quoc Le and Charles Sutton. 2021. Program Synthesis with Large Language Models. http:\/\/arxiv.org\/abs\/2108.07732 arXiv:2108.07732 [cs]."},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1145\/3360585"},{"key":"e_1_3_1_27_2","article-title":"DeepCoder: Learning to Write Programs","author":"Balog Matej","year":"2016","unstructured":"Matej Balog, Alexander L. Gaunt, Marc Brockschmidt, Sebastian Nowozin, and Daniel Tarlow. 2016. DeepCoder: Learning to Write Programs. ArXiv abs\/1611.01989 (2016). https:\/\/api.semanticscholar.org\/CorpusID:2906360","journal-title":"ArXiv"},{"key":"e_1_3_1_28_2","article-title":"Gpt-neox-20b: An open-source autoregressive language model","author":"Black Sid","year":"2022","unstructured":"Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, and others. 2022. Gpt-neox-20b: An open-source autoregressive language model. arXiv preprint arXiv:2204.06745 (2022).","journal-title":"arXiv preprint arXiv:2204.06745"},{"issue":"6","key":"e_1_3_1_29_2","doi-asserted-by":"crossref","first-page":"713","DOI":"10.1145\/945885.945886","article-title":"Escape analysis for JavaTM: Theory and practice","volume":"25","author":"Blanchet Bruno","year":"2003","unstructured":"Bruno Blanchet. 2003. Escape analysis for JavaTM: Theory and practice. ACM Transactions on Programming Languages and Systems (TOPLAS) 25, 6 (2003), 713\u2013775.","journal-title":"ACM Transactions on Programming Languages and Systems (TOPLAS)"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","unstructured":"Shaked Brody Uri Alon and Eran Yahav. 2020. A structural model for contextual code changes. 4 OOPSLA (Nov. 2020). https:\/\/doi.org\/10.1145\/3428283 10.1145\/3428283 Publisher Copyright: \u00a9 2020 Owner\/Author..","DOI":"10.1145\/3428283"},{"key":"e_1_3_1_31_2","first-page":"1901","article-title":"Language models are few-shot learners","volume":"33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems 33 (2020), 1877\u20131901.","journal-title":"Advances in neural information processing systems"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","unstructured":"Max Brunsfeld Andrew Hlynskyi Patrick Thomson Josh Vera Phil Turnbull Timothy Clem Douglas Creager Andrew Helwer Rob Rix Hendrik Van Antwerpen Michael Davis Ika Tun-Anh Nguyn Stafford Brunk Niranjan Hasabnis Bfredl Mingkai Dong Matt Massicotte Jonathan Arnett Vladimir Panteleev Steven Kalt Kolja Lampe Alex Pinkus Mark Schmitz Matthew Krupcale Narpfel Santos Gallegos Vicent Marti and Edgar. 2023. tree-sitter\/tree-sitter: v0.20.8. https:\/\/doi.org\/10.5281\/ZENODO.4619183 10.5281\/ZENODO.4619183","DOI":"10.5281\/ZENODO.4619183"},{"key":"e_1_3_1_33_2","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1007\/BFb0012826","volume-title":"9th International Conference on Automated Deduction: Argonne, Illinois, USA, May 23-26, 1988 Proceedings 9","author":"Bundy Alan","year":"1988","unstructured":"Alan Bundy. 1988. The use of explicit plans to guide inductive proofs. In 9th International Conference on Automated Deduction: Argonne, Illinois, USA, May 23-26, 1988 Proceedings 9. Springer, 111\u2013120."},{"key":"e_1_3_1_34_2","doi-asserted-by":"crossref","first-page":"106","DOI":"10.1007\/978-3-030-20652-9_7","volume-title":"NASA Formal Methods: 11th International Symposium, NFM 2019, Houston, TX, USA, May 7-9, 2019, Proceedings 11","author":"Busi Matteo","year":"2019","unstructured":"Matteo Busi, Pierpaolo Degano, and Letterio Galletta. 2019. Using standard typing algorithms incrementally. In NASA Formal Methods: 11th International Symposium, NFM 2019, Houston, TX, USA, May 7-9, 2019, Proceedings 11. Springer, 106\u2013122."},{"key":"e_1_3_1_35_2","article-title":"Evaluating large language models trained on code","author":"Chen Mark","year":"2021","unstructured":"Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, and others. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021).","journal-title":"arXiv preprint arXiv:2107.03374"},{"key":"e_1_3_1_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/320385.320386"},{"key":"e_1_3_1_37_2","article-title":"Palm: Scaling language modeling with pathways","author":"Chowdhery Aakanksha","year":"2022","unstructured":"Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311 (2022).","journal-title":"arXiv preprint arXiv:2204.02311"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3474624.3474650"},{"key":"e_1_3_1_39_2","first-page":"77","volume-title":"ECOOP\u201995-Object-OrientedProgramming, 9th European Conference, \u00c1arhus, Denmark, August 7-11, 1995 9","author":"Dean Jeffrey","year":"1995","unstructured":"Jeffrey Dean, David Grove, and Craig Chambers. 1995. Optimization of object-oriented programs using static class hierarchy analysis. In ECOOP\u201995-Object-OrientedProgramming, 9th European Conference, \u00c1arhus, Denmark, August 7-11, 1995 9. Springer, 77\u2013101."},{"key":"e_1_3_1_40_2","first-page":"404","volume-title":"ECOOP2006-Object-Oriented Programming: 20th European Conference, Nantes, France, July 3-7, 2006. Proceedings 20","author":"Dig Danny","year":"2006","unstructured":"Danny Dig, Can Comertoglu, Darko Marinov, and Ralph Johnson. 2006. Automated detection of refactorings in evolving components. In ECOOP2006-Object-Oriented Programming: 20th European Conference, Nantes, France, July 3-7, 2006. Proceedings 20. Springer, 404\u2013428."},{"key":"e_1_3_1_41_2","unstructured":"Yangruibo Ding Zijian Wang Wasi Uddin Ahmad Murali Krishna Ramanathan Ramesh Nallapati Parminder Bhatia Dan Roth and Bing Xiang. 2022. CoCoMIC: Code Completion By Jointly Modeling In-file and Cross-file Context. http:\/\/arxiv.org\/abs\/2212.10007 arXiv:2212.10007 [cs]."},{"key":"e_1_3_1_42_2","article-title":"Incoder: A generative model for code infilling and synthesis","author":"Fried Daniel","year":"2022","unstructured":"Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, and Mike Lewis. 2022. Incoder: A generative model for code infilling and synthesis. arXiv preprint arXiv:2204.05999 (2022).","journal-title":"arXiv preprint arXiv:2204.05999"},{"key":"e_1_3_1_43_2","first-page":"71","article-title":"Refactoring-aware code review","author":"Ge Xi","year":"2017","unstructured":"Xi Ge, Saurabh Sarkar, Jim Witschey, and Emerson Murphy-Hill. 2017. Refactoring-aware code review. In 2017 IEEE Symposium on VisualLanguages andHuman-Centric Computing(VL\/HCC). IEEE, 71\u201379.","journal-title":"2017 IEEE Symposium on VisualLanguages andHuman-Centric Computing(VL\/HCC)"},{"key":"e_1_3_1_44_2","volume-title":"Automated Planning: theory and practice","author":"Ghallab Malik","year":"2004","unstructured":"Malik Ghallab, Dana Nau, and Paolo Traverso. 2004. Automated Planning: theory and practice. Elsevier."},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/TITS.2015.2498841"},{"key":"e_1_3_1_46_2","unstructured":"Priyanshu Gupta Avishree Khare Yasharth Bajpai Saikat Chakraborty Sumit Gulwani Aditya Kanade Arjun Radhakrishna Gustavo Soares and Ashish Tiwari. 2023. GrACE: Generation using Associated Code Edits. arXiv:2305.14129 [cs.SE]."},{"key":"e_1_3_1_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/2902362"},{"key":"e_1_3_1_48_2","doi-asserted-by":"crossref","unstructured":"Pascal Hitzler and Md Kamruzzaman Sarker. 2022. Neuro-symbolic artificial intelligence: The state of the art. (2022).","DOI":"10.3233\/FAIA342"},{"key":"e_1_3_1_49_2","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/1512475.1512493","article-title":"Towards a more efficient static software change impact analysis method","author":"Jashki Mohammad-Amin","year":"2008","unstructured":"Mohammad-Amin Jashki, Reza Zafarani, and Ebrahim Bagheri. 2008. Towards a more efficient static software change impact analysis method. In Proceedings of the 8th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering. 84\u201390.","journal-title":"Proceedings of the 8th ACM SIGPLAN-SIGSOFT workshop on Program analysis for software tools and engineering"},{"key":"e_1_3_1_50_2","unstructured":"Xue Jiang Yihong Dong Lecheng Wang Qiwei Shang and Ge Li. 2023. Self-planning Code Generation with Large Language Model. arXiv:2303.06689 [cs.SE]."},{"key":"e_1_3_1_51_2","article-title":"InferFix: End-to-End Program Repair with LLMs","author":"Jin Matthew","year":"2023","unstructured":"Matthew Jin, Syed Shahriar, Michele Tufano, Xin Shi, Shuai Lu, Neel Sundaresan, and Alexey Svyatkovskiy. 2023. InferFix: End-to-End Program Repair with LLMs. arXiv preprint arXiv:2303.07263 (2023).","journal-title":"arXiv preprint arXiv:2303.07263"},{"key":"e_1_3_1_52_2","first-page":"439","article-title":"Automated planning for robotics","volume":"3","author":"Karpas Erez","year":"2020","unstructured":"Erez Karpas and Daniele Magazzeni. 2020. Automated planning for robotics. Annual Review of Control, Robotics, and Autonomous Systems 3 (2020), 417\u2013439.","journal-title":"Annual Review of Control, Robotics, and Autonomous Systems"},{"key":"e_1_3_1_53_2","doi-asserted-by":"crossref","first-page":"1206","DOI":"10.1145\/3510003.3510115","article-title":"Inferring and applying type changes","author":"Ketkar Ameya","year":"2022","unstructured":"Ameya Ketkar, Oleg Smirnov, Nikolaos Tsantalis, Danny Dig, and Timofey Bryksin. 2022. Inferring and applying type changes. In Proceedings of the 44th International Conference on Software Engineering. 1206\u20131218.","journal-title":"Proceedings of the 44th International Conference on Software Engineering"},{"issue":"1","key":"e_1_3_1_54_2","first-page":"45","article-title":"Identifying and summarizing systematic code changes via rule inference","volume":"39","author":"Kim Miryung","year":"2012","unstructured":"Miryung Kim, David Notkin, Dan Grossman, and Gary Wilson. 2012. Identifying and summarizing systematic code changes via rule inference. IEEE Transactions on Software Engineering 39, 1 (2012), 45\u201362.","journal-title":"IEEE Transactions on Software Engineering"},{"key":"e_1_3_1_55_2","doi-asserted-by":"publisher","DOI":"10.1109\/MRA.2011.941635"},{"key":"e_1_3_1_56_2","first-page":"712","article-title":"Symdiff: A language-agnostic semantic diff tool for imperative programs","author":"Lahiri Shuvendu K","year":"2012","unstructured":"Shuvendu K Lahiri, Chris Hawblitzel, Ming Kawaguchi, and Henrique Rebelo. 2012. Symdiff: A language-agnostic semantic diff tool for imperative programs. In Computer Aided Verification: 24th International Conference, CAV 2012, Berkeley, CA, USA, July7-13, 2012 Proceedings 24. Springer, 712\u2013717.","journal-title":"Computer Aided Verification: 24th International Conference, CAV 2012, Berkeley, CA, USA, July7-13, 2012 Proceedings 24"},{"key":"e_1_3_1_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.2988396"},{"key":"e_1_3_1_58_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.abq1158"},{"key":"e_1_3_1_59_2","unstructured":"Tianyang Liu Canwen Xu and Julian McAuley. 2023. RepoBench: Benchmarking Repository-Level Code AutoCompletion Systems. arXiv:2306.03091 [cs.CL]."},{"key":"e_1_3_1_60_2","doi-asserted-by":"crossref","first-page":"554","DOI":"10.1145\/2491411.2501854","article-title":"Scalable and incremental software bug detection","author":"McPeak Scott","year":"2013","unstructured":"Scott McPeak, Charles-Henri Gros, and Murali Krishna Ramanathan. 2013. Scalable and incremental software bug detection. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering. 554\u2013564.","journal-title":"Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering"},{"key":"e_1_3_1_61_2","first-page":"440","article-title":"Sydit: Creating and applying a program transformation from an example","author":"Meng Na","year":"2011","unstructured":"Na Meng, Miryung Kim, and Kathryn S McKinley. 2011. Sydit: Creating and applying a program transformation from an example. In Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering. 440\u2013443.","journal-title":"Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering"},{"key":"e_1_3_1_62_2","doi-asserted-by":"crossref","first-page":"502","DOI":"10.1109\/ICSE.2013.6606596","article-title":"LASE: locating and applying systematic edits by learning from examples","author":"Meng Na","year":"2013","unstructured":"Na Meng, Miryung Kim, and Kathryn S McKinley. 2013. LASE: locating and applying systematic edits by learning from examples. In 2013 35th International Conference on Software Engineering (ICSE). IEEE, 502\u2013511.","journal-title":"2013 35th International Conference on Software Engineering (ICSE)"},{"key":"e_1_3_1_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3360569"},{"key":"e_1_3_1_64_2","article-title":"CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis","author":"Nijkamp Erik","year":"2023","unstructured":"Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2023. CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. In The Eleventh International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=iaYcJKpY2B_","journal-title":"The Eleventh International Conference on Learning Representations"},{"key":"e_1_3_1_65_2","unstructured":"OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL]."},{"key":"e_1_3_1_66_2","first-page":"311","article-title":"Bleu: a method for automatic evaluation of machine translation","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics. 311\u2013318.","journal-title":"Proceedings of the 40th annual meeting of the Association for Computational Linguistics"},{"key":"e_1_3_1_67_2","article-title":"Neuro-Symbolic Program Synthesis","author":"Parisotto Emilio","year":"2016","unstructured":"Emilio Parisotto, Abdel rahman Mohamed, Rishabh Singh, Lihong Li, Dengyong Zhou, and Pushmeet Kohli. 2016. Neuro-Symbolic Program Synthesis. ArXiv abs\/1611.01855 (2016). https:\/\/api.semanticscholar.org\/CorpusID:15904815","journal-title":"ArXiv"},{"key":"e_1_3_1_68_2","unstructured":"Pardis Pashakhanloo Aaditya Naik Yuepeng Wang Hanjun Dai Petros Maniatis and Mayur Naik. 2022. Codetrek: Flexible modeling of code using an extensible relational representation. (2022)."},{"key":"e_1_3_1_69_2","first-page":"1","volume-title":"2023 IEEE Symposium on Security and Privacy (SP)","author":"Pearce Hammond","year":"2022","unstructured":"Hammond Pearce, Benjamin Tan, Baleegh Ahmad, Ramesh Karri, and Brendan Dolan-Gavitt. 2022. Examining ZeroShot Vulnerability Repair with Large Language Models. In 2023 IEEE Symposium on Security and Privacy (SP). IEEE Computer Society, 1\u201318."},{"key":"e_1_3_1_70_2","article-title":"Better context makes better code language models: A case study on function call argument completion","author":"Pei Hengzhi","year":"2023","unstructured":"Hengzhi Pei, Jinman Zhao, Leonard Lausen, Sheng Zha, and George Karypis. 2023. Better context makes better code language models: A case study on function call argument completion. In AAAI. https:\/\/www.amazon.science\/publications\/better-context-makes-better-code-language-models-a-case-study-on-function-call-argument-completion","journal-title":"AAAI"},{"key":"e_1_3_1_71_2","unstructured":"Kexin Pei David Bieber Kensen Shi Charles Sutton and Pengcheng Yin. 2023. Can Large Language Models Reason about Program Invariants? (2023)."},{"key":"e_1_3_1_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/1993316.1993558"},{"key":"e_1_3_1_73_2","doi-asserted-by":"publisher","unstructured":"Machel Reid and Graham Neubig. 2022. Learning to Model Editing Processes. https:\/\/doi.org\/10.48550\/ARXIV.2205.12374 10.48550\/ARXIV.2205.12374","DOI":"10.48550\/ARXIV.2205.12374"},{"key":"e_1_3_1_74_2","first-page":"432","article-title":"Chianti: a tool for change impact analysis of java programs","author":"Ren Xiaoxia","year":"2004","unstructured":"Xiaoxia Ren, Fenil Shah, Frank Tip, Barbara G Ryder, and Ophelia Chesley. 2004. Chianti: a tool for change impact analysis of java programs. In Proceedings of the 19th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications. 432\u2013448.","journal-title":"Proceedings of the 19th annual ACM SIGPLAN conference on Object-oriented programming, systems, languages, and applications"},{"key":"e_1_3_1_75_2","article-title":"Code Llama: Open Foundation Models for Code","author":"Rozi\u00e8re Baptiste","year":"2023","unstructured":"Baptiste Rozi\u00e8re, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Tan, Yossi Adi, Jingyu Liu, Tal Remez, J\u00e9r\u00e9my Rapin, Artyom Kozhevnikov, I. Evtimov, Joanna Bitton, Manish P Bhatt, Cristian Cant\u00f3n Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre D\u2019efossez, Jade Copet, Faisal Azhar, Hugo Touvron, Louis Martin, Nicolas Usunier, Thomas Scialom, and Gabriel Synnaeve. 2023. Code Llama: Open Foundation Models for Code. ArXiv abs\/2308.12950 (2023). https:\/\/api.semanticscholar.org\/CorpusID:261100919","journal-title":"ArXiv abs\/2308.12950 (2023)"},{"key":"e_1_3_1_76_2","volume-title":"Artificial intelligence a modern approach","author":"Russell Stuart J","year":"2010","unstructured":"Stuart J Russell. 2010. Artificial intelligence a modern approach. Pearson Education, Inc."},{"key":"e_1_3_1_77_2","first-page":"167","article-title":"Incremental data flow analysis","author":"Ryder Barbara G","year":"1983","unstructured":"Barbara G Ryder. 1983. Incremental data flow analysis. In Proceedings of the 10th ACM SIGACT-SIGPLAN symposium on Principles ofprogramming languages. 167\u2013176.","journal-title":"Proceedings of the 10th ACM SIGACT-SIGPLAN symposium on Principles ofprogramming languages"},{"key":"e_1_3_1_78_2","article-title":"Adaptive test generation using a large language model","author":"Sch\u00e4fer Max","year":"2023","unstructured":"Max Sch\u00e4fer, Sarah Nadi, Aryaz Eghbali, and Frank Tip. 2023. Adaptive test generation using a large language model. arXiv preprint arXiv:2302.06527 (2023).","journal-title":"arXiv preprint arXiv:2302.06527"},{"key":"e_1_3_1_79_2","unstructured":"Disha Shrivastava Denis Kocetkov Harm de Vries Dzmitry Bahdanau and Torsten Scholak. 2023. RepoFusion: Training Code Models to Understand Your Repository. arXiv:2306.10998 [cs.LG]."},{"key":"e_1_3_1_80_2","article-title":"Repository-level prompt generation for large language models of code","author":"Shrivastava Disha","year":"2022","unstructured":"Disha Shrivastava, Hugo Larochelle, and Daniel Tarlow. 2022. Repository-level prompt generation for large language models of code. arXiv preprint arXiv:2206.12839 (2022).","journal-title":"arXiv preprint arXiv:2206.12839"},{"key":"e_1_3_1_81_2","unstructured":"Haoye Tian Weiqi Lu Tsz On Li Xunzhu Tang Shing-Chi Cheung Jacques Klein and Tegawende F. Bissyande. 2023. Is ChatGPT the Ultimate Programming Assistant - How far is it? arXiv:2304.11938 [cs.SE]."},{"key":"e_1_3_1_82_2","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale Dan Bikel Lukas Blecher Cristian Canton Ferrer Moya Chen Guillem Cucurull David Esiobu Jude Fernandes Jeremy Fu Wenyin Fu Brian Fuller Cynthia Gao Vedanuj Goswami Naman Goyal Anthony Hartshorn Saghar Hosseini Rui Hou Hakan Inan Marcin Kardas Viktor Kerkez Madian Khabsa Isabel Kloumann Artem Korenev Punit Singh Koura Marie-Anne Lachaux Thibaut Lavril Jenya Lee Diana Liskovich Yinghai Lu Yuning Mao Xavier Martinet Todor Mihaylov Pushkar Mishra Igor Molybog Yixin Nie Andrew Poulton Jeremy Reizenstein Rashi Rungta Kalyan Saladi Alan Schelten Ruan Silva Eric Michael Smith Ranjan Subramanian Xiaoqing Ellen Tan Binh Tang Ross Taylor Adina Williams Jian Xiang Kuan Puxin Xu Zheng Yan Iliyan Zarov Yuchen Zhang Angela Fan Melanie Kambadur Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov and Thomas Scialom. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288 [cs.CL]."},{"key":"e_1_3_1_83_2","article-title":"Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples","author":"Vijayakumar Ashwin J.","year":"2018","unstructured":"Ashwin J. Vijayakumar, Abhishek Mohta, Oleksandr Polozov, Dhruv Batra, Prateek Jain, and Sumit Gulwani. 2018. Neural-Guided Deductive Search for Real-Time Program Synthesis from Examples. ArXiv abs\/1804.01186 (2018). https:\/\/api.semanticscholar.org\/CorpusID:4606753","journal-title":"ArXiv abs\/1804.01186 (2018)"},{"key":"e_1_3_1_84_2","unstructured":"Ben Wang and Aran Komatsuzaki. 2021. GPT-J-6B: A 6 billion parameter autoregressive language model."},{"key":"e_1_3_1_85_2","article-title":"CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation","author":"Wang Yue","year":"2021","unstructured":"Yue Wang, Weishi Wang, Shafiq R. Joty, and Steven C. H. Hoi. 2021. CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. ArXiv abs\/2109.00859 (2021).","journal-title":"ArXiv abs\/2109.00859 (2021)"},{"key":"e_1_3_1_86_2","unstructured":"Jiayi Wei Greg Durrett and Isil Dillig. 2023. Coeditor: Leveraging Contextual Changes for Multi-round Code Autoediting. arXiv:2305.18584 [cs.SE]."},{"key":"e_1_3_1_87_2","unstructured":"Jiayi Wei Greg Durrett and Isil Dillig. 2023. TypeT5: Seq2seq Type Inference using Static Analysis. arXiv:2303.09564 [cs.SE]."},{"key":"e_1_3_1_88_2","first-page":"24837","article-title":"Chain-of-thought prompting elicits reasoning in large language models","volume":"35","author":"Wei Jason","year":"2022","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824\u201324837.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_89_2","article-title":"Automated program repair in the era of large pre-trained language models","author":"Xia Chunqiu Steven","year":"2023","unstructured":"Chunqiu Steven Xia, Yuxiang Wei, and Lingming Zhang. 2023. Automated program repair in the era of large pre-trained language models. In Proceedings of the 45th International Conference on Software Engineering (ICSE 2023). Association for Computing Machinery.","journal-title":"Proceedings of the 45th International Conference on Software Engineering (ICSE 2023). Association for Computing Machinery"},{"key":"e_1_3_1_90_2","doi-asserted-by":"publisher","DOI":"10.1145\/3520312.3534862"},{"key":"e_1_3_1_91_2","article-title":"Capturing structural locality in nonparametric language models","author":"Xu Frank F","year":"2021","unstructured":"Frank F Xu, Junxian He, Graham Neubig, and Vincent J Hellendoorn. 2021. Capturing structural locality in nonparametric language models. arXiv preprint arXiv:2110.02870 (2021).","journal-title":"arXiv preprint arXiv:2110.02870"},{"key":"e_1_3_1_92_2","first-page":"335","article-title":"Meditor: inference and application of API migration edits","author":"Xu Shengzhe","year":"2019","unstructured":"Shengzhe Xu, Ziqi Dong, and Na Meng. 2019. Meditor: inference and application of API migration edits. In 2019 IEEE\/ACM 27th International Conference on Program Comprehension (ICPC). IEEE, 335\u2013346.","journal-title":"2019 IEEE\/ACM 27th International Conference on Program Comprehension (ICPC)"},{"key":"e_1_3_1_93_2","article-title":"Learning to Represent Edits","author":"Yin Pengcheng","year":"2019","unstructured":"Pengcheng Yin, Graham Neubig, Miltiadis Allamanis, Marc Brockschmidt, and Alexander Gaunt. 2019. Learning to Represent Edits. In ICLR 2019. https:\/\/www.microsoft.com\/en-us\/research\/publication\/learning-to-represent-edits\/ arXiv:1810.13337 [cs.LG].","journal-title":"ICLR 2019"},{"key":"e_1_3_1_94_2","first-page":"442","article-title":"An incremental flow-and context-sensitive pointer aliasing analysis","author":"Yur Jyh-shiarn","year":"1999","unstructured":"Jyh-shiarn Yur, Barbara G Ryder, and William A Landi. 1999. An incremental flow-and context-sensitive pointer aliasing analysis. In Proceedings of the 21st International conference on Software Engineering. 442\u2013451.","journal-title":"Proceedings of the 21st International conference on Software Engineering"},{"key":"e_1_3_1_95_2","article-title":"RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation","author":"Zhang Fengji","year":"2023","unstructured":"Fengji Zhang, Bei Chen, Yue Zhang, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. 2023. RepoCoder: Repository-Level Code Completion Through Iterative Retrieval and Generation. arXiv preprint arXiv:2303.12570 (2023).","journal-title":"arXiv preprint arXiv:2303.12570"},{"key":"e_1_3_1_96_2","unstructured":"Shun Zhang Zhenfang Chen Yikang Shen Mingyu Ding Joshua B. Tenenbaum and Chuang Gan. 2023. Planning with Large Language Models for Code Generation. arXiv:2303.05510 [cs.LG]."},{"key":"e_1_3_1_97_2","doi-asserted-by":"publisher","DOI":"10.1145\/3563302"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643757","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3643757","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,2,4]],"date-time":"2026-02-04T08:07:08Z","timestamp":1770192428000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3643757"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,12]]},"references-count":96,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2024,7,12]]}},"alternative-id":["10.1145\/3643757"],"URL":"https:\/\/doi.org\/10.1145\/3643757","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,12]]}}}