{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T19:51:31Z","timestamp":1782849091909,"version":"3.54.5"},"reference-count":72,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T00:00:00Z","timestamp":1782777600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2023YFB4503802"],"award-info":[{"award-number":["2023YFB4503802"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62302515"],"award-info":[{"award-number":["62302515"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62172426"],"award-info":[{"award-number":["62172426"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62332005"],"award-info":[{"award-number":["62332005"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>\n                    Code adaptation is a fundamental but challenging task in software development, requiring developers to modify existing code for new contexts. A key challenge is to resolve\n                    <jats:bold>Context Adaptation Bugs (CtxBugs)<\/jats:bold>\n                    , which occurs when code correct in its original context violates constraints in the target environment. Unlike isolated bugs,\n                    <jats:italic toggle=\"yes\">CtxBugs<\/jats:italic>\n                    cannot be resolved through local fixes and require cross-context reasoning to identify semantic mismatches. Overlooking them may lead to critical failures in adaptation. Although Large Language Models (LLMs) show great potential in automating code-related tasks, their ability to resolve\n                    <jats:italic toggle=\"yes\">CtxBugs<\/jats:italic>\n                    remains a significant and unexplored obstacle to their practical use in code adaptation.\n                  <\/jats:p>\n                  <jats:p>\n                    To bridge this gap, we propose\n                    <jats:italic toggle=\"yes\">CtxBugGen<\/jats:italic>\n                    , a novel framework for generating\n                    <jats:italic toggle=\"yes\">CtxBugs<\/jats:italic>\n                    to evaluate LLMs. Its core idea is to leverage LLMs\u2019 tendency to generate plausible but context-free code when contextual constraints are absent. The framework generates\n                    <jats:italic toggle=\"yes\">CtxBugs<\/jats:italic>\n                    through a four-step process to ensure their relevance and validity: (1) Selection of four established context-aware adaptation tasks from the literature, (2) Perturbation via task-specific rules to induce\n                    <jats:italic toggle=\"yes\">CtxBugs<\/jats:italic>\n                    from LLMs while ensuring their plausibility, (3) Generation of candidate variants by prompting LLMs without any context constraints and (4) Identification of valid\n                    <jats:italic toggle=\"yes\">CtxBugs<\/jats:italic>\n                    through syntactic differencing and test execution in the target context. Based on the benchmark constructed by\n                    <jats:italic toggle=\"yes\">CtxBugGen<\/jats:italic>\n                    , we conduct an empirical study with four state-of-the-art LLMs. Our results reveal their unsatisfactory performance in\n                    <jats:italic toggle=\"yes\">CtxBug<\/jats:italic>\n                    resolution. The best performing LLM, Kimi-K2, achieves 55.93% on Pass@1 and resolves just 52.47% of\n                    <jats:italic toggle=\"yes\">CtxBugs<\/jats:italic>\n                    . The presence of\n                    <jats:italic toggle=\"yes\">CtxBugs<\/jats:italic>\n                    degrades LLMs\u2019 adaptation performance by up to 30%. Failure analysis indicates that LLMs often overlook\n                    <jats:italic toggle=\"yes\">CtxBugs<\/jats:italic>\n                    and replicate them in their outputs. This suggests that LLMs overly focus on the local code correctness of the reused code while ignoring its compatibility in the target context. Our study highlights a critical weakness in LLMs\u2019 cross-context reasoning and emphasize the need for new methods to enhance their context awareness for reliable code adaptation. The replication package for this paper is at https:\/\/github.com\/ztwater\/CtxBugGen.\n                  <\/jats:p>","DOI":"10.1145\/3797148","type":"journal-article","created":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:06:14Z","timestamp":1782839174000},"page":"229-252","source":"Crossref","is-referenced-by-count":0,"title":["Coding in a Bubble? Evaluating LLMs in Resolving Context Adaptation Bugs during Code Adaptation"],"prefix":"10.1145","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7241-9730","authenticated-orcid":false,"given":"Tanghaoran","family":"Zhang","sequence":"first","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6003-5748","authenticated-orcid":false,"given":"Xinjun","family":"Mao","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1469-2063","authenticated-orcid":false,"given":"Shangwen","family":"Wang","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-0061-9457","authenticated-orcid":false,"given":"Yuxin","family":"Zhao","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3520-5829","authenticated-orcid":false,"given":"Yao","family":"Lu","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-6902-3270","authenticated-orcid":false,"given":"Zezhou","family":"Tang","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-2088-4976","authenticated-orcid":false,"given":"Wenyu","family":"Xu","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8686-818X","authenticated-orcid":false,"given":"Longfei","family":"Sun","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-6739-7100","authenticated-orcid":false,"given":"Changrong","family":"Xie","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2313-7141","authenticated-orcid":false,"given":"Kang","family":"Yang","sequence":"additional","affiliation":[{"name":"National University of Defense Technology, Changsha, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9865-2212","authenticated-orcid":false,"given":"Yue","family":"Yu","sequence":"additional","affiliation":[{"name":"Peng Cheng Laboratory, Shenzhen, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,30]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"Miltiadis Allamanis and Marc Brockschmidt. 2017. SmartPaste: Learning to Adapt Source Code. http:\/\/arxiv.org\/abs\/ 1705.07867 arXiv:1705.07867 [cs]."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2017.7884629"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/2591062.2591130"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/3586030"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2009.147"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1191\/1478088706qp063oa"},{"key":"e_1_2_1_7_1","first-page":"1877","volume-title":"Lin (Eds.)","volume":"33","author":"Brown Tom","year":"2020","unstructured":"Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877-1901."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE55347.2025.00012"},{"key":"e_1_2_1_9_1","volume-title":"Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al.","author":"Chen Mark","year":"2021","unstructured":"Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021. Evaluating Large Language Models Trained on Code. (2021). arXiv:2107.03374 [cs.LG]"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC66645.2025.00060"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/1370175.1370194"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1453101.1453130"},{"key":"e_1_2_1_14_1","unstructured":"DeepSeek-AI. 2024. DeepSeek-V3 Technical Report. arXiv:2412.19437 [cs.CL] https:\/\/arxiv.org\/abs\/2412.19437"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/2688204.2688208"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3597503.3639219"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/2642937.2642982"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2019.00039"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/sp.2017.31"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","unstructured":"Shuzheng Gao Cuiyun Gao Wenchao Gu and Michael Lyu. 2024. Search-Based LLMs for Code Optimization. doi:10.48550\/arXiv.2408.12159 arXiv:2408.12159 [cs]. 10.48550\/arXiv.2408.12159","DOI":"10.48550\/arXiv.2408.12159"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.15"},{"key":"e_1_2_1_22_1","volume-title":"CodeEditor","author":"Guo Jiawei","unstructured":"Jiawei Guo, Ziming Li, Xueling Liu, Kaijing Ma, Tianyu Zheng, Zhouliang Yu, Ding Pan, Yizhi LI, Ruibo Liu, Yue Wang, Shuyue Guo, Xingwei Qu, Xiang Yue, Ge Zhang, Wenhu Chen, and Jie Fu. 2024. CodeEditorBench: Evaluating Code Editing Capability of Large Language Models. arXiv:2404.03543 [cs.SE] https:\/\/arxiv.org\/abs\/2404.03543"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/9781119196037"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1145\/2377656.2377657"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion58688.2023.00013"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/3524610.3527923"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/1328279.1328283"},{"key":"e_1_2_1_28_1","unstructured":"Barbara Kitchenham and Stuart Charters. 2007. Guidelines for performing Systematic Literature Reviews in Software Engineering. Technical Report. Keele University."},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3729390"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/3540250.3549081"},{"key":"e_1_2_1_31_1","volume-title":"Ahmed Musa Awon, Daniela Damian, and Bowen Xu.","author":"Li Ze Shi","year":"2024","unstructured":"Ze Shi Li, Nowshin Nawar Arony, Ahmed Musa Awon, Daniela Damian, and Bowen Xu. 2024. AI Tool Use and Adoption in Software Development by Individuals and Organizations: A Grounded Theory Study. arXiv:2406.17325 [cs.SE] https:\/\/arxiv.org\/abs\/2406.17325"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616339"},{"key":"e_1_2_1_33_1","unstructured":"Pengfei Liu Weizhe Yuan Jinlan Fu Zhengbao Jiang Hiroaki Hayashi and Graham Neubig. 2021. Pre-train Prompt and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing. http:\/\/arxiv.org\/abs\/2107.13586 arXiv:2107.13586 [cs]."},{"key":"e_1_2_1_34_1","unstructured":"Xiaoyu Liu Jinu Jang Neel Sundaresan Miltiadis Allamanis and Alexey Svyatkovskiy. 2022. AdaptivePaste: Code Adaptation through Learning Semantics-aware Variable Usage Representations. http:\/\/arxiv.org\/abs\/2205.11023 arXiv:2205.11023 [cs]."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASWEC.2018.00027"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3660811"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSME.2019.00026"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/1932682.1869486"},{"key":"e_1_2_1_39_1","unstructured":"OpenAI. 2024. gpt-4o. https:\/\/platform.openai.com\/docs\/models\/gpt-4o"},{"key":"e_1_2_1_40_1","unstructured":"Shuyin Ouyang Jie M. Zhang Mark Harman and Meng Wang. 2023. LLM is Like a Box of Chocolates: the Nondeterminism of ChatGPT in Code Generation. arXiv:2308.02828 [cs.SE] https:\/\/arxiv.org\/abs\/2308.02828"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2019"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2023.3248113"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377929.3398087"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510216"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC52881.2021.00017"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2043174.2043193"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE55347.2025.00040"},{"key":"e_1_2_1_49_1","unstructured":"Kimi Team. 2025. Kimi K2: Open Agentic Intelligence. arXiv:2507.20534 [cs.LG] https:\/\/arxiv.org\/abs\/2507.20534"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/2931037.2931058"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678576"},{"key":"e_1_2_1_52_1","doi-asserted-by":"crossref","unstructured":"Runchu Tian Yining Ye Yujia Qin Xin Cong Yankai Lin Yinxu Pan Yesai Wu Haotian Hui Weichuan Liu Zhiyuan Liu and Maosong Sun. 2024. DebugBench: Evaluating Debugging Capability of Large Language Models. arXiv:2401.04621 [cs.SE] https:\/\/arxiv.org\/abs\/2401.04621","DOI":"10.18653\/v1\/2024.findings-acl.247"},{"key":"e_1_2_1_53_1","unstructured":"Marko Vasic Aditya Kanade Petros Maniatis David Bieber and Rishabh Singh. 2019. Neural Program Repair by Jointly Learning to Localize and Repair. arXiv:1904.01720 [cs.LG] https:\/\/arxiv.org\/abs\/1904.01720"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","unstructured":"Taiming Wang Yanjie Jiang Chunhao Dong Yuxia Zhang and Hui Liu. 2025. Context-Aware Code Wiring Recommendation with LLM-based Agent. doi:10.48550\/arXiv.2507.01315 arXiv:2507.01315 [cs]. 10.48550\/arXiv.2507.01315","DOI":"10.48550\/arXiv.2507.01315"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/2950290.2983934"},{"key":"e_1_2_1_56_1","volume-title":"Proceedings of the 36th International Conference on Neural Information Processing Systems","author":"Wei Jason","year":"2024","unstructured":"Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2024. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems (New Orleans, LA, USA) (NIPS '22). Curran Associates Inc., Red Hook, NY, USA, Article 1800, 14 pages."},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1145\/2380116.2380145"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-018-9634-5"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3540250"},{"key":"e_1_2_1_60_1","unstructured":"Chunqiu Steven Xia and Lingming Zhang. 2023. Keep the Conversation Going: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT. http:\/\/arxiv.org\/abs\/2304.00385 arXiv:2304.00385 [cs]."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2404.08877"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.1145\/2901739.2901767"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2017.13"},{"key":"e_1_2_1_64_1","doi-asserted-by":"crossref","unstructured":"Mingke Yang Yuming Zhou Bixin Li and Yutian Tang. 2023. On Code Reuse from StackOverflow: An Exploratory Study on Jupyter Notebook. http:\/\/arxiv.org\/abs\/2302.11732 arXiv:2302.11732 [cs].","DOI":"10.22541\/au.167748671.10484077\/v1"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3505243"},{"key":"e_1_2_1_66_1","unstructured":"Zezhou Yang Cuiyun Gao Zhaoqiang Guo Zhenhao Li Kui Liu Xin Xia and Yuming Zhou. 2024. A Survey on Modern Code Review: Progresses Challenges and Opportunities. http:\/\/arxiv.org\/abs\/2405.18216 arXiv:2405.18216 [cs]."},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3395519"},{"key":"e_1_2_1_68_1","unstructured":"Tanghaoran Zhang Xinjun Mao Shangwen Wang Yuxin Zhao Yao Lu Jin Zhang Zhang Zhang Kang Yang and Yue Yu. 2026. AdaptEval: A Benchmark for Evaluating Large Language Models on Code Snippet Adaptation. arXiv:2601.04540 [cs.SE] https:\/\/arxiv.org\/abs\/2601.04540"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1145\/3180155.3180260"},{"key":"e_1_2_1_70_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00046"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE55347.2025"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE55347.2025.00082"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3797148","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T19:01:06Z","timestamp":1782846066000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3797148"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,30]]},"references-count":72,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3797148"],"URL":"https:\/\/doi.org\/10.1145\/3797148","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,30]]}}}