{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,2]],"date-time":"2026-07-02T16:32:25Z","timestamp":1783009945032,"version":"3.54.5"},"reference-count":54,"publisher":"Springer Science and Business Media LLC","issue":"2","license":[{"start":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:00:00Z","timestamp":1720742400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T00:00:00Z","timestamp":1720742400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100001661","name":"Ruprecht-Karls-Universit\u00e4t Heidelberg","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100001661","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Autom Softw Eng"],"published-print":{"date-parts":[[2024,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Code generation has become an integral feature of modern IDEs, gathering significant attention. Notable approaches like GitHub Copilot and TabNine have been proposed to tackle this task. However, these tools may shift code writing tasks towards code reviewing, which involves modification from users. Despite the advantages of user feedback, their responses remain transient and lack persistence across interaction sessions. This is attributed to the inherent characteristics of generative AI models, which require explicit re-training for new data integration. Additionally, the non-deterministic and unpredictable nature of AI-powered models limits thorough examination of their unforeseen behaviors. We propose a methodology named <jats:italic>One-shot Correction<\/jats:italic> to mitigate these issues in natural language to code translation models with no additional re-training. We utilize decomposition techniques to break down code translation into sub-problems. The final code is constructed using code snippets of each query chunk, extracted from user feedback or selectively generated from a generative model. Our evaluation indicates comparable or improved performance compared to other models. Moreover, the methodology offers straightforward and interpretable approaches, which enable in-depth examination of unexpected results and facilitate insights for potential enhancements. We also illustrate that user feedback can substantially improve code translation models without re-training. Ultimately, we develop a preliminary GUI application to demonstrate the utility of our methodology in simplifying customization and assessment of suggested code for users.<\/jats:p>","DOI":"10.1007\/s10515-024-00451-y","type":"journal-article","created":{"date-parts":[[2024,7,12]],"date-time":"2024-07-12T13:04:57Z","timestamp":1720789497000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["Rethinking AI code generation: a one-shot correction approach based on user feedback"],"prefix":"10.1007","volume":"31","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4075-1979","authenticated-orcid":false,"given":"Kim Tuyen","family":"Le","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0150-8220","authenticated-orcid":false,"given":"Artur","family":"Andrzejak","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,7,12]]},"reference":[{"key":"451_CR1","doi-asserted-by":"crossref","unstructured":"Abney, S.P.: Parsing by chunks. In: Computation and Psycholinguistics, Principle-Based Parsing, pp. 257\u2013278 (1992)","DOI":"10.1007\/978-94-011-3474-3_10"},{"key":"451_CR2","doi-asserted-by":"crossref","unstructured":"Ahmad, W.U., Chakraborty, S., Ray, B., et\u00a0al.: Unified pre-training for program understanding and generation (2021). arXiv preprint arXiv:2103.06333","DOI":"10.18653\/v1\/2021.naacl-main.211"},{"key":"451_CR3","doi-asserted-by":"crossref","unstructured":"Asare, O., Nagappan, M., Asokan, N.: Is github\u2019s copilot as bad as humans at introducing vulnerabilities in code? (2022). arXiv preprint arXiv:2204.04741","DOI":"10.1007\/s10664-023-10380-1"},{"issue":"OOPSLA1","key":"451_CR4","doi-asserted-by":"publisher","first-page":"85","DOI":"10.1145\/3586030","volume":"7","author":"S Barke","year":"2023","unstructured":"Barke, S., James, M.B., Polikarpova, N.: Grounded copilot: how programmers interact with code-generating models. Proc. ACM Program. Lang. 7(OOPSLA1), 85\u2013111 (2023)","journal-title":"Proc. ACM Program. Lang."},{"key":"451_CR5","doi-asserted-by":"crossref","unstructured":"Bielik, P., Raychev, V., Vechev, M.: Phog: probabilistic model for code. In: International Conference on Machine Learning, PMLR, pp. 2933\u20132942 (2016)","DOI":"10.1145\/2983990.2984041"},{"issue":"6","key":"451_CR6","doi-asserted-by":"publisher","first-page":"35","DOI":"10.1145\/3582083","volume":"20","author":"C Bird","year":"2022","unstructured":"Bird, C., Ford, D., Zimmermann, T., et al.: Taking flight with copilot: early insights and opportunities of AI-powered pair-programming tools. Queue 20(6), 35\u201357 (2022)","journal-title":"Queue"},{"key":"451_CR7","unstructured":"Cai, Y., Mao, S., Wu, W., et\u00a0al.: (2023) Low-code LLM: visual programming over LLMs. arXiv preprint arXiv:2304.08103"},{"key":"451_CR8","unstructured":"Carpenter, A.: 7 tips to help you learn a new programming language fast (2021). https:\/\/www.codecademy.com\/resources\/blog\/how-to-learn-a-new-programming-language-fast\/. Accessed 22 June 2023"},{"key":"451_CR9","doi-asserted-by":"crossref","unstructured":"Charitsis, C., Piech, C., Mitchell, J.C.: Using NLP to quantify program decomposition in CS1. In: Proceedings of the Ninth ACM Conference on Learning@ Scale, pp. 113\u2013120 (2022)","DOI":"10.1145\/3491140.3528272"},{"key":"451_CR10","unstructured":"Chen, M., Tworek, J., Jun, H., et\u00a0al.: Evaluating large language models trained on code (2021). arXiv preprint arXiv:2107.03374"},{"key":"451_CR11","doi-asserted-by":"publisher","first-page":"111734","DOI":"10.1016\/j.jss.2023.111734","volume":"203","author":"AM Dakhel","year":"2023","unstructured":"Dakhel, A.M., Majdinasab, V., Nikanjam, A., et al.: Github copilot AI pair programmer: Asset or liability? J. Syst. Softw. 203, 111734 (2023)","journal-title":"J. Syst. Softw."},{"key":"451_CR12","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1016\/S0573-8555(06)80002-7","volume":"280","author":"M Egidi","year":"2006","unstructured":"Egidi, M.: Decomposition patterns in problem solving. Contrib. Econ. Anal. 280, 15\u201346 (2006)","journal-title":"Contrib. Econ. Anal."},{"key":"451_CR13","doi-asserted-by":"crossref","unstructured":"Elgohary, A., Meek, C., Richardson, M., et\u00a0al.: Nl-edit: correcting semantic parse errors through natural language interaction (2021). arXiv preprint arXiv:2103.14540","DOI":"10.18653\/v1\/2021.naacl-main.444"},{"key":"451_CR14","doi-asserted-by":"publisher","first-page":"111741","DOI":"10.1016\/j.jss.2023.111741","volume":"203","author":"M Evtikhiev","year":"2023","unstructured":"Evtikhiev, M., Bogomolov, E., Sokolov, Y., et al.: Out of the bleu: How should we assess quality of the code generation models? J. Syst. Softw. 203, 111741 (2023)","journal-title":"J. Syst. Softw."},{"key":"451_CR15","doi-asserted-by":"publisher","first-page":"82","DOI":"10.1162\/tacl_a_00356","volume":"9","author":"A Fan","year":"2021","unstructured":"Fan, A., Gardent, C., Braud, C., et al.: Augmenting transformers with KNN-based composite memory for dialog. Trans. Assoc. Comput. Linguist. 9, 82\u201399 (2021)","journal-title":"Trans. Assoc. Comput. Linguist."},{"key":"451_CR16","doi-asserted-by":"crossref","unstructured":"Feng, Z., Guo, D., Tang, D., et\u00a0al.: Codebert: a pre-trained model for programming and natural languages (2020). arXiv preprint arXiv:2002.08155","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"451_CR17","unstructured":"Gozalo-Brizuela, R., Garrido-Merchan, E.C.: ChatGPT is not all you need. A state of the art review of large generative AI models (2023). arXiv preprint arXiv:2301.04655"},{"key":"451_CR18","doi-asserted-by":"crossref","unstructured":"G\u00fcr, I., Yavuz, S., Su, Y., et\u00a0al.: Dialsql: dialogue based structured query generation. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1339\u20131349 (2018)","DOI":"10.18653\/v1\/P18-1124"},{"key":"451_CR19","doi-asserted-by":"crossref","unstructured":"Heyman, G., Huysegems, R., Justen, P., et\u00a0al.: Natural language-guided programming. In: Proceedings of the 2021 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, pp. 39\u201355 (2021)","DOI":"10.1145\/3486607.3486749"},{"issue":"5","key":"451_CR20","doi-asserted-by":"publisher","first-page":"122","DOI":"10.1145\/2902362","volume":"59","author":"A Hindle","year":"2016","unstructured":"Hindle, A., Barr, E.T., Gabel, M., et al.: On the naturalness of software. Commun. ACM 59(5), 122\u2013131 (2016)","journal-title":"Commun. ACM"},{"key":"451_CR21","doi-asserted-by":"crossref","unstructured":"Imai, S. Is github copilot a substitute for human pair-programming? An empirical study. In: Proceedings of the ACM\/IEEE 44th International Conference on Software Engineering: Companion Proceedings, pp. 319\u2013321 (2022)","DOI":"10.1145\/3510454.3522684"},{"key":"451_CR22","doi-asserted-by":"crossref","unstructured":"Iyer, S., Konstas, I., Cheung, A., et\u00a0al.: Learning a neural semantic parser from user feedback (2017). arXiv preprint arXiv:1704.08760","DOI":"10.18653\/v1\/P17-1089"},{"key":"451_CR23","doi-asserted-by":"publisher","unstructured":"Izadi, M., Gismondi, R., Gousios, G.: Codefill: multi-token code completion by jointly learning from structure and naming sequences. In: Proceedings of the 44th International Conference on Software Engineering. Association for Computing Machinery, New York, NY, USA, pp. 401\u2013412 (2022). https:\/\/doi.org\/10.1145\/3510003.3510172","DOI":"10.1145\/3510003.3510172"},{"key":"451_CR24","unstructured":"Kanerva, P.: Sparse distributed memory and related models. Technical Report (1992)"},{"key":"451_CR25","unstructured":"Khandelwal, U., Fan, A., Jurafsky, D., et\u00a0al.: Nearest neighbor machine translation (2020). arXiv preprint arXiv:2010.00710"},{"key":"451_CR26","doi-asserted-by":"crossref","unstructured":"Kim, S., Zhao, J., Tian, Y., et\u00a0al.: Code prediction by feeding trees to transformers. In: 2021 IEEE\/ACM 43rd International Conference on Software Engineering (ICSE), IEEE, pp. 150\u2013162 (2021)","DOI":"10.1109\/ICSE43902.2021.00026"},{"issue":"1","key":"451_CR27","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1007\/s10618-022-00866-9","volume":"37","author":"KT Le","year":"2023","unstructured":"Le, K.T., Rashidi, G., Andrzejak, A.: A methodology for refined evaluation of neural code completion approaches. Data Min. Knowl. Discov. 37(1), 167\u2013204 (2023)","journal-title":"Data Min. Knowl. Discov."},{"key":"451_CR28","doi-asserted-by":"crossref","unstructured":"Li, J., Wang, Y., Lyu, M.R., et\u00a0al.: Code completion with neural attention and pointer networks (2017). arXiv preprint arXiv:1711.09573","DOI":"10.24963\/ijcai.2018\/578"},{"issue":"6624","key":"451_CR29","doi-asserted-by":"publisher","first-page":"1092","DOI":"10.1126\/science.abq1158","volume":"378","author":"Y Li","year":"2022","unstructured":"Li, Y., Choi, D., Chung, J., et al.: Competition-level code generation with alphacode. Science 378(6624), 1092\u20131097 (2022)","journal-title":"Science"},{"key":"451_CR30","doi-asserted-by":"crossref","unstructured":"Lu, S., Duan, N., Han, H., et\u00a0al.: Reacc: a retrieval-augmented code completion framework (2022). arXiv preprint arXiv:2203.07722","DOI":"10.18653\/v1\/2022.acl-long.431"},{"key":"451_CR31","doi-asserted-by":"crossref","unstructured":"Mohapatra, N., Sarraf, N., et\u00a0al.: Domain based chunking. Int. J. Nat. Lang. Comput. (IJNLC) 10 (2021)","DOI":"10.5121\/ijnlc.2021.10401"},{"key":"451_CR32","doi-asserted-by":"crossref","unstructured":"Nguyen, N., Nadi, S. An empirical evaluation of github copilot\u2019s code suggestions. In: Proceedings of the 19th International Conference on Mining Software Repositories, pp. 1\u20135 (2022)","DOI":"10.1145\/3524842.3528470"},{"key":"451_CR33","unstructured":"Nijkamp, E., Pang, B., Hayashi, H., et\u00a0al.: Codegen: an open large language model for code with multi-turn program synthesis (2022). arXiv preprint arXiv:2203.13474"},{"key":"451_CR34","doi-asserted-by":"crossref","unstructured":"Parvez, M.R., Ahmad, W.U., Chakraborty, S., et\u00a0al.: Retrieval augmented code generation and summarization (2021). arXiv preprint arXiv:2108.11601","DOI":"10.18653\/v1\/2021.findings-emnlp.232"},{"key":"451_CR35","doi-asserted-by":"crossref","unstructured":"Pearce, H., Ahmad, B., Tan, B., et\u00a0al.: Asleep at the keyboard? Assessing the security of github copilot\u2019s code contributions. In: 2022 IEEE Symposium on Security and Privacy (SP), IEEE, pp. 754\u2013768 (2022)","DOI":"10.1109\/SP46214.2022.9833571"},{"key":"451_CR36","doi-asserted-by":"crossref","unstructured":"Pourcel, J., Vu, N.S., French, R.M. Online task-free continual learning with dynamic sparse distributed memory. In: European Conference on Computer Vision, Springer, pp. 739\u2013756 (2022)","DOI":"10.1007\/978-3-031-19806-9_42"},{"key":"451_CR37","doi-asserted-by":"crossref","unstructured":"Ramshaw, L.A., Marcus, M.P.: Text chunking using transformation-based learning. In: Natural language processing using very large corpora, pp. 157\u2013176 (1999)","DOI":"10.1007\/978-94-017-2390-9_10"},{"key":"451_CR38","unstructured":"Ren, S., Guo, D., Lu, S., et\u00a0al.: Codebleu: a method for automatic evaluation of code synthesis (2020). arXiv preprint arXiv:2009.10297"},{"key":"451_CR39","doi-asserted-by":"crossref","unstructured":"Reynolds, L., McDonell, K. Prompt programming for large language models: Beyond the few-shot paradigm. In: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, pp. 1\u20137 (2021)","DOI":"10.1145\/3411763.3451760"},{"key":"451_CR40","doi-asserted-by":"crossref","unstructured":"Schlegel, V., Lang, B., Handschuh, S., et\u00a0al.: Vajra: step-by-step programming with natural language. In: Proceedings of the 24th International Conference on Intelligent User Interfaces, pp. 30\u201339 (2019)","DOI":"10.1145\/3301275.3302267"},{"key":"451_CR41","doi-asserted-by":"crossref","unstructured":"Schumacher, M.E.H., Le, K.T., Andrzejak, A. Improving code recommendations by combining neural and classical machine learning approaches. In: Proceedings of the IEEE\/ACM 42nd International Conference on Software Engineering Workshops, pp. 476\u2013482 (2020)","DOI":"10.1145\/3387940.3391489"},{"issue":"3","key":"451_CR42","first-page":"537","volume":"17","author":"J Shin","year":"2021","unstructured":"Shin, J., Nam, J.: A survey of automatic code generation from natural language. J. Inf. Process. Syst. 17(3), 537\u2013555 (2021)","journal-title":"J. Inf. Process. Syst."},{"key":"451_CR43","doi-asserted-by":"crossref","unstructured":"Su, Y., Hassan\u00a0Awadallah, A., Wang, M., et\u00a0al.: Natural language interfaces with fine-grained user interaction: a case study on web apis. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, pp. 855\u2013864 (2018)","DOI":"10.1145\/3209978.3210013"},{"key":"451_CR44","doi-asserted-by":"crossref","unstructured":"Sun, J., Liao, Q.V., Muller, M., et\u00a0al.: Investigating explainability of generative AI for code through scenario-based design. In: 27th International Conference on Intelligent User Interfaces, pp. 212\u2013228 (2022)","DOI":"10.1145\/3490099.3511119"},{"key":"451_CR45","doi-asserted-by":"crossref","unstructured":"Svyatkovskiy, A., Deng, S.K., Fu, S., et\u00a0al.: Intellicode compose: code generation using transformer. In: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, pp. 1433\u20131443 (2020)","DOI":"10.1145\/3368089.3417058"},{"key":"451_CR46","doi-asserted-by":"crossref","unstructured":"Vaithilingam, P., Zhang, T., Glassman, E.L. Expectation vs. experience: evaluating the usability of code generation tools powered by large language models. In: Chi Conference on Human Factors in Computing Systems Extended Abstracts, pp. 1\u20137 (2022)","DOI":"10.1145\/3491101.3519665"},{"key":"451_CR47","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., et\u00a0al.: Attention is all you need. In: Advances in neural information processing systems, vol. 30 (2017)"},{"key":"451_CR48","doi-asserted-by":"crossref","unstructured":"Wang, L., Xu, W., Lan, Y., et\u00a0al.: Plan-and-solve prompting: improving zero-shot chain-of-thought reasoning by large language models (2023). arXiv preprint arXiv:2305.04091","DOI":"10.18653\/v1\/2023.acl-long.147"},{"key":"451_CR49","first-page":"24824","volume":"35","author":"J Wei","year":"2022","unstructured":"Wei, J., Wang, X., Schuurmans, D., et al.: Chain-of-thought prompting elicits reasoning in large language models. Adv. Neural Inf. Process. Syst. 35, 24824\u201324837 (2022)","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"451_CR50","doi-asserted-by":"crossref","unstructured":"Weisz, J.D., Muller, M., Ross, S.I., et\u00a0al.: Better together? An evaluation of AI-supported code translation. In: 27th International Conference on Intelligent User Interfaces, pp. 369\u2013391 (2022)","DOI":"10.1145\/3490099.3511157"},{"key":"451_CR51","unstructured":"Wu, Y., Rabe, M.N., Hutchins, D., et\u00a0al.: Memorizing transformers (2022). arXiv preprint arXiv:2203.08913"},{"issue":"2","key":"451_CR52","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3487569","volume":"31","author":"FF Xu","year":"2022","unstructured":"Xu, F.F., Vasilescu, B., Neubig, G.: In-ide code generation from natural language: promise and challenges. ACM Trans. Softw. Eng. Methodol. (TOSEM) 31(2), 1\u201347 (2022)","journal-title":"ACM Trans. Softw. Eng. Methodol. (TOSEM)"},{"key":"451_CR53","unstructured":"Yao, S., Yu, D., Zhao, J., et\u00a0al.: Tree of thoughts: deliberate problem solving with large language models (2023). arXiv preprint arXiv:2305.10601"},{"key":"451_CR54","unstructured":"Zhang, T., Damerau, F., Johnson, D.: Text chunking based on a generalization of winnow. J. Mach. Learn. Res. 2(4) (2002)"}],"container-title":["Automated Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10515-024-00451-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10515-024-00451-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10515-024-00451-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T15:32:23Z","timestamp":1730129543000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10515-024-00451-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7,12]]},"references-count":54,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["451"],"URL":"https:\/\/doi.org\/10.1007\/s10515-024-00451-y","relation":{},"ISSN":["0928-8910","1573-7535"],"issn-type":[{"value":"0928-8910","type":"print"},{"value":"1573-7535","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,7,12]]},"assertion":[{"value":"13 November 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 May 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"12 July 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"60"}}