{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T21:03:35Z","timestamp":1771621415844,"version":"3.50.1"},"reference-count":49,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T00:00:00Z","timestamp":1768953600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Software"],"abstract":"<jats:p>Background: The implementation of Large Language Models (LLMs) in software engineering has provided new and improved approaches to code synthesis, testing, and refactoring. However, even with these new approaches, the practical efficacy of LLMs is restricted due to their reliance on user-given prompts. The problem is that these prompts can vary a lot in quality and specificity, which results in inconsistent or suboptimal results for the LLM application. Methods: This research therefore aims to alleviate these issues by developing an LLM-based code assistance prototype with a framework based on Retrieval-Augmented Generation (RAG) that automates the prompt-generation process and improves the outputs of LLMs using contextually relevant external knowledge. Results: The tool aims to reduce dependence on the manual preparation of prompts and enhance accessibility and usability for developers of all experience levels. The tool achieved a Code Correctness Score (CCS) of 162.0 and an Average Code Correctness (ACC) score of 98.8% in the refactoring task. These results can be compared to those of the generated tests, which scored CCS 139.0 and ACC 85.3%, respectively. Conclusions: This research contributes to the growing list of Artificial Intelligence (AI)-powered development tools and offers new opportunities for boosting the productivity of developers.<\/jats:p>","DOI":"10.3390\/software5010004","type":"journal-article","created":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T11:11:17Z","timestamp":1768993877000},"page":"4","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Mitigating Prompt Dependency in Large Language Models: A Retrieval-Augmented Framework for Intelligent Code Assistance"],"prefix":"10.3390","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-9813-7226","authenticated-orcid":false,"given":"Saja","family":"Abufarha","sequence":"first","affiliation":[{"name":"Department of Computer Science, University of Calgary, Calgary, AB T2N 1N4, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6520-0749","authenticated-orcid":false,"given":"Ahmed Al","family":"Marouf","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Calgary, Calgary, AB T2N 1N4, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3439-2917","authenticated-orcid":false,"given":"Jon George","family":"Rokne","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Calgary, Calgary, AB T2N 1N4, Canada"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6657-9738","authenticated-orcid":false,"given":"Reda","family":"Alhajj","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Calgary, Calgary, AB T2N 1N4, Canada"},{"name":"Department of Computer Engineering, Istanbul Medipol University, Istanbul 34810, Turkey"},{"name":"Department of Health Informatics, University of Southern Denmark, 5230 Odense, Denmark"}]}],"member":"1968","published-online":{"date-parts":[[2026,1,21]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Rumpe, B. (2017). Principles of Code Generation. Agile Modeling with UML, Springer.","DOI":"10.1007\/978-3-319-58862-9"},{"key":"ref_2","first-page":"1","article-title":"Software Development Life Cycle (SDLC) Methodologies for Information Systems Project Management","volume":"5","author":"Hossain","year":"2023","journal-title":"Int. J. Multidiscip. Res."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Lima, D.L., Santos, R.d.S., Garcia, G.P., da Silva, S.S., Franca, C., and Capretz, L.F. (2023). Software Testing and Code Refactoring: A Survey with Practitioners. arXiv.","DOI":"10.1109\/ICSME58846.2023.00064"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1952","DOI":"10.30574\/ijsra.2024.13.1.1837","article-title":"Integrating Generative AI into the Software Development Lifecycle: Impacts on Code Quality and Maintenance","volume":"13","author":"Ayyappa","year":"2024","journal-title":"Int. J. Sci. Res. Arch."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"726","DOI":"10.18421\/TEM131-76","article-title":"A Comparative Review of AI Techniques for Automated Code Generation in Software Development: Advancements, Challenges, and Future Directions","volume":"13","author":"Odeh","year":"2024","journal-title":"TEM J."},{"key":"ref_6","unstructured":"Minaee, S., Mikolov, T., Nikzad, N., Chenaghlu, M., Socher, R., Amatriain, X., and Gao, J. (2025). Large Language Models: A Survey. arXiv."},{"key":"ref_7","unstructured":"Genkina, D. (2025, December 06). AI Prompt Engineering Is Dead: Long Live AI Prompt Engineering, IEEE Spectrum, Available online: https:\/\/spectrum.ieee.org\/prompt-engineering-is-dead."},{"key":"ref_8","unstructured":"Rayhan, A. (2023). Mastering Prompt Engineering Techniques for Creating Powerful and Effective AI Language Models, Rayhans. Available online: https:\/\/www.kobo.com\/ca\/en\/ebook\/mastering-prompt-engineering."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"544","DOI":"10.1136\/amiajnl-2011-000464","article-title":"Natural Language Processing: An Introduction","volume":"18","author":"Nadkarni","year":"2011","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_10","unstructured":"Bartczak, Z. (2024). From RAG to Riches: Evaluating the Benefits of Retrieval-Augmented Generation in SQL Database Querying. [Master\u2019s Thesis, Uppsala University]."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Khurana, A., Subramonyam, H., and Chilana, P.K. (2024, January 18\u201321). Why and When LLM-Based Assistants Can Go Wrong: Investigating the Effectiveness of Prompt-Based Interactions for Software Help-Seeking. Proceedings of the 29th International Conference on Intelligent User Interfaces (IUI \u201924), Greenville, SC, USA.","DOI":"10.1145\/3640543.3645200"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"14","DOI":"10.4236\/jcc.2024.1210002","article-title":"Prompt Engineering Importance and Applicability with Generative AI","volume":"12","author":"Bansal","year":"2024","journal-title":"J. Comput. Commun."},{"key":"ref_13","unstructured":"Murr, L., Grainger, M., and Gao, D. (2023). Testing LLMs on Code Generation with Varying Levels of Prompt Specificity. arXiv."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Pinto, G., De Souza, C., Neto, J.B., Souza, A., Gotto, T., and Monteiro, E. (2024, January 14\u201320). Lessons from Building StackSpot AI: A Contextualized AI Coding Assistant. Proceedings of the 46th International Conference on Software Engineering: Software Engineering in Practice, Lisbon, Portugal.","DOI":"10.1145\/3639477.3639751"},{"key":"ref_15","unstructured":"Khaliq, Z., Farooq, S.U., and Khan, D.A. (2022). Artificial Intelligence in Software Testing: Impact, Problems, Challenges and Prospect. arXiv."},{"key":"ref_16","first-page":"1","article-title":"A Brief Review of the Development Path of Artificial Intelligence and Its Subfields","volume":"10","author":"Krithiga","year":"2023","journal-title":"Int. J. Eng. Technol. Manag. Res."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Hadi, M.U., Al Tashi, Q., Qureshi, R., Shah, A., Muneer, A., Irfan, M., Zafar, A., Shaikh, M.B., Akhtar, N., and Hassan, S.Z. (2023). A Survey on Large Language Models: Applications, Challenges, Limitations, and Practical Usage. TechRxiv.","DOI":"10.36227\/techrxiv.23589741.v1"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Marvin, G., Hellen, N., Jjingo, D., and Nakatumba-Nabende, J. (2024). Prompt Engineering in Large Language Models. Data Intelligence and Cognitive Informatics, Springer Nature.","DOI":"10.1007\/978-981-99-7962-2_30"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"118","DOI":"10.21541\/apjess.1293702","article-title":"Is ChatGPT Leading Generative AI? What Is Beyond Expectations?","volume":"11","author":"Karaarslan","year":"2023","journal-title":"Acad. Platf. J. Eng. Smart Syst."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Xie, G., Xu, J., Yang, Y., Ding, Y., and Zhang, S. (2024). Large Language Models as Efficient Reward Function Searchers for Custom-Environment Multi-Objective Reinforcement Learning. arXiv.","DOI":"10.1609\/aaai.v39i28.35316"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Kong, A., Zhao, S., Chen, H., Li, Q., Qin, Y., Sun, R., Zhou, X., Wang, E., and Dong, X. (2024). Better Zero-Shot Reasoning with Role-Play Prompting. arXiv.","DOI":"10.18653\/v1\/2024.naacl-long.228"},{"key":"ref_22","unstructured":"Han, Z., and Wang, Z. (November, January 28). Rethinking the Role-Play Prompting in Mathematical Reasoning Tasks. Proceedings of the 1st Workshop on Efficiency, Security, and Generalization of Multimedia Foundation Models (ESGMFM \u201924), Melbourne, VIC, Australia."},{"key":"ref_23","unstructured":"Qian, C., Cong, X., Yang, C., Chen, W., Su, Y., Xu, J., Liu, Z., and Sun, M. (2023). Communicative Agents for Software Development. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Wang, N., Peng, Z., Que, H., Liu, J., Zhou, W., Wu, Y., Guo, H., Gan, R., Ni, Z., and Yang, J. (2024). RoleLLM: Benchmarking, Eliciting, and Enhancing Role-Playing Abilities of Large Language Models. arXiv.","DOI":"10.18653\/v1\/2024.findings-acl.878"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Xian, J., Teofili, T., Pradeep, R., and Lin, J. (2023). Vector Search with OpenAI Embeddings: Lucene Is All You Need. arXiv.","DOI":"10.1145\/3616855.3635691"},{"key":"ref_26","unstructured":"Mickel, M. (2024). Development and Optimization of a Retrieval Augmented Generation System for Enhanced Conversational AI Assistance. [Ph.D. Thesis, Universit\u00e0 degli Studi di Padova]."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Parvez, M.R., Ahmad, W., Chakraborty, S., Ray, B., and Chang, K.-W. (2021). Retrieval augmented code generation and summarization. Findings of the Association for Computational Linguistics: EMNLP 2021, Association for Computational Linguistics.","DOI":"10.18653\/v1\/2021.findings-emnlp.232"},{"key":"ref_28","unstructured":"Tao, Y., Qin, Y., and Liu, Y. (2025). Retrieval-Augmented Code Generation: A Survey with Focus on Repository-Level Approaches. arXiv."},{"key":"ref_29","first-page":"1","article-title":"Large language model-aware in-context learning for code generation","volume":"34","author":"Li","year":"2025","journal-title":"ACM Trans. Softw. Eng. Methodol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Patel, A., Reddy, S., Bahdanau, D., and Dasigi, P. (2024). Evaluating in-context learning of libraries for code generation. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies ((Volume 1: Long Papers), Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.naacl-long.161"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wang, Z., Zhang, T., Wang, Y., and Lu, S. (2024). CodeRAG-Bench: Can Retrieval Augment Code Generation?. arXiv.","DOI":"10.18653\/v1\/2025.findings-naacl.176"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"128596","DOI":"10.1016\/j.eswa.2025.128596","article-title":"Retrieval-augmented code completion for local projects using large language models","volume":"292","author":"Hostnik","year":"2025","journal-title":"Expert Syst. Appl."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Yetistiren, B., Ozsoy, I., and Tuzun, E. (2022, January 14\u201318). Assessing the Quality of GitHub Copilot\u2019s Code Generation. Proceedings of the 18th International Conference on Predictive Models and Data Analytics in Software Engineering, Singapore.","DOI":"10.1145\/3558489.3559072"},{"key":"ref_34","unstructured":"Yetistiren, B., Ozsoy, I., Ayerdem, M., and Tuzun, E. (2023). Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT. arXiv."},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Kazemitabaar, M., Ye, R., Wang, X., Henley, A.Z., Denny, P., Craig, M., and Grossman, T. (2024, January 11\u201316). CodeAid: Evaluating a Classroom Deployment of an LLM-Based Programming Assistant That Balances Student and Educator Needs. Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI \u201924), Honolulu, HI, USA.","DOI":"10.1145\/3613904.3642773"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Barron, R.C., Grantcharov, V., Wanna, S., Eren, M.E., Bhattarai, M., Solovyev, N., Tompkins, G., Nicholas, C., Rasmussen, K.\u00d8., and Matuszek, C. (2024). Domain-Specific Retrieval-Augmented Generation Using Vector Stores, Knowledge Graphs, and Tensor Factorization. arXiv.","DOI":"10.1109\/ICMLA61862.2024.00258"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Jorgensen, P.C. (2013). Software Testing: A Craftsman\u2019s Approach, CRC Press.","DOI":"10.1201\/9781439889503"},{"key":"ref_38","unstructured":"Fowler, M. (2018). Refactoring: Improving the Design of Existing Code, Addison-Wesley Professional."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Douze, M., Guzhva, A., Deng, C., Johnson, J., Szilvasy, G., Mazar\u00e9, P.-E., Lomeli, M., Hosseini, L., and J\u00e9gou, H. (2025). The Faiss Library. arXiv.","DOI":"10.1109\/TBDATA.2025.3618474"},{"key":"ref_40","unstructured":"Chen, M., Tworek, J., Jun, H., Yuan, Q., De Oliveira Pinto, H.P., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., and Brockman, G. (2021). Evaluating Large Language Models Trained on Code. arXiv."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1109\/52.819971","article-title":"What Is Software Testing? Furthermore, Why Is It So Hard?","volume":"17","author":"Whittaker","year":"2000","journal-title":"IEEE Softw."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Dhruv, A., and Dubey, A. (2024). Leveraging Large Language Models for Code Translation and Software Development in Scientific Computing. arXiv.","DOI":"10.1145\/3732775.3733572"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P.N., and Inkpen, K. (2019, January 4\u20139). Guidelines for Human\u2013AI Interaction. Proceedings of the CHI Conference on Human Factors in Computing Systems, Glasgow, UK.","DOI":"10.1145\/3290605.3300233"},{"key":"ref_44","first-page":"2503","article-title":"Hidden Technical Debt in Machine Learning Systems","volume":"28","author":"Sculley","year":"2015","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_45","first-page":"65030","article-title":"Grammar prompting for domain-specific language generation with large language models","volume":"36","author":"Wang","year":"2023","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Nielsen, J. (1994). Usability Engineering, Morgan Kaufmann.","DOI":"10.1016\/B978-0-08-052029-2.50009-7"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Gadiraju, U., M\u00f6ller, S., N\u00f6llenburg, M., Saupe, D., Egger-Lampl, S., Archambault, D., and Fisher, B. (2017). Crowdsourcing versus the laboratory: Towards human-centered experiments using the crowd. Evaluation in the Crowd. Crowdsourcing and Human-Centered Experiments; Revised Contributions, Springer International Publishing.","DOI":"10.1007\/978-3-319-66435-4_2"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"74","DOI":"10.1145\/2408776.2408794","article-title":"The Tail at Scale","volume":"56","author":"Dean","year":"2013","journal-title":"Commun. ACM"},{"key":"ref_49","first-page":"9459","article-title":"Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks","volume":"33","author":"Lewis","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."}],"container-title":["Software"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2674-113X\/5\/1\/4\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,21]],"date-time":"2026-01-21T12:02:31Z","timestamp":1768996951000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2674-113X\/5\/1\/4"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1,21]]},"references-count":49,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["software5010004"],"URL":"https:\/\/doi.org\/10.3390\/software5010004","relation":{},"ISSN":["2674-113X"],"issn-type":[{"value":"2674-113X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1,21]]}}}