{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:43:52Z","timestamp":1760114632999,"version":"build-2065373602"},"reference-count":52,"publisher":"MDPI AG","issue":"9","license":[{"start":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T00:00:00Z","timestamp":1726617600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001871","name":"Funda\u00e7\u00e3o para a Ci\u00eancia e a Tecnologia","doi-asserted-by":"publisher","award":["UIDB\/04466\/2020","UIDP\/04466\/2020","FKZ 13N16581","13N16585"],"award-info":[{"award-number":["UIDB\/04466\/2020","UIDP\/04466\/2020","FKZ 13N16581","13N16585"]}],"id":[{"id":"10.13039\/501100001871","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100002347","name":"Bundesministerium f\u00fcr Bildung und Forschung","doi-asserted-by":"publisher","award":["UIDB\/04466\/2020","UIDP\/04466\/2020","FKZ 13N16581","13N16585"],"award-info":[{"award-number":["UIDB\/04466\/2020","UIDP\/04466\/2020","FKZ 13N16581","13N16585"]}],"id":[{"id":"10.13039\/501100002347","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>Software security is an important topic that is gaining more and more attention due to the rising number of publicly known cybersecurity incidents. Previous research has shown that one way to address software security is by means of a serious game, the CyberSecurity Challenges, which are designed to raise awareness of software developers of secure coding guidelines. This game, proven to be very successful in the industry, makes use of an artificial intelligence technique (laddering technique) to implement a chatbot for human\u2013machine interaction. Recent advances in machine learning have led to a breakthrough, with the implementation and release of large language models, now freely available to the public. Such models are trained on a large amount of data and are capable of analyzing and interpreting not only natural language but also source code in different programming languages. With the advent of ChatGPT, and previous state-of-the-art research in secure software development, a natural question arises: to what extent can ChatGPT aid software developers in writing secure software? In this work, we draw on our experience in the industry, and also on extensive previous work to analyze and reflect on how to use ChatGPT to aid secure software development. Towards this, we conduct two experiments with large language models. Our engagements with ChatGPT and our experience in the field allow us to draw conclusions on the advantages, disadvantages, and limitations of the usage of this new technology.<\/jats:p>","DOI":"10.3390\/info15090572","type":"journal-article","created":{"date-parts":[[2024,9,18]],"date-time":"2024-09-18T09:49:19Z","timestamp":1726652959000},"page":"572","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":5,"title":["May the Source Be with You: On ChatGPT, Cybersecurity, and Secure Coding"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1462-6701","authenticated-orcid":false,"given":"Tiago","family":"Espinha Gasiba","sequence":"first","affiliation":[{"name":"Siemens AG, 81739 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1867-1542","authenticated-orcid":false,"given":"Andrei-Cristian","family":"Iosif","sequence":"additional","affiliation":[{"name":"Siemens AG, 81739 Munich, Germany"},{"name":"Wirtschaftsinformatik, Institut f\u00fcr Schutz und Zuverl\u00e4ssigkeit, Universit\u00e4t der Bundeswehr M\u00fcnchen, 85579 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9297-7502","authenticated-orcid":false,"given":"Ibrahim","family":"Kessba","sequence":"additional","affiliation":[{"name":"Siemens AG, 81739 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6139-5048","authenticated-orcid":false,"given":"Sathwik","family":"Amburi","sequence":"additional","affiliation":[{"name":"TUM School of Computation, Information and Technology, Technische Universit\u00e4t M\u00fcnchen, 85748 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4286-3184","authenticated-orcid":false,"given":"Ulrike","family":"Lechner","sequence":"additional","affiliation":[{"name":"Wirtschaftsinformatik, Institut f\u00fcr Schutz und Zuverl\u00e4ssigkeit, Universit\u00e4t der Bundeswehr M\u00fcnchen, 85579 Munich, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2725-7629","authenticated-orcid":false,"given":"Maria","family":"Pinto-Albuquerque","sequence":"additional","affiliation":[{"name":"Instituto Universit\u00e1rio de Lisboa (ISCTE-IUL), ISTAR, 1600-189 Lisboa, Portugal"}]}],"member":"1968","published-online":{"date-parts":[[2024,9,18]]},"reference":[{"key":"ref_1","unstructured":"(2014). Systems and Software Engineering\u2014Systems and Software Quality Requirements and Evaluation (SQuaRE)\u2014Guide to SQuaRE (Standard No. ISO\/IEC 25000:2014)."},{"key":"ref_2","unstructured":"(2018). Security for Industrial Automation and Control Systems\u2014Part 4-1: Secure Product Development Lifecycle Requirements (Standard No. DIN EN\/IEC 62443-4-1:2018)."},{"key":"ref_3","unstructured":"Bagnara, R., Bagnara, A., and Hill, P.M. (2022). Coding Guidelines and Undecidability. arXiv."},{"key":"ref_4","unstructured":"Patel, S. (2020, July 18). 2019 Global Developer Report: DevSecOps Finds Security Roadblocks Divide Teams. Available online: https:\/\/about.gitlab.com\/blog\/2019\/07\/15\/global-developer-report\/."},{"key":"ref_5","unstructured":"Gasiba, T.E. (2021). Raising Awareness on Secure Coding in the Industry through CyberSecurity Challenges. [Ph.D. Thesis, Universit\u00e4t der Bundeswehr M\u00fcnchen]."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"H\u00e4nsch, N., and Benenson, Z. (2014, January 1\u20135). Specifying IT Security Awareness. Proceedings of the 25th International Workshop on Database and Expert Systems Applications, Munich, Germany.","DOI":"10.1109\/DEXA.2014.71"},{"key":"ref_7","unstructured":"Linux Fountation (2024, July 24). Secure Software Development Education 2024 Survey. Available online: https:\/\/www.linuxfoundation.org\/research\/software-security-education-study."},{"key":"ref_8","unstructured":"(2023, May 08). EU Artificial Intelligence Act. Available online: https:\/\/artificialintelligenceact.eu\/."},{"key":"ref_9","unstructured":"(2023, May 08). AI.gov: Making AI Work for the American People, Available online: https:\/\/ai.gov\/."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"449","DOI":"10.1016\/j.cpet.2021.07.001","article-title":"A Brief History of AI: How to Prevent Another Winter (A Critical Review)","volume":"16","author":"Toosi","year":"2021","journal-title":"PET Clin."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1186\/s42400-020-00064-4","article-title":"Sifu\u2014A CyberSecurity Awareness Platform with Challenge Assessment and Intelligent Coach","volume":"3","author":"Gasiba","year":"2020","journal-title":"Cybersecurity"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Rietz, T., and Maedche, A. (2019, January 23\u201327). LadderBot: A Requirements Self-Elicitation System. Proceedings of the 2019 IEEE 27th International Requirements Engineering Conference (RE), Jeju, Republic of Korea.","DOI":"10.1109\/RE.2019.00045"},{"key":"ref_13","unstructured":"(2023, January 23). OpenAI LP. ChatGPT. Available online: https:\/\/chat.openai.com\/."},{"key":"ref_14","first-page":"2:1","article-title":"I\u2019m Sorry Dave, I\u2019m Afraid I Can\u2019t Fix Your Code: On ChatGPT, CyberSecurity, and Secure Coding","volume":"Volume 112","year":"2023","journal-title":"Proceedings of the 4th International Computer Programming Education Conference (ICPEC 2023)"},{"key":"ref_15","unstructured":"(2024, May 08). Artificial Intelligence at NIST, Available online: https:\/\/www.nist.gov\/artificial-intelligence."},{"key":"ref_16","unstructured":"(2024, May 08). AI Risk Management Framework, Available online: https:\/\/www.nist.gov\/itl\/ai-risk-management-framework."},{"key":"ref_17","unstructured":"(2013). Information Technology\u2014Security Techniques\u2014Information Security Management Systems\u2014Requirements (Standard No. ISO\/IEC 27001:2013)."},{"key":"ref_18","unstructured":"(2022). Information Security, Cybersecurity and Privacy Protection\u2014Information Security Controls (Standard No. ISO\/IEC 27002:2022)."},{"key":"ref_19","unstructured":"OWASP Foundation (2024, May 24). OWASP Top 10 for LLMs. Available online: https:\/\/owasp.org\/www-project-top-10-for-large-language-model-applications."},{"key":"ref_20","unstructured":"MITRE Corporation (2024, July 24). ATLAS Matrix. Available online: https:\/\/atlas.mitre.org\/matrices\/ATLAS."},{"key":"ref_21","unstructured":"(2019). Programming Languages\u2014Guidance to Avoiding Vulnerabilities in Programming Languages\u2014Part 1: Language-Independent Guidance (Standard No. ISO\/IEC 24772-1:2019)."},{"key":"ref_22","unstructured":"Radford, A., and Narasimhan, K. (2024, September 06). Improving Language Understanding by Generative Pre-Training. In Proceedings of the Improving Language Understanding by Generative Pre-Training, Pre-Print 2018. Available online: https:\/\/hayate-lab.com\/wp-content\/uploads\/2023\/05\/43372bfa750340059ad87ac8e538c53b.pdf."},{"key":"ref_23","unstructured":"Brown, T.B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020, January 6\u201312). Language models are few-shot learners. Proceedings of the 34th International Conference on Neural Information Processing Systems (NIPS \u201920), Red Hook, NY, USA."},{"key":"ref_24","unstructured":"Meta (2024, August 03). LLaMA: Large Language Model-Based Automated Assistant. J. AI Res. 2022. Available online: https:\/\/llama.meta.com."},{"key":"ref_25","unstructured":"Russell, S., and Norvig, P. (2021). Artificial Intelligence: A Modern Approach, Global Edition, Pearson Education."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"8858010","DOI":"10.1155\/2020\/8858010","article-title":"A Survey of Automatic Software Vulnerability Detection, Program Repair, and Defect Prediction Techniques","volume":"2020","author":"Shen","year":"2020","journal-title":"Secur. Commun. Netw."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Fu, M., and Tantithamthavorn, C. (2022, January 23\u201324). LineVul: A transformer-based line-level vulnerability prediction. Proceedings of the 19th International Conference on Mining Software Repositories, Pittsburgh, PA, USA.","DOI":"10.1145\/3524842.3528452"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, Y., Wang, S., and Nguyen, T.N. (2021, January 23\u201328). Vulnerability detection with fine-grained interpretations. Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE 2021), New York, NY, USA.","DOI":"10.1145\/3468264.3468597"},{"key":"ref_29","unstructured":"GitHub (1970, January 01). GitHub Copilot. Available online: https:\/\/copilot.github.com\/."},{"key":"ref_30","unstructured":"Niu, L., Mirza, S., Maradni, Z., and P\u00f6pper, C. (2023, January 9\u201311). {CodexLeaks}: Privacy leaks from code generation language models in {GitHub} copilot. Proceedings of the 32nd USENIX Security Symposium (USENIX Security 23), Anaheim, CA, USA."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Perry, N., Srivastava, M., Kumar, D., and Boneh, D. (2023, January 5\u20139). Do Users Write More Insecure Code with AI Assistants?. Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CCS \u201923), Melbourne, Australia.","DOI":"10.1145\/3576915.3623157"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., and Karri, R. (2022, January 22\u201326). Asleep at the Keyboard? Assessing the Security of GitHub Copilot\u2019s Code Contributions. Proceedings of the 2022 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA.","DOI":"10.1109\/SP46214.2022.9833571"},{"key":"ref_33","unstructured":"GitHub (2024, July 07). CodeQL. Available online: https:\/\/codeql.github.com\/."},{"key":"ref_34","unstructured":"MITRE Corporation (2023, July 24). CWE Top 25 Most Dangerous Software Weaknesses. Available online: https:\/\/cwe.mitre.org\/top25\/."},{"key":"ref_35","unstructured":"(2024, May 08). AI TDD: You Write Tests, AI Generates Code. Available online: https:\/\/wonderwhy-er.medium.com\/ai-tdd-you-write-tests-ai-generates-code-c8ad41813c0a."},{"key":"ref_36","unstructured":"MITRE (2020, February 04). Common Weakness Enumeration. Available online: https:\/\/cwe.mitre.org\/."},{"key":"ref_37","unstructured":"Badshah, S., and Sajjad, H. (2024). Quantifying the Capabilities of LLMs across Scale and Precision. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Omar, M. (2023). Detecting software vulnerabilities using Language Models. arXiv.","DOI":"10.1109\/CSR57506.2023.10224924"},{"key":"ref_39","unstructured":"Shestov, A., Levichev, R., Mussabayev, R., Maslov, E., Cheshkov, A., and Zadorozhny, P. (2024). Finetuning Large Language Models for Vulnerability Detection. arXiv."},{"key":"ref_40","unstructured":"Jensen, R.I.T., Tawosi, V., and Alamir, S. (2024). Software Vulnerability and Functionality Assessment using LLMs. arXiv."},{"key":"ref_41","unstructured":"Li, Z., Dutta, S., and Naik, M. (2024). LLM-Assisted Static Analysis for Detecting Security Vulnerabilities. arXiv."},{"key":"ref_42","unstructured":"Tamberg, K., and Bahsi, H. (2024). Harnessing Large Language Models for Software Vulnerability Detection: A Comprehensive Benchmarking Study. arXiv."},{"key":"ref_43","unstructured":"Tarassow, A. (2023). The potential of LLMs for coding with low-resource and domain-specific programming languages. arXiv."},{"key":"ref_44","unstructured":"Jalil, S. (2023). The Transformative Influence of Large Language Models on Software Development. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., and Wang, H. (2024). Large Language Models for Software Engineering: A Systematic Literature Review. arXiv.","DOI":"10.1145\/3695988"},{"key":"ref_46","unstructured":"GitHub (2024, August 16). Measuring the Impact of GitHub Copilot. Available online: https:\/\/resources.github.com\/learn\/pathways\/copilot\/essentials\/measuring-the-impact-of-github-copilot\/."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Pearce, H., Ahmad, B., Tan, B., Dolan-Gavitt, B., and Karri, R. (2021). Asleep at the Keyboard? Assessing the Security of GitHub Copilot\u2019s Code Contributions. arXiv.","DOI":"10.1109\/SP46214.2022.9833571"},{"key":"ref_48","unstructured":"Tambon, F., Dakhel, A.M., Nikanjam, A., Khomh, F., Desmarais, M.C., and Antoniol, G. (2024). Bugs in Large Language Models Generated Code: An Empirical Study. arXiv."},{"key":"ref_49","unstructured":"Fang, C., Miao, N., Srivastav, S., Liu, J., Zhang, R., Fang, R., Tsang, R., Nazari, N., and Wang, H. (2024). Large Language Models for Code Analysis: Do LLMs Really Do Their Job?. arXiv."},{"key":"ref_50","unstructured":"Degges, R. (2024, August 13). Copilot Amplifies Insecure Codebases by Replicating Vulnerabilities in Your Projects. Snyk Blog 2024. Available online: https:\/\/snyk.io\/blog\/copilot-amplifies-insecure-codebases-by-replicating-vulnerabilities\/."},{"key":"ref_51","unstructured":"Sawers, P. (2024, August 13). Samsung Bans Use of Generative AI Tools Like ChatGPT after April Internal Data Leak. TechCrunch 2023. Available online: https:\/\/techcrunch.com\/2023\/05\/02\/samsung-bans-use-of-generative-ai-tools-like-chatgpt-after-april-internal-data-leak\/."},{"key":"ref_52","unstructured":"Franzen, C. (2024, August 16). The AI Feedback Loop: Researchers Warn of \u2019Model Collapse\u2019 as AI Trains on AI-Generated Content. VentureBeat 2023. Available online: https:\/\/venturebeat.com\/ai\/the-ai-feedback-loop-researchers-warn-of-model-collapse-as-ai-trains-on-ai-generated-content\/."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/9\/572\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T15:58:34Z","timestamp":1760111914000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/15\/9\/572"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,18]]},"references-count":52,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2024,9]]}},"alternative-id":["info15090572"],"URL":"https:\/\/doi.org\/10.3390\/info15090572","relation":{},"ISSN":["2078-2489"],"issn-type":[{"type":"electronic","value":"2078-2489"}],"subject":[],"published":{"date-parts":[[2024,9,18]]}}}