{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,13]],"date-time":"2026-06-13T16:37:35Z","timestamp":1781368655497,"version":"3.54.1"},"reference-count":52,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,8,6]],"date-time":"2025-08-06T00:00:00Z","timestamp":1754438400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Instituto de Ci\u00eancia e Tecnologia Ita\u00fa (ICTi)"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>Generative Adversarial Networks (GANs) using Long Short-Term Memory (LSTM) provide a computationally cheaper approach for text generation compared to large language models (LLMs). The low hardware barrier of training GANs poses a threat because it means more bad actors may use them to mass-produce prompt attack messages against LLM systems. Thus, to better understand the threat of GANs being used for prompt attack generation, we train two well-known GAN architectures, SeqGAN and RelGAN, on prompt attack messages. For each architecture, we evaluate generated prompt attack messages, comparing results with each other, with generated attacks from another computationally cheap approach, a 1-billion-parameter Llama 3.2 small language model (SLM), and with messages from the original dataset. This evaluation suggests that GAN architectures like SeqGAN and RelGAN have the potential to be used in conjunction with SLMs to readily generate malicious prompts that impose new threats against LLM-based systems such as chatbots. Analyzing the effectiveness of state-of-the-art defenses against prompt attacks, we also find that GAN-generated attacks can deceive most of these defenses with varying levels of success with the exception of Meta\u2019s PromptGuard. Further, we suggest an improvement of prompt attack defenses based on the analysis of the language quality of the prompts, which we found to be the weakest point of GAN-generated messages.<\/jats:p>","DOI":"10.3390\/make7030077","type":"journal-article","created":{"date-parts":[[2025,8,6]],"date-time":"2025-08-06T10:13:51Z","timestamp":1754475231000},"page":"77","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Evaluating Prompt Injection Attacks with LSTM-Based Generative Adversarial Networks: A Lightweight Alternative to Large Language Models"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-1170-6201","authenticated-orcid":false,"given":"Sharaf","family":"Rashid","sequence":"first","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St., Cambridge, MA 02139, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0993-784X","authenticated-orcid":false,"given":"Edson","family":"Bollis","sequence":"additional","affiliation":[{"name":"Instituto de Ci\u00eancia e Tecnologia Ita\u00fa, Praca Alfredo Egydio De Souza Aranha, 100, T. Olavo Setubal, Parque Jabaquara, Sao Paulo 04344-902, SP, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2827-7602","authenticated-orcid":false,"given":"Lucas","family":"Pellicer","sequence":"additional","affiliation":[{"name":"Instituto de Ci\u00eancia e Tecnologia Ita\u00fa, Praca Alfredo Egydio De Souza Aranha, 100, T. Olavo Setubal, Parque Jabaquara, Sao Paulo 04344-902, SP, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9045-1563","authenticated-orcid":false,"given":"Darian","family":"Rabbani","sequence":"additional","affiliation":[{"name":"Instituto de Ci\u00eancia e Tecnologia Ita\u00fa, Praca Alfredo Egydio De Souza Aranha, 100, T. Olavo Setubal, Parque Jabaquara, Sao Paulo 04344-902, SP, Brazil"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8963-5074","authenticated-orcid":false,"given":"Rafael","family":"Palacios","sequence":"additional","affiliation":[{"name":"Cybersecurity at MIT Sloan (CAMS), Massachusetts Institute of Technology, 77 Massachusetts Avenue, Cambridge, MA 02139, USA"},{"name":"Institute for Research in Technology, Universidad Pontificia Comillas, Alberto Aguilera 23, 28015 Madrid, Spain"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aneesh","family":"Gupta","sequence":"additional","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St., Cambridge, MA 02139, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9306-1256","authenticated-orcid":false,"given":"Amar","family":"Gupta","sequence":"additional","affiliation":[{"name":"Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, 32 Vassar St., Cambridge, MA 02139, USA"},{"name":"AI Institute for Community-Engaged Research (AI-ICER), The University of Texas at El Paso, 500 West University Avenue, El Paso, TX 79968, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,8,6]]},"reference":[{"key":"ref_1","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems 30 (NIPS 2017), Long Beach, CA, USA."},{"key":"ref_2","unstructured":"Wang, F., Zhang, Z., Zhang, X., Wu, Z., Mo, T., Lu, Q., Wang, W., Li, R., Xu, J., and Tang, X. (2024). A Comprehensive Survey of Small Language Models in the Era of Large Language Models: Techniques, Enhancements, Applications, Collaboration with LLMs, and Trustworthiness. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"175","DOI":"10.1007\/s10462-024-10824-0","article-title":"A survey of safety and trustworthiness of large language models through the lens of verification and validation","volume":"57","author":"Huang","year":"2024","journal-title":"Artif. Intell. Rev."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"100211","DOI":"10.1016\/j.hcc.2024.100211","article-title":"A survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly","volume":"4","author":"Yao","year":"2024","journal-title":"High-Confid. Comput."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"152","DOI":"10.1145\/3712001","article-title":"Security and Privacy Challenges of Large Language Models: A Survey","volume":"57","author":"Das","year":"2025","journal-title":"ACM Comput. Surv."},{"key":"ref_6","unstructured":"Perez, F., and Ribeiro, I. (2022). Ignore previous prompt: Attack techniques for language models. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Liu, Y., Deng, G., Xu, Z., Li, Y., Zheng, Y., Zhang, Y., Zhao, L., Zhang, T., Wang, K., and Liu, Y. (2024). Jailbreaking ChatGPT via Prompt Engineering: An Empirical Study. arXiv.","DOI":"10.1145\/3663530.3665021"},{"key":"ref_8","unstructured":"Rossi, S., Michel, A.M., Mukkamala, R.R., and Thatcher, J.B. (2024). An Early Categorization of Prompt Injection Attacks on Large Language Models. arXiv."},{"key":"ref_9","unstructured":"Salem, A., Paverd, A., and K\u00f6pf, B. (2023). Maatphor: Automated Variant Analysis for Prompt Injection Attacks. arXiv."},{"key":"ref_10","unstructured":"Wu, Y., Schuster, M., Chen, Z., Le, Q.V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., and Macherey, K. (2016). Google\u2019s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. arXiv."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Tevet, G., Habib, G., Shwartz, V., and Berant, J. (2019, January 2\u20137). Evaluating Text GANs as Language Models. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, MN, USA.","DOI":"10.18653\/v1\/N19-1233"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"20210068","DOI":"10.1098\/rspa.2021.0068","article-title":"Inductive biases for deep learning of higher-level cognition","volume":"478","author":"Goyal","year":"2022","journal-title":"Proc. R. Soc. A"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Yu, L., Zhang, W., Wang, J., and Yu, Y. (2017, January 4\u20139). SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient. Proceedings of the AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.","DOI":"10.1609\/aaai.v31i1.10804"},{"key":"ref_14","unstructured":"Nie, W., Narodytska, N., and Patel, A. (2019, January 6\u20139). RelGAN: Relational Generative Adversarial Networks for Text Generation. Proceedings of the International Conference on Learning Representations, New Orleans, LA, USA."},{"key":"ref_15","unstructured":"Pfister, N., Volhejn, V., Knott, M., Arias, S., Bazi\u0144ska, J., Bichurin, M., Commike, A., Darling, J., Dienes, P., and Fiedler, M. (2025). Gandalf the Red: Adaptive Security for LLMs. arXiv."},{"key":"ref_16","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative Adversarial Networks. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, Canada."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1117\/1.1526105","article-title":"Feedback-based architecture for reading courtesy amounts on checks","volume":"12","author":"Palacios","year":"2003","journal-title":"J. Electron. Imaging"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., and Aila, T. (2020, January 13\u201319). Analyzing and Improving the Image Quality of StyleGAN. Proceedings of the 2020 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"ref_19","unstructured":"Brock, A., Donahue, J., and Simonyan, K. (2019). Large Scale GAN Training for High Fidelity Natural Image Synthesis. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Park, T., Liu, M.Y., Wang, T.C., and Zhu, J.Y. (2019, January 15\u201320). Semantic Image Synthesis with Spatially-Adaptive Normalization. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00244"},{"key":"ref_21","unstructured":"Lin, K., Li, D., He, X., Zhang, Z., and Sun, M.T. (2017, January 4\u20139). Adversarial Ranking for Language Generation. Proceedings of the Advances in Neural Information Processing Systems, Long Beach, CA, USA."},{"key":"ref_22","unstructured":"Che, T., Li, Y., Zhang, R., Hjelm, R.D., Li, W., Song, Y., and Bengio, Y. (2017). Maximum-Likelihood Augmented Discrete Generative Adversarial Networks. arXiv."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Guo, J., Lu, S., Cai, H., Zhang, W., Yu, Y., and Wang, J. (2018, January 2\u20137). Long text generation via adversarial training with leaked information. Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, LA, USA.","DOI":"10.1609\/aaai.v32i1.11957"},{"key":"ref_24","unstructured":"Zhang, Y., Gan, Z., Fan, K., Chen, Z., Henao, R., Shen, D., and Carin, L. (2017, January 6\u201311). Adversarial feature matching for text generation. Proceedings of the 34th International Conference on Machine Learning-Volume 70, Sydney, NSW, Australia."},{"key":"ref_25","unstructured":"Fedus, W., Goodfellow, I., and Dai, A.M. (2018). MaskGAN: Better Text Generation via Filling in the _______. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chai, Y., Zhang, H., Yin, Q., and Zhang, J. (2021, January 1\u20136). Counter-Contrastive Learning for Language GANs. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2021, Punta Cana, Dominican Republic.","DOI":"10.18653\/v1\/2021.findings-emnlp.415"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"126352","DOI":"10.1016\/j.neucom.2023.126352","article-title":"Feature-aware conditional GAN for category text generation","volume":"547","author":"Li","year":"2023","journal-title":"Neurocomputing"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Liu, Z., Wang, J., and Liang, Z. (2020, January 7\u201312). CatGAN: Category-aware Generative Adversarial Networks with Hierarchical Evolutionary Learning for Category Text Generation. Proceedings of the AAAI Conference on Artificial Intelligence, New York, NY, USA.","DOI":"10.1609\/aaai.v34i05.6361"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Patwardhan, N., Marrone, S., and Sansone, C. (2023). Transformers in the Real World: A Survey on NLP Applications. Information, 14.","DOI":"10.3390\/info14040242"},{"key":"ref_30","first-page":"1","article-title":"Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond","volume":"18","author":"Yang","year":"2024","journal-title":"ACM Trans. Knowl. Discov. Data"},{"key":"ref_31","unstructured":"Devlin, J., Chang, M.W., Lee, K., and Toutanova, K. (2019, January 2\u20137). BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, MN, USA."},{"key":"ref_32","first-page":"1877","article-title":"Language Models are Few-Shot Learners","volume":"Volume 33","author":"Larochelle","year":"2020","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_33","unstructured":"Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.A., Lacroix, T., Rozi\u00e8re, B., Goyal, N., Hambro, E., and Azhar, F. (2023). LLaMA: Open and Efficient Foundation Language Models. arXiv."},{"key":"ref_34","unstructured":"Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., and Vaughan, A. (2024). The Llama 3 Herd of Models. arXiv."},{"key":"ref_35","unstructured":"Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2021). LoRA: Low-Rank Adaptation of Large Language Models. In Proceedings of the International Conference on Learning Representations. arXiv."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Yi, J., Xie, Y., Zhu, B., Kiciman, E., Sun, G., Xie, X., and Wu, F. (2025;, January 3\u20137). Benchmarking and Defending against Indirect Prompt Injection Attacks on Large Language Models. Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1, Toronto, ON, Canada.","DOI":"10.1145\/3690624.3709179"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Shen, X., Chen, Z., Backes, M., Shen, Y., and Zhang, Y. (2024, January 14\u201318). \u201cDo Anything Now\u201d: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models. Proceedings of the ACM SIGSAC Conference on Computer and Communications Security, Salt Lake City, UT, USA.","DOI":"10.1145\/3658644.3670388"},{"key":"ref_38","unstructured":"Tang, Y., Wang, B., Wang, X., Zhao, D., Liu, J., He, R., and Hou, Y. (2024). RoleBreak: Character Hallucination as a Jailbreak Attack in Role-Playing Systems. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Kang, D., Li, X., Stoica, I., Guestrin, C., Zaharia, M., and Hashimoto, T. (, January May). Exploiting Programmatic Behavior of LLMs: Dual-Use Through Standard Security Attacks. Proceedings of the 2024 IEEE Security and Privacy Workshops (SPW), Los Alamitos, CA, USA.","DOI":"10.1109\/SPW63631.2024.00018"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Zhou, Y., Lu, L., Sun, H., Zhou, P., and Sun, L. (2024). Virtual Context: Enhancing Jailbreak Attacks with Special Token Injection. arXiv.","DOI":"10.18653\/v1\/2024.findings-emnlp.692"},{"key":"ref_41","unstructured":"Liu, D., Yang, M., Qu, X., Zhou, P., Cheng, Y., and Hu, W. (2024). A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Iqbal, U., Kohno, T., and Roesner, F. (2024, January 21\u201323). LLM Platform Security: Applying a Systematic Evaluation Framework to OpenAI\u2019s ChatGPT Plugins. Proceedings of the AAAI\/ACM Conference on AI, Ethics, and Society, San Jose, CA, USA.","DOI":"10.1609\/aies.v7i1.31664"},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1109\/TBDATA.2019.2921572","article-title":"Billion-Scale Similarity Search with GPUs","volume":"7","author":"Johnson","year":"2021","journal-title":"IEEE Trans. Big Data"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Lahitani, A.R., Permanasari, A.E., and Setiawan, N.A. (2016, January 26\u201327). Cosine similarity to determine similarity measure: Study case in online essay assessment. Proceedings of the 2016 4th International Conference on Cyber and IT Service Management, Bandung, Indonesia.","DOI":"10.1109\/CITSM.2016.7577578"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Reuther, A., Kepner, J., Byun, C., Samsi, S., Arcand, W., Bestor, D., Bergeron, B., Gadepally, V., Houle, M., and Hubbell, M. (2018, January 25\u201327). Interactive Supercomputing on 40,000 Cores for Machine Learning and Data Analysis. Proceedings of the 2018 IEEE High Performance extreme Computing Conference (HPEC), Waltham, MA, USA.","DOI":"10.1109\/HPEC.2018.8547629"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W.J. (2002;, January 6\u201312). BLEU: A method for automatic evaluation of machine translation. Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_47","unstructured":"Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., and Artzi, Y. (2020). BERTscore: Evaluating Text Generation with BERT. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"201","DOI":"10.1017\/S0305000900012885","article-title":"Type\/Token Ratios: What do they really tell us?","volume":"14","author":"Richards","year":"1987","journal-title":"J. Child Lang."},{"key":"ref_49","first-page":"2579","article-title":"Visualizing Data using t-SNE","volume":"9","author":"Hinton","year":"2008","journal-title":"J. Mach. Learn. Res."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Chen, Y., Wang, R., Jiang, H., Shi, S., and Xu, R. (2023, January 1\u20134). Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study. Proceedings of the Findings of the Association for Computational Linguistics: IJCNLP-AACL 2023 (Findings), Nusa Dua, Indonesia.","DOI":"10.18653\/v1\/2023.findings-ijcnlp.32"},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Reimers, N., and Gurevych, I. (2019, January 3\u20137). Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Hong Kong, China.","DOI":"10.18653\/v1\/D19-1410"},{"key":"ref_52","unstructured":"Liu, Y., Ott, M., Goyal, N., Du, J., Joshi, M., Chen, D., Levy, O., Lewis, M., Zettlemoyer, L., and Stoyanov, V. (2019). RoBERTa: A Robustly Optimized BERT Pretraining Approach. arXiv."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/3\/77\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:24:15Z","timestamp":1760034255000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/3\/77"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,6]]},"references-count":52,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["make7030077"],"URL":"https:\/\/doi.org\/10.3390\/make7030077","relation":{},"ISSN":["2504-4990"],"issn-type":[{"value":"2504-4990","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,6]]}}}