{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,2]],"date-time":"2026-10-02T13:30:30Z","timestamp":1790947830694,"version":"4.1.0"},"reference-count":33,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,3,5]],"date-time":"2025-03-05T00:00:00Z","timestamp":1741132800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Interface Corporation, Japan"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Information"],"abstract":"<jats:p>In recent years, the misuse of large language models (LLMs) has emerged as a significant issue. This paper focuses on a specific attack method known as the greedy coordinate gradient (GCG) jailbreak attack, which compels LLMs to generate responses beyond ethical boundaries. We have developed a tool to suppress the improper use of LLMs by employing a high-precision detection method that combines syntactic tree analysis with the perplexity of generated text. Furthermore, the tool incorporates one of the small language models (SLMs), the DistilBERT model, to evaluate the harmfulness of sentences, thereby preventing harmful content from entering the LLM. Experimental results demonstrate that the tool effectively detects GCG jailbreak attacks and contributes to the secure usage of LLMs. In the test results, the defense success rate reached 90.8%.<\/jats:p>","DOI":"10.3390\/info16030204","type":"journal-article","created":{"date-parts":[[2025,3,5]],"date-time":"2025-03-05T09:37:52Z","timestamp":1741167472000},"page":"204","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["LLM Abuse Prevention Tool Using GCG Jailbreak Attack Detection and DistilBERT-Based Ethics Judgment"],"prefix":"10.3390","volume":"16","author":[{"given":"Qiuyu","family":"Chen","sequence":"first","affiliation":[{"name":"Graduate School of Sciences and Technology for Innovation, Yamaguchi University, 2-16-1 Tokiwadai, Ube 755-8611, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0579-8501","authenticated-orcid":false,"given":"Shingo","family":"Yamaguchi","sequence":"additional","affiliation":[{"name":"Graduate School of Sciences and Technology for Innovation, Yamaguchi University, 2-16-1 Tokiwadai, Ube 755-8611, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yudai","family":"Yamamoto","sequence":"additional","affiliation":[{"name":"Graduate School of Sciences and Technology for Innovation, Yamaguchi University, 2-16-1 Tokiwadai, Ube 755-8611, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,3,5]]},"reference":[{"key":"ref_1","unstructured":"Xu, Z., Liu, Y., Deng, G., Li, Y., and Picek, S. (2024). LLM Jailbreak Attack versus Defense Techniques\u2013A Comprehensive Study. arXiv."},{"key":"ref_2","unstructured":"Wei, A., Haghtalab, N., and Steinhardt, J. (2023). Jailbroken: How Does LLM Safety Training Fail?. Advances in Neural Information Processing Systems, Proceedings of the 37th International Confernece on Nueral Informaiton Precessing Systems, New Orleans, LA, USA, 10\u201316 December 2023, Curran Associated Inc."},{"key":"ref_3","unstructured":"Zou, A., Wang, Z., Kolter, J.Z., and Fredrikson, M. (2023). Universal and transferable adversarial attacks on aligned language models. arXiv."},{"key":"ref_4","unstructured":"Kumar, A., Agarwal, C., Srinivas, S., Feizi, S., and Lakkaraju, H. (2023). Certifying LLM safety against adversarial prompting. arXiv."},{"key":"ref_5","unstructured":"Robey, A., Wong, E., Hassani, H., and Pappas, G.J. (2023). SmoothLLM: Defending large language models against jailbreaking attacks. arXiv."},{"key":"ref_6","unstructured":"Alon, G., and Kamfonas, M. (2023). Detecting language model attacks with perplexity. arXiv."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Pavlopoulos, J., Sorensen, J., Dixon, L., Thain, N., and Androutsopoulos, I. (2020). Toxicity detection: Does context really matter?. arXiv.","DOI":"10.18653\/v1\/2020.acl-main.396"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Lees, A., Tran, V.Q., Tay, Y., Sorensen, J., Gupta, J., Metzler, D., and Vasserman, L. (2022, January 14\u201318). A new generation of perspective API: Efficient multilingual character-level transformers. Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA.","DOI":"10.1145\/3534678.3539147"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"15009","DOI":"10.1609\/aaai.v37i12.26752","article-title":"A holistic approach to undesired content detection in the real world","volume":"Volume 37","author":"Markov","year":"2023","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"ref_10","first-page":"33","article-title":"A Parse Tree Model for Analyzing And Detecting SQL Injection Vulnerabilities","volume":"6","author":"Ogheneovo","year":"2013","journal-title":"West Afr. J. Ind. Acad. Res."},{"key":"ref_11","unstructured":"Park, G., and Taylor, J.M. (2015). Using syntactic features for phishing detection. arXiv."},{"key":"ref_12","unstructured":"Hu, Z., Wu, G., Mitra, S., Zhang, R., Sun, T., Huang, H., and Swaminathan, V. (2023). Token-level adversarial prompt detection based on perplexity measures and contextual information. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"53","DOI":"10.1093\/llc\/fqp031","article-title":"TreeForm: Explaining and exploring grammar through syntax trees","volume":"25","author":"Derrick","year":"2010","journal-title":"Lit. Linguist. Comput."},{"key":"ref_14","unstructured":"Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. (2024, December 25). Language Models Are Unsupervised Multitask Learners. OpenAI. Available online: https:\/\/api.semanticscholar.org\/CorpusID:160025533."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Hajic, J., Ciaramita, M., Johansson, R., Kawahara, D., Mart\u00ed, M.A., M\u00e0rquez, L., and Zhang, Y. (2009, January 4). The CoNLL-2009 Shared Task: Syntactic and Semantic Dependencies in Multiple Languages. Proceedings of the Thirteenth Conference on Computational Natural Language Learning (CoNLL 2009): Shared Task, Boulder, CO, USA.","DOI":"10.3115\/1596409.1596411"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Manning, C.D., Surdeanu, M., Bauer, J., Finkel, J.R., Bethard, S., and McClosky, D. (2014, January 23\u201324). The Stanford CoreNLP natural language processing toolkit. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations, Baltimore, MD, USA.","DOI":"10.3115\/v1\/P14-5010"},{"key":"ref_17","unstructured":"Riedmiller, M., and Lernen, A. (2014). Multi-Layer Perceptron, University of Freiburg. Machine Learning Lab Special Lecture."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Williams, A., Nangia, N., and Bowman, S.R. (2017). A broad-coverage challenge corpus for sentence understanding through inference. arXiv.","DOI":"10.18653\/v1\/N18-1101"},{"key":"ref_19","unstructured":"Phute, M., Helbling, A., Hull, M., Peng, S., Szyller, S., Cornelius, C., and Chau, D.H. (2023). LLM Self Defense: By Self Examination, LLMs Know They Are Being Tricked. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"325","DOI":"10.1146\/annurev-linguistics-011415-040518","article-title":"Sentiment Analysis: An Overview from Linguistics","volume":"2","author":"Taboada","year":"2016","journal-title":"Annu. Rev. Linguist."},{"key":"ref_21","unstructured":"Sanh, V., Debut, L., Chaumond, J., and Wolf, T. (2019). DistilBERT, a distilled version of BERT: Smaller, faster, cheaper, and lighter. arXiv."},{"key":"ref_22","unstructured":"Sundelin, C. (2024, October 21). Comparing Different Transformer Models\u2019 Performance for Identifying Toxic Language Online. Available online: https:\/\/www.diva-portal.org\/smash\/record.jsf?pid=diva2%3A1784346&dswid=-8091."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Hasan, A., Rugina, I., and Wang, A. (2024). Pruning for protection: Increasing jailbreak resistance in aligned LLMs without fine-tuning. arXiv.","DOI":"10.18653\/v1\/2024.blackboxnlp-1.26"},{"key":"ref_24","unstructured":"Bhardwaj, R., and Poria, S. (2023). Red-teaming large language models using chain of utterances for safety-alignment. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Stahlberg, F., Kumar, A., Alberti, C., and Kumar, S. (2022). Conciseness: An Overlooked Language Task. arXiv.","DOI":"10.18653\/v1\/2022.tsar-1.5"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Hartvigsen, T., Gabriel, S., Palangi, H., Sap, M., Ray, D., and Kamar, E. (2022). ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection. arXiv.","DOI":"10.18653\/v1\/2022.acl-long.234"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S.R. (2018). GLUE: A multi-task benchmark and analysis platform for natural language understanding. arXiv.","DOI":"10.18653\/v1\/W18-5446"},{"key":"ref_28","unstructured":"Jain, N., Schwarzschild, A., Wen, Y., Somepalli, G., Kirchenbauer, J., Chiang, P.Y., Goldblum, M., Saha, A., Geiping, J., and Goldstein, T. (2024, March 17). Baseline Defenses for Adversarial Attacks against Aligned Language Models. Available online: https:\/\/github.com\/neelsjain\/baseline-defenses\/tree\/main."},{"key":"ref_29","first-page":"3146","article-title":"LightGBM: A Highly Efficient Gradient Boosting Decision Tree","volume":"Volume 30","author":"Ke","year":"2017","journal-title":"Advances in Neural Information Processing Systems, Proceedings of the NIPS\u201917: Proceedings of the 31st International Conference on Neural Information Processing Systems, Long Beach, CA, USA, 4\u20139 December 2017"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"De Gibert, O., Perez, N., Garc\u00eda-Pablos, A., and Cuadros, M. (2018). Hate speech dataset from a white supremacy forum. arXiv.","DOI":"10.18653\/v1\/W18-5102"},{"key":"ref_31","unstructured":"Lan, Z. (2022). ALBERT: A Lite BERT for Self-Supervised Learning of Language Representations. arXiv."},{"key":"ref_32","unstructured":"Jigsaw (2024, October 20). Perspective API: Toxicity Detection for Online Conversations. Available online: https:\/\/perspectiveapi.com."},{"key":"ref_33","unstructured":"Chen, Q. (2025, January 10). LLM Abuse Prevention Tool Using GCG Jailbreak Attack Detection and DistilBERT-Based Ethics Judgment. Available online: https:\/\/github.com\/jackson12587\/LLM-abuse-prevention-tool-using-GCG-jailbreak-attack-detection-and-DistilBERT-based-ethics-judgment.git."}],"container-title":["Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/3\/204\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T16:47:42Z","timestamp":1760028462000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2078-2489\/16\/3\/204"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,5]]},"references-count":33,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,3]]}},"alternative-id":["info16030204"],"URL":"https:\/\/doi.org\/10.3390\/info16030204","relation":{},"ISSN":["2078-2489"],"issn-type":[{"value":"2078-2489","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,5]]}}}