{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,27]],"date-time":"2026-05-27T20:03:25Z","timestamp":1779912205411,"version":"3.53.1"},"reference-count":35,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T00:00:00Z","timestamp":1725408000000},"content-version":"vor","delay-in-days":247,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,9,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Modern NLP models are often trained on public datasets drawn from diverse sources, rendering them vulnerable to data poisoning attacks. These attacks can manipulate the model\u2019s behavior in ways engineered by the attacker. One such tactic involves the implantation of backdoors, achieved by poisoning specific training instances with a textual trigger and a target class label. Several strategies have been proposed to mitigate the risks associated with backdoor attacks by identifying and removing suspected poisoned examples. However, we observe that these strategies fail to offer effective protection against several advanced backdoor attacks. To remedy this deficiency, we propose a novel defensive mechanism that first exploits training dynamics to identify poisoned samples with high precision, followed by a label propagation step to improve recall and thus remove the majority of poisoned instances. Compared with recent advanced defense methods, our method considerably reduces the success rates of several backdoor attacks while maintaining high classification accuracy on clean test sets.<\/jats:p>","DOI":"10.1162\/tacl_a_00684","type":"journal-article","created":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T13:36:48Z","timestamp":1725457008000},"page":"996-1010","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":4,"title":["SEEP: Training Dynamics Grounds Latent Representation Search for Mitigating Backdoor Poisoning Attacks"],"prefix":"10.1162","volume":"12","author":[{"given":"Xuanli","family":"He","sequence":"first","affiliation":[{"name":"University College London, United Kingdom. z.xuanli.he@gmail.com"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qiongkai","family":"Xu","sequence":"additional","affiliation":[{"name":"Macquarie University, Australia. qiongkai.xu@mq.edu.au"},{"name":"The University of Melbourne, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jun","family":"Wang","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Australia. jun2@student.unimelb.edu.au"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Benjamin I. P.","family":"Rubinstein","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Australia. benjamin.rubinstein@unimelb.edu.au"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Trevor","family":"Cohn","sequence":"additional","affiliation":[{"name":"The University of Melbourne, Australia. trevor.cohn@unimelb.edu.au"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2024,9,4]]},"reference":[{"key":"2024090413342253400_bib1","article-title":"Poisoning and backdooring contrastive learning","volume-title":"International Conference on Learning Representations","author":"Carlini","year":"2022"},{"key":"2024090413342253400_bib2","article-title":"Detecting backdoor attacks on deep neural networks by activation clustering","author":"Chen","year":"2018","journal-title":"CoRR"},{"key":"2024090413342253400_bib3","article-title":"Badpre: Task-agnostic backdoor attacks to pre-trained NLP foundation models","volume-title":"International Conference on Learning Representations","author":"Chen","year":"2022"},{"key":"2024090413342253400_bib4","article-title":"Expose backdoors on the way: A feature-based efficient defense against textual backdoor attacks","author":"Chen","year":"2022","journal-title":"arXiv preprint arXiv:2210.07907"},{"key":"2024090413342253400_bib5","article-title":"Targeted backdoor attacks on deep learning systems using data poisoning","author":"Chen","year":"2017","journal-title":"Journal of Environmental Sciences (China) English Ed"},{"key":"2024090413342253400_bib6","doi-asserted-by":"publisher","first-page":"138872","DOI":"10.1109\/ACCESS.2019.2941376","article-title":"A backdoor attack against LSTM-based text classification systems","volume":"7","author":"Dai","year":"2019","journal-title":"IEEE Access"},{"issue":"4","key":"2024090413342253400_bib7","doi-asserted-by":"publisher","first-page":"2349","DOI":"10.1109\/TDSC.2021.3055844","article-title":"Design and evaluation of a multi-domain trojan detection method on deep neural networks","volume":"19","author":"Gao","year":"2022","journal-title":"IEEE Transactions on Dependable and Secure Computing"},{"key":"2024090413342253400_bib8","article-title":"What doesn\u2019t kill you makes you robust(er): How to adversarially train against data poisoning","author":"Geiping","year":"2022"},{"key":"2024090413342253400_bib9","article-title":"Badnets: Identifying vulnerabilities in the machine learning model supply chain","author":"Tianyu","year":"2017","journal-title":"arXiv preprint arXiv:1708.06733"},{"key":"2024090413342253400_bib10","article-title":"Imbert: Making bert immune to insertion-based backdoor attacks","author":"He","year":"2023","journal-title":"arXiv preprint arXiv:2305.16503"},{"key":"2024090413342253400_bib11","article-title":"Mitigating backdoor poisoning attacks through the lens of spurious correlation","author":"He","year":"2023","journal-title":"arXiv preprint arXiv:2305.11596"},{"key":"2024090413342253400_bib12","doi-asserted-by":"publisher","first-page":"1875","DOI":"10.18653\/v1\/N18-1170","article-title":"Adversarial example generation with syntactically controlled paraphrase networks","volume-title":"Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)","author":"Iyyer","year":"2018"},{"key":"2024090413342253400_bib13","article-title":"Mistral 7b","author":"Jiang","year":"2023","journal-title":"arXiv preprint arXiv:2310.06825"},{"key":"2024090413342253400_bib14","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1007\/978-3-319-46478-7_5","article-title":"Learning visual features from large weakly supervised data","volume-title":"Computer Vision \u2013 ECCV 2016","author":"Joulin","year":"2016"},{"key":"2024090413342253400_bib15","article-title":"Adam: A method for stochastic optimization","author":"Kingma","year":"2014","journal-title":"arXiv preprint arXiv:1412.6980"},{"key":"2024090413342253400_bib16","doi-asserted-by":"publisher","first-page":"2793","DOI":"10.18653\/v1\/2020.acl-main.249","article-title":"Weight poisoning attacks on pretrained models","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Kurita","year":"2020"},{"key":"2024090413342253400_bib17","article-title":"Defending against insertion-based textual backdoor attacks via attribution","author":"Li","year":"2023","journal-title":"arXiv preprint arXiv:2305.02394"},{"key":"2024090413342253400_bib18","doi-asserted-by":"publisher","first-page":"3023","DOI":"10.18653\/v1\/2021.emnlp-main.241","article-title":"Backdoor attacks on pre-trained models by layerwise weight poisoning","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Li","year":"2021"},{"key":"2024090413342253400_bib19","article-title":"Anti-backdoor learning: Training clean models on poisoned data","volume-title":"Advances in Neural Information Processing Systems","author":"Li","year":"2021"},{"key":"2024090413342253400_bib20","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2018.23291","article-title":"Trojaning attack on neural networks","volume-title":"25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February 18-221, 2018","author":"Liu","year":"2018"},{"key":"2024090413342253400_bib21","doi-asserted-by":"crossref","first-page":"9558","DOI":"10.18653\/v1\/2021.emnlp-main.752","article-title":"ONION: A simple and effective defense against textual backdoor attacks","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Qi","year":"2021"},{"key":"2024090413342253400_bib22","first-page":"443","article-title":"Hidden killer: Invisible textual backdoor attacks with syntactic trigger","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Qi","year":"2021"},{"key":"2024090413342253400_bib23","doi-asserted-by":"publisher","first-page":"4873","DOI":"10.18653\/v1\/2021.acl-long.377","article-title":"Turn the combination lock: Learnable textual backdoor attacks via word substitution","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Qi","year":"2021"},{"key":"2024090413342253400_bib24","doi-asserted-by":"publisher","first-page":"13327","DOI":"10.1109\/CVPR52688.2022.01298","article-title":"Backdoor attacks on self-supervised learning","volume-title":"2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Saha","year":"2022"},{"key":"2024090413342253400_bib25","first-page":"1631","article-title":"Recursive deep models for semantic compositionality over a sentiment treebank","volume-title":"Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing","author":"Socher","year":"2013"},{"key":"2024090413342253400_bib26","doi-asserted-by":"publisher","first-page":"9275","DOI":"10.18653\/v1\/2020.emnlp-main.746","article-title":"Dataset cartography: Mapping and diagnosing datasets with training dynamics","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Swayamdipta","year":"2020"},{"key":"2024090413342253400_bib27","first-page":"479","article-title":"OPUS-MT \u2013 building open translation services for the world","volume-title":"Proceedings of the 22nd Annual Conference of the European Association for Machine Translation","author":"Tiedemann","year":"2020"},{"key":"2024090413342253400_bib28","article-title":"Llama 2: Open foundation and fine-tuned chat models","author":"Touvron","year":"2023","journal-title":"arXiv preprint arXiv:2307.09288"},{"key":"2024090413342253400_bib29","article-title":"Spectral signatures in backdoor attacks","volume-title":"Advances in Neural Information Processing Systems","author":"Tran","year":"2018"},{"key":"2024090413342253400_bib30","doi-asserted-by":"publisher","first-page":"353","DOI":"10.18653\/v1\/W18-5446","article-title":"GLUE: A multi-task benchmark and analysis platform for natural language understanding","volume-title":"Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP","author":"Wang","year":"2018"},{"key":"2024090413342253400_bib31","doi-asserted-by":"publisher","first-page":"38","DOI":"10.18653\/v1\/2020.emnlp-demos.6","article-title":"Transformers: State-of-the-art natural language processing","volume-title":"Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations","author":"Wolf","year":"2020"},{"key":"2024090413342253400_bib32","doi-asserted-by":"publisher","first-page":"8365","DOI":"10.18653\/v1\/2021.emnlp-main.659","article-title":"RAP: Robustness-Aware Perturbations for defending against backdoor attacks on NLP models","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Yang","year":"2021"},{"key":"2024090413342253400_bib33","doi-asserted-by":"publisher","first-page":"2041","DOI":"10.1145\/3319535.3354209","article-title":"Latent backdoor attacks on deep neural networks","volume-title":"Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security","author":"Yao","year":"2019"},{"key":"2024090413342253400_bib34","doi-asserted-by":"publisher","first-page":"1415","DOI":"10.18653\/v1\/N19-1144","article-title":"Predicting the type and target of offensive posts in social media","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Zampieri","year":"2019"},{"key":"2024090413342253400_bib35","article-title":"Character-level convolutional networks for text classification","volume":"28","author":"Zhang","year":"2015","journal-title":"Advances in Neural Information Processing Systems"}],"container-title":["Transactions of the Association for Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00684\/2468681\/tacl_a_00684.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/tacl\/article-pdf\/doi\/10.1162\/tacl_a_00684\/2468681\/tacl_a_00684.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T13:37:03Z","timestamp":1725457023000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/tacl\/article\/doi\/10.1162\/tacl_a_00684\/124258\/SEEP-Training-Dynamics-Grounds-Latent"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":35,"URL":"https:\/\/doi.org\/10.1162\/tacl_a_00684","relation":{},"ISSN":["2307-387X"],"issn-type":[{"value":"2307-387X","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}