{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T04:49:05Z","timestamp":1778302145794,"version":"3.51.4"},"reference-count":70,"publisher":"MIT Press","issue":"2","license":[{"start":{"date-parts":[[2023,2,2]],"date-time":"2023-02-02T00:00:00Z","timestamp":1675296000000},"content-version":"vor","delay-in-days":32,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc-nd\/4.0\/"}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,6,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n               <jats:p>Very recently, few certified defense methods have been developed to provably guarantee the robustness of a text classifier to adversarial synonym substitutions. However, all the existing certified defense methods assume that the defenders have been informed of how the adversaries generate synonyms, which is not a realistic scenario. In this study, we propose a certifiably robust defense method by randomly masking a certain proportion of the words in an input text, in which the above unrealistic assumption is no longer necessary. The proposed method can defend against not only word substitution-based attacks, but also character-level perturbations. We can certify the classifications of over 50% of texts to be robust to any perturbation of five words on AGNEWS, and two words on SST2 dataset. The experimental results show that our randomized smoothing method significantly outperforms recently proposed defense methods across multiple datasets under different attack algorithms.<\/jats:p>","DOI":"10.1162\/coli_a_00476","type":"journal-article","created":{"date-parts":[[2023,2,2]],"date-time":"2023-02-02T20:51:52Z","timestamp":1675371112000},"page":"395-427","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":43,"title":["Certified Robustness to Text Adversarial Attacks by Randomized [MASK]"],"prefix":"10.1162","volume":"49","author":[{"given":"Jiehang","family":"Zeng","sequence":"first","affiliation":[{"name":"Fudan University, School of Computer Science. jhzeng18@fudan.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jianhan","family":"Xu","sequence":"additional","affiliation":[{"name":"Fudan University, School of Computer Science. jianhanxu20@fudan.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaoqing","family":"Zheng","sequence":"additional","affiliation":[{"name":"Fudan University, School of Computer Science. zhengxq@fudan.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xuanjing","family":"Huang","sequence":"additional","affiliation":[{"name":"Fudan University, School of Computer Science. xjhuang@fudan.edu.cn"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"281","published-online":{"date-parts":[[2023,6,1]]},"reference":[{"key":"2023061215395491000_bib1","doi-asserted-by":"publisher","first-page":"2890","DOI":"10.18653\/v1\/D18-1316","article-title":"Generating natural language adversarial examples","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Alzantot","year":"2018"},{"key":"2023061215395491000_bib2","article-title":"Interpretable adversarial training for text","author":"Barham","year":"2019","journal-title":"arXiv preprint arXiv:1905.12864"},{"key":"2023061215395491000_bib3","doi-asserted-by":"publisher","first-page":"632","DOI":"10.18653\/v1\/D15-1075","article-title":"A large annotated corpus for learning natural language inference","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Bowman","year":"2015"},{"key":"2023061215395491000_bib4","article-title":"Voting based ensemble improves robustness of defensive models","author":"Cheng","year":"2020","journal-title":"arXiv preprint arXiv:2011.14031"},{"key":"2023061215395491000_bib5","doi-asserted-by":"publisher","first-page":"3325","DOI":"10.18653\/v1\/N19-1336","article-title":"Evaluating and enhancing the robustness of dialogue systems: A case study on a negotiation agent","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL)","author":"Cheng","year":"2019"},{"key":"2023061215395491000_bib6","doi-asserted-by":"publisher","first-page":"3601","DOI":"10.1609\/aaai.v34i04.5767","article-title":"Seq2Sick: Evaluating the robustness of sequence-to-sequence models with adversarial examples","volume-title":"Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI)","author":"Cheng","year":"2020"},{"issue":"4","key":"2023061215395491000_bib7","doi-asserted-by":"publisher","first-page":"404","DOI":"10.1093\/biomet\/26.4.404","article-title":"The use of confidence or fiducial limits illustrated in the case of the binomial","volume":"26","author":"Clopper","year":"1934","journal-title":"Biometrika"},{"key":"2023061215395491000_bib8","first-page":"1310","article-title":"Certified adversarial robustness via randomized smoothing","volume-title":"Proceedings of the International Conference on Machine Learning (ICML)","author":"Cohen","year":"2019"},{"issue":"6","key":"2023061215395491000_bib9","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1109\/MSP.2012.2211477","article-title":"The MNIST database of handwritten digit images for machine learning research [best of the web]","volume":"29","author":"Deng","year":"2012","journal-title":"IEEE Signal Processing Magazine"},{"key":"2023061215395491000_bib10","first-page":"4171","article-title":"BERT: Pre-training of deep bidirectional transformers for language understanding","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin","year":"2019"},{"key":"2023061215395491000_bib11","article-title":"Towards robustness against natural language word substitutions","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Dong","year":"2021"},{"key":"2023061215395491000_bib12","article-title":"Training verified learners with learned verifiers","volume":"1805.10265","author":"Dvijotham","year":"2018","journal-title":"arXiv preprint arXiv:"},{"key":"2023061215395491000_bib13","doi-asserted-by":"publisher","first-page":"31","DOI":"10.18653\/v1\/P18-2006","article-title":"HotFlip: White-box adversarial examples for text classification","volume-title":"Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Ebrahimi","year":"2018"},{"key":"2023061215395491000_bib14","doi-asserted-by":"publisher","first-page":"1634","DOI":"10.18653\/v1\/N19-1165","article-title":"Text processing like humans do: Visually attacking and shielding NLP systems","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL)","author":"Eger","year":"2019"},{"key":"2023061215395491000_bib15","doi-asserted-by":"publisher","first-page":"1625","DOI":"10.1109\/CVPR.2018.00175","article-title":"Robust physical-world attacks on deep learning visual classification","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Eykholt","year":"2018"},{"key":"2023061215395491000_bib16","doi-asserted-by":"publisher","first-page":"50","DOI":"10.1109\/SPW.2018.00016","article-title":"Black-box generation of adversarial text sequences to evade deep learning classifiers","volume-title":"Proceedings of the IEEE Security and Privacy Workshops (SPW)","author":"Gao","year":"2018"},{"key":"2023061215395491000_bib17","article-title":"Explaining and harnessing adversarial examples","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Goodfellow","year":"2015"},{"key":"2023061215395491000_bib18","article-title":"Explaining and harnessing adversarial examples","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Goodfellow","year":"2015"},{"key":"2023061215395491000_bib19","article-title":"On the effectiveness of interval bound propagation for training verifiably robust models","author":"Gowal","year":"2018","journal-title":"arXiv preprint arXiv: 1810.12715"},{"key":"2023061215395491000_bib20","doi-asserted-by":"publisher","first-page":"2327","DOI":"10.18653\/v1\/2020.emnlp-main.182","article-title":"Adversarial attack and defense of structured prediction models","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Han","year":"2020"},{"key":"2023061215395491000_bib21","doi-asserted-by":"publisher","first-page":"1520","DOI":"10.18653\/v1\/P19-1147","article-title":"On the robustness of self-attentive models","volume-title":"Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Hsieh","year":"2019"},{"key":"2023061215395491000_bib22","doi-asserted-by":"publisher","first-page":"4083","DOI":"10.18653\/v1\/D19-1419","article-title":"Achieving verified robustness to symbol substitutions via interval bound propagation","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Huang","year":"2019"},{"key":"2023061215395491000_bib23","first-page":"125","article-title":"Adversarial examples are not bugs, they are features","volume-title":"Proceedings of Advances in Neural Information Processing Systems (NuerIPS)","author":"Ilyas","year":"2019"},{"key":"2023061215395491000_bib24","doi-asserted-by":"publisher","first-page":"2021","DOI":"10.18653\/v1\/D17-1215","article-title":"Adversarial examples for evaluating reading comprehension systems","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Jia","year":"2017"},{"key":"2023061215395491000_bib25","doi-asserted-by":"publisher","first-page":"4129","DOI":"10.18653\/v1\/D19-1423","article-title":"Certified robustness to adversarial word substitutions","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)","author":"Jia","year":"2019"},{"issue":"05","key":"2023061215395491000_bib26","doi-asserted-by":"publisher","first-page":"8018","DOI":"10.1609\/aaai.v34i05.6311","article-title":"Is BERT really robust? A strong baseline for natural language attack on text classification and entailment","volume":"34","author":"Jin","year":"2020","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence (AAAI)"},{"key":"2023061215395491000_bib27","article-title":"Adversarial examples for natural language classification problems","author":"Kuleshov","year":"2018"},{"key":"2023061215395491000_bib28","doi-asserted-by":"publisher","first-page":"656","DOI":"10.1109\/SP.2019.00044","article-title":"Certified robustness to adversarial examples with differential privacy","volume-title":"Proceedings of the IEEE Symposium on Security and Privacy (SP)","author":"L\u00e9cuyer","year":"2019"},{"key":"2023061215395491000_bib29","article-title":"Discrete adversarial attacks and submodular optimization with applications to text classification","volume-title":"Proceedings of Machine Learning and Systems (MLSys)","author":"Lei","year":"2019"},{"key":"2023061215395491000_bib30","doi-asserted-by":"publisher","first-page":"4585","DOI":"10.1609\/aaai.v34i04.5888","article-title":"Robustness certificates for sparse adversarial attacks by randomized ablation","volume-title":"Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI)","author":"Levine","year":"2020"},{"key":"2023061215395491000_bib31","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2019.23138","article-title":"TextBugger: Generating adversarial text against real-world applications","volume-title":"Proceedings of the Annual Network and Distributed System Security Symposium (NDSS)","author":"Li","year":"2019"},{"key":"2023061215395491000_bib32","doi-asserted-by":"publisher","first-page":"6193","DOI":"10.18653\/v1\/2020.emnlp-main.500","article-title":"BERT-ATTACK: Adversarial attack against BERT using BERT","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Li","year":"2020"},{"key":"2023061215395491000_bib33","doi-asserted-by":"publisher","first-page":"3137","DOI":"10.18653\/v1\/2021.emnlp-main.251","article-title":"Searching for an effective defender: Benchmarking defense against adversarial word substitution","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Li","year":"2021"},{"key":"2023061215395491000_bib34","doi-asserted-by":"publisher","first-page":"4208","DOI":"10.24963\/ijcai.2018\/585","article-title":"Deep text classification can be fooled","volume-title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI)","author":"Liang","year":"2018"},{"key":"2023061215395491000_bib35","article-title":"RoBERTa: A robustly optimized BERT pretraining approach","author":"Liu","year":"2019","journal-title":"arXiv preprint arXiv:1907.11692"},{"key":"2023061215395491000_bib36","article-title":"SGDR: Stochastic gradient descent with warm restarts","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Loshchilov","year":"2017"},{"key":"2023061215395491000_bib37","article-title":"Decoupled weight decay regularization","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Loshchilov","year":"2019"},{"key":"2023061215395491000_bib38","first-page":"142","article-title":"Learning word vectors for sentiment analysis","volume-title":"Proceedings of the Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (NAACL)","author":"Maas","year":"2011"},{"key":"2023061215395491000_bib39","article-title":"Towards deep learning models resistant to adversarial attacks","volume-title":"Proceedings of International Conference on Learning Representations (ICLR)","author":"Madry","year":"2018"},{"key":"2023061215395491000_bib40","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i15.17595","article-title":"Generating natural language attacks in a hard label black box setting","author":"Maheshwary","year":"2020","journal-title":"arXiv preprint arXiv:2012.14956"},{"key":"2023061215395491000_bib41","doi-asserted-by":"publisher","first-page":"656","DOI":"10.1109\/SP.2019.00044","article-title":"Certified robustness to adversarial examples with differential privacy","volume-title":"Proceedings of 2019 IEEE Symposium on Security and Privacy (SP)","author":"Mathias","year":"2019"},{"key":"2023061215395491000_bib42","article-title":"Adversarial training methods for semi-supervised text classification","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Miyato","year":"2017"},{"key":"2023061215395491000_bib43","doi-asserted-by":"publisher","first-page":"2574","DOI":"10.1109\/CVPR.2016.282","article-title":"DeepFool: A simple and accurate method to fool deep neural networks","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Moosavi-Dezfooli","year":"2016"},{"key":"2023061215395491000_bib44","doi-asserted-by":"publisher","first-page":"119","DOI":"10.18653\/v1\/2020.emnlp-demos.16","article-title":"TextAttack: A framework for adversarial attacks, data augmentation and adversarial training in NLP","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP)","author":"Morris","year":"2020"},{"key":"2023061215395491000_bib45","doi-asserted-by":"publisher","first-page":"1532","DOI":"10.3115\/v1\/D14-1162","article-title":"GloVe: Global vectors for word representation","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Pennington","year":"2014"},{"key":"2023061215395491000_bib46","doi-asserted-by":"publisher","first-page":"1085","DOI":"10.18653\/v1\/P19-1103","article-title":"Generating natural language adversarial examples through probability weighted word saliency","volume-title":"Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Ren","year":"2019"},{"key":"2023061215395491000_bib47","doi-asserted-by":"publisher","first-page":"1085","DOI":"10.18653\/v1\/P19-1103","article-title":"Generating natural language adversarial examples through probability weighted word saliency","volume-title":"Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Ren","year":"2019"},{"key":"2023061215395491000_bib48","doi-asserted-by":"publisher","first-page":"856","DOI":"10.18653\/v1\/P18-1079","article-title":"Semantically equivalent adversarial rules for debugging NLP models","volume-title":"Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Ribeiro","year":"2018"},{"key":"2023061215395491000_bib49","article-title":"Towards crafting text adversarial samples","author":"Samanta","year":"2017","journal-title":"arXiv preprint arXiv: 1707.02812"},{"key":"2023061215395491000_bib50","doi-asserted-by":"publisher","first-page":"4323","DOI":"10.24963\/ijcai.2018\/601","article-title":"Interpretable adversarial perturbation in input embedding space for text","volume-title":"Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence (IJCAI)","author":"Sato","year":"2018"},{"key":"2023061215395491000_bib51","article-title":"Robustness verification for transformers","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Shi","year":"2020"},{"key":"2023061215395491000_bib52","first-page":"1631","article-title":"Recursive deep models for semantic compositionality over a sentiment treebank","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP)","author":"Socher","year":"2013"},{"key":"2023061215395491000_bib53","article-title":"Intriguing properties of neural networks","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Szegedy","year":"2014"},{"key":"2023061215395491000_bib54","first-page":"5998","article-title":"Attention is all you need","volume-title":"Proceedings of Advances in Neural Information Processing Systems (NeurIPS)","author":"Vaswani","year":"2017"},{"key":"2023061215395491000_bib55","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1221","article-title":"Universal adversarial triggers for NLP","author":"Wallace","year":"2019","journal-title":"arXiv preprint arXiv: 1908.07125"},{"key":"2023061215395491000_bib56","article-title":"A survey: Towards a robust deep neural network in text domain","author":"Wang","year":"2019","journal-title":"arXiv preprint arXiv: 1902.07285"},{"key":"2023061215395491000_bib57","article-title":"DANCin SEQ2SEQ: Fooling text classifiers with adversarial text example generation","author":"Wong","year":"2017","journal-title":"arXiv preprint arXiv: 1712.05419"},{"key":"2023061215395491000_bib58","article-title":"Adversarial attacks and defenses in images, graphs and text: A review","author":"Xu","year":"2019","journal-title":"arXiv preprint arXiv: 1909.08072"},{"key":"2023061215395491000_bib59","article-title":"Automatic perturbation analysis for scalable certified robustness and beyond","volume-title":"Proceedings of the Advances in Neural Information Processing Systems (NeurIPS)","author":"Xu","year":"2020"},{"key":"2023061215395491000_bib60","article-title":"Greedy attack and Gumbel attack: Generating adversarial examples for discrete data","author":"Yang","year":"2018","journal-title":"arXiv preprint arXiv: 1805.12316"},{"key":"2023061215395491000_bib61","doi-asserted-by":"publisher","first-page":"3465","DOI":"10.18653\/v1\/2020.acl-main.317","article-title":"SAFER: A structure-free approach for certified robustness to adversarial word substitutions","volume-title":"Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Ye","year":"2020"},{"key":"2023061215395491000_bib62","doi-asserted-by":"publisher","first-page":"1612","DOI":"10.18653\/v1\/2021.emnlp-main.121","article-title":"On the transferability of adversarial attacks against neural text classifier","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Yuan","year":"2021"},{"key":"2023061215395491000_bib63","doi-asserted-by":"publisher","first-page":"6066","DOI":"10.18653\/v1\/2020.acl-main.540","article-title":"Word-level textual adversarial attacking as combinatorial optimization","volume-title":"Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Zang","year":"2020"},{"key":"2023061215395491000_bib64","article-title":"Word embedding perturbation for sentence classification","author":"Zhang","year":"2018","journal-title":"arXiv preprint arXiv: 1804.08166"},{"key":"2023061215395491000_bib65","first-page":"649","article-title":"Character-level convolutional networks for text classification","volume-title":"Proceedings of the Advances in Neural Information Processing Systems (NeurIPS)","author":"Zhang","year":"2015"},{"key":"2023061215395491000_bib66","article-title":"Generating natural adversarial examples","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Zhao","year":"2018"},{"key":"2023061215395491000_bib67","doi-asserted-by":"publisher","first-page":"6600","DOI":"10.18653\/v1\/2020.acl-main.590","article-title":"Evaluating and enhancing the robustness of neural network-based dependency parsing models with adversarial examples","volume-title":"Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL)","author":"Zheng","year":"2020"},{"key":"2023061215395491000_bib68","doi-asserted-by":"publisher","first-page":"5482","DOI":"10.18653\/v1\/2021.acl-long.426","article-title":"Defense against synonym substitution-based adversarial attacks via Dirichlet neighborhood ensemble","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Zhou","year":"2021"},{"key":"2023061215395491000_bib69","article-title":"FreeLB: Enhanced adversarial training for natural language understanding","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR)","author":"Zhu","year":"2020"},{"key":"2023061215395491000_bib70","doi-asserted-by":"publisher","first-page":"2516","DOI":"10.1109\/ICIP.2019.8803269","article-title":"Generating adversarial examples by makeup attacks on face recognition","volume-title":"Proceedings of the IEEE International Conference on Image Processing (ICIP)","author":"Zhu","year":"2019"}],"container-title":["Computational Linguistics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/coli\/article-pdf\/49\/2\/395\/2125510\/coli_a_00476.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/coli\/article-pdf\/49\/2\/395\/2125510\/coli_a_00476.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,12]],"date-time":"2023-06-12T15:41:30Z","timestamp":1686584490000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/coli\/article\/49\/2\/395\/114730\/Certified-Robustness-to-Text-Adversarial-Attacks"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023]]},"references-count":70,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2023,6,1]]},"published-print":{"date-parts":[[2023,6,1]]}},"URL":"https:\/\/doi.org\/10.1162\/coli_a_00476","relation":{},"ISSN":["0891-2017","1530-9312"],"issn-type":[{"value":"0891-2017","type":"print"},{"value":"1530-9312","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2023]]},"published":{"date-parts":[[2023]]}}}