{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,9,30]],"date-time":"2025-09-30T10:42:03Z","timestamp":1759228923392,"version":"3.41.0"},"reference-count":55,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2024,10,31]],"date-time":"2024-10-31T00:00:00Z","timestamp":1730332800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000006","name":"Office of Naval Research","doi-asserted-by":"crossref","award":["N00014-18-1-2670 and N00014-23-1-2132"],"award-info":[{"award-number":["N00014-18-1-2670 and N00014-23-1-2132"]}],"id":[{"id":"10.13039\/100000006","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100000001","name":"U.S. National Science Foundation","doi-asserted-by":"crossref","award":["CNS-1822094"],"award-info":[{"award-number":["CNS-1822094"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Intell. Syst. Technol."],"published-print":{"date-parts":[[2024,10,31]]},"abstract":"<jats:p>The coronavirus pandemic has fostered an explosion of misinformation about the disease, including the risk and effectiveness of vaccination. AI tools for automatic Scientific Claim Verification (SCV) can be crucial to defeat misinformation campaigns spreading through social media channels. However, over the past years, many concerns have been raised about the robustness of AI to adversarial attacks, and the field of automatic SCV is not exempt. The risk is that such SCV tools may reinforce and legitimize the spread of fake scientific claims rather than refute them. This article investigates the problem of generating adversarial attacks for SCV tools and shows that it is far more difficult than the generic NLP adversarial attack problem. The current NLP adversarial attack generators, when applied to SCV, often generate modified claims with entirely different meaning from the original. Even when the meaning is preserved, the modification of the generated claim is too simplistic (only a single word is changed), leaving many weaknesses of the SCV tools undiscovered. We propose T5-ParEvo, an iterative evolutionary attack generator, that is able to generate more complex and creative attacks while better preserving the semantics of the original claim. Using detailed quantitative and qualitative analyses, we demonstrate the efficacy of T5-ParEvo in comparison with existing attack generators.<\/jats:p>","DOI":"10.1145\/3663481","type":"journal-article","created":{"date-parts":[[2024,5,2]],"date-time":"2024-05-02T18:42:18Z","timestamp":1714675338000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Analyzing Robustness of Automatic Scientific Claim Verification Tools against Adversarial Rephrasing Attacks"],"prefix":"10.1145","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9533-5599","authenticated-orcid":false,"given":"Janet","family":"Layne","sequence":"first","affiliation":[{"name":"Boise State University, Boise, ID, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6738-6302","authenticated-orcid":false,"given":"Qudrat E. Alahy","family":"Ratul","sequence":"additional","affiliation":[{"name":"Boise State University, Boise, ID, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0689-5063","authenticated-orcid":false,"given":"Edoardo","family":"Serra","sequence":"additional","affiliation":[{"name":"Boise State University, Boise, ID, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3210-558X","authenticated-orcid":false,"given":"Sushil","family":"Jajodia","sequence":"additional","affiliation":[{"name":"George Mason University, Fairfax, VA, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,11,6]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0306-4573(02)00021-3"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1316"},{"key":"e_1_3_3_4_2","unstructured":"I. Beltagy M. E. Peters and A. Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150. Retrieved from https:\/\/arxiv.org\/abs\/2004.05150"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D15-1075"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0076540"},{"key":"e_1_3_3_7_2","doi-asserted-by":"crossref","unstructured":"J. DeYoung E. Lehman B. Nye I. J. Marshall and B. C. Wallace. 2020. Evidence inference 2.0: More data better models. arXiv:2005.04177. Retrieved from https:\/\/arxiv.org\/abs\/2005.04177","DOI":"10.18653\/v1\/2020.bionlp-1.13"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","unstructured":"J. Ebrahimi A. Rao D. Lowd and D. Dou. 2017. HotFlip: White-box adversarial examples for text classification. arXiv:1712.06751. Retrieved from 10.48550\/arXiv.1712.06751","DOI":"10.48550\/arXiv.1712.06751"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1037\/h0057532"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/SPW.2018.00016"},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.498"},{"key":"e_1_3_3_12_2","doi-asserted-by":"crossref","unstructured":"M. Glockner V. Shwartz and Y. Goldberg. 2018. Breaking NLI systems with sentences that require simple lexical inferences. arXiv:1805.02266. Retrieved from http:\/\/arxiv.org\/abs\/1805.02266","DOI":"10.18653\/v1\/P18-2103"},{"key":"e_1_3_3_13_2","volume-title":"Technique of Clear Writing","author":"Gunning R.","year":"1952","unstructured":"R. Gunning. 1952. Technique of Clear Writing. McGraw-Hill."},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1170"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","unstructured":"E. Jang S. Gu and B. Poole. 2016. Categorical reparameterization with gumbel-softmax. arXiv:1611.01144. Retrieved from 10.48550\/arXiv.1611.01144","DOI":"10.48550\/arXiv.1611.01144"},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D17-1215"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6311"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1259"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v32i1.12022"},{"key":"e_1_3_3_20_2","unstructured":"J. P. Kincaid R. P. Fishburne R. L. Rogers and B. S. Chissom. 1975. Research Branch Report 8\u201375. Memphis: Naval Air Station."},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2022.102695"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","unstructured":"Z. Lan M. Chen S. Goodman K. Gimpel P. Sharma and R. Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv:1909.11942. Retrieved from 10.48550\/arXiv.1909.11942","DOI":"10.48550\/arXiv.1909.11942"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1093\/bioinformatics\/btz682"},{"key":"e_1_3_3_24_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1371"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","unstructured":"J. Li S. Ji T. Du B. Li and T. Wang. 2018. TextBugger: Generating adversarial text against real-world applications. arXiv:1812.05271. Retrieved from 10.48550\/arXiv.1812.05271","DOI":"10.48550\/arXiv.1812.05271"},{"key":"e_1_3_3_26_2","doi-asserted-by":"crossref","unstructured":"L. Li R. Ma Q. Guo X. Xue and X. Qiu. 2020. BERT-ATTACK: Adversarial attack against BERT using BERT. arXiv:2004.09984. Retrieved from https:\/\/arxiv.org\/abs\/2004.09984","DOI":"10.18653\/v1\/2020.emnlp-main.500"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","unstructured":"X. Li G. A. Burns and N. Peng. 2021. A paragraph-level multi-task learning model for scientific fact-verification. arXiv:2012.14500. Retrieved from 10.48550\/arXiv.2012.14500","DOI":"10.48550\/arXiv.2012.14500"},{"key":"e_1_3_3_28_2","first-page":"4208","volume-title":"Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI \u201918)","author":"Liang B.","year":"2017","unstructured":"B. Liang, H. Li, M. Su, P. Bian, X. Li, and W. Shi. 2017. Deep text classification can be fooled. In Proceedings of the 27th International Joint Conference on Artificial Intelligence (IJCAI \u201918). AAAI Press, 4208\u20134215."},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.522"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v37i11.26553"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.119110"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","unstructured":"Y. Liu M. Ott N. Goyal J. Du M. Joshi D. Chen O. Levy M. Lewis L. Zettlemoyer and V. Stoyanov. 2019. RoBERTa: A robustly optimized bert pretraining approach. arXiv:1907.11692. Retrieved from 10.48550\/arXiv.1907.11692","DOI":"10.48550\/arXiv.1907.11692"},{"key":"e_1_3_3_33_2","first-page":"142","volume-title":"Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Mrk\u0161i\u0107 N.","year":"2016","unstructured":"N. Mrk\u0161i\u0107, D. \u00d3 S\u00e9aghdha, B. Thomson, M. Ga\u0161i\u0107, L. M. Rojas-Barahona, P. H. Su, D. Vandyke, T. H. Wen, S. Young. 2016. Counter-fitting word vectors to linguistic constraints. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Stroudsburg, PA, 142\u2013148."},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1075\/li.30.1.03nad"},{"key":"e_1_3_3_35_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Nie W.","year":"2018","unstructured":"W. Nie, N. Narodytska, and A. Patel. 2018. ReLGAN: Relational generative adversarial networks for text generation. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_3_36_2","unstructured":"OpenAI. 2023. ChatGPT. Retrieved from https:\/\/chat.openai.com\/"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1002\/9781119136378"},{"key":"e_1_3_3_38_2","unstructured":"R. Pradeep X. Ma R. Nogueira and J. Lin. 2020. Scientific claim verification with VerT5erini. arXiv:2010.11930."},{"issue":"8","key":"e_1_3_3_39_2","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford A.","year":"2019","unstructured":"A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"issue":"1","key":"e_1_3_3_40_2","first-page":"67","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel C.","year":"2019","unstructured":"C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu. 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning and Research 21, 1, Article 140 (Jan. 2020), 67.","journal-title":"Journal of Machine Learning and Research"},{"key":"e_1_3_3_41_2","first-page":"1","article-title":"Named entity recognition on bio-medical literature documents using hybrid based approach","author":"Ramachandran R.","year":"2021","unstructured":"R. Ramachandran and K. Arutchelvan. 2021. Named entity recognition on bio-medical literature documents using hybrid based approach. Journal of Ambient Intelligence and Humanized Computing (2021), 1\u201310.","journal-title":"Journal of Ambient Intelligence and Humanized Computing"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1103"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.442"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP43922.2022.9747475"},{"key":"e_1_3_3_45_2","first-page":"16857","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","volume":"33","author":"Song K.","year":"2020","unstructured":"K. Song, X. Tan, T. Qin, J. Lu, and T. Y. Liu. 2020. Mpnet: Masked and permuted pre-training for language understanding. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 33, 16857\u201316867."},{"key":"e_1_3_3_46_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-1074"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","unstructured":"Petter T\u00f6rnberg. 2023. ChatGPT-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning. arXiv:2304.06588. Retrieved from 10.48550\/arXiv.2304.06588","DOI":"10.48550\/arXiv.2304.06588"},{"key":"e_1_3_3_48_2","volume-title":"Proceedings of the Advances in Neural Information Processing Systems,","volume":"30","author":"Vaswani A.","year":"2017","unstructured":"A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, \u0141. Kaiser, and I. Polosukhin. 2017. Attention is all you need. Proceedings of the Advances in Neural Information Processing Systems, Vol. 30."},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.609"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.609"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.findings-naacl.6"},{"key":"e_1_3_3_52_2","unstructured":"D. Wadden K. Lo L. Lu Wang A. Cohan I. Beltagy and H. Hajishirzi. 2021b. LongChecker: Improving scientific claim verification by modeling full-abstract context. arXiv:2112.01640. Retrieved from https:\/\/arxiv.org\/abs\/2112.01640"},{"key":"e_1_3_3_53_2","doi-asserted-by":"crossref","unstructured":"A. Williams N. Nangia and S. R. Bowman. 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv:1704.05426.","DOI":"10.18653\/v1\/N18-1101"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.328"},{"key":"e_1_3_3_55_2","volume-title":"Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL)","author":"Zhang Y.","year":"2019","unstructured":"Y. Zhang, J. Baldridge, and L. He. 2019. PAWS: Paraphrase adversaries from word scrambling. In Proceedings of the North American Chapter of the Association for Computational Linguistics (NAACL), 1298\u20131308."},{"key":"e_1_3_3_56_2","doi-asserted-by":"crossref","unstructured":"Z. Zhang J. Li F. Fukumoto and Y. Ye. 2021. Abstract rationale stance: A joint model for scientific claim verification. arXiv:2110.15116.","DOI":"10.18653\/v1\/2021.emnlp-main.290"}],"container-title":["ACM Transactions on Intelligent Systems and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3663481","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3663481","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T23:56:46Z","timestamp":1750291006000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3663481"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,31]]},"references-count":55,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,10,31]]}},"alternative-id":["10.1145\/3663481"],"URL":"https:\/\/doi.org\/10.1145\/3663481","relation":{},"ISSN":["2157-6904","2157-6912"],"issn-type":[{"type":"print","value":"2157-6904"},{"type":"electronic","value":"2157-6912"}],"subject":[],"published":{"date-parts":[[2024,10,31]]},"assertion":[{"value":"2022-12-02","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-01","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-06","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}