{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,16]],"date-time":"2026-07-16T04:59:06Z","timestamp":1784177946205,"version":"3.55.0"},"reference-count":94,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2023,4,8]],"date-time":"2023-04-08T00:00:00Z","timestamp":1680912000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62006218 and 61902381"],"award-info":[{"award-number":["62006218 and 61902381"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100004739","name":"Youth Innovation Promotion Association CAS","doi-asserted-by":"crossref","award":["20144310 and 2021100"],"award-info":[{"award-number":["20144310 and 2021100"]}],"id":[{"id":"10.13039\/501100004739","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Young Elite Scientist Sponsorship Program by CAST","award":["YESS20200121"],"award-info":[{"award-number":["YESS20200121"]}]},{"name":"Lenovo-CAS Joint Lab Youth Scientist Project"},{"name":"Dutch Ministry of Education, Culture and Science through the Netherlands Organisation for Scientific Research"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Inf. Syst."],"published-print":{"date-parts":[[2023,10,31]]},"abstract":"<jats:p>Neural ranking models (NRMs) have shown remarkable success in recent years, especially with pre-trained language models. However, deep neural models are notorious for their vulnerability to adversarial examples. Adversarial attacks may become a new type of web spamming technique given our increased reliance on neural information retrieval models. Therefore, it is important to study potential adversarial attacks to identify vulnerabilities of NRMs before they are deployed.<\/jats:p><jats:p>In this article, we introduce the Word Substitution Ranking Attack (WSRA) task against NRMs, which aims at promoting a target document in rankings by adding adversarial perturbations to its text. We focus on the decision-based black-box attack setting, where the attackers cannot directly get access to the model information, but can only query the target model to obtain the rank positions of the partial retrieved list. This attack setting is realistic in real-world search engines. We propose a novel Pseudo Relevance-based ADversarial ranking Attack method (PRADA) that learns a surrogate model based on Pseudo Relevance Feedback (PRF) to generate gradients for finding the adversarial perturbations.<\/jats:p><jats:p>Experiments on two web search benchmark datasets show that PRADA can outperform existing attack strategies and successfully fool the NRM with small indiscernible perturbations of text.<\/jats:p>","DOI":"10.1145\/3576923","type":"journal-article","created":{"date-parts":[[2022,12,16]],"date-time":"2022-12-16T13:20:32Z","timestamp":1671196832000},"page":"1-27","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":44,"title":["PRADA: Practical Black-box Adversarial Attacks against Neural Ranking Models"],"prefix":"10.1145","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2492-0640","authenticated-orcid":false,"given":"Chen","family":"Wu","sequence":"first","affiliation":[{"name":"Institute of Computing Technology Academy of Sciences; University of Chinese Academy of Sciences, Haidian District, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4294-2541","authenticated-orcid":false,"given":"Ruqing","family":"Zhang","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology Academy of Sciences; University of Chinese Academy of Sciences, Haidian District, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9509-8674","authenticated-orcid":false,"given":"Jiafeng","family":"Guo","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology Academy of Sciences; University of Chinese Academy of Sciences, Haidian District, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1086-0202","authenticated-orcid":false,"given":"Maarten","family":"De Rijke","sequence":"additional","affiliation":[{"name":"University of Amsterdam, Amsterdam, The Netherlands"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4317-2702","authenticated-orcid":false,"given":"Yixing","family":"Fan","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences; University of Chinese Academy of Sciences, Haidian District, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5201-8195","authenticated-orcid":false,"given":"Xueqi","family":"Cheng","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology, Chinese Academy of Sciences; University of Chinese Academy of Sciences, Haidian District, Beijing, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,4,8]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"crossref","first-page":"89","DOI":"10.1109\/SPW.2018.00022","volume-title":"Proceedings of the 2018 IEEE Security and Privacy Workshops.","author":"Aghakhani Hojjat","year":"2018","unstructured":"Hojjat Aghakhani, Aravind Machiry, Shirin Nilizadeh, Christopher Kruegel, and Giovanni Vigna. 2018. Detecting deceptive reviews using generative adversarial networks. In Proceedings of the 2018 IEEE Security and Privacy Workshops. IEEE, 89\u201395."},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","unstructured":"Moustafa Alzantot Yash Sharma Ahmed Elgohary Bo-Jhang Ho Mani Srivastava and Kai-Wei Chang. 2018. Generating natural language adversarial examples. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing . 2890\u20132896.","DOI":"10.18653\/v1\/D18-1316"},{"key":"e_1_3_2_4_2","unstructured":"Anish Athalye and Nicholas Carlini. 2018. On the robustness of the CVPR 2018 white-box adversarial example defenses. arXiv:1804.03286. Retrieved from https:\/\/arxiv.org\/abs\/1804.03286."},{"key":"e_1_3_2_5_2","volume-title":"Proceedings of the AIRWeb","author":"Benczur Andras A.","year":"2005","unstructured":"Andras A. Benczur, Karoly Csalogany, Tamas Sarlos, and Mate Uher. 2005. SpamRank\u2013Fully automatic link spam detection. In Proceedings of the AIRWeb."},{"issue":"23","key":"e_1_3_2_6_2","first-page":"81","article-title":"From ranknet to lambdarank to lambdamart: An overview","volume":"11","author":"Burges Christopher J. C.","year":"2010","unstructured":"Christopher J. C. Burges. 2010. From ranknet to lambdarank to lambdamart: An overview. Learning 11, 23\u2013581 (2010), 81.","journal-title":"Learning"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401196"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1561\/1500000021"},{"key":"e_1_3_2_9_2","unstructured":"Daniel Cer Yinfei Yang Sheng-yi Kong Nan Hua Nicole Limtiaco Rhomni St John Noah Constant Mario Guajardo-C\u00e9spedes Steve Yuan Chris Tar Yun-Hsuan Sung Brian Strope and Ray Kurzweil. 2018. Universal sentence encoder. arXiv:1803.11175. Retrieved from https:\/\/arxiv.org\/abs\/1803.11175."},{"key":"e_1_3_2_10_2","first-page":"1277","volume-title":"Proceedings of the SP","author":"Chen Jianbo","year":"2020","unstructured":"Jianbo Chen, Michael I. Jordan, and Martin J. Wainwright. 2020. Hopskipjumpattack: A query-efficient decision-based attack. In Proceedings of the SP. IEEE, 1277\u20131294."},{"key":"e_1_3_2_11_2","volume-title":"Proceedings of the SIGIR","author":"Chen Mingyang","year":"2021","unstructured":"Mingyang Chen, Junda Lu, Yi Wang, Jianbin Qin, and Wei Wang. 2021. DAIR: A query-efficient decision-based attack on image retrieval Systems. In Proceedings of the SIGIR."},{"key":"e_1_3_2_12_2","doi-asserted-by":"crossref","unstructured":"Nick Craswell Bhaskar Mitra Emine Yilmaz Daniel Campos and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track. arXiv:2003.07820. Retrieved from https:\/\/arxiv.org\/abs\/2003.07820.","DOI":"10.6028\/NIST.SP.1266.deep-overview"},{"key":"e_1_3_2_13_2","first-page":"985","volume-title":"Proceedings of the SIGIR","author":"Dai Zhuyun","year":"2019","unstructured":"Zhuyun Dai and Jamie Callan. 2019. Deeper text understanding for IR with contextual neural language modeling. In Proceedings of the SIGIR. 985\u2013988."},{"key":"e_1_3_2_14_2","first-page":"126","volume-title":"Proceedings of the WSDM","author":"Dai Zhuyun","year":"2018","unstructured":"Zhuyun Dai, Chenyan Xiong, Jamie Callan, and Zhiyuan Liu. 2018. Convolutional neural networks for soft-matching n-grams in ad-hoc search. In Proceedings of the WSDM. 126\u2013134."},{"key":"e_1_3_2_15_2","first-page":"65","volume-title":"Proceedings of the SIGIR","author":"Dehghani Mostafa","year":"2017","unstructured":"Mostafa Dehghani, Hamed Zamani, Aliaksei Severyn, Jaap Kamps, and W. Bruce Croft. 2017. Neural ranking models with weak supervision. In Proceedings of the SIGIR. 65\u201374."},{"key":"e_1_3_2_16_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401046"},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/3439729"},{"key":"e_1_3_2_18_2","unstructured":"Jacob Devlin Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT (2019) 4171\u20134186."},{"key":"e_1_3_2_19_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Dong Xinshuai","year":"2020","unstructured":"Xinshuai Dong, Anh Tuan Luu, Rongrong Ji, and Hong Liu. 2020. Towards robustness against natural language word substitutions. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1145\/1964114.1964121","volume-title":"Proceedings of the 2011 Joint WICOW\/AIRWeb Workshop on Web Quality","author":"Erd\u00e9lyi Mikl\u00f3s","year":"2011","unstructured":"Mikl\u00f3s Erd\u00e9lyi, Andr\u00e1s Garz\u00f3, and Andr\u00e1s A. Bencz\u00far. 2011. Web spam classification: A few features worth more. In Proceedings of the 2011 Joint WICOW\/AIRWeb Workshop on Web Quality. 27\u201334."},{"issue":"3","key":"e_1_3_2_21_2","article-title":"The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability","volume":"33","author":"Fleiss Joseph L.","year":"1973","unstructured":"Joseph L. Fleiss and Jacob Cohen. 1973. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. EPM 33, 3 (1973), 613\u2013619.","journal-title":"EPM"},{"key":"e_1_3_2_22_2","first-page":"50","volume-title":"Proceedings of the SPW","author":"Gao Ji","year":"2018","unstructured":"Ji Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. Black-box generation of adversarial text sequences to evade deep learning classifiers. In Proceedings of the SPW. IEEE, 50\u201356."},{"key":"e_1_3_2_23_2","unstructured":"Ian J. Goodfellow Jonathon Shlens and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. Stat 1050 (2015) 20."},{"key":"e_1_3_2_24_2","volume-title":"Proceedings of the SIGIR","author":"Goren Gregory","year":"2018","unstructured":"Gregory Goren, Oren Kurland, Moshe Tennenholtz, and Fiana Raiber. 2018. Ranking robustness under adversarial document manipulations. In Proceedings of the SIGIR."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401058"},{"key":"e_1_3_2_26_2","first-page":"2041","volume-title":"Proceedings of the CIKM","author":"Gu Jia-Chen","year":"2020","unstructured":"Jia-Chen Gu, Tianda Li, Quan Liu, Zhen-Hua Ling, Zhiming Su, Si Wei, and Xiaodan Zhu. 2020. Speaker-aware BERT for multi-turn response selection in retrieval-based chatbots. In Proceedings of the CIKM. 2041\u20132044."},{"key":"e_1_3_2_27_2","first-page":"55","volume-title":"Proceedings of the CIKM","author":"Guo Jiafeng","year":"2016","unstructured":"Jiafeng Guo, Yixing Fan, Qingyao Ai, and W. Bruce Croft. 2016. A deep relevance matching model for ad-hoc retrieval. In Proceedings of the CIKM. 55\u201364."},{"key":"e_1_3_2_28_2","volume-title":"Proceedings of the AIRWeb","author":"Gyongyi Zoltan","year":"2005","unstructured":"Zoltan Gyongyi and Hector Garcia-Molina. 2005. Web spam taxonomy. In Proceedings of the AIRWeb."},{"key":"e_1_3_2_29_2","volume-title":"Proceedings of the VLDB","author":"Gyongyi Zoltan","year":"2004","unstructured":"Zoltan Gyongyi, Hector Garcia-Molina, and Jan Pedersen. 2004. Combating web spam with TrustRank. In Proceedings of the VLDB."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3209981"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1145\/2505515.2505665"},{"key":"e_1_3_2_32_2","first-page":"4733","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Huang Qian","year":"2019","unstructured":"Qian Huang, Isay Katsman, Horace He, Zeqi Gu, Serge Belongie, and Ser-Nam Lim. 2019. Enhancing adversarial example transferability with an intermediate level attack. In Proceedings of the IEEE\/CVF International Conference on Computer Vision. 4733\u20134742."},{"key":"e_1_3_2_33_2","first-page":"919","volume-title":"Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval","author":"Izsak Peter","year":"2014","unstructured":"Peter Izsak, Fiana Raiber, Oren Kurland, and Moshe Tennenholtz. 2014. The search duel: A response to a strong ranker. In Proceedings of the 37th International ACM SIGIR Conference on Research & Development in Information Retrieval. 919\u2013922."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10472-018-9612-z"},{"key":"e_1_3_2_35_2","doi-asserted-by":"crossref","unstructured":"Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing . 2021\u20132031.","DOI":"10.18653\/v1\/D17-1215"},{"key":"e_1_3_2_36_2","volume-title":"Proceedings of the EMNLP\/IJCNLP (1)","author":"Jia Robin","year":"2019","unstructured":"Robin Jia, Aditi Raghunathan, Kerem G\u00f6ksel, and Percy Liang. 2019. Certified robustness to adversarial word substitutions. In Proceedings of the EMNLP\/IJCNLP (1)."},{"key":"e_1_3_2_37_2","first-page":"8018","volume-title":"Proceedings of the AAAI","author":"Jin Di","year":"2020","unstructured":"Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. Is BERT really robust? A strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI. 8018\u20138025."},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775067"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00300"},{"key":"e_1_3_2_40_2","volume-title":"Proceedings of the SIGIR","author":"Khattab Omar","year":"2020","unstructured":"Omar Khattab and Matei Zaharia. 2020. Colbert: Efficient and effective passage search via contextualized late interaction over bert. In Proceedings of the SIGIR."},{"key":"e_1_3_2_41_2","article-title":"Generalized badrank with graduated trust","author":"Kolda Tamara G.","year":"2009","unstructured":"Tamara G. Kolda and Michael J. Procopio. 2009. Generalized badrank with graduated trust. Sandia National Laboratories, California (2009).","journal-title":"Sandia National Laboratories, California"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3532771"},{"key":"e_1_3_2_43_2","article-title":"Data poisoning attacks on factorization-based collaborative filtering","volume":"29","author":"Li Bo","year":"2016","unstructured":"Bo Li, Yining Wang, Aarti Singh, and Yevgeniy Vorobeychik. 2016. Data poisoning attacks on factorization-based collaborative filtering. Advances in Neural Information Processing Systems 29, 29 (2016), 1893\u20131901.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.5555\/3019285"},{"key":"e_1_3_2_45_2","first-page":"4899","volume-title":"Proceedings of the ICCV","author":"Li Jie","year":"2019","unstructured":"Jie Li, Rongrong Ji, Hong Liu, Xiaopeng Hong, Yue Gao, and Qi Tian. 2019. Universal perturbation attack against image retrieval. In Proceedings of the ICCV. 4899\u20134908."},{"key":"e_1_3_2_46_2","unstructured":"Jinfeng Li Shouling Ji Tianyu Du Bo Li and Ting Wang. 2019. Textbugger: Generating adversarial text against real-world applications. In Proceedings of the 26th Annual Network and Distributed System Security Symposium ."},{"key":"e_1_3_2_47_2","doi-asserted-by":"crossref","unstructured":"Linyang Li Ruotian Ma Qipeng Guo Xiangyang Xue and Xipeng Qiu. 2020. Bert-attack: Adversarial attack against bert using bert. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP\u201920) . 6193\u20136202.","DOI":"10.18653\/v1\/2020.emnlp-main.500"},{"key":"e_1_3_2_48_2","first-page":"3330","volume-title":"Proceedings of the CVPR","author":"Li Xiaodan","year":"2021","unstructured":"Xiaodan Li, Jinfeng Li, Yuefeng Chen, Shaokai Ye, Yuan He, Shuhui Wang, Hang Su, and Hui Xue. 2021. Qair: Practical query-efficient black-box attacks for image retrieval. In Proceedings of the CVPR. 3330\u20133339."},{"key":"e_1_3_2_49_2","first-page":"3137","volume-title":"Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing","author":"Li Zongyi","year":"2021","unstructured":"Zongyi Li, Jianhan Xu, Jiehang Zeng, Linyang Li, Xiaoqing Zheng, Qi Zhang, Kai-Wei Chang, and Cho-Jui Hsieh. 2021. Searching for an effective defender: Benchmarking defense against adversarial word substitution. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. 3137\u20133147."},{"key":"e_1_3_2_50_2","doi-asserted-by":"crossref","unstructured":"Bin Liang Hongcheng Li Miaoqiang Su Pan Bian Xirong Li and Wenchang Shi. 2018. Deep text classification can be fooled. In Proceedings of the 27th International Joint Conference on Artificial Intelligence . 4208\u20134215.","DOI":"10.24963\/ijcai.2018\/585"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.2200\/S01123ED1V01Y202108HLT053"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-14267-3"},{"key":"e_1_3_2_53_2","doi-asserted-by":"publisher","DOI":"10.1145\/3397271.3401087"},{"key":"e_1_3_2_54_2","first-page":"283","volume-title":"Proceedings of the WSDM","author":"Ma Xinyu","year":"2021","unstructured":"Xinyu Ma, Jiafeng Guo, Ruqing Zhang, Yixing Fan, Xiang Ji, and Xueqi Cheng. 2021. PROP: Pre-training with representative words prediction for ad-hoc retrieval. In Proceedings of the WSDM. 283\u2013291."},{"key":"e_1_3_2_55_2","unstructured":"Xinyu Ma Jiafeng Guo Ruqing Zhang Yixing Fan Yingyan Li and Xueqi Cheng. 2021. B-PROP: Bootstrapped pre-training with representative words prediction for ad-hoc retrieval. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval . 1513\u20131522."},{"key":"e_1_3_2_56_2","unstructured":"Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras and Adrian Vladu. 2019. Towards deep learning models resistant to adversarial attacks. Stat 1050 4 (2019)."},{"key":"e_1_3_2_57_2","first-page":"404","volume-title":"Proceedings of the EMNLP","author":"Mihalcea Rada","year":"2004","unstructured":"Rada Mihalcea and Paul Tarau. 2004. TextRank: Bringing order into text. In Proceedings of the EMNLP. 404\u2013411."},{"key":"e_1_3_2_58_2","doi-asserted-by":"crossref","unstructured":"Pasquale Minervini and Sebastian Riedel. 2018. Adversarially regularising neural NLI models to integrate logical background knowledge. In Proceedings of the 22nd Conference on Computational Natural Language Learning . 65\u201374.","DOI":"10.18653\/v1\/K18-1007"},{"key":"e_1_3_2_59_2","volume-title":"Proceedings of the WWW","author":"Mitra Bhaskar","year":"2017","unstructured":"Bhaskar Mitra, Fernando Diaz, and Nick Craswell. 2017. Learning to match using local and distributed representations of text for web search. In Proceedings of the WWW."},{"key":"e_1_3_2_60_2","doi-asserted-by":"crossref","unstructured":"Nikola Mrk\u0161i\u0107 Diarmuid O. S\u00e9aghdha Blaise Thomson Milica Ga\u0161i\u0107 Lina Rojas-Barahona Pei-Hao Su David Vandyke Tsung-Hsien Wen and Steve Young. 2016. Counter-fitting word vectors to linguistic constraints. In Proceedings of NAACL-HLT . 142\u2013148.","DOI":"10.18653\/v1\/N16-1018"},{"key":"e_1_3_2_61_2","volume-title":"Proceedings of the CoCo@ NIPS","author":"Nguyen Tri","year":"2016","unstructured":"Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated machine reading comprehension dataset. In Proceedings of the CoCo@ NIPS."},{"key":"e_1_3_2_62_2","unstructured":"Rodrigo Nogueira and Kyunghyun Cho. 2019. Passage Re-ranking with BERT. arXiv:1901.04085. Retrieved from https:\/\/arxiv.org\/abs\/1901.04085."},{"key":"e_1_3_2_63_2","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1145\/1135777.1135794","volume-title":"Proceedings of the WWW","author":"Ntoulas Alexandros","year":"2006","unstructured":"Alexandros Ntoulas, Marc Najork, Mark Manasse, and Dennis Fetterly. 2006. Detecting spam web pages through content analysis. In Proceedings of the WWW. 83\u201392."},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-017-9321-y"},{"key":"e_1_3_2_65_2","unstructured":"Nicolas Papernot Patrick McDaniel and Ian Goodfellow. 2016. Transferability in machine learning: From phenomena to black-box attacks using adversarial samples. arXiv:1605.07277. Retrieved from https:\/\/arxiv.org\/abs\/1605.07277."},{"key":"e_1_3_2_66_2","first-page":"506","volume-title":"Proceedings of the ASIA","author":"Papernot Nicolas","year":"2017","unstructured":"Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. 2017. Practical black-box attacks against machine learning. In Proceedings of the ASIA. 506\u2013519."},{"key":"e_1_3_2_67_2","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1162"},{"key":"e_1_3_2_68_2","doi-asserted-by":"crossref","unstructured":"Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies Vol. 1 (Long Papers) Association for Computational Linguistics New Orleans LA 2227\u20132237.","DOI":"10.18653\/v1\/N18-1202"},{"key":"e_1_3_2_69_2","first-page":"25","volume-title":"Proceedings of the AIRWeb","author":"Piskorski Jakub","year":"2008","unstructured":"Jakub Piskorski, Marcin Sydow, and Dawid Weiss. 2008. Exploring linguistic features for web spam detection: A preliminary study. In Proceedings of the AIRWeb. 25\u201328."},{"key":"e_1_3_2_70_2","doi-asserted-by":"crossref","first-page":"275","DOI":"10.1145\/290941.291008","volume-title":"Proceedings of the SIGIR","author":"Ponte Jay M.","year":"1998","unstructured":"Jay M. Ponte and W. Bruce Croft. 1998. A language modeling approach to information retrieval. In Proceedings of the SIGIR. 275\u2013281."},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1561"},{"key":"e_1_3_2_72_2","volume-title":"Proceedings of the LREC","author":"Radev Dragomir R.","year":"2002","unstructured":"Dragomir R. Radev, Hong Qi, Harris Wu, and Weiguo Fan. 2002. Evaluating web-based question answering systems.. In Proceedings of the LREC. Citeseer."},{"key":"e_1_3_2_73_2","unstructured":"Nisarg Raval and Manisha Verma. 2020. One word at a time: Adversarial attacks on retrieval models. arXiv:2008.02197. Retrieved from https:\/\/arxiv.org\/abs\/2008.02197."},{"key":"e_1_3_2_74_2","first-page":"452","volume-title":"Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. 452\u2013461."},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-1-4471-2099-5_24"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1145\/361219.361220"},{"key":"e_1_3_2_77_2","article-title":"An improved framework for content-and link-based web-spam detection: A combined approach","volume":"2021","author":"Shahzad Asim","year":"2021","unstructured":"Asim Shahzad, Nazri Mohd Nawi, Muhammad Zubair Rehman, and Abdullah Khan. 2021. An improved framework for content-and link-based web-spam detection: A combined approach. Complexity 2021 (2021).","journal-title":"Complexity"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"e_1_3_2_79_2","unstructured":"Christian Szegedy Wojciech Zaremba Ilya Sutskever Joan Bruna Dumitru Erhan Ian Goodfellow and Rob Fergus. 2014. Intriguing properties of neural networks. In 2nd International Conference on Learning Representations (ICLR\u201914) ."},{"issue":"1","key":"e_1_3_2_80_2","first-page":"297","article-title":"Ensemble decision for spam detection using term space partition approach","volume":"50","author":"Tan Ying","year":"2018","unstructured":"Ying Tan, Quanbin Wang, and Guyue Mi. 2018. Ensemble decision for spam detection using term space partition approach. IEEE Transactions on Cybernetics 50, 1 (2018), 297\u2013309.","journal-title":"IEEE Transactions on Cybernetics"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2019.2893638"},{"issue":"10","key":"e_1_3_2_82_2","article-title":"Deconstructing Google bombs: A breach of symbolic power or just a goofy prank?","volume":"10","author":"Tatum Clifford","year":"2005","unstructured":"Clifford Tatum. 2005. Deconstructing Google bombs: A breach of symbolic power or just a goofy prank? First Monday 10, 10 (2005).","journal-title":"First Monday"},{"key":"e_1_3_2_83_2","first-page":"395","volume-title":"Proceedings of the SIGIR","author":"Wei Chao","year":"2012","unstructured":"Chao Wei, Yiqun Liu, Min Zhang, Shaoping Ma, Liyun Ru, and Kuo Zhang. 2012. Fighting against web spam: A novel propagation method based on click-through data. In Proceedings of the SIGIR. 395\u2013404."},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1145\/1062745.1062762"},{"key":"e_1_3_2_85_2","unstructured":"Chen Wu Ruqing Zhang Jiafeng Guo Yixing Fan and Xueqi Cheng. 2021. Are neural ranking models robust? arXiv:2108.05018. Retrieved from https:\/\/arxiv.org\/abs\/2108.05018."},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11633-019-1211-x"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.engappai.2020.103641"},{"key":"e_1_3_2_88_2","unstructured":"Ying Xu Xu Zhong Antonio Jimeno Yepes and Jey Han Lau. 2021. Grey-box adversarial attack and defence for sentiment classification. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . 4078\u20134087."},{"issue":"4","key":"e_1_3_2_89_2","article-title":"Adversarial examples for hamming space search","volume":"50","author":"Yang Erkun","year":"2018","unstructured":"Erkun Yang, Tongliang Liu, Cheng Deng, and Dacheng Tao. 2018. Adversarial examples for hamming space search. IEEE Transactions on Cybernetics 50, 4 (2018).","journal-title":"IEEE Transactions on Cybernetics"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1145\/3239571"},{"key":"e_1_3_2_91_2","first-page":"268","volume-title":"Proceedings of the ACM SIGIR Forum","author":"Zhai Chengxiang","year":"2017","unstructured":"Chengxiang Zhai and John Lafferty. 2017. A study of smoothing methods for language models applied to ad hoc information retrieval. In Proceedings of the ACM SIGIR Forum. ACM New York, NY, 268\u2013276."},{"key":"e_1_3_2_92_2","doi-asserted-by":"publisher","DOI":"10.1145\/3545799"},{"key":"e_1_3_2_93_2","doi-asserted-by":"crossref","unstructured":"Peng-Fei Zhang Zi Huang and Xin-Shun Xu. 2021. Proactive privacy-preserving learning for retrieval. In Proceedings of the AAAI Conference on Artificial Intelligence 35 4 (2021) 3369\u20133376.","DOI":"10.1609\/aaai.v35i4.16449"},{"key":"e_1_3_2_94_2","volume-title":"Proceedings of the KDD","author":"Zhou Bin","year":"2009","unstructured":"Bin Zhou and Jian Pei. 2009. OSD: An online web spam detection system. In Proceedings of the KDD."},{"key":"e_1_3_2_95_2","first-page":"781","volume-title":"Proceedings of the ECCV","author":"Zhou Mo","year":"2020","unstructured":"Mo Zhou, Zhenxing Niu, Le Wang, Qilin Zhang, and Gang Hua. 2020. Adversarial ranking attack and defense. In Proceedings of the ECCV. Springer, 781\u2013799."}],"container-title":["ACM Transactions on Information Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3576923","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3576923","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:36:23Z","timestamp":1750178183000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3576923"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,4,8]]},"references-count":94,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2023,10,31]]}},"alternative-id":["10.1145\/3576923"],"URL":"https:\/\/doi.org\/10.1145\/3576923","relation":{},"ISSN":["1046-8188","1558-2868"],"issn-type":[{"value":"1046-8188","type":"print"},{"value":"1558-2868","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,4,8]]},"assertion":[{"value":"2022-05-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2022-11-26","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-04-08","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}