{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,13]],"date-time":"2026-05-13T17:23:56Z","timestamp":1778693036580,"version":"3.51.4"},"reference-count":105,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,6,20]],"date-time":"2024-06-20T00:00:00Z","timestamp":1718841600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"European Commission under the Horizon Europe Programme","award":["101070303"],"award-info":[{"award-number":["101070303"]}]},{"name":"EU-NGEU","award":["Project SERICS (PE00000014)"],"award-info":[{"award-number":["Project SERICS (PE00000014)"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Digital Threats"],"published-print":{"date-parts":[[2024,6,30]]},"abstract":"<jats:p>\n            Existing literature on adversarial Machine Learning (ML) focuses either on showing attacks that break every ML model or defenses that withstand most attacks. Unfortunately, little consideration is given to the actual\n            <jats:italic>feasibility<\/jats:italic>\n            of the attack or the defense. Moreover, adversarial samples are often crafted in the \u201cfeature-space,\u201d making the corresponding evaluations of questionable value. Simply put, the current situation does not allow one to estimate the actual threat posed by adversarial attacks, leading to a lack of secure ML systems.\n          <\/jats:p>\n          <jats:p>\n            We aim to clarify such confusion in this article. By considering the application of ML for Phishing Website Detection (PWD), we formalize the \u201cevasion-space,\u201d in which an adversarial perturbation can be introduced to fool an ML-PWD\u2014demonstrating that even perturbations in the \u201cfeature-space\u201d are useful. Then, we propose a realistic threat model describing evasion attacks against ML-PWD that are cheap to stage, and hence intrinsically more attractive for real phishers. After that, we perform the first statistically validated assessment of state-of-the-art ML-PWD against 12 evasion attacks. Our evaluation shows (i)\u00a0the true efficacy of evasion attempts that are more likely to occur; and (ii)\u00a0the impact of perturbations crafted in different evasion-spaces; our realistic evasion attempts induce a statistically significant degradation (3\u201310% at\n            <jats:italic>p<\/jats:italic>\n            &lt; 0.05), and their cheap cost makes them a subtle threat. Notably, however, some ML-PWD are immune to our most realistic attacks (\n            <jats:italic>p<\/jats:italic>\n            = 0.22).\n          <\/jats:p>\n          <jats:p>\n            Finally, as an additional contribution of this journal publication, we are the first to propose and empirically evaluate the intriguing case wherein an attacker introduces perturbations in multiple evasion-spaces\n            <jats:italic>at the same time<\/jats:italic>\n            . These new results show that simultaneously applying perturbations in the problem- and feature-space can cause a drop in the detection rate from 0.95 to 0.\n          <\/jats:p>\n          <jats:p>Our contribution paves the way for a much-needed re-assessment of adversarial attacks against ML systems for cybersecurity.<\/jats:p>","DOI":"10.1145\/3638253","type":"journal-article","created":{"date-parts":[[2023,12,20]],"date-time":"2023-12-20T12:04:16Z","timestamp":1703073856000},"page":"1-51","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Multi-SpacePhish: Extending the Evasion-space of Adversarial Attacks against Phishing Website Detectors Using Machine Learning"],"prefix":"10.1145","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-9530-4725","authenticated-orcid":false,"given":"Ying","family":"Yuan","sequence":"first","affiliation":[{"name":"University of Padua, Padova, Italy"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6890-9611","authenticated-orcid":false,"given":"Giovanni","family":"Apruzzese","sequence":"additional","affiliation":[{"name":"University of Liechtenstein, Vaduz, Liechtenstein"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3612-1934","authenticated-orcid":false,"given":"Mauro","family":"Conti","sequence":"additional","affiliation":[{"name":"University of Padua, Padova, Italy and Delft University of Technology, Delft, Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,6,20]]},"reference":[{"key":"e_1_3_4_2_2","unstructured":"Rami Mohammad and Lee McCluskey. 2015. UCI Phishing Websites Dataset. Retrieved from https:\/\/archive.ics.uci.edu\/ml\/datasets\/phishing+websites"},{"key":"e_1_3_4_3_2","unstructured":"Federal Bureau of Investigation. 2020. Internet Crime Report. Retrieved from https:\/\/www.ic3.gov\/Media\/PDF\/AnnualReport\/2020_IC3Report.pdf"},{"key":"e_1_3_4_4_2","unstructured":"European Commission. 2020. On Artificial Intelligence - A European Approach to Excellence and Trust. Retrieved from https:\/\/commission.europa.eu\/system\/files\/2020-02\/commission-white-paper-artificial-intelligence-feb2020_en.pdf"},{"key":"e_1_3_4_5_2","unstructured":"US Department of Homeland Security. 2021. S&T Artificial Intelligence and Machine Learning Strategic Plan. Retrieved from https:\/\/www.dhs.gov\/sites\/default\/files\/publications\/21_0730_st_ai_ml_strategic_plan_2021.pdf"},{"key":"e_1_3_4_6_2","unstructured":"Nicholas Carlini. 2022. All Adversarial Examples Papers. Retrieved from https:\/\/nicholas.carlini.com\/writing\/2019\/all-adversarial-example-papers.html"},{"key":"e_1_3_4_7_2","unstructured":"CUJO AI. 2022. Machine Learning Security Evasion Competition. Retrieved from https:\/\/mlsec.io\/"},{"key":"e_1_3_4_8_2","unstructured":"PhishTank. 2022. PhishTank. Retrieved from https:\/\/phishtank.org\/"},{"key":"e_1_3_4_9_2","unstructured":"ProofPoint. 2022. State of the Phish 2022. Retrieved from https:\/\/www.proofpoint.com\/it\/resources\/threat-reports\/state-of-phish"},{"key":"e_1_3_4_10_2","volume-title":"Proceedings of the CCS.","author":"Abdelnabi Sahar","year":"2020","unstructured":"Sahar Abdelnabi, Katharina Krombholz, and Mario Fritz. 2020. VisualPhishNet: Zero-day phishing website detection by visual similarity. In Proceedings of the CCS."},{"key":"e_1_3_4_11_2","volume-title":"Proceedings of the USENIX Security","author":"Acharya Bhupendra","year":"2021","unstructured":"Bhupendra Acharya and Phani Vadrevu. 2021. PhishPrint: Evading phishing detection crawlers by prior profiling. In Proceedings of the USENIX Security."},{"key":"e_1_3_4_12_2","volume-title":"Proceedings of the ISI.","author":"Al-Qurashi Rayah","year":"2021","unstructured":"Rayah Al-Qurashi, Ahmed AlEroud, Ahmad A. Saifan, Mohammad Alsmadi, and Izzat Alsmadi. 2021. Generating optimal attack paths in generative adversarial phishing. In Proceedings of the ISI."},{"key":"e_1_3_4_13_2","volume-title":"Proceedings of the CODASPY.","author":"AlEroud Ahmed","year":"2020","unstructured":"Ahmed AlEroud and George Karabatis. 2020. Bypassing detection of URL-based phishing attacks using generative adversarial deep neur al networks. In Proceedings of the CODASPY."},{"key":"e_1_3_4_14_2","volume-title":"Proceedings of the SaTML","author":"Apruzzese Giovanni","year":"2023","unstructured":"Giovanni Apruzzese, Hyrum S. Anderson, Savino Dambra, David Freeman, Fabio Pierazzi, and Kevin Roundy. 2023. \u201cReal attackers don\u2019t compute gradients\u201d: Bridging the gap between adversarial ml research and practice. In Proceedings of the SaTML."},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/3469659"},{"key":"e_1_3_4_16_2","volume-title":"Proceedings of the NCA.","author":"Apruzzese Giovanni","year":"2018","unstructured":"Giovanni Apruzzese and Michele Colajanni. 2018. Evading botnet detectors based on flows and random forest with adversarial samples. In Proceedings of the NCA."},{"key":"e_1_3_4_17_2","volume-title":"Proceedings of the CyCon.","author":"Apruzzese Giovanni","year":"2019","unstructured":"Giovanni Apruzzese, Michele Colajanni, Luca Ferretti, and Mirco Marchetti. 2019. Addressing adversarial attacks against security systems based on machine learning. In Proceedings of the CyCon."},{"key":"e_1_3_4_18_2","volume-title":"Proceedings of the ACSAC","author":"Apruzzese Giovanni","year":"2022","unstructured":"Giovanni Apruzzese, Mauro Conti, and Ying Yuan. 2022. SpacePhish: The evasion-space of adversarial attacks against phishing website detectors using machine learning. In Proceedings of the ACSAC."},{"key":"e_1_3_4_19_2","doi-asserted-by":"crossref","unstructured":"Giovanni Apruzzese Pavel Laskov Edgardo Montes de Oca Wissam Mallouli Luis Burdalo Rapa Athanasios Vasileios Grammatopoulos and Fabio Di Franco. 2023. The role of machine learning in cybersecurity. ACM Dig. Threats: Res. Pract. 4 (2023) 1\u201338.","DOI":"10.1145\/3545574"},{"key":"e_1_3_4_20_2","volume-title":"Proceedings of the IEEE EuroS&P","author":"Apruzzese Giovanni","year":"2022","unstructured":"Giovanni Apruzzese, Pavel Laskov, and Aliya Tastemirova. 2022. SoK: The impact of unlabelled data in cyberthreat detection. In Proceedings of the IEEE EuroS&P."},{"key":"e_1_3_4_21_2","volume-title":"Proceedings of the USENIX Security Symposium.","author":"Arp Daniel","year":"2022","unstructured":"Daniel Arp, Erwin Quiring, Feargus Pendlebury, Alexander Warnecke, Fabio Pierazzi, Christian Wressnegger, Lorenzo Cavallaro, and Konrad Rieck. 2022. Dos and Don\u2019ts of Machine Learning in Computer Security. In Proceedings of the USENIX Security Symposium."},{"key":"e_1_3_4_22_2","volume-title":"Proceedings of the ICMLANT.","author":"Bac Trinh Nguyen","year":"2021","unstructured":"Trinh Nguyen Bac, Phan The Duy, and Van-Hau Pham. 2021. PWDGAN: Generating adversarial malicious URL examples for deceiving black-box phishing website detector using GANs. In Proceedings of the ICMLANT."},{"key":"e_1_3_4_23_2","volume-title":"Proceedings of the USENIX Security.","author":"Bagdasaryan Eugene","year":"2021","unstructured":"Eugene Bagdasaryan and Vitaly Shmatikov. 2021. Blind backdoors in deep learning models. In Proceedings of the USENIX Security."},{"key":"e_1_3_4_24_2","volume-title":"Proceedings of the eCrime.","author":"Bahnsen Alejandro Correa","year":"2018","unstructured":"Alejandro Correa Bahnsen, Ivan Torroledo, Luis David Camacho, and Sergio Villegas. 2018. DeepPhish: Simulating malicious AI. In Proceedings of the eCrime."},{"key":"e_1_3_4_25_2","volume-title":"Proceedings of the CCS.","author":"Biggio Battista","year":"2018","unstructured":"Battista Biggio and Fabio Roli. 2018. Wild patterns: Ten years after the rise of adversarial machine learning. In Proceedings of the CCS."},{"key":"e_1_3_4_26_2","article-title":"Detecting spam in twitter microblogging services: A novel machine learning approach based on domain popularity","volume":"11","author":"Binsaeed Khalid","year":"2020","unstructured":"Khalid Binsaeed, Gianluca Stringhini, and Ahmed E. Youssef. 2020. Detecting spam in twitter microblogging services: A novel machine learning approach based on domain popularity. Int. J. Adv. Comput. Sci. Appl. 11 (2020).","journal-title":"Int. J. Adv. Comput. Sci. Appl."},{"key":"e_1_3_4_27_2","volume-title":"Proceedings of the MuC.","author":"Boenisch Franziska","year":"2021","unstructured":"Franziska Boenisch, Verena Battis, Nicolas Buchmann, and Maija Poikela. 2021. \u201cNever thought about securing my machine learning systems\u201d: A study of security and privacy awareness of machine learning practitioners. In Proceedings of the MuC."},{"key":"e_1_3_4_28_2","doi-asserted-by":"crossref","unstructured":"Andrei Butnaru Alexios Mylonas and Nikolaos Pitropakis. 2021. Towards lightweight URL-based phishing detection. Future Internet 13 (2021) 154.","DOI":"10.3390\/fi13060154"},{"key":"e_1_3_4_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2013.106"},{"key":"e_1_3_4_30_2","volume-title":"Proceedings of the USENIX Security.","author":"Carlini Nicholas","year":"2021","unstructured":"Nicholas Carlini. 2021. Poisoning the unlabeled dataset of semi-supervised learning. In Proceedings of the USENIX Security."},{"key":"e_1_3_4_31_2","unstructured":"Nicholas Carlini Anish Athalye Nicolas Papernot Wieland Brendel Jonas Rauber Dimitris Tsipras Ian Goodfellow Aleksander Madry and Alexey Kurakin. 2019. On evaluating adversarial robustness. Retrieved from https:\/\/arXiv:1902.06705"},{"key":"e_1_3_4_32_2","unstructured":"Nicholas Carlini and David Wagner. 2016. Defensive distillation is not robust to adversarial examples. Retrieved from https:\/\/arXiv:1607.04311"},{"key":"e_1_3_4_33_2","volume-title":"Proceedings of the EISIC.","author":"Chen Lingwei","year":"2017","unstructured":"Lingwei Chen, Yanfang Ye, and Thirimachos Bourlai. 2017. Adversarial machine learning in malware detection: Arms race between evasion attack and defense. In Proceedings of the EISIC."},{"key":"e_1_3_4_34_2","volume-title":"Proceedings of the ESORICS.","author":"Corona Igino","year":"2017","unstructured":"Igino Corona, Battista Biggio, Matteo Contini, Luca Piras, Roberto Corda, Mauro Mereu, Guido Mureddu, Davide Ariu, and Fabio Roli. 2017. Deltaphish: Detecting phishing webpages in compromised websites. In Proceedings of the ESORICS."},{"key":"e_1_3_4_35_2","volume-title":"Machine Learning in the Age of Cyber AI","year":"2020","unstructured":"Darktrace. 2020. Machine Learning in the Age of Cyber AI. Technical Report. Retrieved from https:\/\/www.darktrace.com\/es\/resources\/wp-machine-learning.pdf"},{"key":"e_1_3_4_36_2","doi-asserted-by":"crossref","unstructured":"Ambra Demontis Marco Melis Battista Biggio Davide Maiorca Daniel Arp Konrad Rieck Igino Corona Giorgio Giacinto and Fabio Roli. 2017. Yes machine learning can be more secure! A case study on android malware detection. IEEE TDSC 16 (2017) 711\u2013724.","DOI":"10.1109\/TDSC.2017.2700270"},{"key":"e_1_3_4_37_2","volume-title":"Proceedings of the USENIX Security.","author":"Demontis Ambra","year":"2019","unstructured":"Ambra Demontis, Marco Melis, Maura Pintor, Matthew Jagielski, Battista Biggio, Alina Oprea, Cristina Nita-Rotaru, and Fabio Roli. 2019. Why do adversarial attacks transfer? Explaining transferability of evasion and poisoning attacks. In Proceedings of the USENIX Security."},{"key":"e_1_3_4_38_2","volume-title":"Proceedings of the COMPSAC.","author":"Duman Sevtap","year":"2016","unstructured":"Sevtap Duman, Kubra Kalkan-Cakmakci, Manuel Egele, William Robertson, and Engin Kirda. 2016. Emailprofiler: Spearphishing filtering with header and stylometric features of emails. In Proceedings of the COMPSAC."},{"key":"e_1_3_4_39_2","article-title":"Stakeholder perspectives and requirements on cybersecurity in europe","author":"Fischer-H\u00fcbner Simone","year":"2021","unstructured":"Simone Fischer-H\u00fcbner, Cristina Alcaraz, Afonso Ferreira, Carmen Fernandez-Gago, Javier Lopez, Evangelos Markatos, Lejla Islami, and Mahdi Akil. 2021. Stakeholder perspectives and requirements on cybersecurity in europe. Elsevier J. Inf. Secur. Appl. 61 (2021), 102916.","journal-title":"Elsevier J. Inf. Secur. Appl."},{"key":"e_1_3_4_40_2","article-title":"Evading anti-phishing models: A field note documenting an experience in the machine learning security evasion competition 2022","author":"Gao Yang","year":"2023","unstructured":"Yang Gao, Benjamin M. Ampel, and Sagar Samtani. 2023. Evading anti-phishing models: A field note documenting an experience in the machine learning security evasion competition 2022. ACM Dig. Threats: Res. Pract. (2023).","journal-title":"ACM Dig. Threats: Res. Pract."},{"key":"e_1_3_4_41_2","unstructured":"Gilad Gressel Niranjan Hegde Archana Sreekumar and Michael Darling. 2021. Feature importance guided attack: A model agnostic adversarial attack. Retrieved from https:\/\/arXiv:2106.14815"},{"key":"e_1_3_4_42_2","volume-title":"Proceedings of the ICWSM.","author":"Guo Zhen","year":"2022","unstructured":"Zhen Guo, Jin-Hee Cho, Ray Chen, Srijan Sengupta, Michin Hong, and Tanushree Mitra. 2022. SAFER: Social capital-based friend recommendation to defend against phishing attacks. In Proceedings of the ICWSM."},{"key":"e_1_3_4_43_2","article-title":"A novel approach for phishing URLs detection using lexical based machine learning in a real-time environment","author":"Gupta Brij B.","year":"2021","unstructured":"Brij B. Gupta, Krishna Yadav, Imran Razzak, Konstantinos Psannis, Arcangelo Castiglione, and Xiaojun Chang. 2021. A novel approach for phishing URLs detection using lexical based machine learning in a real-time environment. Elsevier Comp. Commun. 175 (2021), 47\u201357.","journal-title":"Elsevier Comp. Commun."},{"key":"e_1_3_4_44_2","volume-title":"Proceedings of the AsiaCCS.","author":"Gupta Payas","year":"2018","unstructured":"Payas Gupta, Roberto Perdisci, and Mustaque Ahamad. 2018. Towards measuring the role of phone numbers in twitter-advertised spam. In Proceedings of the AsiaCCS."},{"key":"e_1_3_4_45_2","article-title":"Towards benchmark datasets for machine learning based website phishing detection: An experimental study","author":"Hannousse Abdelhakim","year":"2021","unstructured":"Abdelhakim Hannousse and Salima Yahiouche. 2021. Towards benchmark datasets for machine learning based website phishing detection: An experimental study. Elsevier Eng. Appl. Artifi. Intell. 104 (2021), 104347.","journal-title":"Elsevier Eng. Appl. Artifi. Intell."},{"key":"e_1_3_4_46_2","volume-title":"Proceedings of the APCC.","author":"Haruta Shuichiro","year":"2019","unstructured":"Shuichiro Haruta, Fumitaka Yamazaki, Hiromu Asahina, and Iwao Sasase. 2019. A novel visual similarity-based phishing detection scheme using hue information with auto updating database. In Proceedings of the APCC."},{"key":"e_1_3_4_47_2","volume-title":"Proceedings of the USENIX Security Symp.","author":"Ho Grant","year":"2019","unstructured":"Grant Ho, Asaf Cidon, Lior Gavish, Marco Schweighauser, Vern Paxson, Stefan Savage, Geoffrey M. Voelker, and David Wagner. 2019. Detecting and characterizing lateral phishing at scale. In Proceedings of the USENIX Security Symp."},{"key":"e_1_3_4_48_2","article-title":"Building better data protection with SIEM","author":"Howell David","year":"2015","unstructured":"David Howell. 2015. Building better data protection with SIEM. Elsevier Comput. Fraud Secur. 2015.8 (2015), 19\u201320.","journal-title":"Elsevier Comput. Fraud Secur."},{"key":"e_1_3_4_49_2","article-title":"Development of anti-phishing browser based on random forest and rule of extraction framework","author":"Gowda H. R. Mohith","year":"2020","unstructured":"H. R. Mohith Gowda, M. V. Adithya et\u00a0al. 2020. Development of anti-phishing browser based on random forest and rule of extraction framework. Cybersecurity 3 (2020), 1\u201314.","journal-title":"Cybersecurity"},{"key":"e_1_3_4_50_2","article-title":"Towards detection of phishing websites on client-side using machine learning based approach","author":"Jain Ankit Kumar","year":"2018","unstructured":"Ankit Kumar Jain and Brij B. Gupta. 2018. Towards detection of phishing websites on client-side using machine learning based approach. Telecom. Syst. 68 (2018), 687\u2013700.","journal-title":"Telecom. Syst."},{"key":"e_1_3_4_51_2","article-title":"A machine learning based approach for phishing detection using hyperlinks information","author":"Jain Ankit Kumar","year":"2019","unstructured":"Ankit Kumar Jain and Brij B. Gupta. 2019. A machine learning based approach for phishing detection using hyperlinks information. J. Ambient Intell. Human. Comp. 10 (2019), 2015\u20132028.","journal-title":"J. Ambient Intell. Human. Comp."},{"key":"e_1_3_4_52_2","article-title":"Almost tight l0-norm certified robustness of top-k predictions against adversarial perturbations","author":"Jia Jinyuan","year":"2022","unstructured":"Jinyuan Jia, Binghui Wang, Xiaoyu Cao, Hongbin Liu, and Neil Zhenqiang Gong. 2022. Almost tight l0-norm certified robustness of top-k predictions against adversarial perturbations. Int. Conf. Learn. Repr. (2022).","journal-title":"Int. Conf. Learn. Repr."},{"key":"e_1_3_4_53_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aaa8415"},{"key":"e_1_3_4_54_2","volume-title":"Proceedings of the ICICT.","author":"Kettani Houssain","year":"2019","unstructured":"Houssain Kettani and Polly Wainwright. 2019. On the top threats to cyber systems. In Proceedings of the ICICT."},{"key":"e_1_3_4_55_2","volume-title":"Proceedings of the AsiaCCS.","author":"Kim Doowon","year":"2021","unstructured":"Doowon Kim, Haehyun Cho, Yonghwi Kwon, Adam Doup\u00e9, Sooel Son, Gail-Joon Ahn, and Tudor Dumitras. 2021. Security analysis on practices of certificate authorities in the HTTPS phishing ecosystem. In Proceedings of the AsiaCCS."},{"key":"e_1_3_4_56_2","volume-title":"Proceedings of the IEEE COMPSAC.","author":"Kirda Engin","year":"2005","unstructured":"Engin Kirda and Christopher Kruegel. 2005. Protecting users against phishing attacks with antiphish. In Proceedings of the IEEE COMPSAC."},{"key":"e_1_3_4_57_2","volume-title":"Proceedings of the CCS.","author":"Kondracki Brian","year":"2021","unstructured":"Brian Kondracki, Babak Amin Azad, Oleksii Starov, and Nick Nikiforakis. 2021. Catching transparent phish: Analyzing and detecting MITM phishing toolkits. In Proceedings of the CCS."},{"key":"e_1_3_4_58_2","volume-title":"Proceedings of the SPW.","author":"Kumar Ram Shankar Siva","year":"2020","unstructured":"Ram Shankar Siva Kumar, Magnus Nystr\u00f6m, John Lambert, Andrew Marshall, Mario Goertzel, Andi Comissoneru, Matt Swann, and Sharon Xia. 2020. Adversarial machine learning-industry perspectives. In Proceedings of the SPW."},{"issue":"7553","key":"e_1_3_4_59_2","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun Yann","year":"2015","unstructured":"Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436.","journal-title":"Nature"},{"key":"e_1_3_4_60_2","article-title":"Building robust phishing detection system: An empirical analysis","author":"Lee Jehyun","year":"2020","unstructured":"Jehyun Lee, Pingxiao Ye, Ruofan Liu, Dinil Mon Divakaran, and Mun Choon Chan. 2020. Building robust phishing detection system: An empirical analysis. In Proceedings of the NDSS MADWeb.","journal-title":"Proceedings of the NDSS MADWeb."},{"key":"e_1_3_4_61_2","first-page":"12849","article-title":"Practical no-box adversarial attacks against DNNs","volume":"33","author":"Li Qizhang","year":"2020","unstructured":"Qizhang Li, Yiwen Guo, and Hao Chen. 2020. Practical no-box adversarial attacks against DNNs. Adv. Neural Info. Process. Syst. 33 (2020), 12849\u201312860.","journal-title":"Adv. Neural Info. Process. Syst."},{"key":"e_1_3_4_62_2","volume-title":"Proceedings of the WWW.","author":"Liang Bin","year":"2016","unstructured":"Bin Liang, Miaoqiang Su, Wei You, Wenchang Shi, and Gang Yang. 2016. Cracking classifiers for evasion: A case study on the Google\u2019s Phishing pages filter. In Proceedings of the WWW."},{"key":"e_1_3_4_63_2","volume-title":"Proceedings of the CODASPY.","author":"Liao Cong","year":"2018","unstructured":"Cong Liao, Haoti Zhong, Sencun Zhu, and Anna Squicciarini. 2018. Server-based manipulation attacks against machine learning models. In Proceedings of the CODASPY."},{"key":"e_1_3_4_64_2","volume-title":"Proceedings of the USENIX Security.","author":"Lin Yun","year":"2021","unstructured":"Yun Lin, Ruofan Liu, Dinil Mon Divakaran, Jun Yang Ng, Qing Zhou Chan, Yiwen Lu, Yuxuan Si, Fan Zhang, and Jin Song Dong. 2021. Phishpedia: A hybrid deep learning based approach to visually identify phishing webpages. In Proceedings of the USENIX Security."},{"key":"e_1_3_4_65_2","article-title":"Intelligent rule-based phishing websites classification","author":"Mohammad Rami M.","year":"2014","unstructured":"Rami M. Mohammad, Fadi Thabtah, and Lee McCluskey. 2014. Intelligent rule-based phishing websites classification. IET Inf. Secur. 8 (2014), 153\u2013160.","journal-title":"IET Inf. Secur."},{"key":"e_1_3_4_66_2","article-title":"Predicting phishing websites based on self-structuring neural network","author":"Mohammad Rami M.","year":"2014","unstructured":"Rami M. Mohammad, Fadi Thabtah, and Lee McCluskey. 2014. Predicting phishing websites based on self-structuring neural network. Neur. Comp. Appl. 25 (2014), 443\u2013458.","journal-title":"Neur. Comp. Appl."},{"key":"e_1_3_4_67_2","article-title":"The economics of cybersecurity: Principles and policy options","author":"Moore Tyler","year":"2010","unstructured":"Tyler Moore. 2010. The economics of cybersecurity: Principles and policy options. Elsevier Int. J. Crit. Infrastruct. Protect. (2010).","journal-title":"Elsevier Int. J. Crit. Infrastruct. Protect."},{"key":"e_1_3_4_68_2","volume-title":"Proceedings of the CCS.","author":"Mu Jiaming","year":"2021","unstructured":"Jiaming Mu, Binghui Wang, Qi Li, Kun Sun, Mingwei Xu, and Zhuotao Liu. 2021. A hard label black-box adversarial attack against graph neural networks. In Proceedings of the CCS."},{"key":"e_1_3_4_69_2","volume-title":"Proceedings of the USENIX Security.","author":"Nasr Milad","year":"2021","unstructured":"Milad Nasr, Alireza Bahramali, and Amir Houmansadr. 2021. Defeating DNN-based traffic analysis systems in real-time with blind adversarial perturbations. In Proceedings of the USENIX Security."},{"key":"e_1_3_4_70_2","volume-title":"Proceedings of the ISI.","author":"Niakanlahiji Amirreza","year":"2018","unstructured":"Amirreza Niakanlahiji, Bei-Tseng Chu, and Ehab Al-Shaer. 2018. PhishMon: A machine learning framework for detecting phishing webpages. In Proceedings of the ISI."},{"key":"e_1_3_4_71_2","volume-title":"Proceedings of the USENIX Security.","author":"Oest Adam","year":"2020","unstructured":"Adam Oest, Yeganeh Safaei, Penghui Zhang, Brad Wardman, Kevin Tyers, Yan Shoshitaishvili, and Adam Doup\u00e9. 2020. PhishTime: Continuous longitudinal measurement of the effectiveness of anti-phishing blacklists. In Proceedings of the USENIX Security."},{"key":"e_1_3_4_72_2","volume-title":"Proceedings of the USENIX Security.","author":"Oest Adam","year":"2020","unstructured":"Adam Oest, Penghui Zhang, Brad Wardman, Eric Nunes, Jakub Burgis, Ali Zand, Kurt Thomas, Adam Doup\u00e9, and Gail-Joon Ahn. 2020. Sunrise to sunset: Analyzing the end-to-end life cycle and effectiveness of phishing attacks at scale. In Proceedings of the USENIX Security."},{"key":"e_1_3_4_73_2","volume-title":"Proceedings of the ISPA\/BDCloud\/SocialCom\/SustainCom.","author":"O\u2019Mara Alexander","year":"2021","unstructured":"Alexander O\u2019Mara, Izzat Alsmadi, and Ahmed AlEroud. 2021. Generative adverserial analysis of phishing attacks on static and dynamic content of webpages. In Proceedings of the ISPA\/BDCloud\/SocialCom\/SustainCom."},{"key":"e_1_3_4_74_2","volume-title":"Proceedings of the KDD.","author":"Pang Ren","year":"2020","unstructured":"Ren Pang, Xinyang Zhang, Shouling Ji, Xiapu Luo, and Ting Wang. 2020. AdvMind: Inferring adversary intent of black-box attacks. In Proceedings of the KDD."},{"key":"e_1_3_4_75_2","volume-title":"Proceedings of the USENIX Workshop CSET.","author":"Panum Thomas Kobber","year":"2020","unstructured":"Thomas Kobber Panum, Kaspar Hageman, Ren\u00e9 Rydhof Hansen, and Jens Myrup Pedersen. 2020. Towards adversarial phishing detection. In Proceedings of the USENIX Workshop CSET."},{"key":"e_1_3_4_76_2","volume-title":"Proceedings of the AsiaCCS.","author":"Papernot Nicolas","year":"2017","unstructured":"Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. 2017. Practical black-box attacks against machine learning. In Proceedings of the AsiaCCS."},{"key":"e_1_3_4_77_2","volume-title":"Proceedings of the EuroS&P.","author":"Papernot Nicolas","year":"2018","unstructured":"Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael Wellman. 2018. SoK: Security and privacy in machine learning. In Proceedings of the EuroS&P."},{"key":"e_1_3_4_78_2","volume-title":"Proceedings of the IEEE S&P","author":"Papernot Nicolas","year":"2016","unstructured":"Nicolas Papernot, Patrick McDaniel, Xi Wu, Somesh Jha, and Ananthram Swami. 2016. Distillation as a defense to adversarial perturbations against deep neural networks. In Proceedings of the IEEE S&P."},{"key":"e_1_3_4_79_2","volume-title":"Proceedings of the IEEE S&P.","author":"Pierazzi Fabio","year":"2020","unstructured":"Fabio Pierazzi, Feargus Pendlebury, Jacopo Cortellazzi, and Lorenzo Cavallaro. 2020. Intriguing properties of adversarial Ml attacks in the problem space. In Proceedings of the IEEE S&P."},{"key":"e_1_3_4_80_2","volume-title":"Proceedings of the InfoCOM.","author":"Prakash Pawan","year":"2010","unstructured":"Pawan Prakash, Manish Kumar, Ramana Rao Kompella, and Minaxi Gupta. 2010. Phishnet: Predictive blacklisting to detect phishing attacks. In Proceedings of the InfoCOM."},{"key":"e_1_3_4_81_2","volume-title":"Proceedings of the USENIX Security.","author":"Quiring Erwin","year":"2020","unstructured":"Erwin Quiring, David Klein, Daniel Arp, Martin Johns, and Konrad Rieck. 2020. Adversarial preprocessing: Understanding and preventing image-scaling attacks in machine learning. In Proceedings of the USENIX Security."},{"key":"e_1_3_4_82_2","first-page":"402","volume-title":"Proceedings of the Springer BDA","author":"Rathore Hemant","year":"2018","unstructured":"Hemant Rathore, Swati Agarwal, Sanjay K. Sahay, and Mohit Sewak. 2018. Malware detection using machine learning and deep learning. In Proceedings of the Springer BDA. 402\u2013411."},{"key":"e_1_3_4_83_2","unstructured":"Bushra Sabir M. Ali Babar and Raj Gaire. 2020. An evasion attack against ML-based phishing URL detectors. Retrieved from https:\/\/arXiv:2005.08454"},{"key":"e_1_3_4_84_2","volume-title":"Proceedings of the CCS.","author":"Shan Shawn","year":"2020","unstructured":"Shawn Shan, Emily Wenger, Bolun Wang, Bo Li, Haitao Zheng, and Ben Y. Zhao. 2020. Gotta catch\u2019Em all: Using honeypots to catch adversarial attacks on neural networks. In Proceedings of the CCS."},{"key":"e_1_3_4_85_2","volume-title":"Proceedings of the CONECCT.","author":"Sharma Suhas R.","year":"2020","unstructured":"Suhas R. Sharma, Rahul Parthasarathy, and Prasad B. Honnavalli. 2020. A feature selection comparative study for web phishing datasets. In Proceedings of the CONECCT."},{"key":"e_1_3_4_86_2","volume-title":"Proceedings of the IFIP DBSec","author":"Shirazi Hossein","year":"2019","unstructured":"Hossein Shirazi, Bruhadeshwar Bezawada, Indrakshi Ray, and Charles Anderson. 2019. Adversarial sampling attacks against phishing detection. In Proceedings of the IFIP DBSec."},{"key":"e_1_3_4_87_2","volume-title":"Proceedings of the ACM COMPSAC.","author":"Smutz Charles","year":"2012","unstructured":"Charles Smutz and Angelos Stavrou. 2012. Malicious PDF detection using metadata and structural features. In Proceedings of the ACM COMPSAC."},{"key":"e_1_3_4_88_2","article-title":"Advanced evasion attacks and mitigations on practical ML-based phishing website classifiers","author":"Song Fu","year":"2021","unstructured":"Fu Song, Yusi Lei, Sen Chen, Lingling Fan, and Yang Liu. 2021. Advanced evasion attacks and mitigations on practical ML-based phishing website classifiers. Int. J. Intell. Syst. 36 (2021), 5210\u20135240.","journal-title":"Int. J. Intell. Syst."},{"key":"e_1_3_4_89_2","article-title":"One pixel attack for fooling deep neural networks","author":"Su Jiawei","year":"2019","unstructured":"Jiawei Su, Danilo Vasconcellos Vargas, and Kouichi Sakurai. 2019. One pixel attack for fooling deep neural networks. IEEE T. Evol. Comput. 23 (2019), 828\u2013841.","journal-title":"IEEE T. Evol. Comput."},{"key":"e_1_3_4_90_2","article-title":"PhishWHO: Phishing webpage detection via identity keywords extraction and target domain name finder","author":"Tan Choon Lin","year":"2016","unstructured":"Choon Lin Tan, Kang Leng Chiew, KokSheik Wong et\u00a0al. 2016. PhishWHO: Phishing webpage detection via identity keywords extraction and target domain name finder. Elsevier Decis. Support Syst. 88 (2016), 18\u201327.","journal-title":"Elsevier Decis. Support Syst."},{"key":"e_1_3_4_91_2","article-title":"A survey of machine learning-based solutions for phishing website detection","author":"Tang Lizhen","year":"2021","unstructured":"Lizhen Tang and Qusay H. Mahmoud. 2021. A survey of machine learning-based solutions for phishing website detection. Mach. Learn. Knowl. Extract. 3 (2021), 672\u2013694.","journal-title":"Mach. Learn. Knowl. Extract."},{"key":"e_1_3_4_92_2","volume-title":"Proceedings of the IMC.","author":"Tian Ke","year":"2018","unstructured":"Ke Tian, Steve T. K. Jan, Hang Hu, Danfeng Yao, and Gang Wang. 2018. Needle in a haystack: Tracking down elite phishing domains in the wild. In Proceedings of the IMC."},{"key":"e_1_3_4_93_2","volume-title":"Proceedings of the USENIX Security.","author":"Tong Liang","year":"2019","unstructured":"Liang Tong, Bo Li, Chen Hajaj, Chaowei Xiao, Ning Zhang, and Yevgeniy Vorobeychik. 2019. Improving robustness of ML classifiers against realizable evasion attacks using conserved features. In Proceedings of the USENIX Security."},{"key":"e_1_3_4_94_2","volume-title":"Proceedings of the CCS.","author":"Tram\u00e8r Florian","year":"2019","unstructured":"Florian Tram\u00e8r, Pascal Dupr\u00e9, Gili Rusak, Giancarlo Pellegrino, and Dan Boneh. 2019. Adversarial: Perceptual ad blocking meets adversarial machine learning. In Proceedings of the CCS."},{"key":"e_1_3_4_95_2","volume-title":"Proceedings of the ICLR.","author":"Tram\u00e8r Florian","year":"2018","unstructured":"Florian Tram\u00e8r, Alexey Kurakin, Nicolas Papernot, Ian Goodfellow, Dan Boneh, and Patrick McDaniel. 2018. Ensemble adversarial training: Attacks and defenses. In Proceedings of the ICLR."},{"key":"e_1_3_4_96_2","volume-title":"Proceedings of the ARES.","author":"Dooremaal Bram Van","year":"2021","unstructured":"Bram Van Dooremaal, Pavlo Burda, Luca Allodi, and Nicola Zannone. 2021. Combining text and visual features to improve the identification of cloned webpages for early phishing detection. In Proceedings of the ARES."},{"key":"e_1_3_4_97_2","volume-title":"Proceedings of the BigDataSecurity.","author":"Veeramachaneni Kalyan","year":"2016","unstructured":"Kalyan Veeramachaneni, Ignacio Arnaldo, Vamsi Korrapati, Constantinos Bassias, and Ke Li. 2016. AI2: Training a big data machine to defend. In Proceedings of the BigDataSecurity."},{"key":"e_1_3_4_98_2","volume-title":"Proceedings of the CODASPY.","author":"Verma Rakesh","year":"2015","unstructured":"Rakesh Verma and Keith Dyer. 2015. On the character of phishing URLs: Accurate and robust statistical learning classifiers. In Proceedings of the CODASPY."},{"key":"e_1_3_4_99_2","doi-asserted-by":"crossref","DOI":"10.1016\/j.comnet.2020.107275","article-title":"Accurate and fast URL phishing detector: A convolutional neural network approach","author":"Wei Wei","year":"2020","unstructured":"Wei Wei, Qiao Ke, Jakub Nowak, Marcin Korytkowski, Rafa\u0142 Scherer, and Marcin Wo\u017aniak. 2020. Accurate and fast URL phishing detector: A convolutional neural network approach. Elsevier Comp. Netw. 178 (2020), 107275.","journal-title":"Elsevier Comp. Netw."},{"key":"e_1_3_4_100_2","doi-asserted-by":"crossref","DOI":"10.1109\/ACCESS.2014.2305658","article-title":"Some fundamental Cybersecurity concepts","author":"Wilson Kelce S.","year":"2014","unstructured":"Kelce S. Wilson and M\u00fcge Ayse Kiy. 2014. Some fundamental Cybersecurity concepts. IEEE Access 2 (2014), 116\u2013124.","journal-title":"IEEE Access"},{"key":"e_1_3_4_101_2","article-title":"Embedding training within warnings improves skills of identifying phishing webpages","author":"Xiong Aiping","year":"2019","unstructured":"Aiping Xiong, Robert W. Proctor, Weining Yang, and Ninghui Li. 2019. Embedding training within warnings improves skills of identifying phishing webpages. Hum. Fact. 61 (2019), 577\u2013595.","journal-title":"Hum. Fact."},{"key":"e_1_3_4_102_2","volume-title":"Proceedings of the WWW.","author":"Yoon Changhoon","year":"2019","unstructured":"Changhoon Yoon, Kwanwoo Kim, Yongdae Kim, Seungwon Shin, and Sooel Son. 2019. Doppelg\u00e4ngers on the dark web: A large-scale assessment on phishing hidden web services. In Proceedings of the WWW."},{"key":"e_1_3_4_103_2","volume-title":"Proceedings of the IEEE S&P.","author":"Zhang Penghui","year":"2021","unstructured":"Penghui Zhang, Adam Oest, Haehyun Cho, Zhibo Sun, RC Johnson, Brad Wardman, Shaown Sarker, Alexandros Kapravelos, Tiffany Bao, Ruoyu Wang et\u00a0al. 2021. Crawlphish: Large-scale analysis of client-side cloaking techniques in phishing. In Proceedings of the IEEE S&P."},{"key":"e_1_3_4_104_2","volume-title":"Proceedings of the CCS.","author":"Zheng Baolin","year":"2021","unstructured":"Baolin Zheng, Peipei Jiang, Qian Wang, Qi Li, Chao Shen, Cong Wang, Yunjie Ge, Qingyang Teng, and Shenyi Zhang. 2021. Black-box adversarial attacks on commercial speech platforms with minimal information. In Proceedings of the CCS."},{"key":"e_1_3_4_105_2","volume-title":"Proceedings of the RAID.","author":"Zuo Fei","year":"2019","unstructured":"Fei Zuo, Bokai Yang, Xiaopeng Li, and Qiang Zeng. 2019. Exploiting the inherent limitation of l0 adversarial examples. In Proceedings of the RAID."},{"key":"e_1_3_4_106_2","volume-title":"Proceedings of the IEEE S&P.","author":"\u0160rndic Nedim","year":"2014","unstructured":"Nedim \u0160rndic and Pavel Laskov. 2014. Practical evasion of a learning-based classifier: A case study. In Proceedings of the IEEE S&P."}],"container-title":["Digital Threats: Research and Practice"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3638253","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3638253","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:53:35Z","timestamp":1750287215000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3638253"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,20]]},"references-count":105,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,6,30]]}},"alternative-id":["10.1145\/3638253"],"URL":"https:\/\/doi.org\/10.1145\/3638253","relation":{},"ISSN":["2692-1626","2576-5337"],"issn-type":[{"value":"2692-1626","type":"print"},{"value":"2576-5337","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,20]]},"assertion":[{"value":"2023-07-17","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-07","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}