{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,18]],"date-time":"2026-07-18T16:28:04Z","timestamp":1784392084696,"version":"3.55.0"},"reference-count":213,"publisher":"Association for Computing Machinery (ACM)","issue":"6","license":[{"start":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T00:00:00Z","timestamp":1708646400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Cyber Security Research Centre Limited"},{"name":"Australian Government\u2019s Cooperative Research Centres Programme"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2024,6,30]]},"abstract":"<jats:p>The rapid development of Machine Learning (ML) has demonstrated superior performance in many areas, such as computer vision and video and speech recognition. It has now been increasingly leveraged in software systems to automate the core tasks. However, how to securely develop the machine learning-based modern software systems (MLBSS) remains a big challenge, for which the insufficient consideration will largely limit its application in safety-critical domains. One concern is that the present MLBSS development tends to be rushed, and the latent vulnerabilities and privacy issues exposed to external users and attackers will be largely neglected and hard to be identified. Additionally, machine learning-based software systems exhibit different liabilities towards novel vulnerabilities at different development stages from requirement analysis to system maintenance, due to its inherent limitations from the model and data and the external adversary capabilities. The successful generation of such intelligent systems will thus solicit dedicated efforts jointly from different research areas, i.e., software engineering, system security, and machine learning. Most of the recent works regarding the security issues for ML have a strong focus on the data and models, which has brought adversarial attacks into consideration. In this work, we consider that security for machine learning-based software systems may arise from inherent system defects or external adversarial attacks, and the secure development practices should be taken throughout the whole lifecycle. While machine learning has become a new threat domain for existing software engineering practices, there is no such review work covering the topic. Overall, we present a holistic review regarding the security for MLBSS, which covers a systematic understanding from a structure review of three distinct aspects in terms of security threats. Moreover, it provides a thorough state-of-the-practice for MLBSS secure development. Finally, we summarize the literature for system security assurance and motivate the future research directions with open challenges. We anticipate this work provides sufficient discussion and novel insights to incorporate system security engineering for future exploration.<\/jats:p>","DOI":"10.1145\/3638531","type":"journal-article","created":{"date-parts":[[2023,12,28]],"date-time":"2023-12-28T21:57:45Z","timestamp":1703800665000},"page":"1-38","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["Security for Machine Learning-based Software Systems: A Survey of Threats, Practices, and Challenges"],"prefix":"10.1145","volume":"56","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-5678-472X","authenticated-orcid":false,"given":"Huaming","family":"Chen","sequence":"first","affiliation":[{"name":"The University of Sydney, Sydney, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9696-3626","authenticated-orcid":false,"given":"M. Ali","family":"Babar","sequence":"additional","affiliation":[{"name":"CREST - The Centre for Research on Engineering Software Technologies, The University of Adelaide, Australia and Cyber Security Cooperative Research Centre, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,2,23]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2016.10.005"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"issue":"5","key":"e_1_3_2_4_2","first-page":"111","article-title":"Assuring the machine learning lifecycle: Desiderata, methods, and challenges","volume":"54","author":"Ashmore Rob","year":"2021","unstructured":"Rob Ashmore, Radu Calinescu, and Colin Paterson. 2021. Assuring the machine learning lifecycle: Desiderata, methods, and challenges. ACM Comput. Surv. 54, 5, Article 111 (2021), 39 pages.","journal-title":"ACM Comput. Surv."},{"key":"e_1_3_2_5_2","first-page":"1","volume-title":"Proceedings of the CHI Conference on Human Factors in Computing Systems","author":"Assal Hala","year":"2019","unstructured":"Hala Assal and Sonia Chiasson. 2019. \u201cThink secure from the beginning\u201d A survey with software developers. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1\u201313."},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1504\/IJSN.2015.071829"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1145\/3095797"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2019.8802997"},{"key":"e_1_3_2_9_2","first-page":"333","volume-title":"Proceedings of the 12th International Conference on Information Quality (ICIQ\u201907)","author":"Batini Carlo","year":"2007","unstructured":"Carlo Batini, Daniele Barone, Michele Mastrella, Andrea Maurino, and Claudio Ruffini. 2007. A framework and a methodology for data quality assessment and monitoring. In Proceedings of the 12th International Conference on Information Quality (ICIQ\u201907). Citeseer, 333\u2013346."},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-16664-3"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2007.55"},{"key":"e_1_3_2_12_2","first-page":"634","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Bhagoji Arjun Nitin","year":"2019","unstructured":"Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. 2019. Analyzing federated learning through an adversarial lens. In Proceedings of the International Conference on Machine Learning. PMLR, 634\u2013643."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2018.07.023"},{"key":"e_1_3_2_14_2","first-page":"118","volume-title":"Proceedings of the 31st International Conference on Neural Information Processing Systems","author":"Blanchard Peva","year":"2017","unstructured":"Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. 2017. Machine learning with adversaries: Byzantine tolerant gradient descent. In Proceedings of the 31st International Conference on Neural Information Processing Systems. 118\u2013128."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/TechDebt52882.2021.00016"},{"key":"e_1_3_2_16_2","article-title":"Bad characters: Imperceptible NLP attacks","author":"Boucher Nicholas","year":"2021","unstructured":"Nicholas Boucher, Ilia Shumailov, Ross Anderson, and Nicolas Papernot. 2021. Bad characters: Imperceptible NLP attacks. arXiv preprint arXiv:2106.09898 (2021).","journal-title":"arXiv preprint arXiv:2106.09898"},{"issue":"4","key":"e_1_3_2_17_2","first-page":"84","article-title":"Vision-based autonomous vehicle recognition: A new challenge for deep learning-based systems","volume":"54","author":"Boukerche Azzedine","year":"2021","unstructured":"Azzedine Boukerche and Xiren Ma. 2021. Vision-based autonomous vehicle recognition: A new challenge for deep learning-based systems. ACM Comput. Surv. 54, 4, Article 84 (2021), 37 pages.","journal-title":"ACM Comput. Surv."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.5555\/2616205"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2020.110542"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scs.2018.02.039"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.1984.5010196"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.3390\/electronics9091379"},{"key":"e_1_3_2_23_2","article-title":"Adversarial attacks and defences: A survey","author":"Chakraborty Anirban","year":"2018","unstructured":"Anirban Chakraborty, Manaar Alam, Vishal Dey, Anupam Chattopadhyay, and Debdeep Mukhopadhyay. 2018. Adversarial attacks and defences: A survey. arXiv preprint arXiv:1810.00069 (2018).","journal-title":"arXiv preprint arXiv:1810.00069"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1145\/3372297.3417238"},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/647"},{"issue":"4","key":"e_1_3_2_26_2","first-page":"77","article-title":"Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities","volume":"54","author":"Chen Kaixuan","year":"2021","unstructured":"Kaixuan Chen, Dalin Zhang, Lina Yao, Bin Guo, Zhiwen Yu, and Yunhao Liu. 2021. Deep learning for sensor-based human activity recognition: Overview, challenges, and opportunities. ACM Comput. Surv. 54, 4, Article 77 (2021), 40 pages.","journal-title":"ACM Comput. Surv."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409759"},{"key":"e_1_3_2_28_2","first-page":"1964","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Choquette-Choo Christopher A.","year":"2021","unstructured":"Christopher A. Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. 2021. Label-only membership inference attacks. In Proceedings of the International Conference on Machine Learning. PMLR, 1964\u20131974."},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/VTS48691.2020.9107564"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN.2018.8489592"},{"key":"e_1_3_2_31_2","article-title":"Covert channel attack to federated learning systems","author":"Costa Gabriele","year":"2021","unstructured":"Gabriele Costa, Fabio Pinelli, Simone Soderi, and Gabriele Tolomei. 2021. Covert channel attack to federated learning systems. arXiv preprint arXiv:2104.10561 (2021).","journal-title":"arXiv preprint arXiv:2104.10561"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW50498.2020.00406"},{"key":"e_1_3_2_33_2","volume-title":"Secure Software Development Life Cycle Processes: A Technology Scouting Report","author":"Davis Noopur","year":"2005","unstructured":"Noopur Davis. 2005. Secure Software Development Life Cycle Processes: A Technology Scouting Report. Technical Report. Carnegie-Mellon Univ Pittsburgh PA Software Engineering Institute."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2019.8870157"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3338906.3338954"},{"key":"e_1_3_2_36_2","unstructured":"Dick Fairley Alice Squires and Keith Willett. 2020. System security\u2014Guide to the Systems Engineering Body of Knowledge. Retrieved from https:\/\/www.sebokwiki.org\/wiki\/System_Security"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2021.3103064"},{"key":"e_1_3_2_38_2","first-page":"1605","volume-title":"Proceedings of the 29th USENIX Security Symposium (USENIX Security\u201920)","author":"Fang Minghong","year":"2020","unstructured":"Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Gong. 2020. Local model poisoning attacks to byzantine-robust federated learning. In Proceedings of the 29th USENIX Security Symposium (USENIX Security\u201920). 1605\u20131622."},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3452750"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3452795"},{"key":"e_1_3_2_41_2","article-title":"Quality assurance for AI-based systems: Overview and challenges","author":"Felderer Michael","year":"2021","unstructured":"Michael Felderer and Rudolf Ramler. 2021. Quality assurance for AI-based systems: Overview and challenges. In Proceedings of the International Conference on Software Quality.","journal-title":"Proceedings of the International Conference on Software Quality"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3340482.3342743"},{"key":"e_1_3_2_43_2","article-title":"DataExposer: Exposing disconnect between data and systems","author":"Galhotra Sainyam","year":"2021","unstructured":"Sainyam Galhotra, Anna Fariha, Raoni Louren\u00e7o, Juliana Freire, Alexandra Meliou, and Divesh Srivastava. 2021. DataExposer: Exposing disconnect between data and systems. arXiv preprint arXiv:2105.06058 (2021).","journal-title":"arXiv preprint arXiv:2105.06058"},{"key":"e_1_3_2_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243834"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-54549-9_13"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380391"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.iot.2022.100514"},{"key":"e_1_3_2_48_2","article-title":"MLDemon: Deployment monitoring for machine learning systems","author":"Ginart Antonio","year":"2021","unstructured":"Antonio Ginart, Martin Zhang, and James Zou. 2021. MLDemon: Deployment monitoring for machine learning systems. arXiv preprint arXiv:2104.13621 (2021).","journal-title":"arXiv preprint arXiv:2104.13621"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2021.111031"},{"key":"e_1_3_2_50_2","article-title":"Dataset security for machine learning: Data poisoning, backdoor attacks, and defenses","author":"Goldblum Micah","year":"2020","unstructured":"Micah Goldblum, Dimitris Tsipras, Chulin Xie, Xinyun Chen, Avi Schwarzschild, Dawn Song, Aleksander Madry, Bo Li, and Tom Goldstein. 2020. Dataset security for machine learning: Data poisoning, backdoor attacks, and defenses. arXiv preprint arXiv:2012.10544 (2020).","journal-title":"arXiv preprint arXiv:2012.10544"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2909068"},{"key":"e_1_3_2_52_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00080"},{"key":"e_1_3_2_53_2","article-title":"Tabor: A highly accurate approach to inspecting and restoring trojan backdoors in AI systems","author":"Guo Wenbo","year":"2019","unstructured":"Wenbo Guo, Lun Wang, Xinyu Xing, Min Du, and Dawn Song. 2019. Tabor: A highly accurate approach to inspecting and restoring trojan backdoors in AI systems. arXiv preprint arXiv:1908.01763 (2019).","journal-title":"arXiv preprint arXiv:1908.01763"},{"key":"e_1_3_2_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2007.70754"},{"key":"e_1_3_2_55_2","doi-asserted-by":"crossref","unstructured":"Gaku Fujii Koichi Hamada Fuyuki Ishikawa Satoshi Masuda Mineo Matsuya Tomoyuki Myojin Yasuharu Nishi Hideto Ogawa Takahiro Toku Susumu Tokumoto Kazunori Tsuchiya and Yasuhiro Ujita. 2020. Guidelines for quality assurance of machine learning-based artificial intelligence. International Journal of Software Engineering and Knowledge Engineering 30 11\u201312 (2020) 1589\u20131606.","DOI":"10.1142\/S0218194020400227"},{"key":"e_1_3_2_56_2","article-title":"Towards security threats of deep learning systems: A survey","author":"He Yingzhe","year":"2020","unstructured":"Yingzhe He, Guozhu Meng, Kai Chen, Xingbo Hu, and Jinwen He. 2020. Towards security threats of deep learning systems: A survey. IEEE Trans. Softw. Eng. 48, 5 (2020), 1743\u20131770.","journal-title":"IEEE Trans. Softw. Eng."},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1145\/3148238"},{"key":"e_1_3_2_58_2","article-title":"Securing artificial intelligence\u2013Part 1: The attack surface of machine learning and its implications","author":"Herpig Sven","year":"2019","unstructured":"Sven Herpig. 2019. Securing artificial intelligence\u2013Part 1: The attack surface of machine learning and its implications. Think Tank at the Intersection of Technology and Society Retrieved from https:\/\/www.stiftung-nv.de\/sites\/default\/files\/securing_artificial_intelligence.pdf","journal-title":"Think Tank at the Intersection of Technology and Society"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1109\/WAIN52551.2021.00020"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3134012"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/SDS.2019.8768572"},{"key":"e_1_3_2_62_2","doi-asserted-by":"publisher","DOI":"10.1109\/RE.2019.00050"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3523273"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2020.2995347"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP52600.2021.00019"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380395"},{"key":"e_1_3_2_67_2","first-page":"1","volume-title":"Proceedings of the NIPS MLSys Workshop","author":"Hynes Nick","year":"2017","unstructured":"Nick Hynes, D. Sculley, and Michael Terry. 2017. The data linter: Lightweight, automated sanity checking for ML data sets. In Proceedings of the NIPS MLSys Workshop. 1\u20137."},{"key":"e_1_3_2_68_2","article-title":"What Is Data Security? Data Security Definition and Overview","year":"2021","unstructured":"IBM. 2021. What Is Data Security? Data Security Definition and Overview. Retrieved from https:\/\/www.ibm.com\/topics\/data-security","journal-title":"R"},{"key":"e_1_3_2_69_2","first-page":"2137","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Ilyas Andrew","year":"2018","unstructured":"Andrew Ilyas, Logan Engstrom, Anish Athalye, and Jessy Lin. 2018. Black-box adversarial attacks with limited queries and information. In Proceedings of the International Conference on Machine Learning. PMLR, 2137\u20132146."},{"key":"e_1_3_2_70_2","volume-title":"26262-1-Road Vehicles Functional Safety Part 1 Vocabulary","author":"ISO ISO","year":"2011","unstructured":"ISO ISO. 2011. 26262-1-Road Vehicles Functional Safety Part 1 Vocabulary. Technical Report. Technical report, International Organization for Standardization\/Technical Committee."},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.5555\/3489212.3489288"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2018.00057"},{"key":"e_1_3_2_73_2","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243757"},{"key":"e_1_3_2_74_2","first-page":"14","volume-title":"Proceedings of the International Conference on Software Business","author":"John Meenu Mary","year":"2020","unstructured":"Meenu Mary John, Helena Holmstr\u00f6m Olsson, and Jan Bosch. 2020. Architecting AI deployment: A systematic review of state-of-the-art and state-of-practice literature. In Proceedings of the International Conference on Software Business. Springer, 14\u201329."},{"key":"e_1_3_2_75_2","doi-asserted-by":"crossref","unstructured":"John Jumper Richard Evans Alexander Pritzel Tim Green Michael Figurnov Olaf Ronneberger Kathryn Tunyasuvunakool Russ Bates Augustin \u017d\u00eddek Anna Potapenko Alex Bridgland Clemens Meyer Simon A. A. Kohl Andrew J. Ballard Andrew Cowie Bernardino Romera-Paredes Stanislav Nikolov Rishub Jain Jonas Adler Trevor Back Stig Petersen David Reiman Ellen Clancy Michal Zielinski Martin Steinegger Michalina Pacholska Tamas Berghammer Sebastian Bodenstein David Silver Oriol Vinyals Andrew W. Senior Koray Kavukcuoglu Pushmeet Kohli and Demis Hassabis. 2021. Highly accurate protein structure prediction with AlphaFold. Nature 596 7873 (2021) 583\u2013589.","DOI":"10.1038\/s41586-021-03819-2"},{"key":"e_1_3_2_76_2","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP.2019.00044"},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377814.3381714"},{"key":"e_1_3_2_78_2","volume-title":"The Impact of Platform Vulnerabilities in AI Systems","author":"Kim Ashley Ashley Hyowon","year":"2020","unstructured":"Ashley Ashley Hyowon Kim. 2020. The Impact of Platform Vulnerabilities in AI Systems. Ph.D. Dissertation. Massachusetts Institute of Technology."},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00108"},{"key":"e_1_3_2_80_2","article-title":"Designing machine learning toolboxes: Concepts, principles and patterns","author":"Kir\u00e1ly Franz J.","year":"2021","unstructured":"Franz J. Kir\u00e1ly, Markus L\u00f6ning, Anthony Blaom, Ahmed Guecioueur, and Raphael Sonabend. 2021. Designing machine learning toolboxes: Concepts, principles and patterns. arXiv preprint arXiv:2101.04938 (2021).","journal-title":"arXiv preprint arXiv:2101.04938"},{"key":"e_1_3_2_81_2","first-page":"1","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Krishna Kalpesh","year":"2020","unstructured":"Kalpesh Krishna, Gaurav Singh Tomar, Ankur P. Parikh, Nicolas Papernot, and Mohit Iyyer. 2020. Thieves on sesame street! Model extraction of BERT-based APIs. In Proceedings of the International Conference on Learning Representations. 1\u201319."},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/1822327.1822341"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.3233\/IDT-190160"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.249"},{"key":"e_1_3_2_85_2","doi-asserted-by":"crossref","unstructured":"Alexander Lavin Ciar\u00e1n M. Gilligan-Lee Alessya Visnjic Siddha Ganju Dava Newman Sujoy Ganguly Danny Lange At\u0131l\u0131m G\u00fcne\u015f Baydin Amit Sharma Adam Gibson Stephan Zheng Eric P. Xing Chris Mattmann James Parr and Yarin Gal. 2022. Technology readiness levels for machine learning systems. Nature Communications 13 1 (2022) 6039.","DOI":"10.1038\/s41467-022-33128-9"},{"key":"e_1_3_2_86_2","volume-title":"Proceedings of the ICML Workshop on Challenges in Deploying Machine Learning Systems","author":"Lavin Alexander","year":"2020","unstructured":"Alexander Lavin and Gregory Renard. 2020. Technology readiness levels for AI & ML. In Proceedings of the ICML Workshop on Challenges in Deploying Machine Learning Systems."},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-4048(02)00413-3"},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1109\/WAIN52551.2021.00028"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2022.3181682"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.1145\/3372297.3423338"},{"key":"e_1_3_2_91_2","doi-asserted-by":"publisher","DOI":"10.1145\/3359789.3359831"},{"key":"e_1_3_2_92_2","article-title":"Rethinking the trigger of backdoor attack","author":"Li Yiming","year":"2020","unstructured":"Yiming Li, Tongqing Zhai, Baoyuan Wu, Yong Jiang, Zhifeng Li, and Shutao Xia. 2020. Rethinking the trigger of backdoor attack. arXiv preprint arXiv:2004.04692 (2020).","journal-title":"arXiv preprint arXiv:2004.04692"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1145\/3436755"},{"key":"e_1_3_2_94_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377815.3381377"},{"key":"e_1_3_2_95_2","doi-asserted-by":"publisher","DOI":"10.1145\/3287624.3288751"},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-32409-4_1"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISQED48828.2020.9137011"},{"key":"e_1_3_2_98_2","article-title":"RoBERTa: A robustly optimized BERT pretraining approach","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692 (2019).","journal-title":"arXiv preprint arXiv:1907.11692"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCAD.2017.8203770"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCD.2017.16"},{"key":"e_1_3_2_101_2","doi-asserted-by":"publisher","DOI":"10.1109\/VTS48691.2020.9107582"},{"key":"e_1_3_2_102_2","article-title":"Backdoor attacks on network certification via data poisoning","author":"Lorenz Tobias","year":"2021","unstructured":"Tobias Lorenz, Marta Kwiatkowska, and Mario Fritz. 2021. Backdoor attacks on network certification via data poisoning. arXiv preprint arXiv:2108.11299 (2021).","journal-title":"arXiv preprint arXiv:2108.11299"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2020.106368"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-SEIP52600.2021.00034"},{"key":"e_1_3_2_105_2","article-title":"Secure deep learning engineering: A software quality assurance perspective","author":"Ma Lei","year":"2018","unstructured":"Lei Ma, Felix Juefei-Xu, Minhui Xue, Qiang Hu, Sen Chen, Bo Li, Yang Liu, Jianjun Zhao, Jianxiong Yin, and Simon See. 2018. Secure deep learning engineering: A software quality assurance perspective. arXiv preprint arXiv:1810.04538 (2018).","journal-title":"arXiv preprint arXiv:1810.04538"},{"key":"e_1_3_2_106_2","doi-asserted-by":"publisher","DOI":"10.1109\/SANER.2019.8668044"},{"key":"e_1_3_2_107_2","doi-asserted-by":"crossref","unstructured":"Lei Ma Felix Juefei-Xu Fuyuan Zhang Jiyuan Sun Minhui Xue Bo Li Chunyang Chen Ting Su Li Li Yang Liu Jianjun Zhao and Yadong Wang. 2018. DeepGauge: Multi-granularity testing criteria for deep learning systems. In Proceedings of the 33rd ACM\/IEEE International Conference on Automated Software Engineering 120\u2013131.","DOI":"10.1145\/3238147.3238202"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2018.00021"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2019.23415"},{"key":"e_1_3_2_110_2","first-page":"4274","volume-title":"Proceedings of the 36th International Conference on Machine Learning","author":"Mahloujifar Saeed","year":"2019","unstructured":"Saeed Mahloujifar. 2019. Universal multi-party poisoning attacks. In Proceedings of the 36th International Conference on Machine Learning. 4274\u20134283."},{"key":"e_1_3_2_111_2","article-title":"Software engineering for AI-based systems: A survey","author":"Mart\u00ednez-Fern\u00e1ndez Silverio","year":"2021","unstructured":"Silverio Mart\u00ednez-Fern\u00e1ndez, Justus Bogner, Xavier Franch, Marc Oriol, Julien Siebert, Adam Trendowicz, Anna Maria Vollmer, and Stefan Wagner. 2021. Software engineering for AI-based systems: A survey. arXiv preprint arXiv:2105.01984 (2021).","journal-title":"arXiv preprint arXiv:2105.01984"},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1038\/498255a"},{"key":"e_1_3_2_113_2","doi-asserted-by":"publisher","DOI":"10.1109\/MSECP.2004.1281254"},{"key":"e_1_3_2_114_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2019.2909955"},{"key":"e_1_3_2_115_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2020.2984868"},{"key":"e_1_3_2_116_2","article-title":"An architectural risk analysis of machine learning systems: Toward more secure machine learning","volume":"23","author":"McGraw Gary","year":"2020","unstructured":"Gary McGraw, Harold Figueroa, Victor Shepardson, and Richie Bonett. 2020. An architectural risk analysis of machine learning systems: Toward more secure machine learning. Berryville Institute of Machine Learning, Clarke County, VA. Accessed on: Mar 23 (2020).","journal-title":"Berryville Institute of Machine Learning, Clarke County, VA. Accessed on: Mar"},{"key":"e_1_3_2_117_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.csi.2010.01.006"},{"key":"e_1_3_2_118_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sysarc.2021.102163"},{"key":"e_1_3_2_119_2","article-title":"Towards a framework for evaluating the safety, acceptability and efficacy of AI systems for health: An initial synthesis","author":"Morley Jessica","year":"2021","unstructured":"Jessica Morley, Caroline Morton, Kassandra Karpathakis, Mariarosaria Taddeo, and Luciano Floridi. 2021. Towards a framework for evaluating the safety, acceptability and efficacy of AI systems for health: An initial synthesis. arXiv preprint arXiv:2104.06910 (2021).","journal-title":"arXiv preprint arXiv:2104.06910"},{"key":"e_1_3_2_120_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2020.10.007"},{"key":"e_1_3_2_121_2","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2014.2344095"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1145\/3243734.3243831"},{"key":"e_1_3_2_123_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-77385-4_10"},{"key":"e_1_3_2_124_2","doi-asserted-by":"publisher","DOI":"10.1109\/CBI.2017.63"},{"key":"e_1_3_2_125_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00766-020-00343-z"},{"key":"e_1_3_2_126_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46397-1_3"},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2021.3054129"},{"key":"e_1_3_2_128_2","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00078"},{"key":"e_1_3_2_129_2","first-page":"1","article-title":"Exploiting machine learning to subvert your spam filter.","volume":"8","author":"Nelson Blaine","year":"2008","unstructured":"Blaine Nelson, Marco Barreno, Fuching Jack Chi, Anthony D. Joseph, Benjamin I. P. Rubinstein, Udam Saini, Charles Sutton, J. Doug Tygar, and Kai Xia. 2008. Exploiting machine learning to subvert your spam filter. Proc. 1st USENIX Worksh. Large-Scale Exploits Emerg. Threats 8 (2008), 1\u20139.","journal-title":"Proc. 1st USENIX Worksh. Large-Scale Exploits Emerg. Threats"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.1145\/3374664.3379534"},{"key":"e_1_3_2_131_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSREW.2019.00087"},{"key":"e_1_3_2_132_2","first-page":"4901","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Odena Augustus","year":"2019","unstructured":"Augustus Odena, Catherine Olsson, David Andersen, and Ian Goodfellow. 2019. TensorFuzz: Debugging neural networks with coverage-guided fuzzing. In Proceedings of the International Conference on Machine Learning. PMLR, 4901\u20134911."},{"key":"e_1_3_2_133_2","first-page":"121","volume-title":"Explainable AI: Interpreting, Explaining and Visualizing Deep Learning","author":"Oh Seong Joon","year":"2019","unstructured":"Seong Joon Oh, Bernt Schiele, and Mario Fritz. 2019. Towards reverse-engineering black-box neural networks. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Springer, 121\u2013144."},{"key":"e_1_3_2_134_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00509"},{"key":"e_1_3_2_135_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-4009"},{"key":"e_1_3_2_136_2","article-title":"Challenges in deploying machine learning: A survey of case studies","author":"Paleyes Andrei","year":"2020","unstructured":"Andrei Paleyes, Raoul-Gabriel Urma, and Neil D. Lawrence. 2020. Challenges in deploying machine learning: A survey of case studies. arXiv preprint arXiv:2011.09926 (2020).","journal-title":"arXiv preprint arXiv:2011.09926"},{"key":"e_1_3_2_137_2","doi-asserted-by":"publisher","DOI":"10.1145\/3270101.3270102"},{"key":"e_1_3_2_138_2","article-title":"Transferability in machine learning: From phenomena to black-box attacks using adversarial samples","author":"Papernot Nicolas","year":"2016","unstructured":"Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. 2016. Transferability in machine learning: From phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 (2016).","journal-title":"arXiv preprint arXiv:1605.07277"},{"key":"e_1_3_2_139_2","doi-asserted-by":"publisher","DOI":"10.1145\/3052973.3053009"},{"key":"e_1_3_2_140_2","article-title":"Towards the science of security and privacy in machine learning","author":"Papernot Nicolas","year":"2016","unstructured":"Nicolas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael Wellman. 2016. Towards the science of security and privacy in machine learning. arXiv preprint arXiv:1611.03814 (2016).","journal-title":"arXiv preprint arXiv:1611.03814"},{"key":"e_1_3_2_141_2","doi-asserted-by":"publisher","DOI":"10.1109\/EuroSP.2018.00035"},{"key":"e_1_3_2_142_2","doi-asserted-by":"publisher","DOI":"10.1145\/3132747.3132785"},{"key":"e_1_3_2_143_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3417063"},{"key":"e_1_3_2_144_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2006.26"},{"key":"e_1_3_2_145_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00107"},{"key":"e_1_3_2_146_2","first-page":"47","volume-title":"Proceedings of the 13th International Symposium on Software Reliability Engineering","author":"Piessens Frank","year":"2002","unstructured":"Frank Piessens. 2002. A taxonomy of causes of software vulnerabilities in internet software. In Proceedings of the 13th International Symposium on Software Reliability Engineering. Citeseer, 47\u201352."},{"key":"e_1_3_2_147_2","doi-asserted-by":"publisher","DOI":"10.1145\/3035918.3054782"},{"key":"e_1_3_2_148_2","unstructured":"Jennifer Prendki. 2018. The curse of big data labeling and three ways to solve it. https:\/\/aws.amazon.com\/blogs\/apn\/the-curse-of-big-data-labeling-and-threeways-to-solve-it\/. Accessed 15 January 2024."},{"key":"e_1_3_2_149_2","volume-title":"The System Development Life Cycle (SDLC)","author":"Radack Shirley","year":"2009","unstructured":"Shirley Radack. 2009. The System Development Life Cycle (SDLC). Technical Report. National Institute of Standards and Technology."},{"key":"e_1_3_2_150_2","article-title":"Machine learning software engineering in practice: An industrial case study","author":"Rahman Md Saidur","year":"2019","unstructured":"Md Saidur Rahman, Emilio Rivera, Foutse Khomh, Yann-Ga\u00ebl Gu\u00e9h\u00e9neuc, and Bernd Lehnert. 2019. Machine learning software engineering in practice: An industrial case study. arXiv preprint arXiv:1906.07154 (2019).","journal-title":"arXiv preprint arXiv:1906.07154"},{"key":"e_1_3_2_151_2","doi-asserted-by":"crossref","unstructured":"Francesco Regazzoni Shivam Bhasin Amir Ali Pour Ihab Alshaer Furkan Aydin Aydin Aysu Vincent Beroulle Giorgio Di Natale Paul Franzon David Hely Naofumi Homma Akira Ito Dirmanto Jap Priyank Kashyap Ilia Polian Seetal Potluri Rei Ueno Elena-Ioana Vatajelu and Ville Yli-M\u00e4yry. 2020. Machine learning and hardware security: Challenges and opportunities. In Proceedings of the 39th International Conference on Computer-Aided Design 1\u20136.","DOI":"10.1145\/3400302.3416260"},{"key":"e_1_3_2_152_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10664-020-09881-0"},{"key":"e_1_3_2_153_2","doi-asserted-by":"publisher","unstructured":"Ronald Ross Michael McEvilley and Janet Oren. 2018. Systems Security Engineering: Considerations for a Multidisciplinary Approach in the Engineering of Trustworthy Secure Systems. DOI:10.6028\/NIST.SP.800-160","DOI":"10.6028\/NIST.SP.800-160"},{"key":"e_1_3_2_154_2","doi-asserted-by":"publisher","DOI":"10.1145\/1644893.1644895"},{"key":"e_1_3_2_155_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2016.09.001"},{"key":"e_1_3_2_156_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2019.23119"},{"key":"e_1_3_2_157_2","first-page":"1","volume-title":"Proceedings of the CHI Conference on Human Factors in Computing Systems","author":"Sambasivan Nithya","year":"2021","unstructured":"Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen Paritosh, and Lora M. Aroyo. 2021. \u201cEveryone wants to do the model work, not the data work\u201d: Data Cascades in High-Stakes AI. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1\u201315."},{"issue":"3","key":"e_1_3_2_158_2","article-title":"The CIA strikes back: Redefining confidentiality, integrity and availability in security.","volume":"10","author":"Samonas Spyridon","year":"2014","unstructured":"Spyridon Samonas and David Coss. 2014. The CIA strikes back: Redefining confidentiality, integrity and availability in security. J. Inf. Syst. Secur. 10, 3 (2014).","journal-title":"J. Inf. Syst. Secur."},{"key":"e_1_3_2_159_2","first-page":"3309","volume-title":"Proceedings of the 30th USENIX Security Symposium (USENIX Security\u201921)","author":"Sato Takami","year":"2021","unstructured":"Takami Sato, Junjie Shen, Ningfei Wang, Yunhan Jia, Xue Lin, and Qi Alfred Chen. 2021. Dirty road can attack: Security of deep learning based automated lane centering under physical-world attack. In Proceedings of the 30th USENIX Security Symposium (USENIX Security\u201921). 3309\u20133326."},{"key":"e_1_3_2_160_2","first-page":"1","volume-title":"Proceedings of the CHI Conference on Human Factors in Computing Systems","author":"Schlesinger Ari","year":"2018","unstructured":"Ari Schlesinger, Kenton P. O\u2019Hara, and Alex S. Taylor. 2018. Let\u2019s talk about race: Identity, chatbots, and AI. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1\u201314."},{"key":"e_1_3_2_161_2","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2020.2980761"},{"key":"e_1_3_2_162_2","first-page":"9389","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Schwarzschild Avi","year":"2021","unstructured":"Avi Schwarzschild, Micah Goldblum, Arjun Gupta, John P. Dickerson, and Tom Goldstein. 2021. Just how toxic is data poisoning? A unified benchmark for backdoor and data poisoning attacks. In Proceedings of the International Conference on Machine Learning. PMLR, 9389\u20139398."},{"key":"e_1_3_2_163_2","first-page":"2503","article-title":"Hidden technical debt in machine learning systems","volume":"28","author":"Sculley David","year":"2015","unstructured":"David Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, Michael Young, Jean-Francois Crespo, and Dan Dennison. 2015. Hidden technical debt in machine learning systems. Adv. Neural Inf. Process. Syst. 28 (2015), 2503\u20132511.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_164_2","unstructured":"SEI\/CERT. 2020. CERT\/CC Vulnerability Note VU#425163 - Machine learning classifiers trained via gradient descent are vulnerable to arbitrary misclassification attack. Retrieved from https:\/\/kb.cert.org\/vuls\/id\/425163"},{"key":"e_1_3_2_165_2","doi-asserted-by":"publisher","DOI":"10.1145\/3382494.3410681"},{"key":"e_1_3_2_166_2","unstructured":"Burr Settles. 2009. Active learning literature survey. Technical Report TR-1648. University of Wisconsin-Madison Department of Computer Sciences."},{"key":"e_1_3_2_167_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395352.3404070"},{"key":"e_1_3_2_168_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.41"},{"key":"e_1_3_2_169_2","first-page":"1","volume-title":"Proceedings of the Workshop on Artificial Intelligence Safety (SafeAI\u201920), co-located with 34th AAAI Conference on Artificial Intelligence (AAAI\u201920)","author":"Smith Colin","year":"2020","unstructured":"Colin Smith, Ewen Denney, and Ganesh Pai. 2020. Hazard contribution modes of machine learning components. In Proceedings of the Workshop on Artificial Intelligence Safety (SafeAI\u201920), co-located with 34th AAAI Conference on Artificial Intelligence (AAAI\u201920). 1\u20139."},{"key":"e_1_3_2_170_2","doi-asserted-by":"publisher","DOI":"10.1145\/3133956.3134077"},{"key":"e_1_3_2_171_2","doi-asserted-by":"publisher","DOI":"10.1145\/3448016.3457250"},{"issue":"2","key":"e_1_3_2_172_2","first-page":"40","article-title":"A taxonomy of supervised learning for IDSs in SCADA environments","volume":"53","author":"Suaboot Jakapan","year":"2020","unstructured":"Jakapan Suaboot, Adil Fahad, Zahir Tari, John Grundy, Abdun Naser Mahmood, Abdulmohsen Almalawi, Albert Y. Zomaya, and Khalil Drira. 2020. A taxonomy of supervised learning for IDSs in SCADA environments. ACM Comput. Surv. 53, 2, Article 40 (2020), 37 pages.","journal-title":"ACM Comput. Surv."},{"key":"e_1_3_2_173_2","first-page":"1299","volume-title":"Proceedings of the 27th USENIX Security Symposium (USENIX Security\u201918)","author":"Suciu Octavian","year":"2018","unstructured":"Octavian Suciu, Radu Marginean, Yigitcan Kaya, Hal Daume III, and Tudor Dumitras. 2018. When does machine learning FAIL? Generalized transferability for evasion and poisoning attacks. In Proceedings of the 27th USENIX Security Symposium (USENIX Security\u201918). 1299\u20131316."},{"key":"e_1_3_2_174_2","article-title":"Adversarial attack and defense on graph data: A survey","author":"Sun Lichao","year":"2018","unstructured":"Lichao Sun, Yingtong Dou, Carl Yang, Ji Wang, Philip S. Yu, Lifang He, and Bo Li. 2018. Adversarial attack and defense on graph data: A survey. arXiv preprint arXiv:1812.10528 (2018).","journal-title":"arXiv preprint arXiv:1812.10528"},{"key":"e_1_3_2_175_2","doi-asserted-by":"publisher","DOI":"10.1145\/3302504.3311802"},{"key":"e_1_3_2_176_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00033"},{"key":"e_1_3_2_177_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58951-6_24"},{"key":"e_1_3_2_178_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2012.12.052"},{"key":"e_1_3_2_179_2","first-page":"601","volume-title":"Proceedings of the 25th USENIX Security Symposium (USENIX Security\u201916)","author":"Tram\u00e8r Florian","year":"2016","unstructured":"Florian Tram\u00e8r, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. 2016. Stealing machine learning models via prediction apis. In Proceedings of the 25th USENIX Security Symposium (USENIX Security\u201916). 601\u2013618."},{"key":"e_1_3_2_180_2","volume-title":"Fault-injection and Neural Trojan Attacks on Spiking Neural Networks","author":"Venceslai Valerio","year":"2020","unstructured":"Valerio Venceslai. 2020. Fault-injection and Neural Trojan Attacks on Spiking Neural Networks. Ph.D. Dissertation. Politecnico di Torino."},{"key":"e_1_3_2_181_2","article-title":"How does machine learning change software development practices?","author":"Wan Zhiyuan","year":"2019","unstructured":"Zhiyuan Wan, Xin Xia, David Lo, and Gail C. Murphy. 2019. How does machine learning change software development practices? IEEE Trans. Softw. Eng. 47, 9 (2019), 1857\u20131871.","journal-title":"IEEE Trans. Softw. Eng."},{"key":"e_1_3_2_182_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2018.00038"},{"key":"e_1_3_2_183_2","first-page":"10859","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Wang Yunjuan","year":"2021","unstructured":"Yunjuan Wang, Poorya Mianjy, and Raman Arora. 2021. Robust learning for data poisoning attacks. In Proceedings of the International Conference on Machine Learning. PMLR, 10859\u201310869."},{"key":"e_1_3_2_184_2","doi-asserted-by":"publisher","DOI":"10.1145\/3368089.3409761"},{"key":"e_1_3_2_185_2","doi-asserted-by":"publisher","DOI":"10.1109\/DSN48063.2020.00031"},{"key":"e_1_3_2_186_2","article-title":"A comprehensive study on security bug characteristics","author":"Wei Ying","unstructured":"Ying Wei, Xiaobing Sun, Lili Bo, Sicong Cao, Xin Xia, and Bin Li. [n.d.]. A comprehensive study on security bug characteristics. J. Softw.: Evolut. Process 33, 10 (n.d.), e2376.","journal-title":"J. Softw.: Evolut. Process"},{"key":"e_1_3_2_187_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-89960-2_22"},{"key":"e_1_3_2_188_2","doi-asserted-by":"publisher","DOI":"10.1109\/QRS-C51114.2020.00078"},{"key":"e_1_3_2_189_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-55583-2_25"},{"issue":"4","key":"e_1_3_2_190_2","first-page":"71","article-title":"Computer vision and natural language processing: Recent approaches in multimedia and robotics","volume":"49","author":"Wiriyathammabhum Peratham","year":"2016","unstructured":"Peratham Wiriyathammabhum, Douglas Summers-Stay, Cornelia Ferm\u00fcller, and Yiannis Aloimonos. 2016. Computer vision and natural language processing: Recent approaches in multimedia and robotics. ACM Comput. Surv. 49, 4, Article 71 (2016), 44 pages.","journal-title":"ACM Comput. Surv."},{"key":"e_1_3_2_191_2","article-title":"Detection of backdoors in trained classifiers without access to the training set","author":"Xiang Zhen","year":"2020","unstructured":"Zhen Xiang, David J. Miller, and George Kesidis. 2020. Detection of backdoors in trained classifiers without access to the training set. IEEE Trans. Neural Netw. Learn. Syst. 33, 3 (2020), 1177\u20131191.","journal-title":"IEEE Trans. Neural Netw. Learn. Syst."},{"key":"e_1_3_2_192_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00044"},{"key":"e_1_3_2_193_2","doi-asserted-by":"publisher","DOI":"10.1145\/3293882.3330579"},{"key":"e_1_3_2_194_2","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/800"},{"key":"e_1_3_2_195_2","article-title":"Towards a robust and trustworthy machine learning system development","author":"Xiong Pulei","year":"2021","unstructured":"Pulei Xiong, Scott Buffett, Shahrear Iqbal, Philippe Lamontagne, Mohammad Mamun, and Heather Molyneaux. 2021. Towards a robust and trustworthy machine learning system development. arXiv preprint arXiv:2101.03042 (2021).","journal-title":"arXiv preprint arXiv:2101.03042"},{"key":"e_1_3_2_196_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394885.3431639"},{"key":"e_1_3_2_197_2","doi-asserted-by":"publisher","DOI":"10.1049\/iet-cdt.2020.0041"},{"key":"e_1_3_2_198_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSICT49897.2020.9278162"},{"key":"e_1_3_2_199_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3009876"},{"key":"e_1_3_2_200_2","doi-asserted-by":"publisher","DOI":"10.1145\/3298981"},{"key":"e_1_3_2_201_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00165-021-00543-6"},{"key":"e_1_3_2_202_2","doi-asserted-by":"publisher","DOI":"10.1109\/ATS.2018.00024"},{"key":"e_1_3_2_203_2","article-title":"Machine learning testing: Survey, landscapes and horizons","author":"Zhang Jie M.","year":"2020","unstructured":"Jie M. Zhang, Mark Harman, Lei Ma, and Yang Liu. 2020. Machine learning testing: Survey, landscapes and horizons. IEEE Trans. Softw. Eng. 48, 1 (2020), 1\u201336.","journal-title":"IEEE Trans. Softw. Eng."},{"key":"e_1_3_2_204_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sysarc.2021.102205"},{"key":"e_1_3_2_205_2","doi-asserted-by":"publisher","DOI":"10.1145\/3158369"},{"key":"e_1_3_2_206_2","doi-asserted-by":"publisher","DOI":"10.1109\/ISSRE.2019.00020"},{"issue":"3","key":"e_1_3_2_207_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3374217","article-title":"Adversarial attacks on deep-learning models in natural language processing: A survey","volume":"11","author":"Zhang Wei Emma","year":"2020","unstructured":"Wei Emma Zhang, Quan Z. Sheng, Ahoud Alhazmi, and Chenliang Li. 2020. Adversarial attacks on deep-learning models in natural language processing: A survey. ACM Trans. Intell. Syst. Technol. 11, 3 (2020), 1\u201341.","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"e_1_3_2_208_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380368"},{"key":"e_1_3_2_209_2","article-title":"Software engineering practice in the development of deep learning applications","author":"Zhang Xufan","year":"2019","unstructured":"Xufan Zhang, Yilin Yang, Yang Feng, and Zhenyu Chen. 2019. Software engineering practice in the development of deep learning applications. arXiv preprint arXiv:1910.03156 (2019).","journal-title":"arXiv preprint arXiv:1910.03156"},{"key":"e_1_3_2_210_2","doi-asserted-by":"publisher","DOI":"10.1145\/3213846.3213866"},{"key":"e_1_3_2_211_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-54549-9_16"},{"key":"e_1_3_2_212_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.2971519"},{"key":"e_1_3_2_213_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.future.2018.09.019"},{"key":"e_1_3_2_214_2","volume-title":"Proceedings of the 30th USENIX Security Symposium (USENIX Security\u201921)","author":"Zhu Yuankun","year":"2021","unstructured":"Yuankun Zhu, Yueqiang Cheng, Husheng Zhou, and Yantao Lu. 2021. Hermes attack: Steal DNN models with lossless inference accuracy. In Proceedings of the 30th USENIX Security Symposium (USENIX Security\u201921)."}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3638531","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3638531","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:06:12Z","timestamp":1750291572000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3638531"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,23]]},"references-count":213,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2024,6,30]]}},"alternative-id":["10.1145\/3638531"],"URL":"https:\/\/doi.org\/10.1145\/3638531","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,23]]},"assertion":[{"value":"2022-01-11","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-15","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}