{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:58:55Z","timestamp":1750309135896,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":57,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,9,11]],"date-time":"2024-09-11T00:00:00Z","timestamp":1726012800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,9,11]]},"DOI":"10.1145\/3650212.3680375","type":"proceedings-article","created":{"date-parts":[[2024,9,11]],"date-time":"2024-09-11T11:44:25Z","timestamp":1726055065000},"page":"1479-1490","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["TeDA: A Testing Framework for Data Usage Auditing in Deep Learning Model Development"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8335-2746","authenticated-orcid":false,"given":"Xiangshan","family":"Gao","sequence":"first","affiliation":[{"name":"Zhejiang University, Hangzhou, China \/ Huawei Technology, Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4322-4285","authenticated-orcid":false,"given":"Jialuo","family":"Chen","sequence":"additional","affiliation":[{"name":"Zhejiang University, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7113-7635","authenticated-orcid":false,"given":"Jingyi","family":"Wang","sequence":"additional","affiliation":[{"name":"Zhejiang University, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4983-0676","authenticated-orcid":false,"given":"Jie","family":"Shi","sequence":"additional","affiliation":[{"name":"Huawei International, Singapore, Singapore"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4221-2162","authenticated-orcid":false,"given":"Peng","family":"Cheng","sequence":"additional","affiliation":[{"name":"Zhejiang University, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3155-3145","authenticated-orcid":false,"given":"Jiming","family":"Chen","sequence":"additional","affiliation":[{"name":"Zhejiang University, Hangzhou, China \/ Hangzhou Dianzi University, Hangzhou, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,9,11]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"[n. d.]. https:\/\/edition.cnn.com\/2020\/02\/26\/tech\/clearview-ai-hack\/index.html"},{"key":"e_1_3_2_1_2_1","unstructured":"2022. GitHub Copilot sets off AI alarm bells. https:\/\/www.theregister.com\/2022\/11\/11\/githubs_copilot_opinion\/"},{"key":"e_1_3_2_1_3_1","unstructured":"2023. AI Art Copyright Lawsuit. https:\/\/www.theverge.com\/2023\/2\/6\/23587393\/ai-art-copyright-lawsuit-getty-images-stable-diffusion"},{"key":"e_1_3_2_1_4_1","volume-title":"International conference on machine learning. 284\u2013293","author":"Athalye Anish","year":"2018","unstructured":"Anish Athalye, Logan Engstrom, Andrew Ilyas, and Kevin Kwok. 2018. Synthesizing robust adversarial examples. In International conference on machine learning. 284\u2013293."},{"volume-title":"Handbook of tables for probability and statistics","author":"Beyer William H","key":"e_1_3_2_1_5_1","unstructured":"William H Beyer. 2019. Handbook of tables for probability and statistics. Crc Press."},{"key":"e_1_3_2_1_6_1","volume-title":"2022 IEEE Symposium on Security and Privacy (SP). 1897\u20131914","author":"Carlini Nicholas","year":"2022","unstructured":"Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. 2022. Membership inference attacks from first principles. In 2022 IEEE Symposium on Security and Privacy (SP). 1897\u20131914."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Nicholas Carlini and David Wagner. 2017. Towards evaluating the robustness of neural networks. In 2017 ieee symposium on security and privacy (sp). 39\u201357.","DOI":"10.1109\/SP.2017.49"},{"key":"e_1_3_2_1_8_1","volume-title":"2022 IEEE symposium on security and privacy (SP). 824\u2013841","author":"Chen Jialuo","year":"2022","unstructured":"Jialuo Chen, Jingyi Wang, Tinglan Peng, Youcheng Sun, Peng Cheng, Shouling Ji, Xingjun Ma, Bo Li, and Dawn Song. 2022. Copy, right? a testing framework for copyright protection of deep learning models. In 2022 IEEE symposium on security and privacy (SP). 824\u2013841."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3128572.3140448"},{"key":"e_1_3_2_1_10_1","volume-title":"International conference on machine learning. 1964\u20131974","author":"Choquette-Choo Christopher A","year":"2021","unstructured":"Christopher A Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. 2021. Label-only membership inference attacks. In International conference on machine learning. 1964\u20131974."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206848"},{"key":"e_1_3_2_1_12_1","volume-title":"An interactive web-based dashboard to track COVID-19 in real time. The Lancet infectious diseases, 20, 5","author":"Dong Ensheng","year":"2020","unstructured":"Ensheng Dong, Hongru Du, and Lauren Gardner. 2020. An interactive web-based dashboard to track COVID-19 in real time. The Lancet infectious diseases, 20, 5 (2020), 533\u2013534."},{"key":"e_1_3_2_1_13_1","volume-title":"Exploring Memorization in Adversarial Training. In International Conference on Learning Representations.","author":"Dong Yinpeng","year":"2021","unstructured":"Yinpeng Dong, Ke Xu, Xiao Yang, Tianyu Pang, Zhijie Deng, Hang Su, and Jun Zhu. 2021. Exploring Memorization in Adversarial Training. In International Conference on Learning Representations."},{"volume-title":"The age of generative AI and AI-generated everything","author":"Du Hongyang","key":"e_1_3_2_1_14_1","unstructured":"Hongyang Du, Dusit Niyato, Jiawen Kang, Zehui Xiong, Ping Zhang, Shuguang Cui, Xuemin Shen, Shiwen Mao, Zhu Han, and Abbas Jamalipour. 2024. The age of generative AI and AI-generated everything. IEEE Network."},{"key":"e_1_3_2_1_15_1","unstructured":"European Commission. 2022. Artificial Intelligence Act. https:\/\/artificialintelligenceact.eu\/the-act\/"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3357713.3384290"},{"key":"e_1_3_2_1_17_1","first-page":"2881","article-title":"What neural networks memorize and why: Discovering the long tail via influence estimation","volume":"33","author":"Feldman Vitaly","year":"2020","unstructured":"Vitaly Feldman and Chiyuan Zhang. 2020. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 33 (2020), 2881\u20132891.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1177\/2053951719860542"},{"key":"e_1_3_2_1_19_1","article-title":"Verifi: Towards verifiable federated unlearning","author":"Gao Xiangshan","year":"2024","unstructured":"Xiangshan Gao, Xingjun Ma, Jingyi Wang, Youcheng Sun, Bo Li, Shouling Ji, Peng Cheng, and Jiming Chen. 2024. Verifi: Towards verifiable federated unlearning. IEEE Transactions on Dependable and Secure Computing.","journal-title":"IEEE Transactions on Dependable and Secure Computing."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2016.37"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"crossref","first-page":"37","DOI":"10.1104\/pp.101.1.37","article-title":"Photosynthetic fractionation of the stable isotopes of oxygen and carbon","volume":"101","author":"Guy Robert D","year":"1993","unstructured":"Robert D Guy, Marilyn L Fogel, and Joseph A Berry. 1993. Photosynthetic fractionation of the stable isotopes of oxygen and carbon. Plant Physiology, 101, 1 (1993), 37\u201347.","journal-title":"Plant Physiology"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_1_23_1","unstructured":"Hongsheng Hu Zoran Salcic Gillian Dobbie Jinjun Chen Lichao Sun and Xuyun Zhang. 2022. Membership Inference via Backdooring. arXiv preprint arXiv:2206.04823."},{"key":"e_1_3_2_1_24_1","volume-title":"A new defense against adversarial images: Turning a weakness into a strength. Advances in neural information processing systems, 32","author":"Hu Shengyuan","year":"2019","unstructured":"Shengyuan Hu, Tao Yu, Chuan Guo, Wei-Lun Chao, and Kilian Q Weinberger. 2019. A new defense against adversarial images: Turning a weakness into a strength. Advances in neural information processing systems, 32 (2019)."},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.243"},{"key":"e_1_3_2_1_26_1","volume-title":"Workshop on faces in\u2019Real-Life\u2019Images: detection, alignment, and recognition.","author":"Huang Gary B","year":"2008","unstructured":"Gary B Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. 2008. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on faces in\u2019Real-Life\u2019Images: detection, alignment, and recognition."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/3065386"},{"key":"e_1_3_2_1_28_1","unstructured":"Alexey Kurakin Ian Goodfellow and Samy Bengio. 2016. Adversarial examples in the physical world."},{"key":"e_1_3_2_1_29_1","unstructured":"Yiming Li Ziqi Zhang Jiawang Bai Baoyuan Wu Yong Jiang and Shu-Tao Xia. 2020. Open-sourced dataset protection via backdoor watermarking. arXiv preprint arXiv:2010.05821."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.425"},{"key":"e_1_3_2_1_31_1","unstructured":"Aleksander Madry Aleksandar Makelov Ludwig Schmidt Dimitris Tsipras and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083."},{"key":"e_1_3_2_1_32_1","unstructured":"Douglas C Montgomery and George C Runger. 2010. Applied statistics and probability for engineers. John wiley & sons."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.282"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"crossref","unstructured":"Nina Narodytska and Shiva Prasad Kasiviswanathan. 2016. Simple black-box adversarial perturbations for deep networks. arXiv preprint arXiv:1612.06299.","DOI":"10.1109\/CVPRW.2017.172"},{"key":"e_1_3_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICIP.2014.7025068"},{"key":"e_1_3_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2009.191"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/3052973.3053009"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2016.41"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijinfomgt.2013.11.002"},{"key":"e_1_3_2_1_40_1","volume-title":"International Conference on Machine Learning. 8326\u20138335","author":"Sablayrolles Alexandre","year":"2020","unstructured":"Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, and Herv\u00e9 J\u00e9gou. 2020. Radioactive data: tracing through training. In International Conference on Machine Learning. 8326\u20138335."},{"key":"e_1_3_2_1_41_1","volume-title":"ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models. In Network and Distributed Systems Security Symposium","author":"Salem Ahmed","year":"2019","unstructured":"Ahmed Salem, Yang Zhang, Mathias Humbert, Mario Fritz, and Michael Backes. 2019. ML-Leaks: Model and Data Independent Membership Inference Attacks and Defenses on Machine Learning Models. In Network and Distributed Systems Security Symposium 2019."},{"key":"e_1_3_2_1_42_1","unstructured":"Amazon Web Services. 2021. Amazon Rekognition. https:\/\/aws.amazon.com\/rekognition\/"},{"key":"e_1_3_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.41"},{"key":"e_1_3_2_1_44_1","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556."},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330885"},{"key":"e_1_3_2_1_46_1","volume-title":"30th USENIX Security Symposium (USENIX Security 21)","author":"Song Liwei","year":"2021","unstructured":"Liwei Song and Prateek Mittal. 2021. Systematic evaluation of privacy risks of machine learning models. In 30th USENIX Security Symposium (USENIX Security 21). 2615\u20132632."},{"key":"e_1_3_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/3319535.3354211"},{"key":"e_1_3_2_1_48_1","volume-title":"Proceedings of the AAAI conference on artificial intelligence. 31","author":"Szegedy Christian","year":"2017","unstructured":"Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Proceedings of the AAAI conference on artificial intelligence. 31."},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.220"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/TrustCom.2012.282"},{"key":"e_1_3_2_1_51_1","unstructured":"Thomas Tanay and Lewis Griffin. 2016. A boundary tilting persepective on the phenomenon of adversarial examples. arXiv preprint arXiv:1608.07690."},{"key":"e_1_3_2_1_52_1","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence. 32","author":"Tian Shixin","year":"2018","unstructured":"Shixin Tian, Guolei Yang, and Ying Cai. 2018. Detecting adversarial examples through image transformation. In Proceedings of the AAAI Conference on Artificial Intelligence. 32."},{"key":"e_1_3_2_1_53_1","unstructured":"Alexander Turner Dimitris Tsipras and Aleksander Madry. 2018. Clean-label backdoor attacks."},{"key":"e_1_3_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3654677"},{"key":"e_1_3_2_1_55_1","volume-title":"Proceedings on Privacy Enhancing Technologies.","author":"Wenger Emily","year":"2024","unstructured":"Emily Wenger, Xiuyu Li, Ben Y Zhao, and Vitaly Shmatikov. 2024. Data Isotopes for Data Provenance in DNNs. Proceedings on Privacy Enhancing Technologies."},{"volume-title":"Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF). 268\u2013282","author":"Yeom Samuel","key":"e_1_3_2_1_56_1","unstructured":"Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. 2018. Privacy risk in machine learning: Analyzing the connection to overfitting. In 2018 IEEE 31st computer security foundations symposium (CSF). 268\u2013282."},{"key":"e_1_3_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.463"}],"event":{"name":"ISSTA '24: 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering","AITO"],"location":"Vienna Austria","acronym":"ISSTA '24"},"container-title":["Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3650212.3680375","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3650212.3680375","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:50:08Z","timestamp":1750287008000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3650212.3680375"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,11]]},"references-count":57,"alternative-id":["10.1145\/3650212.3680375","10.1145\/3650212"],"URL":"https:\/\/doi.org\/10.1145\/3650212.3680375","relation":{},"subject":[],"published":{"date-parts":[[2024,9,11]]},"assertion":[{"value":"2024-09-11","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}