{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,5]],"date-time":"2026-07-05T21:52:25Z","timestamp":1783288345471,"version":"3.54.6"},"reference-count":87,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2025,4,28]],"date-time":"2025-04-28T00:00:00Z","timestamp":1745798400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100020951","name":"Fonds de recherche du Qu\u00e9bec","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100020951","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100007631","name":"Canadian Institute for Advanced Research","doi-asserted-by":"crossref","id":[{"id":"10.13039\/100007631","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,5,31]]},"abstract":"<jats:p>Code auditing ensures that the developed code adheres to standards, regulations, and copyright protection by verifying that it does not contain code from protected sources. The recent advent of Large Language Models (LLMs) as coding assistants in the software development process poses new challenges for code auditing. The dataset for training these models is mainly collected from publicly available sources. This raises the issue of intellectual property infringement as developers\u2019 codes are already included in the dataset. Therefore, auditing code developed using LLMs is challenging, as it is difficult to reliably assert if an LLM used during development has been trained on specific copyrighted codes, given that we do not have access to the training datasets of these models. Given the non-disclosure of the training datasets, traditional approaches such as code clone detection are insufficient for asserting copyright infringement. To address this challenge, we propose a new approach, TraWiC; a model-agnostic and interpretable method based on membership inference for detecting code inclusion in an LLM\u2019s training dataset. We extract syntactic and semantic identifiers unique to each program to train a classifier for detecting code inclusion. In our experiments, we observe that TraWiC is capable of detecting 83.87% of codes that were used to train an LLM. In comparison, the prevalent clone detection tool NiCad is only capable of detecting 47.64%. In addition to its remarkable performance, TraWiC has low resource overhead in contrast to pairwise clone detection that is conducted during the auditing process of tools like CodeWhisperer reference tracker, across thousands of code snippets.<\/jats:p>","DOI":"10.1145\/3702980","type":"journal-article","created":{"date-parts":[[2024,11,2]],"date-time":"2024-11-02T15:26:49Z","timestamp":1730561209000},"page":"1-46","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Trained without My Consent: Detecting Code Inclusion in Language Models Trained on Code"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4411-0810","authenticated-orcid":false,"given":"Vahid","family":"Majdinasab","sequence":"first","affiliation":[{"name":"Polytechnique Montreal, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0440-6839","authenticated-orcid":false,"given":"Amin","family":"Nikanjam","sequence":"additional","affiliation":[{"name":"Polytechnique Montreal, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5704-4173","authenticated-orcid":false,"given":"Foutse","family":"Khomh","sequence":"additional","affiliation":[{"name":"Polytechnique Montreal, Montreal, Quebec, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,4,28]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Style Guide. 2023. Common LiSP style guide\u2014Common LiSP. Retrieved from https:\/\/lisp-lang.org\/style-guide\/"},{"key":"e_1_3_2_3_2","unstructured":"Python. 2023. PEP 8\u2014Style guide for Python code\u2014peps.python.org. Retrieved from https:\/\/peps.python.org\/pep-0008\/"},{"key":"e_1_3_2_4_2","unstructured":"Simian. 2023. Simian similarity analyzer. Retrieved from https:\/\/simian.quandarypeak.com\/"},{"key":"e_1_3_2_5_2","unstructured":"McKinsey & Company. 2023. The state of AI in 2023: Generative AI\u2019s breakout year. Retrieved from https:\/\/www.mckinsey.com\/capabilities\/quantumblack\/our-insights\/the-state-of-ai-in-2023-generative-ais-breakout-year#"},{"key":"e_1_3_2_6_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380405"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00122"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2019.2918202"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.298"},{"key":"e_1_3_2_10_2","unstructured":"Loubna Ben Allal Raymond Li Denis Kocetkov Chenghao Mou Christopher Akiki Carlos Munoz Ferrandis Niklas Muennighoff Mayank Mishra Alex Gu Manan Dey et al. 2023. SantaCoder: Don\u2019T reach for the stars! arXiv:2301.03988."},{"key":"e_1_3_2_11_2","unstructured":"Rohan Anil Andrew M. Dai Orhan Firat Melvin Johnson Dmitry Lepikhin Alexandre Passos Siamak Shakeri Emanuel Taropa Paige Bailey Zhifeng Chen et al. 2023. Palm 2 technical report. arXiv:2305.10403."},{"key":"e_1_3_2_12_2","unstructured":"Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski David Dohan Ellen Jiang Carrie Cai Michael Terry Quoc Le et al. 2021. Program synthesis with large language models. arXiv:2108.07732."},{"key":"e_1_3_2_13_2","unstructured":"Mohammad Bavarian Heewoo Jun Nikolas Tezak John Schulman Christine McLeavey Jerry Tworek and Mark Chen. 2022. Efficient training of language models to fill in the middle. arXiv:2207.14255."},{"key":"e_1_3_2_14_2","unstructured":"Iz Beltagy Matthew E. Peters and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv:2004.05150."},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1201\/9781315139470"},{"key":"e_1_3_2_16_2","unstructured":"Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. arXiv:2005.14165."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-65687-8_21"},{"key":"e_1_3_2_18_2","unstructured":"Nicholas Carlini Daphne Ippolito Matthew Jagielski Katherine Lee Florian Tramer and Chiyuan Zhang. 2022. Quantifying memorization across neural language models. arXiv:2202.07646."},{"key":"e_1_3_2_19_2","first-page":"2633","article-title":"Extracting training data from large language models","author":"Carlini Nicholas","year":"2021","unstructured":"Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. 2021. Extracting training data from large language models. In Proceedings of the 30th USENIX Security Symposium (USENIX Security \u201921), 2633\u20132650.","journal-title":"Proceedings of the 30th USENIX Security Symposium (USENIX Security \u201921)"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.453"},{"key":"e_1_3_2_21_2","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman et al. 2021. Evaluating large language models trained on code. arXiv:2107.03374."},{"key":"e_1_3_2_22_2","first-page":"1154","article-title":"Tools for verifying neural models\u2019 training data","volume":"36","author":"Choi Dami","year":"2023","unstructured":"Dami Choi, Yonadav Shavit, and David Duvenaud. 2023. Tools for verifying neural models\u2019 training data. In Advances in Neural Information Processing Systems. A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Curran Associates, Inc., Vol. 36, 1154\u20131188. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2023\/file\/03e33e1f62e3302b47fe1d38a235921e-Paper-Conference.pdf","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524842.3528440"},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.14722\/bar.2020.23009"},{"key":"e_1_3_2_25_2","first-page":"10088","article-title":"QLORA: Efficient finetuning of quantized LLMs","author":"Dettmers Tim","year":"2024","unstructured":"Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2024. QLORA: Efficient finetuning of quantized LLMs. In Proceedings of the 37th International Conference on Neural Information Processing Systems (NIPS \u201923). Curran Associates Inc., Red Hook, NY, Article 441, 10088\u201310115.","journal-title":"Proceedings of the 37th International Conference on Neural Information Processing Systems (NIPS \u201923)"},{"key":"e_1_3_2_26_2","unstructured":"Jacob Devlin Ming-Wei Chang Kenton Lee and Kristina Toutanova. 2018. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-023-00626-4"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2020.2976920"},{"key":"e_1_3_2_29_2","unstructured":"Joshua Feldman Joe Davison and Alexander M. Rush. 2019. Commonsense knowledge mining from pretrained models. arXiv:1909.00505."},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/WCRE.2007.21"},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-57454-7_53"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1109\/ESEM.2011.22"},{"key":"e_1_3_2_33_2","unstructured":"Xinlei He Rui Wen Yixin Wu Michael Backes Yun Shen and Yang Zhang. 2021. Node-level membership inference attacks against graph neural networks. arXiv:2102.05429."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN52387.2021.9534381"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1145\/3523273"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1109\/TR.2020.3001918"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.14722\/ndss.2021.24293"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/3319535.3363201"},{"key":"e_1_3_2_39_2","unstructured":"Albert Q. Jiang Alexandre Sablayrolles Arthur Mensch Chris Bamford Devendra Singh Chaplot Diego de las Casas Florian Bressand Gianna Lengyel Guillaume Lample Lucile Saulnier et al. 2023. Mistral 7B. arXiv:2310.06825."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3422337.3450321"},{"key":"e_1_3_2_41_2","unstructured":"Denis Kocetkov Raymond Li Loubna Ben Allal Jia Li Chenghao Mou Carlos Mu\u00f1oz Ferrandis Yacine Jernite Margaret Mitchell Sean Hughes Thomas Wolf Dzmitry Bahdanau et al.. 2022. The Stack: 3 TB of permissively licensed source code. arXiv:2211.15533."},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1038\/nature14539"},{"key":"e_1_3_2_43_2","first-page":"1605","article-title":"Stolen memories: Leveraging model memorization for calibrated  \\(\\{\\)","author":"Leino Klas","year":"2020","unstructured":"Klas Leino and Matt Fredrikson. 2020. Stolen memories: Leveraging model memorization for calibrated \\(\\{\\) White-Box \\(\\}\\) membership inference. In Proceedings of the 29th USENIX Security Symposium (USENIX Security \u201920), 1605\u20131622.","journal-title":"Proceedings of the 29th USENIX Security Symposium (USENIX Security \u201920)"},{"key":"e_1_3_2_44_2","first-page":"707","article-title":"Binary codes capable of correcting deletions, insertions and reversals","volume":"10","author":"Levenshtein V. I.","year":"1966","unstructured":"V. I. Levenshtein. 1966. Binary codes capable of correcting deletions, insertions and reversals. Soviet Physics Doklady 10 (Feb. 1966), 707.","journal-title":"Soviet Physics Doklady"},{"key":"e_1_3_2_45_2","unstructured":"Yunhui Long Vincent Bindschaedler Lei Wang Diyue Bu Xiaofeng Wang Haixu Tang Carl A. Gunter and Kai Chen. 2018. Understanding membership inferences on well-generalized learning models. arXiv:1802.04889."},{"key":"e_1_3_2_46_2","article-title":"WizardCoder: Empowering code large language models with evol-instruct","author":"Luo Ziyang","year":"2024","unstructured":"Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. 2024. WizardCoder: Empowering code large language models with evol-instruct. In The Twelfth International Conference on Learning Representations. Retrieved from https:\/\/openreview.net\/forum?id=UnUwSIgK5W","journal-title":"The Twelfth International Conference on Learning Representations"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1145\/3395363.3397369"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1145\/3143434.3143445"},{"key":"e_1_3_2_49_2","volume-title":"Clean Code: A Handbook of Agile Software Craftsmanship","author":"Martin Robert C.","year":"2009","unstructured":"Robert C. Martin. 2009. Clean Code: A Handbook of Agile Software Craftsmanship. Pearson Education."},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2019.00029"},{"key":"e_1_3_2_51_2","unstructured":"Shervin Minaee Tomas Mikolov Narjes Nikzad Meysam Chenaghlu Richard Socher Xavier Amatriain and Jianfeng Gao. 2024. Large language models: A survey. arXiv:2402.06196."},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","unstructured":"Matthew E. Peters Mark Neumann Mohit Iyyer Matt Gardner Christopher Clark Kenton Lee and Luke Zettlemoyer. 2018. Deep contextualized word representations. arXiv:1802.05365.","DOI":"10.18653\/v1\/N18-1202"},{"key":"e_1_3_2_53_2","doi-asserted-by":"crossref","unstructured":"Fabio Petroni Tim Rockt\u00e4schel Patrick Lewis Anton Bakhtin Yuxiang Wu Alexander H. Miller and Sebastian Riedel. 2019. Language models as knowledge bases? arXiv:1909.01066. Retrieved from https:\/\/arxiv.org\/abs\/1909.01066","DOI":"10.18653\/v1\/D19-1250"},{"issue":"11","key":"e_1_3_2_54_2","first-page":"1016","article-title":"Finding plagiarisms among a set of programs with JPlag","volume":"8","author":"Prechelt Lutz","year":"2002","unstructured":"Lutz Prechelt, Guido Malpohl, and Michael Philippsen. 2002. Finding plagiarisms among a set of programs with JPlag. Journal of Universal Computer Science 8, 11 (2002), 1016.","journal-title":"Journal of Universal Computer Science"},{"key":"e_1_3_2_55_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11431-020-1647-3"},{"issue":"8","key":"e_1_3_2_56_2","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCS54944.2021.00056"},{"key":"e_1_3_2_58_2","unstructured":"Lucas Rosenblatt Xiaoyan Liu Samira Pouyanfar Eduardo de Leon Anuj Desai and Joshua Allen. 2020. Differentially private synthetic data: Applied evaluations and enhancements. arXiv:2011.05537. Retrieved from https:\/\/arxiv.org\/abs\/2011.05537"},{"issue":"115","key":"e_1_3_2_59_2","first-page":"64","article-title":"A survey on software clone detection research","volume":"541","author":"Roy Chanchal Kumar","year":"2007","unstructured":"Chanchal Kumar Roy and James R. Cordy. 2007. A survey on software clone detection research. Queen\u2019s School of Computing TR 541, 115 (2007), 64\u201368.","journal-title":"Queen\u2019s School of Computing TR"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICPC.2008.41"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.scico.2009.02.007"},{"key":"e_1_3_2_62_2","unstructured":"Baptiste Rozi\u00e8re Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat Xiaoqing Ellen Tan Yossi Adi Jingyu Liu Tal Remez J\u00e9r\u00e9my Rapin et al. 2023. Code Llama: Open foundation models for code. arXiv:2308.12950."},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.procs.2018.05.080"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1145\/2884781.2884877"},{"key":"e_1_3_2_65_2","doi-asserted-by":"crossref","unstructured":"Ahmed Salem Yang Zhang Mathias Humbert Pascal Berrang Mario Fritz and Michael Backes. 2018. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. arXiv:1806.01246. Retrieved from https:\/\/arxiv.org\/abs\/1806.01246","DOI":"10.14722\/ndss.2019.23119"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/STARTUP.2016.7583913"},{"key":"e_1_3_2_67_2","unstructured":"Amazon Web Services. 2023. Code references - CodeWhisperer. Retrieved from https:\/\/docs.aws.amazon.com\/codewhisperer\/latest\/userguide\/code-reference.html"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i11.17150"},{"key":"e_1_3_2_69_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.41"},{"key":"e_1_3_2_70_2","unstructured":"Aarohi Srivastava Abhinav Rastogi Abhishek Rao Abu Awal Md Shoeb Abubakar Abid Adam Fisch Adam R. Brown Adam Santoro Aditya Gupta Adri\u00e0 Garriga-Alonso et al. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv:2206.04615."},{"key":"e_1_3_2_71_2","first-page":"38274","article-title":"Memorization without overfitting: Analyzing the training dynamics of large language models","volume":"35","author":"Tirumala Kushal","year":"2022","unstructured":"Kushal Tirumala, Aram Markosyan, Luke Zettlemoyer, and Armen Aghajanyan. 2022. Memorization without overfitting: Analyzing the training dynamics of large language models. In Advances in Neural Information Processing Systems, Vol. 35, 38274\u201338290.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_72_2","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv:2307.09288."},{"key":"e_1_3_2_73_2","unstructured":"trawic. 2024. TraWiC. Retrieved from https:\/\/github.com\/CommissarSilver\/TraWiC"},{"key":"e_1_3_2_74_2","unstructured":"Ashish Vaswani Noam Shazeer Niki Parmar Jakob Uszkoreit Llion Jones Aidan N. Gomez Lukasz Kaiser and Illia Polosukhin. 2023. Attention is all you need. arXiv:1706.03762."},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.3390\/info11090421"},{"key":"e_1_3_2_76_2","unstructured":"Michael J. Wise. 1993. String similarity via greedy string tiling and running Karp-Rabin matching. Department of Computer Science Technical Report Sydney University Sydney."},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2019.2897874"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-45871-7_20"},{"key":"e_1_3_2_79_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3482719"},{"issue":"4","key":"e_1_3_2_80_2","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/1082983.1083152","article-title":"Source code that talks: An exploration of Eclipse task comments and their implication to repository mining","volume":"30","author":"Ying Annie T. T.","year":"2005","unstructured":"Annie T. T. Ying, James L. Wright, and Steven Abrams. 2005. Source code that talks: An exploration of Eclipse task comments and their implication to repository mining. ACM SIGSOFT Software Engineering Notes 30, 4 (2005), 1\u20135.","journal-title":"ACM SIGSOFT Software Engineering Notes"},{"key":"e_1_3_2_81_2","doi-asserted-by":"publisher","DOI":"10.1002\/0470011815.b2a15150"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1145\/3446776"},{"key":"e_1_3_2_83_2","first-page":"39321","article-title":"Counterfactual memorization in neural language models","volume":"36","author":"Zhang Chiyuan","year":"2021","unstructured":"Chiyuan Zhang, Daphne Ippolito, Katherine Lee, Matthew Jagielski, Florian Tram\u00e8r, and Nicholas Carlini. 2021. Counterfactual memorization in neural language models. In Advances in Neural Information Processing Systems. A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Curran Associates, Inc., Vol. 36, 39321\u201339362. Retrieved from https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2023\/file\/7bc4f74e35bcfe8cfe43b0a860786d6a-Paper-Conference.pdf","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00086"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1145\/3579990.3580007"},{"key":"e_1_3_2_86_2","doi-asserted-by":"crossref","unstructured":"Sheng Zhang and Hui Li. 2023. Code membership inference for detecting unauthorized data use in code pre-trained language models. arXiv:2312.07200.","DOI":"10.18653\/v1\/2024.findings-emnlp.621"},{"key":"e_1_3_2_87_2","first-page":"452","volume-title":"Neurocomputing","author":"Zheng Junxiang","year":"2021","unstructured":"Junxiang Zheng, Yongzhi Cao, and Hanpin Wang. 2021. Resisting membership inference attacks through knowledge distillation. Neurocomputing 452 (2021), 114\u2013126."},{"key":"e_1_3_2_88_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2017.10.004"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3702980","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3702980","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:18:04Z","timestamp":1750295884000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3702980"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,28]]},"references-count":87,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2025,5,31]]}},"alternative-id":["10.1145\/3702980"],"URL":"https:\/\/doi.org\/10.1145\/3702980","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,28]]},"assertion":[{"value":"2024-01-26","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-26","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}