{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:39:10Z","timestamp":1785537550094,"version":"3.56.0"},"publisher-location":"New York, NY, USA","reference-count":35,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,11,26]],"date-time":"2023-11-26T00:00:00Z","timestamp":1700956800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,11,30]]},"DOI":"10.1145\/3605770.3625215","type":"proceedings-article","created":{"date-parts":[[2023,11,23]],"date-time":"2023-11-23T11:46:12Z","timestamp":1700739972000},"page":"17-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":21,"title":["Distinguishing AI- and Human-Generated Code: A Case Study"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-5103-9067","authenticated-orcid":false,"given":"Sufiyan","family":"Bukhari","sequence":"first","affiliation":[{"name":"University of Calgary, Calgary, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7642-3638","authenticated-orcid":false,"given":"Benjamin","family":"Tan","sequence":"additional","affiliation":[{"name":"University of Calgary, Calgary, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0432-3686","authenticated-orcid":false,"given":"Lorenzo","family":"De Carli","sequence":"additional","affiliation":[{"name":"University of Calgary, Calgary, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,11,26]]},"reference":[{"key":"e_1_3_2_1_1_1","unstructured":"2021. Executive Order on Improving the Nation's Cybersecurity. https:\/\/www.whitehouse.gov\/briefing-room\/presidentialactions\/2021\/05\/12\/executive-order-on-improving-the-nations-cybersecurity\/."},{"key":"e_1_3_2_1_2_1","unstructured":"2022. Software Security in Supply Chains. https:\/\/www.nist.gov\/itl\/executiveorder-14028-improving-nations-cybersecurity\/software-security-supplychains"},{"key":"e_1_3_2_1_3_1","unstructured":"2023. Networkx 3.1 Documentation. https:\/\/networkx.org\/documentation\/stable\/index.html."},{"key":"e_1_3_2_1_4_1","unstructured":"2023. Scikit-Learn: Machine Learning in Python - Scikit-Learn 1.2.2 Documentation. https:\/\/scikit-learn.org\/stable\/."},{"key":"e_1_3_2_1_5_1","unstructured":"2023. SigStore - Open Source Security Foundation. https:\/\/openssf.org\/ community\/sigstore\/"},{"key":"e_1_3_2_1_6_1","unstructured":"2023. Tree-sitter | Introduction. https:\/\/tree-sitter.github.io\/tree-sitter\/."},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"crossref","unstructured":"Adriana Sejfia and Max Schafer. 2022. Practical Automated Detection of Malicious Npm Packages. In ICSE.","DOI":"10.1145\/3510003.3510104"},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","unstructured":"Paul E. Black Vadim Okun and Barbara Guttman. 2021. Guidelines on Minimum Standards for Developer Verification of Software. https:\/\/doi.org\/10.6028\/NIST. IR.8397","DOI":"10.6028\/NIST"},{"key":"e_1_3_2_1_9_1","unstructured":"Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language Models are Few-Shot Learners. arXiv:2005.14165 [cs.CL]"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"crossref","unstructured":"Aylin Caliskan Fabian Yamaguchi Edwin Dauber Richard Harang Konrad Rieck Rachel Greenstadt and Arvind Narayanan. 2018. When Coding Style Survives Compilation: De-anonymizing Programmers from Executable Binaries. In NDSS. https:\/\/www.ndss-symposium.org\/wp-content\/uploads\/sites\/25\/2018\/ 02\/ndss2018_06B-2_Caliskan_paper.pdf","DOI":"10.14722\/ndss.2018.23304"},{"key":"e_1_3_2_1_11_1","volume-title":"USENIX Security Symposium.","author":"Caliskan-Islam Aylin","year":"2015","unstructured":"Aylin Caliskan-Islam, Richard Harang, Andrew Liu, Arvind Narayanan, Clare Voss, Fabian Yamaguchi, and Rachel Greenstadt. 2015. De-Anonymizing Programmers via Code Stylometry. In USENIX Security Symposium."},{"key":"e_1_3_2_1_12_1","unstructured":"Mark Chen Jerry Tworek Heewoo Jun Qiming Yuan Henrique Ponde de Oliveira Pinto Jared Kaplan Harri Edwards Yuri Burda Nicholas Joseph Greg Brockman Alex Ray Raul Puri Gretchen Krueger Michael Petrov Heidy Khlaaf Girish Sastry Pamela Mishkin Brooke Chan Scott Gray Nick Ryder Mikhail Pavlov Alethea Power Lukasz Kaiser Mohammad Bavarian Clemens Winter Philippe Tillet Felipe Petroski Such Dave Cummings Matthias Plappert Fotios Chantzis Elizabeth Barnes Ariel Herbert-Voss William Hebgen Guss Alex Nichol Alex Paino Nikolas Tezak Jie Tang Igor Babuschkin Suchir Balaji Shantanu Jain William Saunders Christopher Hesse Andrew N. Carr Jan Leike Josh Achiam Vedant Misra Evan Morikawa Alec Radford Matthew Knight Miles Brundage Mira Murati Katie Mayer Peter Welinder Bob McGrew Dario Amodei Sam McCandlish Ilya Sutskever and Wojciech Zaremba. 2021. Evaluating Large Language Models Trained on Code. arXiv:2107.03374 [cs.LG]"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","unstructured":"George Christou Grigoris Ntousakis Eric Lahtinen Sotiris Ioannidis Vasileios P Kemerlis and Nikos Vasilakis. 2023. BinWrap: Hybrid Protection against Native Node.Js Add-ons. In ACM AsiaCCS.","DOI":"10.1145\/3579856.3590330"},{"key":"e_1_3_2_1_14_1","unstructured":"Thomas Claburn. 2023. GitHub and OpenAI Fail to Wriggle out of Copilot Lawsuit. https:\/\/www.theregister.com\/2023\/05\/12\/github_microsoft_openai_copilot\/."},{"key":"e_1_3_2_1_15_1","unstructured":"Lucian Constantin. 2020. SolarWinds Attack Explained: And Why It Was so Hard to Detect | CSO Online. https:\/\/www.csoonline.com\/article\/3601508\/solarwindssupply-chain-attack-explained-why-organizations-were-not-prepared.html."},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.2478\/popets-2021-0080"},{"key":"e_1_3_2_1_17_1","unstructured":"Thomas Dohmke. 2022. GitHub Copilot Is Generally Available to All Developers. https:\/\/github.blog\/2022-06--21-github-copilot-is-generally-available-to-alldevelopers\/."},{"key":"e_1_3_2_1_18_1","volume-title":"Bad Snakes: Understanding and Improving Python Package Index Malware Scanning. In ICSE.","author":"Vu Duc Ly","year":"2023","unstructured":"Duc Ly Vu, Zachary Newman, and John Speed Meyers. 2023. Bad Snakes: Understanding and Improving Python Package Index Malware Scanning. In ICSE."},{"key":"e_1_3_2_1_19_1","unstructured":"Emily Dreibelbis. 2023. OpenAI Quietly Shuts Down AI Text-Detection Tool Over Inaccuracies. https:\/\/www.pcmag.com\/news\/openai-quietly-shuts-downai-text-detection-tool-over-inaccuracies."},{"key":"e_1_3_2_1_20_1","unstructured":"Mark Chen et al. 2021. Evaluating Large Language Models Trained on Code. arXiv:2107.03374 (July 2021). arXiv:2107.03374 [cs] http:\/\/arxiv.org\/abs\/2107. 03374"},{"key":"e_1_3_2_1_21_1","volume-title":"Detection of Intrusions and Malware, and Vulnerability Assessment.","author":"Fass Aurore","unstructured":"Aurore Fass, Robert P. Krawczyk, Michael Backes, and Ben Stock. 2018. JaSt: Fully Syntactic Detection of Malicious (Obfuscated) JavaScript. In Detection of Intrusions and Malware, and Vulnerability Assessment. Vol. 10885. Springer International Publishing, Cham, 303--325. http:\/\/link.springer.com\/10.1007\/978- 3--319--93411--2_14"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"crossref","unstructured":"Shafi Goldwasser Michael P. Kim Vinod Vaikuntanathan and Or Zamir. 2022. Planting Undetectable Backdoors in Machine Learning Models. arXiv:2204.06974 [cs] http:\/\/arxiv.org\/abs\/2204.06974","DOI":"10.1109\/FOCS54457.2022.00092"},{"key":"e_1_3_2_1_23_1","volume-title":"Lorenzo De Carli, and Drew Davidson","author":"Hansen Niels","year":"2020","unstructured":"Niels Hansen, Lorenzo De Carli, and Drew Davidson. 2020. Assessing Adaptive Attacks Against Trained JavaScript Classifiers. In Security and Privacy in Communication Networks. Vol. 335. Springer International Publishing, Cham, 190--210. https:\/\/link.springer.com\/10.1007\/978--3-030--63086--7_12"},{"key":"e_1_3_2_1_24_1","volume-title":"CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. arXiv:2203.13474 (Sept","author":"Nijkamp Erik","year":"2022","unstructured":"Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2022. CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis. arXiv:2203.13474 (Sept. 2022). arXiv:2203.13474 [cs] http:\/\/arxiv.org\/abs\/2203.13474"},{"key":"e_1_3_2_1_25_1","unstructured":"Chris O'Donnell. 2018. The \"event-Stream\" Vulnerability. https:\/\/medium.com\/@codfish\/the-event-stream-vulnerability-6acd4c515aae."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP46214.2022.9833571"},{"key":"e_1_3_2_1_27_1","volume-title":"Do Users Write More Insecure Code with AI Assistants? arXiv:2211.03622 (Nov","author":"Perry Neil","year":"2022","unstructured":"Neil Perry, Megha Srivastava, Deepak Kumar, and Dan Boneh. 2022. Do Users Write More Insecure Code with AI Assistants? arXiv:2211.03622 (Nov. 2022). arXiv:2211.03622 [cs] http:\/\/arxiv.org\/abs\/2211.03622"},{"key":"e_1_3_2_1_28_1","volume-title":"32nd USENIX Security Symposium (USENIX Security 23)","author":"Sandoval Gustavo","year":"2023","unstructured":"Gustavo Sandoval, Hammond Pearce, Teo Nys, Ramesh Karri, Siddharth Garg, and Brendan Dolan-Gavitt. 2023. Lost at C: A User Study on the Security Implications of Large Language Model Code Assistants. In 32nd USENIX Security Symposium (USENIX Security 23). USENIX Association, Anaheim, CA. https:\/\/www.usenix. org\/conference\/usenixsecurity23\/presentation\/sandoval"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","unstructured":"Gustavo Sandoval Hammond Pearce Teo Nys Ramesh Karri Siddharth Garg and Brendan Dolan-Gavitt. 2023. Lost at C: Data from the Security-focused User Study. https:\/\/doi.org\/10.5281\/ZENODO.7708658","DOI":"10.5281\/ZENODO.7708658"},{"key":"e_1_3_2_1_30_1","volume-title":"You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion. In USENIX Security Symposium.","author":"Schuster Roei","year":"2021","unstructured":"Roei Schuster, Congzheng Song, Eran Tromer, and Vitaly Shmatikov. 2021. You Autocomplete Me: Poisoning Vulnerabilities in Neural Code Completion. In USENIX Security Symposium."},{"key":"e_1_3_2_1_31_1","volume-title":"Drew Davidson, and Vaibhav Rastogi.","author":"Vaidya Ruturaj K.","year":"2019","unstructured":"Ruturaj K. Vaidya, Lorenzo De Carli, Drew Davidson, and Vaibhav Rastogi. 2019. Security Issues in Language-based Sofware Ecosystems. CoRR abs\/1903.02613 (2019). arXiv:1903.02613 http:\/\/arxiv.org\/abs\/1903.02613"},{"key":"e_1_3_2_1_32_1","volume-title":"CoRR abs\/1706.03762","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. CoRR abs\/1706.03762 (2017). arXiv:1706.03762 http:\/\/arxiv.org\/abs\/ 1706.03762"},{"key":"e_1_3_2_1_33_1","unstructured":"Alexander Wan Eric Wallace Sheng Shen and Dan Klein. 2023. Poisoning Language Models During Instruction Tuning. arXiv:2305.00944 [cs.CL]"},{"key":"e_1_3_2_1_34_1","volume-title":"Lorenzo De Carli, and Drew Davidson","author":"Wyss Elizabeth","year":"2022","unstructured":"Elizabeth Wyss, Lorenzo De Carli, and Drew Davidson. 2022. What the Fork?: Finding Hidden Code Clones in Npm. In ICSE."},{"key":"e_1_3_2_1_35_1","unstructured":"Elizabeth Wyss Alexander Wittman Drew Davidson and Lorenzo De Carli. 2022. Wolf at the Door: Preventing Install-Time Attacks in Npm with Latch. In ACM AsiaCCS."}],"event":{"name":"CCS '23: ACM SIGSAC Conference on Computer and Communications Security","location":"Copenhagen Denmark","acronym":"CCS '23","sponsor":["SIGSAC ACM Special Interest Group on Security, Audit, and Control"]},"container-title":["Proceedings of the 2023 Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3605770.3625215","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3605770.3625215","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:36:18Z","timestamp":1750178178000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3605770.3625215"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,11,26]]},"references-count":35,"alternative-id":["10.1145\/3605770.3625215","10.1145\/3605770"],"URL":"https:\/\/doi.org\/10.1145\/3605770.3625215","relation":{},"subject":[],"published":{"date-parts":[[2023,11,26]]},"assertion":[{"value":"2023-11-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}