{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T20:30:39Z","timestamp":1792787439430,"version":"4.5.0"},"reference-count":303,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2025,5,24]],"date-time":"2025-05-24T00:00:00Z","timestamp":1748044800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Key R&D Program of Hubei Province","award":["2023BAB017, 2023BAB079"],"award-info":[{"award-number":["2023BAB017, 2023BAB079"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62072046, 62302181"],"award-info":[{"award-number":["62072046, 62302181"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"HUST CSE-FiberHome"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>The rapid advancement of large language models (LLMs) has revolutionized artificial intelligence, introducing unprecedented capabilities in natural language processing and multimodal content generation. However, the increasing complexity and scale of these models have given rise to a multifaceted supply chain that presents unique challenges across infrastructure, foundation models, and downstream applications. This article provides the first comprehensive research agenda of the LLM supply chain, offering a structured approach to identify critical challenges and opportunities through the dual lenses of software engineering (SE) and security and privacy (S&amp;P). We begin by establishing a clear definition of the LLM supply chain, encompassing its components and dependencies. We then analyze each layer of the supply chain, presenting a vision for robust and secure LLM development, reviewing the current state of practices and technologies, and identifying key challenges and research opportunities. This work aims to bridge the existing research gap in systematically understanding the multifaceted issues within the LLM supply chain, offering valuable insights to guide future efforts in this rapidly evolving domain.<\/jats:p>","DOI":"10.1145\/3708531","type":"journal-article","created":{"date-parts":[[2024,12,13]],"date-time":"2024-12-13T11:12:12Z","timestamp":1734088332000},"page":"1-46","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":35,"title":["Large Language Model Supply Chain: A Research Agenda"],"prefix":"10.1145","volume":"34","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3818-3343","authenticated-orcid":false,"given":"Shenao","family":"Wang","sequence":"first","affiliation":[{"name":"Hubei Key Laboratory of Distributed System Security, Hubei Engineering Research Center on Big Data Security, School of Cyber Science and Engineering, Huazhong University of Science and Technology, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8793-5367","authenticated-orcid":false,"given":"Yanjie","family":"Zhao","sequence":"additional","affiliation":[{"name":"Hubei Key Laboratory of Distributed System Security, Hubei Engineering Research Center on Big Data Security, School of Cyber Science and Engineering, Huazhong University of Science and Technology, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-9965-2109","authenticated-orcid":false,"given":"Xinyi","family":"Hou","sequence":"additional","affiliation":[{"name":"Hubei Key Laboratory of Distributed System Security, Hubei Engineering Research Center on Big Data Security, School of Cyber Science and Engineering, Huazhong University of Science and Technology, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1100-8633","authenticated-orcid":false,"given":"Haoyu","family":"Wang","sequence":"additional","affiliation":[{"name":"Hubei Key Laboratory of Distributed System Security, Hubei Engineering Research Center on Big Data Security, School of Cyber Science and Engineering, Huazhong University of Science and Technology, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2025,5,24]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"Sara Abdali Richard Anarfi C. J. Barberan and Jia He. 2024. Securing large language models: Threats vulnerabilities and responsible practices. arXiv:2403.12503. Retrieved from https:\/\/arxiv.org\/abs\/2403.12503"},{"key":"e_1_3_2_3_2","first-page":"79","volume-title":"Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec \u201923)","author":"Abdelnabi Sahar","year":"2023","unstructured":"Sahar Abdelnabi, Kai Greshake, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. Not what you\u2019ve signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security (AISec \u201923). Maura Pintor, Xinyun Chen, and Florian Tram\u00e8r (Eds.), ACM, New York, NY, 79\u201390. DOI: 10.1145\/3605764.3623985"},{"key":"e_1_3_2_4_2","unstructured":"Adrian Wood. 2023. Model confusion \u2013 Weaponizing ML models for red teams and bounty hunters. Retrieved September 26 2024 from https:\/\/5stars217.github.io\/2023-08-08-red-teaming-with-ml-models\/"},{"key":"e_1_3_2_5_2","unstructured":"Vibhor Agarwal Yulong Pei Salwa Alamir and Xiaomo Liu. 2024. CodeMirage: Hallucinations in code generated by large language models. arXiv:2408.08333. Retrieved from https:\/\/arxiv.org\/abs\/2408.08333"},{"key":"e_1_3_2_6_2","doi-asserted-by":"crossref","first-page":"296","DOI":"10.1109\/SP.2007.36","volume-title":"Proceedings of the IEEE Symposium on Security and Privacy (S&P 2007)","author":"Agrawal Dakshi","year":"2007","unstructured":"Dakshi Agrawal, Sel\u00e7uk Baktir, Deniz Karakoyunlu, Pankaj Rohatgi, and Berk Sunar. 2007. Trojan detection using IC fingerprinting. In Proceedings of the IEEE Symposium on Security and Privacy (S&P 2007). IEEE Computer Society, 296\u2013310. DOI: 10.1109\/SP.2007.36"},{"key":"e_1_3_2_7_2","unstructured":"Arcee AI. 2024. Mergekit. Retrieved November 11 2024 from https:\/\/github.com\/arcee-ai\/mergekit"},{"key":"e_1_3_2_8_2","doi-asserted-by":"crossref","unstructured":"Adekunle Ajibode Abdul Ali Bangash Filipe Roseiro C\u00f4go Bram Adams and Ahmed E. Hassan. 2024. Towards semantic versioning of open pre-trained language model releases on hugging face. arXiv:2409.10472. Retrieved from https:\/\/arxiv.org\/abs\/2409.10472","DOI":"10.1007\/s10664-025-10631-3"},{"key":"e_1_3_2_9_2","first-page":"1","article-title":"A survey on data selection for language models","volume":"2024","author":"Albalak Alon","year":"2024","unstructured":"Alon Albalak, Yanai Elazar, Sang Michael Xie, Shayne Longpre, Nathan Lambert, Xinyi Wang, Niklas Muennighoff, Bairu Hou, Liangming Pan, Haewon Jeong, et al. 2024. A survey on data selection for language models. Transactions on Machine Learning Research 2024 (2024), 1\u201381. Retrieved from https:\/\/openreview.net\/forum?id=XfHWcNTSHp","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_3_2_10_2","unstructured":"Alien and Nicky. 2023. Beware of hugging face open-source component risks exploited in large language model supply chain attacks. Retrieved September 22 2024 from https:\/\/security.tencent.com\/index.php\/blog\/msg\/209. Accessed: 2024-09-22."},{"key":"e_1_3_2_11_2","first-page":"7","volume-title":"Proceedings of the ACM\/IEEE 2nd International Workshop on Interpretability, Robustness, and Benchmarking in Neural Software Engineering (InteNSE \u201924)","author":"Alshahwan Nadia","year":"2024","unstructured":"Nadia Alshahwan, Mark Harman, Inna Harper, Alexandru Marginean, Shubho Sengupta, and Eddy Wang. 2024. Assured offline LLM-based software engineering. In Proceedings of the ACM\/IEEE 2nd International Workshop on Interpretability, Robustness, and Benchmarking in Neural Software Engineering (InteNSE \u201924). ACM, New York, NY, 7\u201312. DOI: 10.1145\/3643661.3643953"},{"key":"e_1_3_2_12_2","unstructured":"Eleni Angelou. 2022. Three scenarios of Pseudo-Alignment. Retrieved March 28 2024 from https:\/\/www.lesswrong.com\/posts\/W5nnfgWkCPxDvJMpe\/three-scenarios-of-pseudo-alignment"},{"key":"e_1_3_2_13_2","unstructured":"Rohan Anil Sebastian Borgeaud Yonghui Wu Jean-Baptiste Alayrac Jiahui Yu Radu Soricut Johan Schalkwyk Andrew M. Dai Anja Hauth Katie Millican et al. 2023. Gemini: A family of highly capable multimodal models. arXiv:2312.11805. Retrieved from https:\/\/arxiv.org\/abs\/2312.11805"},{"key":"e_1_3_2_14_2","unstructured":"Avi Lumelsky. 2024. TensorFlow keras downgrade attack: CVE-2024-3660 bypass. Retrieved September 23 2024 from https:\/\/www.oligo.security\/blog\/tensorflow-keras-downgrade-attack-cve-2024-3660-bypass"},{"key":"e_1_3_2_15_2","first-page":"2245","volume-title":"Proceedings of the International Conference on Findings of the Association for Computational Linguistics (ACL \u201922)","author":"Baldini Ioana","year":"2022","unstructured":"Ioana Baldini, Dennis Wei, Karthikeyan Natesan Ramamurthy, Moninder Singh, and Mikhail Yurochkin. 2022. Your fairness may vary: Pretrained language model fairness in toxic text classification. In Proceedings of the International Conference on Findings of the Association for Computational Linguistics (ACL \u201922). Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.), ACL, 2245\u20132262. DOI: 10.18653\/V1\/2022.FINDINGS-ACL.176"},{"key":"e_1_3_2_16_2","first-page":"67","volume-title":"Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL \u201924)","author":"Balloccu Simone","year":"2024","unstructured":"Simone Balloccu, Patr\u00edcia Schmidtov\u00e1, Mateusz Lango, and Ondrej Dusek. 2024. Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source LLMs. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL \u201924). Yvette Graham and Matthew Purver (Eds.), ACL, 67\u201393. Retrieved from https:\/\/aclanthology.org\/2024.eacl-long.5"},{"key":"e_1_3_2_17_2","unstructured":"Bar Lanyado. 2023. More Than 1500 HuggingFace API Tokens Were Exposed Leaving Millions of Meta-Llama Bloom and Pythia Users Vulnerable. Retrieved September 26 2024 from https:\/\/www.lasso.security\/blog\/1500-huggingface-api-tokens-were-exposed-leaving-millions-of-meta-llama-bloom-and-pythia-users-for-supply-chain-attacks"},{"key":"e_1_3_2_18_2","first-page":"581","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: Industry Track (ACL \u201923)Association for Computational Linguistics","author":"Bespalov Dmitriy","year":"2023","unstructured":"Dmitriy Bespalov, Sourav Bhabesh, Yi Xiang, Liutong Zhou, and Yanjun Qi. 2023. Towards building a robust toxicity predictor. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics: Industry Track (ACL \u201923). Sunayana Sitaram, Beata Beigman Klebanov, and Jason D. Williams (Eds.), Association for Computational Linguistics, 581\u2013598. DOI: 10.18653\/V1\/2023.ACL-INDUSTRY.56"},{"key":"e_1_3_2_19_2","unstructured":"Manish Bhatt Sahana Chennabasappa Cyrus Nikolaidis Shengye Wan Ivan Evtimov Dominik Gabi Daniel Song Faizan Ahmad Cornelius Aschermann Lorenzo Fontana et al. 2023. Purple llama CyberSecEval: A secure coding benchmark for language models. arXiv:2312.04724. Retrieved from https:\/\/arxiv.org\/abs\/2312.04724"},{"key":"e_1_3_2_20_2","first-page":"12574","volume-title":"Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI \u201921), 33rd Conference on Innovative Applications of Artificial Intelligence (IAAI \u201921), The 11th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201921)","author":"Bian Ning","year":"2021","unstructured":"Ning Bian, Xianpei Han, Bo Chen, and Le Sun. 2021. Benchmarking knowledge-enhanced commonsense question answering via knowledge-to-text transformation. In Proceedings of the 35th AAAI Conference on Artificial Intelligence (AAAI \u201921), 33rd Conference on Innovative Applications of Artificial Intelligence (IAAI \u201921), The 11th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201921). AAAI Press, 12574\u201312582. DOI: 10.1609\/AAAI.V35I14.17490"},{"key":"e_1_3_2_21_2","unstructured":"Bill Doerrfeld. 2024. Will the Rise of Generative AI Increase Technical Debt? Retrieved September 20 2024 from https:\/\/devops.com\/will-the-rise-of-generative-ai-increase-technical-debt\/"},{"key":"e_1_3_2_22_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Birhane Abeba","year":"2023","unstructured":"Abeba Birhane, Vinay Uday Prabhu, Sanghyun Han, Vishnu Boddeti, and Sasha Luccioni. 2023. Into the LAION\u2019s Den: Investigating hate in multimodal datasets. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/42f225509e8263e2043c9d834ccd9a2b-Abstract-Datasets_and_Benchmarks.html"},{"key":"e_1_3_2_23_2","unstructured":"Rishi Bommasani Drew A. Hudson Ehsan Adeli Russ B. Altman Simran Arora Sydney von Arx Michael S. Bernstein Jeannette Bohg Antoine Bosselut Emma Brunskill et al. 2021. On the opportunities and risks of foundation models. arXiv:2108.07258. Retrieved from https:\/\/arxiv.org\/abs\/2108.07258"},{"key":"e_1_3_2_24_2","unstructured":"Boyan Milanov. 2024. Exploiting ML Models with Pickle File Attacks: Part 1. Retrieved September 23 2024 from https:\/\/blog.trailofbits.com\/2024\/06\/11\/exploiting-ml-models-with-pickle-file-attacks-part-1\/"},{"key":"e_1_3_2_25_2","unstructured":"Boyan Milanov. 2024. Exploiting ML Models with Pickle File Attacks: Part 2. Retrieved September 23 2024 from https:\/\/blog.trailofbits.com\/2024\/06\/11\/exploiting-ml-models-with-pickle-file-attacks-part-2\/"},{"key":"e_1_3_2_26_2","volume-title":"Proceedings of the 2nd Conference on Machine Learning and Systems (SysML \u201919)","author":"Breck Eric","year":"2019","unstructured":"Eric Breck, Neoklis Polyzotis, Sudip Roy, Steven Whang, and Martin Zinkevich. 2019. Data validation for machine learning. In Proceedings of the 2nd Conference on Machine Learning and Systems (SysML \u201919). Ameet Talwalkar, Virginia Smith, and Matei Zaharia (Eds.), mlsys.org. Retrieved from https:\/\/proceedings.mlsys.org\/paper_files\/paper\/2019\/hash\/928f1160e52192e3e0017fb63ab65391-Abstract.html"},{"key":"e_1_3_2_27_2","first-page":"1970","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL \u201922)","volume":"1","author":"Broscheit Samuel","year":"2022","unstructured":"Samuel Broscheit, Quynh Do, and Judith Gaspers. 2022. Distributionally robust finetuning BERT for covariate drift in spoken language understanding. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL \u201922). Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.), Vol. 1: Long Papers, Association for Computational Linguistics, 1970\u20131985. DOI: 10.18653\/V1\/2022.ACL-LONG.139"},{"key":"e_1_3_2_28_2","doi-asserted-by":"crossref","first-page":"2280","DOI":"10.1145\/3531146.3534642","volume-title":"Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT \u201922)","author":"Brown Hannah","year":"2022","unstructured":"Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, and Florian Tram\u00e8r. 2022. What does it Mean for a language model to preserve privacy? In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT \u201922). ACM, New York, NY, 2280\u20132292. DOI: 10.1145\/3531146.3534642"},{"key":"e_1_3_2_29_2","unstructured":"Jialun Cao Wuqi Zhang and Shing-Chi Cheung. 2024. Concerned with data contamination? Assessing countermeasures in code language model. arXiv:2403.16898. Retrieved from https:\/\/arxiv.org\/abs\/2403.16898"},{"key":"e_1_3_2_30_2","first-page":"5253","volume-title":"Proceedings of the 32nd USENIX Security Symposium, USENIX SecurityUSENIX Association","author":"Carlini Nicholas","year":"2023","unstructured":"Nicholas Carlini, Jamie Hayes, Milad Nasr, Matthew Jagielski, Vikash Sehwag, Florian Tram\u00e8r, Borja Balle, Daphne Ippolito, and Eric Wallace. 2023. Extracting training data from diffusion models. In Proceedings of the 32nd USENIX Security Symposium, USENIX Security. Joseph A. Calandrino and Carmela Troncoso (Eds.), USENIX Association, 5253\u20135270. Retrieved from https:\/\/www.usenix.org\/conference\/usenixsecurity23\/presentation\/carlini"},{"key":"e_1_3_2_31_2","first-page":"2633","volume-title":"Proceedings of the 30th USENIX Security Symposium, USENIX SecurityUSENIX Association","author":"Carlini Nicholas","year":"2021","unstructured":"Nicholas Carlini, Florian Tram\u00e8r, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom B. Brown, Dawn Song, \u00dalfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. Extracting training data from large language models. In Proceedings of the 30th USENIX Security Symposium, USENIX Security. Michael D. Bailey and Rachel Greenstadt (Eds.), USENIX Association, 2633\u20132650. Retrieved from https:\/\/www.usenix.org\/conference\/usenixsecurity21\/presentation\/carlini-extracting"},{"key":"e_1_3_2_32_2","unstructured":"CERT Vulnerability Notes Database. 2024. Keras 2 Lambda Layers Allow Arbitrary Code Injection in TensorFlow Models. Retrieved September 23 2024 from https:\/\/kb.cert.org\/vuls\/id\/253266"},{"key":"e_1_3_2_33_2","doi-asserted-by":"crossref","first-page":"396","DOI":"10.1007\/978-3-642-04138-9_28","volume-title":"Cryptographic Hardware and Embedded Systems - CHES 2009, Proceedings of the 11th International Workshop on Cryptographic Hardware and Embedded Systems (CHES \u201909)","volume":"5747","author":"Chakraborty Rajat Subhra","year":"2009","unstructured":"Rajat Subhra Chakraborty, Francis G. Wolff, Somnath Paul, Christos A. Papachristou, and Swarup Bhunia. 2009. MERO: A statistical approach for hardware trojan detection. In Cryptographic Hardware and Embedded Systems - CHES 2009, Proceedings of the 11th International Workshop on Cryptographic Hardware and Embedded Systems (CHES \u201909). Christophe Clavier and Kris Gaj (Eds.), Lecture Notes in Computer Science, Vol. 5747, Springer, 396\u2013410. DOI: 10.1007\/978-3-642-04138-9_28"},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1145\/3641289"},{"key":"e_1_3_2_35_2","unstructured":"Patrick Chao Edoardo Debenedetti Alexander Robey Maksym Andriushchenko Francesco Croce Vikash Sehwag Edgar Dobriban Nicolas Flammarion George J. Pappas Florian Tram\u00e8r et al. 2024. JailbreakBench: An open robustness benchmark for jailbreaking large language models. arXiv:2404.01318. Retrieved from https:\/\/arxiv.org\/abs\/2404.01318"},{"key":"e_1_3_2_36_2","unstructured":"Charlie Hull. 2023. Is llama 2 Open Source? No \u2013 and Perhaps We Need a New Definition of Open. Retrieved September 25 2024 from https:\/\/opensourceconnections.com\/blog\/2023\/07\/19\/is-llama-2-open-source-no-and-perhaps-we-need-a-new-definition-of-open\/"},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","unstructured":"Jayeeta Chaudhuri Dhruv Thapar Arjun Chaudhuri Farshad Firouzi and Krishnendu Chakrabarty. 2024. SPICED: Syntactical bug and trojan pattern identification in a\/MS circuits using LLM-enhanced detection. arXiv:2408.16018. Retrieved from https:\/\/arxiv.org\/abs\/2408.16018","DOI":"10.1109\/PAINE62042.2024.10792717"},{"key":"e_1_3_2_38_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Chen Shiqi","year":"2023","unstructured":"Shiqi Chen, Yiran Zhao, Jinghan Zhang, I-Chun Chern, Siyang Gao, Pengfei Liu, and Junxian He. 2023. FELM: Benchmarking factuality evaluation of large language models. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/8b8a7960d343e023a6a0afe37eee6022-Abstract-Datasets_and_Benchmarks.html"},{"key":"e_1_3_2_39_2","first-page":"364","volume-title":"Proceedings of the 5th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT)","author":"Chen Tianyi","year":"2024","unstructured":"Tianyi Chen. 2024. Challenges and opportunities in integrating LLMs Into continuous integration\/continuous deployment (CI\/CD) pipelines. In Proceedings of the 5th International Seminar on Artificial Intelligence, Networking and Information Technology (AINIT), 364\u2013367. DOI: 10.1109\/AINIT61980.2024.10581784"},{"key":"e_1_3_2_40_2","unstructured":"Junjie Chu Yugeng Liu Ziqing Yang Xinyue Shen Michael Backes and Yang Zhang. 2024. Comprehensive assessment of jailbreak attacks against LLMs. arXiv:2402.05668."},{"key":"e_1_3_2_41_2","first-page":"17871","volume-title":"Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914)","author":"Chu Timothy","year":"2024","unstructured":"Timothy Chu, Zhao Song, and Chiwun Yang. 2024. How to protect copyright data in optimization of large language models? In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914). Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.), AAAI Press, 17871\u201317879. DOI: 10.1609\/AAAI.V38I16.29741"},{"key":"e_1_3_2_42_2","unstructured":"Cleanlab. 2024. Cleanlab. Retrieved November 11 2024 from https:\/\/github.com\/cleanlab\/cleanlab"},{"key":"e_1_3_2_43_2","unstructured":"Codacy. 2024. Shift Left Security: A Complete Guide. Retrieved September 28 2024 from https:\/\/blog.codacy.com\/what-is-shift-left-security"},{"key":"e_1_3_2_44_2","unstructured":"Tianyu Cui Yanling Wang Chuanpu Fu Yong Xiao Sijia Li Xinhao Deng Yunpeng Liu Qinglin Zhang Ziyi Qiu Peiyang Li et al. 2024. Risk taxonomy mitigation and assessment benchmarks of large language model systems. arXiv:2401.05778. Retrieved from https:\/\/arxiv.org\/abs\/2401.05778"},{"key":"e_1_3_2_45_2","unstructured":"Daniel Huynh Jade Hardouin. 2023. PoisonGPT: How We Hid a Lobotomized LLM on Hugging Face to Spread Fake News. Retrieved September 25 2024 from https:\/\/blog.mithrilsecurity.io\/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news\/"},{"key":"e_1_3_2_46_2","unstructured":"David Mor-Ofek. 2024. It\u2019s Time to Talk about AI\/ML BOM (Artificial Intelligence Bill of Materials) And Vulnerability Management. Retrieved September 26 2024 from https:\/\/c2a-sec.com\/its-time-to-talk-about-ai-ml-bom-artificial-intelligence-bill-of-materials-and-vulnerability-management\/"},{"key":"e_1_3_2_47_2","first-page":"17","volume-title":"Proceedings of the IEEE John Vincent Atanasoff International Symposium on Modern Computing (JVA \u201923)","author":"Davis James C.","year":"2023","unstructured":"James C. Davis, Purvish Jajal, Wenxin Jiang, Taylor R. Schorlemmer, Nicholas Synovic, and George K. Thiruvathukal. 2023. Reusing deep learning models: Challenges and directions in software engineering. In Proceedings of the IEEE John Vincent Atanasoff International Symposium on Modern Computing (JVA \u201923). Shaukat Ali, Claudio A. Ardagna, Nimanthi L. Atukorala, Johanna Barzen, Carl K. Chang, Rong Chang, Jing Fan, Ismael Faro, Sebastian Feld, Geoffrey Fox, Zhi Jin, Frank Leymann, Florian Neukart, Salvador de la Puente, and Manuel Wimmer (Eds.), IEEE, 17\u201330. DOI: 10.1109\/JVA60410.2023.00015"},{"key":"e_1_3_2_48_2","unstructured":"Jasper Dekoninck Mark Niklas M\u00fcller Maximilian Baader Marc Fischer and Martin T. Vechev. 2024. Evading data contamination detection for language models is (too) Easy. arXiv:2402.02823. Retrieved from https:\/\/arxiv.org\/abs\/2402.02823"},{"key":"e_1_3_2_49_2","volume-title":"Proceedings of the Workshop on Backdoors in Deep Learning - The Good, the Bad, and the Ugly","author":"Deng Chunyuan","year":"2024","unstructured":"Chunyuan Deng, Yilun Zhao, Xiangru Tang, Mark Gerstein, and Arman Cohan. 2024. Benchmark probing: Investigating data leakage in large language models. In Proceedings of the Workshop on Backdoors in Deep Learning - The Good, the Bad, and the Ugly. Retrieved from https:\/\/openreview.net\/forum?id=a34bgvner1"},{"key":"e_1_3_2_50_2","volume-title":"Proceedings of the 31st Annual Network and Distributed System Security Symposium (NDSS \u201924)","author":"Deng Gelei","year":"2024","unstructured":"Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. 2024. MASTERKEY: Automated jailbreaking of large language model chatbots. In Proceedings of the 31st Annual Network and Distributed System Security Symposium (NDSS \u201924). The Internet Society. Retrieved from https:\/\/www.ndss-symposium.org\/ndss-paper\/masterkey-automated-jailbreaking-of-large-language-model-chatbots\/"},{"key":"e_1_3_2_51_2","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1145\/3643788.3648014","volume-title":"Proceedings of the 5th ACM\/IEEE International Workshop on Automated Program Repair (APR \u201924)","author":"Diaz-De-Arcaya Josu","year":"2024","unstructured":"Josu Diaz-De-Arcaya, Juan L\u00f3pez-De-Armentia, Gorka Z\u00e1rate, and Ana I. Torre-Bastida. 2024. Towards the self-healing of infrastructure as code projects using constrained LLM technologies. In Proceedings of the 5th ACM\/IEEE International Workshop on Automated Program Repair (APR \u201924). ACM, New York, NY, 22\u201325. DOI: 10.1145\/3643788.3648014"},{"key":"e_1_3_2_52_2","unstructured":"Guoliang Dong Haoyu Wang Jun Sun and Xinyu Wang. 2024b. Evaluating and mitigating linguistic discrimination in large language models. arXiv:2404.18534. Retrieved from https:\/\/arxiv.org\/abs\/2404.18534"},{"key":"e_1_3_2_53_2","unstructured":"Yihong Dong Jiazheng Ding Xue Jiang Zhuo Li Ge Li and Zhi Jin. 2023. CodeScore: Evaluating code generation by learning code execution. arXiv:2301.09043. Retrieved from https:\/\/arxiv.org\/abs\/2301.09043"},{"key":"e_1_3_2_54_2","doi-asserted-by":"crossref","first-page":"12039","DOI":"10.18653\/v1\/2024.findings-acl.716","volume-title":"Proceedings of the Findings of the Association for Computational Linguistics (ACL \u201924)","author":"Dong Yihong","year":"2024","unstructured":"Yihong Dong, Xue Jiang, Huanyu Liu, Zhi Jin, Bin Gu, Mengfei Yang, and Ge Li. 2024a. Generalization or memorization: Data contamination and trustworthy evaluation for large language models. In Proceedings of the Findings of the Association for Computational Linguistics (ACL \u201924). Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.), Association for Computational Linguistics, 12039\u201312050. DOI: 10.18653\/V1\/2024.FINDINGS-ACL.716"},{"key":"e_1_3_2_55_2","unstructured":"Dorian Maillard. 2024. Nvidia\u2019s AI Market Dominance: Can Anyone Mount a Serious Challenge? Retrieved September 20 2024 from https:\/\/www.techradar.com\/pro\/nvidias-ai-market-dominance-can-anyone-mount-a-serious-challenge"},{"key":"e_1_3_2_56_2","first-page":"81:1","volume-title":"Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924)","author":"Du Xueying","year":"2024","unstructured":"Xueying Du, Mingwei Liu, Kaixin Wang, Hanlin Wang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. 2024. Evaluating large language models in class-level code generation. In Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924). ACM, New York, NY, 81:1\u201381:13. DOI: 10.1145\/3597503.3639219"},{"key":"e_1_3_2_57_2","volume-title":"Proceedings of the 28th Annual Network and Distributed System Security Symposium (NDSS \u201921)","author":"Duan Ruian","year":"2021","unstructured":"Ruian Duan, Omar Alrawi, Ranjita Pai Kasturi, Ryan Elder, Brendan Saltaformaggio, and Wenke Lee. 2021. Towards measuring supply chain attacks on package managers for interpreted languages. In Proceedings of the 28th Annual Network and Distributed System Security Symposium (NDSS \u201921). The Internet Society. Retrieved from https:\/\/www.ndss-symposium.org\/ndss-paper\/towards-measuring-supply-chain-attacks-on-package-managers-for-interpreted-languages\/"},{"key":"e_1_3_2_58_2","unstructured":"Eclypsium. 2024. Securing Supply Chains for GenAI Hardware and Models. Retrieved September 20 2024 from https:\/\/eclypsium.com\/blog\/securing-supply-chains-for-genai-models-and-hardware\/"},{"key":"e_1_3_2_59_2","doi-asserted-by":"crossref","first-page":"10476","DOI":"10.18653\/v1\/2023.findings-acl.666","volume-title":"Proceedings of the Findings of the Association for Computational Linguistics (ACL \u201923)Association for Computational Linguistics","author":"Efrat Avia","year":"2023","unstructured":"Avia Efrat, Or Honovich, and Omer Levy. 2023. LMentry: A language model benchmark of elementary language tasks. In Proceedings of the Findings of the Association for Computational Linguistics (ACL \u201923). Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki (Eds.), Association for Computational Linguistics, 10476\u201310501. DOI: 10.18653\/V1\/2023.FINDINGS-ACL.666"},{"key":"e_1_3_2_60_2","unstructured":"Eoin Wickens Kasimir Schulz. 2024. Hijacking Safetensors Conversion on Hugging Face. Retrieved September 25 2024 from https:\/\/hiddenlayer.com\/research\/silent-sabotage\/"},{"key":"e_1_3_2_61_2","unstructured":"Hugging Face. 2024. Hugging Face. Retrieved March 28 2024 from https:\/\/huggingface.co\/"},{"key":"e_1_3_2_62_2","unstructured":"facebookresearch. 2024. Faiss. Retrieved September 28 2024 from https:\/\/github.com\/facebookresearch\/faiss"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.5210\/FM.V28I11.13346"},{"key":"e_1_3_2_64_2","unstructured":"Center for Research on Foundation Models (CRFM). 2024. The Foundation Model Transparency Index. Retrieved March 28 2024 from https:\/\/crfm.stanford.edu\/fmti\/"},{"key":"e_1_3_2_65_2","first-page":"1276","volume-title":"Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD \u201913)","author":"Fu Bin","year":"2013","unstructured":"Bin Fu, Jialiu Lin, Lei Li, Christos Faloutsos, Jason I. Hong, and Norman M. Sadeh. 2013. Why people hate your app: Making sense of user feedback in a Mobile app store. In Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD \u201913). Inderjit S. Dhillon, Yehuda Koren, Rayid Ghani, Ted E. Senator, Paul Bradley, Rajesh Parekh, Jingrui He, Robert L. Grossman, and Ramasamy Uthurusamy (Eds.), ACM, New York, NY, 1276\u20131284. DOI: 10.1145\/2487575.2488202"},{"key":"e_1_3_2_66_2","unstructured":"Xiaohan Fu Zihan Wang Shuheng Li Rajesh K. Gupta Niloofar Mireshghallah Taylor Berg-Kirkpatrick and Earlence Fernandes. 2023. Misusing Tools in Large Language Models with Visual Adversarial Examples. arXiv:2310.03185. Retrieved from https:\/\/arxiv.org\/abs\/2310.03185"},{"key":"e_1_3_2_67_2","unstructured":"Isabel O. Gallegos Ryan A. Rossi Joe Barrow Md. Mehrab Tanjim Sungchul Kim Franck Dernoncourt Tong Yu Ruiyi Zhang and Nesreen K. Ahmed. 2023. Bias and fairness in large language models: A survey. arXiv:2309.00770. Retrieved from https:\/\/arxiv.org\/abs\/2309.00770"},{"key":"e_1_3_2_68_2","doi-asserted-by":"publisher","DOI":"10.13328\/j.cnki.jos.006975"},{"key":"e_1_3_2_69_2","first-page":"16477","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL \u201923)","author":"Gao Luyu","year":"2023","unstructured":"Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Y. Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, et al. 2023. RARR: Researching and revising what language models say, using language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL \u201923). Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki (Eds.), Association for Computational Linguistics, 16477\u201316508. DOI: 10.18653\/V1\/2023.ACL-LONG.910"},{"key":"e_1_3_2_70_2","first-page":"1927","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (NAACL \u201924)Association for Computational Linguistics","author":"Ge Suyu","year":"2024","unstructured":"Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao. 2024. MART: Improving LLM safety with multi-round automatic red-teaming. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (NAACL \u201924). Kevin Duh, Helena G\u00f3mez-Adorno, and Steven Bethard (Eds.), Association for Computational Linguistics, 1927\u20131937. DOI: 10.18653\/V1\/2024.NAACL-LONG.107"},{"key":"e_1_3_2_71_2","doi-asserted-by":"crossref","first-page":"3356","DOI":"10.18653\/v1\/2020.findings-emnlp.301","volume-title":"Proceedings of the Findings of the Association for Computational Linguistics (EMNLP \u201920)Association for Computational Linguistics","author":"Gehman Samuel","year":"2020","unstructured":"Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxicityPrompts: Evaluating neural toxic degeneration in language models. In Proceedings of the Findings of the Association for Computational Linguistics (EMNLP \u201920) Trevor Cohn, Yulan He, and Yang Liu (Eds.), Association for Computational Linguistics, 3356\u20133369. DOI: 10.18653\/V1\/2020.FINDINGS-EMNLP.301"},{"key":"e_1_3_2_72_2","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1145\/1858996.1859088","volume-title":"Proceedings of the 25th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201910)","author":"Germ\u00e1n Daniel M.","year":"2010","unstructured":"Daniel M. Germ\u00e1n, Yuki Manabe, and Katsuro Inoue. 2010. A sentence-matching method for automatic license identification of source code files. In Proceedings of the 25th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201910). Charles Pecheur, Jamie Andrews, and Elisabetta Di Nitto (Eds.), ACM, New York, NY, 437\u2013446. DOI: 10.1145\/1858996.1859088"},{"key":"e_1_3_2_73_2","unstructured":"Ggerganov. 2024. GGML. Retrieved September 28 2024 from https:\/\/github.com\/ggerganov\/ggml"},{"key":"e_1_3_2_74_2","first-page":"1275","volume-title":"Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS \u201922)","author":"Gohil Vasudev","year":"2022","unstructured":"Vasudev Gohil, Hao Guo, Satwik Patnaik, and Jeyavijayan Rajendran. 2022. ATTRITION: Attacking static hardware trojan detection techniques using reinforcement learning. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS \u201922). Heng Yin, Angelos Stavrou, Cas Cremers, and Elaine Shi (Eds.), ACM, New York, NY, 1275\u20131289. DOI: 10.1145\/3548606.3560690"},{"key":"e_1_3_2_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2023.3309731"},{"key":"e_1_3_2_76_2","unstructured":"Shahriar Golchin and Mihai Surdeanu. 2023. Data contamination quiz: A Tool to detect and estimate contamination in large language models. arXiv:2311.06233. Retrieved from https:\/\/arxiv.org\/abs\/2311.06233"},{"key":"e_1_3_2_77_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations (ICLR \u201924)","author":"Golchin Shahriar","year":"2024","unstructured":"Shahriar Golchin and Mihai Surdeanu. 2024. Time travel in LLMs: Tracing data contamination in large language models. In Proceedings of the 12th International Conference on Learning Representations (ICLR \u201924). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=2Rwq6c3tvr"},{"key":"e_1_3_2_78_2","first-page":"18099","volume-title":"Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914)","author":"Gu Zhouhong","year":"2024","unstructured":"Zhouhong Gu, Xiaoxuan Zhu, Haoning Ye, Lin Zhang, Jianchen Wang, Yixin Zhu, Sihang Jiang, Zhuozhi Xiong, Zihan Li, Weijie Wu, et al. 2024. Xiezhi: An ever-updating benchmark for holistic domain knowledge evaluation. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914). Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.), AAAI Press, 18099\u201318107. DOI: 10.1609\/AAAI.V38I16.29767"},{"key":"e_1_3_2_79_2","first-page":"18135","volume-title":"Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914)","author":"Gunjal Anisha","year":"2024","unstructured":"Anisha Gunjal, Jihan Yin, and Erhan Bas. 2024. Detecting and preventing hallucinations in large vision language models. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914). Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.), AAAI Press, 18135\u201318143. DOI: 10.1609\/AAAI.V38I16.29771"},{"key":"e_1_3_2_80_2","first-page":"8048","volume-title":"Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI \u201924)","author":"Guo Taicheng","year":"2024","unstructured":"Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. 2024. Large language model based multi-agents: A survey of progress and challenges. In Proceedings of the 33rd International Joint Conference on Artificial Intelligence (IJCAI \u201924). ijcai.org, 8048\u20138057. Retrieved from https:\/\/www.ijcai.org\/proceedings\/2024\/890"},{"key":"e_1_3_2_81_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Guo Taicheng","year":"2023","unstructured":"Taicheng Guo, Kehan Guo, Bozhao Nan, Zhenwen Liang, Zhichun Guo, Nitesh V. Chawla, Olaf Wiest, and Xiangliang Zhang. 2023a. What can large language models do in chemistry? A comprehensive benchmark on eight tasks. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/bbb330189ce02be00cf7346167028ab1-Abstract-Datasets_and_Benchmarks.html"},{"key":"e_1_3_2_82_2","first-page":"166","volume-title":"Proceedings of the 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201923)","author":"Guo Wenbo","year":"2023","unstructured":"Wenbo Guo, Zhengzi Xu, Chengwei Liu, Cheng Huang, Yong Fang, and Yang Liu. 2023. An empirical study of malicious code In PyPI ecosystem. In Proceedings of the 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201923). IEEE, 166\u2013177. DOI: 10.1109\/ASE56229.2023.00135"},{"key":"e_1_3_2_83_2","unstructured":"Hasan Abed Al Kader Hammoud Umberto Michieli Fabio Pizzati Philip Torr Adel Bibi Bernard Ghanem and Mete Ozay. 2024. Model merging and safety alignment: One bad model spoils the bunch. arXiv:2406.14563. Retrieved from https:\/\/arxiv.org\/abs\/2406.14563"},{"key":"e_1_3_2_84_2","unstructured":"Yikun Han Chunjiang Liu and Pengfei Wang. 2023. A comprehensive survey on vector database: Storage and retrieval technique challenge. arXiv:2310.11703. Retrieved from https:\/\/arxiv.org\/abs\/2310.11703"},{"key":"e_1_3_2_85_2","doi-asserted-by":"publisher","DOI":"10.1109\/COMST.2019.2944748"},{"key":"e_1_3_2_86_2","unstructured":"Julian Hazell. 2023. Spear phishing with large language models. arXiv:2305.06972. Retrieved from https:\/\/arxiv.org\/abs\/2305.06972"},{"key":"e_1_3_2_87_2","unstructured":"Feng He Tianqing Zhu Dayong Ye Bo Liu Wanlei Zhou and Philip S. Yu. 2024. The emerged security and privacy of LLM agent: A survey with case studies. arXiv:2407.19354. Retrieved from https:\/\/arxiv.org\/abs\/2407.19354"},{"key":"e_1_3_2_88_2","first-page":"10218","volume-title":"Proceedings of the International Conference on Findings of the Association for Computational Linguistics (ACL \u201924)","author":"He Jinwen","year":"2024","unstructured":"Jinwen He, Yujia Gong, Zijin Lin, Cheng\u2019an Wei, Yue Zhao, and Kai Chen. 2024. LLM factoscope: Uncovering LLMs\u2019 factual discernment through measuring inner States. In Proceedings of the International Conference on Findings of the Association for Computational Linguistics (ACL \u201924). Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, 10218\u201310230. DOI: 10.18653\/v1\/2024.findings-acl.608"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1109\/TVLSI.2017.2727985"},{"key":"e_1_3_2_90_2","doi-asserted-by":"publisher","DOI":"10.19363\/J.cnki.cn10-1380\/tn.2020.01.06"},{"key":"e_1_3_2_91_2","unstructured":"Yifeng He Ethan Wang Yuyang Rong Zifei Cheng and Hao Chen. 2024. Security of AI agents. arXiv:2406.08689. Retrieved from https:\/\/arxiv.org\/abs\/2406.08689"},{"key":"e_1_3_2_92_2","first-page":"101","volume-title":"Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results, ICSE (NIER \u201918)","author":"Hejderup Joseph","year":"2018","unstructured":"Joseph Hejderup, Arie van Deursen, and Georgios Gousios. 2018. Software ecosystem call graph for dependency management. In Proceedings of the 40th International Conference on Software Engineering: New Ideas and Emerging Results, ICSE (NIER \u201918). Andrea Zisman and Sven Apel (Eds.), ACM, New York, NY, 101\u2013104. DOI: 10.1145\/3183399.3183417"},{"key":"e_1_3_2_93_2","unstructured":"Xinyi Hou Yanjie Zhao and Haoyu Wang. 2024. On the (In)Security of LLM App stores. arXiv:2407.08422. Retrieved from https:\/\/arxiv.org\/abs\/2407.08422"},{"key":"e_1_3_2_94_2","unstructured":"Xinyi Hou Yanjie Zhao and Haoyu Wang. 2024b. Voices from the frontier: A comprehensive analysis of the OpenAI developer forum. arXiv:2408.01687. Retrieved from https:\/\/arxiv.org\/abs\/2408.01687"},{"key":"e_1_3_2_95_2","first-page":"212:1","volume-title":"Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924)","author":"Hu Jinchang","year":"2024","unstructured":"Jinchang Hu, Lyuye Zhang, Chengwei Liu, Sen Yang, Song Huang, and Yang Liu. 2024. Empirical analysis of vulnerabilities Life cycle in golang ecosystem. In Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924). ACM, New York, NY, 212:1\u2013212:13. DOI: 10.1145\/3597503.3639230"},{"key":"e_1_3_2_96_2","unstructured":"Kexin Huang Xiangyang Liu Qianyu Guo Tianxiang Sun Jiawei Sun Yaru Wang Zeyang Zhou Yixu Wang Yan Teng Xipeng Qiu et al. 2023. Flames: Benchmarking value alignment of chinese large language models. arXiv:2311.06899. Retrieved from https:\/\/arxiv.org\/abs\/2311.06899"},{"key":"e_1_3_2_97_2","unstructured":"Evan Hubinger Chris van Merwijk Vladimir Mikulik Joar Skalse and Scott Garrabrant. 2019. Risks from learned optimization in advanced machine learning systems. arXiv:1906.01820. Retrieved from http:\/\/arxiv.org\/abs\/1906.01820"},{"key":"e_1_3_2_98_2","unstructured":"HuggingFace. 2024. Datasets. Retrieved November 11 2024 from https:\/\/github.com\/huggingface\/datasets"},{"key":"e_1_3_2_99_2","doi-asserted-by":"crossref","first-page":"560","DOI":"10.1145\/3442188.3445918","volume-title":"Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT \u201921)","author":"Hutchinson Ben","year":"2021","unstructured":"Ben Hutchinson, Andrew Smart, Alex Hanna, Emily Denton, Christina Greer, Oddur Kjartansson, Parker Barnes, and Margaret Mitchell. 2021. Towards accountability for machine learning datasets: Practices from software engineering and infrastructure. In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT \u201921). Madeleine Clare Elish, William Isaac, and Richard S. Zemel (Eds.), ACM, New York, NY, 560\u2013575. DOI: 10.1145\/3442188.3445918"},{"key":"e_1_3_2_100_2","unstructured":"InfiniFlow. 2024. RAGFlow. Retrieved November 11 2024 from https:\/\/github.com\/infiniflow\/ragflow"},{"key":"e_1_3_2_101_2","unstructured":"Ionut Ilascu. 2021. Cybercriminal Sells Tool to Hide Malware in AMD NVIDIA GPUs. Retrieved September 20 2024 from https:\/\/www.bleepingcomputer.com\/news\/security\/cybercriminal-sells-tool-to-hide-malware-in-amd-nvidia-gpus\/"},{"key":"e_1_3_2_102_2","unstructured":"iumiro. 2023. CVE-2023-1177 (MLFlow Path Traversal). Retrieved September 23 2024 from https:\/\/github.com\/iumiro\/CVE-2023-1177-MLFlow"},{"key":"e_1_3_2_103_2","first-page":"3561","volume-title":"KDD \u201920: The Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD \u201920)","author":"Jain Abhinav","year":"2020","unstructured":"Abhinav Jain, Hima Patel, Lokesh Nagalapatti, Nitin Gupta, Sameep Mehta, Shanmukha C. Guttula, Shashank Mujumdar, Shazia Afzal, Ruhi Sharma Mittal, and Vitobha Munigala. 2020. Overview and importance of data quality for machine learning tasks. In KDD \u201920: The Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD \u201920). Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash (Eds.), ACM, New York, NY, 3561\u20133562. DOI: 10.1145\/3394486.3406477"},{"key":"e_1_3_2_104_2","unstructured":"Naman Jain King Han Alex Gu Wen-Ding Li Fanjia Yan Tianjun Zhang Sida Wang Armando Solar-Lezama Koushik Sen and Ion Stoica. 2024a. LiveCodeBench: Holistic and contamination free evaluation of large language models for code. arXiv:2403.07974. Retrieved from https:\/\/arxiv.org\/abs\/2403.07974"},{"key":"e_1_3_2_105_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations (ICLR \u201924)","author":"Jain Naman","year":"2024","unstructured":"Naman Jain, Tianjun Zhang, Wei-Lin Chiang, Joseph E. Gonzalez, Koushik Sen, and Ion Stoica. 2024. LLM-assisted code cleaning for training accurate code generators. In Proceedings of the 12th International Conference on Learning Representations (ICLR \u201924). OpenReview.Net. Retrieved from https:\/\/openreview.net\/forum?id=maRYffiUpI"},{"key":"e_1_3_2_106_2","unstructured":"Jair Ribeiro. 2024. On the risks of overreliance on a single AI Cloud services provider. Retrieved September 20 2024 from https:\/\/medium.com\/technology-hits\/on-the-risks-of-overreliance-on-a-single-ai-cloud-services-provider-c9a32f20b7e8"},{"key":"e_1_3_2_107_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Ji Jiaming","year":"2023","unstructured":"Jiaming Ji, Mickel Liu, Josef Dai, Xuehai Pan, Chi Zhang, Ce Bian, Boyuan Chen, Ruiyang Sun, Yizhou Wang, and Yaodong Yang. 2023b. BeaverTails: Towards improved safety alignment of LLM via a Human-preference dataset. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/4dbb61cb68671edc4ca3712d70083b9f-Abstract-Datasets_and_Benchmarks.html"},{"key":"e_1_3_2_108_2","doi-asserted-by":"publisher","DOI":"10.13328\/j.cnki.jos.006717"},{"key":"e_1_3_2_109_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571730"},{"key":"e_1_3_2_110_2","unstructured":"Ziwei Ji Tiezheng Yu Yan Xu Nayeon Lee Etsuko Ishii and Pascale Fung. 2023. Towards mitigating hallucination in large language models via Self-reflection. arXiv:2310.06271. Retrieved from https:\/\/arxiv.org\/abs\/arXiv:2310.06271"},{"key":"e_1_3_2_111_2","first-page":"2463","volume-title":"Proceedings of the 45th IEEE\/ACM International Conference on Software Engineering (ICSE \u201923)IEEE","author":"Jiang Wenxin","year":"2023","unstructured":"Wenxin Jiang, Nicholas Synovic, Matt Hyatt, Taylor R. Schorlemmer, Rohan Sethi, Yung-Hsiang Lu, George K. Thiruvathukal, and James C. Davis. 2023. An empirical study of pre-trained model reuse in the hugging face deep learning model registry. In Proceedings of the 45th IEEE\/ACM International Conference on Software Engineering (ICSE \u201923). IEEE, 2463\u20132475. DOI: 10.1109\/ICSE48619.2023.00206"},{"key":"e_1_3_2_112_2","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1145\/3560835.3564547","volume-title":"Proceedings of the ACM Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses (SCORED \u201922)","author":"Jiang Wenxin","year":"2022","unstructured":"Wenxin Jiang, Nicholas Synovic, Rohan Sethi, Aryan Indarapu, Matt Hyatt, Taylor R. Schorlemmer, George K. Thiruvathukal, and James C. Davis. 2022. An empirical study of artifacts and security risks in the pre-trained model supply chain. In Proceedings of the ACM Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses (SCORED \u201922). Santiago Torres-Arias, Marcela S. Melara, and Laurent Simon (Eds.), ACM, New York, NY, 105\u2013114. DOI: 10.1145\/3560835.3564547"},{"key":"e_1_3_2_113_2","first-page":"5372","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL\/IJCNLP \u201921)","author":"Jiang Zhongtao","year":"2021","unstructured":"Zhongtao Jiang, Yuanzhe Zhang, Zhao Yang, Jun Zhao, and Kang Liu. 2021. Alignment rationale for natural language inference. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL\/IJCNLP \u201921). Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.), Vol. 1 Long Papers, Association for Computational Linguistics, 5372\u20135387. DOI: 10.18653\/V1\/2021.ACL-LONG.417"},{"key":"e_1_3_2_114_2","doi-asserted-by":"publisher","DOI":"10.1145\/3361870"},{"key":"e_1_3_2_115_2","unstructured":"Haolin Jin Linghan Huang Haipeng Cai Jun Yan Bo Li and Huaming Chen. 2024. From LLMs to LLM-based agents for software engineering: A survey of current challenges and future. arXiv:2408.02479. Retrieved from https:\/\/arxiv.org\/abs\/2408.02479"},{"key":"e_1_3_2_116_2","first-page":"51","volume-title":"Proceedings of the IEEE International Workshop on Hardware-Oriented Security and Trust (HOST \u201908)IEEE Computer Society","author":"Jin Yier","year":"2008","unstructured":"Yier Jin and Yiorgos Makris. 2008. Hardware trojan detection using path delay fingerprint. In Proceedings of the IEEE International Workshop on Hardware-Oriented Security and Trust (HOST \u201908). Mohammad Tehranipoor and Jim Plusquellic (Eds.), IEEE Computer Society, 51\u201357. DOI: 10.1109\/HST.2008.4559049"},{"key":"e_1_3_2_117_2","series-title":"Proceedings of Machine Learning Research","first-page":"15696","volume-title":"Proceedings of the International Conference on Machine Learning (ICML \u201923)","volume":"202","author":"Kandpal Nikhil","year":"2023","unstructured":"Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. 2023. Large language models struggle to Learn long-tail knowledge. In Proceedings of the International Conference on Machine Learning (ICML \u201923). Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.), Proceedings of Machine Learning Research, Vol. 202, PMLR, 15696\u201315707. Retrieved from https:\/\/proceedings.mlr.press\/v202\/kandpal23a.html"},{"key":"e_1_3_2_118_2","first-page":"7403","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP \u201923)Association for Computational Linguistics","author":"Karamolegkou Antonia","year":"2023","unstructured":"Antonia Karamolegkou, Jiaang Li, Li Zhou, and Anders S\u00f8gaard. 2023. Copyright violations and large language models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP \u201923). Houda Bouamor, Juan Pino, and Kalika Bali (Eds.), Association for Computational Linguistics, 7403\u20137412. DOI: 10.18653\/V1\/2023.EMNLP-MAIN.458"},{"key":"e_1_3_2_119_2","first-page":"99","volume-title":"Proceedings of the 3rd IEEE International Conference on Data Mining (ICDM \u201903)","author":"Kargupta Hillol","year":"2003","unstructured":"Hillol Kargupta, Souptik Datta, Qi Wang, and Krishnamoorthy Sivakumar. 2003. On the privacy preserving properties of random data perturbation techniques. In Proceedings of the 3rd IEEE International Conference on Data Mining (ICDM \u201903). IEEE Computer Society, 99\u2013106. DOI: 10.1109\/ICDM.2003.1250908"},{"key":"e_1_3_2_120_2","first-page":"1","volume-title":"Proceedings of the IEEE International Conference on Intelligence and Security Informatics (ISI \u201923)IEEE","author":"Kathikar Adhishree","year":"2023","unstructured":"Adhishree Kathikar, Aishwarya Nair, Ben Lazarine, Agrim Sachdeva, and Sagar Samtani. 2023. Assessing the vulnerabilities of the open-source artificial intelligence (AI) landscape: A large-scale analysis of the hugging face platform. In Proceedings of the IEEE International Conference on Intelligence and Security Informatics (ISI \u201923). IEEE, 1\u20136. DOI: 10.1109\/ISI58743.2023.10297271"},{"key":"e_1_3_2_121_2","first-page":"74","volume-title":"Proceedings of the IEEE\/ACM First International Conference on AI Foundation Models and Software Engineering (FORGE \u201924)","author":"Katzy Jonathan","year":"2024","unstructured":"Jonathan Katzy, Razvan Mihai Popescu, Arie van Deursen, and Maliheh Izadi. 2024. An exploratory investigation into code license infringements in large language model training datasets. In Proceedings of the IEEE\/ACM First International Conference on AI Foundation Models and Software Engineering (FORGE \u201924). David Lo, Xin Xia, Massimiliano Di Penta, and Xing Hu (Eds.), ACM, New York, NY, 74\u201385. DOI: 10.1145\/3650105.3652298"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491209"},{"key":"e_1_3_2_123_2","doi-asserted-by":"crossref","first-page":"5805","DOI":"10.1145\/3580305.3599557","volume-title":"Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD \u201923)","author":"Kenthapadi Krishnaram","year":"2023","unstructured":"Krishnaram Kenthapadi, Himabindu Lakkaraju, and Nazneen Rajani. 2023. Generative AI meets responsible AI: Practical challenges and opportunities. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD \u201923) Ambuj K. Singh, Yizhou Sun, Leman Akoglu, Dimitrios Gunopulos, Xifeng Yan, Ravi Kumar, Fatma Ozcan, and Jieping Ye (Eds.), ACM, New York, NY, 5805\u20135806. DOI: 10.1145\/3580305.3599557"},{"key":"e_1_3_2_124_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Kim Siwon","year":"2023","unstructured":"Siwon Kim, Sangdoo Yun, Hwaran Lee, Martin Gubri, Sungroh Yoon, and Seong Joon Oh. 2023. ProPILE: Probing privacy leakage in large language models. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923) Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/420678bb4c8251ab30e765bc27c3b047-Abstract-Conference.html"},{"key":"e_1_3_2_125_2","first-page":"2611","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201921)","author":"Kirk Hannah Rose","year":"2021","unstructured":"Hannah Rose Kirk, Yennie Jun, Filippo Volpin, Haider Iqbal, Elias Benussi, Fr\u00e9d\u00e9ric A. Dreyer, Aleksandar Shtedritski, and Yuki M. Asano. 2021. Bias out-of-the-box: An empirical analysis of intersectional occupational biases in popular generative language models. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201921). Marc\u2019Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (Eds.), 2611\u20132624. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2021\/hash\/1531beb762df4029513ebf9295e0d34f-Abstract.html"},{"key":"e_1_3_2_126_2","unstructured":"Max Klabunde Tobias Schumacher Markus Strohmaier and Florian Lemmerich. 2023. Similarity of neural network models: A survey of functional and representational measures. arXiv:2305.06329. Retrieved from https:\/\/arxiv.org\/abs\/2305.06329"},{"key":"e_1_3_2_127_2","doi-asserted-by":"crossref","first-page":"176","DOI":"10.1007\/978-3-031-61486-6_11","volume-title":"Proceedings of the Satellite Workshops on Applied Cryptography and Network Security Workshops (ACNS \u201924)","volume":"14586","author":"Kokolakis Georgios","year":"2024","unstructured":"Georgios Kokolakis, Athanasios Moschos, and Angelos D. Keromytis. 2024. Harnessing the power of general-purpose LLMs in hardware trojan design. In Proceedings of the Satellite Workshops on Applied Cryptography and Network Security Workshops (ACNS \u201924). Martin Andreoni (Ed.), Lecture Notes in Computer Science, Vol. 14586, Springer, 176\u2013194. DOI: 10.1007\/978-3-031-61486-6_11"},{"key":"e_1_3_2_128_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"K\u00f6pf Andreas","year":"2023","unstructured":"Andreas K\u00f6pf, Yannic Kilcher, Dimitri von R\u00fctte, Sotiris Anagnostidis, Zhi Rui Tam, Keith Stevens, Abdullah Barhoum, Duc Nguyen, Oliver Stanley, Rich\u00e1rd Nagyfi, et al. 2023. OpenAssistant conversations - democratizing large language model alignment. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/949f0f8f32267d297c2d4e3ee10a2e7e-Abstract-Datasets_and_Benchmarks.html"},{"key":"e_1_3_2_129_2","unstructured":"LangChain-AI. 2024. LangChain. Retrieved March 28 2024 from https:\/\/github.com\/langchain-ai\/langchain"},{"key":"e_1_3_2_130_2","doi-asserted-by":"publisher","DOI":"10.2753\/MIS0742-1222310206"},{"key":"e_1_3_2_131_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201922)","author":"Lee Nayeon","year":"2022","unstructured":"Nayeon Lee, Wei Ping, Peng Xu, Mostofa Patwary, Pascale Fung, Mohammad Shoeybi, and Bryan Catanzaro. 2022. Factuality enhanced language models for open-ended text generation. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201922). Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2022\/hash\/df438caa36714f69277daa92d608dd63-Abstract-Conference.html"},{"key":"e_1_3_2_132_2","volume-title":"Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914)","author":"Li Changmao","year":"2024","unstructured":"Changmao Li and Jeffrey Flanigan. 2024. Task contamination: Language models May not be Few-shot anymore. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914). Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.), AAAI Press, 18471\u201318480. DOI: 10.1609\/AAAI.V38I16.29808"},{"key":"e_1_3_2_133_2","unstructured":"Haodong Li Gelei Deng Yi Liu Kailong Wang Yuekang Li Tianwei Zhang Yang Liu Guoai Xu Guosheng Xu and Haoyu Wang. 2024. Digger: Detecting copyright content mis-usage in large language model training. arXiv:2401.00676."},{"key":"e_1_3_2_134_2","first-page":"6449","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP \u201923)Association for Computational Linguistics","author":"Li Junyi","year":"2023","unstructured":"Junyi Li, Xiaoxue Cheng, Xin Zhao, Jian-Yun Nie, and Ji-Rong Wen. 2023. HaluEval: A large-scale hallucination evaluation benchmark for large language models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP \u201923). Houda Bouamor, Juan Pino, and Kalika Bali (Eds.), Association for Computational Linguistics, 6449\u20136464. DOI: 10.18653\/V1\/2023.EMNLP-MAIN.397"},{"key":"e_1_3_2_135_2","unstructured":"Kenneth Li Tianle Liu Naomi Bashkansky David Bau Fernanda B. Vi\u00e9gas Hanspeter Pfister and Martin Wattenberg. 2024. Measuring and controlling persona drift in language model dialogs. arXiv:2402.10962. Retrieved from https:\/\/arxiv.org\/abs\/2402.10962"},{"key":"e_1_3_2_136_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Li Kenneth","year":"2023","unstructured":"Kenneth Li, Oam Patel, Fernanda B. Vi\u00e9gas, Hanspeter Pfister, and Martin Wattenberg. 2023. Inference-time intervention: Eliciting truthful answers from a language model. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/81b8390039b7302c909cb769f8b6cd93-Abstract-Conference.html"},{"key":"e_1_3_2_137_2","first-page":"1993","volume-title":"Proceedings of the 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201923)","author":"Li Ningke","year":"2023","unstructured":"Ningke Li, Shenao Wang, Mingxi Feng, Kailong Wang, Meizhen Wang, and Haoyu Wang. 2023e. MalWuKong: Towards fast, accurate, and multilingual detection of malicious code poisoning in OSS supply chains. In Proceedings of the 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201923). IEEE, 1993\u20132005. DOI: 10.1109\/ASE56229.2023.00073"},{"key":"e_1_3_2_138_2","unstructured":"Rongao Li Jie Fu Bo-Wen Zhang Tao Huang Zhihong Sun Chen Lyu Guang Liu Zhi Jin and Ge Li. 2023. TACO: Topics in algorithmic COde generation dataset. arXiv:2312.14852. Retrieved from https:\/\/arxiv.org\/abs\/2312.14852"},{"key":"e_1_3_2_139_2","first-page":"3123","volume-title":"Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS \u201921)","author":"Li Shaofeng","year":"2021","unstructured":"Shaofeng Li, Hui Liu, Tian Dong, Benjamin Zi Hao Zhao, Minhui Xue, Haojin Zhu, and Jialiang Lu. 2021. Hidden backdoors in human-centric language models. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS \u201921). Yongdae Kim, Jong Kim, Giovanni Vigna, and Elaine Shi (Eds.), ACM, New York, NY, 3123\u20133140. DOI: 10.1145\/3460120.3484576"},{"key":"e_1_3_2_140_2","unstructured":"Yucheng Li. 2023. Estimating contamination via perplexity: Quantifying memorisation in language model evaluation. arXiv:2309.10677. Retrieved from https:\/\/arxiv.org\/abs\/2309.10677"},{"key":"e_1_3_2_141_2","first-page":"18600","volume-title":"Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914)","author":"Li Yucheng","year":"2024","unstructured":"Yucheng Li, Frank Guerin, and Chenghua Lin. 2024. LatestEval: Addressing data contamination in language model evaluation through dynamic and time-sensitive test construction. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914). Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.), AAAI Press, 18600\u201318607. DOI: 10.1609\/AAAI.V38I17.29822"},{"key":"e_1_3_2_142_2","first-page":"7236","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL \u201923)","author":"Li Yanzhou","year":"2023","unstructured":"Yanzhou Li, Shangqing Liu, Kangjie Chen, Xiaofei Xie, Tianwei Zhang, and Yang Liu. 2023c. Multi-target backdoor attacks for code Pre-trained models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL \u201923). Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki (Eds.), Association for Computational Linguistics, 7236\u20137254. DOI: 10.18653\/V1\/2023.ACL-LONG.399"},{"key":"e_1_3_2_143_2","first-page":"2075","volume-title":"Proceedings of the ACM on Software Engineering","volume":"1","author":"Li Yuxi","year":"2024","unstructured":"Yuxi Li, Yi Liu, Gelei Deng, Ying Zhang, Wenjia Song, Ling Shi, Kailong Wang, Yuekang Li, Yang Liu, and Haoyu Wang. 2024d. Glitch tokens in large language models: Categorization taxonomy and effective detection. Proceedings of the ACM on Software Engineering 1, FSE (2024), 2075\u20132097. DOI: 10.1145\/3660799"},{"key":"e_1_3_2_144_2","unstructured":"Yuanchun Li Hao Wen Weijun Wang Xiangyu Li Yizhen Yuan Guohong Liu Jiacheng Liu Wenxing Xu Xiang Wang Yi Sun et al. 2024. Personal LLM agents: Insights and survey about the capability efficiency and security. arXiv:2401.05459. Retrieved from https:\/\/arxiv.org\/abs\/2401.05459"},{"key":"e_1_3_2_145_2","first-page":"139","volume-title":"Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA \u201921)","author":"Li Yuanchun","year":"2021","unstructured":"Yuanchun Li, Ziqi Zhang, Bingyan Liu, Ziyue Yang, and Yunxin Liu. 2021b. ModelDiff: Testing-based DNN similarity comparison for model reuse detection. In Proceedings of the 30th ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA \u201921). Cristian Cadar and Xiangyu Zhang (Eds.), ACM, New York, NY, 139\u2013151. DOI: 10.1145\/3460319.3464816"},{"key":"e_1_3_2_146_2","article-title":"Holistic evaluation of language models","volume":"2023","author":"Liang Percy","year":"2023","unstructured":"Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. 2023. Holistic evaluation of language models. Transactions on Machine Learning Research 2023 (2023). Retrieved from https:\/\/openreview.net\/forum?id=iO4LZibEqW","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_3_2_147_2","first-page":"6565","volume-title":"Proceedings of the 38th International Conference on Machine Learning (ICML \u201921)","volume":"139","author":"Liang Paul Pu","year":"2021","unstructured":"Paul Pu Liang, Chiyu Wu, Louis-Philippe Morency, and Ruslan Salakhutdinov. 2021. Towards understanding and mitigating social biases in language models. In Proceedings of the 38th International Conference on Machine Learning (ICML \u201921). Marina Meila and Tong Zhang (Eds.), Proceedings of Machine Learning Research, Vol. 139, PMLR, 6565\u20136576. Retrieved from http:\/\/proceedings.mlr.press\/v139\/liang21a.html"},{"key":"e_1_3_2_148_2","unstructured":"Weixin Liang Nazneen Rajani Xinyu Yang Ezinwanne Ozoani Eric Wu Yiqun Chen Daniel Scott Smith and James Zou. 2024. What\u2019s documented in AI? Systematic analysis of 32K AI model cards. arXiv:2402.05160. Retrieved from https:\/\/arxiv.org\/abs\/2402.05160"},{"key":"e_1_3_2_149_2","first-page":"3214","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL \u201922)","author":"Lin Stephanie","year":"2022","unstructured":"Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. TruthfulQA: Measuring how models mimic Human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL \u201922). Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.), Association for Computational Linguistics, 3214\u20133252. DOI: 10.18653\/V1\/2022.ACL-LONG.229"},{"key":"e_1_3_2_150_2","first-page":"672","volume-title":"Proceedings of the 44th IEEE\/ACM 44th International Conference on Software Engineering (ICSE \u201922)","author":"Liu Chengwei","year":"2022","unstructured":"Chengwei Liu, Sen Chen, Lingling Fan, Bihuan Chen, Yang Liu, and Xin Peng. 2022. Demystifying the vulnerability propagation and its evolution via dependency trees in the NPM ecosystem. In Proceedings of the 44th IEEE\/ACM 44th International Conference on Software Engineering (ICSE \u201922). ACM, New York, NY, 672\u2013684. DOI: 10.1145\/3510003.3510142"},{"key":"e_1_3_2_151_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations (ICLR \u201924)","author":"Liu Hao","year":"2024","unstructured":"Hao Liu, Carmelo Sferrazza, and Pieter Abbeel. 2024b. Chain of hindsight aligns language models with feedback. In Proceedings of the 12th International Conference on Learning Representations (ICLR \u201924). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=6xfe4IVcOu"},{"key":"e_1_3_2_152_2","unstructured":"Junwei Liu Kaixin Wang Yixuan Chen Xin Peng Zhenpeng Chen Lingming Zhang and Yiling Lou. 2024. Large language model-based agents for software engineering: A survey. arXiv:2409.02977. Retrieved from https:\/\/arxiv.org\/abs\/2409.02977"},{"key":"e_1_3_2_153_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Liu Jiawei","year":"2023","unstructured":"Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023. Is your code generated by ChatGPT really correct? Rigorous evaluation of large language models for code generation. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/43e9d647ccd3e4b7b5baab53f0368686-Abstract-Conference.html"},{"key":"e_1_3_2_154_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.ARTINT.2021.103654"},{"key":"e_1_3_2_155_2","unstructured":"Tong Liu Zizhuang Deng Guozhu Meng Yuekang Li and Kai Chen. 2023. Demystifying RCE vulnerabilities in LLM-integrated apps. arXiv:2309.02926. Retrieved from https:\/\/arxiv.org\/abs\/2309.02926"},{"key":"e_1_3_2_156_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations (ICLR \u201924)","author":"Liu Tianyang","year":"2024","unstructured":"Tianyang Liu, Canwen Xu, and Julian J. McAuley. 2024d. RepoBench: Benchmarking repository-level code Auto-completion systems. In Proceedings of the 12th International Conference on Learning Representations (ICLR \u201924). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=pPjZIOuQuF"},{"key":"e_1_3_2_157_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597204"},{"key":"e_1_3_2_158_2","doi-asserted-by":"crossref","unstructured":"Yang Liu Jiahuan Cao Chongyu Liu Kai Ding and Lianwen Jin. 2024a. Datasets for large language models: A comprehensive survey. arXiv:2402.18041. Retrieved from https:\/\/arxiv.org\/abs\/arXiv:2402.18041","DOI":"10.21203\/rs.3.rs-3996137\/v1"},{"key":"e_1_3_2_159_2","unstructured":"Yi Liu Gelei Deng Yuekang Li Kailong Wang Tianwei Zhang Yepang Liu Haoyu Wang Yan Zheng and Yang Liu. 2023. Prompt injection attack against LLM-integrated applications. arXiv:2306.05499. Retrieved from https:\/\/arxiv.org\/abs\/2306.05499"},{"key":"e_1_3_2_160_2","unstructured":"Yi Liu Gelei Deng Zhengzi Xu Yuekang Li Yaowen Zheng Ying Zhang Lida Zhao Tianwei Zhang and Yang Liu. 2023. Jailbreaking ChatGPT via prompt engineering: An empirical study. arXiv:2305.13860. Retrieved from https:\/\/arxiv.org\/abs\/2305.13860"},{"key":"e_1_3_2_161_2","unstructured":"Yupei Liu Yuqi Jia Runpeng Geng Jinyuan Jia and Neil Zhenqiang Gong. 2023. Prompt injection attacks and defenses in LLM-integrated applications. arXiv:2310.12815. Retrieved from https:\/\/arxiv.org\/abs\/2310.12815"},{"key":"e_1_3_2_162_2","unstructured":"Yang Liu Yuanshun Yao Jean-Francois Ton Xiaoying Zhang Ruocheng Guo Hao Cheng Yegor Klochkov Muhammad Faaiz Taufiq and Hang Li. 2023. Trustworthy LLMs: A survey and guideline for evaluating large language models\u2019 Alignment. arXiv:2308.05374. Retrieved from https:\/\/arxiv.org\/abs\/2308.05374"},{"key":"e_1_3_2_163_2","unstructured":"Shayne Longpre Robert Mahari Anthony Chen Naana Obeng-Marnu Damien Sileo William Brannon Niklas Muennighoff Nathan Khazam Jad Kabbara Kartik Perisetla et al. 2023. The data provenance initiative: A large scale audit of dataset licensing & attribution in AI. arXiv:2310.16787. Retrieved from https:\/\/arxiv.org\/abs\/2310.16787"},{"key":"e_1_3_2_164_2","doi-asserted-by":"crossref","first-page":"346","DOI":"10.1109\/SP46215.2023.10179300","volume-title":"44th IEEE Symposium on Security and Privacy, SP 2023","author":"Lukas Nils","year":"2023","unstructured":"Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella B\u00e9guelin. 2023. Analyzing leakage of personally identifiable information in language models. In 44th IEEE Symposium on Security and Privacy, SP 2023, San Francisco, CA, USA, May 21-25, 2023. IEEE, 346\u2013363. DOI: 10.1109\/SP46215.2023.10179300"},{"key":"e_1_3_2_165_2","first-page":"248","volume-title":"Proceedings of the 43rd IEEE\/ACM International Conference on Software Engineering: Software Engineering in Practice, ICSE (SEIP \u201921)IEEE","author":"Lwakatare Lucy Ellen","year":"2021","unstructured":"Lucy Ellen Lwakatare, Ellinor R\u00e5nge, Ivica Crnkovic, and Jan Bosch. 2021. On the experiences of adopting automated data validation in an industrial machine learning project. In Proceedings of the 43rd IEEE\/ACM International Conference on Software Engineering: Software Engineering in Practice, ICSE (SEIP \u201921). IEEE, 248\u2013257. DOI: 10.1109\/ICSE-SEIP52600.2021.00034"},{"key":"e_1_3_2_166_2","unstructured":"Lingjuan Lyu Han Yu and Qiang Yang. 2020. Threats to federated learning: A survey. arXiv:2003.02133. Retrieved from https:\/\/arxiv.org\/abs\/2003.02133"},{"key":"e_1_3_2_167_2","first-page":"458","volume-title":"Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings (ICSE \u201918)","author":"Ma Yuxing","year":"2018","unstructured":"Yuxing Ma. 2018. Constructing supply chains in open source software. In Proceedings of the 40th International Conference on Software Engineering: Companion Proceeedings (ICSE \u201918). Michel Chaudron, Ivica Crnkovic, Marsha Chechik, and Mark Harman (Eds.), ACM, New York, NY, 458\u2013459. DOI: 10.1145\/3183440.3183454"},{"key":"e_1_3_2_168_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Madaan Aman","year":"2023","unstructured":"Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with Self-feedback. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/91edff07232fb1b55a505a9e9f6c0ff3-Abstract-Conference.html"},{"key":"e_1_3_2_169_2","first-page":"15767","volume-title":"Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI \u201923), 35th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201923), 13th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201923)","author":"Madaan Nishtha","year":"2023","unstructured":"Nishtha Madaan, Adithya Manjunatha, Hrithik Nambiar, Aviral Kumar Goel, Harivansh Kumar, Diptikalyan Saha, and Srikanta Bedathur. 2023a. DetAIL: A tool to automatically detect and analyze drift in language. In Proceedings of the 37th AAAI Conference on Artificial Intelligence (AAAI \u201923), 35th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201923), 13th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201923). Brian Williams, Yiling Chen, and Jennifer Neville (Eds.), AAAI Press, 15767\u201315773. DOI: 10.1609\/AAAI.V37I13.26872"},{"key":"e_1_3_2_170_2","first-page":"74","volume-title":"Proceedings of the 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA \u201923)IEEE","author":"Madani Pooria","year":"2023","unstructured":"Pooria Madani. 2023. Metamorphic malware evolution: The potential and peril of large language models. In Proceedings of the 5th IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA \u201923). IEEE, 74\u201381. DOI: 10.1109\/TPS-ISA58951.2023.00019"},{"key":"e_1_3_2_171_2","first-page":"157","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (ACL \u201922)Association for Computational Linguistics","author":"Magar Inbal","year":"2022","unstructured":"Inbal Magar and Roy Schwartz. 2022. Data contamination: From memorization to exploitation. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) (ACL \u201922). Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.), Association for Computational Linguistics, 157\u2013165. DOI: 10.18653\/V1\/2022.ACL-SHORT.18"},{"key":"e_1_3_2_172_2","first-page":"155","volume-title":"Proceedings of the 17th USENIX Symposium on Operating Systems Design and Implementation (OSDI \u201923)USENIX Association","author":"Mai Haohui","year":"2023","unstructured":"Haohui Mai, Jiacheng Zhao, Hongren Zheng, Yiyang Zhao, Zibin Liu, Mingyu Gao, Cong Wang, Huimin Cui, Xiaobing Feng, and Christos Kozyrakis. 2023. Honeycomb: Secure and efficient GPU executions via static validation. In Proceedings of the 17th USENIX Symposium on Operating Systems Design and Implementation (OSDI \u201923). Roxana Geambasu and Ed Nightingale (Eds.), USENIX Association, 155\u2013172. Retrieved from https:\/\/www.usenix.org\/conference\/osdi23\/presentation\/mai"},{"key":"e_1_3_2_173_2","unstructured":"Vahid Majdinasab Amin Nikanjam and Foutse Khomh. 2024. Trained without my consent: Detecting code inclusion in language models trained on code. arXiv:2402.09299. Retrieved from https:\/\/arxiv.org\/abs\/2402.09299"},{"key":"e_1_3_2_174_2","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2016.2630689"},{"key":"e_1_3_2_175_2","volume-title":"Proceedings of the 41st International Conference on Machine Learning (ICML \u201924)","author":"Mazeika Mantas","year":"2024","unstructured":"Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, et al. 2024. HarmBench: A standardized evaluation framework for automated red teaming and robust refusal. In Proceedings of the 41st International Conference on Machine Learning (ICML \u201924). OpenReview.Net. Retrieved from https:\/\/openreview.net\/forum?id=f3TUipYU3U"},{"key":"e_1_3_2_176_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Mazumder Mark","year":"2023","unstructured":"Mark Mazumder, Colby R. Banbury, Xiaozhe Yao, Bojan Karlas, William Gaviria Rojas, Sudnya Frederick Diamos, Greg Diamos, Lynn He, Alicia Parrish, Hannah Rose Kirk, et al. 2023. DataPerf: Benchmarks for data-centric AI development. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/112db88215e25b3ae2750e9eefcded94-Abstract-Datasets_and_Benchmarks.html"},{"key":"e_1_3_2_177_2","doi-asserted-by":"crossref","unstructured":"Timothy R. McIntosh Teo Susnjak Tong Liu Paul A. Watters and Malka N. Halgamuge. 2024. Inadequacies of large language model benchmarks in the era of generative artificial intelligence. arXiv:2402.09880.","DOI":"10.1109\/TAI.2025.3569516"},{"key":"e_1_3_2_178_2","volume-title":"Proceedings of the 41st International Conference on Machine Learning (ICML \u201924)","author":"Meeus Matthieu","year":"2024","unstructured":"Matthieu Meeus, Igor Shilov, Manuel Faysse, and Yves-Alexandre de Montjoye. 2024. Copyright traps for large language models. In Proceedings of the 41st International Conference on Machine Learning (ICML \u201924). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=LDq1JPdc55"},{"key":"e_1_3_2_179_2","unstructured":"Kai Mei Zelong Li Shuyuan Xu Ruosong Ye Yingqiang Ge and Yongfeng Zhang. 2024. LLM agent operating system. arXiv:2403.16971. Retrieved from https:\/\/arxiv.org\/abs\/2403.16971"},{"key":"e_1_3_2_180_2","unstructured":"MLC-AI. 2024. MLC-LLM. Retrieved September 28 2024 from https:\/\/github.com\/mlc-ai\/mlc-llm"},{"key":"e_1_3_2_181_2","first-page":"49","volume-title":"Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL \u201924)","volume":"1","author":"Muhlgay Dor","year":"2024","unstructured":"Dor Muhlgay, Ori Ram, Inbal Magar, Yoav Levine, Nir Ratner, Yonatan Belinkov, Omri Abend, Kevin Leyton-Brown, Amnon Shashua, and Yoav Shoham. 2024. Generating benchmarks for factuality evaluation of language models. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL \u201924). Yvette Graham and Matthew Purver (Eds.), Volume 1 Long Papers, Association for Computational Linguistics, 49\u201366. Retrieved from https:\/\/aclanthology.org\/2024.eacl-long.4"},{"key":"e_1_3_2_182_2","unstructured":"Nadav Noy. 2023. Legit discovers \u201cAI jacking\u201d vulnerability in popular hugging face AI platform. Retrieved September 26 2024 from https:\/\/www.legitsecurity.com\/blog\/tens-of-thousands-of-developers-were-potentially-impacted-by-the-hugging-face-aijacking-attack"},{"key":"e_1_3_2_183_2","first-page":"2139","volume-title":"Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS \u201918)","author":"Naghibijouybari Hoda","year":"2018","unstructured":"Hoda Naghibijouybari, Ajaya Neupane, Zhiyun Qian, and Nael B. Abu-Ghazaleh. 2018. Rendered insecure: GPU side channel attacks are practical. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS \u201918). David Lie, Mohammad Mannan, Michael Backes, and XiaoFeng Wang (Eds.), ACM, New York, NY, 2139\u20132153. DOI: 10.1145\/3243734.3243831"},{"key":"e_1_3_2_184_2","first-page":"21454","volume-title":"Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada","author":"Nagireddy Manish","year":"2024","unstructured":"Manish Nagireddy, Lamogha Chiazor, Moninder Singh, and Ioana Baldini. 2024. SocialStigmaQA: A benchmark to uncover stigma amplification in generative language models. In Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.). AAAI Press, 21454\u201321462. DOI: 10.1609\/AAAI.V38I19.30142"},{"key":"e_1_3_2_185_2","first-page":"1542","volume-title":"Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD \u201920)","author":"Namaki Mohammad Hossein","year":"2020","unstructured":"Mohammad Hossein Namaki, Avrilia Floratou, Fotis Psallidas, Subru Krishnan, Ashvin Agrawal, Yinghui Wu, Yiwen Zhu, and Markus Weimer. 2020. Vamsa: Automated provenance tracking in data Science scripts. In Proceedings of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD \u201920). Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash (Eds.), ACM, New York, NY, 1542\u20131551. DOI: 10.1145\/3394486.3403205"},{"key":"e_1_3_2_186_2","unstructured":"Nate Nelson. 2024. Hundreds of LLM Servers Expose Corporate Health & Other Online Data. Retrieved September 28 2024 from https:\/\/www.darkreading.com\/application-security\/hundreds-of-llm-servers-expose-corporate-health-and-other-online-data"},{"key":"e_1_3_2_187_2","doi-asserted-by":"publisher","DOI":"10.1145\/3597307"},{"key":"e_1_3_2_188_2","unstructured":"The Hacker News. 2024. New Hugging Face Vulnerability Exposes AI Models to Supply Chain Attacks. Retrieved March 28 2024 from https:\/\/thehackernews.com\/2024\/02\/new-hugging-face-vulnerability-exposes.html"},{"key":"e_1_3_2_189_2","unstructured":"NVIDIA. 2024. Security bulletin: NVIDIA DGX A100 - January 2024. Retrieved September 9 2024 from https:\/\/nvidia.custhelp.com\/app\/answers\/detail\/a_id\/5510\/\/security-bulletin:-nvidia-dgx-h100-and-a100\u2014january-2024"},{"key":"e_1_3_2_190_2","unstructured":"OpenAI. 2023. GPT-4 technical report. arXiv:2303.08774. Retrieved from https:\/\/arxiv.org\/abs\/2303.08774"},{"key":"e_1_3_2_191_2","unstructured":"OpenAI. 2024. Introducing the GPT Store. Retrieved March 28 2024 from https:\/\/openai.com\/blog\/introducing-the-gpt-store"},{"key":"e_1_3_2_192_2","unstructured":"OpenGVLab. 2024. OmniQuant. Retrieved November 11 2024 from https:\/\/github.com\/OpenGVLab\/OmniQuant"},{"key":"e_1_3_2_193_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations (ICLR \u201924)","author":"Oren Yonatan","year":"2024","unstructured":"Yonatan Oren, Nicole Meister, Niladri S. Chatterji, Faisal Ladhak, and Tatsunori Hashimoto. 2024. Proving test set contamination in black-box language models. In Proceedings of the 12th International Conference on Learning Representations (ICLR \u201924). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=KS8mIvetg2"},{"key":"e_1_3_2_194_2","first-page":"4262","volume-title":"Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL\/IJCNLP \u201921)","author":"Ousidhoum Nedjma","year":"2021","unstructured":"Nedjma Ousidhoum, Xinran Zhao, Tianqing Fang, Yangqiu Song, and Dit-Yan Yeung. 2021. Probing toxic content in large pre-trained language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL\/IJCNLP \u201921). Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (Eds.), Vol. 1 Long Papers, Association for Computational Linguistics, 4262\u20134274. DOI: 10.18653\/V1\/2021.ACL-LONG.329"},{"key":"e_1_3_2_195_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201922)","author":"Ouyang Long","year":"2022","unstructured":"Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with Human feedback. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201922). Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2022\/hash\/b1efde53be364a73914f58805a001731-Abstract-Conference.html"},{"key":"e_1_3_2_196_2","unstructured":"OWASP. 2024. OWASP top 10 for large language model applications. Retrieved March 28 2024 from https:\/\/owasp.org\/www-project-top-10-for-large-language-model-applications\/"},{"key":"e_1_3_2_197_2","unstructured":"OWASP. 2024. OWASP top 10 LLM V2 candidates. Retrieved September 26 2024 from https:\/\/owasp.org\/www-project-top-10-for-large-language-model-applications\/2_0_voting\/v2_candidates.pdf"},{"key":"e_1_3_2_198_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533378"},{"key":"e_1_3_2_199_2","first-page":"408","volume-title":"Proceedings of the 26th Asia and South Pacific Design Automation Conference (ASPDAC \u201921)","author":"Pan Zhixin","year":"2021","unstructured":"Zhixin Pan and Prabhat Mishra. 2021. Automated test generation for hardware trojan detection using reinforcement learning. In Proceedings of the 26th Asia and South Pacific Design Automation Conference (ASPDAC \u201921). ACM, New York, NY, 408\u2013413. DOI: 10.1145\/3394885.3431595"},{"key":"e_1_3_2_200_2","first-page":"2799","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing","author":"Park Ji Ho","year":"2018","unstructured":"Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. Reducing gender bias in abusive language detection. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun\u2019ichi Tsujii (Eds.), Association for Computational Linguistics, 2799\u20132804. DOI: 10.18653\/V1\/D18-1302"},{"key":"e_1_3_2_201_2","unstructured":"Patrick Peng. 2024. Supply-Chain Attacks in LLMs: From GGUF Model Format Metadata RCE to State-of-the-Art NLP project RCEs. Retrieved September 28 2024 from https:\/\/0reg.dev\/blog\/from-gguf-model-format-metadata-rce-to-state-of-the-art-nlp-project-rces"},{"key":"e_1_3_2_202_2","doi-asserted-by":"crossref","first-page":"4296","DOI":"10.18653\/v1\/2020.acl-main.396","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL \u201920)Association for Computational Linguistics","author":"Pavlopoulos John","year":"2020","unstructured":"John Pavlopoulos, Jeffrey Sorensen, Lucas Dixon, Nithum Thain, and Ion Androutsopoulos. 2020. Toxicity detection: Does context really Matter? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL \u201920). Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.), Association for Computational Linguistics, 4296\u20134305. DOI: 10.18653\/V1\/2020.ACL-MAIN.396"},{"key":"e_1_3_2_203_2","unstructured":"Baolin Peng Michel Galley Pengcheng He Hao Cheng Yujia Xie Yu Hu Qiuyuan Huang Lars Liden Zhou Yu Weizhu Chen and Jianfeng Gao. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv:2302.12813. Retrieved from https:\/\/arxiv.org\/abs\/2302.12813"},{"key":"e_1_3_2_204_2","first-page":"145","volume-title":"Proceedings of the 32nd ACM\/IEEE International Conference on Software Engineering (ICSE \u201910)","volume":"1","author":"Penta Massimiliano Di","year":"2010","unstructured":"Massimiliano Di Penta, Daniel M. Germ\u00e1n, Yann-Ga\u00ebl Gu\u00e9h\u00e9neuc, and Giuliano Antoniol. 2010. An exploratory study of the evolution of software licensing. In Proceedings of the 32nd ACM\/IEEE International Conference on Software Engineering (ICSE \u201910). Jeff Kramer, Judith Bishop, Premkumar T. Devanbu, and Sebasti\u00e1n Uchitel (Eds.), Vol. 1, ACM, New York, NY, 145\u2013154. DOI: 10.1145\/1806799.1806824"},{"key":"e_1_3_2_205_2","first-page":"370","volume-title":"Proceedings of the 32nd IEEE\/ACM International Conference on Program Comprehension (ICPC \u201924)","author":"Pepe Federica","year":"2024","unstructured":"Federica Pepe, Vittoria Nardone, Antonio Mastropaolo, Gabriele Bavota, Gerardo Canfora, and Massimiliano Di Penta. 2024. How do hugging face models document datasets, bias, and licenses? An empirical study. In Proceedings of the 32nd IEEE\/ACM International Conference on Program Comprehension (ICPC \u201924). April 15\u201316, 2024, Igor Steinmacher, Mario Linares-V\u00e1squez, Kevin Patrick Moran, and Olga Baysal (Eds.), ACM, New York, NY, 370\u2013381. DOI: 10.1145\/3643916.3644412"},{"key":"e_1_3_2_206_2","doi-asserted-by":"publisher","DOI":"10.1162\/COLI_A_00458"},{"key":"e_1_3_2_207_2","doi-asserted-by":"publisher","unstructured":"Mahesh Datta Sai Ponnuru Likhitha Amasala and Guna Garikipati. 2024. Unveiling the Veil: A Comprehensive Analysis of Data Contamination in Leading Language Models. DOI: 10.21203\/rs.3.rs-3978251\/v1","DOI":"10.21203\/rs.3.rs-3978251\/v1"},{"key":"e_1_3_2_208_2","unstructured":"PyTorch. 2024. PyTorch. Retrieved March 28 2024 from https:\/\/github.com\/pytorch\/pytorch"},{"key":"e_1_3_2_209_2","first-page":"21527","volume-title":"Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914)","author":"Qi Xiangyu","year":"2024","unstructured":"Xiangyu Qi, Kaixuan Huang, Ashwinee Panda, Peter Henderson, Mengdi Wang, and Prateek Mittal. 2024. Visual adversarial examples jailbreak aligned large language models. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914). Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.), AAAI Press, 21527\u201321536. DOI: 10.1609\/AAAI.V38I19.30150"},{"key":"e_1_3_2_210_2","unstructured":"Ray. 2024. Ray: Productionizing and scaling Python ML workloads simply. Retrieved September 23 2024 from https:\/\/www.ray.io\/"},{"key":"e_1_3_2_211_2","unstructured":"The Record. 2024. Thousands of Companies Using Ray Framework Exposed to Cyberattacks Researchers Say. Retrieved March 28 2024 from https:\/\/therecord.media\/thousands-exposed-to-ray-framework-vulnerability"},{"key":"e_1_3_2_212_2","unstructured":"Richard Harang. 2024. Practical LLM Security: Takeaways from a Year in the Trenches. Retrieved September 23 2024 from https:\/\/www.blackhat.com\/us-24\/briefings\/schedule\/#practical-llm-security-takeaways-from-a-year-in-the-trenches-39468"},{"key":"e_1_3_2_213_2","first-page":"3354","volume-title":"Proceedings of the VLDB Endowment","volume":"13","author":"Rupprecht Lukas","year":"2020","unstructured":"Lukas Rupprecht, James C. Davis, Constantine Arnold, Yaniv Gur, and Deepavali Bhagwat. 2020. Improving reproducibility of data Science pipelines through transparent provenance capture. Proceedings of the VLDB Endowment 13, 12 (2020), 3354\u20133368. DOI: 10.14778\/3415478.3415556"},{"key":"e_1_3_2_214_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2011.2164908"},{"key":"e_1_3_2_215_2","first-page":"39:1","volume-title":"Proceedings of the Conference on Human Factors in Computing Systems (CHI \u201921)","author":"Sambasivan Nithya","year":"2021","unstructured":"Nithya Sambasivan, Shivani Kapania, Hannah Highfill, Diana Akrong, Praveen K. Paritosh, and Lora Aroyo. 2021. \u201cEveryone wants to do the model work, not the data work\u201d: Data cascades in high-stakes AI. In Proceedings of the Conference on Human Factors in Computing Systems (CHI \u201921). Yoshifumi Kitamura, Aaron Quigley, Katherine Isbister, Takeo Igarashi, Pernille Bj\u00f8rn, and Steven Mark Drucker (Eds.), ACM, New York, NY, 39:1\u201339:15. DOI: 10.1145\/3411764.3445518"},{"key":"e_1_3_2_216_2","series-title":"Proceedings of Machine Learning Research","first-page":"9389","volume-title":"Proceedings of the 38th International Conference on Machine Learning (ICML \u201921)","volume":"139","author":"Schwarzschild Avi","year":"2021","unstructured":"Avi Schwarzschild, Micah Goldblum, Arjun Gupta, John P. Dickerson, and Tom Goldstein. 2021. Just how toxic is data poisoning? A unified benchmark for backdoor and data poisoning attacks. In Proceedings of the 38th International Conference on Machine Learning (ICML \u201921). Marina Meila and Tong Zhang (Eds.), Proceedings of Machine Learning Research, Vol. 139, PMLR, 9389\u20139398. Retrieved from http:\/\/proceedings.mlr.press\/v139\/schwarzschild21a.html"},{"key":"e_1_3_2_217_2","unstructured":"Shachar Menashe. 2024. From MLOps to MLOops - Exposing the Attack Surface of Machine Learning Platforms. Retrieved September 23 2024 from https:\/\/www.blackhat.com\/us-24\/briefings\/schedule\/#from-mlops-to-mloops\u2014exposing-the-attack-surface-of-machine-learning-platforms-39309"},{"key":"e_1_3_2_218_2","unstructured":"Shaerul Haque J. 2024. Navigating the Cloud: Reflecting on the World\u2019s 10 Biggest Outages and Critical Preventive Measures. Retrieved September 20 2024 from https:\/\/www.linkedin.com\/pulse\/navigating-cloud-reflecting-worlds-10-biggest-outages-joarder-kemac\/"},{"key":"e_1_3_2_219_2","unstructured":"Tianhao Shen Renren Jin Yufei Huang Chuang Liu Weilong Dong Zishan Guo Xinwei Wu Yan Liu and Deyi Xiong. 2023. Large language model alignment: A survey. arXiv:2309.15025. Retrieved from https:\/\/arxiv.org\/abs\/2309.15025"},{"key":"e_1_3_2_220_2","first-page":"18952","volume-title":"Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914)","author":"Shi Dan","year":"2024","unstructured":"Dan Shi, Chaobin You, Jiantao Huang, Taihao Li, and Deyi Xiong. 2024. CORECODE: A common sense annotated dialogue dataset with benchmark tasks for chinese large language models. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914). Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.), AAAI Press, 18952\u201318960. DOI: 10.1609\/AAAI.V38I17.29861"},{"key":"e_1_3_2_221_2","doi-asserted-by":"crossref","unstructured":"Jieke Shi Zhou Yang and David Lo. 2024. Efficient and green large language models for software engineering: Vision and the road ahead. arXiv:2404.04566. Retrieved from https:\/\/arxiv.org\/abs\/2404.04566","DOI":"10.1145\/3708525"},{"key":"e_1_3_2_222_2","unstructured":"Taiwei Shi Zhuoer Wang Longqi Yang Ying-Chun Lin Zexue He Mengting Wan Pei Zhou Sujay Kumar Jauhar Xiaofeng Xu Xia Song and Jennifer Neville. 2024a. WildFeedback: Aligning LLMs with in-situ user interactions and feedback. arXiv:2408.15549. Retrieved from https:\/\/arxiv.org\/abs\/2408.15549"},{"key":"e_1_3_2_223_2","first-page":"60","volume-title":"Proceedings of the 48th Euromicro Conference on Software Engineering and Advanced Applications (SEAA \u201922)","author":"Shivashankar Karthik","year":"2022","unstructured":"Karthik Shivashankar and Antonio Martini. 2022. Maintainability challenges in ML: A systematic literature review. In Proceedings of the 48th Euromicro Conference on Software Engineering and Advanced Applications (SEAA \u201922). Gustavo Marrero Callic\u00f3, Regina Hebig, and Andreas Wortmann (Eds.), IEEE, 60\u201367. DOI: 10.1109\/SEAA56994.2022.00018"},{"key":"e_1_3_2_224_2","volume-title":"Proceedings of the 11th International Conference on Learning Representations (ICLR \u201923)","author":"Si Chenglei","year":"2023","unstructured":"Chenglei Si, Zhe Gan, Zhengyuan Yang, Shuohang Wang, Jianfeng Wang, Jordan L. Boyd-Graber, and Lijuan Wang. 2023. Prompting GPT-3 to be reliable. In Proceedings of the 11th International Conference on Learning Representations (ICLR \u201923). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=98p5x51L5af"},{"key":"e_1_3_2_225_2","unstructured":"Significant-Gravitas. 2024. AutoGPT. Retrieved November 11 2024 from https:\/\/github.com\/Significant-Gravitas\/AutoGPT"},{"key":"e_1_3_2_226_2","unstructured":"Chandan Singh Jeevana Priya Inala Michel Galley Rich Caruana and Jianfeng Gao. 2024. Rethinking interpretability in the era of large language models. arXiv:2402.01761. Retrieved from https:\/\/arxiv.org\/abs\/2402.01761"},{"key":"e_1_3_2_227_2","unstructured":"Smartkeyss. 2024. RCE in Python NLTK (CVE-2024-39705). Retrieved July 5 2024 from https:\/\/www.vicarius.io\/vsociety\/posts\/rce-in-python-nltk-cve-2024-39705-39706"},{"key":"e_1_3_2_228_2","unstructured":"Tyler Sorensen and Heidy Khlaaf. 2024. LeftoverLocals: Listening to LLM responses through leaked GPU local memory. arXiv:2401.16603. Retrieved from https:\/\/arxiv.org\/abs\/2401.16603"},{"key":"e_1_3_2_229_2","unstructured":"Sriram Parthasarathy. 2024. Is llama 3.1 really open source? Retrieved September 25 2024 from https:\/\/medium.com\/gptalk\/is-llama-3-1-really-open-source-73ac220f9aa2"},{"key":"e_1_3_2_230_2","first-page":"44:1","volume-title":"Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924)","author":"Stalnaker Trevor","year":"2024","unstructured":"Trevor Stalnaker, Nathan Wintersgill, Oscar Chaparro, Massimiliano Di Penta, Daniel M. Germ\u00e1n, and Denys Poshyvanyk. 2024. BOMs Away! Inside the minds of stakeholders: A comprehensive study of bills of materials for software systems. In Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924). ACM, New York, NY, 44:1\u201344:13. DOI: 10.1145\/3597503.3623347"},{"key":"e_1_3_2_231_2","doi-asserted-by":"crossref","first-page":"3524","DOI":"10.18653\/v1\/2022.acl-long.247","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL \u201922)","author":"Steed Ryan","year":"2022","unstructured":"Ryan Steed, Swetasudha Panda, Ari Kobren, and Michael L. Wick. 2022. Upstream mitigation is not all you need: Testing the bias transfer hypothesis in pre-trained language models. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (ACL \u201922). Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (Eds.), Association for Computational Linguistics, 3524\u20133542. DOI: 10.18653\/V1\/2022.ACL-LONG.247"},{"key":"e_1_3_2_232_2","unstructured":"Dongxun Su Yanjie Zhao Xinyi Hou Shenao Wang and Haoyu Wang. 2024. GPT store mining and analysis. arXiv:2405.10210. Retrieved from https:\/\/arxiv.org\/abs\/2405.10210"},{"key":"e_1_3_2_233_2","unstructured":"Chuanneng Sun Songjun Huang and Dario Pompili. 2024b. LLM-based multi-agent reinforcement learning: Current and future directions. arXiv:2405.11106. Retrieved from https:\/\/arxiv.org\/abs\/2405.11106"},{"key":"e_1_3_2_234_2","first-page":"19053","volume-title":"Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 38th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914)","author":"Sun Liangtai","year":"2024","unstructured":"Liangtai Sun, Yang Han, Zihan Zhao, Da Ma, Zhennan Shen, Baocai Chen, Lu Chen, and Kai Yu. 2024. SciEval: A multi-level large language model evaluation benchmark for scientific research. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 38th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914). Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.), AAAI Press, 19053\u201319061. DOI: 10.1609\/AAAI.V38I17.29872"},{"key":"e_1_3_2_235_2","unstructured":"Lichao Sun Yue Huang Haoran Wang Siyuan Wu Qihui Zhang Chujie Gao Yixin Huang Wenhan Lyu Yixuan Zhang Xiner Li et al. 2024. TrustLLM: Trustworthiness in large language models. arXiv:2401.05561."},{"key":"e_1_3_2_236_2","unstructured":"Yashar Talebirad and Amirhossein Nadiri. 2023. Multi-agent collaboration: Harnessing the power of intelligent LLM agents. arXiv:2306.03314. Retrieved from https:\/\/arxiv.org\/abs\/2306.03314"},{"key":"e_1_3_2_237_2","first-page":"21619","volume-title":"Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914)","author":"Tan Zhen","year":"2024","unstructured":"Zhen Tan, Tianlong Chen, Zhenyu Zhang, and Huan Liu. 2024. Sparsity-guided holistic explanation for LLMs with interpretable inference-time intervention. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914). Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.), AAAI Press, 21619\u201321627. DOI: 10.1609\/AAAI.V38I19.30160"},{"key":"e_1_3_2_238_2","first-page":"6275","volume-title":"Proceedings of the 32nd USENIX Security Symposium (USENIX Security \u201923)USENIX Association","author":"Taneja Hritvik","year":"2023","unstructured":"Hritvik Taneja, Jason Kim, Jie Jeff Xu, Stephan van Schaik, Daniel Genkin, and Yuval Yarom. 2023. Hot pixels: Frequency, power, and temperature attacks on GPUs and Arm SoCs. In Proceedings of the 32nd USENIX Security Symposium (USENIX Security \u201923). Joseph A. Calandrino and Carmela Troncoso (Eds.), USENIX Association, 6275\u20136292. Retrieved from https:\/\/www.usenix.org\/conference\/usenixsecurity23\/presentation\/taneja"},{"key":"e_1_3_2_239_2","first-page":"1964","volume-title":"Proceedings of the 35th IEEE International Conference on Data Engineering (ICDE \u201919)","author":"Tang MingJie","year":"2019","unstructured":"MingJie Tang, Saisai Shao, Weiqing Yang, Yanbo Liang, Yongyang Yu, Bikas Saha, and Dongjoon Hyun. 2019. SAC: A system for big data lineage tracking. In Proceedings of the 35th IEEE International Conference on Data Engineering (ICDE \u201919). IEEE, 1964\u20131967. DOI: 10.1109\/ICDE.2019.00215"},{"key":"e_1_3_2_240_2","first-page":"512","volume-title":"Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER \u201924)IEEE","author":"Taraghi Mina","year":"2024","unstructured":"Mina Taraghi, Gianolli Dorcelus, Armstrong Foundjem, Florian Tambon, and Foutse Khomh. 2024. Deep learning model reuse in the HuggingFace community: Challenges, benefit and trends. In Proceedings of the IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER \u201924). IEEE, 512\u2013523. DOI: 10.1109\/SANER60148.2024.00059"},{"key":"e_1_3_2_241_2","unstructured":"TensorFlow. 2024. TensorFlow. Retrieved March 28 2024 from https:\/\/github.com\/tensorflow\/tensorflow"},{"key":"e_1_3_2_242_2","unstructured":"Lionel Nganyewou Tidjon and Foutse Khomh. 2022. Never trust always verify: A roadmap for trustworthy AI? arXiv:2206.11981. Retrieved from https:\/\/arxiv.org\/abs\/2206.11981"},{"key":"e_1_3_2_243_2","unstructured":"Tom Bonner. 2023. Models Are Code: A Deep Dive into Security Risks in TensorFlow and Keras. Retrieved September 23 2024 from https:\/\/hiddenlayer.com\/research\/models-are-code\/"},{"key":"e_1_3_2_244_2","unstructured":"S. M. Towhidul Islam Tonmoy S. M. Mehedi Zaman Vinija Jain Anku Rani Vipula Rawte Aman Chadha and Amitava Das. 2024. A comprehensive survey of hallucination mitigation techniques in large language models. arXiv:2401.01313. Retrieved from https:\/\/arxiv.org\/abs\/2401.01313"},{"key":"e_1_3_2_245_2","unstructured":"Hugo Touvron Thibaut Lavril Gautier Izacard Xavier Martinet Marie-Anne Lachaux Timoth\u00e9e Lacroix Baptiste Rozi\u00e8re Naman Goyal Eric Hambro Faisal Azhar Aur\u00e9lien Rodriguez Armand Joulin Edouard Grave and Guillaume Lample. 2023. LLaMA: Open and efficient foundation language models. arXiv:2302.13971. Retrieved from https:\/\/arxiv.org\/abs\/2302.13971"},{"key":"e_1_3_2_246_2","doi-asserted-by":"publisher","DOI":"10.1007\/S10515-009-0054-Z"},{"key":"e_1_3_2_247_2","unstructured":"Valdez Ladd. 2024. Generative AI Exposure to PyTorch\u2019s distributed RPC and CVE-2024\u20135480 Vulnerability. Retrieved September 20 2024 from https:\/\/medium.com\/atoracle_43885\/generative-ai-exposure-topytorchs-distributed-rpc-system-and-cve-2024-5480-vulnerability-663bfd604913"},{"key":"e_1_3_2_248_2","volume-title":"Advances in Neural Information Processing Systems 36: Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Valmeekam Karthik","year":"2023","unstructured":"Karthik Valmeekam, Matthew Marquez, Alberto Olmo Hernandez, Sarath Sreedharan, and Subbarao Kambhampati. 2023. PlanBench: An extensible benchmark for evaluating large language models on planning and reasoning about change. In Advances in Neural Information Processing Systems 36: Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/7a92bcdede88c7afd108072faf5485c8-Abstract-Datasets_and_Benchmarks.html"},{"key":"e_1_3_2_249_2","doi-asserted-by":"crossref","first-page":"731","DOI":"10.1145\/2642937.2643013","volume-title":"Proceedings of the ACM\/IEEE International Conference on Automated Software Engineering (ASE \u201914)","author":"Burg Sander van der","year":"2014","unstructured":"Sander van der Burg, Eelco Dolstra, Shane McIntosh, Julius Davies, Daniel M. Germ\u00e1n, and Armijn Hemel. 2014. Tracing software build processes to uncover license compliance inconsistencies. In Proceedings of the ACM\/IEEE International Conference on Automated Software Engineering (ASE \u201914). Ivica Crnkovic, Marsha Chechik, and Paul Gr\u00fcnbacher (Eds.), ACM, New York, NY, 731\u2013742. DOI: 10.1145\/2642937.2643013"},{"key":"e_1_3_2_250_2","unstructured":"Neeraj Varshney Wenlin Yao Hongming Zhang Jianshu Chen and Dong Yu. 2023. A stitch in time saves nine: Detecting and mitigating hallucinations of LLMs by validating low-confidence generation. arXiv:2307.03987. Retrieved from https:\/\/arxiv.org\/abs\/2307.03987"},{"key":"e_1_3_2_251_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCAD.2021.3098740"},{"key":"e_1_3_2_252_2","first-page":"772","volume-title":"Proceedings of the 37th IEEE\/ACM International Conference on Software Engineering (ICSE \u201915)","volume":"2","author":"Vendome Christopher","year":"2015","unstructured":"Christopher Vendome. 2015. A large scale study of license usage on GitHub. In Proceedings of the 37th IEEE\/ACM International Conference on Software Engineering (ICSE \u201915). Antonia Bertolino, Gerardo Canfora, and Sebastian G. Elbaum (Eds.), Vol. 2, IEEE Computer Society, 772\u2013774. DOI: 10.1109\/ICSE.2015.245"},{"key":"e_1_3_2_253_2","doi-asserted-by":"publisher","DOI":"10.1007\/S10664-016-9438-4"},{"key":"e_1_3_2_254_2","first-page":"811","volume-title":"Proceedings of the 38th International Conference on Software Engineering (ICSE \u201916)","author":"Vendome Christopher","year":"2016","unstructured":"Christopher Vendome and Denys Poshyvanyk. 2016. Assisting developers with license compliance. In Proceedings of the 38th International Conference on Software Engineering (ICSE \u201916). Laura K. Dillon, Willem Visser, and Laurie A. Williams (Eds.), ACM, New York, NY, 811\u2013814. DOI: 10.1145\/2889160.2889259"},{"key":"e_1_3_2_255_2","first-page":"218","volume-title":"Proceedings of the IEEE 23rd International Conference on Program Comprehension (ICPC \u201915)","author":"Vendome Christopher","year":"2015","unstructured":"Christopher Vendome, Mario Linares V\u00e1squez, Gabriele Bavota, Massimiliano Di Penta, Daniel M. Germ\u00e1n, and Denys Poshyvanyk. 2015. License usage and changes: A large-scale study of Java projects on GitHub. In Proceedings of the IEEE 23rd International Conference on Program Comprehension (ICPC \u201915). Andrea De Lucia, Christian Bird, and Rocco Oliveto (Eds.), IEEE Computer Society, 218\u2013228. DOI: 10.1109\/ICPC.2015.32"},{"key":"e_1_3_2_256_2","first-page":"118","volume-title":"Proceedings of the 39th International Conference on Software Engineering (ICSE \u201917)","author":"Vendome Christopher","year":"2017","unstructured":"Christopher Vendome, Mario Linares V\u00e1squez, Gabriele Bavota, Massimiliano Di Penta, Daniel M. Germ\u00e1n, and Denys Poshyvanyk. 2017b. Machine learning-based detection of open source license exceptions. In Proceedings of the 39th International Conference on Software Engineering (ICSE \u201917). Sebasti\u00e1n Uchitel, Alessandro Orso, and Martin P. Robillard (Eds.), IEEE\/ACM, New York, NY, 118\u2013129. DOI: 10.1109\/ICSE.2017.19"},{"key":"e_1_3_2_257_2","doi-asserted-by":"publisher","DOI":"10.1145\/3377454"},{"key":"e_1_3_2_258_2","unstructured":"Chaozheng Wang Zongjie Li Cuiyun Gao Wenxuan Wang Ting Peng Hailiang Huang Yuetang Deng Shuai Wang and Michael R. Lyu. 2024. Exploring multi-lingual bias of large code models in code generation. arXiv:2404.19368. Retrieved from https:\/\/arxiv.org\/abs\/2404.19368"},{"key":"e_1_3_2_259_2","first-page":"2509","volume-title":"Proceedings of the IEEE International Conference on Big Data (IEEE BigData \u201915)IEEE Computer Society","author":"Wang Jianwu","year":"2015","unstructured":"Jianwu Wang, Daniel Crawl, Shweta Purawat, Mai H. Nguyen, and Ilkay Altintas. 2015. Big data provenance: Challenges, state of the art and opportunities. In Proceedings of the IEEE International Conference on Big Data (IEEE BigData \u201915). IEEE Computer Society, 2509\u20132516. DOI: 10.1109\/BIGDATA.2015.7364047"},{"key":"e_1_3_2_260_2","unstructured":"Jiongxiao Wang Zichen Liu Keun Hee Park Muhao Chen and Chaowei Xiao. 2023. Adversarial demonstration attacks on large language models. arXiv:2305.14950. Retrieved from https:\/\/arxiv.org\/abs\/2305.14950"},{"key":"e_1_3_2_261_2","first-page":"472","volume-title":"Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering: Companion Proceedings (ICSE Companion \u201924)","author":"Wang Xinchen","year":"2024","unstructured":"Xinchen Wang, Ruida Hu, Cuiyun Gao, Xin-Cheng Wen, Yujia Chen, and Qing Liao. 2024. ReposVul: A repository-level high-quality vulnerability dataset. In Proceedings of the IEEE\/ACM 46th International Conference on Software Engineering: Companion Proceedings (ICSE Companion \u201924). ACM, New York, NY, 472\u2013483. DOI: 10.1145\/3639478.3647634"},{"key":"e_1_3_2_262_2","unstructured":"Yuntao Wang Yanghe Pan Quan Zhao Yi Deng Zhou Su Linkang Du and Tom H. Luan. 2024. Large model agents: State-of-the-Art cooperation paradigms Security and privacy and future trends. arXiv:2409.14457. Retrieved from https:\/\/arxiv.org\/abs\/2409.14457"},{"key":"e_1_3_2_263_2","unstructured":"Yanlin Wang Wanjun Zhong Yanxian Huang Ensheng Shi Min Yang Jiachi Chen Hui Li Yuchi Ma Qianxiang Wang and Zibin Zheng. 2024. Agents in software engineering: Survey landscape and vision. arXiv:2409.09030. Retrieved from https:\/\/arxiv.org\/abs\/2409.09030"},{"key":"e_1_3_2_264_2","doi-asserted-by":"crossref","first-page":"214","DOI":"10.1145\/3531146.3533088","volume-title":"Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT \u201922)","author":"Weidinger Laura","year":"2022","unstructured":"Laura Weidinger, Jonathan Uesato, Maribeth Rauh, Conor Griffin, Po-Sen Huang, John Mellor, Amelia Glaese, Myra Cheng, Borja Balle, Atoosa Kasirzadeh, et al. 2022. Taxonomy of risks posed by language models. In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT \u201922). ACM, New York, NY, 214\u2013229. DOI: 10.1145\/3531146.3533088"},{"key":"e_1_3_2_265_2","unstructured":"Weights & Biases. 2024. Weights & Biases: The AI Developer Platform. Retrieved September 23 2024 from https:\/\/wandb.ai\/site"},{"key":"e_1_3_2_266_2","first-page":"2447","volume-title":"Proceedings of the International Conference on Findings of the Association for Computational Linguistics (EMNLP \u201921)","author":"Welbl Johannes","year":"2021","unstructured":"Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. 2021. Challenges in detoxifying language models. In Proceedings of the International Conference on Findings of the Association for Computational Linguistics (EMNLP \u201921). Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (Eds.), Association for Computational Linguistics, 2447\u20132469. DOI: 10.18653\/V1\/2021.FINDINGS-EMNLP.210"},{"key":"e_1_3_2_267_2","volume-title":"Proceedings of the 41st International Conference on Machine Learning (ICML \u201924)","author":"Wolf Yotam","year":"2024","unstructured":"Yotam Wolf, Noam Wies, Oshri Avnery, Yoav Levine, and Amnon Shashua. 2024. Fundamental limitations of alignment in large language models. In Proceedings of the 41st International Conference on Machine Learning (ICML \u201924). OpenReview.net. Retrieved from https:\/\/openreview.net\/forum?id=KXsUCgn9Ks"},{"key":"e_1_3_2_268_2","doi-asserted-by":"publisher","DOI":"10.1145\/3571852"},{"key":"e_1_3_2_269_2","unstructured":"Fangzhou Wu Ning Zhang Somesh Jha Patrick D. McDaniel and Chaowei Xiao. 2024. A new era in LLM security: Exploring security concerns in real-world LLM-based systems. arXiv:2402.18649. Retrieved from https:\/\/arxiv.org\/abs\/2402.18649"},{"key":"e_1_3_2_270_2","doi-asserted-by":"publisher","DOI":"10.1007\/S10664-016-9487-8"},{"key":"e_1_3_2_271_2","unstructured":"Zihui Wu Haichang Gao Jianping He and Ping Wang. 2024. The Dark Side of function calling: Pathways to jailbreaking large language models. arXiv:2407.17915. Retrieved from https:\/\/arxiv.org\/abs\/2407.17915"},{"key":"e_1_3_2_272_2","unstructured":"Zhiheng Xi Wenxiang Chen Xin Guo Wei He Yiwen Ding Boyang Hong Ming Zhang Junzhe Wang Senjie Jin Enyu Zhou et al. 2023. The rise and potential of large language model based agents: A survey. arXiv:2309.07864. Retrieved from https:\/\/arxiv.org\/abs\/2309.07864"},{"key":"e_1_3_2_273_2","first-page":"2630","volume-title":"Proceedings of the 45th IEEE\/ACM International Conference on Software Engineering (ICSE \u201923)IEEE","author":"Xia Boming","year":"2023","unstructured":"Boming Xia, Tingting Bi, Zhenchang Xing, Qinghua Lu, and Liming Zhu. 2023. An empirical study on software bill of materials: Where We stand and the road ahead. In Proceedings of the 45th IEEE\/ACM International Conference on Software Engineering (ICSE \u201923). IEEE, 2630\u20132642. DOI: 10.1109\/ICSE48619.2023.00219"},{"key":"e_1_3_2_274_2","first-page":"2639","volume-title":"Proceedings of the International Conference on Management of Data (SIGMOD \u201921)","author":"Xin Doris","year":"2021","unstructured":"Doris Xin, Hui Miao, Aditya G. Parameswaran, and Neoklis Polyzotis. 2021. Production machine learning pipelines: Empirical analysis and optimization opportunities. In Proceedings of the International Conference on Management of Data (SIGMOD \u201921). Guoliang Li, Zhanhuai Li, Stratos Idreos, and Divesh Srivastava (Eds.), ACM, New York, NY, 2639\u20132652. DOI: 10.1145\/3448016.3457566"},{"key":"e_1_3_2_275_2","unstructured":"Huiyu Xu Wenhui Zhang Zhibo Wang Feng Xiao Rui Zheng Yunhe Feng Zhongjie Ba and Kui Ren. 2024. RedAgent: Red teaming large language models with context-aware autonomous language agent. arXiv:2407.16667. Retrieved from https:\/\/arxiv.org\/abs\/2407.16667"},{"key":"e_1_3_2_276_2","first-page":"11547","volume-title":"Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI \u201922), 34th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201922), The 12th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201922)","author":"Xu Runxin","year":"2022","unstructured":"Runxin Xu, Fuli Luo, Chengyu Wang, Baobao Chang, Jun Huang, Songfang Huang, and Fei Huang. 2022. From dense to sparse: Contrastive pruning for better pre-trained language model compression. In Proceedings of the 36th AAAI Conference on Artificial Intelligence (AAAI \u201922), 34th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201922), The 12th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201922). AAAI Press, 11547\u201311555. DOI: 10.1609\/AAAI.V36I10.21408"},{"key":"e_1_3_2_277_2","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1109\/SP.2016.10","volume-title":"Proceedings of the IEEE Symposium on Security and Privacy (SP \u201916)","author":"Yang Kaiyuan","year":"2016","unstructured":"Kaiyuan Yang, Matthew Hicks, Qing Dong, Todd M. Austin, and Dennis Sylvester. 2016. A2: Analog malicious hardware. In Proceedings of the IEEE Symposium on Security and Privacy (SP \u201916). IEEE Computer Society, 18\u201337. DOI: 10.1109\/SP.2016.10"},{"key":"e_1_3_2_278_2","series-title":"Proceedings of Machine Learning Research","first-page":"39299","volume-title":"Proceedings of the International Conference on Machine Learning (ICML \u201923)","volume":"202","author":"Yang Ziqing","year":"2023","unstructured":"Ziqing Yang, Xinlei He, Zheng Li, Michael Backes, Mathias Humbert, Pascal Berrang, and Yang Zhang. 2023. Data poisoning attacks against multimodal encoders. In Proceedings of the International Conference on Machine Learning (ICML \u201923). Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (Eds.), Proceedings of Machine Learning Research, Vol. 202, PMLR, 39299\u201339313. Retrieved from https:\/\/proceedings.mlr.press\/v202\/yang23f.html"},{"key":"e_1_3_2_279_2","first-page":"72:1","volume-title":"Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924)","author":"Yang Zhou","year":"2024","unstructured":"Zhou Yang, Zhipeng Zhao, Chenyu Wang, Jieke Shi, Dongsun Kim, DongGyun Han, and David Lo. 2024. Unveiling memorization in code models. In Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924). ACM, New York, NY, 72:1\u201372:13. DOI: 10.1145\/3597503.3639074"},{"key":"e_1_3_2_280_2","unstructured":"Yifan Yao Jinhao Duan Kaidi Xu Yuanfang Cai Eric Sun and Yue Zhang. 2023. A survey on large language model (LLM) security and privacy: The good the bad and the ugly. arXiv:2312.02003. Retrieved from https:\/\/arxiv.org\/abs\/2312.02003"},{"key":"e_1_3_2_281_2","unstructured":"Sibo Yi Yule Liu Zhen Sun Tianshuo Cong Xinlei He Jiaxing Song Ke Xu and Qi Li. 2024. Jailbreak attacks and defenses against large language models: A survey. arXiv:2407.04295."},{"key":"e_1_3_2_282_2","first-page":"6032","volume-title":"Proceedings of the Findings of the Association for Computational Linguistics (ACL \u201923)Association for Computational Linguistics","author":"Yu Charles","year":"2023","unstructured":"Charles Yu, Sullam Jeoung, Anish Kasi, Pengfei Yu, and Heng Ji. 2023. Unlearning bias in language models by partitioning gradients. In Proceedings of the Findings of the Association for Computational Linguistics (ACL \u201923). Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki (Eds.), Association for Computational Linguistics, 6032\u20136048. DOI: 10.18653\/V1\/2023.FINDINGS-ACL.375"},{"key":"e_1_3_2_283_2","first-page":"37:1","volume-title":"Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924)","author":"Yu Hao","year":"2024","unstructured":"Hao Yu, Bo Shen, Dezhi Ran, Jiaxin Zhang, Qi Zhang, Yuchi Ma, Guangtai Liang, Ying Li, Qianxiang Wang, and Tao Xie. 2024. CoderEval: A benchmark of pragmatic code generation with generative pre-trained models. In Proceedings of the 46th IEEE\/ACM International Conference on Software Engineering (ICSE \u201924). ACM, New York, NY, 37:1\u201337:12. DOI: 10.1145\/3597503.3623316"},{"key":"e_1_3_2_284_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Yu Yue","year":"2023","unstructured":"Yue Yu, Yuchen Zhuang, Jieyu Zhang, Yu Meng, Alexander J. Ratner, Ranjay Krishna, Jiaming Shen, and Chao Zhang. 2023. Large language model as attributed training data generator: A tale of diversity and Bias. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/ae9500c4f5607caf2eff033c67daa9d7-Abstract-Datasets_and_Benchmarks.html"},{"key":"e_1_3_2_285_2","unstructured":"Yifan Zeng Yiran Wu Xiao Zhang Huazheng Wang and Qingyun Wu. 2024. AutoDefense: Multi-agent LLM defense against jailbreak attacks. arXiv:2403.04783. Retrieved from https:\/\/arxiv.org\/abs\/2403.04783"},{"key":"e_1_3_2_286_2","doi-asserted-by":"crossref","first-page":"1577","DOI":"10.1145\/3581783.3612108","volume-title":"Proceedings of the 31st ACM International Conference on Multimedia (MM \u201923)","author":"Zhai Shengfang","year":"2023","unstructured":"Shengfang Zhai, Yinpeng Dong, Qingni Shen, Shi Pu, Yuejian Fang, and Hang Su. 2023. Text-to-image diffusion models can be easily backdoored through multimodal data poisoning. In Proceedings of the 31st ACM International Conference on Multimedia (MM \u201923). Abdulmotaleb El-Saddik, Tao Mei, Rita Cucchiara, Marco Bertini, Diana Patricia Tobon Vallejo, Pradeep K. Atrey, and M. Shamim Hossain (Eds.), ACM, New York, NY, 1577\u20131587. DOI: 10.1145\/3581783.3612108"},{"key":"e_1_3_2_287_2","doi-asserted-by":"crossref","first-page":"1440","DOI":"10.1109\/SP46214.2022.9833773","volume-title":"Proceedings of the 43rd IEEE Symposium on Security and Privacy (SP \u201922)IEEE","author":"Zhan Zihao","year":"2022","unstructured":"Zihao Zhan, Zhenkai Zhang, Sisheng Liang, Fan Yao, and Xenofon D. Koutsoukos. 2022. Graphics peeping unit: Exploiting EM side-channel information of GPUs to eavesdrop on your neighbors. In Proceedings of the 43rd IEEE Symposium on Security and Privacy (SP \u201922). IEEE, 1440\u20131457. DOI: 10.1109\/SP46214.2022.9833773"},{"key":"e_1_3_2_288_2","unstructured":"Boyang Zhang Yicong Tan Yun Shen Ahmed Salem Michael Backes Savvas Zannettou and Yang Zhang. 2024. Breaking agents: Compromising autonomous LLM agents through malfunction amplification. arXiv:2407.20859. Retrieved from https:\/\/arxiv.org\/abs\/2407.20859"},{"key":"e_1_3_2_289_2","unstructured":"Hangfan Zhang Zhimeng Guo Huaisheng Zhu Bochuan Cao Lu Lin Jinyuan Jia Jinghui Chen and Dinghao Wu. 2023. On the safety of open-sourced large language models: Does alignment really prevent them from being misused? arXiv:2310.01581. Retrieved from https:\/\/arxiv.org\/abs\/2310.01581"},{"key":"e_1_3_2_290_2","first-page":"21779","volume-title":"Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914)","author":"Zhang Jiang","year":"2024","unstructured":"Jiang Zhang, Qiong Wu, Yiming Xu, Cheng Cao, Zheng Du, and Konstantinos Psounis. 2024. Efficient toxic content detection by bootstrapping and distilling large language models. In Proceedings of the 38th AAAI Conference on Artificial Intelligence (AAAI \u201924), 36th Conference on Innovative Applications of Artificial Intelligence (IAAI \u201924), 14th Symposium on Educational Advances in Artificial Intelligence (EAAI \u201914). Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (Eds.), AAAI Press, 21779\u201321787. DOI: 10.1609\/AAAI.V38I19.30178"},{"key":"e_1_3_2_291_2","first-page":"191","volume-title":"Proceedings of the 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201923)","author":"Zhang Lyuye","year":"2023","unstructured":"Lyuye Zhang, Chengwei Liu, Sen Chen, Zhengzi Xu, Lingling Fan, Lida Zhao, Yiran Zhang, and Yang Liu. 2023. Mitigating persistence of open-source vulnerabilities in Maven ecosystem. In Proceedings of the 38th IEEE\/ACM International Conference on Automated Software Engineering (ASE \u201923). IEEE, 191\u2013203. DOI: 10.1109\/ASE56229.2023.00058"},{"key":"e_1_3_2_292_2","doi-asserted-by":"crossref","first-page":"4416","DOI":"10.18653\/v1\/2020.acl-main.406","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL \u201920)Association for Computational Linguistics","author":"Zhang Yi","year":"2020","unstructured":"Yi Zhang, Zachary G. Ives, and Dan Roth. 2020. \u201cWho said it, and why?\u201d Provenance for natural language claims. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics (ACL \u201920). Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault (Eds.), Association for Computational Linguistics, 4416\u20134426. DOI: 10.18653\/V1\/2020.ACL-MAIN.406"},{"key":"e_1_3_2_293_2","unstructured":"Yue Zhang Yafu Li Leyang Cui Deng Cai Lemao Liu Tingchen Fu Xinting Huang Enbo Zhao Yu Zhang Yulong Chen et al. 2023. Siren\u2019s song in the AI Ocean: A Survey on Hallucination in Large Language Models. arXiv:2309.01219. Retrieved from https:\/\/arxiv.org\/abs\/2309.01219"},{"key":"e_1_3_2_294_2","doi-asserted-by":"crossref","unstructured":"Zhibo Zhang Wuxia Bai Yuxi Li Mark Huasong Meng Kailong Wang Ling Shi Li Li Jun Wang and Haoyu Wang. 2024. GlitchProber: Advancing effective detection and mitigation of glitch tokens in large language models. arXiv:2408.04905. Retrieved from https:\/\/arxiv.org\/abs\/2408.04905","DOI":"10.1145\/3691620.3695060"},{"key":"e_1_3_2_295_2","doi-asserted-by":"publisher","DOI":"10.1109\/TDSC.2018.2864733"},{"key":"e_1_3_2_296_2","doi-asserted-by":"crossref","unstructured":"Jian Zhao Shenao Wang Yanjie Zhao Xinyi Hou Kailong Wang Peiming Gao Yuanchao Zhang Chen Wei and Haoyu Wang. 2024. Models are codes: Towards measuring malicious code poisoning attacks on pre-trained model hubs. arXiv:2409.09368. Retrieved from https:\/\/arxiv.org\/abs\/2409.09368","DOI":"10.1145\/3691620.3695271"},{"key":"e_1_3_2_297_2","doi-asserted-by":"crossref","unstructured":"Yanjie Zhao Xinyi Hou Shenao Wang and Haoyu Wang. 2024a. LLM app store analysis: A vision and roadmap. arXiv:2404.12737. Retrieved from https:\/\/arxiv.org\/abs\/2404.12737","DOI":"10.1145\/3708530"},{"key":"e_1_3_2_298_2","volume-title":"Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923)","author":"Zhao Yunqing","year":"2023","unstructured":"Yunqing Zhao, Tianyu Pang, Chao Du, Xiao Yang, Chongxuan Li, Ngai-Man Cheung, and Min Lin. 2023. On evaluating adversarial robustness of large vision-language models. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS \u201923). Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). Retrieved from http:\/\/papers.nips.cc\/paper_files\/paper\/2023\/hash\/a97b58c4f7551053b0512f92244b0810-Abstract-Conference.html"},{"key":"e_1_3_2_299_2","unstructured":"Weikang Zhou Xiao Wang Limao Xiong Han Xia Yingshuang Gu Mingxu Chai Fukang Zhu Caishuang Huang Shihan Dou Zhiheng Xi et al. 2024. EasyJailbreak: A unified framework for jailbreaking large language models. arXiv:2403.12171. Retrieved from https:\/\/arxiv.org\/abs\/2403.12171"},{"key":"e_1_3_2_300_2","first-page":"1450","volume-title":"Proceedings of the IEEE Symposium on Security and Privacy (SP \u201920)IEEE","author":"Zhu Jianping","year":"2020","unstructured":"Jianping Zhu, Rui Hou, XiaoFeng Wang, Wenhao Wang, Jiangfeng Cao, Boyan Zhao, Zhongpu Wang, Yuhui Zhang, Jiameng Ying, Lixin Zhang, et al. 2020. Enabling rack-scale confidential computing using heterogeneous trusted execution environment. In Proceedings of the IEEE Symposium on Security and Privacy (SP \u201920). IEEE, 1450\u20131465. DOI: 10.1109\/SP40000.2020.00054"},{"key":"e_1_3_2_301_2","first-page":"1","volume-title":"Proceedings of the General Purpose GPUs (GPGPU at PPoPP)","author":"Zhu Zhiting","year":"2017","unstructured":"Zhiting Zhu, Sangman Kim, Yuri Rozhanski, Yige Hu, Emmett Witchel, and Mark Silberstein. 2017. Understanding the security of discrete GPUs. In Proceedings of the General Purpose GPUs (GPGPU at PPoPP). ACM, New York, NY, 1\u201311. DOI: 10.1145\/3038228.3038233"},{"key":"e_1_3_2_302_2","unstructured":"Terry Yue Zhuo Yujin Huang Chunyang Chen and Zhenchang Xing. 2023. Red teaming ChatGPT via jailbreaking: Bias robustness reliability and toxicity. arXiv:2301.12867. Retrieved from https:\/\/arxiv.org\/abs\/2301.12867"},{"key":"e_1_3_2_303_2","unstructured":"Andy Zou Long Phan Sarah Chen James Campbell Phillip Guo Richard Ren Alexander Pan Xuwang Yin Mantas Mazeika Ann-Kathrin Dombrowski et al. 2023. Representation engineering: A top-down approach to AI transparency. arXiv:2310.01405. Retrieved from https:\/\/arxiv.org\/abs\/2310.01405"},{"key":"e_1_3_2_304_2","unstructured":"Andy Zou Zifan Wang J. Zico Kolter and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv:2307.15043. Retrieved from https:\/\/arxiv.org\/abs\/2307.15043"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3708531","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3708531","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T21:17:46Z","timestamp":1750281466000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3708531"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,24]]},"references-count":303,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3708531"],"URL":"https:\/\/doi.org\/10.1145\/3708531","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,24]]},"assertion":[{"value":"2024-04-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-20","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-05-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}