{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,28]],"date-time":"2026-03-28T00:47:29Z","timestamp":1774658849907,"version":"3.50.1"},"reference-count":91,"publisher":"Association for Computing Machinery (ACM)","issue":"11","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,7]]},"abstract":"<jats:p>Large Language Models (LLMs) have become integral to numerous domains, significantly advancing applications in data management, mining, and analysis. Their profound capabilities in processing and interpreting complex language data, however, bring to light pressing concerns regarding data privacy, especially the risk of unintentional training data leakage. Despite the critical nature of this issue, there has been no existing literature to offer a comprehensive assessment of data privacy risks in LLMs. Addressing this gap, our paper introduces LLM-PBE, a toolkit crafted specifically for the systematic evaluation of data privacy risks in LLMs. LLM-PBE is designed to analyze privacy across the entire lifecycle of LLMs, incorporating diverse attack and defense strategies, and handling various data types and metrics. Through detailed experimentation with multiple LLMs, LLM-PBE facilitates an in-depth exploration of data privacy concerns, shedding light on influential factors such as model size, data characteristics, and evolving temporal dimensions. This study not only enriches the understanding of privacy issues in LLMs but also serves as a vital resource for future research in the field. Aimed at enhancing the breadth of knowledge in this area, the findings, resources, and our full technical report are made available at https:\/\/llm-pbe.github.io\/, providing an open platform for academic and practical advancements in LLM privacy assessment.<\/jats:p>","DOI":"10.14778\/3681954.3681994","type":"journal-article","created":{"date-parts":[[2024,8,30]],"date-time":"2024-08-30T16:23:36Z","timestamp":1725035016000},"page":"3201-3214","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":25,"title":["LLM-PBE: Assessing Data Privacy in Large Language Models"],"prefix":"10.14778","volume":"17","author":[{"given":"Qinbin","family":"Li","sequence":"first","affiliation":[{"name":"University of California, Berkeley"}]},{"given":"Junyuan","family":"Hong","sequence":"additional","affiliation":[{"name":"University of Texas at Austin"}]},{"given":"Chulin","family":"Xie","sequence":"additional","affiliation":[{"name":"University of Illinois Urbana-Champaign"}]},{"given":"Jeffrey","family":"Tan","sequence":"additional","affiliation":[{"name":"University of California, Berkeley"}]},{"given":"Rachel","family":"Xin","sequence":"additional","affiliation":[{"name":"University of California, Berkeley"}]},{"given":"Junyi","family":"Hou","sequence":"additional","affiliation":[{"name":"National University of Singapore"}]},{"given":"Xavier","family":"Yin","sequence":"additional","affiliation":[{"name":"University of California, Berkeley"}]},{"given":"Zhun","family":"Wang","sequence":"additional","affiliation":[{"name":"University of California, Berkeley"}]},{"given":"Dan","family":"Hendrycks","sequence":"additional","affiliation":[{"name":"Center for AI Safety"}]},{"given":"Zhangyang","family":"Wang","sequence":"additional","affiliation":[{"name":"University of Texas at Austin"}]},{"given":"Bo","family":"Li","sequence":"additional","affiliation":[{"name":"University of Chicago"}]},{"given":"Bingsheng","family":"He","sequence":"additional","affiliation":[{"name":"National University of Singapore"}]},{"given":"Dawn","family":"Song","sequence":"additional","affiliation":[{"name":"University of California, Berkeley"}]}],"member":"320","published-online":{"date-parts":[[2024,8,30]]},"reference":[{"key":"e_1_2_1_1_1","unstructured":"2023. https:\/\/news.ycombinator.com\/item?id=34482318 Accessed: 2024-07-16."},{"key":"e_1_2_1_2_1","unstructured":"2023. Jailbreak Chat. https:\/\/www.jailbreakchat.com\/ Accessed: 2024-07-16."},{"key":"e_1_2_1_3_1","unstructured":"2023. Leaked-GPTs. https:\/\/github.com\/friuns2\/Leaked-GPTs Accessed: 2024-07-16."},{"key":"e_1_2_1_4_1","unstructured":"2024. Hugging Face - The AI community building the future. https:\/\/huggingface.co\/. Accessed: 2024-07-16."},{"key":"e_1_2_1_5_1","unstructured":"2024. Together.ai. https:\/\/www.together.ai\/. Accessed: 2024-07-16."},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/2976749.2978318"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1145\/342009.335438"},{"key":"e_1_2_1_8_1","volume-title":"Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics (demonstrations). 54--59","author":"Akbik Alan","year":"2019","unstructured":"Alan Akbik, Tanja Bergmann, Duncan Blythe, Kashif Rasul, Stefan Schweter, and Roland Vollgraf. 2019. FLAIR: An easy-to-use framework for state-of-the-art NLP. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics (demonstrations). 54--59."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3631504.3631518"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5281\/zenodo.5584996"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE.2005.42"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1145\/3543507.3587431"},{"key":"e_1_2_1_13_1","volume-title":"International Conference on Machine Learning. PMLR, 2397--2430","author":"Biderman Stella","year":"2023","unstructured":"Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. 2023. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning. PMLR, 2397--2430."},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP40001.2021.00019"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP46214.2022.9833649"},{"key":"e_1_2_1_16_1","volume-title":"Quantifying memorization across neural language models. arXiv preprint arXiv:2202.07646","author":"Carlini Nicholas","year":"2022","unstructured":"Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2022. Quantifying memorization across neural language models. arXiv preprint arXiv:2202.07646 (2022)."},{"key":"e_1_2_1_17_1","volume-title":"Quantifying Memorization Across Neural Language Models. In The Eleventh International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=TatRHT_1cK Accessed: 2024-07-16","author":"Carlini Nicholas","year":"2023","unstructured":"Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. 2023. Quantifying Memorization Across Neural Language Models. In The Eleventh International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=TatRHT_1cK Accessed: 2024-07-16."},{"key":"e_1_2_1_18_1","volume-title":"The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks. In 28th USENIX Security Symposium, USENIX Security","author":"Carlini Nicholas","year":"2019","unstructured":"Nicholas Carlini, Chang Liu, \u00dalfar Erlingsson, Jernej Kos, and Dawn Song. 2019. The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks. In 28th USENIX Security Symposium, USENIX Security 2019."},{"key":"e_1_2_1_19_1","volume-title":"30th USENIX Security Symposium (USENIX Security 21)","author":"Carlini Nicholas","year":"2021","unstructured":"Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. Extracting training data from large language models. In 30th USENIX Security Symposium (USENIX Security 21). 2633--2650."},{"key":"e_1_2_1_20_1","volume-title":"Neural legal judgment prediction in English. arXiv preprint arXiv:1906.02059","author":"Chalkidis Ilias","year":"2019","unstructured":"Ilias Chalkidis, Ion Androutsopoulos, and Nikolaos Aletras. 2019. Neural legal judgment prediction in English. arXiv preprint arXiv:1906.02059 (2019)."},{"key":"e_1_2_1_21_1","volume-title":"Paragraph-level rationale extraction through regularization: A case study on European court of human rights cases. arXiv preprint arXiv:2103.13084","author":"Chalkidis Ilias","year":"2021","unstructured":"Ilias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapatsanis, Nikolaos Aletras, Ion Androutsopoulos, and Prodromos Malakasiotis. 2021. Paragraph-level rationale extraction through regularization: A case study on European court of human rights cases. arXiv preprint arXiv:2103.13084 (2021)."},{"key":"e_1_2_1_22_1","volume-title":"Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419","author":"Chao Patrick","year":"2023","unstructured":"Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. 2023. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419 (2023)."},{"key":"e_1_2_1_23_1","volume-title":"Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457","author":"Clark Peter","year":"2018","unstructured":"Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457 (2018)."},{"key":"e_1_2_1_24_1","volume-title":"Do Membership Inference Attacks Work on Large Language Models? arXiv preprint arXiv:2402.07841","author":"Duan Michael","year":"2024","unstructured":"Michael Duan, Anshuman Suri, Niloofar Mireshghallah, Sewon Min, Weijia Shi, Luke Zettlemoyer, Yulia Tsvetkov, Yejin Choi, David Evans, and Hannaneh Hajishirzi. 2024. Do Membership Inference Attacks Work on Large Language Models? arXiv preprint arXiv:2402.07841 (2024)."},{"key":"e_1_2_1_25_1","volume-title":"International colloquium on automata, languages, and programming","author":"Dwork Cynthia","unstructured":"Cynthia Dwork. 2006. Differential privacy. In International colloquium on automata, languages, and programming. Springer, 1--12."},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/11681878_14"},{"key":"e_1_2_1_27_1","doi-asserted-by":"crossref","unstructured":"Cynthia Dwork Aaron Roth et al. 2014. The algorithmic foundations of differential privacy. Foundations and Trends\u00ae in Theoretical Computer Science 9 3--4 (2014) 211--407.","DOI":"10.1561\/0400000042"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.14778\/3611479.3611527"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.14778\/3583140.3583165"},{"key":"e_1_2_1_30_1","volume-title":"NM Anoop Krishnan, and Mausam","author":"Gupta Tanishq","year":"2022","unstructured":"Tanishq Gupta, Mohd Zaki, NM Anoop Krishnan, and Mausam. 2022. MatSciBERT: A materials domain language model for text mining and information extraction. npj Computational Materials 8, 1 (2022), 102."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-84858-7"},{"key":"e_1_2_1_32_1","volume-title":"Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300","author":"Hendrycks Dan","year":"2020","unstructured":"Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2020. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300 (2020)."},{"key":"e_1_2_1_33_1","volume-title":"DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer. arXiv preprint arXiv:2312.03724","author":"Hong Junyuan","year":"2023","unstructured":"Junyuan Hong, Jiachen T Wang, Chenhui Zhang, Zhangheng Li, Bo Li, and Zhangyang Wang. 2023. DP-OPT: Make Large Language Model Your Privacy-Preserving Prompt Engineer. arXiv preprint arXiv:2312.03724 (2023)."},{"key":"e_1_2_1_34_1","volume-title":"Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685","author":"Hu Edward J","year":"2021","unstructured":"Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)."},{"key":"e_1_2_1_35_1","volume-title":"PLeak: Prompt Leaking Attacks against Large Language Model Applications. arXiv preprint arXiv:2405.06823","author":"Hui Bo","year":"2024","unstructured":"Bo Hui, Haolin Yuan, Neil Gong, Philippe Burlina, and Yinzhi Cao. 2024. PLeak: Prompt Leaking Attacks against Large Language Model Applications. arXiv preprint arXiv:2405.06823 (2024)."},{"key":"e_1_2_1_36_1","volume-title":"Knowledge unlearning for mitigating privacy risks in language models. arXiv preprint arXiv:2210.01504","author":"Jang Joel","year":"2022","unstructured":"Joel Jang, Dongkeun Yoon, Sohee Yang, Sungmin Cha, Moontae Lee, Lajanugen Logeswaran, and Minjoon Seo. 2022. Knowledge unlearning for mitigating privacy risks in language models. arXiv preprint arXiv:2210.01504 (2022)."},{"key":"e_1_2_1_37_1","volume-title":"Active data pattern extraction attacks on generative language models. arXiv preprint arXiv:2207.10802","author":"Jayaraman Bargav","year":"2022","unstructured":"Bargav Jayaraman, Esha Ghosh, Huseyin Inan, Melissa Chase, Sambuddha Roy, and Wei Dai. 2022. Active data pattern extraction attacks on generative language models. arXiv preprint arXiv:2207.10802 (2022)."},{"key":"e_1_2_1_38_1","volume-title":"Exploiting programmatic behavior of llms: Dual-use through standard security attacks. arXiv preprint arXiv:2302.05733","author":"Kang Daniel","year":"2023","unstructured":"Daniel Kang, Xuechen Li, Ion Stoica, Carlos Guestrin, Matei Zaharia, and Tatsunori Hashimoto. 2023. Exploiting programmatic behavior of llms: Dual-use through standard security attacks. arXiv preprint arXiv:2302.05733 (2023)."},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.14778\/3401960.3401970"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-30115-8_22"},{"key":"e_1_2_1_41_1","volume-title":"Privacy adhering machine un-learning in nlp. arXiv preprint arXiv:2212.09573","author":"Kumar Vinayshekhar Bannihatti","year":"2022","unstructured":"Vinayshekhar Bannihatti Kumar, Rashmi Gangadharaiah, and Dan Roth. 2022. Privacy adhering machine un-learning in nlp. arXiv preprint arXiv:2212.09573 (2022)."},{"key":"e_1_2_1_42_1","volume-title":"Multi-step jailbreaking privacy attacks on chatgpt. arXiv preprint arXiv:2304.05197","author":"Li Haoran","year":"2023","unstructured":"Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, and Yangqiu Song. 2023. Multi-step jailbreaking privacy attacks on chatgpt. arXiv preprint arXiv:2304.05197 (2023)."},{"key":"e_1_2_1_43_1","unstructured":"Qinbin Li Junyuan Hong Chulin Xie Jeffrey Tan Rachel Xin Junyi Hou Xavier Yin Zhun Wang Dan Hendrycks Zhangyang Wang Bo Li Bingsheng He and Dawn Song. 2024. LLM-PBE: Assessing Data Privacy in Large Language Model. https:\/\/llm-pbe.github.io\/paper. Accessed: 2024-07-16."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01929"},{"key":"e_1_2_1_45_1","unstructured":"Kevin Liu. 2023. https:\/\/twitter.com\/kliu128\/status\/1623472922374574080 Accessed: 2024-07-16."},{"key":"e_1_2_1_46_1","volume-title":"Analyzing leakage of personally identifiable information in language models. arXiv preprint arXiv:2302.00539","author":"Lukas Nils","year":"2023","unstructured":"Nils Lukas, Ahmed Salem, Robert Sim, Shruti Tople, Lukas Wutschitz, and Santiago Zanella-B\u00e9guelin. 2023. Analyzing leakage of personally identifiable information in language models. arXiv preprint arXiv:2302.00539 (2023)."},{"key":"e_1_2_1_47_1","volume-title":"Membership Inference Attacks against Language Models via Neighbourhood Comparison. arXiv preprint arXiv:2305.18462","author":"Mattern Justus","year":"2023","unstructured":"Justus Mattern, Fatemehsadat Mireshghallah, Zhijing Jin, Bernhard Sch\u00f6lkopf, Mrinmaya Sachan, and Taylor Berg-Kirkpatrick. 2023. Membership Inference Attacks against Language Models via Neighbourhood Comparison. arXiv preprint arXiv:2305.18462 (2023)."},{"key":"e_1_2_1_48_1","unstructured":"Ryan Mac Michael M. Grynbaum. 2023. The Times Sues OpenAI and Microsoft Over A.I. Use of Copyrighted Work. https:\/\/www.nytimes.com\/2023\/12\/27\/business\/media\/new-york-times-open-ai-microsoft-lawsuit.html"},{"key":"e_1_2_1_49_1","volume-title":"Quantifying privacy risks of masked language models using membership inference attacks. arXiv preprint arXiv:2203.03929","author":"Mireshghallah Fatemehsadat","year":"2022","unstructured":"Fatemehsadat Mireshghallah, Kartik Goyal, Archit Uniyal, Taylor Berg-Kirkpatrick, and Reza Shokri. 2022. Quantifying privacy risks of masked language models using membership inference attacks. arXiv preprint arXiv:2203.03929 (2022)."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.119"},{"key":"e_1_2_1_51_1","volume-title":"Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory. arXiv preprint arXiv:2310.17884","author":"Mireshghallah Niloofar","year":"2023","unstructured":"Niloofar Mireshghallah, Hyunwoo Kim, Xuhui Zhou, Yulia Tsvetkov, Maarten Sap, Reza Shokri, and Yejin Choi. 2023. Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory. arXiv preprint arXiv:2310.17884 (2023)."},{"key":"e_1_2_1_52_1","volume-title":"Pathways language model (palm): Scaling to 540 billion parameters for breakthrough performance. Google AI Blog","author":"Narang Sharan","year":"2022","unstructured":"Sharan Narang and Aakanksha Chowdhery. 2022. Pathways language model (palm): Scaling to 540 billion parameters for breakthrough performance. Google AI Blog (2022)."},{"key":"e_1_2_1_53_1","unstructured":"Leandro von Werra Alex Havrilla Nathan Lambert Louis Castricato. 2022. Illustrating Reinforcement Learning from Human Feedback (RLHF). IllustratingReinforcementLearningfromHumanFeedback(RLHF)"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP40000.2020.00095"},{"key":"e_1_2_1_56_1","volume-title":"The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=qo21ZlfNu6 Accessed: 2024-07-16","author":"Panda Ashwinee","year":"2024","unstructured":"Ashwinee Panda, Christopher A. Choquette-Choo, Zhengming Zhang, Yaoqing Yang, and Prateek Mittal. 2024. Teach LLMs to Phish: Stealing Private Information from Language Models. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=qo21ZlfNu6 Accessed: 2024-07-16."},{"key":"e_1_2_1_57_1","volume-title":"Differentially Private In-Context Learning. arXiv preprint arXiv:2305.01639","author":"Panda Ashwinee","year":"2023","unstructured":"Ashwinee Panda, Tong Wu, Jiachen T Wang, and Prateek Mittal. 2023. Differentially Private In-Context Learning. arXiv preprint arXiv:2305.01639 (2023)."},{"key":"e_1_2_1_58_1","volume-title":"Teach GPT To Phish. In The Second Workshop on New Frontiers in Adversarial Machine Learning. https:\/\/openreview.net\/forum?id=tGvWCD9BEP Accessed: 2024-07-16","author":"Panda Ashwinee","year":"2023","unstructured":"Ashwinee Panda, Zhengming Zhang, Yaoqing Yang, and Prateek Mittal. 2023. Teach GPT To Phish. In The Second Workshop on New Frontiers in Adversarial Machine Learning. https:\/\/openreview.net\/forum?id=tGvWCD9BEP Accessed: 2024-07-16."},{"key":"e_1_2_1_59_1","volume-title":"Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527","author":"Perez F\u00e1bio","year":"2022","unstructured":"F\u00e1bio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models. arXiv preprint arXiv:2211.09527 (2022)."},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDM.2017.48"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00458"},{"key":"e_1_2_1_62_1","volume-title":"Detecting pretraining data from large language models. arXiv preprint arXiv:2310.16789","author":"Shi Weijia","year":"2023","unstructured":"Weijia Shi, Anirudh Ajith, Mengzhou Xia, Yangsibo Huang, Daogao Liu, Terra Blevins, Danqi Chen, and Luke Zettlemoyer. 2023. Detecting pretraining data from large language models. arXiv preprint arXiv:2310.16789 (2023)."},{"key":"e_1_2_1_63_1","volume-title":"Proceedings of the 22nd ACM SIGSAC conference on computer and communications security. 1310--1321","author":"Shokri Reza","year":"2015","unstructured":"Reza Shokri and Vitaly Shmatikov. 2015. Privacy-preserving deep learning. In Proceedings of the 22nd ACM SIGSAC conference on computer and communications security. 1310--1321."},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2017.41"},{"key":"e_1_2_1_65_1","volume-title":"The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=kmn0BhQk7p Accessed: 2024-07-16","author":"Staab Robin","year":"2024","unstructured":"Robin Staab, Mark Vero, Mislav Balunovic, and Martin Vechev. 2024. Beyond Memorization: Violating Privacy via Inference with Large Language Models. In The Twelfth International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=kmn0BhQk7p Accessed: 2024-07-16."},{"key":"e_1_2_1_66_1","unstructured":"Yu Sun Shuohuan Wang Shikun Feng Siyu Ding Chao Pang Junyuan Shang Jiaxiang Liu Xuyi Chen Yanbin Zhao Yuxiang Lu et al. 2021. Ernie 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation. arXiv preprint arXiv:2107.02137 (2021)."},{"key":"e_1_2_1_67_1","volume-title":"Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation. arXiv preprint arXiv:2309.11765","author":"Tang Xinyu","year":"2023","unstructured":"Xinyu Tang, Richard Shin, Huseyin A Inan, Andre Manoel, Fatemehsadat Mireshghallah, Zinan Lin, Sivakanth Gopi, Janardhan Kulkarni, and Robert Sim. 2023. Privacy-Preserving In-Context Learning with Differentially Private Few-Shot Generation. arXiv preprint arXiv:2309.11765 (2023)."},{"key":"e_1_2_1_68_1","volume-title":"Fast yet effective machine unlearning","author":"Tarun Ayush K","year":"2023","unstructured":"Ayush K Tarun, Vikram S Chundawat, Murari Mandal, and Mohan Kankanhalli. 2023. Fast yet effective machine unlearning. IEEE Transactions on Neural Networks and Learning Systems (2023)."},{"key":"e_1_2_1_69_1","unstructured":"Hugo Touvron Louis Martin Kevin Stone Peter Albert Amjad Almahairi Yasmine Babaei Nikolay Bashlykov Soumya Batra Prajjwal Bhargava Shruti Bhosale Dan Bikel Lukas Blecher Cristian Canton Ferrer Moya Chen Guillem Cucurull David Esiobu Jude Fernandes Jeremy Fu Wenyin Fu Brian Fuller Cynthia Gao Vedanuj Goswami Naman Goyal Anthony Hartshorn Saghar Hosseini Rui Hou Hakan Inan Marcin Kardas Viktor Kerkez Madian Khabsa Isabel Kloumann Artem Korenev Punit Singh Koura Marie-Anne Lachaux Thibaut Lavril Jenya Lee Diana Liskovich Yinghai Lu Yuning Mao Xavier Martinet Todor Mihaylov Pushkar Mishra Igor Molybog Yixin Nie Andrew Poulton Jeremy Reizenstein Rashi Rungta Kalyan Saladi Alan Schelten Ruan Silva Eric Michael Smith Ranjan Subramanian Xiaoqing Ellen Tan Binh Tang Ross Taylor Adina Williams Jian Xiang Kuan Puxin Xu Zheng Yan Iliyan Zarov Yuchen Zhang Angela Fan Melanie Kambadur Sharan Narang Aurelien Rodriguez Robert Stojnic Sergey Edunov and Thomas Scialom. 2023. Llama 2: Open Foundation and Fine-Tuned Chat Models. arXiv:2307.09288 [cs.CL]"},{"key":"e_1_2_1_70_1","volume-title":"Considerations for Differentially Private Learning with Large-Scale Public Pretraining. arXiv:2212.06470","author":"Tram'er Florian","year":"2022","unstructured":"Florian Tram'er, Kamath Gautam, and Nicholas Carlini Carlini. 2022. Considerations for Differentially Private Learning with Large-Scale Public Pretraining. arXiv:2212.06470 (2022)."},{"key":"e_1_2_1_71_1","volume-title":"Roger Baker Grosse, and Owain Evans","author":"Treutlein Johannes","year":"2024","unstructured":"Johannes Treutlein, Dami Choi, Jan Betley, Cem Anil, Samuel Marks, Roger Baker Grosse, and Owain Evans. 2024. Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data. arXiv preprint arXiv:2406.14546 (2024)."},{"key":"e_1_2_1_72_1","volume-title":"From bert to gpt-3 codex: harnessing the potential of very large language models for data management. arXiv preprint arXiv:2306.09339","author":"Trummer Immanuel","year":"2023","unstructured":"Immanuel Trummer. 2023. From bert to gpt-3 codex: harnessing the potential of very large language models for data management. arXiv preprint arXiv:2306.09339 (2023)."},{"key":"e_1_2_1_73_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCIDS.2019.8862080"},{"key":"e_1_2_1_74_1","doi-asserted-by":"publisher","DOI":"10.1145\/3593078.3593933"},{"key":"e_1_2_1_75_1","unstructured":"Yuli Vasiliev. 2020. Natural language processing with Python and spaCy: A practical introduction. No Starch Press."},{"key":"e_1_2_1_76_1","volume-title":"Attention is all you need. Advances in neural information processing systems 30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_2_1_77_1","unstructured":"Boxin Wang Weixin Chen Hengzhi Pei Chulin Xie Mintong Kang Chenhui Zhang Chejian Xu Zidi Xiong Ritik Dutta Rylan Schaeffer et al. 2023. DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Models. arXiv preprint arXiv:2306.11698 (2023)."},{"key":"e_1_2_1_78_1","volume-title":"KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment. arXiv preprint arXiv:2305.06535","author":"Wang Lingzhi","year":"2023","unstructured":"Lingzhi Wang, Tong Chen, Wei Yuan, Xingshan Zeng, Kam-Fai Wong, and Hongzhi Yin. 2023. KGA: A General Machine Unlearning Framework Based on Knowledge Gap Alignment. arXiv preprint arXiv:2305.06535 (2023)."},{"key":"e_1_2_1_79_1","volume-title":"Machine unlearning of features and labels. arXiv preprint arXiv:2108.11577","author":"Warnecke Alexander","year":"2021","unstructured":"Alexander Warnecke, Lukas Pirch, Christian Wressnegger, and Konrad Rieck. 2021. Machine unlearning of features and labels. arXiv preprint arXiv:2108.11577 (2021)."},{"key":"e_1_2_1_80_1","volume-title":"On the importance of difficulty calibration in membership inference attacks. arXiv preprint arXiv:2111.08440","author":"Watson Lauren","year":"2021","unstructured":"Lauren Watson, Chuan Guo, Graham Cormode, and Alex Sablayrolles. 2021. On the importance of difficulty calibration in membership inference attacks. arXiv preprint arXiv:2111.08440 (2021)."},{"key":"e_1_2_1_81_1","volume-title":"Jailbroken: How does llm safety training fail? arXiv preprint arXiv:2307.02483","author":"Wei Alexander","year":"2023","unstructured":"Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does llm safety training fail? arXiv preprint arXiv:2307.02483 (2023)."},{"key":"e_1_2_1_82_1","volume-title":"Proceedings of the 32nd international conference on Very large data bases. 139--150","author":"Xiao Xiaokui","year":"2006","unstructured":"Xiaokui Xiao and Yufei Tao. 2006. Anatomy: Simple and effective privacy preservation. In Proceedings of the 32nd international conference on Very large data bases. 139--150."},{"key":"e_1_2_1_83_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2010.247"},{"key":"e_1_2_1_84_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-013-0309-y"},{"key":"e_1_2_1_85_1","doi-asserted-by":"publisher","DOI":"10.1145\/3548606.3560675"},{"key":"e_1_2_1_86_1","volume-title":"Bag of tricks for training data extraction from language models. arXiv preprint arXiv:2302.04460","author":"Yu Weichen","year":"2023","unstructured":"Weichen Yu, Tianyu Pang, Qian Liu, Chao Du, Bingyi Kang, Yan Huang, Min Lin, and Shuicheng Yan. 2023. Bag of tricks for training data extraction from language models. arXiv preprint arXiv:2302.04460 (2023)."},{"key":"e_1_2_1_87_1","volume-title":"A Synthetic Dataset for Personal Attribute Inference. arXiv preprint arXiv:2406.07217","author":"Yukhymenko Hanna","year":"2024","unstructured":"Hanna Yukhymenko, Robin Staab, Mark Vero, and Martin Vechev. 2024. A Synthetic Dataset for Personal Attribute Inference. arXiv preprint arXiv:2406.07217 (2024)."},{"key":"e_1_2_1_88_1","doi-asserted-by":"publisher","DOI":"10.1145\/3580305.3599921"},{"key":"e_1_2_1_89_1","volume-title":"Prompts should not be seen as secrets: Systematically measuring prompt extraction attack success. arXiv preprint arXiv:2307.06865","author":"Zhang Yiming","year":"2023","unstructured":"Yiming Zhang and Daphne Ippolito. 2023. Prompts should not be seen as secrets: Systematically measuring prompt extraction attack success. arXiv preprint arXiv:2307.06865 (2023)."},{"key":"e_1_2_1_90_1","first-page":"13433","article-title":"Prompt certified machine unlearning with randomized gradient smoothing and quantization","volume":"35","author":"Zhang Zijie","year":"2022","unstructured":"Zijie Zhang, Yang Zhou, Xin Zhao, Tianshi Che, and Lingjuan Lyu. 2022. Prompt certified machine unlearning with randomized gradient smoothing and quantization. Advances in Neural Information Processing Systems 35 (2022), 13433--13455.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_91_1","volume-title":"DB-GPT: Large Language Model Meets Database. Data Science and Engineering","author":"Zhou Xuanhe","year":"2024","unstructured":"Xuanhe Zhou, Zhaoyan Sun, and Guoliang Li. 2024. DB-GPT: Large Language Model Meets Database. Data Science and Engineering (2024), 1--10."},{"key":"e_1_2_1_92_1","volume-title":"Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593","author":"Ziegler Daniel M","year":"2019","unstructured":"Daniel M Ziegler, Nisan Stiennon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. 2019. Fine-tuning language models from human preferences. arXiv preprint arXiv:1909.08593 (2019)."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3681954.3681994","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,4]],"date-time":"2024-09-04T18:38:02Z","timestamp":1725475082000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3681954.3681994"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,7]]},"references-count":91,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2024,7]]}},"alternative-id":["10.14778\/3681954.3681994"],"URL":"https:\/\/doi.org\/10.14778\/3681954.3681994","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,7]]},"assertion":[{"value":"2024-08-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}