{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T19:39:09Z","timestamp":1776886749167,"version":"3.51.2"},"reference-count":128,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,11,7]],"date-time":"2024-11-07T00:00:00Z","timestamp":1730937600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2025,2,28]]},"abstract":"<jats:p>Text watermarking algorithms are crucial for protecting the copyright of textual content. Historically, their capabilities and application scenarios were limited. However, recent advancements in large language models (LLMs) have revolutionized these techniques. LLMs not only enhance text watermarking algorithms with their advanced abilities but also create a need for employing these algorithms to protect their own copyrights or prevent potential misuse. This work conducts a comprehensive survey of the current state of text watermarking technology, covering four main aspects: (1) an overview and comparison of different text watermarking techniques; (2) evaluation methods for text watermarking algorithms, including their detectability, impact on text or LLM quality, and robustness under target or untargeted attacks; (3) potential application scenarios for text watermarking technology; and (4) current challenges and future directions for text watermarking. This survey aims to provide researchers with a thorough understanding of text watermarking technology in the era of LLMs, thereby promoting its further advancement.<\/jats:p>","DOI":"10.1145\/3691626","type":"journal-article","created":{"date-parts":[[2024,9,3]],"date-time":"2024-09-03T15:34:05Z","timestamp":1725377645000},"page":"1-36","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":42,"title":["A Survey of Text Watermarking in the Era of Large Language Models"],"prefix":"10.1145","volume":"57","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-4965-8263","authenticated-orcid":false,"given":"Aiwei","family":"Liu","sequence":"first","affiliation":[{"name":"Tsinghua University, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-0859-2203","authenticated-orcid":false,"given":"Leyi","family":"Pan","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-1852-9829","authenticated-orcid":false,"given":"Yijian","family":"Lu","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, Hong Kong"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9018-3039","authenticated-orcid":false,"given":"Jingjing","family":"Li","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, Hong Kong"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6075-4224","authenticated-orcid":false,"given":"Xuming","family":"Hu","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology - Guangzhou Campus, Guangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2111-7385","authenticated-orcid":false,"given":"Xi","family":"Zhang","sequence":"additional","affiliation":[{"name":"Beijing University of Posts and Telecommunications, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0358-3160","authenticated-orcid":false,"given":"Lijie","family":"Wen","sequence":"additional","affiliation":[{"name":"Tsinghua University, Beijing, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8106-6447","authenticated-orcid":false,"given":"Irwin","family":"King","sequence":"additional","affiliation":[{"name":"The Chinese University of Hong Kong, Hong Kong, Hong Kong"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6016-6465","authenticated-orcid":false,"given":"Hui","family":"Xiong","sequence":"additional","affiliation":[{"name":"The Hong Kong University of Science and Technology - Guangzhou Campus, Guangzhou, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3491-5968","authenticated-orcid":false,"given":"Philip","family":"Yu","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Illinois at Chicago, Chicago, United States"}]}],"member":"320","published-online":{"date-parts":[[2024,11,7]]},"reference":[{"key":"e_1_3_2_2_2","unstructured":"S. Aaronson and H. Kirchner. 2022. Watermarking GPT Outputs. Retrieved September 5 2024 from https:\/\/www.scottaaronson.com\/talks\/watermark.ppt"},{"key":"e_1_3_2_3_2","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1109\/SP40001.2021.00083","volume-title":"Proceedings of the 2021 IEEE Symposium on Security and Privacy (SP \u201921)","author":"Abdelnabi Sahar","year":"2021","unstructured":"Sahar Abdelnabi and Mario Fritz. 2021. Adversarial watermarking transformer: Towards tracing text provenance with data hiding. In Proceedings of the 2021 IEEE Symposium on Security and Privacy (SP \u201921). IEEE, 121\u2013140."},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1002\/sec.1738"},{"key":"e_1_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Mikhail J. Atallah Victor Raskin Michael Crogan Christian Hempelmann Florian Kerschbaum Dina Mohamed and Sanket Naik. 2001. Natural language watermarking: Design analysis and a proof-of-concept implementation. In Information Hiding. Lecture Notes in Computer Science Vol. 2137. Springer 185\u2013200.","DOI":"10.1007\/3-540-45496-9_14"},{"key":"e_1_3_2_6_2","first-page":"65","volume-title":"Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization","author":"Banerjee Satanjeev","year":"2005","unstructured":"Satanjeev Banerjee and Alon Lavie. 2005. METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization. 65\u201372."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.3390\/info11020110"},{"key":"e_1_3_2_8_2","article-title":"Model leeching: An extraction attack targeting LLMs","author":"Birch Lewis","year":"2023","unstructured":"Lewis Birch, William Hackett, Stefan Trawicki, Neeraj Suri, and Peter Garraghan. 2023. Model leeching: An extraction attack targeting LLMs. arXiv preprint arXiv:2309.10544 (2023).","journal-title":"arXiv preprint arXiv:2309.10544"},{"key":"e_1_3_2_9_2","doi-asserted-by":"crossref","first-page":"1987","DOI":"10.1109\/SP46214.2022.9833641","volume-title":"Proceedings of the 2022 IEEE Symposium on Security and Privacy (SP \u201922)","author":"Boucher Nicholas","year":"2022","unstructured":"Nicholas Boucher, Ilia Shumailov, Ross Anderson, and Nicolas Papernot. 2022. Bad characters: Imperceptible NLP attacks. In Proceedings of the 2022 IEEE Symposium on Security and Privacy (SP \u201922). IEEE, 1987\u20132004."},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/49.464718"},{"key":"e_1_3_2_11_2","article-title":"Sparks of artificial general intelligence: Early experiments with GPT-4","author":"Bubeck S\u00e9bastien","year":"2023","unstructured":"S\u00e9bastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. Sparks of artificial general intelligence: Early experiments with GPT-4. arXiv preprint arXiv:2303.12712 (2023).","journal-title":"arXiv preprint arXiv:2303.12712"},{"key":"e_1_3_2_12_2","article-title":"Universal sentence encoder","author":"Cer Daniel","year":"2018","unstructured":"Daniel Cer, Yinfei Yang, Sheng-Yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. 2018. Universal sentence encoder. arXiv preprint arXiv:1803.11175 (2018).","journal-title":"arXiv preprint arXiv:1803.11175"},{"key":"e_1_3_2_13_2","article-title":"Combating misinformation in the age of LLMs: Opportunities and challenges","author":"Chen Canyu","year":"2023","unstructured":"Canyu Chen and Kai Shu. 2023. Combating misinformation in the age of LLMs: Opportunities and challenges. arXiv preprint arXiv:2311.05656 (2023).","journal-title":"arXiv preprint arXiv:2311.05656"},{"key":"e_1_3_2_14_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Chen Canyu","year":"2024","unstructured":"Canyu Chen and Kai Shu. 2024. Can LLM-generated misinformation be detected? In Proceedings of the 12th International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=ccxD4mtkTU"},{"key":"e_1_3_2_15_2","article-title":"Evaluating large language models trained on code","author":"Chen Mark","year":"2021","unstructured":"Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khalaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, Williams Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandish, Ilya Sutskever, and Wojciech Zaremba. 2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021).","journal-title":"arXiv preprint arXiv:2107.03374"},{"key":"e_1_3_2_16_2","first-page":"1125","volume-title":"Proceedings of the 37th Annual Conference on Learning Theory","author":"Christ Miranda","year":"2024","unstructured":"Miranda Christ, Sam Gunn, and Or Zamir. 2024. Undetectable watermarks for language models. In Proceedings of the 37th Annual Conference on Learning Theory. 1125\u20131139."},{"key":"e_1_3_2_17_2","article-title":"No language left behind: Scaling human-centered machine translation","author":"Costa-Juss\u00e0 Marta R.","year":"2022","unstructured":"Marta R. Costa-Juss\u00e0, James Cross, Onur \u00c7elebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzman, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. 2022. No language left behind: Scaling human-centered machine translation. arXiv preprint arXiv:2207.04672 (2022).","journal-title":"arXiv preprint arXiv:2207.04672"},{"key":"e_1_3_2_18_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171\u20134186."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D18-1045"},{"key":"e_1_3_2_20_2","article-title":"Publicly-Detectable Watermarking for Language Models","author":"Fairoze Jaiden","year":"2023","unstructured":"Jaiden Fairoze, Sanjam Garg, Somesh Jha, Saeed Mahloujifar, Mohammad Mahmoody, and Mingyuan Wang. 2023. Publicly-Detectable Watermarking for Language Models. Paper 2923.1661, Cryptology ePrint Archive. https:\/\/eprint.iacr.org\/2023\/1661https:\/\/eprint.iacr.org\/2023\/1661","journal-title":"Cryptology ePrint Archive"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1346"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.7551\/mitpress\/7287.001.0001"},{"key":"e_1_3_2_23_2","first-page":"1","volume-title":"Proceedings of the 2023 IEEE International Workshop on Information Forensics and Security (WIFS \u201923)","author":"Fernandez Pierre","year":"2023","unstructured":"Pierre Fernandez, Antoine Chaffin, Karim Tit, Vivien Chappelier, and Teddy Furon. 2023. Three bricks to consolidate watermarks for large language models. In Proceedings of the 2023 IEEE International Workshop on Information Forensics and Security (WIFS \u201923). IEEE, 1\u20136."},{"key":"e_1_3_2_24_2","first-page":"18003","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"38","author":"Fu Yu","year":"2024","unstructured":"Yu Fu, Deyi Xiong, and Yue Dong. 2024. Watermarking conditional text generation for AI detection: Unveiling challenges and a semantic-aware watermark remedy. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 18003\u201318011."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/503124.503156"},{"key":"e_1_3_2_26_2","article-title":"WaterMax: Breaking the LLM watermark detectability-robustness-quality trade-off","author":"Giboulot Eva","year":"2024","unstructured":"Eva Giboulot and Furon Teddy. 2024. WaterMax: Breaking the LLM watermark detectability-robustness-quality trade-off. arXiv preprint arXiv:2403.04808 (2024).","journal-title":"arXiv preprint arXiv:2403.04808"},{"key":"e_1_3_2_27_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Gu Chenchen","year":"2024","unstructured":"Chenchen Gu, Xiang Lisa Li, Percy Liang, and Tatsunori Hashimoto. 2024. On the learnability of watermarks for language models. In Proceedings of the 12th International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=9k0krNzvlV"},{"key":"e_1_3_2_28_2","article-title":"CodeIP: A grammar-guided multi-bit watermark for large language models of code","author":"Guan Batu","year":"2024","unstructured":"Batu Guan, Yao Wan, Zhangqian Bi, Zheng Wang, Hongyu Zhang, Yulei Sui, Pan Zhou, and Lichao Sun. 2024. CodeIP: A grammar-guided multi-bit watermark for large language models of code. arXiv preprint arXiv:2404.15639 (2024).","journal-title":"arXiv preprint arXiv:2404.15639"},{"key":"e_1_3_2_29_2","first-page":"10758","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"36","author":"He Xuanli","year":"2022","unstructured":"Xuanli He, Qiongkai Xu, Lingjuan Lyu, Fangzhao Wu, and Chenguang Wang. 2022. Protecting intellectual property of language generation APIs with lexical watermark. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 10758\u201310766."},{"key":"e_1_3_2_30_2","first-page":"5431","article-title":"CATER: Intellectual property protection on text generation APIs via conditional watermarks","volume":"35","author":"He Xuanli","year":"2022","unstructured":"Xuanli He, Qiongkai Xu, Yi Zeng, Lingjuan Lyu, Fangzhao Wu, Jiwei Li, and Ruoxi Jia. 2022. CATER: Intellectual property protection on text generation APIs via conditional watermarks. Advances in Neural Information Processing Systems 35 (2022), 5431\u20135445.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_31_2","article-title":"Can watermarks survive translation? On the cross-lingual consistency of text watermark for large language models","author":"He Zhiwei","year":"2024","unstructured":"Zhiwei He, Binglin Zhou, Hongkun Hao, Aiwei Liu, Xing Wang, Zhaopeng Tu, Zhuosheng Zhang, and Rui Wang. 2024. Can watermarks survive translation? On the cross-lingual consistency of text watermark for large language models. arXiv preprint arXiv:2402.14007 (2024).","journal-title":"arXiv preprint arXiv:2402.14007"},{"key":"e_1_3_2_32_2","article-title":"How good are GPT models at machine translation? A comprehensive evaluation","author":"Hendy Amr","year":"2023","unstructured":"Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. How good are GPT models at machine translation? A comprehensive evaluation. arXiv preprint arXiv:2302.09210 (2023).","journal-title":"arXiv preprint arXiv:2302.09210"},{"key":"e_1_3_2_33_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Holtzman Ari","year":"2020","unstructured":"Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In Proceedings of the International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=rygGQyrFvH"},{"key":"e_1_3_2_34_2","first-page":"4067","volume-title":"Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (NAACL \u201924)","author":"Hou Abe","year":"2024","unstructured":"Abe Hou, Jingyu Zhang, Tianxing He, Yichen Wang, Yung-Sung Chuang, Hongwei Wang, Lingfeng Shen, Benjamin Van Durme, Daniel Khashabi, and Yulia Tsvetkov. 2024. SemStamp: A semantic watermark with paraphrastic robustness for text generation. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) (NAACL \u201924). 4067\u20134082."},{"key":"e_1_3_2_35_2","article-title":"k-SemStamp: A clustering-based semantic watermark for detection of machine-generated text","author":"Hou Abe Bohan","year":"2024","unstructured":"Abe Bohan Hou, Jingyu Zhang, Yichen Wang, Daniel Khashabi, and Tianxing He. 2024. k-SemStamp: A clustering-based semantic watermark for detection of machine-generated text. arXiv preprint arXiv:2402.11399 (2024).","journal-title":"arXiv preprint arXiv:2402.11399"},{"key":"e_1_3_2_36_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Hu Zhengmian","year":"2024","unstructured":"Zhengmian Hu, Lichang Chen, Xidong Wu, Yihan Wu, Hongyang Zhang, and Heng Huang. 2024. Unbiased watermark for large language models. In Proceedings of the 12th International Conference on Learning Representations."},{"key":"e_1_3_2_37_2","doi-asserted-by":"crossref","first-page":"216","DOI":"10.18653\/v1\/2023.sigdial-1.21","volume-title":"Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue","author":"Hude\u010dek Vojt\u011bch","year":"2023","unstructured":"Vojt\u011bch Hude\u010dek and Ond\u0159ej Du\u0161ek. 2023. Are large language models all you need for task-oriented dialogue? In Proceedings of the 24th Annual Meeting of the Special Interest Group on Discourse and Dialogue. 216\u2013228."},{"key":"e_1_3_2_38_2","first-page":"1940","volume-title":"Proceedings of the 2019 15th International Wireless Communications and Mobile Computing Conference (IWCMC \u201919)","author":"Iqbal Muhammad Munwar","year":"2019","unstructured":"Muhammad Munwar Iqbal, Umair Khadam, Ki Jun Han, Jihun Han, and Sohail Jabbar. 2019. A robust digital watermarking algorithm for text document copyright protection based on feature coding. In Proceedings of the 2019 15th International Wireless Communications and Mobile Computing Conference (IWCMC \u201919). IEEE, 1940\u20131945."},{"key":"e_1_3_2_39_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Jang Eric","year":"2017","unstructured":"Eric Jang, Shixiang Gu, and Ben Poole. 2017. Categorical reparameterization with Gumbel-Softmax. In Proceedings of the International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=rkE3y85ee"},{"key":"e_1_3_2_40_2","article-title":"TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension","author":"Joshi Mandar","year":"2017","unstructured":"Mandar Joshi, Eunsol Choi, Daniel S. Weld, and Luke Zettlemoyer. 2017. TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension. arXiv preprint arXiv:1705.03551 (2017).","journal-title":"arXiv preprint arXiv:1705.03551"},{"key":"e_1_3_2_41_2","article-title":"Watermark stealing in large language models","author":"Jovanovi\u0107 Nikola","year":"2024","unstructured":"Nikola Jovanovi\u0107, Robin Staab, and Martin Vechev. 2024. Watermark stealing in large language models. arXiv preprint arXiv:2402.19361 (2024).","journal-title":"arXiv preprint arXiv:2402.19361"},{"key":"e_1_3_2_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2796585"},{"key":"e_1_3_2_43_2","unstructured":"John Kirchenbauer Jonas Geiping Yuxin Wen Jonathan Katz Ian Miers and Tom Goldstein. 2023. A watermark for large language models. Proceedings of Machine Learning Research 202 (2023) 17061\u201317084. https:\/\/proceedings.mlr.press\/v202\/kirchenbauer23a.html"},{"key":"e_1_3_2_44_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Kirchenbauer John","year":"2024","unstructured":"John Kirchenbauer, Jonas Geiping, Yuxin Wen, Manli Shu, Khalid Saifullah, Kezhi Kong, Kasun Fernando, Aniruddha Saha, Micah Goldblum, and Tom Goldstein. 2024. On the reliability of watermarks for large language models. In Proceedings of the 12th International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=DEJIDCmWOz"},{"key":"e_1_3_2_45_2","article-title":"Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense","volume":"36","author":"Krishna Kalpesh","year":"2023","unstructured":"Kalpesh Krishna, Yixiao Song, Marzena Karpinska, John Wieting, and Mohit Iyyer. 2023. Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense. Advances in Neural Information Processing Systems 36 (2023), 1\u201332.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_46_2","article-title":"Robust distortion-free watermarks for language models","author":"Kuditipudi Rohith","year":"2024","unstructured":"Rohith Kuditipudi, John Thickstun, Tatsunori Hashimoto, and Percy Liang. 2024. Robust distortion-free watermarks for language models. Transactions on Machine Learning Research 2024 (2024), 1\u201351. https:\/\/openreview.net\/forum?id=FpaCL1MO2C","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_3_2_47_2","doi-asserted-by":"crossref","first-page":"1583","DOI":"10.1007\/978-3-031-54144-5_153","article-title":"Academic integrity and artificial intelligence: An overview","author":"Kumar Rahul","year":"2024","unstructured":"Rahul Kumar, Sarah Elaine Eaton, Michael Mindzak, and Ryan Morrison. 2024. Academic integrity and artificial intelligence: An overview. In Handbook of Academic Integrity. Springer, 1583\u20131596.","journal-title":"Handbook of Academic Integrity."},{"key":"e_1_3_2_48_2","article-title":"Adaptive ensembles of fine-tuned transformers for LLM-generated text detection","author":"Lai Zhixin","year":"2024","unstructured":"Zhixin Lai, Xuesheng Zhang, and Suiyao Chen. 2024. Adaptive ensembles of fine-tuned transformers for LLM-generated text detection. arXiv preprint arXiv:2403.13335 (2024).","journal-title":"arXiv preprint arXiv:2403.13335"},{"key":"e_1_3_2_49_2","article-title":"Waterfall: Framework for robust and scalable text watermarking","author":"Lau Gregory Kang Ruey","year":"2024","unstructured":"Gregory Kang Ruey Lau, Xinyuan Niu, Hieu Dao, Jiangwei Chen, Chuan-Sheng Foo, and Bryan Kian Hsiang Low. 2024. Waterfall: Framework for robust and scalable text watermarking. arXiv preprint arXiv:2407.04411 (2024).","journal-title":"arXiv preprint arXiv:2407.04411"},{"key":"e_1_3_2_50_2","article-title":"Who wrote this code? Watermarking for code generation","author":"Lee Taehyun","year":"2023","unstructured":"Taehyun Lee, Seokhee Hong, Jaewoo Ahn, Ilgee Hong, Hwaran Lee, Sangdoo Yun, Jamin Shin, and Gunhee Kim. 2023. Who wrote this code? Watermarking for code generation. arXiv preprint arXiv:2305.15060 (2023).","journal-title":"arXiv preprint arXiv:2305.15060"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","unstructured":"Jingjing Li Zichao Li Tao Ge Irwin King and Michael R. Lyu. 2022. Text revision by on-the-fly representation optimization. In Proceedings of the 1st Workshop on Intelligent and Interactive Writing Assistants (In2Writing \u201922). 58\u201359.","DOI":"10.18653\/v1\/2022.in2writing-1.7"},{"key":"e_1_3_2_53_2","first-page":"10820","article-title":"Unsupervised text generation by learning from search","volume":"33","author":"Li Jingjing","year":"2020","unstructured":"Jingjing Li, Zichao Li, Lili Mou, Xin Jiang, Michael Lyu, and Irwin King. 2020. Unsupervised text generation by learning from search. Advances in Neural Information Processing Systems 33 (2020), 10820\u201310831.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_54_2","first-page":"5358","volume-title":"Findings of the Association for Computational Linguistics: ACL 2023","author":"Li Tong","year":"2023","unstructured":"Tong Li, Zhihao Wang, Liangying Shao, Xuling Zheng, Xiaoli Wang, and Jinsong Su. 2023. A sequence-to-sequence&set model for text-to-table generation. In Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics, 5358\u20135370."},{"key":"e_1_3_2_55_2","volume-title":"Proceedings of the Workshop on Secure and Trustworthy Large Language Models (ICLR \u201924)","author":"Chen Liang","year":"2024","unstructured":"Liang Chen, Yatao Bian, Yang Deng, Deng Cai, Shuaiyi Li, Peilin Zhao, and Kam-Fai Wong. 2024. WatME: Towards lossless watermarking through lexical redundancy. In Proceedings of the Workshop on Secure and Trustworthy Large Language Models (ICLR \u201924)."},{"key":"e_1_3_2_56_2","article-title":"TruthfulQA: Measuring how models mimic human falsehoods","author":"Lin Stephanie","year":"2021","unstructured":"Stephanie Lin, Jacob Hilton, and Owain Evans. 2021. TruthfulQA: Measuring how models mimic human falsehoods. arXiv preprint arXiv:2109.07958 (2021).","journal-title":"arXiv preprint arXiv:2109.07958"},{"key":"e_1_3_2_57_2","unstructured":"Aiwei Liu Leyi Pan Xuming Hu Shu\u2019ang Li Lijie Wen Irwin King and Philip S. Yu. 2023. An unforgeable publicly verifiable watermark for large language models. arxiv:2307.16230 [cs.CL] (2023)."},{"key":"e_1_3_2_58_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Liu Aiwei","year":"2024","unstructured":"Aiwei Liu, Leyi Pan, Xuming Hu, Shiao Meng, and Lijie Wen. 2024. A semantic invariant robust watermark for large language models. In Proceedings of the 12th International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=6p8lpe4MNf"},{"key":"e_1_3_2_59_2","doi-asserted-by":"publisher","DOI":"10.1145\/3626772.3661377"},{"key":"e_1_3_2_60_2","volume-title":"Proceedings of the 41st International Conference on Machine Learning","author":"Liu Yepeng","year":"2024","unstructured":"Yepeng Liu and Yuheng Bu. 2024. Adaptive text watermark for large language models. In Proceedings of the 41st International Conference on Machine Learning. https:\/\/openreview.net\/forum?id=7emOSb5UfX"},{"key":"e_1_3_2_61_2","article-title":"An entropy-based text watermarking detection method","author":"Lu Yijian","year":"2024","unstructured":"Yijian Lu, Aiwei Liu, Dianzhi Yu, Jingjing Li, and Irwin King. 2024. An entropy-based text watermarking detection method. arXiv preprint arXiv:2403.13485 (2024).","journal-title":"arXiv preprint arXiv:2403.13485"},{"key":"e_1_3_2_62_2","article-title":"Lost in overlap: Exploring watermark collision in LLMs","author":"Luo Yiyang","year":"2024","unstructured":"Yiyang Luo, Ke Lin, and Chao Gu. 2024. Lost in overlap: Exploring watermark collision in LLMs. arXiv preprint arXiv:2403.10020 (2024).","journal-title":"arXiv preprint arXiv:2403.10020"},{"key":"e_1_3_2_63_2","doi-asserted-by":"publisher","DOI":"10.1145\/3465481.3470088"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.csl.2008.04.001"},{"key":"e_1_3_2_65_2","article-title":"PanGu-Bot: Efficient generative dialogue pre-training from pre-trained language model","author":"Mi Fei","year":"2022","unstructured":"Fei Mi, Yitong Li, Yulong Zeng, Jingyan Zhou, Yasheng Wang, Chuanfei Xu, Lifeng Shang, Xin Jiang, Shiqi Zhao, and Qun Liu. 2022. PanGu-Bot: Efficient generative dialogue pre-training from pre-trained language model. arXiv preprint arXiv:2203.17090 (2022).","journal-title":"arXiv preprint arXiv:2203.17090"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2013.07.040"},{"key":"e_1_3_2_67_2","first-page":"24950","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Mitchell Eric","year":"2023","unstructured":"Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. DetectGPT: Zero-shot machine-generated text detection using probability curvature. In Proceedings of the International Conference on Machine Learning. 24950\u201324962."},{"key":"e_1_3_2_68_2","doi-asserted-by":"crossref","first-page":"3515","DOI":"10.18653\/v1\/2024.findings-naacl.223","volume-title":"Findings of the Association for Computational Linguistics: NAACL 2024","author":"Molenda Piotr","year":"2024","unstructured":"Piotr Molenda, Adian Liusie, and Mark Gales. 2024. WaterJudge: Quality-detection trade-off when watermarking large language models. In Findings of the Association for Computational Linguistics: NAACL 2024. Association for Computational Linguistics, 3515\u20133525."},{"key":"e_1_3_2_69_2","article-title":"DeepTextMark: Deep learning based text watermarking for detection of large language model generated text","author":"Munyer Travis","year":"2023","unstructured":"Travis Munyer and Xin Zhong. 2023. DeepTextMark: Deep learning based text watermarking for detection of large language model generated text. arXiv preprint arXiv:2305.05773 (2023).","journal-title":"arXiv preprint arXiv:2305.05773"},{"key":"e_1_3_2_70_2","first-page":"26106","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Ni Ansong","year":"2023","unstructured":"Ansong Ni, Srini Iyer, Dragomir Radev, Veselin Stoyanov, Wen-Tau Yih, Sida Wang, and Xi Victoria Lin. 2023. LEVER: Learning to verify language-to-code generation with execution. In Proceedings of the International Conference on Machine Learning. 26106\u201326128."},{"key":"e_1_3_2_71_2","volume-title":"Proceedings of the 11th International Conference on Learning Representations","author":"Nijkamp Erik","year":"2023","unstructured":"Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. 2023. CodeGen: An open large language model for code with multi-turn program synthesis. In Proceedings of the 11th International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=iaYcJKpY2B_"},{"key":"e_1_3_2_72_2","unstructured":"OpenAI. 2023. GPT-4 technical report. arXiv:abs\/2303.08774 (2023). https:\/\/api.semanticscholar.org\/CorpusID:257532815"},{"key":"e_1_3_2_73_2","article-title":"MarkLLM: An open-source toolkit for LLM watermarking","author":"Pan Leyi","year":"2024","unstructured":"Leyi Pan, Aiwei Liu, Zhiwei He, Zitian Gao, Xuandong Zhao, Yijian Lu, Binglin Zhou, Shuliang Liu, Xuming Hu, Lijie Wen, Irwin King, and Philip S. Yu. 2024. MarkLLM: An open-source toolkit for LLM watermarking. arXiv preprint arXiv:2405.10051 (2024).","journal-title":"arXiv preprint arXiv:2405.10051"},{"key":"e_1_3_2_74_2","first-page":"311","volume-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics","author":"Papineni Kishore","year":"2002","unstructured":"Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. 311\u2013318."},{"key":"e_1_3_2_75_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Patil Vaidehi","year":"2024","unstructured":"Vaidehi Patil, Peter Hase, and Mohit Bansal. 2024. Can sensitive information be deleted from LLMs? Objectives for defending against extraction attacks. In Proceedings of the 12th International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=7erlRDoaV8"},{"key":"e_1_3_2_76_2","doi-asserted-by":"crossref","first-page":"7653","DOI":"10.18653\/v1\/2023.acl-long.423","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Volume 1 (Long Papers)","author":"Peng Wenjun","year":"2023","unstructured":"Wenjun Peng, Jingwei Yi, Fangzhao Wu, Shangxi Wu, Bin Bin Zhu, Lingjuan Lyu, Binxing Jiao, Tong Xu, Guangzhong Sun, and Xing Xie. 2023. Are you copying my model? Protecting the copyright of large language models for EaaS via backdoor watermark. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Volume 1 (Long Papers). 7653\u20137668."},{"key":"e_1_3_2_77_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.225"},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.53761\/1.20.02.07"},{"key":"e_1_3_2_79_2","article-title":"Mark my words: Analyzing and evaluating language model watermarks","author":"Piet Julien","year":"2023","unstructured":"Julien Piet, Chawin Sitawarin, Vivian Fang, Norman Mu, and David Wagner. 2023. Mark my words: Analyzing and evaluating language model watermarks. arXiv preprint arXiv:2312.00273 (2023).","journal-title":"arXiv preprint arXiv:2312.00273"},{"key":"e_1_3_2_80_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jss.2011.12.023"},{"key":"e_1_3_2_81_2","unstructured":"Alec Radford Karthik Narasimhan Tim Salimans and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. Preprint."},{"issue":"8","key":"e_1_3_2_82_2","first-page":"9","article-title":"Language models are unsupervised multitask learners","volume":"1","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog"},{"key":"e_1_3_2_83_2","doi-asserted-by":"publisher","DOI":"10.5555\/3455716.3455856"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D19-1410"},{"key":"e_1_3_2_85_2","first-page":"Association for","volume-title":"Findings of the Association for Computational Linguistics: NAACL 2024","author":"Ren Jie","year":"2024","unstructured":"Jie Ren, Han Xu, Yiding Liu, Yingqian Cui, Shuaiqiang Wang, Dawei Yin, and Jiliang Tang. 2024. A robust semantics-based watermark for large language model against paraphrasing. In Findings of the Association for Computational Linguistics: NAACL 2024. Association for Computational Linguistics, 613\u2013625."},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1145\/2938503.2938510"},{"key":"e_1_3_2_87_2","unstructured":"Vinu Sankar Sadasivan Aounon Kumar Sriram Balasubramanian Wenxiao Wang and Soheil Feizi. 2023. Can AI-generated text be reliably detected? arxiv:2303.11156[cs.CL] (2023)."},{"key":"e_1_3_2_88_2","first-page":"8887","article-title":"Review of the literature on the steganography concept","volume":"975","author":"\u015eahin Fatih","year":"2021","unstructured":"Fatih \u015eahin, Taner \u00c7evik, and Mustafa Takao\u011flu. 2021. Review of the literature on the steganography concept. International Journal of Computer Applications 975 (2021), 8887.","journal-title":"International Journal of Computer Applications"},{"key":"e_1_3_2_89_2","article-title":"Watermarking makes language models radioactive","author":"Sander Tom","year":"2024","unstructured":"Tom Sander, Pierre Fernandez, Alain Durmus, Matthijs Douze, and Teddy Furon. 2024. Watermarking makes language models radioactive. arXiv preprint arXiv:2402.14904 (2024).","journal-title":"arXiv preprint arXiv:2402.14904"},{"key":"e_1_3_2_90_2","article-title":"Embarrassingly simple text watermarks","author":"Sato Ryoma","year":"2023","unstructured":"Ryoma Sato, Yuki Takezawa, Han Bao, Kenta Niwa, and Makoto Yamada. 2023. Embarrassingly simple text watermarks. arXiv preprint arXiv:2310.08920 (2023).","journal-title":"arXiv preprint arXiv:2310.08920"},{"key":"e_1_3_2_91_2","doi-asserted-by":"crossref","first-page":"373","DOI":"10.18653\/v1\/2022.findings-emnlp.27","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2022","author":"Shuster Kurt","year":"2022","unstructured":"Kurt Shuster, Mojtaba Komeili, Leonard Adolphs, Stephen Roller, Arthur Szlam, and Jason Weston. 2022. Language models that seek for knowledge: Modular search & generation for dialogue and prompt completion. In Findings of the Association for Computational Linguistics: EMNLP 2022. Association for Computational Linguistics, 373\u2013393."},{"key":"e_1_3_2_92_2","article-title":"CodeMark: Imperceptible watermarking for code datasets against neural code completion models","author":"Sun Zhensu","year":"2023","unstructured":"Zhensu Sun, Xiaoning Du, Fu Song, and Li Li. 2023. CodeMark: Imperceptible watermarking for code datasets against neural code completion models. arXiv preprint arXiv:2308.14401 (2023).","journal-title":"arXiv preprint arXiv:2308.14401"},{"key":"e_1_3_2_93_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485447.3512225"},{"key":"e_1_3_2_94_2","volume-title":"The Second Tiny Papers Track at ICLR 2024","author":"Suresh Tarun","year":"2024","unstructured":"Tarun Suresh, Shubham Ugare, Gagandeep Singh, and Sasa Misailovic. 2024. Is watermarking LLM-generated code robust? In The Second Tiny Papers Track at ICLR 2024. https:\/\/openreview.net\/forum?id=8PhI1PzSYY"},{"key":"e_1_3_2_95_2","article-title":"A comparative analysis of information hiding techniques for copyright protection of text documents","volume":"2018","author":"Ahvanooey Milad Taleby","year":"2018","unstructured":"Milad Taleby Ahvanooey, Qianmu Li, Hiuk Jae Shim, and Yanyan Huang. 2018. A comparative analysis of information hiding techniques for copyright protection of text documents. Security and Communication Networks 2018, 1 (2018), 1\u201322.","journal-title":"Security and Communication Networks"},{"key":"e_1_3_2_96_2","article-title":"Did you train on my dataset? Towards public dataset protection with clean-label backdoor watermarking","author":"Tang Ruixiang","year":"2023","unstructured":"Ruixiang Tang, Qizhang Feng, Ninghao Liu, Fan Yang, and Xia Hu. 2023. Did you train on my dataset? Towards public dataset protection with clean-label backdoor watermarking. arXiv preprint arXiv:2303.11470 (2023).","journal-title":"arXiv preprint arXiv:2303.11470"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02448-8"},{"key":"e_1_3_2_98_2","article-title":"LaMDA: Language models for dialog applications","author":"Thoppilan Romal","year":"2022","unstructured":"Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzhong Xu, Zhifeng Chen, Adam Roberts, Maarten Bosma, Vincent Zhao, Yanqi Zhou, Chung-Ching Chang, Igor Krivokon, Will Rusch, Marc Pickett, Pranesh Srinivasan, Laichee Man, Kathleen Meier-Hellstern, Meredith Ringel Morris, Tulsee Doshi, Renelito Delos Santos, Toju Duke, Johnny Soraker, Ben Zevenbergen, Vinodkumar Prabhakaran, Mark Diaz, Ben Hutchinson, Kristen Olson, Alejandra Molina, Erin Hoffman-John, Josh Lee, Lora Aroyo, Ravi Rajakumar, Alena Butryna, Matthew Lamm, Viktoriya Kuzmina, Joe Fenton, Aaron Cohen, Rachel Bernstein, Ray Kurzweil, Blaise Aguera-Arcas, Claire Cui, Marian Croak, Ed Chi, and Quoc Le. 2022. LaMDA: Language models for dialog applications. arXiv preprint arXiv:2201.08239 (2022).","journal-title":"arXiv preprint arXiv:2201.08239"},{"key":"e_1_3_2_99_2","doi-asserted-by":"publisher","DOI":"10.1145\/1178766.1178777"},{"key":"e_1_3_2_100_2","doi-asserted-by":"publisher","DOI":"10.1145\/1161366.1161397"},{"key":"e_1_3_2_101_2","article-title":"LLaMA: Open and efficient foundation language models","author":"Touvron Hugo","year":"2023","unstructured":"Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth\u00e9e Lacroix, Baptiste Rozi\u00e8re, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample.2023. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023).","journal-title":"arXiv preprint arXiv:2302.13971"},{"key":"e_1_3_2_102_2","article-title":"WaterBench: Towards holistic evaluation of watermarks for large language models","author":"Tu Shangqing","year":"2023","unstructured":"Shangqing Tu, Yuliang Sun, Yushi Bai, Jifan Yu, Lei Hou, and Juanzi Li. 2023. WaterBench: Towards holistic evaluation of watermarks for large language models. arXiv preprint arXiv:2311.07138 (2023).","journal-title":"arXiv preprint arXiv:2311.07138"},{"key":"e_1_3_2_103_2","doi-asserted-by":"publisher","DOI":"10.1145\/3078971.3078974"},{"key":"e_1_3_2_104_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491101.3519665"},{"key":"e_1_3_2_105_2","article-title":"HowkGPT: Investigating the detection of ChatGPT-generated university student homework through context-aware perplexity analysis","author":"Vasilatos Christoforos","year":"2023","unstructured":"Christoforos Vasilatos, Manaar Alam, Talal Rahwan, Yasir Zaki, and Michail Maniatakos. 2023. HowkGPT: Investigating the detection of ChatGPT-generated university student homework through context-aware perplexity analysis. arXiv preprint arXiv:2305.18226 (2023).","journal-title":"arXiv preprint arXiv:2305.18226"},{"key":"e_1_3_2_106_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Wang Lean","year":"2024","unstructured":"Lean Wang, Wenkai Yang, Deli Chen, Hao Zhou, Yankai Lin, Fandong Meng, Jie Zhou, and Xu Sun. 2024. Towards codable watermarking for injecting multi-bits information to LLMs. In Proceedings of the 12th International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=JYu5Flqm9D"},{"key":"e_1_3_2_107_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-demos.38"},{"key":"e_1_3_2_108_2","article-title":"Optimizing watermarks for large language models","author":"Wouters Bram","year":"2023","unstructured":"Bram Wouters. 2023. Optimizing watermarks for large language models. arXiv preprint arXiv:2312.17295 (2023).","journal-title":"arXiv preprint arXiv:2312.17295"},{"key":"e_1_3_2_109_2","article-title":"Bypassing LLM watermarks with color-aware substitutions","author":"Wu Qilong","year":"2024","unstructured":"Qilong Wu and Varun Chandrasekaran. 2024. Bypassing LLM watermarks with color-aware substitutions. arXiv preprint arXiv:2403.14719 (2024).","journal-title":"arXiv preprint arXiv:2403.14719"},{"key":"e_1_3_2_110_2","article-title":"Distortion-free watermarks are not truly distortion-free under watermark key collisions","author":"Wu Yihan","year":"2024","unstructured":"Yihan Wu, Ruibo Chen, Zhengmian Hu, Yanshuo Chen, Junfeng Guo, Hongyang Zhang, and Heng Huang. 2024. Distortion-free watermarks are not truly distortion-free under watermark key collisions. arXiv preprint arXiv:2406.02603 (2024).","journal-title":"arXiv preprint arXiv:2406.02603"},{"key":"e_1_3_2_111_2","article-title":"DiPmark: A stealthy, efficient and resilient watermark for large language models","author":"Wu Yihan","year":"2023","unstructured":"Yihan Wu, Zhengmian Hu, Hongyang Zhang, and Heng Huang. 2023. DiPmark: A stealthy, efficient and resilient watermark for large language models. arXiv preprint arXiv:2310.07710 (2023).","journal-title":"arXiv preprint arXiv:2310.07710"},{"key":"e_1_3_2_112_2","doi-asserted-by":"publisher","DOI":"10.1145\/3520312.3534862"},{"key":"e_1_3_2_113_2","article-title":"Hufu: A modality-agnositc watermarking system for pre-trained transformers via permutation equivariance","author":"Xu Hengyuan","year":"2024","unstructured":"Hengyuan Xu, Liyao Xiang, Xingjun Ma, Borui Yang, and Baochun Li. 2024. Hufu: A modality-agnositc watermarking system for pre-trained transformers via permutation equivariance. arXiv preprint arXiv:2403.05842 (2024).","journal-title":"arXiv preprint arXiv:2403.05842"},{"key":"e_1_3_2_114_2","article-title":"Learning to watermark LLM-generated text via reinforcement learning","author":"Xu Xiaojun","year":"2024","unstructured":"Xiaojun Xu, Yuanshun Yao, and Yang Liu. 2024. Learning to watermark LLM-generated text via reinforcement learning. arXiv preprint arXiv:2403.10553 (2024).","journal-title":"arXiv preprint arXiv:2403.10553"},{"key":"e_1_3_2_115_2","article-title":"Watermarking text generated by black-box language models","author":"Yang Xi","year":"2023","unstructured":"Xi Yang, Kejiang Chen, Weiming Zhang, Chang Liu, Yuang Qi, Jie Zhang, Han Fang, and Nenghai Yu. 2023. Watermarking text generated by black-box language models. arXiv preprint arXiv:2305.08883 (2023).","journal-title":"arXiv preprint arXiv:2305.08883"},{"key":"e_1_3_2_116_2","first-page":"11613","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"36","author":"Yang Xi","year":"2022","unstructured":"Xi Yang, Jie Zhang, Kejiang Chen, Weiming Zhang, Zehua Ma, Feng Wang, and Nenghai Yu. 2022. Tracing text provenance via context-aware lexical substitution. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 11613\u201311621."},{"key":"e_1_3_2_117_2","first-page":"100211","article-title":"A survey on large language model (LLM) security and privacy: The good, the bad, and the ugly","author":"Yao Yifan","year":"2024","unstructured":"Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024. A survey on large language model (LLM) security and privacy: The good, the bad, and the ugly. High-Confidence Computing 4, 2 (2024), 100211.","journal-title":"High-Confidence Computing"},{"key":"e_1_3_2_118_2","first-page":"2092","volume-title":"Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Volume 1 (Long Papers)","author":"Yoo KiYoon","year":"2023","unstructured":"KiYoon Yoo, Wonhyuk Ahn, Jiho Jang, and Nojun Kwak. 2023. Robust multi-bit natural language watermarking through invariant features. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Volume 1 (Long Papers). 2092\u20132115."},{"key":"e_1_3_2_119_2","article-title":"Advancing beyond identification: Multi-bit watermark for language models","author":"Yoo KiYoon","year":"2023","unstructured":"KiYoon Yoo, Wonhyuk Ahn, and Nojun Kwak. 2023. Advancing beyond identification: Multi-bit watermark for language models. arXiv preprint arXiv:2308.00221 (2023).","journal-title":"arXiv preprint arXiv:2308.00221"},{"key":"e_1_3_2_120_2","article-title":"REMARK-LLM: A robust and efficient watermarking framework for generative large language models","author":"Zhang Ruisi","year":"2023","unstructured":"Ruisi Zhang, Shehzeen Samarah Hussain, Paarth Neekhara, and Farinaz Koushanfar. 2023. REMARK-LLM: A robust and efficient watermarking framework for generative large language models. arXiv preprint arXiv:2310.12362 (2023).","journal-title":"arXiv preprint arXiv:2310.12362"},{"key":"e_1_3_2_121_2","article-title":"OPT: Open pre-trained transformer language models","author":"Zhang Susan","year":"2022","unstructured":"Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, Anjali Sridhar, Tianlu Wang, and Luke Zettlemoyer. 2022. OPT: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068 (2022).","journal-title":"arXiv preprint arXiv:2205.01068"},{"key":"e_1_3_2_122_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00632"},{"key":"e_1_3_2_123_2","article-title":"Large language model watermark stealing with mixed integer programming","author":"Zhang Zhaoxi","year":"2024","unstructured":"Zhaoxi Zhang, Xiaomei Zhang, Yanjun Zhang, Leo Yu Zhang, Chao Chen, Shengshan Hu, Asif Gill, and Shirui Pan. 2024. Large language model watermark stealing with mixed integer programming. arXiv preprint arXiv:2405.19677 (2024).","journal-title":"arXiv preprint arXiv:2405.19677"},{"key":"e_1_3_2_124_2","unstructured":"Wayne Xin Zhao Kun Zhou Junyi Li Tianyi Tang Xiaolei Wang Yupeng Hou Yingqian Min Beichen Zhang Junjie Zhang Zican Dong Yifan Du Chen Yang Yushuo Chen Zhipeng Chen Jinhao Jiang Ruiyang Ren Yifan Li Xinyu Tang Zikang Liu Peiyu Liu Jian-Yun Nie and Ji-Rong Wen. 2023. A survey of large language models. arxiv:2303.18223[cs.CL] (2023)."},{"key":"e_1_3_2_125_2","volume-title":"Proceedings of the 12th International Conference on Learning Representations","author":"Zhao Xuandong","year":"2024","unstructured":"Xuandong Zhao, Prabhanjan Vijendra Ananth, Lei Li, and Yu-Xiang Wang. 2024. Provable robust watermarking for AI-generated text. In Proceedings of the 12th International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=SsmT8aO45L"},{"key":"e_1_3_2_126_2","first-page":"42187","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Zhao Xuandong","year":"2023","unstructured":"Xuandong Zhao, Yu-Xiang Wang, and Lei Li. 2023. Protecting language generation models via invisible watermarking. In Proceedings of the International Conference on Machine Learning. 42187\u201342199."},{"key":"e_1_3_2_127_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544548.3581318"},{"key":"e_1_3_2_128_2","article-title":"Generative AI security: Challenges and countermeasures","author":"Zhu Banghua","year":"2024","unstructured":"Banghua Zhu, Norman Mu, Jiantao Jiao, and David Wagner. 2024. Generative AI security: Challenges and countermeasures. arXiv preprint arXiv:2402.12617 (2024).","journal-title":"arXiv preprint arXiv:2402.12617"},{"key":"e_1_3_2_129_2","article-title":"Incorporating bert into neural machine translation","author":"Zhu Jinhua","year":"2020","unstructured":"Jinhua Zhu, Yingce Xia, Lijun Wu, Di He, Tao Qin, Wengang Zhou, Houqiang Li, and Tie-Yan Liu. 2020. Incorporating bert into neural machine translation. arXiv preprint arXiv:2002.06823 (2020).","journal-title":"arXiv preprint arXiv:2002.06823"}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3691626","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3691626","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:09:40Z","timestamp":1750295380000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3691626"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,7]]},"references-count":128,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,2,28]]}},"alternative-id":["10.1145\/3691626"],"URL":"https:\/\/doi.org\/10.1145\/3691626","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,7]]},"assertion":[{"value":"2024-01-28","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-25","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-07","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}