{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T19:40:23Z","timestamp":1774554023785,"version":"3.50.1"},"reference-count":36,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2025,1,21]],"date-time":"2025-01-21T00:00:00Z","timestamp":1737417600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGKDD Explor. Newsl."],"published-print":{"date-parts":[[2025,1,21]]},"abstract":"<jats:p>Text-to-image generative models, especially those based on latent diffusion models (LDMs), have demonstrated outstanding ability in generating high-quality and high-resolution images from textual prompts. With this advancement, various fine-tuning methods have been developed to personalize text-to-image models for specific applications such as artistic style adaptation and human face transfer. However, such advancements have raised copyright concerns, especially when the data are used for personalization without authorization. For example, a malicious user can employ fine-tuning techniques to replicate the style of an artist without consent. In light of this concern, we propose FT-Shield, a watermarking solution tailored for the fine-tuning of text-to-image diffusion models. FT-Shield addresses copyright protection challenges by designing new watermark generation and detection strategies. In particular, it introduces an innovative algorithm for watermark generation. It ensures the seamless transfer of watermarks from training images to generated outputs, facilitating the identification of copyrighted material use. To tackle the variability in fine-tuning methods and their impact on watermark detection, FT-Shield integrates a Mixture of Experts (MoE) approach for watermark detection. Comprehensive experiments validate the effectiveness of our proposed FT-Shield.<\/jats:p>","DOI":"10.1145\/3715073.3715080","type":"journal-article","created":{"date-parts":[[2025,1,29]],"date-time":"2025-01-29T23:19:36Z","timestamp":1738192776000},"page":"76-88","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":6,"title":["FT-Shield: A Watermark Against Unauthorized Fine-tuning in Text-to-Image Diffusion Models"],"prefix":"10.1145","volume":"26","author":[{"given":"Yingqian","family":"Cui","sequence":"first","affiliation":[{"name":"Michigan State University"}]},{"given":"Jie","family":"Ren","sequence":"additional","affiliation":[{"name":"Michigan State University"}]},{"given":"Yuping","family":"Lin","sequence":"additional","affiliation":[{"name":"Michigan State University"}]},{"given":"Han","family":"Xu","sequence":"additional","affiliation":[{"name":"The University of Arizona"}]},{"given":"Pengfei","family":"He","sequence":"additional","affiliation":[{"name":"Michigan State University"}]},{"given":"Yue","family":"Xing","sequence":"additional","affiliation":[{"name":"Michigan State University"}]},{"given":"Lingjuan","family":"Lyu","sequence":"additional","affiliation":[{"name":"Sony AI"}]},{"given":"Wenqi","family":"Fan","sequence":"additional","affiliation":[{"name":"The Hong Kong Polytechnic University"}]},{"given":"Hui","family":"Liu","sequence":"additional","affiliation":[{"name":"Michigan State University"}]},{"given":"Jiliang","family":"Tang","sequence":"additional","affiliation":[{"name":"Michigan State University"}]}],"member":"320","published-online":{"date-parts":[[2025,1,29]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1109\/FOCS52979.2021.00098"},{"key":"e_1_2_1_2_1","volume-title":"A pathway towards responsible AI generated content. arXiv preprint arXiv:2303.01325","author":"Chen Chen","year":"2023","unstructured":"Chen Chen, Jie Fu, and Lingjuan Lyu. A pathway towards responsible AI generated content. arXiv preprint arXiv:2303.01325, 2023."},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1117\/1.1494075"},{"key":"e_1_2_1_4_1","volume-title":"Diffusion models beat GANs on image synthesis. Advances in neural information processing systems, 34:8780--8794","author":"Dhariwal Prafulla","year":"2021","unstructured":"Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. Advances in neural information processing systems, 34:8780--8794, 2021."},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.5555\/1209579"},{"key":"e_1_2_1_6_1","volume-title":"An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618","author":"Gal Rinon","year":"2022","unstructured":"Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit H Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022."},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_8_1","volume-title":"GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Advances in neural information processing systems, 30","author":"Heusel Martin","year":"2017","unstructured":"Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. GANs trained by a two time-scale update rule converge to a local Nash equilibrium. Advances in neural information processing systems, 30, 2017."},{"key":"e_1_2_1_9_1","first-page":"6840","article-title":"Denoising diffusion probabilistic models","volume":"33","author":"Ho Jonathan","year":"2020","unstructured":"Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840--6851, 2020.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_10_1","volume-title":"Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598","author":"Ho Jonathan","year":"2022","unstructured":"Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598, 2022."},{"key":"e_1_2_1_11_1","volume-title":"LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685","author":"Hu Edward J","year":"2021","unstructured":"Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021."},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1991.3.1.79"},{"key":"e_1_2_1_13_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980","author":"Kingma Diederik P","year":"2014","unstructured":"Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014."},{"key":"e_1_2_1_14_1","volume-title":"Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples","author":"Liang Chumeng","year":"2023","unstructured":"Chumeng Liang, Xiaoyu Wu, Yang Hua, Jiaru Zhang, Yiming Xue, Tao Song, Xue Zhengui, Ruhui Ma, and Haibing Guan. Adversarial example does good: Preventing painting imitation from diffusion models via adversarial examples. 2023."},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.425"},{"key":"e_1_2_1_16_1","volume-title":"Generative watermarking against unauthorized subject-driven image synthesis. arXiv preprint arXiv:2306.07754","author":"Ma Yihan","year":"2023","unstructured":"Yihan Ma, Zhengyu Zhao, Xinlei He, Zheng Li, Michael Backes, and Yang Zhang. Generative watermarking against unauthorized subject-driven image synthesis. arXiv preprint arXiv:2306.07754, 2023."},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the international conference on learning representations","author":"Madry Aleksander","year":"2018","unstructured":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In Proceedings of the international conference on learning representations, 2018."},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/COMSWA.2008.4554423"},{"key":"e_1_2_1_19_1","volume-title":"WANet--imperceptible warping-based backdoor attack. arXiv preprint arXiv:2102.10369","author":"Nguyen Anh","year":"2021","unstructured":"Anh Nguyen and Anh Tran. WANet--imperceptible warping-based backdoor attack. arXiv preprint arXiv:2102.10369, 2021."},{"key":"e_1_2_1_20_1","volume-title":"Pokemon blip captions. https:\/\/huggingface.co\/datasets\/lambdalabs\/pokemon-blip-captions\/","author":"Pinkney Justin N. M.","year":"2022","unstructured":"Justin N. M. Pinkney. Pokemon blip captions. https:\/\/huggingface.co\/datasets\/lambdalabs\/pokemon-blip-captions\/, 2022."},{"key":"e_1_2_1_21_1","volume-title":"Digital watermarking: Algorithms and applications","author":"Podilchuk Christine I","year":"2001","unstructured":"Christine I Podilchuk and Edward J Delp. Digital watermarking: Algorithms and applications. IEEE signal processing Magazine, 18(4):33--46, 2001."},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"e_1_2_1_23_1","volume-title":"U-Net: Convolutional networks for biomedical image segmentation","author":"Ronneberger Olaf","year":"2015","unstructured":"Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation, 2015."},{"key":"e_1_2_1_24_1","volume-title":"Dreambooth: Finetuning text-to-image diffusion models for subject-driven generation","author":"Ruiz Nataniel","year":"2022","unstructured":"Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Finetuning text-to-image diffusion models for subject-driven generation. 2022."},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.02155"},{"key":"e_1_2_1_26_1","volume-title":"Raising the cost of malicious AI-powered image editing. arXiv preprint arXiv:2302.06588","author":"Salman Hadi","year":"2023","unstructured":"Hadi Salman, Alaa Khaddaj, Guillaume Leclerc, Andrew Ilyas, and Aleksander Madry. Raising the cost of malicious AI-powered image editing. arXiv preprint arXiv:2302.06588, 2023."},{"key":"e_1_2_1_27_1","volume-title":"Glaze: Protecting artists from style mimicry by text-to-image models. arXiv preprint arXiv:2302.04222","author":"Shan Shawn","year":"2023","unstructured":"Shawn Shan, Jenna Cryan, Emily Wenger, Haitao Zheng, Rana Hanocka, and Ben Y Zhao. Glaze: Protecting artists from style mimicry by text-to-image models. arXiv preprint arXiv:2302.04222, 2023."},{"key":"e_1_2_1_28_1","volume-title":"Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538","author":"Shazeer Noam","year":"2017","unstructured":"Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538, 2017."},{"key":"e_1_2_1_29_1","volume-title":"Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502","author":"Song Jiaming","year":"2020","unstructured":"Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020."},{"key":"e_1_2_1_30_1","volume-title":"Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456","author":"Song Yang","year":"2020","unstructured":"Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456, 2020."},{"key":"e_1_2_1_31_1","volume-title":"Quan Dao, Ngoc Tran, and Anh Tran. Anti-dreambooth: Protecting users from personalized text-to-image synthesis. arXiv preprint arXiv:2303.15433","author":"Le Thanh Van","year":"2023","unstructured":"Thanh Van Le, Hao Phung, Thuan Hoang Nguyen, Quan Dao, Ngoc Tran, and Anh Tran. Anti-dreambooth: Protecting users from personalized text-to-image synthesis. arXiv preprint arXiv:2303.15433, 2023."},{"key":"e_1_2_1_32_1","volume-title":"Diffusers: State-of-the-art diffusion models. https:\/\/github.com\/huggingface\/diffusers\/tree\/main\/examples\/text_to_image","author":"von Platen Patrick","year":"2022","unstructured":"Patrick von Platen, Suraj Patil, Anton Lozhkov, Pedro Cuenca, Nathan Lambert, Kashif Rasul, Mishig Davaadorj, and Thomas Wolf. Diffusers: State-of-the-art diffusion models. https:\/\/github.com\/huggingface\/diffusers\/tree\/main\/examples\/text_to_image, 2022."},{"key":"e_1_2_1_33_1","volume-title":"International Conference on Learning Representations","author":"Wang Zhenting","year":"2024","unstructured":"Zhenting Wang, Chen Chen, Lingjuan Lyu, Dimitris Metaxas, and Shiqing Ma. Diagnosis: Detecting unauthorized data usages in text-to-image diffusion models. In International Conference on Learning Representations, 2024."},{"key":"e_1_2_1_34_1","first-page":"36","article-title":"Origin attribution of AI-generated images","author":"Wang Zhenting","year":"2024","unstructured":"Zhenting Wang, Chen Chen, Yi Zeng, Lingjuan Lyu, and Shiqing Ma. Where did I come from? Origin attribution of AI-generated images. Advances in Neural Information Processing Systems, 36, 2024.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_35_1","volume-title":"The Twelfth International Conference on Learning Representations","author":"Wen Yuxin","year":"2023","unstructured":"Yuxin Wen, Yuchen Liu, Chen Chen, and Lingjuan Lyu. Detecting, explaining, and mitigating memorization in diffusion models. In The Twelfth International Conference on Learning Representations, 2023."},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-01267-0_40"}],"container-title":["ACM SIGKDD Explorations Newsletter"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715073.3715080","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3715073.3715080","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:56:53Z","timestamp":1750298213000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3715073.3715080"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,1,21]]},"references-count":36,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,1,21]]}},"alternative-id":["10.1145\/3715073.3715080"],"URL":"https:\/\/doi.org\/10.1145\/3715073.3715080","relation":{},"ISSN":["1931-0145","1931-0153"],"issn-type":[{"value":"1931-0145","type":"print"},{"value":"1931-0153","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,21]]},"assertion":[{"value":"2025-01-29","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}