{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,28]],"date-time":"2025-10-28T03:21:49Z","timestamp":1761621709759,"version":"3.41.0"},"reference-count":52,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,12,30]],"date-time":"2024-12-30T00:00:00Z","timestamp":1735516800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100012166","name":"National Key R&D Program of China","doi-asserted-by":"crossref","award":["2021ZD0112100"],"award-info":[{"award-number":["2021ZD0112100"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National NSF of China","award":["62336001, 62120106009"],"award-info":[{"award-number":["62336001, 62120106009"]}]},{"name":"Beijing NSF","award":["4222014"],"award-info":[{"award-number":["4222014"]}]},{"DOI":"10.13039\/501100010244","name":"Science Foundation of China University of Petroleum, Beijing","doi-asserted-by":"crossref","award":["2462024YJRC023"],"award-info":[{"award-number":["2462024YJRC023"]}],"id":[{"id":"10.13039\/501100010244","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2025,2,28]]},"abstract":"<jats:p>With the rapid evolution of deep learning and the advent of AI, the metaverse has emerged as a significant technology. Within the metaverse, diverse elements such as rich applications and realistic digital avatars provide users with immersive experiences, but it poses a series of security problems. Current research predominantly focuses on the data storage and transmission processes from the perspective of blockchain and the Internet of Things to achieve the protection of the metaverse. However, there exists a gap in security research on the digital avatar generation process. Given that digital avatars are the primary entities engaging in social activities within the metaverse and are crafted based on real face images, the virtual character can be generated easily by stealing the user\u2019s face image and controlled to interact with others. In order to deal with the above problems, we propose a novel method to prevent the misuse of faces, which maintains the security of the metaverse by protecting facial data and thus preventing its misuse. We explore the common architecture of generative models and propose a defense method based on copyright protection to prevent face embezzling. Firstly, we utilize the copyright protection module to obtain copyright protection information. Secondly, we utilized the defense control module to ensure the representation of the protected images occurs errors in the latent space of the generation model. Therefore, the subsequent generation task output fails, which effectively protects the face data and prevents the generation of digital avatars. Furthermore, the results on public datasets and across multiple generative models present unnatural outputs, indicating the excellence of our defense and transfer capabilities.<\/jats:p>","DOI":"10.1145\/3699718","type":"journal-article","created":{"date-parts":[[2024,10,8]],"date-time":"2024-10-08T15:39:00Z","timestamp":1728401940000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["FaceDefend: Copyright Protection to Prevent Face Embezzle"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-1797-8139","authenticated-orcid":false,"given":"Rui","family":"Zhai","sequence":"first","affiliation":[{"name":"Institute of Information Science, Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing Jiaotong University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5096-8752","authenticated-orcid":false,"given":"Rongrong","family":"Ni","sequence":"additional","affiliation":[{"name":"Institute of Information Science, Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing Jiaotong University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5204-2929","authenticated-orcid":false,"given":"Yang","family":"Yu","sequence":"additional","affiliation":[{"name":"Department of Electronic Information Engineering, School of Artificial Intelligence, China University of Petroleum, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8581-9554","authenticated-orcid":false,"given":"Yao","family":"Zhao","sequence":"additional","affiliation":[{"name":"Institute of Information Science, Beijing Key Laboratory of Advanced Information Science and Network Technology, Beijing Jiaotong University, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,12,30]]},"reference":[{"doi-asserted-by":"publisher","key":"e_1_3_1_2_2","DOI":"10.1109\/ICCV.2019.00453"},{"doi-asserted-by":"publisher","key":"e_1_3_1_3_2","DOI":"10.1007\/11513988_27"},{"key":"e_1_3_1_4_2","first-page":"187","volume-title":"Proceedings of the 26th annual conference on Computer Graphics and Interactive Techniques","author":"Blanz Volker","unstructured":"Volker Blanz and Thomas Vetter. A morphable model for the synthesis of 3D faces. In Proceedings of the 26th annual conference on Computer Graphics and Interactive Techniques, 187\u2013194."},{"doi-asserted-by":"publisher","key":"e_1_3_1_5_2","DOI":"10.1109\/CVPR52688.2022.01565"},{"doi-asserted-by":"publisher","key":"e_1_3_1_6_2","DOI":"10.1007\/s10676-018-9480-6"},{"doi-asserted-by":"publisher","key":"e_1_3_1_7_2","DOI":"10.1109\/CVPR.2018.00916"},{"doi-asserted-by":"publisher","key":"e_1_3_1_8_2","DOI":"10.1109\/ICCV.2017.335"},{"doi-asserted-by":"publisher","key":"e_1_3_1_9_2","DOI":"10.1145\/3359626"},{"doi-asserted-by":"publisher","key":"e_1_3_1_10_2","DOI":"10.1109\/MTS.2018.2826060"},{"doi-asserted-by":"publisher","key":"e_1_3_1_11_2","DOI":"10.32604\/cmc.2018.02356"},{"doi-asserted-by":"publisher","key":"e_1_3_1_12_2","DOI":"10.1109\/TCSS.2022.3226717"},{"doi-asserted-by":"publisher","key":"e_1_3_1_13_2","DOI":"10.1145\/3292500.3330866"},{"key":"e_1_3_1_14_2","first-page":"2468","volume-title":"Proceedings of the IEEE Smartworld, Ubiquitous Intelligence & Computing, Scalable Computing & Communications, Digital Twin, Privacy Computing, Metaverse, Autonomous & Trusted Vehicles (SmartWorld\/UIC\/ScalCom\/DigitalTwin\/PriComp\/Meta \u201922)","author":"Ghirmai Siem","year":"2022","unstructured":"Siem Ghirmai, Daniel Mebrahtom, Moayad Aloqaily, Mohsen Guizani, and Merouane Debbah. 2022. Self-sovereign identity for trust and interoperability in the metaverse. In Proceedings of the IEEE Smartworld, Ubiquitous Intelligence & Computing, Scalable Computing & Communications, Digital Twin, Privacy Computing, Metaverse, Autonomous & Trusted Vehicles (SmartWorld\/UIC\/ScalCom\/DigitalTwin\/PriComp\/Meta \u201922). IEEE, 2468\u20132475."},{"issue":"2","key":"e_1_3_1_15_2","first-page":"1","article-title":"Hierarchical multi-attention transfer for knowledge distillation","volume":"20","author":"Gou Jianping","year":"2023","unstructured":"Jianping Gou, Liyuan Sun, Baosheng Yu, Shaohua Wan, and Dacheng Tao. 2023. Hierarchical multi-attention transfer for knowledge distillation. ACM Transactions on Multimedia Computing, Communications and Applications 20, 2 (2023), 1\u201320.","journal-title":"ACM Transactions on Multimedia Computing, Communications and Applications"},{"key":"e_1_3_1_16_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Gu Jiatao","year":"2021","unstructured":"Jiatao Gu, Lingjie Liu, Peng Wang, and Christian Theobalt. 2021. StyleNeRF: A style-based 3D-aware generator for high-resolution image synthesis. In Proceedings of the International Conference on Learning Representations, 2022."},{"issue":"4","key":"e_1_3_1_17_2","first-page":"2118","article-title":"Paradefender: A scenario-driven parallel system for defending metaverses","volume":"53","author":"Han Jinpeng","year":"2022","unstructured":"Jinpeng Han, Manzhi Yang, Xiaoguang Chen, Hongtao Liu, Yuntao Wang, Jianhao Li, Zhou Su, Zhen Li, and Xiaobo Ma. 2022. Paradefender: A scenario-driven parallel system for defending metaverses. IEEE Transactions on Systems, Man, and Cybernetics: Systems 53, 4 (2022), 2118\u20132127.","journal-title":"IEEE Transactions on Systems, Man, and Cybernetics: Systems"},{"unstructured":"Y. Han D. Niyato C. Leung D. I. Kim K. Zhu S. Feng and C. Miao. 2022. A dynamic hierarchical framework for loT-assisted metaverse synchronization. arXiv:2203.03969. Retrieved from https:\/\/arxiv.org\/abs\/2203.03969v3","key":"e_1_3_1_18_2"},{"issue":"9","key":"e_1_3_1_19_2","doi-asserted-by":"crossref","first-page":"3998","DOI":"10.1002\/mp.13656","article-title":"Paired cycle-GAN-based image correction for quantitative cone-beam computed tomography","volume":"46","author":"Harms Joseph","year":"2019","unstructured":"Joseph Harms, Yang Lei, Tonghe Wang, Rongxiao Zhang, Jun Zhou, Xiangyang Tang, Walter J. Curran, Tian Liu, and Xiaofeng Yang. 2019. Paired cycle-GAN-based image correction for quantitative cone-beam computed tomography. Medical Physics 46, 9 (2019), 3998\u20134009.","journal-title":"Medical Physics"},{"doi-asserted-by":"publisher","key":"e_1_3_1_20_2","DOI":"10.1109\/TIP.2019.2916751"},{"doi-asserted-by":"publisher","key":"e_1_3_1_21_2","DOI":"10.1109\/CVPR.2019.00453"},{"doi-asserted-by":"publisher","key":"e_1_3_1_22_2","DOI":"10.1109\/CVPR42600.2020.00813"},{"doi-asserted-by":"publisher","unstructured":"Diederik P. Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv:1312.6114. Retrieved from https:\/\/doi.org\/10.48550\/arXiv.1312.6114","key":"e_1_3_1_23_2","DOI":"10.48550\/arXiv.1312.6114"},{"key":"e_1_3_1_24_2","first-page":"2967","volume-title":"Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision","author":"Ko Jaehoon","year":"2023","unstructured":"Jaehoon Ko, Kyusun Cho, Daewon Choi, Kwangrok Ryoo, and Seungryong Kim. 2023. 3D GAN inversion with pose optimization. In Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, 2967\u20132976."},{"key":"e_1_3_1_25_2","first-page":"20940","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Lan Yushi","year":"2023","unstructured":"Yushi Lan, Xuyi Meng, Shuai Yang, Chen Change Loy, and Bo Dai. 2023. Self-supervised geometry-aware encoder for style-based 3d gan inversion. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 20940\u201320949."},{"doi-asserted-by":"publisher","key":"e_1_3_1_26_2","DOI":"10.1145\/3532640.3532665"},{"doi-asserted-by":"publisher","key":"e_1_3_1_27_2","DOI":"10.1109\/CVPR.2019.00379"},{"issue":"6","key":"e_1_3_1_28_2","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1109\/MNET.2023.3317320","article-title":"Privacy attacks and defenses for digital twin migrations in vehicular metaverses","volume":"37","author":"Luo Xiaofeng","year":"2023","unstructured":"Xiaofeng Luo, Jinbo Wen, Jiawen Kang, Jiangtian Nie, Zehui Xiong, Yang Zhang, Zhaohui Yang, and Shengli Xie. 2023. Privacy attacks and defenses for digital twin migrations in vehicular metaverses. IEEE Network 37, 6 (2023), 82\u201391.","journal-title":"IEEE Network"},{"doi-asserted-by":"publisher","key":"e_1_3_1_29_2","DOI":"10.1016\/j.compag.2022.107544"},{"doi-asserted-by":"publisher","key":"e_1_3_1_30_2","DOI":"10.1109\/ICCV48922.2021.00554"},{"key":"e_1_3_1_31_2","first-page":"13503","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Or-El Roy","year":"2022","unstructured":"Roy Or-El, Xuan Luo, Mengyi Shan, Eli Shechtman, Jeong Joon Park, and Ira Kemelmacher-Shlizerman. 2022. StyleSDF: High-resolution 3D-consistent image and geometry generation. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 13503\u201313513."},{"doi-asserted-by":"publisher","key":"e_1_3_1_32_2","DOI":"10.1007\/s11948-014-9621-1"},{"doi-asserted-by":"publisher","key":"e_1_3_1_33_2","DOI":"10.1007\/s40860-020-00098-y"},{"doi-asserted-by":"publisher","key":"e_1_3_1_34_2","DOI":"10.1109\/AVSS.2009.58"},{"unstructured":"Justin N. M. Pinkney and Doron Adler. 2020. Resolution dependent GAN interpolation for controllable image synthesis between domains. arXiv:2010.05334. Retrieved from https:\/\/arxiv.org\/abs\/2010.05334","key":"e_1_3_1_35_2"},{"doi-asserted-by":"publisher","key":"e_1_3_1_36_2","DOI":"10.1109\/CVPR46437.2021.00232"},{"doi-asserted-by":"publisher","key":"e_1_3_1_37_2","DOI":"10.1145\/3550469.3555402"},{"doi-asserted-by":"publisher","key":"e_1_3_1_38_2","DOI":"10.1145\/3450626.3459771"},{"doi-asserted-by":"publisher","key":"e_1_3_1_39_2","DOI":"10.1097\/APO.0000000000000537"},{"doi-asserted-by":"publisher","key":"e_1_3_1_40_2","DOI":"10.1145\/3450626.3459838"},{"doi-asserted-by":"publisher","key":"e_1_3_1_41_2","DOI":"10.1109\/CVPR.2019.00122"},{"key":"e_1_3_1_42_2","first-page":"27171","article-title":"Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction","volume":"34","author":"Wang Peng","year":"2021","unstructured":"Peng Wang, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. 2021. Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction. Advances in Neural Information Processing Systems 34, (2021), 27171\u201327183.","journal-title":"Advances in Neural Information Processing Systems"},{"doi-asserted-by":"publisher","key":"e_1_3_1_43_2","DOI":"10.1109\/COMST.2022.3202047"},{"issue":"11","key":"e_1_3_1_44_2","first-page":"1","article-title":"Semantic map guided identity transfer GAN for person re-identification","volume":"20","author":"Wu Tian","year":"2023","unstructured":"Tian Wu, Rongbo Zhu, and Shaohua Wan. 2023. Semantic map guided identity transfer GAN for person re-identification. ACM Transactions on Multimedia Computing, Communications and Applications 20, 11 (2023), 1\u201320.","journal-title":"ACM Transactions on Multimedia Computing, Communications and Applications"},{"doi-asserted-by":"publisher","key":"e_1_3_1_45_2","DOI":"10.1109\/TNSM.2023.3236673"},{"key":"e_1_3_1_46_2","volume-title":"Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Xie Jiaxin","year":"2023","unstructured":"Jiaxin Xie, Hao Ouyang, Jingtan Piao, Chenyang Lei, and Qifeng Chen. 2023. High-fidelity 3D GAN inversion by pseudo-multi-view optimization. In Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition."},{"key":"e_1_3_1_47_2","first-page":"342","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Yin Fei","year":"2023","unstructured":"Fei Yin, Yong Zhang, Xuan Wang, Tengfei Wang, Xiaoyu Li, Yuan Gong, Yanbo Fan, Xiaodong Cun, Ying Shan, Cengiz Oztireli, et al. 2023. 3D GAN inversion with facial symmetry prior. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, 342\u2013351."},{"key":"e_1_3_1_48_2","first-page":"2437","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Yuan Ziyang","year":"2023","unstructured":"Ziyang Yuan, Yiming Zhu, Yu Li, Hongyu Liu, and Chun Yuan. 2023. Make encoder great again in 3D GAN inversion through geometry and occlusion-aware encoding. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 2437\u20132447."},{"doi-asserted-by":"publisher","key":"e_1_3_1_49_2","DOI":"10.1109\/CVPR.2018.00068"},{"doi-asserted-by":"publisher","key":"e_1_3_1_50_2","DOI":"10.57019\/jmv.1286526"},{"doi-asserted-by":"publisher","key":"e_1_3_1_51_2","DOI":"10.1109\/MNET.2023.3318890"},{"issue":"2","key":"e_1_3_1_52_2","first-page":"1484","article-title":"A multigranularity forensics and analysis method on privacy leakage in cloud environment","volume":"6","author":"Zou Deqing","year":"2018","unstructured":"Deqing Zou, Jian Zhao, Weiming Li, Yueming Wu, Weizhong Qiang, Hai Jin, Ye Wu, and Yifei Yang. 2018. A multigranularity forensics and analysis method on privacy leakage in cloud environment. IEEE Internet of Things Journal 6, 2 (2018), 1484\u20131494.","journal-title":"IEEE Internet of Things Journal"},{"key":"e_1_3_1_53_2","first-page":"3925","volume-title":"Proceedings of the 32nd USENIX Security Symposium (USENIX Security \u201923)","author":"Zuo Chaoshun","year":"2023","unstructured":"Chaoshun Zuo, Chao Wang, and Zhiqiang Lin. 2023. A peek into the metaverse: Detecting 3D model clones in mobile games. In Proceedings of the 32nd USENIX Security Symposium (USENIX Security \u201923), 3925\u20133942."}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3699718","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3699718","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T01:09:53Z","timestamp":1750295393000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3699718"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,30]]},"references-count":52,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,2,28]]}},"alternative-id":["10.1145\/3699718"],"URL":"https:\/\/doi.org\/10.1145\/3699718","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2024,12,30]]},"assertion":[{"value":"2023-12-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-23","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}