{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,7]],"date-time":"2026-04-07T16:13:29Z","timestamp":1775578409523,"version":"3.50.1"},"reference-count":64,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2024,8,9]],"date-time":"2024-08-09T00:00:00Z","timestamp":1723161600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62102162"],"award-info":[{"award-number":["62102162"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100012401","name":"Beijing Science and Technology Plan Project","doi-asserted-by":"crossref","award":["Z231100005923033"],"award-info":[{"award-number":["Z231100005923033"]}],"id":[{"id":"10.13039\/501100012401","id-type":"DOI","asserted-by":"crossref"}]},{"name":"242 project","award":["2023A078"],"award-info":[{"award-number":["2023A078"]}]},{"name":"National Science and Technology Council","award":["111-2221-E-006-112-MY3"],"award-info":[{"award-number":["111-2221-E-006-112-MY3"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Graph."],"published-print":{"date-parts":[[2024,10,31]]},"abstract":"<jats:p>In this study, we revisit the fundamental setting of face-swapping models and reveal that only using implicit supervision for training leads to the difficulty of advanced methods to preserve the source identity. We propose a novel reverse pseudo-input generation approach to offer supplemental data for training face-swapping models, which addresses the aforementioned issue. Unlike the traditional pseudo-label-based training strategy, we assume that arbitrary real facial images could serve as the ground-truth outputs for the face-swapping network and try to generate corresponding input &lt;source, target&gt; pair data. Specifically, we involve a source-creating surrogate that alters the attributes of the real image while keeping the identity, and a target-creating surrogate intends to synthesize attribute-preserved target images with different identities. Our framework, which utilizes proxy-paired data as explicit supervision to direct the face-swapping training process, partially fulfills a credible and effective optimization direction to boost the identity-preserving capability. We design explicit and implicit adaption strategies to better approximate the explicit supervision for face swapping. Quantitative and qualitative experiments on FF++, FFHQ, and wild images show that our framework could improve the performance of various face-swapping pipelines in terms of visual fidelity and ID preserving. Furthermore, we display applications with our method on re-aging, swappable attribute customization, cross-domain, and video face swapping. Code is available under https:\/\/github.com\/ ICTMCG\/CSCS.<\/jats:p>","DOI":"10.1145\/3676165","type":"journal-article","created":{"date-parts":[[2024,7,1]],"date-time":"2024-07-01T11:12:51Z","timestamp":1719832371000},"page":"1-19","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":11,"title":["Identity-Preserving Face Swapping via Dual Surrogate Generative Models"],"prefix":"10.1145","volume":"43","author":[{"ORCID":"https:\/\/orcid.org\/0009-0004-3141-9979","authenticated-orcid":false,"given":"Ziyao","family":"Huang","sequence":"first","affiliation":[{"name":"Institute of Computing Technology Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3975-2483","authenticated-orcid":false,"given":"Fan","family":"Tang","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0066-3448","authenticated-orcid":false,"given":"Yong","family":"Zhang","sequence":"additional","affiliation":[{"name":"Tencent, Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7857-1546","authenticated-orcid":false,"given":"Juan","family":"Cao","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-6373-9941","authenticated-orcid":false,"given":"Chengyu","family":"Li","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3573-2407","authenticated-orcid":false,"given":"Sheng","family":"Tang","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4597-8534","authenticated-orcid":false,"given":"Jintao","family":"Li","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology Chinese Academy of Sciences, Beijing, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6699-2944","authenticated-orcid":false,"given":"Tong-Yee","family":"Lee","sequence":"additional","affiliation":[{"name":"National Cheng Kung University, Tainan, Taiwan"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,8,9]]},"reference":[{"key":"e_1_3_4_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/1399504.1360638"},{"key":"e_1_3_4_3_1","doi-asserted-by":"crossref","unstructured":"Volker Blanz Kristina Scherbaum Thomas Vetter and Hans-Peter Seidel. 2004. Exchanging faces in images. In Computer Graphics Forum. Wiley Online Library 669\u2013676.","DOI":"10.1111\/j.1467-8659.2004.00799.x"},{"key":"e_1_3_4_4_1","doi-asserted-by":"publisher","DOI":"10.1145\/311535.311556"},{"key":"e_1_3_4_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3413630"},{"key":"e_1_3_4_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00482"},{"key":"e_1_3_4_7_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3051251"},{"key":"e_1_3_4_8_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2006.90"},{"key":"e_1_3_4_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2009.24"},{"key":"e_1_3_4_10_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00341"},{"key":"e_1_3_4_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3422622"},{"key":"e_1_3_4_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58529-7_10"},{"key":"e_1_3_4_13_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46487-9_6"},{"key":"e_1_3_4_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3114308"},{"key":"e_1_3_4_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3527168"},{"key":"e_1_3_4_16_1","volume-title":"Proceedings of International Conference on Learning Representations (ICLR\u201918)","author":"Karras Tero","year":"2018","unstructured":"Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive growing of GANs for improved quality, stability, and variation. In Proceedings of International Conference on Learning Representations (ICLR\u201918)."},{"key":"e_1_3_4_17_1","first-page":"12104","article-title":"Training generative adversarial networks with limited data","volume":"33","author":"Karras Tero","year":"2020","unstructured":"Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. 2020a. Training generative adversarial networks with limited data. Adv. Neural Inf. Process. Syst. 33 (2020), 12104\u201312114.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_4_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_3_4_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"e_1_3_4_20_1","first-page":"18661","article-title":"Supervised contrastive learning","volume":"33","author":"Khosla Prannay","year":"2020","unstructured":"Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. 2020. Supervised contrastive learning. Adv. Neural Inf. Process. Syst. 33 (2020), 18661\u201318673.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_4_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01051"},{"key":"e_1_3_4_22_1","volume-title":"Proceedings of the 3rd International Conference for Learning Representations","author":"Kingma Diederik P.","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference for Learning Representations."},{"key":"e_1_3_4_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01006"},{"key":"e_1_3_4_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00512"},{"key":"e_1_3_4_25_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14704"},{"key":"e_1_3_4_26_1","doi-asserted-by":"crossref","unstructured":"Jingwang Ling Zhibo Wang Ming Lu Quan Wang Chen Qian and Feng Xu. 2022. Semantically disentangled variational autoencoder for modeling 3d facial details. IEEE Transactions on Visualization and Computer Graphics 29 8 (2022) 3630\u20133641.","DOI":"10.1109\/TVCG.2022.3166666"},{"key":"e_1_3_4_27_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3107669"},{"key":"e_1_3_4_28_1","first-page":"6629","article-title":"AdaAttN: Revisit attention mechanism in arbitrary neural style transfer","author":"Liu Songhua","year":"2021","unstructured":"Songhua Liu, Tianwei Lin, Dongliang He, Fu Li, Meiling Wang, Xin Li, Zhengxing Sun, Qian Li, and Errui Ding. 2021a. AdaAttN: Revisit attention mechanism in arbitrary neural style transfer. In Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV\u201921), 6629\u20136638.","journal-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision (ICCV\u201921)"},{"key":"e_1_3_4_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.2967036"},{"key":"e_1_3_4_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00829"},{"key":"e_1_3_4_31_1","first-page":"3481","volume-title":"Proceedings of the International Conference on Machine Learning (ICML\u201918)","author":"Mescheder Lars","year":"2018","unstructured":"Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. 2018. Which training methods for GANs do actually converge? In Proceedings of the International Conference on Machine Learning (ICML\u201918). PMLR, 3481\u20133490."},{"key":"e_1_3_4_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58452-8_24"},{"key":"e_1_3_4_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3425780"},{"key":"e_1_3_4_34_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14062"},{"key":"e_1_3_4_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00728"},{"key":"e_1_3_4_36_1","first-page":"98","article-title":"On face segmentation, face swapping, and face perception","author":"Nirkin Yuval","year":"2017","unstructured":"Yuval Nirkin, Iacopo Masi, A. Tran, Tal Hassner, and G\u00e9rard G. Medioni. 2017. On face segmentation, face swapping, and face perception. In Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG\u201918), 98\u2013105.","journal-title":"Proceedings of the 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG\u201918)"},{"key":"e_1_3_4_37_1","doi-asserted-by":"publisher","DOI":"10.1111\/cgf.14705"},{"key":"e_1_3_4_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3052973.3053009"},{"key":"e_1_3_4_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2007.22"},{"key":"e_1_3_4_40_1","doi-asserted-by":"crossref","unstructured":"Kunlin Liu Ivan Perov Daiheng Gao Nikolay Chervoniy Wenbo Zhou and Weiming Zhang. 2023. Deepfacelab: Integrated flexible and extensible face-swapping framework. Pattern Recognition 141 (2023) 109628.","DOI":"10.1016\/j.patcog.2023.109628"},{"key":"e_1_3_4_41_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01350"},{"key":"e_1_3_4_42_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00009"},{"key":"e_1_3_4_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3272127.3275019"},{"key":"e_1_3_4_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"e_1_3_4_45_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2020.3034267"},{"key":"e_1_3_4_46_1","doi-asserted-by":"publisher","unstructured":"Xinhui Song Chen Liu Youyi Zheng Zunlei Feng Lincheng Li Kun Zhou and Xin Yu. 2024. HairStyle editing via parametric controllable strokes. IEEE Transactions on Visualization and Computer Graphics 30 7 (2024) 3857\u20133870. 10.1109\/TVCG.2023.3241894","DOI":"10.1109\/TVCG.2023.3241894"},{"key":"e_1_3_4_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/2929464.2929475"},{"key":"e_1_3_4_48_1","article-title":"Stitch it in Time: GAN-based facial editing of real videos","author":"Tzaban Rotem","year":"2022","unstructured":"Rotem Tzaban, Ron Mokady, Rinon Gal, Amit Haim Bermano, and Daniel Cohen-Or. 2022. Stitch it in Time: GAN-based facial editing of real videos. In Proceedings of the SIGGRAPH Asia Conference.","journal-title":"Proceedings of the SIGGRAPH Asia Conference"},{"key":"e_1_3_4_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2021.3139913"},{"key":"e_1_3_4_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00552"},{"key":"e_1_3_4_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00991"},{"key":"e_1_3_4_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00905"},{"key":"e_1_3_4_53_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2021\/157"},{"key":"e_1_3_4_54_1","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR\u201921)","author":"Wang Yaohui","year":"2021","unstructured":"Yaohui Wang, Di Yang, Francois Bremond, and Antitza Dantcheva. 2021. Latent image animator: Learning to animate images via latent space navigation. In Proceedings of the International Conference on Learning Representations (ICLR\u201921)."},{"key":"e_1_3_4_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/TVCG.2020.3023573"},{"key":"e_1_3_4_56_1","doi-asserted-by":"publisher","unstructured":"Wenpeng Xiao Cheng Xu Jiajie Mai Xuemiao Xu Yue Li Chengze Li Xueting Liu and Shengfeng He. 2024. Appearance-preserved portrait-to-anime translation via proxy-guided domain adaptation. IEEE Transactions on Visualization and Computer Graphics 30 7 (2024) 3104\u20133120. 10.1109\/TVCG.2022.3228707","DOI":"10.1109\/TVCG.2022.3228707"},{"key":"e_1_3_4_57_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00748"},{"key":"e_1_3_4_58_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00749"},{"key":"e_1_3_4_59_1","doi-asserted-by":"publisher","unstructured":"Zipeng Ye Mengfei Xia Yanan Sun Ran Yi Minjing Yu Juyong Zhang Yu-Kun Lai and Yong-Jin Liu. 2023. 3D-CariGAN: An end-to-end solution to 3D caricature generation from normal face photos. IEEE Transactions on Visualization and Computer Graphics 29 4 (2023) 2203\u20132210. 10.1109\/TVCG.2021.3126659","DOI":"10.1109\/TVCG.2021.3126659"},{"key":"e_1_3_4_60_1","doi-asserted-by":"publisher","DOI":"10.1145\/3528233.3530720"},{"key":"e_1_3_4_61_1","doi-asserted-by":"publisher","unstructured":"Jie Zhang Kangneng Zhou Yan Luximon Tong-Yee Lee and Ping Li. 2024. MeshWGAN: Mesh-to-mesh Wasserstein GAN with Multi-Task Gradient Penalty for 3D facial geometric age transformation. IEEE Transactions on Visualization and Computer Graphics 30 8 (2024) 4927\u20134940. 10.1109\/TVCG.2023.3284500","DOI":"10.1109\/TVCG.2023.3284500"},{"key":"e_1_3_4_62_1","doi-asserted-by":"crossref","unstructured":"Lvmin Zhang Anyi Rao and Maneesh Agrawala. 2023. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE\/CVF International Conference on Computer Vision. 3836\u20133847.","DOI":"10.1109\/ICCV51070.2023.00355"},{"key":"e_1_3_4_63_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.01224"},{"key":"e_1_3_4_64_1","doi-asserted-by":"publisher","unstructured":"Wen-Yang Zhou Lu Yuan Shu-Yu Chen Lin Gao and Shi-Min Hu. 2024. LC-NeRF: Local controllable face generation in neural radiance field. IEEE Transactions on Visualization and Computer Graphics 30 8 (2024) 5437\u20135448. 10.1109\/TVCG.2023.3293653","DOI":"10.1109\/TVCG.2023.3293653"},{"key":"e_1_3_4_65_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00480"}],"container-title":["ACM Transactions on Graphics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3676165","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3676165","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:05:34Z","timestamp":1750291534000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3676165"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,9]]},"references-count":64,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,10,31]]}},"alternative-id":["10.1145\/3676165"],"URL":"https:\/\/doi.org\/10.1145\/3676165","relation":{},"ISSN":["0730-0301","1557-7368"],"issn-type":[{"value":"0730-0301","type":"print"},{"value":"1557-7368","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,9]]},"assertion":[{"value":"2023-09-21","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-24","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}