{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T02:32:32Z","timestamp":1760236352196,"version":"build-2065373602"},"reference-count":31,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2021,11,22]],"date-time":"2021-11-22T00:00:00Z","timestamp":1637539200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["62002334","U20B2047"],"award-info":[{"award-number":["62002334","U20B2047"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Anhui Science Foundation of China","award":["2008085QF296"],"award-info":[{"award-number":["2008085QF296"]}]},{"name":"Exploration Fund Project of University of Science and Technology of China","award":["YD3480002001"],"award-info":[{"award-number":["YD3480002001"]}]},{"name":"Fundamental Research Funds for Central Universities","award":["WK2100000011"],"award-info":[{"award-number":["WK2100000011"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Future Internet"],"abstract":"<jats:p>Deepfake aims to swap a face of an image with someone else\u2019s likeness in a reasonable manner. Existing methods usually perform deepfake frame by frame, thus ignoring video consistency and producing incoherent results. To address such a problem, we propose a novel framework Neural Identity Carrier (NICe), which learns identity transformation from an arbitrary face-swapping proxy via a U-Net. By modeling the incoherence between frames as noise, NICe naturally suppresses its disturbance and preserves primary identity information. Concretely, NICe inputs the original frame and learns transformation supervised by swapped pseudo labels. As the temporal incoherence has an uncertain or stochastic pattern, NICe can filter out such outliers and well maintain the target content by uncertainty prediction. With the predicted temporally stable appearance, NICe enhances its details by constraining 3D geometry consistency, making NICe learn fine-grained facial structure across the poses. In this way, NICe guarantees the temporal stableness of deepfake approaches and predicts detailed results against over-smoothness. Extensive experiments on benchmarks demonstrate that NICe significantly improves the quality of existing deepfake methods on video-level. Besides, data generated by our methods can benefit video-level deepfake detection methods.<\/jats:p>","DOI":"10.3390\/fi13110298","type":"journal-article","created":{"date-parts":[[2021,11,23]],"date-time":"2021-11-23T02:55:17Z","timestamp":1637636117000},"page":"298","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Face Swapping Consistency Transfer with Neural Identity Carrier"],"prefix":"10.3390","volume":"13","author":[{"given":"Kunlin","family":"Liu","sequence":"first","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7912-3235","authenticated-orcid":false,"given":"Ping","family":"Wang","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wenbo","family":"Zhou","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhenyu","family":"Zhang","sequence":"additional","affiliation":[{"name":"Tencent Youtu, Shanghai 200233, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yanhao","family":"Ge","sequence":"additional","affiliation":[{"name":"Tencent Youtu, Shanghai 200233, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9294-9624","authenticated-orcid":false,"given":"Honggu","family":"Liu","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Weiming","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nenghai","family":"Yu","sequence":"additional","affiliation":[{"name":"School of Information Science and Technology, University of Science and Technology of China, Hefei 230026, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,11,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. (2009, January 12\u201313). Creating a photoreal digital actor: The digital emily project. Proceedings of the 2009 Conference for Visual Media Production, London, UK.","DOI":"10.1109\/CVMP.2009.29"},{"key":"ref_2","first-page":"669","article-title":"Exchanging faces in images","volume":"23","author":"Blanz","year":"2004","journal-title":"CGF"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Li, L., Bao, J., Yang, H., Chen, D., and Wen, F. (2020, January 14\u201319). Advancing High Fidelity Identity Swapping for Forgery Detection. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00512"},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Nirkin, Y., Keller, Y., and Hassner, T. (2019, January 27\u201328). FSGAN: Subject agnostic face swapping and reenactment. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00728"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Nirkin, Y., Masi, I., Tuan, A.T., Hassner, T., and Medioni, G. (2018, January 15\u201319). On face segmentation, face swapping, and face perception. Proceedings of the 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), Xi\u2019an, China.","DOI":"10.1109\/FG.2018.00024"},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Chen, R., Chen, X., Ni, B., and Ge, Y. (2020, January 12). SimSwap: An Efficient Framework For High Fidelity Face Swapping. Proceedings of the MM \u201920: The 28th ACM International Conference on Multimedia, New York, NY, USA.","DOI":"10.1145\/3394171.3413630"},{"key":"ref_7","unstructured":"Zhou, Z.H. (2021). HifiFace: 3D Shape and Semantic Prior Guided High Fidelity Face Swapping. Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, Montr\u00e9al, QC, Canada, 21 August 2021, International Joint Conferences on Artificial Intelligence Organization. Available online: https:\/\/arxiv.org\/pdf\/2106.09965.pdf."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Li Fan, W.L., and Cui, X. (2021). Deepfake-Image Anti-Forensics with Adversarial Examples Attacks. Future Internet, 13.","DOI":"10.3390\/fi13110288"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Hewage, C., and Ekmekcioglu, E. (2020). Multimedia Quality of Experience (QoE): Current Status and Future Direction. Future Internet, 12.","DOI":"10.3390\/fi12070121"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Khalil, S.S., Youssef, S.M., and Saleh, S.N. (2021). iCaps-Dfake: An Integrated Capsule-Based Model for Deepfake Image and Video Detection. Future Internet, 13.","DOI":"10.3390\/fi13040093"},{"key":"ref_11","unstructured":"Ulyanov, D., Vedaldi, A., and Lempitsky, V. (2018, January 18\u201323). Deep image prior. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA."},{"key":"ref_12","unstructured":"Lei, C., Xing, Y., and Chen, Q. (2021, November 18). Blind Video Temporal Consistency via Deep Video Prior. Advances in Neural Information Processing Systems. Available online: https:\/\/proceedings.neurips.cc\/\/paper\/2020\/hash\/0c0a7566915f4f24853fc4192689aa7e-Abstract.html."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"39","DOI":"10.1145\/1360612.1360638","article-title":"Face Swapping: Automatically replacing faces in photographs","volume":"27","author":"Bitouk","year":"2008","journal-title":"ACM SIGGRAPH"},{"key":"ref_14","unstructured":"(2019, February 06). DeepFakes. FaceSwap. Available online: https:\/\/github.com\/deepfakes\/faceswap."},{"key":"ref_15","unstructured":"Perov, I., Gao, D., Chervoniy, N., Liu, K., Marangonda, S., Um\u2019e, C., Dpfks, M., Luis, R., Jiang, J., and Zhang, S. (2020). DeepFaceLab: A simple, flexible and extensible face swapping framework. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.strusafe.2008.06.020","article-title":"Aleatory or epistemic? Does it matter?","volume":"31","author":"Kiureghian","year":"2009","journal-title":"Struct. Saf."},{"key":"ref_17","unstructured":"Kendall, A., and Gal, Y. (2021, November 18). What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?. NIPS, Available online: https:\/\/arxiv.org\/abs\/1703.04977."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Nix, D., and Weigend, A. (July, January 28). Estimating the mean and variance of the target probability distribution. Proceedings of the 1994 IEEE International Conference on Neural Networks (ICNN\u201994), Orlando, FL, USA. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/374138.","DOI":"10.1109\/ICNN.1994.374138"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Le, Q.V., Smola, A.J., and Canu, S. (2005, January 7\u201311). Heteroscedastic Gaussian Process Regression. Proceedings of the 22nd International Conference on Machine Learning, Bonn, Germany. Available online: https:\/\/dl.acm.org\/doi\/abs\/10.1145\/1102351.1102413.","DOI":"10.1145\/1102351.1102413"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Nagano, K., Seo, J., Xing, J., Wei, L., Li, Z., Saito, S., Agarwal, A., Fursund, J., and Li, H. (2018). PaGAN: Real-Time Avatars Using Dynamic Textures. ACM Trans. Graph., 37.","DOI":"10.1145\/3272127.3275075"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Thies, J., Zollh\u00f6fer, M., and Nie\u00dfner, M. (2019). Deferred Neural Rendering: Image Synthesis Using Neural Textures. ACM Trans. Graph., 38.","DOI":"10.1145\/3306346.3323035"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Chaudhuri, B., Vesdapunt, N., Shapiro, L., and Wang, B. (2020). Personalized face modeling for improved face reconstruction and motion retargeting. arXiv.","DOI":"10.1007\/978-3-030-58558-7_9"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Feng, Y., Feng, H., Black, M.J., and Bolkart, T. (2021, November 18). Learning an Animatable Detailed 3D Face Model from In-The-Wild Images, Available online: http:\/\/xxx.lanl.gov\/abs\/2012.04012.","DOI":"10.1145\/3476576.3476646"},{"key":"ref_24","first-page":"194","article-title":"Learning a model of facial shape and expression from 4D scans","volume":"36","author":"Li","year":"2017","journal-title":"ACM Trans. Graph. (Proc. SIGGRAPH Asia)"},{"key":"ref_25","unstructured":"Ravi, N., Reizenstein, J., Novotny, D., Gordon, T., Lo, W.Y., Johnson, J., and Gkioxari, G. (2020). Accelerating 3D Deep Learning with PyTorch3D. arXiv."},{"key":"ref_26","unstructured":"Wang, Y., Tao, X., Qi, X., Shen, X., and Jia, J. (2018). Image Inpainting via Generative Multi-column Convolutional Neural Networks. Advances in Neural Information Processing Systems, Available online: https:\/\/arxiv.org\/abs\/1810.08771."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"R\u00f6ssler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Nie\u00dfner, M. (2021, November 18). FaceForensics++: Learning to Detect Manipulated Facial Images. International Conference on Computer Vision (ICCV). Available online: https:\/\/openaccess.thecvf.com\/content_ICCV_2019\/html\/Rossler_FaceForensics_Learning_to_Detect_Manipulated_Facial_Images_ICCV_2019_paper.html.","DOI":"10.1109\/ICCV.2019.00009"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Chen, D., Liao, J., Yuan, L., Yu, N., and Hua, G. (2017, January 22\u201329). Coherent Online Video Style Transfer. Proceedings of the IEEE International Conference on Computer Vision (ICCV), Venice, Italy. Available online: https:\/\/openaccess.thecvf.com\/content_iccv_2017\/html\/Chen_Coherent_Online_Video_ICCV_2017_paper.html.","DOI":"10.1109\/ICCV.2017.126"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Baltrusaitis, T., Zadeh, A., Lim, Y.C., and Morency, L. (2018, January 15\u201319). OpenFace 2.0: Facial Behavior Analysis Toolkit. Proceedings of the 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018), Xi\u2019an, China. Available online: https:\/\/ieeexplore.ieee.org\/abstract\/document\/8373812.","DOI":"10.1109\/FG.2018.00019"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Carreira, J., and Zisserman, A. (2017, January 21\u201326). Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA. Available online: https:\/\/openaccess.thecvf.com\/content_cvpr_2017\/html\/Carreira_Quo_Vadis_Action_CVPR_2017_paper.html.","DOI":"10.1109\/CVPR.2017.502"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. (2020, January 14\u201319). Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics. Proceedings of the IEEE Conference on Computer Vision and Patten Recognition (CVPR), Seattle, WA, USA. Available online: https:\/\/openaccess.thecvf.com\/content_CVPR_2020\/html\/Li_Celeb-DF_A_LargeScale_Challenging_Dataset_for_DeepFake_Forensics_CVPR_2020_paper.html.","DOI":"10.1109\/CVPR42600.2020.00327"}],"container-title":["Future Internet"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1999-5903\/13\/11\/298\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:34:08Z","timestamp":1760168048000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1999-5903\/13\/11\/298"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,11,22]]},"references-count":31,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2021,11]]}},"alternative-id":["fi13110298"],"URL":"https:\/\/doi.org\/10.3390\/fi13110298","relation":{},"ISSN":["1999-5903"],"issn-type":[{"type":"electronic","value":"1999-5903"}],"subject":[],"published":{"date-parts":[[2021,11,22]]}}}