{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T19:00:21Z","timestamp":1771614021023,"version":"3.50.1"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,12,26]],"date-time":"2024-12-26T00:00:00Z","timestamp":1735171200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Guangxi Science and Technology Project","award":["AB22035052 and AD20159034"],"award-info":[{"award-number":["AB22035052 and AD20159034"]}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62172120, 62002082 and 6202780103"],"award-info":[{"award-number":["62172120, 62002082 and 6202780103"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Guangxi Key Laboratory of Image and Graphic Intelligent Processing Project","award":["GIIP2209"],"award-info":[{"award-number":["GIIP2209"]}]},{"name":"Innovation Project of GUET Graduate Education","award":["2023YCXB09"],"award-info":[{"award-number":["2023YCXB09"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2025,2,28]]},"abstract":"<jats:p>With the development of Metaverse technology, the avatar in Metaverse has faced serious security and privacy concerns. Analyzing facial features to distinguish between genuine and manipulated facial videos holds significant research importance for ensuring the authenticity of characters in the virtual world and for mitigating discrimination as well as preventing malicious use of facial data. To address this issue, the Facial Feature Points and Class-head-Transformer (FFP-ChT) deepfake video detection model is designed based on the clues of different FFPs distribution in real and fake videos and different displacement distances of real and fake FFPs between frames. The face video input is first detected by the BlazeFace model, and the face detection results are fed into the FaceMesh model to extract 468 FFPs. Then, the Lucas\u2013Kanade (LK) optical flow method is used to track the points of the face, the face calibration algorithm is introduced to re-calibrate the FFPs, and the jitter displacement is calculated by tracking the FFPs between frames. Finally, the Ch is designed in the transformer, and the FFPs and FFP displacement are jointly classified through the ChT model. In this way, the designed ChT classifier is able to accurately and effectively identify deepfake videos. Experiments on open datasets clearly demonstrate the effectiveness and generalization capabilities of our approach.<\/jats:p>","DOI":"10.1145\/3672566","type":"journal-article","created":{"date-parts":[[2024,6,12]],"date-time":"2024-06-12T14:42:41Z","timestamp":1718203361000},"page":"1-22","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":5,"title":["Deepfake Video Detection Using Facial Feature Points and Ch-Transformer"],"prefix":"10.1145","volume":"21","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5669-7105","authenticated-orcid":false,"given":"Rui","family":"Yang","sequence":"first","affiliation":[{"name":"The Guangxi Key Laboratory of Image and Graphic Intelligent Processing, Guilin University of Electronic Technology, Guilin, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9488-8236","authenticated-orcid":false,"given":"Rushi","family":"Lan","sequence":"additional","affiliation":[{"name":"The Guangxi Key Laboratory of Image and Graphic Intelligent Processing, Guilin University of Electronic Technology, Guilin, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5751-1843","authenticated-orcid":false,"given":"Zhenrong","family":"Deng","sequence":"additional","affiliation":[{"name":"The Guangxi Key Laboratory of Image and Graphic Intelligent Processing, Guilin University of Electronic Technology, Guilin, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0751-5045","authenticated-orcid":false,"given":"Xiaonan","family":"Luo","sequence":"additional","affiliation":[{"name":"The Guangxi Key Laboratory of Image and Graphic Intelligent Processing, Guilin University of Electronic Technology, Guilin, China"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4227-2499","authenticated-orcid":false,"given":"Xiyan","family":"Sun","sequence":"additional","affiliation":[{"name":"The Nanning Research Institute, Guilin University of Electronic Technology, Nanning, China"}]}],"member":"320","published-online":{"date-parts":[[2024,12,26]]},"reference":[{"key":"e_1_3_1_2_2","first-page":"1","volume-title":"Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS \u201918)","author":"Afchar Darius","year":"2018","unstructured":"Darius Afchar, Vincent Nozick, Junichi Yamagishi, and Isao Echizen. 2018. Mesonet: A compact facial video forgery detection network. In Proceedings of the IEEE International Workshop on Information Forensics and Security (WIFS \u201918). IEEE, 1\u20137."},{"key":"e_1_3_1_3_2","unstructured":"Valentin Bazarevsky Yury Kartynnik Andrey Vakunov Karthik Raveendran and Matthias Grundmann. 2019. BlazeFace: Sub-millisecond neural face detection on mobile gpus. arXiv:1907.05047. Retrieved from https:\/\/arxiv.org\/pdf\/1907.05047"},{"key":"e_1_3_1_4_2","doi-asserted-by":"publisher","DOI":"10.5555\/1028916.1035417"},{"key":"e_1_3_1_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.dsp.2023.104233"},{"issue":"6","key":"e_1_3_1_6_2","doi-asserted-by":"crossref","first-page":"3527","DOI":"10.1109\/TCSVT.2021.3116679","article-title":"A robust GAN-generated face detection method based on dual-color spaces and an improved Xception","volume":"32","author":"Chen Beijing","year":"2021","unstructured":"Beijing Chen, Xin Liu, Yuhui Zheng, Guoying Zhao, and Yun-Qing Shi. 2021a. A robust GAN-generated face detection method based on dual-color spaces and an improved Xception. IEEE Transactions on Circuits and Systems for Video Technology 32, 6 (2021), 3527\u20133538.","journal-title":"IEEE Transactions on Circuits and Systems for Video Technology"},{"key":"e_1_3_1_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01815"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i2.16193"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.195"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCB48548.2020.9304909"},{"key":"e_1_3_1_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3292500.3330866"},{"key":"e_1_3_1_12_2","first-page":"744","volume-title":"Proceedings of the AAAI Conference on Artificial Intelligence","volume":"36","author":"Gu Zhihao","year":"2022","unstructured":"Zhihao Gu, Yang Chen, Taiping Yao, Shouhong Ding, Jilin Li, and Lizhuang Ma. 2022a. Delving into the local: Dynamic inconsistency learning for deepfake video detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 36. 744\u2013752."},{"key":"e_1_3_1_13_2","first-page":"596","volume-title":"Proceedings of the Computer Vision\u2013ECCV 2022: 17th European Conference","author":"Gu Zhihao","year":"2022","unstructured":"Zhihao Gu, Taiping Yao, Yang Chen, Shouhong Ding, and Lizhuang Ma. 2022b. Hierarchical contrastive inconsistency learning for deepfake video detection. In Proceedings of the Computer Vision\u2013ECCV 2022: 17th European Conference, Part XII. Springer, 596\u2013613."},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.3390\/app10010370"},{"key":"e_1_3_1_15_2","unstructured":"Hyeonseong Jeon Youngoh Bang Junyaup Kim and Simon S. Woo. 2020b. T-gd:Transferable gan-generated images detection framework. arXiv:2008.04115. Retrieved from https:\/\/arxiv.org\/abs\/2008.04115"},{"key":"e_1_3_1_16_2","doi-asserted-by":"crossref","first-page":"416","DOI":"10.1007\/978-3-030-58201-2_28","volume-title":"Proceedings of the ICT Systems Security and Privacy Protection: 35th IFIP TC 11 International Conference (SEC \u201920)","author":"Jeon Hyeonseong","year":"2020","unstructured":"Hyeonseong Jeon, Youngoh Bang, and Simon S. Woo. 2020a. Fdftnet: Facing off fake images using fake detection fine-tuning network. In Proceedings of the ICT Systems Security and Privacy Protection: 35th IFIP TC 11 International Conference (SEC \u201920). Springer, 416\u2013430."},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"e_1_3_1_18_2","unstructured":"Yury Kartynnik Artsiom Ablavatski Ivan Grishchenko and Matthias Grundmann. 2019. Real-time facial surface geometry from monocular video on mobile GPUs. arXiv:1907.06724. Retrieved from https:\/\/arxiv.org\/abs\/1907.06724"},{"key":"e_1_3_1_19_2","first-page":"656","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Workshops","author":"Khalid Hasam","year":"2020","unstructured":"Hasam Khalid and Simon S. Woo. 2020. Oc-fakedect: Classifying deepfakes using one-class variational autoencoder. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Workshops. 656\u2013657."},{"key":"e_1_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.bushor.2019.11.006"},{"key":"e_1_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.397"},{"key":"e_1_3_1_22_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sigpro.2020.107616"},{"key":"e_1_3_1_23_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00505"},{"key":"e_1_3_1_24_2","unstructured":"Yuezun Li and Siwei Lyu. 2018. Exposing deepfake videos by detecting face warping artifacts. arXiv:1811.00656. Retrieved from https:\/\/arxiv.org\/abs\/1811.00656"},{"key":"e_1_3_1_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00327"},{"key":"e_1_3_1_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00083"},{"key":"e_1_3_1_27_2","unstructured":"Sitong Liu Zhichao Lian Siqi Gu and Liang Xiao. 2022. Block shuffling learning for deepfake detection. arXiv:2202.02819. Retrieved from https:\/\/arxiv.org\/abs\/2202.02819"},{"key":"e_1_3_1_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00808"},{"key":"e_1_3_1_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01605"},{"key":"e_1_3_1_30_2","doi-asserted-by":"crossref","unstructured":"Scott McCloskey and Michael Albright. 2018. Detecting GAN-generated imagery using color cues. arXiv:1812.08247. Retrieved from https:\/\/arxiv.org\/abs\/1812.08247","DOI":"10.1109\/ICIP.2019.8803661"},{"key":"e_1_3_1_31_2","unstructured":"Lakshmanan Nataraj Tajuddin Manhar Mohammed Shivkumar Chandrasekaran Arjuna Flenner Jawadul H. Bappy Amit K. Roy-Chowdhury and B. S. Manjunath. 2019. Detecting GAN generated fake images using co-occurrence matrices. arXiv:1903.06836. Retrieved from https:\/\/arxiv.org\/abs\/1903.06836"},{"key":"e_1_3_1_32_2","first-page":"2307","volume-title":"Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP \u201919)","author":"Nguyen Huy H.","year":"2019","unstructured":"Huy H. Nguyen, Junichi Yamagishi, and Isao Echizen. 2019. Capsule-forensics: Using capsule networks to detect forged images and videos. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP \u201919). IEEE, 2307\u20132311."},{"key":"e_1_3_1_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00728"},{"key":"e_1_3_1_34_2","doi-asserted-by":"crossref","first-page":"235","DOI":"10.1201\/9781003373384-12","volume-title":"Big Data Analytics and Intelligent Systems for Cyber Threat Intelligence","author":"Nowroozi Ehsan","year":"2023","unstructured":"Ehsan Nowroozi and Yassine Mekdad. 2023. Detecting high-quality GAN-generated face images using neural networks. In Big Data Analytics and Intelligent Systems for Cyber Threat Intelligence. River Publishers, 235\u2013252."},{"key":"e_1_3_1_35_2","unstructured":"Ivan Perov Daiheng Gao Nikolay Chervoniy Kunlin Liu Sugasa Marangonda Chris Um\u00e9 Mr Dpfks Carl Shift Facenheim Luis R. P. Jian Jiang Sheng Zhang Pingyu Wu Bo Zhou and Weiming Zhang. 2020. DeepFaceLab: Integrated flexible and extensible face-swapping framework. arXiv:2005.05535. Retrieved from https:\/\/arxiv.org\/abs\/2005.05535"},{"key":"e_1_3_1_36_2","first-page":"1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision","author":"Rossler Andreas","year":"2019","unstructured":"Andreas Rossler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nie\u00dfner. 2019. Faceforensics \\({+}{+}\\) : Learning to detect manipulated facial images. In Proceedings of the IEEE\/CVF Conference on Computer Vision. 1\u201311."},{"key":"e_1_3_1_37_2","first-page":"3856","article-title":"Dynamic routing between capsules","volume":"30","author":"Sabour Sara","year":"2017","unstructured":"Sara Sabour, Nicholas Frosst, and Geoffrey E. Hinton. 2017. Dynamic routing between capsules. Advances in Neural Information Processing Systems 30, 3856\u20133866.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/TBIOM.2019.2942395"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01816"},{"key":"e_1_3_1_40_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v36i2.20130"},{"key":"e_1_3_1_41_2","first-page":"633","volume-title":"Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision","author":"Sun YuYang","year":"2023","unstructured":"YuYang Sun, ZhiYong Zhang, Isao Echizen, Huy H. Nguyen, ChangZhen Qiu, and Lu Sun. 2023. Face forgery detection based on facial region displacement trajectory series. In Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision. 633\u2013642."},{"key":"e_1_3_1_42_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00361"},{"issue":"5","key":"e_1_3_1_43_2","doi-asserted-by":"crossref","first-page":"6177","DOI":"10.1029\/JC094iC05p06177","article-title":"The role of the Hessian matrix in fitting models to measurements","volume":"94","author":"Thacker William Carlisle","year":"1989","unstructured":"William Carlisle Thacker. 1989. The role of the Hessian matrix in fitting models to measurements. Journal of Geophysical Research: Oceans 94, C5 (1989), 6177\u20136196.","journal-title":"Journal of Geophysical Research: Oceans"},{"key":"e_1_3_1_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3306346.3323035"},{"key":"e_1_3_1_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/2929464.2929475"},{"key":"e_1_3_1_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/70.34766"},{"key":"e_1_3_1_47_2","first-page":"1288","volume-title":"Proceedings of the International Joint Conference on Artificial Intelligence","author":"Zhang Daichi","year":"2021","unstructured":"Daichi Zhang, Chenyu Li, Fanzhao Lin, Dan Zeng, and Shiming Ge. 2021. Detecting deepfake videos with temporal dropout 3DCNN. In Proceedings of the International Joint Conference on Artificial Intelligence. 1288\u20131294."},{"key":"e_1_3_1_48_2","doi-asserted-by":"crossref","unstructured":"Daichi Zhang Fanzhao Lin Yingying Hua Pengju Wang Dan Zeng and Shiming Ge. 2022. Deepfake video detection with spatiotemporal dropout transformer. arXiv:2207.06612. Retrieved from https:\/\/arxiv.org\/abs\/2207.06612","DOI":"10.1145\/3503161.3547913"},{"key":"e_1_3_1_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.01477"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3672566","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3672566","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:06:14Z","timestamp":1750291574000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3672566"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12,26]]},"references-count":48,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,2,28]]}},"alternative-id":["10.1145\/3672566"],"URL":"https:\/\/doi.org\/10.1145\/3672566","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,12,26]]},"assertion":[{"value":"2023-12-12","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-25","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-26","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}