{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T17:02:18Z","timestamp":1783098138028,"version":"3.54.6"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"31","license":[{"start":{"date-parts":[[2024,8,8]],"date-time":"2024-08-08T00:00:00Z","timestamp":1723075200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2024,8,8]],"date-time":"2024-08-08T00:00:00Z","timestamp":1723075200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100003789","name":"Helwan University","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100003789","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Neural Comput &amp; Applic"],"published-print":{"date-parts":[[2024,11]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>Deepfake technology has rapidly advanced in recent years, creating highly realistic fake videos that can be difficult to distinguish from real ones. The rise of social media platforms and online forums has exacerbated the challenges of detecting misinformation and malicious content. This study leverages many papers on artificial intelligence techniques to address deepfake detection. This research proposes a deep learning (DL)-based method for detecting deepfakes. The system comprises three components: preprocessing, detection, and prediction. Preprocessing includes frame extraction, face detection, alignment, and feature cropping. Convolutional neural networks (CNNs) are employed in the eye and nose feature detection phase. A CNN combined with a vision transformer is also used for face detection. The prediction component employs a majority voting approach, merging results from the three models applied to different features, leading to three individual predictions. The model is trained on various face images using FaceForensics++ and DFDC datasets. Multiple performance metrics, including accuracy, precision, F1, and recall, are used to assess the proposed model\u2019s performance. The experimental results indicate the potential and strengths of the proposed CNN that achieved enhanced performance with an accuracy of 97%, while the CViT-based model achieved 85% using the FaceForences++ dataset and demonstrated significant improvements in deepfake detection compared to recent studies, affirming the potential of the suggested framework for detecting deepfakes on social media. This study contributes to a broader understanding of CNN-based DL methods for deepfake detection.<\/jats:p>","DOI":"10.1007\/s00521-024-10181-7","type":"journal-article","created":{"date-parts":[[2024,8,8]],"date-time":"2024-08-08T11:03:25Z","timestamp":1723115005000},"page":"19759-19775","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":100,"title":["Deepfake detection using convolutional vision transformers and convolutional neural networks"],"prefix":"10.1007","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-6919-3962","authenticated-orcid":false,"given":"Ahmed Hatem","family":"Soudy","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Omnia","family":"Sayed","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hala","family":"Tag-Elser","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Rewaa","family":"Ragab","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sohaila","family":"Mohsen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tarek","family":"Mostafa","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Amr A.","family":"Abohany","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Salwa O.","family":"Slim","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2024,8,8]]},"reference":[{"key":"10181_CR1","unstructured":"Al-Hussein M, Venkataraman S, Jawahar C (2020) Deepfake detection for video: an open source challenge. arXiv preprint arXiv:2006.06058"},{"issue":"7","key":"10181_CR2","doi-asserted-by":"publisher","first-page":"2249","DOI":"10.1007\/s00521-020-05070-8","volume":"33","author":"N Altwaijry","year":"2021","unstructured":"Altwaijry N, Al-Turaiki I (2021) Arabic handwriting recognition system using convolutional neural network. Neural Comput Appl 33(7):2249\u20132261","journal-title":"Neural Comput Appl"},{"issue":"3","key":"10181_CR3","doi-asserted-by":"publisher","first-page":"144","DOI":"10.3390\/bdcc7030144","volume":"7","author":"MS Arif","year":"2023","unstructured":"Arif MS, Mukheimer A, Asif D (2023) Enhancing the early detection of chronic kidney disease: a robust machine learning model. Big Data Cogn Comput 7(3):144","journal-title":"Big Data Cogn Comput"},{"issue":"6","key":"10181_CR4","doi-asserted-by":"publisher","first-page":"308","DOI":"10.3390\/a16060308","volume":"16","author":"D Asif","year":"2023","unstructured":"Asif D, Bibi M, Arif MS, Mukheimer A (2023) Enhancing heart disease prediction through ensemble learning techniques with hyperparameter optimization. Algorithms 16(6):308","journal-title":"Algorithms"},{"key":"10181_CR5","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.ceh.2020.11.002","volume":"4","author":"M Desai","year":"2021","unstructured":"Desai M, Shah M (2021) An anatomization on breast cancer detection and diagnosis employing multi-layer perceptron neural network (mlp) and convolutional neural network (cnn). Clin eHealth 4:1\u201311","journal-title":"Clin eHealth"},{"key":"10181_CR6","unstructured":"Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S et\u00a0al (2020) An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929"},{"key":"10181_CR7","doi-asserted-by":"publisher","first-page":"3155","DOI":"10.1007\/s00521-020-05209-7","volume":"33","author":"M Dua","year":"2021","unstructured":"Dua M, Shakshi Singla R, Raj S, Jangra A (2021) Deep cnn models-based ensemble approach to driver drowsiness detection. Neural Comput Appl 33:3155\u20133168","journal-title":"Neural Comput Appl"},{"key":"10181_CR8","unstructured":"Hasan MJ et\u00a0al (2020) Understanding the influence of epochs and learning rate on deep learning-based sentiment analysis. In: Proceedings of the international conference on artificial intelligence and applications, pp 553\u2013561"},{"issue":"16","key":"10181_CR9","doi-asserted-by":"publisher","first-page":"5413","DOI":"10.3390\/s21165413","volume":"21","author":"A Ismail","year":"2021","unstructured":"Ismail A, Elpeltagy M, Zaki SM, Eldahshan K (2021) A new deep learning-based methodology for video deepfake detection using xgboost. Sensors 21(16):5413","journal-title":"Sensors"},{"issue":"6","key":"10181_CR10","doi-asserted-by":"publisher","DOI":"10.1002\/cpe.6767","volume":"34","author":"B Jena","year":"2022","unstructured":"Jena B, Nayak GK, Saxena S (2022) Convolutional neural network and its pretrained models for image classification and object detection: a survey. Concurr Comput Pract Exp 34(6):e6767","journal-title":"Concurr Comput Pract Exp"},{"key":"10181_CR11","doi-asserted-by":"crossref","unstructured":"Jing TW, Murugesan RK (2021) Protecting data privacy and prevent fake news and deepfakes in social media via blockchain technology. In: Advances in cyber security: second international conference, ACeS 2020, Penang, Malaysia, December 8\u20139, 2020, Revised Selected Papers 2. Springer, Berlin, pp 674\u2013684","DOI":"10.1007\/978-981-33-6835-4_44"},{"issue":"2","key":"10181_CR12","doi-asserted-by":"publisher","first-page":"1311","DOI":"10.30534\/ijatcse\/2020\/62922020","volume":"9","author":"A Karandikar","year":"2020","unstructured":"Karandikar A, Deshpande V, Singh S, Nagbhidkar S, Agrawal S (2020) Deepfake video detection using convolutional neural network. Int J Adv Trends Comput Sci Eng 9(2):1311\u20131315","journal-title":"Int J Adv Trends Comput Sci Eng"},{"key":"10181_CR13","doi-asserted-by":"publisher","DOI":"10.47164\/ijngc.v14i1.1046","author":"A Karandikar","year":"2023","unstructured":"Karandikar A, Thakare Y, Sah O, Sah R, Nafde S, Kumar S (2023) Detection of deepfake video using residual neural network and long short-term memory. Int J Next-Gener Comput. https:\/\/doi.org\/10.47164\/ijngc.v14i1.1046","journal-title":"Int J Next-Gener Comput"},{"key":"10181_CR14","unstructured":"Kfir O, Kfir A, Friedman A (2019) Generative adversarial attacks in social media. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & sata mining, pp 2558\u20132566"},{"key":"10181_CR15","doi-asserted-by":"crossref","unstructured":"Kumar A, Bhavsar A, Verma R (2020) Detecting deepfakes with metric learning. In: 2020 8th international workshop on biometrics and forensics (IWBF). IEEE, pp 1\u20136","DOI":"10.1109\/IWBF49977.2020.9107951"},{"key":"10181_CR16","doi-asserted-by":"crossref","unstructured":"Li Y, Chang MC, Lyu S (2018) In ictu oculi: exposing ai created fake videos by detecting eye blinking. In: 2018 IEEE International workshop on information forensics and security (WIFS). IEEE, pp 1\u20137","DOI":"10.1109\/WIFS.2018.8630787"},{"key":"10181_CR17","doi-asserted-by":"crossref","unstructured":"Li Y, Yang X, Sun P, Qi H, Lyu S (2020) Celeb-df: a large-scale challenging dataset for deepfake forensics. In: Proceedings of the IEEE\/CVF conference on computer vision and pattern recognition, pp 3207\u20133216","DOI":"10.1109\/CVPR42600.2020.00327"},{"key":"10181_CR18","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s42979-021-00495-x","volume":"2","author":"A Mitra","year":"2021","unstructured":"Mitra A, Mohanty SP, Corcoran P, Kougianos E (2021) A machine learning based approach for deepfake detection in social media through key video frame extraction. SN Comput Sci 2:1\u201318","journal-title":"SN Comput Sci"},{"issue":"8","key":"10181_CR19","doi-asserted-by":"publisher","first-page":"3809","DOI":"10.3390\/s23083809","volume":"23","author":"JN Mogan","year":"2023","unstructured":"Mogan JN, Lee CP, Lim KM, Ali M, Alqahtani A (2023) Gait-cnn-vit: multi-model gait recognition with convolutional neural networks and vision transformer. Sensors 23(8):3809","journal-title":"Sensors"},{"issue":"12","key":"10181_CR20","doi-asserted-by":"publisher","first-page":"3396","DOI":"10.3390\/en14123396","volume":"14","author":"Y Nawaz","year":"2021","unstructured":"Nawaz Y, Arif MS, Shatanawi W, Nazeer A (2021) An explicit fourth-order compact numerical scheme for heat transfer of boundary layer flow. Energies 14(12):3396","journal-title":"Energies"},{"issue":"7","key":"10181_CR21","doi-asserted-by":"publisher","first-page":"920","DOI":"10.1002\/fld.5078","volume":"94","author":"Y Nawaz","year":"2022","unstructured":"Nawaz Y, Arif MS, Abodayeh K (2022) An explicit-implicit numerical scheme for time fractional boundary layer flows. Int J Numer Methods Fluids 94(7):920\u2013940","journal-title":"Int J Numer Methods Fluids"},{"issue":"10","key":"10181_CR22","doi-asserted-by":"publisher","DOI":"10.1115\/1.4054800","volume":"17","author":"Y Nawaz","year":"2022","unstructured":"Nawaz Y, Arif MS, Abodayeh K (2022) A third-order two-stage numerical scheme for fractional stokes problems: a comparative computational study. J Comput Nonlinear Dyn 17(10):101004","journal-title":"J Comput Nonlinear Dyn"},{"key":"10181_CR23","doi-asserted-by":"publisher","DOI":"10.1016\/j.cviu.2022.103525","volume":"223","author":"TT Nguyen","year":"2022","unstructured":"Nguyen TT, Nguyen QVH, Nguyen DT, Nguyen DT, Huynh-The T, Nahavandi S, Nguyen TT, Pham QV, Nguyen CM (2022) Deep learning for deepfakes creation and detection: a survey. Comput Vis Image Underst 223:103525","journal-title":"Comput Vis Image Underst"},{"issue":"1","key":"10181_CR24","doi-asserted-by":"publisher","first-page":"7422","DOI":"10.1038\/s41598-023-34629-3","volume":"13","author":"R Rafique","year":"2023","unstructured":"Rafique R, Gantassi R, Amin R, Frnda J, Mustapha A, Alshehri AH (2023) Deep fake detection and classification using error-level analysis and deep learning. Sci Rep 13(1):7422","journal-title":"Sci Rep"},{"issue":"1","key":"10181_CR25","first-page":"80","volume":"3","author":"E Sabir","year":"2019","unstructured":"Sabir E, Cheng J, Jaiswal A, AbdAlmageed W, Masi I, Natarajan P (2019) Recurrent convolutional strategies for face manipulation detection in videos. Interfaces (GUI) 3(1):80\u201387","journal-title":"Interfaces (GUI)"},{"key":"10181_CR26","doi-asserted-by":"publisher","unstructured":"Singh KA, Vats O, Shankar M (2022) Deepfake detection using deep learning methods: a systematic and comprehensive review. Wiley Interdiscip Rev Data Min Knowl Discov 14(3):e1520. https:\/\/doi.org\/10.1002\/widm.1520","DOI":"10.1002\/widm.1520"},{"issue":"1","key":"10181_CR27","doi-asserted-by":"publisher","first-page":"89","DOI":"10.3390\/jcp2010007","volume":"2","author":"M Taeb","year":"2022","unstructured":"Taeb M, Chi H (2022) Comparison of deepfake detection techniques through deep learning. J Cybersecur Priv 2(1):89\u2013106","journal-title":"J Cybersecur Priv"},{"key":"10181_CR28","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser \u0141, Polosukhin I (2017) Attention is all you need. In: Advances in Neural Information Processing Systems 30 (NIPS 2017)"},{"key":"10181_CR29","doi-asserted-by":"publisher","DOI":"10.1016\/j.compag.2021.106090","volume":"184","author":"Y Wang","year":"2021","unstructured":"Wang Y, Zhang Z, Feng L, Ma Y, Du Q (2021) A new attention-based cnn approach for crop mapping using time series sentinel-2 images. Comput Electron Agric 184:106090","journal-title":"Comput Electron Agric"},{"key":"10181_CR30","unstructured":"Wodajo D, Atnafu S (2021) Deepfake video detection using convolutional vision transformer. arXiv preprint arXiv:2102.11126"},{"issue":"14","key":"10181_CR31","doi-asserted-by":"publisher","first-page":"3491","DOI":"10.3390\/rs15143491","volume":"15","author":"J Xie","year":"2023","unstructured":"Xie J, Hua J, Chen S, Wu P, Gao P, Sun D, Lyu Z, Lyu S, Xue X, Lu J (2023) Hypersformer: a transformer-based end-to-end hyperspectral image classification method for crop classification. Remote Sens 15(14):3491","journal-title":"Remote Sens"},{"key":"10181_CR32","doi-asserted-by":"publisher","first-page":"611","DOI":"10.1007\/s13244-018-0639-9","volume":"9","author":"R Yamashita","year":"2018","unstructured":"Yamashita R, Nishio M, Do RKG, Togashi K (2018) Convolutional neural networks: an overview and application in radiology. Insights Imaging 9:611\u2013629","journal-title":"Insights Imaging"},{"key":"10181_CR33","doi-asserted-by":"publisher","first-page":"1696","DOI":"10.1109\/TIFS.2023.3249566","volume":"18","author":"Z Yang","year":"2023","unstructured":"Yang Z, Liang J, Xu Y, Zhang XY, He R (2023) Masked relation learning for deepfake detection. IEEE Trans Inf Forensics Secur 18:1696\u20131708","journal-title":"IEEE Trans Inf Forensics Secur"},{"issue":"6","key":"10181_CR34","doi-asserted-by":"publisher","first-page":"607","DOI":"10.1049\/bme2.12031","volume":"10","author":"P Yu","year":"2021","unstructured":"Yu P, Xia Z, Fei J, Lu Y (2021) A survey on deepfake video detection. IET Biometrics 10(6):607\u2013624","journal-title":"IET Biometrics"},{"issue":"5","key":"10181_CR35","doi-asserted-by":"publisher","first-page":"6259","DOI":"10.1007\/s11042-021-11733-y","volume":"81","author":"T Zhang","year":"2022","unstructured":"Zhang T (2022) Deepfake generation and detection, a survey. Multimed Tools Appl 81(5):6259\u20136276","journal-title":"Multimed Tools Appl"},{"issue":"8","key":"10181_CR36","first-page":"1295","volume":"23","author":"M Zhao","year":"2020","unstructured":"Zhao M, Qin W, Yu W, Li X, Gao B (2020) Adversarial examples: attacks on deepfake detection models. IEEE Trans Multimed 23(8):1295\u20131305","journal-title":"IEEE Trans Multimed"},{"issue":"3","key":"10181_CR37","first-page":"196","volume":"14","author":"XY Zhou","year":"2020","unstructured":"Zhou XY, Jiang WJ, Yu ZH (2020) Deepfakes detection using deep learning methods: a survey. Int J Imaging Robot 14(3):196\u2013211","journal-title":"Int J Imaging Robot"}],"container-title":["Neural Computing and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-10181-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s00521-024-10181-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s00521-024-10181-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,9,27]],"date-time":"2024-09-27T12:07:26Z","timestamp":1727438846000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s00521-024-10181-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,8]]},"references-count":37,"journal-issue":{"issue":"31","published-print":{"date-parts":[[2024,11]]}},"alternative-id":["10181"],"URL":"https:\/\/doi.org\/10.1007\/s00521-024-10181-7","relation":{},"ISSN":["0941-0643","1433-3058"],"issn-type":[{"value":"0941-0643","type":"print"},{"value":"1433-3058","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,8,8]]},"assertion":[{"value":"19 August 2023","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 July 2024","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 August 2024","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}