{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,20]],"date-time":"2026-04-20T10:24:58Z","timestamp":1776680698737,"version":"3.51.2"},"reference-count":46,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2022,3,24]],"date-time":"2022-03-24T00:00:00Z","timestamp":1648080000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100016714","name":"University of Sharjah","doi-asserted-by":"publisher","award":["Open UAE"],"award-info":[{"award-number":["Open UAE"]}],"id":[{"id":"10.13039\/100016714","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Creating deepfake multimedia, and especially deepfake videos, has become much easier these days due to the availability of deepfake tools and the virtually unlimited numbers of face images found online. Research and industry communities have dedicated time and resources to develop detection methods to expose these fake videos. Although these detection methods have been developed over the past few years, synthesis methods have also made progress, allowing for the production of deepfake videos that are harder and harder to differentiate from real videos. This research paper proposes an improved optical flow estimation-based method to detect and expose the discrepancies between video frames. Augmentation and modification are experimented upon to try to improve the system\u2019s overall accuracy. Furthermore, the system is trained on graphics processing units (GPUs) and tensor processing units (TPUs) to explore the effects and benefits of each type of hardware in deepfake detection. TPUs were found to have shorter training times compared to GPUs. VGG-16 is the best performing model when used as a backbone for the system, as it achieved around 82.0% detection accuracy when trained on GPUs and 71.34% accuracy on TPUs.<\/jats:p>","DOI":"10.3390\/s22072500","type":"journal-article","created":{"date-parts":[[2022,3,24]],"date-time":"2022-03-24T23:31:43Z","timestamp":1648164703000},"page":"2500","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":9,"title":["Improved Optical Flow Estimation Method for Deepfake Videos"],"prefix":"10.3390","volume":"22","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1570-0897","authenticated-orcid":false,"given":"Ali Bou","family":"Nassif","sequence":"first","affiliation":[{"name":"Department of Computer Engineering, University of Sharjah, Sharjah 27272, United Arab Emirates"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Qassim","family":"Nasir","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, University of Sharjah, Sharjah 27272, United Arab Emirates"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Manar Abu","family":"Talib","sequence":"additional","affiliation":[{"name":"Department of Computer Science, University of Sharjah, Sharjah 27272, United Arab Emirates"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Omar Mohamed","family":"Gouda","sequence":"additional","affiliation":[{"name":"Department of Computer Engineering, University of Sharjah, Sharjah 27272, United Arab Emirates"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,24]]},"reference":[{"key":"ref_1","unstructured":"De Lima, O., Franklin, S., Basu, S., Karwoski, B., and George, A. (2020). Deepfake detection using spatiotemporal convolutional networks. arXiv."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1016\/j.patrec.2021.03.005","article-title":"Optical Flow based CNN for detection of unlearnt deepfake manipulations","volume":"146","author":"Caldelli","year":"2021","journal-title":"Pattern Recognit. Lett."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Fagni, T., Falchi, F., Gambini, M., Martella, A., and Tesconi, M. (2020). TweepFake: About detecting deepfake tweets. arXiv.","DOI":"10.1371\/journal.pone.0251415"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1016\/j.inffus.2020.06.014","article-title":"DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection","volume":"64","author":"Tolosana","year":"2020","journal-title":"Inf. Fusion"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Khalid, H., and Woo, S.S. (2020, January 14\u201319). OC-FakeDect: Classifying Deepfakes Using One-Class Variational Autoencoder. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, Seattle, WA, USA.","DOI":"10.1109\/CVPRW50498.2020.00336"},{"key":"ref_6","unstructured":"(2022, February 25). FaceApp\u2014Free Neural Face Transformation Filters. Available online: https:\/\/www.faceapp.com\/."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Qi, H., Guo, Q., Juefei-Xu, F., Xie, X., Ma, L., Feng, W., Liu, Y., and Zhao, J. (2020, January 12\u201316). DeepRhythm. Proceedings of the 28th ACM International Conference on Multimedia, Seattle, WA, USA.","DOI":"10.1145\/3394171.3413707"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"910","DOI":"10.1109\/JSTSP.2020.3002101","article-title":"Media Forensics and DeepFakes: An Overview","volume":"14","author":"Verdoliva","year":"2020","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., and Niessner, M. (2019, January 27\u201328). FaceForensics++: Learning to Detect Manipulated Facial Images. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00009"},{"key":"ref_10","first-page":"416","article-title":"FDFtNet: Facing Off Fake Images Using Fake Detection Fine-Tuning Network","volume":"580","author":"Jeon","year":"2020","journal-title":"IFIP Adv. Inf. Commun. Technol."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Matern, F., Riess, C., and Stamminger, M. (2019, January 7\u201311). Exploiting Visual Artifacts to Expose Deepfakes and Face Manipulations. Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision Workshops (WACVW), Waikoloa Village, HI, USA.","DOI":"10.1109\/WACVW.2019.00020"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"2575","DOI":"10.1007\/s00521-018-3760-2","article-title":"Novel cascaded Gaussian mixture model-deep neural network classifier for speaker identification in emotional talking environments","volume":"32","author":"Shahin","year":"2018","journal-title":"Neural Comput. Appl."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"19143","DOI":"10.1109\/ACCESS.2019.2896880","article-title":"Speech Recognition Using Deep Neural Networks: A Systematic Review","volume":"7","author":"Nassif","year":"2019","journal-title":"IEEE Access"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Li, Y., Yang, X., Sun, P., Qi, H., and Lyu, S. (2020, January 14\u201319). Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00327"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Gao, H., Pei, J., and Huang, H. (2020, January 13\u201319). Progan: Network Embedding via Proximity Generative Adversarial Network. Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Seattle, WA, USA.","DOI":"10.1145\/3292500.3330866"},{"key":"ref_16","first-page":"10215","article-title":"Glow: Generative Flow with Invertible 1 \u00d7 1 Convolutions","volume":"2018","author":"Kingma","year":"2018","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_17","unstructured":"Thies, J., Zollh\u00f6fer, M., Stamminger, M., Theobalt, C., and Nie\u00dfner, M. (2019, January 11\u201315). Face2Face: Real-Time Face Capture and Reenactment of RGB Videos. Proceedings of the Communications of the ACM, London, UK."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Balakrishnan, G., Durand, F., and Guttag, J. (2013, January 23\u201328). Detecting Pulse from Head Motions in Video. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.440"},{"key":"ref_19","unstructured":"Dolhansky, B., Bitton, J., Pflaum, B., Lu, J., Howes, R., Wang, M., and Ferrer, C.C. (2020). The DeepFake detection challenge dataset. arXiv."},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Guera, D., and Delp, E.J. (2018, January 27\u201330). Deepfake Video Detection Using Recurrent Neural Networks. Proceedings of the AVSS 2018 15th IEEE International Conference on Advanced Video and Signal-Based Surveillance (AVSS), Auckland, New Zealand.","DOI":"10.1109\/AVSS.2018.8639163"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Laptev, I., Marsza\u0142ek, M., Schmid, C., and Rozenfeld, B. (2008, January 23\u201328). Learning Realistic Human Actions from Movies. Proceedings of the 26th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, AK, USA.","DOI":"10.1109\/CVPR.2008.4587756"},{"key":"ref_22","unstructured":"Amerini, I., Galteri, L., Caldelli, R., and Bimbo, A. (November, January 27). Del Deepfake Video Detection through Optical Flow based CNN. Proceedings of the International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1408","DOI":"10.1109\/TPAMI.2019.2894353","article-title":"Models Matter, so Does Training: An Empirical Study of CNNs for Optical Flow Estimation","volume":"42","author":"Sun","year":"2020","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"769","DOI":"10.1016\/j.infsof.2008.09.005","article-title":"Supporting user-oriented analysis for multi-view domain-specific visual languages","volume":"51","author":"Guerra","year":"2009","journal-title":"Inf. Softw. Technol."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep Residual Learning for Image Recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Chollet, F. (2016, January 21\u201326). Xception: Deep Learning with Depthwise Separable Convolutions. Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Honolulu, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.195"},{"key":"ref_27","unstructured":"Karras, T., Aila, T., Laine, S., and Lehtinen, J. (May, January 30). Progressive Growing of GANs for Improved Quality, Stability, and Variation. Proceedings of the 6th International Conference on Learning Representations (ICLR), Vancouver, BC, Canada."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Li, L., Bao, J., Zhang, T., Yang, H., Chen, D., Wen, F., and Guo, B. (2020;, January 14\u201319). Face X-ray for More General Face Forgery Detection. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00505"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Li, Y., Chang, M.C., and Lyu, S. (2018, January 11\u201313). In Ictu Oculi: Exposing AI Created Fake Videos by Detecting Eye Blinking. Proceedings of the 10th IEEE International Workshop on Information Forensics and Security (WIFS), Hong Kong, China.","DOI":"10.1109\/WIFS.2018.8630787"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Chintha, A., Rao, A., Sohrawardi, S., Bhatt, K., Wright, M., and Ptucha, R. (October, January 28). Leveraging Edges and Optical Flow on Faces for Deepfake Detection. Proceedings of the 2020 IEEE International Joint Conference on Biometrics (IJCB), Houston, TX, USA.","DOI":"10.1109\/IJCB48548.2020.9304936"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Dai, G., Xie, J., and Fang, Y. (2017, January 23\u201327). Metric-Based Generative Adversarial Network. Proceedings of the 2017 ACM Multimedia Conference, Mountain View, CA, USA.","DOI":"10.1145\/3123266.3123334"},{"key":"ref_32","first-page":"1","article-title":"Synthesizing obama: Learning lip sync from audio","volume":"36","author":"Suwajanakorn","year":"2017","journal-title":"Assoc. Comput. Mach. Trans. Graph."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Prenger, R., Valle, R., and Catanzaro, B. (2019, January 12\u201317). Waveglow: A Flow-Based Generative Network for Speech Synthesis. Proceedings of the ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, UK.","DOI":"10.1109\/ICASSP.2019.8683143"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Zhao, W., Xie, Q., Ma, Y., Liu, Y., and Xiong, S. (2020;, January 25\u201328). Pose Guided Person Image Generation Based on Pose Skeleton Sequence and 3D Convolution. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/ICIP40778.2020.9190773"},{"key":"ref_35","unstructured":"Baker, S., Roth, S., Scharstein, D., Black, M.J., Lewis, J.P., and Szeliski, R. (2017, January 22\u201329). A Database and Evaluation Methodology for Optical Flow. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/0004-3702(81)90024-2","article-title":"Determining Optical Flow","volume":"17","author":"Horn","year":"1981","journal-title":"Artif. Intell."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Dosovitskiy, A., Fischery, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., van der Smagt, P., Cremers, D., and Brox, T. (2015, January 7\u201313). FlowNet: Learning optical flow with convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.316"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1499","DOI":"10.1109\/LSP.2016.2603342","article-title":"Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks","volume":"23","author":"Zhang","year":"2016","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"085201","DOI":"10.1088\/1751-8113\/44\/8\/085201","article-title":"A \u201cmissing\u201d family of classical orthogonal polynomials","volume":"44","author":"Vinet","year":"2011","journal-title":"J. Phys. A Math. Theor."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Ketkar, N., and Ketkar, N. (2017). Introduction to Keras. Deep Learning with Python, Apress.","DOI":"10.1007\/978-1-4842-2766-4"},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"84","DOI":"10.1145\/3065386","article-title":"ImageNet classification with deep convolutional neural networks","volume":"60","author":"Krizhevsky","year":"2017","journal-title":"Commun. ACM"},{"key":"ref_42","first-page":"1","article-title":"In-Datacenter Performance Analysis of a Tensor Processing Unit","volume":"Volume F1286","author":"Jouppi","year":"2017","journal-title":"Proceedings of the International Symposium on Computer Architecture"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., and Brox, T. (2016, January 21\u201326). FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks. Proceedings of the 30th IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.179"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., and Wojna, Z. (2016, January 27\u201330). Rethinking the Inception Architecture for Computer Vision. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.308"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Howard, A., Sandler, M., Chen, B., Wang, W., Chen, L.C., Tan, M., Chu, G., Vasudevan, V., Zhu, Y., and Pang, R. (2019;, January 27\u201328). Searching for mobileNetV3. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea.","DOI":"10.1109\/ICCV.2019.00140"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.neucom.2018.01.091","article-title":"Segmentation of images by color features: A survey","volume":"292","author":"Cervantes","year":"2018","journal-title":"Neurocomputing"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/7\/2500\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:42:33Z","timestamp":1760136153000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/22\/7\/2500"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,24]]},"references-count":46,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["s22072500"],"URL":"https:\/\/doi.org\/10.3390\/s22072500","relation":{"has-preprint":[{"id-type":"doi","id":"10.21203\/rs.3.rs-882636\/v1","asserted-by":"object"}]},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,24]]}}}