{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,8]],"date-time":"2026-08-08T17:30:38Z","timestamp":1786210238362,"version":"3.56.0"},"reference-count":48,"publisher":"Association for Computing Machinery (ACM)","issue":"11","license":[{"start":{"date-parts":[[2024,9,12]],"date-time":"2024-09-12T00:00:00Z","timestamp":1726099200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,11,30]]},"abstract":"<jats:p>Detecting and recognizing deepfakes is a pressing issue in the digital age. In this study, we first collected a dataset of pristine images and fake ones properly generated by nine different Generative Adversarial Network (GAN) architectures and four Diffusion Models (DM). The dataset contained a total of 83,000 images, with equal distribution between the real and deepfake data. Then, to address different deepfake detection and recognition tasks, we proposed a hierarchical multi-level approach. At the first level, we classified real images from AI-generated ones. At the second level, we distinguished between images generated by GANs and DMs. At the third level (composed of two additional sub-levels), we recognized the specific GAN and DM architectures used to generate the synthetic data. Experimental results demonstrated that our approach achieved more than 97% classification accuracy, outperforming existing state-of-the-art methods. The models obtained in the different levels turn out to be robust to various attacks such as JPEG compression (with different quality factor values) and resize (and others), demonstrating that the framework can be used and applied in real-world contexts (such as the analysis of multimedia data shared in the various social platforms) for support even in forensic investigations to counter the illicit use of these powerful and modern generative models. We are able to identify the specific GAN and DM architecture used to generate the image, which is critical in tracking down the source of the deepfake. Our hierarchical multi-level approach to deepfake detection and recognition shows promising results in identifying deepfakes allowing focus on underlying task by improving (about 2% on the average) standard multiclass flat detection systems. The proposed method has the potential to enhance the performance of deepfake detection systems, aid in the fight against the spread of fake images, and safeguard the authenticity of digital media.<\/jats:p>","DOI":"10.1145\/3652027","type":"journal-article","created":{"date-parts":[[2024,3,9]],"date-time":"2024-03-09T09:43:56Z","timestamp":1709977436000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":72,"title":["Mastering Deepfake Detection: A Cutting-edge Approach to Distinguish GAN and Diffusion-model Images"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8315-351X","authenticated-orcid":false,"given":"Luca","family":"Guarnera","sequence":"first","affiliation":[{"name":"Department of Mathematics and Computer Science, University of Catania, Catania, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8343-2049","authenticated-orcid":false,"given":"Oliver","family":"Giudice","sequence":"additional","affiliation":[{"name":"Banca d'Italia, Roma, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6127-2470","authenticated-orcid":false,"given":"Sebastiano","family":"Battiato","sequence":"additional","affiliation":[{"name":"Department of Mathematics and Computer Science, University of Catania, Catania, Italy"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,9,12]]},"reference":[{"key":"e_1_3_2_2_2","article-title":"A siamese-based verification system for open-set architecture attribution of synthetic images","author":"Abady Lydia","year":"2023","unstructured":"Lydia Abady, Jun Wang, Benedetta Tondi, and Mauro Barni. 2023. A siamese-based verification system for open-set architecture attribution of synthetic images. arXiv preprint arXiv:2307.09822 (2023).","journal-title":"arXiv preprint arXiv:2307.09822"},{"key":"e_1_3_2_3_2","article-title":"Parents and children: Distinguishing multimodal deepfakes from natural images","author":"Amoroso Roberto","year":"2023","unstructured":"Roberto Amoroso, Davide Morelli, Marcella Cornia, Lorenzo Baraldi, Alberto Del Bimbo, and Rita Cucchiara. 2023. Parents and children: Distinguishing multimodal deepfakes from natural images. arXiv preprint arXiv:2304.00500 (2023).","journal-title":"arXiv preprint arXiv:2304.00500"},{"key":"e_1_3_2_4_2","first-page":"5","volume-title":"Proceedings of the 17th International Conference on Computer Systems and Technologies","author":"Battiato Sebastiano","year":"2016","unstructured":"Sebastiano Battiato, Oliver Giudice, and Antonino Paratore. 2016. Multimedia forensics: Discovering the history of multimedia contents. In Proceedings of the 17th International Conference on Computer Systems and Technologies. 5\u201316."},{"key":"e_1_3_2_5_2","first-page":"602","volume-title":"Proceedings of the 11th International Conference on Image Analysis and Processing","author":"Battiato Sebastiano","year":"2001","unstructured":"Sebastiano Battiato, Massimo Mancuso, Angelo Bosco, and Mirko Guarnera. 2001. Psychovisual and statistical optimization of quantization tables for DCT compression engines. In Proceedings of the 11th International Conference on Image Analysis and Processing. IEEE, 602\u2013606."},{"key":"e_1_3_2_6_2","first-page":"10639","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Cho Wonwoong","year":"2019","unstructured":"Wonwoong Cho, Sungha Choi, David Keetae Park, Inkyu Shin, and Jaegul Choo. 2019. Image-to-image translation via group-wise deep whitening-and-coloring transformation. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 10639\u201310647."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00916"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00821"},{"key":"e_1_3_2_9_2","first-page":"1","volume-title":"Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201923)","author":"Corvi Riccardo","year":"2023","unstructured":"Riccardo Corvi, Davide Cozzolino, Giada Zingarini, Giovanni Poggi, Koki Nagano, and Luisa Verdoliva. 2023. On the detection of synthetic images generated by diffusion models. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP\u201923). IEEE, 1\u20135."},{"key":"e_1_3_2_10_2","first-page":"8780","article-title":"Diffusion models beat GANs on image synthesis","volume":"34","author":"Dhariwal Prafulla","year":"2021","unstructured":"Prafulla Dhariwal and Alexander Nichol. 2021. Diffusion models beat GANs on image synthesis. Adv. Neural Inf. Process. Syst. 34 (2021), 8780\u20138794.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52729.2023.00389"},{"key":"e_1_3_2_12_2","volume-title":"Proceedings of the International Conference on Learning Representations","author":"Dosovitskiy Alexey","year":"2020","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, and Sylvain Gelly. 2020. An image is worth 16 \\(\\times\\) 16 words: Transformers for image recognition at scale. In Proceedings of the International Conference on Learning Representations."},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2014.05.014"},{"key":"e_1_3_2_14_2","doi-asserted-by":"publisher","DOI":"10.3390\/jimaging7080128"},{"key":"e_1_3_2_15_2","first-page":"2672","volume-title":"Adv. Neural Inf. Process. Syst.","author":"Goodfellow Ian","year":"2014","unstructured":"Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Adv. Neural Inf. Process. Syst.. 2672\u20132680."},{"key":"e_1_3_2_16_2","first-page":"1","volume-title":"Proceedings of the IEEE International Conference on Multimedia and Expo (ICME)","author":"Gragnaniello Diego","year":"2021","unstructured":"Diego Gragnaniello, Davide Cozzolino, Francesco Marra, Giovanni Poggi, and Luisa Verdoliva. 2021. Are GAN generated images easy to detect? a critical analysis of the state-of-the-art. In Proceedings of the IEEE International Conference on Multimedia and Expo (ICME). IEEE, 1\u20136."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3023037"},{"key":"e_1_3_2_18_2","series-title":"Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/978-3-031-06430-2_13","volume-title":"International Conference on Image Analysis and Processing","volume":"13232","author":"Guarnera Luca","year":"2022","unstructured":"Luca Guarnera, Oliver Giudice, and Sebastiano Battiato. 2022. Deepfake style transfer mixture: A first forensic ballistics study on synthetic images. In International Conference on Image Analysis and Processing(Lecture Notes in Computer Science, Vol. 13232). Springer, Cham, 151\u2013163."},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.23919\/AEIT50178.2020.9241108"},{"key":"e_1_3_2_20_2","first-page":"61","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops","author":"Guarnera Luca","year":"2022","unstructured":"Luca Guarnera, Oliver Giudice, Matthias Nie\u00dfner, and Sebastiano Battiato. 2022. On the exploitation of deepfake model recognition. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops. 61\u201370."},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2916751"},{"key":"e_1_3_2_23_2","first-page":"6840","article-title":"Denoising diffusion probabilistic models","volume":"33","author":"Ho Jonathan","year":"2020","unstructured":"Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Adv. Neural Inf. Process. Syst. 33 (2020), 6840\u20136851.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_24_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.243"},{"key":"e_1_3_2_25_2","volume-title":"Proceedings of the International Conference on Learning Representations (ICLR\u201918)","author":"Karras Tero","year":"2018","unstructured":"Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2018. Progressive growing of GANs for improved quality, stability, and variation. In Proceedings of the International Conference on Learning Representations (ICLR\u201918)."},{"key":"e_1_3_2_26_2","first-page":"852","article-title":"Alias-free generative adversarial networks","volume":"34","author":"Karras Tero","year":"2021","unstructured":"Tero Karras, Miika Aittala, Samuli Laine, Erik H\u00e4rk\u00f6nen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. 2021. Alias-free generative adversarial networks. Adv. Neural Inf. Process. Syst. 34 (2021), 852\u2013863.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_3_2_28_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"e_1_3_2_29_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV56688.2023.00139"},{"key":"e_1_3_2_30_2","first-page":"4220","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Li Ke","year":"2019","unstructured":"Ke Li, Tianhao Zhang, and Jitendra Malik. 2019. Diverse image synthesis from semantic layouts via conditional IMLE. In Proceedings of the IEEE International Conference on Computer Vision. 4220\u20134229."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.425"},{"key":"e_1_3_2_32_2","doi-asserted-by":"crossref","first-page":"506","DOI":"10.1109\/MIPR.2019.00103","article-title":"Do GANs leave artificial fingerprints?","author":"Marra Francesco","year":"2019","unstructured":"Francesco Marra, Diego Gragnaniello, Luisa Verdoliva, and Giovanni Poggi. 2019. Do GANs leave artificial fingerprints? In Proceedings of the IEEE Conference on Multimedia Information Processing and Retrieval (MIPR\u201919). 506\u2013511.","journal-title":"Proceedings of the IEEE Conference on Multimedia Information Processing and Retrieval (MIPR\u201919)"},{"key":"e_1_3_2_33_2","doi-asserted-by":"crossref","unstructured":"Momina Masood Mariam Nawaz Khalid Mahmood Malik Ali Javed Aun Irtaza and Hafiz Malik. 2023. Deepfakes generation and detection: State-of-the-art open challenges countermeasures and way forward. Applied intelligence 53 4 (2023) 3974\u20134026.","DOI":"10.1007\/s10489-022-03766-z"},{"key":"e_1_3_2_34_2","first-page":"16784","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Nichol Alexander Quinn","year":"2022","unstructured":"Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mcgrew, Ilya Sutskever, and Mark Chen. 2022. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. In Proceedings of the International Conference on Machine Learning. PMLR, 16784\u201316804."},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cose.2020.102092"},{"key":"e_1_3_2_36_2","article-title":"Hierarchical text-conditional image generation with clip latents","author":"Ramesh Aditya","year":"2022","unstructured":"Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. 2022. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125 (2022).","journal-title":"arXiv preprint arXiv:2204.06125"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01042"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00009"},{"key":"e_1_3_2_39_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_3_2_40_2","article-title":"DE-FAKE: Detection and attribution of fake images generated by text-to-image diffusion models","author":"Sha Zeyang","year":"2022","unstructured":"Zeyang Sha, Zheng Li, Ning Yu, and Yang Zhang. 2022. DE-FAKE: Detection and attribution of fake images generated by text-to-image diffusion models. arXiv preprint arXiv:2210.06998 (2022).","journal-title":"arXiv preprint arXiv:2210.06998"},{"key":"e_1_3_2_41_2","article-title":"Deep image fingerprint: Accurate and low budget synthetic image detector","author":"Sinitsa Sergey","year":"2023","unstructured":"Sergey Sinitsa and Ohad Fried. 2023. Deep image fingerprint: Accurate and low budget synthetic image detector. arXiv preprint arXiv:2303.10762 (2023).","journal-title":"arXiv preprint arXiv:2303.10762"},{"key":"e_1_3_2_42_2","first-page":"2256","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Sohl-Dickstein Jascha","year":"2015","unstructured":"Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. 2015. Deep unsupervised learning using nonequilibrium thermodynamics. In Proceedings of the International Conference on Machine Learning. PMLR, 2256\u20132265."},{"key":"e_1_3_2_43_2","first-page":"6105","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Tan Mingxing","year":"2019","unstructured":"Mingxing Tan and Quoc Le. 2019. EfficientNet: Rethinking model scaling for convolutional neural networks. In Proceedings of the International Conference on Machine Learning. PMLR, 6105\u20136114."},{"key":"e_1_3_2_44_2","first-page":"3444","volume-title":"Proceedings of the 29th International Conference on International Joint Conferences on Artificial Intelligence","author":"Wang Run","year":"2021","unstructured":"Run Wang, Felix Juefei-Xu, Lei Ma, Xiaofei Xie, Yihao Huang, Jian Wang, and Yang Liu. 2021. FakeSpotter: A simple yet robust baseline for spotting AI-synthesized fake faces. In Proceedings of the 29th International Conference on International Joint Conferences on Artificial Intelligence. 3444\u20133451."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00872"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.634"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/WIFS47025.2019.9035107"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.544"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.244"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3652027","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3652027","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:12Z","timestamp":1750291392000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3652027"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,9,12]]},"references-count":48,"journal-issue":{"issue":"11","published-print":{"date-parts":[[2024,11,30]]}},"alternative-id":["10.1145\/3652027"],"URL":"https:\/\/doi.org\/10.1145\/3652027","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"value":"1551-6857","type":"print"},{"value":"1551-6865","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,9,12]]},"assertion":[{"value":"2023-04-20","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-25","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-12","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}