{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,7,20]],"date-time":"2025-07-20T04:07:08Z","timestamp":1752984428148,"version":"3.41.0"},"reference-count":66,"publisher":"Association for Computing Machinery (ACM)","issue":"8","license":[{"start":{"date-parts":[[2024,6,13]],"date-time":"2024-06-13T00:00:00Z","timestamp":1718236800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"FWO","award":["G0A4720N and 1SB5721N"],"award-info":[{"award-number":["G0A4720N and 1SB5721N"]}]},{"DOI":"10.13039\/501100011878","name":"Flemish Government","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100011878","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Onderzoeksprogramma Artificiele Intelligentie (AI) Vlaanderen"},{"name":"Trustworthy AI Methods"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Multimedia Comput. Commun. Appl."],"published-print":{"date-parts":[[2024,8,31]]},"abstract":"<jats:p>\n            Explainable Artificial Intelligence (XAI) attempts to help humans understand machine learning decisions better and has been identified as a critical component toward increasing the trustworthiness of complex black-box systems, such as deep neural networks. In this article, we propose a generic and comprehensive framework named\n            <jats:italic>SNIPPET<\/jats:italic>\n            and create a user interface for the subjective evaluation of visual explanations, focusing on finding human-friendly explanations. SNIPPET considers human-centered evaluation tasks and incorporates the collection of human annotations. These annotations can serve as valuable feedback to validate the qualitative results obtained from the subjective assessment tasks. Moreover, we consider different user background categories during the evaluation process to ensure diverse perspectives and comprehensive evaluation. We demonstrate SNIPPET on a DeepFake face dataset. Distinguishing real from fake faces is a non-trivial task even for humans that depends on rather subtle features, making it a challenging use case. Using SNIPPET, we evaluate four popular XAI methods which provide visual explanations: Gradient-weighted Class Activation Mapping, Layer-wise Relevance Propagation, attention rollout, and Transformer Attribution. Based on our experimental results, we observe preference variations among different user categories. We find that most people are more favorable to the explanations of rollout. Moreover, when it comes to XAI-assisted understanding, those who have no or lack relevant background knowledge often consider that visual explanations are insufficient to help them understand. We open-source our framework for continued data collection and annotation at\n            <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" ext-link-type=\"url\" xlink:href=\"https:\/\/github.com\/XAI-SubjEvaluation\/SNIPPET\">https:\/\/github.com\/XAI-SubjEvaluation\/SNIPPET<\/jats:ext-link>\n            .\n          <\/jats:p>","DOI":"10.1145\/3665248","type":"journal-article","created":{"date-parts":[[2024,5,22]],"date-time":"2024-05-22T11:34:06Z","timestamp":1716377646000},"page":"1-29","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["SNIPPET: A Framework for Subjective Evaluation of Visual Explanations Applied to DeepFake Detection"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2110-9318","authenticated-orcid":false,"given":"Yuqing","family":"Yang","sequence":"first","affiliation":[{"name":"Departement of Electronics and Informatics, VUB, Brussel, Belgium"},{"name":"Imec, Leuven Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2881-2727","authenticated-orcid":false,"given":"Boris","family":"Joukovsky","sequence":"additional","affiliation":[{"name":"Departement of Electronics and Informatics, VUB, Brussel Belgium"},{"name":"Imec, Leuven Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8607-5067","authenticated-orcid":false,"given":"Jos\u00e9","family":"Oramas Mogrovejo","sequence":"additional","affiliation":[{"name":"University of Antwerp, Antwerpen Belgium"},{"name":"Imec-IDLab, Antwerpen Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3307-9723","authenticated-orcid":false,"given":"Tinne","family":"Tuytelaars","sequence":"additional","affiliation":[{"name":"KU Leuven, Leuven Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9300-5860","authenticated-orcid":false,"given":"Nikos","family":"Deligiannis","sequence":"additional","affiliation":[{"name":"Departement of Electronics and Informatics, VUB, Brussel Belgium"},{"name":"Imec, Leuven Belgium"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,6,13]]},"reference":[{"key":"e_1_3_2_2_2","article-title":"Quantifying attention flow in transformers","author":"Abnar Samira","year":"2020","unstructured":"Samira Abnar and Willem Zuidema. 2020. Quantifying attention flow in transformers. arXiv preprint arXiv:2005.00928 (2020).","journal-title":"arXiv preprint arXiv:2005.00928"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0130140"},{"key":"e_1_3_2_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2015.06.018"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.chb.2017.08.045"},{"key":"e_1_3_2_6_2","article-title":"Exemplary natural images explain CNN activations better than state-of-the-art feature visualization","author":"Borowski Judy","year":"2020","unstructured":"Judy Borowski, Roland S. Zimmermann, Judith Schepers, Robert Geirhos, Thomas S. A. Wallis, Matthias Bethge, and Wieland Brendel. 2020. Exemplary natural images explain CNN activations better than state-of-the-art feature visualization. arXiv preprint arXiv:2010.12606 (2020).","journal-title":"arXiv preprint arXiv:2010.12606"},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2018.2815601"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00084"},{"key":"e_1_3_2_9_2","article-title":"Symbolic discovery of optimization algorithms","author":"Chen Xiangning","year":"2023","unstructured":"Xiangning Chen, Chen Liang, Da Huang, Esteban Real, Kaiyuan Wang, Yao Liu, Hieu Pham, Xuanyi Dong, Thang Luong, Cho-Jui Hsieh, Yifeng Lu, and Quoc V. Le. 2023. Symbolic discovery of optimization algorithms. arXiv preprint arXiv:2302.06675 (2023).","journal-title":"arXiv preprint arXiv:2302.06675"},{"key":"e_1_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3460940"},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1145\/3336498"},{"key":"e_1_3_2_12_2","first-page":"2832","article-title":"What I cannot predict, I do not understand: A human-centered evaluation framework for explainability methods","volume":"35","author":"Colin Julien","year":"2022","unstructured":"Julien Colin, Thomas Fel, R\u00e9mi Cad\u00e8ne, and Thomas Serre. 2022. What I cannot predict, I do not understand: A human-centered evaluation framework for explainability methods. Advances in Neural Information Processing Systems 35 (2022), 2832\u20132845.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_13_2","article-title":"The DeepFake Detection Challenge (DFDC) dataset","author":"Dolhansky Brian","year":"2020","unstructured":"Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. 2020. The DeepFake Detection Challenge (DFDC) dataset. arXiv preprint arXiv:2006.07397 (2020).","journal-title":"arXiv preprint arXiv:2006.07397"},{"key":"e_1_3_2_14_2","article-title":"An image is worth 16x16 words: Transformers for image recognition at scale","author":"Dosovitskiy Alexey","year":"2020","unstructured":"Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).","journal-title":"arXiv preprint arXiv:2010.11929"},{"issue":"3","key":"e_1_3_2_15_2","doi-asserted-by":"crossref","first-page":"226","DOI":"10.1016\/j.diin.2011.06.003","article-title":"Perceptual discrimination of computer generated and photographic faces","volume":"8","author":"Farid Hany","year":"2012","unstructured":"Hany Farid and Mary J. Bravo. 2012. Perceptual discrimination of computer generated and photographic faces. Digital Investigation 8, 3-4 (2012), 226\u2013235.","journal-title":"Digital Investigation"},{"key":"e_1_3_2_16_2","first-page":"2950","volume-title":"Proceedings of the IEEE International Conference on Computer Vision","author":"Fong Ruth","year":"2019","unstructured":"Ruth Fong, Mandela Patrick, and Andrea Vedaldi. 2019. Understanding deep networks via extremal perturbations and smooth masks. In Proceedings of the IEEE International Conference on Computer Vision. 2950\u20132958."},{"key":"e_1_3_2_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00910"},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2014.81"},{"key":"e_1_3_2_19_2","article-title":"Evaluating explainable AI: Which algorithmic explanations help users predict model behavior?","author":"Hase Peter","year":"2020","unstructured":"Peter Hase and Mohit Bansal. 2020. Evaluating explainable AI: Which algorithmic explanations help users predict model behavior? arXiv preprint arXiv:2005.01831 (2020).","journal-title":"arXiv preprint arXiv:2005.01831"},{"key":"e_1_3_2_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00434"},{"key":"e_1_3_2_22_2","doi-asserted-by":"publisher","DOI":"10.1145\/3570507"},{"key":"e_1_3_2_23_2","first-page":"4211","article-title":"How can I explain this to you? An empirical study of deep neural network explanation methods","volume":"33","author":"Jeyakumar Jeya Vikranth","year":"2020","unstructured":"Jeya Vikranth Jeyakumar, Joseph Noor, Yu-Hsi Cheng, Luis Garcia, and Mani Srivastava. 2020. How can I explain this to you? An empirical study of deep neural network explanation methods. Advances in Neural Information Processing Systems 33 (2020), 4211\u20134222.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_24_2","unstructured":"Tilke Judd Fr\u00e9do Durand and Antonio Torralba. 2012. A Benchmark of Computational Models of Saliency to Predict Human Fixations. Technical Report MIT-CSAIL-TR-2021-001. Computer Science and Artificial Intelligence Lab (CSAIL)."},{"key":"e_1_3_2_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00813"},{"key":"e_1_3_2_27_2","doi-asserted-by":"crossref","unstructured":"Sunnie S. Y. Kim Nicole Meister Vikram V. Ramaswamy Ruth Fong and Olga Russakovsky. 2022. Hive: Evaluating the human interpretability of visual explanations. In Computer Vision\u2014ECCV 2022. Lecture Notes in Computer Science Vol. 13672. Springer 280\u2013298.","DOI":"10.1007\/978-3-031-19775-8_17"},{"key":"e_1_3_2_28_2","article-title":"Deepfake detection: Humans vs. machines","author":"Korshunov Pavel","year":"2020","unstructured":"Pavel Korshunov and S\u00e9bastien Marcel. 2020. Deepfake detection: Humans vs. machines. arXiv preprint arXiv:2009.03155 (2020).","journal-title":"arXiv preprint arXiv:2009.03155"},{"key":"e_1_3_2_29_2","article-title":"ImageNet classification with deep convolutional neural networks","volume":"25","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25 (2012), 1\u20139.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_30_2","first-page":"259","volume-title":"Proceedings of Vision Interface","volume":"32","author":"Lalonde Marc","year":"2001","unstructured":"Marc Lalonde, Langis Gagnon, and Marie-Carole Boucher. 2001. Automatic visual quality assessment in optical fundus images. In Proceedings of Vision Interface, Vol. 32. 259\u2013264."},{"key":"e_1_3_2_31_2","doi-asserted-by":"publisher","DOI":"10.3758\/s13428-012-0226-9"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.visres.2007.06.015"},{"key":"e_1_3_2_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00559"},{"key":"e_1_3_2_34_2","first-page":"1581","volume-title":"Medical Imaging 1999: Image Processing","author":"Lee Samuel C.","year":"1999","unstructured":"Samuel C. Lee and Yiming Wang. 1999. Automatic retinal image quality assessment and enhancement. In Medical Imaging 1999: Image Processing, Vol. 3661. SPIE, 1581\u20131590."},{"key":"e_1_3_2_35_2","article-title":"FaceShifter: Towards high fidelity and occlusion aware face swapping","author":"Li Lingzhi","year":"2019","unstructured":"Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, and Fang Wen. 2019. FaceShifter: Towards high fidelity and occlusion aware face swapping. arXiv preprint arXiv:1912.13457 (2019).","journal-title":"arXiv preprint arXiv:1912.13457"},{"key":"e_1_3_2_36_2","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467213"},{"key":"e_1_3_2_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"e_1_3_2_38_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01167"},{"key":"e_1_3_2_39_2","unstructured":"He Lyu Ningyu Sha Shuyang Qin Ming Yan Yuying Xie and Rongrong Wang. 2019. Manifold denoising by nonlinear robust principal component analysis. Advances in Neural Information Processing Systems 32 (2019) 1\u201311."},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/3387166"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patcog.2016.11.008"},{"key":"e_1_3_2_42_2","first-page":"23296","article-title":"Intriguing properties of vision transformers","volume":"34","author":"Naseer Muhammad Muzammal","year":"2021","unstructured":"Muhammad Muzammal Naseer, Kanchana Ranasinghe, Salman H. Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang. 2021. Intriguing properties of vision transformers. Advances in Neural Information Processing Systems 34 (2021), 23296\u201323308.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_43_2","first-page":"26422","article-title":"The effectiveness of feature attribution methods and its correlation with automatic evaluation scores: Evaluating explainable AI","volume":"34","author":"Nguyen Giang","year":"2021","unstructured":"Giang Nguyen, Daeyoung Kim, and Anh Nguyen. 2021. The effectiveness of feature attribution methods and its correlation with automatic evaluation scores: Evaluating explainable AI. Advances in Neural Information Processing Systems 34 (2021), 26422\u201326436.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_44_2","article-title":"Visual correspondence-based explanations improve AI robustness and human-AI team accuracy","author":"Nguyen Giang","year":"2022","unstructured":"Giang Nguyen, Mohammad Reza Taesiri, and Anh Nguyen. 2022. Visual correspondence-based explanations improve AI robustness and human-AI team accuracy. In Proceedings of the Conference on Neural Information Processing Systems (NeurIPS\u201922).","journal-title":"Proceedings of the Conference on Neural Information Processing Systems (NeurIPS\u201922)."},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.5555\/2821575"},{"key":"e_1_3_2_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445315"},{"key":"e_1_3_2_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00009"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-015-0816-y"},{"key":"e_1_3_2_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2016.2599820"},{"key":"e_1_3_2_50_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-28954-6"},{"key":"e_1_3_2_51_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.74"},{"key":"e_1_3_2_52_2","first-page":"1","volume-title":"Proceedings of the 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG\u201921)","author":"Shen Bingyu","year":"2021","unstructured":"Bingyu Shen, Brandon RichardWebster, Alice O\u2019Toole, Kevin Bowyer, and Walter J. Scheirer. 2021. A study of the human perception of synthetic faces. In Proceedings of the 2021 16th IEEE International Conference on Automatic Face and Gesture Recognition (FG\u201921). IEEE, 1\u20138."},{"issue":"3","key":"e_1_3_2_53_2","first-page":"1","article-title":"Objective object segmentation visual quality evaluation: Quality measure and pooling method","volume":"18","author":"Shi Ran","year":"2022","unstructured":"Ran Shi, Jing Ma, King Ngi Ngan, Jian Xiong, and Tong Qiao. 2022. Objective object segmentation visual quality evaluation: Quality measure and pooling method. ACM Transactions on Multimedia Computing, Communications, and Applications 18, 3 (2022), 1\u201319.","journal-title":"ACM Transactions on Multimedia Computing, Communications, and Applications"},{"key":"e_1_3_2_54_2","first-page":"3145","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Shrikumar Avanti","year":"2017","unstructured":"Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. 2017. Learning important features through propagating activation differences. In Proceedings of the International Conference on Machine Learning. 3145\u20133153."},{"key":"e_1_3_2_55_2","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014).","journal-title":"arXiv preprint arXiv:1409.1556"},{"key":"e_1_3_2_56_2","article-title":"SmoothGrad: Removing noise by adding noise","author":"Smilkov Daniel","year":"2017","unstructured":"Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Vi\u00e9gas, and Martin Wattenberg. 2017. SmoothGrad: Removing noise by adding noise. arXiv preprint arXiv:1706.03825 (2017).","journal-title":"arXiv preprint arXiv:1706.03825"},{"key":"e_1_3_2_57_2","doi-asserted-by":"publisher","DOI":"10.1016\/0959-4752(94)90003-5"},{"key":"e_1_3_2_58_2","article-title":"Attention is all you need","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Processing Systems 30 (2017), 1\u201311.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_59_2","unstructured":"Catherine Wah Steve Branson Peter Welinder Pietro Perona and Serge Belongie. 2011. The Caltech-UCSD Birds-200-2011 Dataset. California Institute of Technology."},{"key":"e_1_3_2_60_2","article-title":"Towards human-understandable visual explanations: Imperceptible high-frequency cues can better be removed","author":"Wang Kaili","year":"2021","unstructured":"Kaili Wang, Jose Oramas, and Tinne Tuytelaars. 2021. Towards human-understandable visual explanations: Imperceptible high-frequency cues can better be removed. arXiv preprint arXiv:2104.07954 (2021).","journal-title":"arXiv preprint arXiv:2104.07954"},{"key":"e_1_3_2_61_2","article-title":"ONE-PEACE: Exploring one general representation model toward unlimited modalities","author":"Wang Peng","year":"2023","unstructured":"Peng Wang, Shijie Wang, Junyang Lin, Shuai Bai, Xiaohuan Zhou, Jingren Zhou, Xinggang Wang, and Chang Zhou. 2023. ONE-PEACE: Exploring one general representation model toward unlimited modalities. arXiv preprint arXiv:2305.11172 (2023).","journal-title":"arXiv preprint arXiv:2305.11172"},{"key":"e_1_3_2_62_2","article-title":"Evaluating explanation without ground truth in interpretable machine learning","author":"Yang Fan","year":"2019","unstructured":"Fan Yang, Mengnan Du, and Xia Hu. 2019. Evaluating explanation without ground truth in interpretable machine learning. arXiv preprint arXiv:1907.06831 (2019).","journal-title":"arXiv preprint arXiv:1907.06831"},{"key":"e_1_3_2_63_2","volume-title":"Proceedings of the International Conference on Quality of Multimedia Experience (QoMEX\u201923)","author":"Yang Yuqing","year":"2023","unstructured":"Yuqing Yang, Saeed Mahmoudpour, Peter Schelkens, and Nikos Deligiannis. 2023. Evaluating quality of visual explanations of deep learning models for vision tasks. In Proceedings of the International Conference on Quality of Multimedia Experience (QoMEX\u201923)."},{"issue":"3","key":"e_1_3_2_64_2","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.visinf.2018.09.001","article-title":"A user-based taxonomy for deep learning visualization","volume":"2","author":"Yu Rulei","year":"2018","unstructured":"Rulei Yu and Lei Shi. 2018. A user-based taxonomy for deep learning visualization. Visual Informatics 2, 3 (2018), 147\u2013154.","journal-title":"Visual Informatics"},{"key":"e_1_3_2_65_2","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3372852"},{"key":"e_1_3_2_66_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11633-022-1348-x"},{"key":"e_1_3_2_67_2","article-title":"DETRs with collaborative hybrid assignments training","author":"Zong Zhuofan","year":"2022","unstructured":"Zhuofan Zong, Guanglu Song, and Yu Liu. 2022. DETRs with collaborative hybrid assignments training. arXiv preprint arXiv:2211.12860 (2022).","journal-title":"arXiv preprint arXiv:2211.12860"}],"container-title":["ACM Transactions on Multimedia Computing, Communications, and Applications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3665248","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3665248","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:58:33Z","timestamp":1750294713000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3665248"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,13]]},"references-count":66,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2024,8,31]]}},"alternative-id":["10.1145\/3665248"],"URL":"https:\/\/doi.org\/10.1145\/3665248","relation":{},"ISSN":["1551-6857","1551-6865"],"issn-type":[{"type":"print","value":"1551-6857"},{"type":"electronic","value":"1551-6865"}],"subject":[],"published":{"date-parts":[[2024,6,13]]},"assertion":[{"value":"2023-09-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-05-12","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-06-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}