{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,13]],"date-time":"2026-02-13T14:54:47Z","timestamp":1770994487402,"version":"3.50.1"},"reference-count":77,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,2,5]],"date-time":"2024-02-05T00:00:00Z","timestamp":1707091200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61902148"],"award-info":[{"award-number":["61902148"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100007129","name":"Shandong Provincial Natural Science Foundation","doi-asserted-by":"crossref","award":["ZR2020MF055, ZR2021LZH007, ZR2020LZH002, ZR2020QF045"],"award-info":[{"award-number":["ZR2020MF055, ZR2021LZH007, ZR2020LZH002, ZR2020QF045"]}],"id":[{"id":"10.13039\/501100007129","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Priv. Secur."],"published-print":{"date-parts":[[2024,2,29]]},"abstract":"<jats:p>Deepfake data contains realistically manipulated faces\u2014its abuses pose a huge threat to the security and privacy-critical applications. Intensive research from academia and industry has produced many deepfake\/detection models, leading to a constant race of attack and defense. However, due to the lack of a unified evaluation platform, many critical questions on this subject remain largely unexplored. How is the anti-detection ability of the existing deepfake models? How generalizable are existing detection models against different deepfake samples? How effective are the detection APIs provided by the cloud-based vendors? How evasive and transferable are adversarial deepfakes in the lab and real-world environment? How do various factors impact the performance of deepfake and detection models?<\/jats:p>\n          <jats:p>\n            To bridge the gap, we design and implement\n            <jats:monospace>DEEPFAKER<\/jats:monospace>\n            <jats:xref ref-type=\"fn\">\n              <jats:sup>1<\/jats:sup>\n            <\/jats:xref>\n            a unified and comprehensive deepfake detection evaluation platform. Specifically,\n            <jats:monospace>DEEPFAKER<\/jats:monospace>\n            has integrated 10 state-of-the-art deepfake methods and 9 representative detection methods, while providing a user-friendly interface and modular design that allows for easy integration of new methods. Leveraging\n            <jats:monospace>DEEPFAKER<\/jats:monospace>\n            , we conduct a large-scale empirical study of facial deepfake\/detection models and draw a set of key findings: (i)\u00a0the detection methods have poor generalization on samples generated by different deepfake methods; (ii)\u00a0there is no significant correlation between anti-detection ability and visual quality of deepfake samples; (iii)\u00a0the current detection APIs have poor detection performance and adversarial deepfakes can achieve about 70% attack success rate on all cloud-based vendors, calling for an urgent need to deploy effective and robust detection APIs; (iv)\u00a0the detection methods in the lab are more robust against transfer attacks than the detection APIs in the real-world environment; and (v)\u00a0deepfake videos may not always be more difficult to detect after video compression. We envision that\n            <jats:monospace>DEEPFAKER<\/jats:monospace>\n            will benefit future research on facial deepfake and detection.\n          <\/jats:p>","DOI":"10.1145\/3634914","type":"journal-article","created":{"date-parts":[[2023,11,29]],"date-time":"2023-11-29T12:00:36Z","timestamp":1701259236000},"page":"1-34","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["DEEPFAKER: A Unified Evaluation Platform for Facial Deepfake and Detection Models"],"prefix":"10.1145","volume":"27","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1289-5457","authenticated-orcid":false,"given":"Li","family":"Wang","sequence":"first","affiliation":[{"name":"School of Cyber Science and Technology, Shandong University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-5613-8932","authenticated-orcid":false,"given":"Xiangtao","family":"Meng","sequence":"additional","affiliation":[{"name":"School of Cyber Science and Technology, Shandong University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-4079-2091","authenticated-orcid":false,"given":"Dan","family":"Li","sequence":"additional","affiliation":[{"name":"Information Security Center, China Electronic Product Reliability and Environmental Testing Research Institute, and Key Laboratory of Ministry of Industry and Information Technology, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8571-9780","authenticated-orcid":false,"given":"Xuhong","family":"Zhang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Zhejiang University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4268-372X","authenticated-orcid":false,"given":"Shouling","family":"Ji","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Zhejiang University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3367-0951","authenticated-orcid":false,"given":"Shanqing","family":"Guo","sequence":"additional","affiliation":[{"name":"School of Cyber Science and Technology, Shandong University, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,2,5]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/WIFS.2018.8630761"},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0031-3203(96)00142-2"},{"key":"e_1_3_3_4_2","unstructured":"John Brandon. 2018. Terrifying High-Tech Porn: Creepy \u2018Deepfake\u2019 Videos Are on the Rise. Retrieved December 6 2023 from https:\/\/www.foxnews.com\/tech\/terrifying-high-tech-porn-creepy-deepfake-videos-are-on-the-rise"},{"key":"e_1_3_3_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.116"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1109\/FG.2018.00020"},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW50498.2020.00337"},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICME51207.2021.9428361"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3413630"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3544746"},{"key":"e_1_3_3_11_2","first-page":"1","volume-title":"Proceedings of the International Conference of the Biometrics Special Interest Group (BIOSIG\u201912)","author":"Chingovska Ivana","year":"2012","unstructured":"Ivana Chingovska, Andr\u00e9 Anjos, and S\u00e9bastien Marcel. 2012. On the effectiveness of local binary patterns in face anti-spoofing. In Proceedings of the International Conference of the Biometrics Special Interest Group (BIOSIG\u201912). IEEE, Los Alamitos, CA, 1\u20137."},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00916"},{"key":"e_1_3_3_13_2","article-title":"The Deepfake Detection Challenge (DFDC) dataset","author":"Dolhansky Brian","year":"2020","unstructured":"Brian Dolhansky, Joanna Bitton, Ben Pflaum, Jikuo Lu, Russ Howes, Menglin Wang, and Cristian Canton Ferrer. 2020. The Deepfake Detection Challenge (DFDC) dataset. arXiv preprint arXiv:2006.07397 (2020).","journal-title":"arXiv preprint arXiv:2006.07397"},{"key":"e_1_3_3_14_2","unstructured":"Paul Ekman and Wallace V. Friesen. 1978. FacialActionCodingSystem:A Technique for the Measurement of FacialMovement. Consulting Psychologists Press Palo Alto CA."},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN48605.2020.9207034"},{"key":"e_1_3_3_16_2","article-title":"Explaining and harnessing adversarial examples","author":"Goodfellow Ian J.","year":"2014","unstructured":"Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2014. Explaining and harnessing adversarial examples. arXiv preprint arXiv:1412.6572 (2014).","journal-title":"arXiv preprint arXiv:1412.6572"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1109\/AVSS.2018.8639163"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00434"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/TIP.2019.2916751"},{"key":"e_1_3_3_20_2","first-page":"6840","article-title":"Denoising diffusion probabilistic models","volume":"33","author":"Ho Jonathan","year":"2020","unstructured":"Jonathan Ho, Ajay Jain, and Pieter Abbeel. 2020. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33 (2020), 6840\u20136851.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV48630.2021.00339"},{"key":"e_1_3_3_22_2","doi-asserted-by":"publisher","DOI":"10.5555\/1324787"},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11263-022-01606-8"},{"key":"e_1_3_3_24_2","article-title":"Progressive growing of GANs for improved quality, stability, and variation","author":"Karras Tero","year":"2017","unstructured":"Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. 2017. Progressive growing of GANs for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196 (2017).","journal-title":"arXiv preprint arXiv:1710.10196"},{"key":"e_1_3_3_25_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00453"},{"key":"e_1_3_3_26_2","article-title":"FakeAVCeleb: A novel audio-video multimodal deepfake dataset","author":"Khalid Hasam","year":"2021","unstructured":"Hasam Khalid, Shahroz Tariq, Minha Kim, and Simon S. Woo. 2021. FakeAVCeleb: A novel audio-video multimodal deepfake dataset. arXiv preprint arXiv:2108.05080 (2021).","journal-title":"arXiv preprint arXiv:2108.05080"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00246"},{"key":"e_1_3_3_28_2","article-title":"Deepfakes: A new threat to face recognition? Assessment and detection","author":"Korshunov Pavel","year":"2018","unstructured":"Pavel Korshunov and S\u00e9bastien Marcel. 2018. Deepfakes: A new threat to face recognition? Assessment and detection. arXiv preprint arXiv:1812.08685 (2018).","journal-title":"arXiv preprint arXiv:1812.08685"},{"key":"e_1_3_3_29_2","doi-asserted-by":"crossref","first-page":"99","DOI":"10.1201\/9781351251389-8","volume-title":"Artificial Intelligence Safety and Security","author":"Kurakin Alexey","year":"2018","unstructured":"Alexey Kurakin, Ian J. Goodfellow, and Samy Bengio. 2018. Adversarial examples in the physical world. In Artificial Intelligence Safety and Security. Chapman & Hall\/CRC, 99\u2013112."},{"key":"e_1_3_3_30_2","unstructured":"Guillaume Lample Neil Zeghidour Nicolas Usunier Antoine Bordes Ludovic Denoyer and Marc\u2019Aurelio Ranzato. 2017. Fader networks: Manipulating images by sliding attributes. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS\u201917). 5969\u20135978."},{"key":"e_1_3_3_31_2","article-title":"Seeing is living? Rethinking the security of facial liveness verification in the deepfake era","author":"Li Changjiang","year":"2022","unstructured":"Changjiang Li, Li Wang, Shouling Ji, Xuhong Zhang, Zhaohan Xi, Shanqing Guo, and Ting Wang. 2022. Seeing is living? Rethinking the security of facial liveness verification in the deepfake era. arXiv preprint arXiv:2202.10673 (2022).","journal-title":"arXiv preprint arXiv:2202.10673"},{"key":"e_1_3_3_32_2","article-title":"FaceShifter: Towards high fidelity and occlusion aware face swapping","author":"Li Lingzhi","year":"2019","unstructured":"Lingzhi Li, Jianmin Bao, Hao Yang, Dong Chen, and Fang Wen. 2019. FaceShifter: Towards high fidelity and occlusion aware face swapping. arXiv preprint arXiv:1912.13457 (2019).","journal-title":"arXiv preprint arXiv:1912.13457"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00505"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00853"},{"key":"e_1_3_3_35_2","unstructured":"Yuezun Li. 2020. DSP-FWA. Retrieved December 6 2023 from https:\/\/github.com\/yuezunli\/DSP-FWA"},{"key":"e_1_3_3_36_2","article-title":"Exposing deepfake videos by detecting face warping artifacts","author":"Li Yuezun","year":"2018","unstructured":"Yuezun Li and Siwei Lyu. 2018. Exposing deepfake videos by detecting face warping artifacts. arXiv preprint arXiv:1811.00656 (2018).","journal-title":"arXiv preprint arXiv:1811.00656"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00327"},{"key":"e_1_3_3_38_2","article-title":"Towards benchmarking and evaluating deepfake detection","author":"Lin Chenhao","year":"2022","unstructured":"Chenhao Lin, Jingyi Deng, Pengbin Hu, Chao Shen, Qian Wang, and Qi Li. 2022. Towards benchmarking and evaluating deepfake detection. arXiv preprint arXiv:2203.02115 (2022).","journal-title":"arXiv preprint arXiv:2203.02115"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2019.00023"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00481"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2015.425"},{"key":"e_1_3_3_42_2","article-title":"Towards deep learning models resistant to adversarial attacks","author":"Madry Aleksander","year":"2017","unstructured":"Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2017. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083 (2017).","journal-title":"arXiv preprint arXiv:1706.06083"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACVW.2019.00020"},{"key":"e_1_3_3_44_2","doi-asserted-by":"publisher","DOI":"10.1145\/3425780"},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3394171.3413570"},{"key":"e_1_3_3_46_2","article-title":"VoxCeleb: A large-scale speaker identification dataset","author":"Nagrani Arsha","year":"2017","unstructured":"Arsha Nagrani, Joon Son Chung, and Andrew Zisserman. 2017. VoxCeleb: A large-scale speaker identification dataset. arXiv preprint arXiv:1706.08612 (2017).","journal-title":"arXiv preprint arXiv:1706.08612"},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPRW53098.2021.00103"},{"key":"e_1_3_3_48_2","article-title":"Multi-task learning for detecting and segmenting manipulated facial images and videos","author":"Nguyen Huy H.","year":"2019","unstructured":"Huy H. Nguyen, Fuming Fang, Junichi Yamagishi, and Isao Echizen. 2019. Multi-task learning for detecting and segmenting manipulated facial images and videos. arXiv preprint arXiv:1906.06876 (2019).","journal-title":"arXiv preprint arXiv:1906.06876"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2019.8682602"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00728"},{"key":"e_1_3_3_51_2","article-title":"Transferability in machine learning: From phenomena to black-box attacks using adversarial samples","author":"Papernot Nicolas","year":"2016","unstructured":"Nicolas Papernot, Patrick McDaniel, and Ian Goodfellow. 2016. Transferability in machine learning: From phenomena to black-box attacks using adversarial samples. arXiv preprint arXiv:1605.07277 (2016).","journal-title":"arXiv preprint arXiv:1605.07277"},{"key":"e_1_3_3_52_2","doi-asserted-by":"publisher","DOI":"10.1145\/1201775.882269"},{"key":"e_1_3_3_53_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.01036"},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-58610-2_6"},{"key":"e_1_3_3_55_2","doi-asserted-by":"publisher","DOI":"10.1145\/3491199"},{"key":"e_1_3_3_56_2","unstructured":"Valuates Reports. 2020. Facial Recognition Market Size to Reach USD 9.99 Billion by 2025. Retrieved December 6 2023 from https:\/\/www.prnewswire.com\/news-releases\/facial-recognition-market-size-to-reach-usd-9-99-billion-by-2025--valuates-reports-301071952.html"},{"key":"e_1_3_3_57_2","article-title":"FaceForensics: A large-scale video dataset for forgery detection in human faces","author":"R\u00f6ssler Andreas","year":"2018","unstructured":"Andreas R\u00f6ssler, Davide Cozzolino, Luisa Verdoliva, Christian Riess, Justus Thies, and Matthias Nie\u00dfner. 2018. FaceForensics: A large-scale video dataset for forgery detection in human faces. arXiv preprint arXiv:1803.09179 (2018).","journal-title":"arXiv preprint arXiv:1803.09179"},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00009"},{"issue":"1","key":"e_1_3_3_59_2","first-page":"80","article-title":"Recurrent convolutional strategies for face manipulation detection in videos","volume":"3","author":"Sabir Ekraam","year":"2019","unstructured":"Ekraam Sabir, Jiaxin Cheng, Ayush Jaiswal, Wael AbdAlmageed, Iacopo Masi, and Prem Natarajan. 2019. Recurrent convolutional strategies for face manipulation detection in videos. Interfaces (GUI) 3, 1 (2019), 80\u201387.","journal-title":"Interfaces (GUI)"},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298682"},{"key":"e_1_3_3_61_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.74"},{"key":"e_1_3_3_62_2","doi-asserted-by":"publisher","DOI":"10.1145\/3317611"},{"key":"e_1_3_3_63_2","article-title":"First order motion model for image animation","author":"Siarohin Aliaksandr","year":"2019","unstructured":"Aliaksandr Siarohin, St\u00e9phane Lathuili\u00e8re, Sergey Tulyakov, Elisa Ricci, and Nicu Sebe. 2019. First order motion model for image animation. In Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS\u201919). 1\u201311.","journal-title":"Proceedings of the 33rd Conference on Neural Information Processing Systems (NeurIPS\u201919)."},{"key":"e_1_3_3_64_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01344"},{"key":"e_1_3_3_65_2","article-title":"Denoising diffusion implicit models","author":"Song Jiaming","year":"2020","unstructured":"Jiaming Song, Chenlin Meng, and Stefano Ermon. 2020. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502 (2020).","journal-title":"arXiv preprint arXiv:2010.02502"},{"key":"e_1_3_3_66_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00361"},{"key":"e_1_3_3_67_2","article-title":"Intriguing properties of neural networks","author":"Szegedy Christian","year":"2013","unstructured":"Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2013. Intriguing properties of neural networks. arXiv preprint arXiv:1312.6199 (2013).","journal-title":"arXiv preprint arXiv:1312.6199"},{"key":"e_1_3_3_68_2","doi-asserted-by":"publisher","DOI":"10.1145\/2929464.2929475"},{"key":"e_1_3_3_69_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2020.06.014"},{"key":"e_1_3_3_70_2","doi-asserted-by":"publisher","DOI":"10.1109\/WACV45572.2020.9093474"},{"key":"e_1_3_3_71_2","doi-asserted-by":"publisher","DOI":"10.1109\/JSTSP.2020.3002101"},{"key":"e_1_3_3_72_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00872"},{"key":"e_1_3_3_73_2","article-title":"Deepfake video detection using convolutional vision transformer","author":"Wodajo Deressa","year":"2021","unstructured":"Deressa Wodajo and Solomon Atnafu. 2021. Deepfake video detection using convolutional vision transformer. arXiv preprint arXiv:2102.11126 (2021).","journal-title":"arXiv preprint arXiv:2102.11126"},{"key":"e_1_3_3_74_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01267"},{"key":"e_1_3_3_75_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICASSP.2019.8683164"},{"key":"e_1_3_3_76_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2019.00765"},{"key":"e_1_3_3_77_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.00222"},{"key":"e_1_3_3_78_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.244"}],"container-title":["ACM Transactions on Privacy and Security"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3634914","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3634914","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:51:07Z","timestamp":1750287067000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3634914"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,5]]},"references-count":77,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,2,29]]}},"alternative-id":["10.1145\/3634914"],"URL":"https:\/\/doi.org\/10.1145\/3634914","relation":{},"ISSN":["2471-2566","2471-2574"],"issn-type":[{"value":"2471-2566","type":"print"},{"value":"2471-2574","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,5]]},"assertion":[{"value":"2023-05-15","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-11-17","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-05","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}