{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,19]],"date-time":"2025-12-19T09:59:54Z","timestamp":1766138394652,"version":"3.41.0"},"reference-count":77,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2024,2,23]],"date-time":"2024-02-23T00:00:00Z","timestamp":1708646400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["62250710682"],"award-info":[{"award-number":["62250710682"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Guangdong Provincial Key Laboratory","award":["2020B121201001"],"award-info":[{"award-number":["2020B121201001"]}]},{"name":"Program for Guangdong Introducing Innovative and Entrepreneurial Teams","award":["2017ZT07X386"],"award-info":[{"award-number":["2017ZT07X386"]}]},{"name":"Science and Technology Innovation Committee Foundation of Shenzhen","award":["ZDSYS201703031748284"],"award-info":[{"award-number":["ZDSYS201703031748284"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Evol. Learn. Optim."],"published-print":{"date-parts":[[2024,3,31]]},"abstract":"<jats:p>The feature attribution-based explanation (FAE) methods, which indicate how much each input feature contributes to the model\u2019s output for a given data point, are one of the most popular categories of explainable machine learning techniques. Although various metrics have been proposed to evaluate the explanation quality, no single metric could capture different aspects of the explanations. Different conclusions might be drawn using different metrics. Moreover, during the processes of generating explanations, existing FAE methods either do not consider any evaluation metric or only consider the faithfulness of the explanation, failing to consider multiple metrics simultaneously. To address this issue, we formulate the problem of creating FAE explainable models as a multi-objective learning problem that considers multiple explanation quality metrics simultaneously. We first reveal conflicts between various explanation quality metrics, including faithfulness, sensitivity, and complexity. Then, we define the considered multi-objective explanation problem and propose a multi-objective feature attribution explanation (MOFAE) framework to address this newly defined problem. Subsequently, we instantiate the framework by simultaneously considering the explanation\u2019s faithfulness, sensitivity, and complexity. Experimental results comparing with six state-of-the-art FAE methods on eight datasets demonstrate that our method can optimize multiple conflicting metrics simultaneously and can provide explanations with higher faithfulness, lower sensitivity, and lower complexity than the compared methods. Moreover, the results have shown that our method has better diversity, i.e., it provides various explanations that achieve different tradeoffs between multiple conflicting explanation quality metrics. Therefore, it can provide tailored explanations to different stakeholders based on their specific requirements.<\/jats:p>","DOI":"10.1145\/3617380","type":"journal-article","created":{"date-parts":[[2023,8,29]],"date-time":"2023-08-29T11:23:03Z","timestamp":1693308183000},"page":"1-32","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":10,"title":["Multi-objective Feature Attribution Explanation For Explainable Machine Learning"],"prefix":"10.1145","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3118-8742","authenticated-orcid":false,"given":"Ziming","family":"Wang","sequence":"first","affiliation":[{"name":"Research Institute of Trustworthy Autonomous Systems, and Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Departmentof Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3685-2822","authenticated-orcid":false,"given":"Changwu","family":"Huang","sequence":"additional","affiliation":[{"name":"Research Institute of Trustworthy Autonomous Systems, and Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Departmentof Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0004-6239-1534","authenticated-orcid":false,"given":"Yun","family":"Li","sequence":"additional","affiliation":[{"name":"The Advanced Cognitive Technology Lab, Huawei Technologies Co., Ltd., Shanghai, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8837-4442","authenticated-orcid":false,"given":"Xin","family":"Yao","sequence":"additional","affiliation":[{"name":"Research Institute of Trustworthy Autonomous Systems, and Guangdong Provincial Key Laboratory of Brain-inspired Intelligent Computation, Department of Computer Science and Engineering, Southern University of Science and Technology, China and School of Computer Science, University of Birmingham, Birmingham, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,2,23]]},"reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.artint.2021.103502"},{"key":"e_1_3_2_3_1","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2018.2870052"},{"key":"e_1_3_2_4_1","first-page":"9525","article-title":"Sanity checks for saliency maps","volume":"31","author":"Adebayo Julius","year":"2018","unstructured":"Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. 2018. Sanity checks for saliency maps. Adv. Neural Inf. Process. Syst. 31 (2018), 9525\u20139536.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_5_1","first-page":"110","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Agarwal Sushant","year":"2021","unstructured":"Sushant Agarwal, Shahin Jabbari, Chirag Agarwal, Sohini Upadhyay, Steven Wu, and Himabindu Lakkaraju. 2021. Towards the unification and robustness of perturbation and gradient based explanations. In Proceedings of the International Conference on Machine Learning. PMLR, 110\u2013119."},{"key":"e_1_3_2_6_1","article-title":"Can explainable AI explain unfairness? A framework for evaluating explainable AI","author":"Alikhademi Kiana","year":"2021","unstructured":"Kiana Alikhademi, Brianna Richardson, Emma Drobina, and Juan E. Gilbert. 2021. Can explainable AI explain unfairness? A framework for evaluating explainable AI. arXiv preprint arXiv:2106.07483 (2021).","journal-title":"arXiv preprint arXiv:2106.07483"},{"key":"e_1_3_2_7_1","first-page":"7786","article-title":"Towards robust interpretability with self-explaining neural networks","volume":"31","author":"Melis David Alvarez","year":"2018","unstructured":"David Alvarez Melis and Tommi Jaakkola. 2018. Towards robust interpretability with self-explaining neural networks. Adv. Neural Inf. Process. Syst. 31 (2018), 7786\u20137795.","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_8_1","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1007\/978-3-030-28954-6_9","volume-title":"Explainable AI: Interpreting, Explaining and Visualizing Deep Learning","author":"Ancona Marco","year":"2019","unstructured":"Marco Ancona, Enea Ceolini, Cengiz \u00d6ztireli, and Markus Gross. 2019. Gradient-based attribution methods. In Explainable AI: Interpreting, Explaining and Visualizing Deep Learning. Springer, 169\u2013191."},{"key":"e_1_3_2_9_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.inffus.2019.12.012"},{"key":"e_1_3_2_10_1","first-page":"3016","volume-title":"Proceedings of the 29th International Joint Conference on Artificial Intelligence","author":"Bhatt Umang","year":"2021","unstructured":"Umang Bhatt, Adrian Weller, and Jos\u00e9 M. F. Moura. 2021. Evaluating and aggregating feature-based model explanations. In Proceedings of the 29th International Joint Conference on Artificial Intelligence. 3016\u20133022."},{"key":"e_1_3_2_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/3351095.3375624"},{"key":"e_1_3_2_12_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10852-005-9020-3"},{"key":"e_1_3_2_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2010.26"},{"key":"e_1_3_2_14_1","doi-asserted-by":"publisher","DOI":"10.3390\/make5010006"},{"key":"e_1_3_2_15_1","unstructured":"Chinese New Generation AI Governance Expert Committee. 2019. Next Generation AI Governance Principles - Developing Responsible AI. Retrieved from https:\/\/www.most.gov.cn\/kjbgz\/201906\/t20190617_147107.html"},{"key":"e_1_3_2_16_1","article-title":"Opportunities and challenges in explainable artificial intelligence (XAI): A survey","author":"Das Arun","year":"2020","unstructured":"Arun Das and Paul Rad. 2020. Opportunities and challenges in explainable artificial intelligence (XAI): A survey. arXiv preprint arXiv:2006.11371 (2020).","journal-title":"arXiv preprint arXiv:2006.11371"},{"key":"e_1_3_2_17_1","article-title":"Explainable AI meets healthcare: A study on heart disease dataset","author":"Dave Devam","year":"2020","unstructured":"Devam Dave, Het Naik, Smiti Singhal, and Pankesh Patel. 2020. Explainable AI meets healthcare: A study on heart disease dataset. arXiv preprint arXiv:2011.03195 (2020).","journal-title":"arXiv preprint arXiv:2011.03195"},{"key":"e_1_3_2_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2013.2281535"},{"key":"e_1_3_2_19_1","article-title":"Towards a rigorous science of interpretable machine learning","author":"Doshi-Velez Finale","year":"2017","unstructured":"Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:1702.08608 (2017).","journal-title":"arXiv preprint arXiv:1702.08608"},{"key":"e_1_3_2_20_1","unstructured":"Dheeru Dua and Casey Graff. 2017. UCI Machine Learning Repository. Retrieved from http:\/\/archive.ics.uci.edu\/ml"},{"key":"e_1_3_2_21_1","doi-asserted-by":"publisher","DOI":"10.1109\/WACV51458.2022.00163"},{"key":"e_1_3_2_22_1","doi-asserted-by":"crossref","unstructured":"Will Fleisher. 2022. Understanding idealization and explainable AI. Episteme 19 4 (2022) 534\u2013560.","DOI":"10.1017\/epi.2022.39"},{"key":"e_1_3_2_23_1","unstructured":"Food and Drug Administration. 2021. Artificial Intelligence and Machine Learning (AI\/ML) Software as a Medical Device Action Plan. Retrieved from https:\/\/www.fda.gov\/medical-devices\/software-medical-device-samd\/artificial-intelligence-and-machine-learning-software-medical-device"},{"key":"e_1_3_2_24_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-83620-7_7"},{"key":"e_1_3_2_25_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33013681"},{"key":"e_1_3_2_26_1","doi-asserted-by":"publisher","DOI":"10.1109\/DSAA.2018.00018"},{"key":"e_1_3_2_27_1","first-page":"60","volume-title":"Proceedings of the 18th International Conference on Artificial Intelligence and Law","author":"G\u00f3rski \u0141ukasz","year":"2021","unstructured":"\u0141ukasz G\u00f3rski and Shashishekar Ramakrishna. 2021. Explainable artificial intelligence, lawyer\u2019s perspective. In Proceedings of the 18th International Conference on Artificial Intelligence and Law. 60\u201368."},{"key":"e_1_3_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3236009"},{"key":"e_1_3_2_29_1","unstructured":"Patrick Hall Navdeep Gill Megan Kurka and Wen Phan. 2017. Machine learning interpretability with H2O driverless AI. H2O. AI."},{"key":"e_1_3_2_30_1","article-title":"The promise and peril of human evaluation for model interpretability","author":"Herman Bernease","year":"2017","unstructured":"Bernease Herman. 2017. The promise and peril of human evaluation for model interpretability. arXiv preprint arXiv: 1711.07414 (2017).","journal-title":"arXiv preprint arXiv: 1711.07414"},{"key":"e_1_3_2_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TAI.2022.3194503"},{"key":"e_1_3_2_32_1","volume-title":"Ethics Guidelines for Trustworthy AI","author":"Intelligence European Commission High-level Expert Group Artificial","year":"2019","unstructured":"European Commission High-level Expert Group Artificial Intelligence. 2019. Ethics Guidelines for Trustworthy AI. Publications Office."},{"key":"e_1_3_2_33_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.386"},{"key":"e_1_3_2_34_1","doi-asserted-by":"publisher","DOI":"10.3905\/jfds.2021.1.066"},{"key":"e_1_3_2_35_1","article-title":"On two XAI cultures: A case study of non-technical explanations in deployed AI system","author":"Jiang Helen","year":"2021","unstructured":"Helen Jiang and Erwen Senge. 2021. On two XAI cultures: A case study of non-technical explanations in deployed AI system. arXiv preprint arXiv:2112.01016 (2021).","journal-title":"arXiv preprint arXiv:2112.01016"},{"key":"e_1_3_2_36_1","volume-title":"Explainable Artificial Intelligence: An Introduction to Interpretable Machine Learning","author":"Kamath Uday","year":"2021","unstructured":"Uday Kamath and John Liu. 2021. Explainable Artificial Intelligence: An Introduction to Interpretable Machine Learning. Springer."},{"key":"e_1_3_2_37_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41568-020-00327-9"},{"key":"e_1_3_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/2792984"},{"key":"e_1_3_2_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2016.2549267"},{"key":"e_1_3_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/3300148"},{"key":"e_1_3_2_41_1","article-title":"Human-centered explainable AI (XAI): From algorithms to user experiences","author":"Liao Q. Vera","year":"2021","unstructured":"Q. Vera Liao and Kush R. Varshney. 2021. Human-centered explainable AI (XAI): From algorithms to user experiences. arXiv preprint arXiv:2110.10790 (2021).","journal-title":"arXiv preprint arXiv:2110.10790"},{"key":"e_1_3_2_42_1","doi-asserted-by":"publisher","DOI":"10.5555\/3295222.3295230"},{"key":"e_1_3_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467265"},{"key":"e_1_3_2_44_1","doi-asserted-by":"publisher","DOI":"10.1145\/2522920.2522928"},{"key":"e_1_3_2_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3387166"},{"key":"e_1_3_2_46_1","volume-title":"Interpretable Machine Learning","author":"Molnar Christoph","year":"2020","unstructured":"Christoph Molnar. 2020. Interpretable Machine Learning. Lulu.com."},{"key":"e_1_3_2_47_1","article-title":"On quantitative aspects of model interpretability","author":"Nguyen An-phi","year":"2020","unstructured":"An-phi Nguyen and Mar\u00eda Rodr\u00edguez Mart\u00ednez. 2020. On quantitative aspects of model interpretability. arXiv preprint arXiv:2007.07584 (2020).","journal-title":"arXiv preprint arXiv:2007.07584"},{"key":"e_1_3_2_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-82017-6_12"},{"key":"e_1_3_2_49_1","article-title":"Rise: Randomized input sampling for explanation of black-box models","author":"Petsiuk Vitali","year":"2018","unstructured":"Vitali Petsiuk, Abir Das, and Kate Saenko. 2018. Rise: Randomized input sampling for explanation of black-box models. arXiv preprint arXiv:1806.07421 (2018).","journal-title":"arXiv preprint arXiv:1806.07421"},{"key":"e_1_3_2_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3411764.3445315"},{"key":"e_1_3_2_51_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2021.3098962"},{"key":"e_1_3_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/2939672.2939778"},{"key":"e_1_3_2_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10458-019-09408-y"},{"key":"e_1_3_2_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2016.2599820"},{"key":"e_1_3_2_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.74"},{"key":"e_1_3_2_56_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-97929-4_3"},{"key":"e_1_3_2_57_1","article-title":"Not just a black box: Learning important features through propagating activation differences","author":"Shrikumar Avanti","year":"2016","unstructured":"Avanti Shrikumar, Peyton Greenside, Anna Shcherbina, and Anshul Kundaje. 2016. Not just a black box: Learning important features through propagating activation differences. arXiv preprint arXiv:1605.01713 (2016).","journal-title":"arXiv preprint arXiv:1605.01713"},{"key":"e_1_3_2_58_1","volume-title":"Proceedings of the Workshop at International Conference on Learning Representations","author":"Simonyan Karen","year":"2014","unstructured":"Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. 2014. Deep inside convolutional networks: Visualising image classification models and saliency maps. In Proceedings of the Workshop at International Conference on Learning Representations. Citeseer."},{"key":"e_1_3_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/3375627.3375830"},{"key":"e_1_3_2_60_1","article-title":"SmoothGrad: Removing noise by adding noise","author":"Smilkov Daniel","year":"2017","unstructured":"Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Vi\u00e9gas, and Martin Wattenberg. 2017. SmoothGrad: Removing noise by adding noise. arXiv preprint arXiv:1706.03825 (2017).","journal-title":"arXiv preprint arXiv:1706.03825"},{"key":"e_1_3_2_61_1","doi-asserted-by":"publisher","DOI":"10.1006\/jeem.1993.1004"},{"key":"e_1_3_2_62_1","first-page":"3319","volume-title":"Proceedings of the International Conference on Machine Learning","author":"Sundararajan Mukund","year":"2017","unstructured":"Mukund Sundararajan, Ankur Taly, and Qiqi Yan. 2017. Axiomatic attribution for deep networks. In Proceedings of the International Conference on Machine Learning. 3319\u20133328."},{"key":"e_1_3_2_63_1","volume-title":"Interpretable AI","author":"Thampi Ajay","year":"2022","unstructured":"Ajay Thampi. 2022. Interpretable AI. Simon and Schuster."},{"issue":"8","key":"e_1_3_2_64_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3470971","article-title":"Evolutionary large-scale multi-objective optimization: A survey","volume":"54","author":"Tian Ye","year":"2021","unstructured":"Ye Tian, Langchun Si, Xingyi Zhang, Ran Cheng, Cheng He, Kay Chen Tan, and Yaochu Jin. 2021. Evolutionary large-scale multi-objective optimization: A survey. ACM Comput. Surv. 54, 8 (2021), 1\u201334.","journal-title":"ACM Comput. Surv."},{"key":"e_1_3_2_65_1","unstructured":"European Union. 2018. General Data Protection Regulation (GDPR). Retrieved from https:\/\/gdpr-info.eu\/"},{"key":"e_1_3_2_66_1","first-page":"714","volume-title":"Proceedings of the Intelligent Systems Conference (IntelliSys\u201917)","author":"Vishnukumar Harsha Jakkanahalli","year":"2017","unstructured":"Harsha Jakkanahalli Vishnukumar, Bj\u00f6rn Butting, Christian M\u00fcller, and Eric Sax. 2017. Machine learning and deep neural network\u2014Artificial intelligence core for lab and real-world test and validation for ADAS and autonomous vehicles: AI for efficient and quality test and validation. In Proceedings of the Intelligent Systems Conference (IntelliSys\u201917). 714\u2013721."},{"key":"e_1_3_2_67_1","doi-asserted-by":"publisher","DOI":"10.1109\/IJCNN54540.2023.10191221"},{"key":"e_1_3_2_68_1","article-title":"On the (in) fidelity and sensitivity of explanations","volume":"32","author":"Yeh Chih-Kuan","year":"2019","unstructured":"Chih-Kuan Yeh, Cheng-Yu Hsieh, Arun Suggala, David I. Inouye, and Pradeep K. Ravikumar. 2019. On the (in) fidelity and sensitivity of explanations. Adv. Neural Inf. Process. Syst. 32 (2019).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"e_1_3_2_69_1","article-title":"DLIME: A deterministic local interpretable model-agnostic explanations approach for computer-aided diagnosis systems","author":"Zafar Muhammad Rehman","year":"2019","unstructured":"Muhammad Rehman Zafar and Naimul Mefraz Khan. 2019. DLIME: A deterministic local interpretable model-agnostic explanations approach for computer-aided diagnosis systems. arXiv preprint arXiv:1906.10263 (2019).","journal-title":"arXiv preprint arXiv:1906.10263"},{"key":"e_1_3_2_70_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.cam.2019.112530"},{"key":"e_1_3_2_71_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2007.892759"},{"key":"e_1_3_2_72_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-86380-7_10"},{"key":"e_1_3_2_73_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2022.3209544"},{"key":"e_1_3_2_74_1","doi-asserted-by":"publisher","DOI":"10.1109\/TEVC.2014.2378512"},{"key":"e_1_3_2_75_1","doi-asserted-by":"publisher","DOI":"10.1109\/TETCI.2021.3100641"},{"key":"e_1_3_2_76_1","first-page":"887","volume-title":"Uncertainty in Artificial Intelligence","author":"Zhao Xingyu","year":"2021","unstructured":"Xingyu Zhao, Wei Huang, Xiaowei Huang, Valentin Robu, and David Flynn. 2021. BayLIME: Bayesian local interpretable model-agnostic explanations. In Uncertainty in Artificial Intelligence. PMLR, 887\u2013896."},{"key":"e_1_3_2_77_1","doi-asserted-by":"publisher","DOI":"10.1162\/106365600568202"},{"key":"e_1_3_2_78_1","doi-asserted-by":"publisher","DOI":"10.1007\/BFb0056872"}],"container-title":["ACM Transactions on Evolutionary Learning and Optimization"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3617380","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3617380","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:45:54Z","timestamp":1750178754000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3617380"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,23]]},"references-count":77,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2024,3,31]]}},"alternative-id":["10.1145\/3617380"],"URL":"https:\/\/doi.org\/10.1145\/3617380","relation":{},"ISSN":["2688-3007"],"issn-type":[{"type":"electronic","value":"2688-3007"}],"subject":[],"published":{"date-parts":[[2024,2,23]]},"assertion":[{"value":"2022-09-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-08-14","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-23","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}