{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,9]],"date-time":"2026-01-09T02:08:44Z","timestamp":1767924524659,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":76,"publisher":"ACM","license":[{"start":{"date-parts":[[2024,8,24]],"date-time":"2024-08-24T00:00:00Z","timestamp":1724457600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,8,25]]},"DOI":"10.1145\/3637528.3671883","type":"proceedings-article","created":{"date-parts":[[2024,8,25]],"date-time":"2024-08-25T04:55:12Z","timestamp":1724561712000},"page":"2307-2317","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Reliable Confidence Intervals for Information Retrieval Evaluation Using Generative A.I."],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0458-9233","authenticated-orcid":false,"given":"Harrie","family":"Oosterhuis","sequence":"first","affiliation":[{"name":"Google Research & Radboud University, Amsterdam, Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5169-495X","authenticated-orcid":false,"given":"Rolf","family":"Jagerman","sequence":"additional","affiliation":[{"name":"Google Research, Amsterdam, Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6739-134X","authenticated-orcid":false,"given":"Zhen","family":"Qin","sequence":"additional","affiliation":[{"name":"Google Research, New York City, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-1388-1423","authenticated-orcid":false,"given":"Xuanhui","family":"Wang","sequence":"additional","affiliation":[{"name":"Google Research, Mountain View, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2941-6240","authenticated-orcid":false,"given":"Michael","family":"Bendersky","sequence":"additional","affiliation":[{"name":"Google Research, Mountain View, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,8,24]]},"reference":[{"key":"e_1_3_2_2_1_1","volume-title":"Maria Korobeynikova, and Fabrizio Gilardi.","author":"Alizadeh Meysam","year":"2023","unstructured":"Meysam Alizadeh, Ma\u00ebl Kubli, Zeynab Samei, Shirin Dehghani, Juan Diego Bermeo, Maria Korobeynikova, and Fabrizio Gilardi. 2023. Open-source large language models outperform crowd workers and approach ChatGPT in text-annotation tasks. arXiv preprint arXiv:2307.02179 (2023)."},{"key":"e_1_3_2_2_2_1","volume-title":"A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511","author":"Angelopoulos Anastasios N","year":"2021","unstructured":"Anastasios N Angelopoulos and Stephen Bates. 2021. A gentle introduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511 (2021)."},{"key":"e_1_3_2_2_3_1","volume-title":"Prediction-powered inference. arXiv preprint arXiv:2301.09633","author":"Angelopoulos Anastasios N","year":"2023","unstructured":"Anastasios N Angelopoulos, Stephen Bates, Clara Fannjiang, Michael I Jordan, and Tijana Zrnic. 2023. Prediction-powered inference. arXiv preprint arXiv:2301.09633 (2023)."},{"key":"e_1_3_2_2_4_1","volume-title":"Conformal risk control. arXiv preprint arXiv:2208.02814","author":"Angelopoulos Anastasios N","year":"2022","unstructured":"Anastasios N Angelopoulos, Stephen Bates, Adam Fisch, Lihua Lei, and Tal Schuster. 2022. Conformal risk control. arXiv preprint arXiv:2208.02814 (2022)."},{"key":"e_1_3_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148263"},{"key":"e_1_3_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390447"},{"key":"e_1_3_2_2_7_1","unstructured":"Vineeth Balasubramanian Shen-Shyang Ho and Vladimir Vovk. 2014. Conformal prediction for reliable machine learning: theory adaptations and applications. Newnes."},{"key":"e_1_3_2_2_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3442188.3445922"},{"key":"e_1_3_2_2_9_1","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531863"},{"key":"e_1_3_2_2_10_1","doi-asserted-by":"crossref","unstructured":"F Jay Breidt and Jean D Opsomer. 2017. Model-assisted survey estimation with modern prediction techniques. (2017).","DOI":"10.1214\/16-STS589"},{"key":"e_1_3_2_2_11_1","unstructured":"Tom Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared D Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell et al. 2020. Language models are few-shot learners. Advances in neural information processing systems Vol. 33 (2020) 1877--1901."},{"key":"e_1_3_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.5555\/184656.180369"},{"key":"e_1_3_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148219"},{"key":"e_1_3_2_2_14_1","volume-title":"Proceedings of the learning to rank challenge. PMLR, 1--24","author":"Chapelle Olivier","year":"2011","unstructured":"Olivier Chapelle and Yi Chang. 2011. Yahoo! learning to rank challenge overview. In Proceedings of the learning to rank challenge. PMLR, 1--24."},{"key":"e_1_3_2_2_15_1","volume-title":"HMC: A Spectrum of Human--Machine-Collaborative Relevance Judgment Frameworks. Frontiers of Information Access Experimentation for Research and Education","author":"Clarke Charles LA","year":"2023","unstructured":"Charles LA Clarke, Gianluca Demartini, Laura Dietz, Guglielmo Faggioli, Matthias Hagen, Claudia Hauff, Noriko Kando, Evangelos Kanoulas, Martin Potthast, Ian Soboroff, et al. 2023. 4.2 HMC: A Spectrum of Human--Machine-Collaborative Relevance Judgment Frameworks. Frontiers of Information Access Experimentation for Research and Education (2023), 41."},{"key":"e_1_3_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1148170.1148262"},{"key":"e_1_3_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3404835.3463249"},{"key":"e_1_3_2_2_18_1","volume-title":"Generative adversarial networks: An overview","author":"Creswell Antonia","year":"2018","unstructured":"Antonia Creswell, Tom White, Vincent Dumoulin, Kai Arulkumaran, Biswa Sengupta, and Anil A Bharath. 2018. Generative adversarial networks: An overview. IEEE signal processing magazine, Vol. 35, 1 (2018), 53--65."},{"key":"e_1_3_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASRU.2015.7404803"},{"key":"e_1_3_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1145\/3477495.3531740"},{"key":"e_1_3_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-015-9275-x"},{"key":"e_1_3_2_2_22_1","volume-title":"Bootstrap confidence intervals. Statistical science","author":"DiCiccio Thomas J","year":"1996","unstructured":"Thomas J DiCiccio and Bradley Efron. 1996. Bootstrap confidence intervals. Statistical science, Vol. 11, 3 (1996), 189--228."},{"key":"e_1_3_2_2_23_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.2517-6161.1988.tb01732.x"},{"key":"e_1_3_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1987.10478410"},{"key":"e_1_3_2_2_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3578337.3605136"},{"key":"e_1_3_2_2_26_1","volume-title":"Acm sigir forum","author":"Fuhr Norbert","unstructured":"Norbert Fuhr. 2018. Some common mistakes in IR evaluation, and how they can be avoided. In Acm sigir forum, Vol. 51. ACM New York, NY, USA, 32--41."},{"key":"e_1_3_2_2_27_1","volume-title":"Chatgpt outperforms crowd-workers for text-annotation tasks. arXiv preprint arXiv:2303.15056","author":"Gilardi Fabrizio","year":"2023","unstructured":"Fabrizio Gilardi, Meysam Alizadeh, and Ma\u00ebl Kubli. 2023. Chatgpt outperforms crowd-workers for text-annotation tasks. arXiv preprint arXiv:2303.15056 (2023)."},{"key":"e_1_3_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3422622"},{"key":"e_1_3_2_2_29_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2019.102067"},{"key":"e_1_3_2_2_30_1","volume-title":"Information retrieval evaluation","author":"Harman Donna","unstructured":"Donna Harman. 2011. Information retrieval evaluation. Morgan & Claypool Publishers."},{"key":"e_1_3_2_2_31_1","unstructured":"Donna K Harman. 2005. The TREC test collections. (2005)."},{"key":"e_1_3_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1002\/wics.182"},{"key":"e_1_3_2_2_33_1","doi-asserted-by":"publisher","DOI":"10.1197\/jamia.M1733"},{"key":"e_1_3_2_2_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3539597.3570468"},{"key":"e_1_3_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1145\/3130348.3130374"},{"key":"e_1_3_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2022.3192720"},{"key":"e_1_3_2_2_38_1","volume-title":"Proceedings of the first NTCIR workshop on research in Japanese text retrieval and term recognition. 11--44","author":"Kando Noriko","year":"1999","unstructured":"Noriko Kando, Kazuko Kuriyama, Toshihiko Nozue, Koji Eguchi, Hiroyuki Kato, and Souichiro Hidaka. 1999. Overview of IR tasks at the first NTCIR workshop. In Proceedings of the first NTCIR workshop on research in Japanese text retrieval and term recognition. 11--44."},{"key":"e_1_3_2_2_39_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.10137"},{"key":"e_1_3_2_2_40_1","volume-title":"Relevance assessments and retrieval system evaluation. Information storage and retrieval","author":"Lesk Michael E","year":"1968","unstructured":"Michael E Lesk and Gerard Salton. 1968. Relevance assessments and retrieval system evaluation. Information storage and retrieval, Vol. 4, 4 (1968), 343--359."},{"key":"e_1_3_2_2_41_1","volume-title":"Generative artificial intelligence and its applications in materials science: Current situation and future perspectives. Journal of Materiomics","author":"Liu Yue","year":"2023","unstructured":"Yue Liu, Zhengwei Yang, Zhenyao Yu, Zitu Liu, Dahui Liu, Hailong Lin, Mingqing Li, Shuchang Ma, Maxim Avdeev, and Siqi Shi. 2023. Generative artificial intelligence and its applications in materials science: Current situation and future perspectives. Journal of Materiomics (2023)."},{"key":"e_1_3_2_2_42_1","volume-title":"Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al.","author":"Longpre Shayne","year":"2023","unstructured":"Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. 2023. The flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688 (2023)."},{"key":"e_1_3_2_2_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3209978.3210048"},{"key":"e_1_3_2_2_44_1","volume-title":"One-Shot Labeling for Automatic Relevance Estimation. arXiv preprint arXiv:2302.11266","author":"MacAvaney Sean","year":"2023","unstructured":"Sean MacAvaney and Luca Soldaini. 2023. One-Shot Labeling for Automatic Relevance Estimation. arXiv preprint arXiv:2302.11266 (2023)."},{"key":"e_1_3_2_2_45_1","volume-title":"Neural models for information retrieval. arXiv preprint arXiv:1705.01509","author":"Mitra Bhaskar","year":"2017","unstructured":"Bhaskar Mitra and Nick Craswell. 2017. Neural models for information retrieval. arXiv preprint arXiv:1705.01509 (2017)."},{"key":"e_1_3_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1561\/9781680835335"},{"key":"e_1_3_2_2_47_1","volume-title":"MS MARCO: A human generated machine reading comprehension dataset. choice","author":"Nguyen Tri","year":"2016","unstructured":"Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. MS MARCO: A human generated machine reading comprehension dataset. choice, Vol. 2640 (2016), 660."},{"key":"e_1_3_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-017-9321-y"},{"key":"e_1_3_2_2_49_1","first-page":"27730","article-title":"Training language models to follow instructions with human feedback","volume":"35","author":"Ouyang Long","year":"2022","unstructured":"Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, Vol. 35 (2022), 27730--27744.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_2_50_1","doi-asserted-by":"crossref","unstructured":"Harris Papadopoulos. 2008. Inductive conformal prediction: Theory and application to neural networks. In Tools in artificial intelligence. Citeseer.","DOI":"10.5772\/6078"},{"key":"e_1_3_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1177\/10776958221149577"},{"key":"e_1_3_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1007\/3-540-44645-1"},{"key":"e_1_3_2_2_53_1","volume-title":"arXiv preprint arXiv:1306.2597","author":"Qin Tao","year":"2013","unstructured":"Tao Qin and Tie-Yan Liu. 2013. Introducing LETOR 4.0 datasets. arXiv preprint arXiv:1306.2597 (2013)."},{"key":"e_1_3_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10791-009-9123-y"},{"key":"e_1_3_2_2_55_1","doi-asserted-by":"crossref","unstructured":"Zhen Qin Rolf Jagerman Kai Hui Honglei Zhuang Junru Wu Jiaming Shen Tianqi Liu Jialu Liu Donald Metzler Xuanhui Wang et al. 2023. Large language models are effective text rankers with pairwise ranking prompting. arXiv preprint arXiv:2306.17563 (2023).","DOI":"10.18653\/v1\/2024.findings-naacl.97"},{"key":"e_1_3_2_2_56_1","doi-asserted-by":"publisher","DOI":"10.1145\/2641383.2641385"},{"key":"e_1_3_2_2_57_1","volume-title":"Foundations and Trends\u00ae in Information Retrieval","volume":"4","author":"Mark","year":"2010","unstructured":"Mark Sanderson et al. 2010. Test collection based evaluation of information retrieval systems. Foundations and Trends\u00ae in Information Retrieval, Vol. 4, 4 (2010), 247--375."},{"key":"e_1_3_2_2_58_1","doi-asserted-by":"publisher","DOI":"10.1145\/1076034.1076064"},{"key":"e_1_3_2_2_59_1","doi-asserted-by":"publisher","DOI":"10.1145\/1321440.1321528"},{"key":"e_1_3_2_2_60_1","volume-title":"Neil Houlsby, and Donald Metzler.","author":"Tay Yi","year":"2022","unstructured":"Yi Tay, Mostafa Dehghani, Vinh Q Tran, Xavier Garcia, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Neil Houlsby, and Donald Metzler. 2022. Unifying language learning paradigms. arXiv preprint arXiv:2205.05131 (2022)."},{"key":"e_1_3_2_2_61_1","unstructured":"Gemini Team Rohan Anil Sebastian Borgeaud Yonghui Wu Jean-Baptiste Alayrac Jiahui Yu Radu Soricut Johan Schalkwyk Andrew M Dai Anja Hauth et al. 2023. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023)."},{"key":"e_1_3_2_2_62_1","volume-title":"Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models. arXiv preprint arXiv:2104.08663","author":"Thakur Nandan","year":"2021","unstructured":"Nandan Thakur, Nils Reimers, Andreas R\u00fcckl\u00e9, Abhishek Srivastava, and Iryna Gurevych. 2021. Beir: A heterogenous benchmark for zero-shot evaluation of information retrieval models. arXiv preprint arXiv:2104.08663 (2021)."},{"key":"e_1_3_2_2_63_1","volume-title":"Large language models can accurately predict searcher preferences. arXiv preprint arXiv:2309.10621","author":"Thomas Paul","year":"2023","unstructured":"Paul Thomas, Seth Spielman, Nick Craswell, and Bhaskar Mitra. 2023. Large language models can accurately predict searcher preferences. arXiv preprint arXiv:2309.10621 (2023)."},{"key":"e_1_3_2_2_64_1","volume-title":"Chatgpt-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning. arXiv preprint arXiv:2304.06588","author":"T\u00f6rnberg Petter","year":"2023","unstructured":"Petter T\u00f6rnberg. 2023. Chatgpt-4 outperforms experts and crowd workers in annotating political twitter messages with zero-shot learning. arXiv preprint arXiv:2304.06588 (2023)."},{"key":"e_1_3_2_2_65_1","doi-asserted-by":"crossref","unstructured":"George Tsatsaronis Georgios Balikas Prodromos Malakasiotis Ioannis Partalas Matthias Zschunke Michael R Alvers Dirk Weissenborn Anastasia Krithara Sergios Petridis Dimitris Polychronopoulos et al. 2015. An overview of the BIOASQ large-scale biomedical semantic indexing and question answering competition. BMC bioinformatics Vol. 16 1 (2015) 1--28.","DOI":"10.1186\/s12859-015-0564-6"},{"key":"e_1_3_2_2_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331259"},{"key":"e_1_3_2_2_67_1","doi-asserted-by":"publisher","DOI":"10.1016\/0306-4573(75)90006-0"},{"key":"e_1_3_2_2_68_1","volume-title":"ACM SIGIR Forum","volume":"54","author":"Voorhees Ellen","year":"2021","unstructured":"Ellen Voorhees, Tasmeer Alam, Steven Bedrick, Dina Demner-Fushman, William R Hersh, Kyle Lo, Kirk Roberts, Ian Soboroff, and Lucy Lu Wang. 2021. TREC-COVID: constructing a pandemic information retrieval test collection. In ACM SIGIR Forum, Vol. 54. ACM New York, NY, USA, 1--12."},{"key":"e_1_3_2_2_69_1","volume-title":"The evolution of cranfield. Information Retrieval Evaluation in a Changing World: Lessons Learned from 20 Years of CLEF","author":"Voorhees Ellen M","year":"2019","unstructured":"Ellen M Voorhees. 2019. The evolution of cranfield. Information Retrieval Evaluation in a Changing World: Lessons Learned from 20 Years of CLEF (2019), 45--69."},{"key":"e_1_3_2_2_70_1","volume-title":"Overview of the TREC 2003 robust retrieval track.. In Trec. 69--77","author":"Ellen","unstructured":"Ellen M Voorhees et al. 2003. Overview of the TREC 2003 robust retrieval track.. In Trec. 69--77."},{"key":"e_1_3_2_2_71_1","volume-title":"TREC: Experiment and evaluation in information retrieval.","author":"Voorhees Ellen M","year":"2005","unstructured":"Ellen M Voorhees, Donna K Harman, et al. 2005. TREC: Experiment and evaluation in information retrieval. Vol. 63. MIT press Cambridge."},{"key":"e_1_3_2_2_72_1","doi-asserted-by":"publisher","DOI":"10.1145\/2414782.2414784"},{"key":"e_1_3_2_2_73_1","volume-title":"Measurement in information retrieval evaluation. Ph.,D. Dissertation","author":"Webber William Edward","unstructured":"William Edward Webber. 2010. Measurement in information retrieval evaluation. Ph.,D. Dissertation. University of Melbourne, Department of Computer Science and Software Engineering."},{"key":"e_1_3_2_2_74_1","volume-title":"Proceedings of Machine Translation Summit XVII: Research Track. 12--20","author":"Yarmohammadi Mahsa","year":"2019","unstructured":"Mahsa Yarmohammadi, Xutai Ma, Sorami Hisamoto, Muhammad Rahman, Yiming Wang, Hainan Xu, Daniel Povey, Philipp Koehn, and Kevin Duh. 2019. Robust document representations for cross-lingual information retrieval in low-resource settings. In Proceedings of Machine Translation Summit XVII: Research Track. 12--20."},{"key":"e_1_3_2_2_75_1","doi-asserted-by":"publisher","DOI":"10.1145\/1390334.1390437"},{"key":"e_1_3_2_2_76_1","volume-title":"Beyond yes and no: Improving zero-shot llm rankers via scoring fine-grained relevance labels. arXiv preprint arXiv:2310.14122","author":"Zhuang Honglei","year":"2023","unstructured":"Honglei Zhuang, Zhen Qin, Kai Hui, Junru Wu, Le Yan, Xuanhui Wang, and Michael Berdersky. 2023. Beyond yes and no: Improving zero-shot llm rankers via scoring fine-grained relevance labels. arXiv preprint arXiv:2310.14122 (2023)."}],"event":{"name":"KDD '24: The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","location":"Barcelona Spain","acronym":"KDD '24","sponsor":["SIGMOD ACM Special Interest Group on Management of Data","SIGKDD ACM Special Interest Group on Knowledge Discovery in Data"]},"container-title":["Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3637528.3671883","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3637528.3671883","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:04:15Z","timestamp":1750291455000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3637528.3671883"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,8,24]]},"references-count":76,"alternative-id":["10.1145\/3637528.3671883","10.1145\/3637528"],"URL":"https:\/\/doi.org\/10.1145\/3637528.3671883","relation":{},"subject":[],"published":{"date-parts":[[2024,8,24]]},"assertion":[{"value":"2024-08-24","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}