{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T19:39:02Z","timestamp":1785526742034,"version":"3.56.0"},"reference-count":50,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,11,28]],"date-time":"2024-11-28T00:00:00Z","timestamp":1732752000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"Algorithms, Data, and Democracy project"},{"name":"Villum Foundation and Velux Foundation"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Recomm. Syst."],"published-print":{"date-parts":[[2025,6,30]]},"abstract":"<jats:p>Fairness is an emerging and challenging topic in recommender systems. In recent years, various ways of evaluating and therefore improving fairness have emerged. In this study, we examine existing evaluation measures of fairness in recommender systems. Specifically, we focus solely on exposure-based fairness measures of individual items that aim at quantifying the disparity in how individual items are recommended to users, separate from item relevance to users. We gather all such measures and we critically analyse their theoretical properties. We identify a series of limitations in each of them, which collectively may render the affected measures hard or impossible to interpret, to compute, or to use for comparing recommendations. We resolve these limitations by redefining or correcting the affected measures, or we argue why certain limitations cannot be resolved. We further perform a comprehensive empirical analysis of both the original and our corrected versions of these fairness measures, using real-world and synthetic datasets. Our analysis provides novel insights into the relationship between measures based on different fairness concepts, and different levels of measure sensitivity and strictness. We conclude with practical suggestions of which fairness measures should be used and when. Our code is publicly available. To our knowledge, this is the first critical comparison of individual item fairness measures in recommender systems.<\/jats:p>","DOI":"10.1145\/3631943","type":"journal-article","created":{"date-parts":[[2023,11,9]],"date-time":"2023-11-09T11:45:34Z","timestamp":1699530334000},"page":"1-52","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":16,"title":["Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study"],"prefix":"10.1145","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1233-7690","authenticated-orcid":false,"given":"Theresia Veronika","family":"Rampisela","sequence":"first","affiliation":[{"name":"University of Copenhagen, Copenhagen, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7001-4817","authenticated-orcid":false,"given":"Maria","family":"Maistro","sequence":"additional","affiliation":[{"name":"University of Copenhagen, Copenhagen, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2203-4928","authenticated-orcid":false,"given":"Tuukka","family":"Ruotsalo","sequence":"additional","affiliation":[{"name":"University of Copenhagen, Copenhagen, Denmark and LUT University, Lappeenranta, Finland"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2600-2701","authenticated-orcid":false,"given":"Christina","family":"Lioma","sequence":"additional","affiliation":[{"name":"University of Copenhagen, Copenhagen, Denmark"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2024,11,28]]},"reference":[{"issue":"6","key":"e_1_3_4_2_2","doi-asserted-by":"crossref","first-page":"865","DOI":"10.2307\/2094626","article-title":"Measures of inequality","volume":"43","author":"Allison Paul D.","year":"1978","unstructured":"Paul D. Allison. 1978. Measures of inequality. American Sociological Review 43, 6 (1978), 865\u2013880.","journal-title":"American Sociological Review"},{"key":"e_1_3_4_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/J.IPM.2022.103115"},{"key":"e_1_3_4_4_2","doi-asserted-by":"publisher","DOI":"10.1111\/J.2517-6161.1995.TB02031.X"},{"key":"e_1_3_4_5_2","first-page":"405","volume-title":"Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Biega Asia J.","year":"2018","unstructured":"Asia J. Biega, Krishna P. Gummadi, and Gerhard Weikum. 2018. Equity of attention: Amortizing individual fairness in rankings. In Proceedings of the 41st International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, Inc, 405\u2013414. DOI:10.1145\/3209978.3210063"},{"key":"e_1_3_4_6_2","doi-asserted-by":"crossref","first-page":"113","DOI":"10.1145\/3523227.3546775","volume-title":"Proceedings of the 16th ACM Conference on Recommender Systems","author":"Bruun Simone Borg","year":"2022","unstructured":"Simone Borg Bruun, Maria Maistro, and Christina Lioma. 2022. Learning recommendations from user actions in the item-poor insurance domain. In Proceedings of the 16th ACM Conference on Recommender Systems. Association for Computing Machinery, Inc, 113\u2013123. DOI:10.1145\/3523227.3546775"},{"key":"e_1_3_4_7_2","volume-title":"Proceedings of the 11th International Conference on Management of Digital EcoSystems","author":"Borges Rodrigo","year":"2019","unstructured":"Rodrigo Borges and Kostas Stefanidis. 2019. Enhancing long term fairness in recommendations with variational autoencoders. In Proceedings of the 11th International Conference on Management of Digital EcoSystems. ACM, New York, NY. DOI:10.1145\/3297662"},{"key":"e_1_3_4_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10888-011-9188-x"},{"key":"e_1_3_4_9_2","doi-asserted-by":"publisher","DOI":"10.1145\/963770.963776"},{"key":"e_1_3_4_10_2","volume-title":"Proceedings of the 29th ACM International Conference on Information & Knowledge Management","author":"Diaz Fernando","year":"2020","unstructured":"Fernando Diaz, Bhaskar Mitra, Michael D. Ekstrand, Asia J. Biega, and Ben Carterette. 2020. Evaluating stochastic rankings with expected exposure. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. ACM, New York, NY. DOI:10.1145\/3340531"},{"key":"e_1_3_4_11_2","first-page":"8596","volume-title":"Proceedings of the Advances in Neural Information Processing Systems","author":"Do Virginie","year":"2021","unstructured":"Virginie Do, Sam Corbett-Davies, Jamal Atif, and Nicolas Usunier. 2021. Two-sided fairness in rankings via Lorenz dominance. In Proceedings of the Advances in Neural Information Processing Systems. 8596\u20138608."},{"key":"e_1_3_4_12_2","first-page":"737","volume-title":"Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Do Virginie","year":"2022","unstructured":"Virginie Do and Nicolas Usunier. 2022. Optimizing generalized gini indices for fairness in rankings. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, 737\u2013747. DOI:10.1145\/3477495"},{"key":"e_1_3_4_13_2","first-page":"214","volume-title":"Proceedings of the Innovations in Theoretical Computer Science Conference","author":"Dwork Cynthia","year":"2012","unstructured":"Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the Innovations in Theoretical Computer Science Conference. 214\u2013226. DOI:10.1145\/2090236.2090255"},{"key":"e_1_3_4_14_2","article-title":"Variabilit\u00e0 e mutabilit\u00e0","author":"Gini C.","year":"1912","unstructured":"C. Gini. 1912. Variabilit\u00e0 e mutabilit\u00e0. Contributo allo Studio delle Distribuzioni e delle Relazioni Statistiche, Tipografia di Paolo Cuppini (Ed.). Bologna.","journal-title":"Contributo allo Studio delle Distribuzioni e delle Relazioni Statistiche"},{"key":"e_1_3_4_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/2827872"},{"key":"e_1_3_4_16_2","first-page":"173","volume-title":"Proceedings of the 26th International World Wide Web Conference, WWW 2017","author":"He Xiangnan","year":"2017","unstructured":"Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat Seng Chua. 2017. Neural collaborative filtering. In Proceedings of the 26th International World Wide Web Conference, WWW 2017. International World Wide Web Conferences Steering Committee, 173\u2013182. DOI:10.1145\/3038912.3052569"},{"key":"e_1_3_4_17_2","doi-asserted-by":"publisher","DOI":"10.1145\/963770.963772"},{"key":"e_1_3_4_18_2","article-title":"A simple sequentially rejective multiple test procedure","author":"Holm S.","year":"1979","unstructured":"S. Holm. 1979. A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics 6, 2 (1979), 65\u201370. http:\/\/www.jstor.org\/stable\/4615733","journal-title":"Scandinavian Journal of Statistics"},{"key":"e_1_3_4_19_2","unstructured":"Rajendra K. Jain Dah-Ming W. Chiu William R. Hawe and others. 1984. A Quantitative Measure Of Fairness And Discrimination For Resource Allocation In Shared Computer Systems. Eastern Research Laboratory Digital Equipment Corporation Hudson MA."},{"key":"e_1_3_4_20_2","doi-asserted-by":"publisher","DOI":"10.1145\/582415.582418"},{"key":"e_1_3_4_21_2","volume-title":"Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings","author":"Kingma Diederik P.","year":"2014","unstructured":"Diederik P. Kingma and Jimmy Lei Ba. 2014. Adam: A method for stochastic optimization. In Proceedings of the 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings. International Conference on Learning Representations, ICLR. DOI:10.48550\/arxiv.1412.6980"},{"key":"e_1_3_4_22_2","doi-asserted-by":"crossref","first-page":"1748","DOI":"10.1145\/3394486.3403226","volume-title":"Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining","author":"Krichene Walid","year":"2020","unstructured":"Walid Krichene and Steffen Rendle. 2020. On sampled metrics for item recommendation. In Proceedings of the ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Association for Computing Machinery, 1748\u20131757. DOI:10.1145\/3394486.3403226"},{"key":"e_1_3_4_23_2","doi-asserted-by":"publisher","DOI":"10.1145\/3610302"},{"key":"e_1_3_4_24_2","first-page":"689","volume-title":"The Web Conference 2018 - Proceedings of the World Wide Web Conference","author":"Liang Dawen","year":"2018","unstructured":"Dawen Liang, Rahul G. Krishnan, Matthew D. Hoffman, and Tony Jebara. 2018. Variational autoencoders for collaborative filtering. In The Web Conference 2018 - Proceedings of the World Wide Web Conference. Association for Computing Machinery, Inc, 689\u2013698. DOI:10.1145\/3178876.3186150"},{"key":"e_1_3_4_25_2","doi-asserted-by":"publisher","DOI":"10.1145\/3585006"},{"key":"e_1_3_4_26_2","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1145\/3340631.3394860","volume-title":"Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization","author":"Mansoury Masoud","year":"2020","unstructured":"Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, and Robin Burke. 2020. FairMatch: A graph-based approach for improving aggregate diversity in recommender systems. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. ACM, 154\u2013162. DOI:10.1145\/3340631.3394860"},{"key":"e_1_3_4_27_2","doi-asserted-by":"publisher","DOI":"10.1145\/3470948"},{"key":"e_1_3_4_28_2","first-page":"1","volume-title":"Proceedings of the Information Retrieval Technology.","author":"Moffat Alistair","year":"2013","unstructured":"Alistair Moffat. 2013. Seven numeric properties of effectiveness metrics. In Proceedings of the Information Retrieval Technology.Rafael E. Banchs, Fabrizio Silvestri, Tie-Yan Liu, Min Zhang, Sheng Gao, and Jun Lang (Eds.), Springer, Berlin, 1\u201312."},{"key":"e_1_3_4_29_2","doi-asserted-by":"publisher","DOI":"10.1145\/1416950.1416952"},{"key":"e_1_3_4_30_2","doi-asserted-by":"crossref","first-page":"429","DOI":"10.1145\/3397271.3401100","volume-title":"Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Morik Marco","year":"2020","unstructured":"Marco Morik, Ashudeep Singh, Jessica Hong, and Thorsten Joachims. 2020. Controlling fairness and bias in dynamic learning-to-rank. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, Inc, 429\u2013438. DOI:10.1145\/3397271.3401100"},{"key":"e_1_3_4_31_2","first-page":"188","volume-title":"Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference","author":"Ni Jianmo","year":"2019","unstructured":"Jianmo Ni, Jiacheng Li, and Julian McAuley. 2019. Justifying recommendations using distantly-labeled reviews and fine-grained aspects. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing, Proceedings of the Conference. Association for Computational Linguistics, 188\u2013197. DOI:10.18653\/V1\/D19-1018"},{"key":"e_1_3_4_32_2","first-page":"497","volume-title":"Proceedings - IEEE International Conference on Data Mining","author":"Ning Xia","year":"2011","unstructured":"Xia Ning and George Karypis. 2011. SLIM: Sparse LInear methods for top-N recommender systems. In Proceedings - IEEE International Conference on Data Mining. 497\u2013506. DOI:10.1109\/ICDM.2011.134"},{"key":"e_1_3_4_33_2","first-page":"1194","volume-title":"Proceedings of the World Wide Web Conference","author":"Patro Gourab K.","year":"2020","unstructured":"Gourab K. Patro, Arpita Biswas, Niloy Ganguly, Krishna P. Gummadi, and Abhijnan Chakraborty. 2020. FairRec: Two-sided fairness for personalized recommendations in two-sided platforms. In Proceedings of the World Wide Web Conference. Association for Computing Machinery, Inc, 1194\u20131204. DOI:10.1145\/3366423.3380196"},{"key":"e_1_3_4_34_2","first-page":"726","volume-title":"Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Raj Amifa","year":"2022","unstructured":"Amifa Raj and Michael D. Ekstrand. 2022. Measuring fairness in ranked results. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, New York, NY, 726\u2013736. DOI:10.1145\/3477495.3532018"},{"key":"e_1_3_4_35_2","doi-asserted-by":"crossref","first-page":"127","DOI":"10.1145\/502716.502737","volume-title":"Proceedings of the 7th International Conference on Intelligent User Interfaces","author":"Rashid Al Mamunur","year":"2002","unstructured":"Al Mamunur Rashid, Istvan Albert, Dan Cosley, Shyong K. Lam, Sean M. McNee, Joseph A. Konstan, and John Riedl. 2002. Getting to know you. In Proceedings of the 7th International Conference on Intelligent User Interfaces. ACM, New York, 127. DOI:10.1145\/502716.502737"},{"key":"e_1_3_4_36_2","first-page":"452","volume-title":"Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence","author":"Rendle Steffen","year":"2009","unstructured":"Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. 2009. BPR: Bayesian personalized ranking from implicit feedback. In Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. AUAI Press, Arlington, Virginia, 452\u2013461."},{"key":"e_1_3_4_37_2","doi-asserted-by":"crossref","first-page":"1514","DOI":"10.1145\/3534678.3539353","volume-title":"Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","author":"Saito Yuta","year":"2022","unstructured":"Yuta Saito and Thorsten Joachims. 2022. Fair ranking as fair division: Impact-based individual fairness in ranking. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 1514\u20131524. DOI:10.1145\/3534678.3539353"},{"key":"e_1_3_4_38_2","doi-asserted-by":"crossref","first-page":"103","DOI":"10.1145\/2911996.2912004","volume-title":"Proceedings of the 2016 ACM International Conference on Multimedia Retrieval","author":"Schedl Markus","year":"2016","unstructured":"Markus Schedl. 2016. The LFM-1b dataset for music retrieval and recommendation. In Proceedings of the 2016 ACM International Conference on Multimedia Retrieval. Association for Computing Machinery, Inc, 103\u2013110. DOI:10.1145\/2911996.2912004"},{"key":"e_1_3_4_39_2","doi-asserted-by":"crossref","first-page":"623","DOI":"10.1002\/j.1538-7305.1948.tb00917.x","article-title":"A mathematical theory of communication","volume":"27","author":"Shannon C. E.","year":"1948","unstructured":"C. E. Shannon. 1948. A mathematical theory of communication. The Bell System Technical Journal 27, 3 (1948), 623\u2013656.","journal-title":"The Bell System Technical Journal"},{"key":"e_1_3_4_40_2","first-page":"165","volume-title":"SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Wang Xiang","year":"2019","unstructured":"Xiang Wang, Xiangnan He, Meng Wang, Fuli Feng, and Tat Seng Chua. 2019. Neural graph collaborative filtering. In SIGIR 2019 - Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, Inc, 165\u2013174. DOI:10.1145\/3331184.3331267"},{"key":"e_1_3_4_41_2","first-page":"117","volume-title":"Proceedings of the ACM International Conference Proceeding Series","author":"Wang Xiuling","year":"2022","unstructured":"Xiuling Wang and Wendy Hui Wang. 2022. Providing item-side individual fairness for deep recommender systems. In Proceedings of the ACM International Conference Proceeding Series. Association for Computing Machinery, 117\u2013127. DOI:10.1145\/3531146.3533079"},{"key":"e_1_3_4_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/3547333"},{"key":"e_1_3_4_43_2","first-page":"695","volume-title":"Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings","author":"Webber William","year":"2008","unstructured":"William Webber, Alistair Moffat, Justin Zobel, and Tetsuya Sakai. 2008. Precision-at-ten considered redundant. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, Proceedings. Singapore, 695\u2013696. DOI:10.1145\/1390334.1390456"},{"key":"e_1_3_4_44_2","first-page":"703","volume-title":"Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Wu Haolun","year":"2022","unstructured":"Haolun Wu, Bhaskar Mitra, Chen Ma, Fernando Diaz, and Xue Liu. 2022. Joint multisided exposure fairness for recommendation. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, Inc, 703\u2013714. DOI:10.1145\/3477495.3532007"},{"key":"e_1_3_4_45_2","doi-asserted-by":"crossref","first-page":"119","DOI":"10.1145\/2481492.2481505","volume-title":"Proceedings of the 24th ACM Conference on Hypertext and Social Media","author":"Yang Dingqi","year":"2013","unstructured":"Dingqi Yang, Daqing Zhang, Zhiyong Yu, and Zhu Wang. 2013. A sentiment-enhanced personalized location recommendation system. In Proceedings of the 24th ACM Conference on Hypertext and Social Media. 119\u2013128. DOI:10.1145\/2481492.2481505"},{"key":"e_1_3_4_46_2","doi-asserted-by":"publisher","DOI":"10.1145\/3533380"},{"key":"e_1_3_4_47_2","first-page":"4653","volume-title":"Proceedings of the International Conference on Information and Knowledge Management, Proceedings","author":"Zhao Wayne Xin","year":"2021","unstructured":"Wayne Xin Zhao, Shanlei Mu, Yupeng Hou, Zihan Lin, Yushuo Chen, Xingyu Pan, Kaiyuan Li, Yujie Lu, Hui Wang, Changxin Tian, Yingqian Min, Zhichao Feng, Xinyan Fan, Xu Chen, Pengfei Wang, Wendi Ji, Yaliang Li, Xiaoling Wang, and Ji Rong Wen. 2021. RecBole: Towards a unified, comprehensive and efficient framework for recommendation algorithms. In Proceedings of the International Conference on Information and Knowledge Management, Proceedings. ACM, New York, NY, 4653\u20134664. DOI:10.1145\/3459637.3482016"},{"key":"e_1_3_4_48_2","doi-asserted-by":"publisher","DOI":"10.1109\/ACCESS.2020.3007617"},{"key":"e_1_3_4_49_2","first-page":"85","volume-title":"Proceedings of the 14th ACM International Conference on Web Search and Data Mining","author":"Zhu Ziwei","year":"2021","unstructured":"Ziwei Zhu, Yun He, Xing Zhao, Yin Zhang, Jianling Wang, and James Caverlee. 2021. Popularity-opportunity bias in collaborative filtering. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. Association for Computing Machinery, Inc, 85\u201393. DOI:10.1145\/3437963.3441820"},{"key":"e_1_3_4_50_2","first-page":"767","volume-title":"Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval","author":"Zhu Ziwei","year":"2021","unstructured":"Ziwei Zhu, Jingu Kim, Trung Nguyen, Aish Fenton, and James Caverlee. 2021. Fairness among new items in cold start recommender systems. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, Inc, 767\u2013776. DOI:10.1145\/3404835.3462948"},{"key":"e_1_3_4_51_2","doi-asserted-by":"crossref","first-page":"22","DOI":"10.1145\/1060745.1060754","volume-title":"Proceedings of the 14th International Conference on World Wide Web.","author":"Ziegler Cai-Nicolas","year":"2005","unstructured":"Cai-Nicolas Ziegler, Sean M. McNee, Joseph A. Konstan, and Georg Lausen. 2005. Improving recommendation lists through topic diversification. In Proceedings of the 14th International Conference on World Wide Web.Association for Computing Machinery, New York, NY, 22\u201332. DOI:10.1145\/1060745.1060754"}],"container-title":["ACM Transactions on Recommender Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3631943","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3631943","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:51:02Z","timestamp":1750287062000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3631943"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,28]]},"references-count":50,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,6,30]]}},"alternative-id":["10.1145\/3631943"],"URL":"https:\/\/doi.org\/10.1145\/3631943","relation":{},"ISSN":["2770-6699"],"issn-type":[{"value":"2770-6699","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,28]]},"assertion":[{"value":"2023-03-31","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-19","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-11-28","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}