{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,14]],"date-time":"2026-03-14T09:50:26Z","timestamp":1773481826339,"version":"3.50.1"},"reference-count":69,"publisher":"Association for Computing Machinery (ACM)","issue":"5","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,1]]},"abstract":"<jats:p>Biased data can lead to unfair machine learning models, highlighting the importance of embedding fairness at the beginning of data analysis, particularly during dataset curation and labeling. In response, we propose Falcon, a scalable fair active learning framework. Falcon adopts a data-centric approach that improves machine learning model fairness via strategic sample selection. Given a user-specified group fairness measure, Falcon identifies samples from \"target groups\" (e.g., (attribute=female, label=positive)) that are the most informative for improving fairness. However, a challenge arises since these target groups are defined using ground truth labels that are not available during sample selection. To handle this, we propose a novel trial-and-error method, where we postpone using a sample if the predicted label is different from the expected one and falls outside the target group. We also observe the trade-off that selecting more informative samples results in higher likelihood of postponing due to undesired label prediction, and the optimal balance varies per dataset. We capture the trade-off between informativeness and postpone rate as policies and propose to automatically select the best policy using adversarial multi-armed bandit methods, given their computational efficiency and theoretical guarantees. Experiments show that Falcon significantly outperforms existing fair active learning approaches in terms of fairness and accuracy and is more efficient. In particular, only Falcon supports a proper trade-off between accuracy and fairness where its maximum fairness score is 1.8--4.5x higher than the second-best results.<\/jats:p>","DOI":"10.14778\/3641204.3641207","type":"journal-article","created":{"date-parts":[[2024,5,2]],"date-time":"2024-05-02T22:05:43Z","timestamp":1714687543000},"page":"952-965","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Falcon: Fair Active Learning Using Multi-Armed Bandits"],"prefix":"10.14778","volume":"17","author":[{"given":"Ki Hyun","family":"Tae","sequence":"first","affiliation":[{"name":"KAIST, Daejeon, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hantian","family":"Zhang","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jaeyoung","family":"Park","sequence":"additional","affiliation":[{"name":"KAIST, Daejeon, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kexin","family":"Rong","sequence":"additional","affiliation":[{"name":"Georgia Institute of Technology, Atlanta, USA"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Steven Euijong","family":"Whang","sequence":"additional","affiliation":[{"name":"KAIST, Daejeon, Republic of Korea"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2024,5,2]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"ICML.","author":"Abe Naoki","unstructured":"Naoki Abe and Hiroshi Mamitsuka. 1998. Query Learning Strategies Using Boosting and Bagging. In ICML. San Francisco, CA, USA, 1--9."},{"key":"e_1_2_1_2_1","unstructured":"Jacob Abernethy Pranjal Awasthi Matth\u00e4us Kleindessner Jamie Morgenstern Chris Russell and Jie Zhang. 2022. Active Sampling for Min-Max Fairness. In ICML."},{"key":"e_1_2_1_3_1","unstructured":"Jacob D. Abernethy Pranjal Awasthi Matth\u00e4us Kleindessner Jamie Morgenstern Chris Russell and Jie Zhang. 2022. Active Sampling for Min-Max Fairness. In ICML. 53--65."},{"key":"e_1_2_1_4_1","doi-asserted-by":"crossref","unstructured":"Umang Aggarwal Adrian Popescu and C\u00e9line Hudelot. 2020. Active Learning for Imbalanced Datasets. In WACV. 1417--1426.","DOI":"10.1109\/WACV45572.2020.9093475"},{"key":"e_1_2_1_5_1","doi-asserted-by":"crossref","unstructured":"Umang Aggarwal Adrian Popescu and C\u00e9line Hudelot. 2021. Minority Class Oriented Active Learning for Imbalanced Datasets. In ICPR. 9920--9927.","DOI":"10.1109\/ICPR48806.2021.9412182"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2022.116981"},{"key":"e_1_2_1_7_1","unstructured":"J. Angwin J. Larson S. Mattu and L. Kirchner. 2016. Machine bias: There's software used across the country to predict future criminals. And its biased against blacks."},{"key":"e_1_2_1_8_1","unstructured":"Jordan T. Ash Chicheng Zhang Akshay Krishnamurthy John Langford and Alekh Agarwal. 2020. Deep Batch Active Learning by Diverse Uncertain Gradient Lower Bounds. In ICLR."},{"key":"e_1_2_1_9_1","volume-title":"Finite-Time Analysis of the Multiarmed Bandit Problem. 47, 2--3","author":"Auer Peter","year":"2002","unstructured":"Peter Auer, Nicol\u00f2 Cesa-Bianchi, and Paul Fischer. 2002. Finite-Time Analysis of the Multiarmed Bandit Problem. 47, 2--3 (2002), 235--256."},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1137\/S0097539701398375"},{"key":"e_1_2_1_11_1","first-page":"9","article-title":"Seiden: Revisiting Query Processing in Video Database Systems","volume":"16","author":"Bang Jaeho","year":"2023","unstructured":"Jaeho Bang, Gaurav Tarlok Kakkar, Pramod Chunduri, Subrata Mitra, and Joy Arulraj. 2023. Seiden: Revisiting Query Processing in Video Database Systems. Proc. VLDB Endow. 16, 9 (may 2023), 2289--2301.","journal-title":"Proc. VLDB Endow."},{"key":"e_1_2_1_12_1","unstructured":"Solon Barocas Moritz Hardt and Arvind Narayanan. 2019. Fairness and Machine Learning: Limitations and Opportunities. fairmlbook.org. http:\/\/www.fairmlbook.org."},{"key":"e_1_2_1_13_1","volume-title":"John Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, and Yunfeng Zhang.","author":"Bellamy Rachel K. E.","year":"2018","unstructured":"Rachel K. E. Bellamy, Kuntal Dey, Michael Hind, Samuel C. Hoffman, Stephanie Houde, Kalapriya Kannan, Pranay Lohia, Jacquelyn Martino, Sameep Mehta, Aleksandra Mojsilovic, Seema Nagar, Karthikeyan Natesan Ramamurthy, John Richards, Diptikalyan Saha, Prasanna Sattigeri, Moninder Singh, Kush R. Varshney, and Yunfeng Zhang. 2018. AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Bias."},{"key":"e_1_2_1_14_1","first-page":"19","article-title":"Contextual Bandit Algorithms with Supervised Learning Guarantees","volume":"15","author":"Beygelzimer Alina","year":"2011","unstructured":"Alina Beygelzimer, John Langford, Lihong Li, Lev Reyzin, and Robert E. Schapire. 2011. Contextual Bandit Algorithms with Supervised Learning Guarantees. In AISTATS (JMLR Proceedings), Vol. 15. 19--26.","journal-title":"AISTATS (JMLR Proceedings)"},{"key":"e_1_2_1_15_1","volume-title":"Fairlearn: A toolkit for assessing and improving fairness in AI. Technical Report MSR-TR-2020-32. Microsoft.","author":"Bird Sarah","year":"2020","unstructured":"Sarah Bird, Miro Dudik, Richard Edgar, Brandon Horn, Roman Lutz, Vanessa Milan, Mehrnoosh Sameki, Hanna Wallach, and Kathleen Walker. 2020. Fairlearn: A toolkit for assessing and improving fairness in AI. Technical Report MSR-TR-2020-32. Microsoft."},{"key":"e_1_2_1_16_1","first-page":"11","article-title":"Effective Entity Augmentation by Querying External Data Sources","volume":"16","author":"Buss Christopher","year":"2023","unstructured":"Christopher Buss, Jasmin Mousavi, Mikhail Tokarev, Arash Termehchy, David Maier, and Stefan Lee. 2023. Effective Entity Augmentation by Querying External Data Sources. Proc. VLDB Endow. 16, 11 (jul 2023), 3404--3417.","journal-title":"Proc. VLDB Endow."},{"key":"e_1_2_1_17_1","doi-asserted-by":"crossref","unstructured":"Yiting Cao and Chao Lan. 2022. Fairness-Aware Active Learning for Decoupled Model. In IJCNN. 1--9.","DOI":"10.1109\/IJCNN55064.2022.9892608"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.14778\/3523210.3523223"},{"key":"e_1_2_1_19_1","volume-title":"Sontag","author":"Chen Irene Y.","year":"2018","unstructured":"Irene Y. Chen, Fredrik D. Johansson, and David A. Sontag. 2018. Why Is My Classifier Discriminatory?. In NeurIPS. 3543--3554."},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1089\/big.2016.0047"},{"key":"e_1_2_1_21_1","volume-title":"Amazon scraps secret AI recruiting tool that showed bias against women","author":"Dastin Jeffrey","year":"2018","unstructured":"Jeffrey Dastin. 2018. Amazon scraps secret AI recruiting tool that showed bias against women. Reuters (2018)."},{"key":"e_1_2_1_22_1","volume-title":"Retiring Adult: New Datasets for Fair Machine Learning. In NeurIPS. 6478--6490.","author":"Ding Frances","year":"2021","unstructured":"Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. 2021. Retiring Adult: New Datasets for Fair Machine Learning. In NeurIPS. 6478--6490."},{"key":"e_1_2_1_23_1","first-page":"2803","article-title":"Is There a Trade-Off Between Fairness and Accuracy? A Perspective Using Mismatched Hypothesis Testing","volume":"119","author":"Dutta Sanghamitra","year":"2020","unstructured":"Sanghamitra Dutta, Dennis Wei, Hazar Yueksel, Pin-Yu Chen, Sijia Liu, and Kush R. Varshney. 2020. Is There a Trade-Off Between Fairness and Accuracy? A Perspective Using Mismatched Hypothesis Testing. In ICML, Vol. 119. 2803--2813.","journal-title":"ICML"},{"key":"e_1_2_1_24_1","doi-asserted-by":"crossref","unstructured":"Ehsan Elhamifar Guillermo Sapiro Allen Y. Yang and S. Shankar Sastry. 2013. A Convex Optimization Framework for Active Learning. In ICCV. 209--216.","DOI":"10.1109\/ICCV.2013.33"},{"key":"e_1_2_1_25_1","volume-title":"Regret Bounds for Batched Bandits. 35, 8 (May","author":"Esfandiari Hossein","year":"2021","unstructured":"Hossein Esfandiari, Amin Karbasi, Abbas Mehrabian, and Vahab Mirrokni. 2021. Regret Bounds for Batched Bandits. 35, 8 (May 2021), 7340--7348."},{"key":"e_1_2_1_26_1","doi-asserted-by":"crossref","unstructured":"Michael Feldman Sorelle A. Friedler John Moeller Carlos Scheidegger and Suresh Venkatasubramanian. 2015. Certifying and Removing Disparate Impact. In KDD. 259--268.","DOI":"10.1145\/2783258.2783311"},{"key":"e_1_2_1_27_1","first-page":"12","article-title":"Ml-whatif: What If You Could Stop Re-Implementing Your Machine Learning Pipeline Analyses over and Over","volume":"16","author":"Grafberger Stefan","year":"2023","unstructured":"Stefan Grafberger, Shubha Guha, Paul Groth, and Sebastian Schelter. 2023. Ml-whatif: What If You Could Stop Re-Implementing Your Machine Learning Pipeline Analyses over and Over? Proc. VLDB Endow. 16, 12 (aug 2023), 4002--4005.","journal-title":"Proc. VLDB Endow."},{"key":"e_1_2_1_28_1","unstructured":"Moritz Hardt Eric Price and Nati Srebro. 2016. Equality of Opportunity in Supervised Learning. In NeurIPS. 3315--3323."},{"key":"e_1_2_1_29_1","unstructured":"Wei-Ning Hsu and Hsuan-Tien Lin. 2015. Active Learning by Learning. In AAAI. 2659--2665."},{"key":"e_1_2_1_30_1","article-title":"Machine Learning and Data Cleaning: Which Serves the Other","volume":"14","author":"Ilyas Ihab F.","year":"2022","unstructured":"Ihab F. Ilyas and Theodoros Rekatsinas. 2022. Machine Learning and Data Cleaning: Which Serves the Other? ACM J. Data Inf. Qual. 14, 3 (2022), 13:1--13:11.","journal-title":"ACM J. Data Inf. Qual."},{"key":"e_1_2_1_31_1","doi-asserted-by":"crossref","unstructured":"Vasileios Iosifidis and Eirini Ntoutsi. 2019. AdaFair: Cumulative Fairness Adaptive Boosting. In CIKM. ACM 781--790.","DOI":"10.1145\/3357384.3357974"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10115-011-0463-8"},{"key":"e_1_2_1_33_1","volume-title":"ICML (Proceedings of Machine Learning Research)","volume":"162","author":"Kim Jung-Hun","year":"2022","unstructured":"Jung-Hun Kim, Milan Vojnovic, and Se-Young Yun. 2022. Rotting Infinitely Many-Armed Bandits. In ICML (Proceedings of Machine Learning Research), Vol. 162. PMLR, 11229--11254."},{"key":"e_1_2_1_34_1","volume-title":"Boost-clean: Automated error detection and repair for machine learning. arXiv preprint arXiv:1711.01299","author":"Krishnan Sanjay","year":"2017","unstructured":"Sanjay Krishnan, Michael J Franklin, Ken Goldberg, and Eugene Wu. 2017. Boost-clean: Automated error detection and repair for machine learning. arXiv preprint arXiv:1711.01299 (2017)."},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.14778\/3372716.3372723"},{"key":"e_1_2_1_36_1","volume-title":"Bandit Algorithms","author":"Lattimore Tor","unstructured":"Tor Lattimore and Csaba Szepesv\u00e1ri. 2020. Bandit Algorithms. Cambridge University Press."},{"key":"e_1_2_1_37_1","unstructured":"Dong-Hyun Lee. 2013. Pseudo-Label: The Simple and Efficient Semi-Supervised Learning Method for Deep Neural Networks."},{"key":"e_1_2_1_38_1","unstructured":"Nir Levine Koby Crammer and Shie Mannor. 2017. Rotting Bandits. In NeurIPS. 3074--3083."},{"key":"e_1_2_1_39_1","volume-title":"Gale","author":"Lewis David D.","year":"1994","unstructured":"David D. Lewis and William A. Gale. 1994. A Sequential Algorithm for Training Text Classifiers. In SIGIR. 3--12."},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1007\/s00778-021-00699-w"},{"key":"e_1_2_1_41_1","first-page":"107","article-title":"The cost of fairness in binary classification","volume":"81","author":"Menon Aditya Krishna","year":"2018","unstructured":"Aditya Krishna Menon and Robert C. Williamson. 2018. The cost of fairness in binary classification. In FAT, Vol. 81. 107--118.","journal-title":"FAT"},{"key":"e_1_2_1_42_1","volume-title":"Smeulders","author":"Nguyen Hieu Tat","year":"2004","unstructured":"Hieu Tat Nguyen and Arnold W. M. Smeulders. 2004. Active learning using pre-clustering. In ICML."},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.5555\/1953048.2078195"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.14778\/3157794.3157797"},{"key":"e_1_2_1_45_1","unstructured":"Yuji Roh Kangwook Lee Steven Whang and Changho Suh. 2021. Sample Selection for Fair and Robust Training. In NeurIPS. 815--827."},{"key":"e_1_2_1_46_1","volume-title":"Steven Euijong Whang, and Changho Suh","author":"Roh Yuji","year":"2021","unstructured":"Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh. 2021. Fair-Batch: Batch Selection for Model Fairness. In ICLR."},{"key":"e_1_2_1_47_1","volume-title":"ECML","volume":"4212","author":"Roth Dan","year":"2006","unstructured":"Dan Roth and Kevin Small. 2006. Margin-Based Active Learning for Structured Output Spaces. In ECML, Vol. 4212. Springer, 413--424."},{"key":"e_1_2_1_48_1","volume-title":"Toward Optimal Active Learning Through Sampling Estimation of Error Reduction","author":"Roy Nicholas","unstructured":"Nicholas Roy and Andrew McCallum. 2001. Toward Optimal Active Learning Through Sampling Estimation of Error Reduction. In ICML. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 441--448."},{"key":"e_1_2_1_49_1","doi-asserted-by":"crossref","unstructured":"Babak Salimi Luke Rodriguez Bill Howe and Dan Suciu. 2019. Interventional fairness: Causal database repair for algorithmic fairness. In SIGMOD. 793--810.","DOI":"10.1145\/3299869.3319901"},{"key":"e_1_2_1_50_1","unstructured":"Ozan Sener and Silvio Savarese. 2018. Active Learning for Convolutional Neural Networks: A Core-Set Approach. In ICLR."},{"key":"e_1_2_1_51_1","volume-title":"Active Learning","author":"Settles Burr","unstructured":"Burr Settles. 2012. Active Learning. Morgan & Claypool Publishers."},{"key":"e_1_2_1_52_1","doi-asserted-by":"crossref","unstructured":"Burr Settles and Mark Craven. 2008. An Analysis of Active Learning Strategies for Sequence Labeling Tasks. In EMNLP. 1070--1079.","DOI":"10.3115\/1613715.1613855"},{"key":"e_1_2_1_53_1","volume-title":"NIPS (Vancouver, British Columbia, Canada)","author":"Settles Burr","unstructured":"Burr Settles, Mark Craven, and Soumya Ray. 2007. Multiple-instance Active Learning. In NIPS (Vancouver, British Columbia, Canada). Curran Associates Inc., USA, 1289--1296."},{"key":"e_1_2_1_54_1","doi-asserted-by":"crossref","unstructured":"H. S. Seung M. Opper and H. Sompolinsky. 1992. Query by Committee. In COLT (Pittsburgh Pennsylvania USA). ACM New York NY USA 287--294.","DOI":"10.1145\/130385.130417"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/584091.584093"},{"key":"e_1_2_1_56_1","volume-title":"Hal Daum\u00e9 III, and Renkun Ni","author":"Sharaf Amr","year":"2022","unstructured":"Amr Sharaf, Hal Daum\u00e9 III, and Renkun Ni. 2022. Promoting Fairness in Learned Models by Learning to Active Learn under Parity Constraints. In FAccT. ACM, 2149--2156."},{"key":"e_1_2_1_57_1","unstructured":"Shubhanshu Shekhar Greg Fields Mohammad Ghavamzadeh and Tara Javidi. 2021. Adaptive Sampling for Minimax Fair Classification. In NeurIPS. 24535--24544."},{"key":"e_1_2_1_58_1","volume-title":"Adams","author":"Snoek Jasper","year":"2012","unstructured":"Jasper Snoek, Hugo Larochelle, and Ryan P. Adams. 2012. Practical Bayesian Optimization of Machine Learning Algorithms. In NeurIPS (NIPS'12). 2951--2959."},{"key":"e_1_2_1_59_1","volume-title":"Slice Tuner: A Selective Data Acquisition Framework for Accurate and Fair Machine Learning Models. In SIGMOD. ACM, 1771--1783.","author":"Tae Ki Hyun","year":"2021","unstructured":"Ki Hyun Tae and Steven Euijong Whang. 2021. Slice Tuner: A Selective Data Acquisition Framework for Accurate and Fair Machine Learning Models. In SIGMOD. ACM, 1771--1783."},{"key":"e_1_2_1_60_1","volume-title":"Falcon: Fair Active Learning using MABs. https:\/\/github.com\/khtae8250\/Falcon\/blob\/main\/techreport.pdf.","author":"Tae Ki Hyun","year":"2023","unstructured":"Ki Hyun Tae, Hantian Zhang, Jaeyoung Park, Kexin Rong, and Steven Euijong Whang. 2023. Falcon: Fair Active Learning using MABs. https:\/\/github.com\/khtae8250\/Falcon\/blob\/main\/techreport.pdf."},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.14778\/2831360.2831371"},{"key":"e_1_2_1_62_1","doi-asserted-by":"crossref","unstructured":"Sahil Verma and Julia Rubin. 2018. Fairness definitions explained. In Fair-Ware@ICSE. 1--7.","DOI":"10.1145\/3194770.3194776"},{"key":"e_1_2_1_63_1","volume-title":"Multi-armed Bandit Algorithms and Empirical Evaluation","author":"Vermorel Joann\u00e8s","unstructured":"Joann\u00e8s Vermorel and Mehryar Mohri. 2005. Multi-armed Bandit Algorithms and Empirical Evaluation. In ECML. Springer Berlin Heidelberg, 437--448."},{"key":"e_1_2_1_64_1","doi-asserted-by":"crossref","unstructured":"Dan Wang and Yi Shang. 2014. A new active labeling method for deep learning. In IJCNN. 112--119.","DOI":"10.1109\/IJCNN.2014.6889457"},{"key":"e_1_2_1_65_1","volume-title":"NeurIPS","volume":"21","author":"Wang Yizao","year":"2008","unstructured":"Yizao Wang, Jean-yves Audibert, and R\u00e9mi Munos. 2008. Algorithms for Infinitely Many-Armed Bandits. In NeurIPS, Vol. 21. Curran Associates, Inc."},{"key":"e_1_2_1_66_1","unstructured":"Steven Euijong Whang Yuji Roh Hwanjun Song and Jae-Gil Lee. 2023. Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective. VLDB J. (2023)."},{"key":"e_1_2_1_67_1","volume-title":"Navathe","author":"Zhang Hantian","year":"2021","unstructured":"Hantian Zhang, Xu Chu, Abolfazl Asudeh, and Shamkant B. Navathe. 2021. OmniFair: A Declarative System for Model-Agnostic Group Fairness in Machine Learning. In SIGMOD. ACM, 2076--2088."},{"key":"e_1_2_1_68_1","first-page":"1","volume-title":"Proc. ACM Manag. Data 1","author":"Zhang Hantian","year":"2023","unstructured":"Hantian Zhang, Ki Hyun Tae, Jaeyoung Park, Xu Chu, and Steven Euijong Whang. 2023. iFlipper: Label Flipping for Individual Fairness. Proc. ACM Manag. Data 1, 1 (2023), 8:1--8:26."},{"key":"e_1_2_1_69_1","volume-title":"Gordon","author":"Zhao Han","year":"2019","unstructured":"Han Zhao and Geoffrey J. Gordon. 2019. Inherent Tradeoffs in Learning Fair Representations. In NeurIPS. 15649--15659."}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3641204.3641207","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,5,2]],"date-time":"2024-05-02T22:07:08Z","timestamp":1714687628000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3641204.3641207"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1]]},"references-count":69,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2024,1]]}},"alternative-id":["10.14778\/3641204.3641207"],"URL":"https:\/\/doi.org\/10.14778\/3641204.3641207","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,1]]},"assertion":[{"value":"2024-05-02","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}