{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,9]],"date-time":"2026-05-09T19:14:55Z","timestamp":1778354095468,"version":"3.51.4"},"reference-count":96,"publisher":"Association for Computing Machinery (ACM)","issue":"4","license":[{"start":{"date-parts":[[2024,2,13]],"date-time":"2024-02-13T00:00:00Z","timestamp":1707782400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"TUBITAK 1001 grant","award":["120E346"],"award-info":[{"award-number":["120E346"]}]},{"name":"BAGEP 2023 Young Scientist Award"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2024,5,31]]},"abstract":"<jats:p>Transformer-based contextualized language models constitute the state-of-the-art in several natural language processing (NLP) tasks and applications. Despite their utility, contextualized models can contain human-like social biases, as their training corpora generally consist of human-generated text. Evaluating and removing social biases in NLP models has been a major research endeavor. In parallel, NLP approaches in the legal domain, namely, legal NLP or computational law, have also been increasing. Eliminating unwanted bias in legal NLP is crucial, since the law has the utmost importance and effect on people. In this work, we focus on the gender bias encoded in BERT-based models. We propose a new template-based bias measurement method with a new bias evaluation corpus using crime words from the FBI database. This method quantifies the gender bias present in BERT-based models for legal applications. Furthermore, we propose a new fine-tuning-based debiasing method using the European Court of Human Rights (ECtHR) corpus to debias legal pre-trained models. We test the debiased models\u2019 language understanding performance on the LexGLUE benchmark to confirm that the underlying semantic vector space is not perturbed during the debiasing process. Finally, we propose a bias penalty for the performance scores to emphasize the effect of gender bias on model performance.<\/jats:p>","DOI":"10.1145\/3628602","type":"journal-article","created":{"date-parts":[[2023,10,18]],"date-time":"2023-10-18T21:38:04Z","timestamp":1697665084000},"page":"1-26","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Measuring and Mitigating Gender Bias in Legal Contextualized Language Models"],"prefix":"10.1145","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0009-0007-9090-8555","authenticated-orcid":false,"given":"Mustafa","family":"Bozdag","sequence":"first","affiliation":[{"name":"Dept. of Electrical and Electronics Engineering and UMRAM, Bilkent University, Turkey"}]},{"ORCID":"https:\/\/orcid.org\/0009-0000-0790-0587","authenticated-orcid":false,"given":"Nurullah","family":"Sevim","sequence":"additional","affiliation":[{"name":"Dept. of Electrical and Electronics Engineering and UMRAM, Bilkent University, Turkey"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6348-2663","authenticated-orcid":false,"given":"Aykut","family":"Ko\u00e7","sequence":"additional","affiliation":[{"name":"Dept. of Electrical and Electronics Engineering and UMRAM, Bilkent University, Turkey"}]}],"member":"320","published-online":{"date-parts":[[2024,2,13]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.7717\/peerj-cs.93"},{"key":"e_1_3_2_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0004-3702(03)00105-X"},{"key":"e_1_3_2_4_2","first-page":"1","article-title":"Thirty years of artificial intelligence and law: Overviews","author":"Araszkiewicz Micha\u0142","year":"2022","unstructured":"Micha\u0142 Araszkiewicz, Trevor Bench-Capon, Enrico Francesconi, Marc Lauritsen, and Antonino Rotolo. 2022. Thirty years of artificial intelligence and law: Overviews. Artif. Intell. Law (2022), 1\u201318.","journal-title":"Artif. Intell. Law"},{"issue":"02","key":"e_1_3_2_5_2","article-title":"Gender attitudes in the judiciary: Evidence from US circuit courts","volume":"2019","author":"Ash Elliott","year":"2021","unstructured":"Elliott Ash, Daniel L. Chen, and Arianna Ornaghi. 2021. Gender attitudes in the judiciary: Evidence from US circuit courts. Cent. Law Econ. Work. Pap. Series 2019, 02 (2021).","journal-title":"Cent. Law Econ. Work. Pap. Series"},{"key":"e_1_3_2_6_2","volume-title":"Modelling Legal Argument: Reasoning with Cases and Hypotheticals","author":"Ashley Kevin D.","year":"1988","unstructured":"Kevin D. Ashley. 1988. Modelling Legal Argument: Reasoning with Cases and Hypotheticals. Ph. D. Dissertation. University of Massachusetts."},{"key":"e_1_3_2_7_2","doi-asserted-by":"publisher","DOI":"10.1016\/0020-7373(91)90011-U"},{"key":"e_1_3_2_8_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF00114920"},{"key":"e_1_3_2_9_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-009-9077-9"},{"key":"e_1_3_2_10_2","article-title":"Gender differences in judicial decisions under incomplete information: Evidence from child support cases","author":"Asmat Roberto","year":"2021","unstructured":"Roberto Asmat and Lajos Kossuth. 2021. Gender differences in judicial decisions under incomplete information: Evidence from child support cases. Retrieved from: SSRN 3964747 (2021).","journal-title":"Retrieved from: SSRN 3964747"},{"issue":"1","key":"e_1_3_2_11_2","first-page":"3","article-title":"A two-phase framework for learning logical structures of paragraphs in legal articles","volume":"12","author":"Bach Ngo Xuan","year":"2013","unstructured":"Ngo Xuan Bach, Nguyen Le Minh, Tran Thi Oanh, and Akira Shimazu. 2013. A two-phase framework for learning logical structures of paragraphs in legal articles. ACM Trans. Asian Lang. Inf. Process. 12, 1, Article 3 (Mar. 2013), 32 pages.","journal-title":"ACM Trans. Asian Lang. Inf. Process."},{"key":"e_1_3_2_12_2","first-page":"1","volume-title":"Proceedings of the 2nd Workshop on Gender Bias in Natural Language Processing","author":"Bartl Marion","year":"2020","unstructured":"Marion Bartl, Malvina Nissim, and Albert Gatt. 2020. Unmasking contextual stereotypes: Measuring and mitigating BERT\u2019s gender bias. In Proceedings of the 2nd Workshop on Gender Bias in Natural Language Processing. Association for Computational Linguistics, 1\u201316. Retrieved from https:\/\/aclanthology.org\/2020.gebnlp-1.1"},{"key":"e_1_3_2_13_2","article-title":"Longformer: The long-document transformer","volume":"2004","author":"Beltagy Iz","year":"2020","unstructured":"Iz Beltagy, Matthew E. Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. CoRR abs\/2004.05150 (2020).","journal-title":"CoRR"},{"key":"e_1_3_2_14_2","first-page":"1","article-title":"Thirty years of artificial intelligence and law: Editor\u2019s introduction","author":"Bench-Capon Trevor","year":"2022","unstructured":"Trevor Bench-Capon. 2022. Thirty years of artificial intelligence and law: Editor\u2019s introduction. Artif. Intell. Law (2022), 1\u20135.","journal-title":"Artif. Intell. Law"},{"key":"e_1_3_2_15_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-012-9131-x"},{"issue":"4","key":"e_1_3_2_16_2","doi-asserted-by":"crossref","first-page":"1008","DOI":"10.1007\/s12559-021-09881-2","article-title":"Investigating gender bias in BERT","volume":"13","author":"Bhardwaj Rishabh","year":"2021","unstructured":"Rishabh Bhardwaj, Navonil Majumder, and Soujanya Poria. 2021. Investigating gender bias in BERT. Cog. Comput. 13, 4 (2021), 1008\u20131018.","journal-title":"Cog. Comput."},{"key":"e_1_3_2_17_2","first-page":"435","volume-title":"Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS\u201916)","author":"Bolukbasi Tolga","year":"2016","unstructured":"Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man is to computer programmer as woman is to homemaker? Debiasing word embeddings. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS\u201916). Curran Associates Inc., Red Hook, NY, 435\u2013364."},{"key":"e_1_3_2_18_2","doi-asserted-by":"publisher","DOI":"10.2307\/1227753"},{"key":"e_1_3_2_19_2","doi-asserted-by":"publisher","DOI":"10.1126\/science.aal4230"},{"key":"e_1_3_2_20_2","doi-asserted-by":"crossref","first-page":"4317","DOI":"10.18653\/v1\/P19-1424","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Chalkidis Ilias","year":"2019","unstructured":"Ilias Chalkidis, Ion Androutsopoulos, and Nikolaos Aletras. 2019. Neural legal judgment prediction in English. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 4317\u20134323. DOI:10.18653\/v1\/P19-1424"},{"key":"e_1_3_2_21_2","first-page":"19","volume-title":"Proceedings of the 16th Edition of the International Conference on Articial Intelligence and Law (ICAIL\u201917)","author":"Chalkidis Ilias","year":"2017","unstructured":"Ilias Chalkidis, Ion Androutsopoulos, and Achilleas Michos. 2017. Extracting contract elements. In Proceedings of the 16th Edition of the International Conference on Articial Intelligence and Law (ICAIL\u201917). Association for Computing Machinery, New York, NY, 19\u201328. DOI:10.1145\/3086512.3086515"},{"key":"e_1_3_2_22_2","first-page":"254","volume-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics","author":"Chalkidis Ilias","year":"2018","unstructured":"Ilias Chalkidis, Ion Androutsopoulos, and Achilleas Michos. 2018. Obligation and prohibition extraction using hierarchical RNNs. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 254\u2013259. DOI:10.18653\/v1\/P18-2041"},{"key":"e_1_3_2_23_2","first-page":"78","volume-title":"Proceedings of the Natural Legal Language Processing Workshop","author":"Chalkidis Ilias","year":"2019","unstructured":"Ilias Chalkidis, Emmanouil Fergadiotis, Prodromos Malakasiotis, Nikolaos Aletras, and Ion Androutsopoulos. 2019. Extreme multi-label legal text classification: A case study in EU legislation. In Proceedings of the Natural Legal Language Processing Workshop. Association for Computational Linguistics, 78\u201387. DOI:10.18653\/v1\/W19-2209"},{"key":"e_1_3_2_24_2","doi-asserted-by":"crossref","first-page":"6314","DOI":"10.18653\/v1\/P19-1636","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Chalkidis Ilias","year":"2019","unstructured":"Ilias Chalkidis, Emmanouil Fergadiotis, Prodromos Malakasiotis, and Ion Androutsopoulos. 2019. Large-scale multi-label text classification on EU legislation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 6314\u20136322. DOI:10.18653\/v1\/P19-1636"},{"key":"e_1_3_2_25_2","first-page":"6974","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing","author":"Chalkidis Ilias","year":"2021","unstructured":"Ilias Chalkidis, Manos Fergadiotis, and Ion Androutsopoulos. 2021. MultiEurlEX\u2014A multi-lingual and multi-label legal document classification dataset for zero-shot cross-lingual transfer. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 6974\u20136996. Retrieved from https:\/\/aclanthology.org\/2021.emnlp-main.559"},{"key":"e_1_3_2_26_2","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.261"},{"key":"e_1_3_2_27_2","volume-title":"Proceedings of the Workshop on Document Intelligence at NeurIPS","author":"Chalkidis Ilias","year":"2019","unstructured":"Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, and Ion Androutsopoulos. 2019. Neural contract element extraction revisited. In Proceedings of the Workshop on Document Intelligence at NeurIPS. Retrieved from https:\/\/openreview.net\/forum?id=B1x6fa95UH"},{"key":"e_1_3_2_28_2","first-page":"226","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Chalkidis Ilias","year":"2021","unstructured":"Ilias Chalkidis, Manos Fergadiotis, Dimitrios Tsarapatsanis, Nikolaos Aletras, Ion Androutsopoulos, and Prodromos Malakasiotis. 2021. Paragraph-level rationale extraction through regularization: A case study on European Court of Human Rights Cases. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 226\u2013241. DOI:10.18653\/v1\/2021.naacl-main.22"},{"key":"e_1_3_2_29_2","first-page":"4310","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics)","author":"Chalkidis Ilias","year":"2022","unstructured":"Ilias Chalkidis, Abhik Jana, Dirk Hartung, Michael Bommarito, Ion Androutsopoulos, Daniel Katz, and Nikolaos Aletras. 2022. LexGLUE: A benchmark dataset for legal language understanding in English. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics). Association for Computational Linguistics, 4310\u20134330. DOI:10.18653\/v1\/2022.acl-long.297"},{"key":"e_1_3_2_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-018-9238-9"},{"key":"e_1_3_2_31_2","first-page":"4389","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics","author":"Chalkidis Ilias","year":"2022","unstructured":"Ilias Chalkidis, Tommaso Pasini, Sheng Zhang, Letizia Tomada, Sebastian Schwemer, and Anders S\u00f8gaard. 2022. FairLex: A multilingual benchmark for evaluating fairness in legal text processing. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 4389\u20134406. Retrieved from https:\/\/aclanthology.org\/2022.acl-long.301"},{"key":"e_1_3_2_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3503157"},{"issue":"3","key":"e_1_3_2_33_2","doi-asserted-by":"crossref","first-page":"1067","DOI":"10.1017\/S000305542100143X","article-title":"Ethnic bias in judicial decision making: Evidence from criminal appeals in Kenya","volume":"116","author":"Choi Donghyun Danny","year":"2022","unstructured":"Donghyun Danny Choi, J. Andrew Harris, and Fiona Shen-Bayh. 2022. Ethnic bias in judicial decision making: Evidence from criminal appeals in Kenya. Am. Polit. Sci. Rev. 116, 3 (2022), 1067\u20131080.","journal-title":"Am. Polit. Sci. Rev."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324922000481"},{"key":"e_1_3_2_35_2","first-page":"4171","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 4171\u20134186. DOI:10.18653\/v1\/N19-1423"},{"key":"e_1_3_2_36_2","first-page":"23","volume-title":"Proceedings of the Artificial Intelligence and Cloud Computing Conference (AICCC\u201918)","author":"Elnaggar Ahmed","year":"2018","unstructured":"Ahmed Elnaggar, Robin Otto, and Florian Matthes. 2018. Deep learning for named-entity linking with transfer learning for legal documents. In Proceedings of the Artificial Intelligence and Cloud Computing Conference (AICCC\u201918). Association for Computing Machinery, New York, NY, 23\u201328. DOI:10.1145\/3299819.3299846"},{"key":"e_1_3_2_37_2","first-page":"115","volume-title":"Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data (HYBRID\u201912)","author":"Galgani Filippo","year":"2012","unstructured":"Filippo Galgani, Paul Compton, and Achim Hoffmann. 2012. Combining different summarization techniques for legal text. In Proceedings of the Workshop on Innovative Hybrid Approaches to the Processing of Textual Data (HYBRID\u201912). Association for Computational Linguistics, 115\u2013123."},{"key":"e_1_3_2_38_2","first-page":"60","volume-title":"Proceedings of the Workshop on Widening NLP","author":"Gonen Hila","year":"2019","unstructured":"Hila Gonen and Yoav Goldberg. 2019. Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. In Proceedings of the Workshop on Widening NLP. Association for Computational Linguistics, 60\u201363. Retrieved from https:\/\/aclanthology.org\/W19-3621"},{"key":"e_1_3_2_39_2","first-page":"1","article-title":"Thirty years of artificial intelligence and law: The first decade","author":"Governatori Guido","year":"2022","unstructured":"Guido Governatori, Trevor Bench-Capon, Bart Verheij, Micha\u0142 Araszkiewicz, Enrico Francesconi, and Matthias Grabmair. 2022. Thirty years of artificial intelligence and law: The first decade. Artif. Intell. Law (2022), 1\u201339.","journal-title":"Artif. Intell. Law"},{"key":"e_1_3_2_40_2","doi-asserted-by":"publisher","DOI":"10.1037\/0022-3514.74.6.1464"},{"key":"e_1_3_2_41_2","doi-asserted-by":"publisher","DOI":"10.1023\/A:1019516031847"},{"key":"e_1_3_2_42_2","article-title":"DeBERTa: Decoding-enhanced BERT with disentangled attention","volume":"2006","author":"He Pengcheng","year":"2020","unstructured":"Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. DeBERTa: Decoding-enhanced BERT with disentangled attention. CoRR abs\/2006.03654 (2020).","journal-title":"CoRR"},{"key":"e_1_3_2_43_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_2_44_2","article-title":"Bidirectional LSTM-CRF models for sequence tagging","author":"Huang Zhiheng","year":"2015","unstructured":"Zhiheng Huang, Wei Xu, and Kai Yu. 2015. Bidirectional LSTM-CRF models for sequence tagging. arXiv preprint arXiv:1508.01991 (2015).","journal-title":"arXiv preprint arXiv:1508.01991"},{"key":"e_1_3_2_45_2","doi-asserted-by":"publisher","DOI":"10.1145\/3310254"},{"key":"e_1_3_2_46_2","doi-asserted-by":"crossref","first-page":"1641","DOI":"10.18653\/v1\/P19-1160","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Kaneko Masahiro","year":"2019","unstructured":"Masahiro Kaneko and Danushka Bollegala. 2019. Gender-preserving debiasing for pre-trained word embeddings. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1641\u20131650. DOI:10.18653\/v1\/P19-1160"},{"key":"e_1_3_2_47_2","first-page":"1256","volume-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics","author":"Kaneko Masahiro","year":"2021","unstructured":"Masahiro Kaneko and Danushka Bollegala. 2021. Debiasing pre-trained contextualised embeddings. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 1256\u20131266. DOI:10.18653\/v1\/2021.eacl-main.107"},{"key":"e_1_3_2_48_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0174698"},{"key":"e_1_3_2_49_2","doi-asserted-by":"crossref","first-page":"43","DOI":"10.18653\/v1\/S18-2005","volume-title":"Proceedings of the 7th Joint Conference on Lexical and Computational Semantics (SEM@NAACL-HLT\u201918)","author":"Kiritchenko Svetlana","year":"2018","unstructured":"Svetlana Kiritchenko and Saif Mohammad. 2018. Examining gender and race bias in two hundred sentiment analysis systems. In Proceedings of the 7th Joint Conference on Lexical and Computational Semantics (SEM@NAACL-HLT\u201918), Malvina Nissim, Jonathan Berant, and Alessandro Lenci (Eds.). Association for Computational Linguistics, 43\u201353. DOI:10.18653\/v1\/s18-2005"},{"key":"e_1_3_2_50_2","doi-asserted-by":"crossref","first-page":"166","DOI":"10.18653\/v1\/W19-3823","volume-title":"Proceedings of the 1st Workshop on Gender Bias in Natural Language Processing","author":"Kurita Keita","year":"2019","unstructured":"Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W. Black, and Yulia Tsvetkov. 2019. Measuring bias in contextualized word representations. In Proceedings of the 1st Workshop on Gender Bias in Natural Language Processing. Association for Computational Linguistics, 166\u2013172. DOI:10.18653\/v1\/W19-3823"},{"key":"e_1_3_2_51_2","first-page":"17","volume-title":"Proceedings of the 8th International Conference on Learning Representations (ICLR\u201920)","author":"Lan Zhenzhong","year":"2020","unstructured":"Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2020. ALBERT: A lite BERT for self-supervised learning of language representations. In Proceedings of the 8th International Conference on Learning Representations (ICLR\u201920). OpenReview.net, 17 pages. Retrieved from https:\/\/openreview.net\/forum?id=H1eA7AEtvS"},{"key":"e_1_3_2_52_2","doi-asserted-by":"crossref","first-page":"117","DOI":"10.1007\/s10506-019-09243-2","article-title":"CLAUDETTE: An automated detector of potentially unfair clauses in online terms of service","volume":"27","author":"Lippi Marco","year":"2019","unstructured":"Marco Lippi, Przemyslaw Palka, Giuseppe Contissa, Francesca Lagioia, Hans Wolfgang Micklitz, Giovanni Sartor, and Paolo Torroni. 2019. CLAUDETTE: An automated detector of potentially unfair clauses in online terms of service. Artif. Intell. Law 27 (2019), 117\u2013139.","journal-title":"Artif. Intell. Law"},{"key":"e_1_3_2_53_2","article-title":"A survey on contextual embeddings","volume":"2003","author":"Liu Qi","year":"2020","unstructured":"Qi Liu, Matt J. Kusner, and Phil Blunsom. 2020. A survey on contextual embeddings. CoRR abs\/2003.07278 (2020).","journal-title":"CoRR"},{"key":"e_1_3_2_54_2","article-title":"RoBERTa: A robustly optimized BERT pretraining approach","volume":"1907","author":"Liu Yinhan","year":"2019","unstructured":"Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A robustly optimized BERT pretraining approach. CoRR abs\/1907.11692 (2019).","journal-title":"CoRR"},{"key":"e_1_3_2_55_2","first-page":"615","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Manzini Thomas","year":"2019","unstructured":"Thomas Manzini, Lim Yao Chong, Alan W. Black, and Yulia Tsvetkov. 2019. Black is to criminal as caucasian is to police: Detecting and removing multiclass bias in word embeddings. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 615\u2013621. DOI:10.18653\/v1\/N19-1062"},{"key":"e_1_3_2_56_2","doi-asserted-by":"publisher","DOI":"10.1017\/S1537592704040502"},{"key":"e_1_3_2_57_2","first-page":"5266","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP\u201919)","author":"Maudslay Rowan Hall","year":"2019","unstructured":"Rowan Hall Maudslay, Hila Gonen, Ryan Cotterell, and Simone Teufel. 2019. It\u2019s all in the name: Mitigating gender bias with name-based counterfactual data substitution. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP\u201919), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, 5266\u20135274. DOI:10.18653\/v1\/D19-1530"},{"key":"e_1_3_2_58_2","first-page":"622","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"May Chandler","year":"2019","unstructured":"Chandler May, Alex Wang, Shikha Bordia, Samuel R. Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 622\u2013628. DOI:10.18653\/v1\/N19-1063"},{"key":"e_1_3_2_59_2","volume-title":"Proceedings of the 1st International Conference on Learning Representations (ICLR\u201913)","author":"Mikolov Tomas","year":"2013","unstructured":"Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of the 1st International Conference on Learning Representations (ICLR\u201913), Yoshua Bengio and Yann LeCun (Eds.). Retrieved from http:\/\/arxiv.org\/abs\/1301.3781"},{"key":"e_1_3_2_60_2","doi-asserted-by":"publisher","DOI":"10.1145\/3524887"},{"key":"e_1_3_2_61_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2021.102684"},{"issue":"2","key":"e_1_3_2_62_2","first-page":"10","article-title":"Biases in large language models: Origins, inventory, and discussion","volume":"15","author":"Navigli Roberto","year":"2023","unstructured":"Roberto Navigli, Simone Conia, and Bj\u00f6rn Ross. 2023. Biases in large language models: Origins, inventory, and discussion. J. Data Inf. Qual. 15, 2, Article 10 (June 2023), 21 pages.","journal-title":"J. Data Inf. Qual."},{"key":"e_1_3_2_63_2","first-page":"159","volume-title":"Proceedings of the 16th Edition of the International Conference on Articial Intelligence and Law (ICAIL\u201917)","author":"Neill James O\u2019","year":"2017","unstructured":"James O\u2019 Neill, Paul Buitelaar, Cecile Robin, and Leona O\u2019 Brien. 2017. Classifying sentential modality in legal language: A use case in financial regulations, acts and directives. In Proceedings of the 16th Edition of the International Conference on Articial Intelligence and Law (ICAIL\u201917). Association for Computing Machinery, New York, NY, 159\u2013168. DOI:10.1145\/3086512.3086528"},{"key":"e_1_3_2_64_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-018-9225-1"},{"key":"e_1_3_2_65_2","volume-title":"Proceedings of the 27th AIAI Irish Conference on Artificial Intelligence and Cognitive Science","author":"O\u2019Sullivan Conor","year":"2019","unstructured":"Conor O\u2019Sullivan and Joeran Beel. 2019. Predicting the outcome of judicial decisions made by the European Court of Human Rights. In Proceedings of the 27th AIAI Irish Conference on Artificial Intelligence and Cognitive Science."},{"key":"e_1_3_2_66_2","first-page":"1532","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201914)","author":"Pennington Jeffrey","year":"2014","unstructured":"Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP\u201914). Association for Computational Linguistics, 1532\u20131543. DOI:10.3115\/v1\/D14-1162"},{"key":"e_1_3_2_67_2","first-page":"2227","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Peters Matthew","year":"2018","unstructured":"Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word representations. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 2227\u20132237. DOI:10.18653\/v1\/N18-1202"},{"key":"e_1_3_2_68_2","article-title":"Language models are unsupervised multitask learners","author":"Radford Alec","year":"2019","unstructured":"Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.","journal-title":"OpenAI Blog 1, 8 (2019), 9."},{"key":"e_1_3_2_69_2","article-title":"DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter","volume":"1910","author":"Sanh Victor","year":"2019","unstructured":"Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter. CoRR abs\/1910.01108 (2019).","journal-title":"CoRR"},{"key":"e_1_3_2_70_2","first-page":"1","article-title":"Thirty years of artificial intelligence and law: The second decade","author":"Sartor Giovanni","year":"2022","unstructured":"Giovanni Sartor, Micha\u0142 Araszkiewicz, Katie Atkinson, Floris Bex, Tom van Engers, Enrico Francesconi, Henry Prakken, Giovanni Sileno, Frank Schilder, Adam Wyner, et\u00a0al. 2022. Thirty years of artificial intelligence and law: The second decade. Artif. Intell. Law (2022), 1\u201337.","journal-title":"Artif. Intell. Law"},{"key":"e_1_3_2_71_2","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324922000122"},{"key":"e_1_3_2_72_2","doi-asserted-by":"publisher","DOI":"10.1145\/3569929"},{"issue":"11","key":"e_1_3_2_73_2","doi-asserted-by":"crossref","first-page":"1297","DOI":"10.1093\/jamia\/ocz096","article-title":"Enhancing clinical concept extraction with contextual embeddings","volume":"26","author":"Si Yuqi","year":"2019","unstructured":"Yuqi Si, Jingqi Wang, Hua Xu, and Kirk Roberts. 2019. Enhancing clinical concept extraction with contextual embeddings. J. Am. Med. Inform. Assoc. 26, 11 (2019), 1297\u20131304.","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"e_1_3_2_74_2","first-page":"67","volume-title":"Proceedings of the Natural Legal Language Processing Workshop","author":"Soh Jerrold","year":"2019","unstructured":"Jerrold Soh, How Khang Lim, and Ian Ernst Chai. 2019. Legal area classification: A comparative study of text classifiers on Singapore Supreme Court Judgments. In Proceedings of the Natural Legal Language Processing Workshop. Association for Computational Linguistics, 67\u201377. DOI:10.18653\/v1\/W19-2208"},{"key":"e_1_3_2_75_2","unstructured":"Harold J. Spaeth Lee Epstein Andrew D. Martin Jeffrey A. Segal Theodore J. Ruger and Sara C. Benesh. 2020. 2020 Supreme court database version 2021 release 01. Retrieved from http:\/\/Supremecourtdatabase.org"},{"key":"e_1_3_2_76_2","doi-asserted-by":"crossref","first-page":"1679","DOI":"10.18653\/v1\/P19-1164","volume-title":"Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics","author":"Stanovsky Gabriel","year":"2019","unstructured":"Gabriel Stanovsky, Noah A. Smith, and Luke Zettlemoyer. 2019. Evaluating gender bias in machine translation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 1679\u20131684. DOI:10.18653\/v1\/P19-1164"},{"key":"e_1_3_2_77_2","first-page":"13230","volume-title":"Advances in Neural Information Processing Systems","author":"Tan Yi Chern","year":"2019","unstructured":"Yi Chern Tan and L. Elisa Celis. 2019. Assessing social and intersectional biases in contextualized word representations. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d\u2019Alch\u00e9 Buc, E. Fox, and R. Garnett (Eds.), Vol. 32. Curran Associates, Inc., 13230\u201313241."},{"key":"e_1_3_2_78_2","doi-asserted-by":"publisher","DOI":"10.1145\/3134734"},{"key":"e_1_3_2_79_2","first-page":"1235","volume-title":"Proceedings of the 12th Language Resources and Evaluation Conference","author":"Tuggener Don","year":"2020","unstructured":"Don Tuggener, Pius von D\u00e4niken, Thomas Peetz, and Mark Cieliebak. 2020. LEDGAR: A large-scale multi-label corpus for text classification of legal provisions in contracts. In Proceedings of the 12th Language Resources and Evaluation Conference. European Language Resources Association, 1235\u20131241. Retrieved from https:\/\/aclanthology.org\/2020.lrec-1.155"},{"key":"e_1_3_2_80_2","volume-title":"Crime In the United States, 2019","author":"UCR FBI:","year":"2019","unstructured":"FBI: UCR. 2019. Crime In the United States, 2019. FBI. Retrieved from https:\/\/ucr.fbi.gov\/crime-in-the-u.s\/2019\/crime-in-the-u.s.-2019\/topic-pages\/tables\/table-42"},{"key":"e_1_3_2_81_2","volume-title":"Advances in Neural Information Processing Systems","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, \u0141ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.), Vol. 30. Curran Associates, Inc. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2017\/file\/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf"},{"key":"e_1_3_2_82_2","doi-asserted-by":"publisher","DOI":"10.1109\/TCSS.2021.3068519"},{"key":"e_1_3_2_83_2","first-page":"1","article-title":"Thirty years of artificial intelligence and law: The third decade","author":"Villata Serena","year":"2022","unstructured":"Serena Villata, Michal Araszkiewicz, Kevin Ashley, Trevor Bench-Capon, L. Karl Branting, Jack G. Conrad, and Adam Wyner. 2022. Thirty years of artificial intelligence and law: The third decade. Artif. Intell. Law (2022), 1\u201331.","journal-title":"Artif. Intell. Law"},{"key":"e_1_3_2_84_2","doi-asserted-by":"publisher","DOI":"10.1145\/3551390"},{"key":"e_1_3_2_85_2","doi-asserted-by":"crossref","first-page":"353","DOI":"10.18653\/v1\/W18-5446","volume-title":"Proceedings of the EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP","author":"Wang Alex","year":"2018","unstructured":"Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP. Association for Computational Linguistics, 353\u2013355. DOI:10.18653\/v1\/W18-5446"},{"key":"e_1_3_2_86_2","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00240"},{"key":"e_1_3_2_87_2","doi-asserted-by":"publisher","DOI":"10.1145\/3469887"},{"key":"e_1_3_2_88_2","first-page":"17283","volume-title":"Advances in Neural Information Processing Systems","author":"Zaheer Manzil","year":"2020","unstructured":"Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago Ontanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. 2020. Big Bird: Transformers for longer sequences. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 17283\u201317297. Retrieved from https:\/\/proceedings.neurips.cc\/paper\/2020\/file\/c8512d142a2d849725f31a9a7a361ab9-Paper.pdf"},{"key":"e_1_3_2_89_2","doi-asserted-by":"publisher","DOI":"10.1145\/3485244"},{"key":"e_1_3_2_90_2","first-page":"15","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Zhao Jieyu","year":"2018","unstructured":"Jieyu Zhao, TianluWang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 15\u201320."},{"key":"e_1_3_2_91_2","first-page":"629","volume-title":"Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","author":"Zhao Jieyu","year":"2019","unstructured":"Jieyu Zhao, Tianlu Wang, Mark Yatskar, Ryan Cotterell, Vicente Ordonez, and Kai-Wei Chang. 2019. Gender bias in contextualized word embeddings. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, 629\u2013634. DOI:10.18653\/v1\/N19-1064"},{"key":"e_1_3_2_92_2","first-page":"2979","volume-title":"Proceedings of the Conference on Empirical Methods in Natural Language Processing","author":"Zhao Jieyu","year":"2017","unstructured":"Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 2979\u20132989. DOI:10.18653\/v1\/D17-1323"},{"key":"e_1_3_2_93_2","first-page":"159","volume-title":"Proceedings of the 18th International Conference on Artificial Intelligence and Law (ICAIL\u201921)","author":"Zheng Lucia","year":"2021","unstructured":"Lucia Zheng, Neel Guha, Brandon R. Anderson, Peter Henderson, and Daniel E. Ho. 2021. When does pretraining help? Assessing self-supervised learning for law and the casehold dataset of 53,000+ legal holdings. In Proceedings of the 18th International Conference on Artificial Intelligence and Law (ICAIL\u201921). Association for Computing Machinery, New York, NY, 159\u2013168. DOI:10.1145\/3462757.3466088"},{"key":"e_1_3_2_94_2","first-page":"159","volume-title":"Proceedings of the 18th International Conference on Artificial Intelligence and Law (ICAIL\u201921)","author":"Zheng Lucia","year":"2021","unstructured":"Lucia Zheng, Neel Guha, Brandon R. Anderson, Peter Henderson, and Daniel E. Ho. 2021. When does pretraining help? Assessing self-supervised learning for law and the casehold dataset of 53,000+ legal holdings. In Proceedings of the 18th International Conference on Artificial Intelligence and Law (ICAIL\u201921). Association for Computing Machinery, New York, NY, 159\u2013168. DOI:10.1145\/3462757.3466088"},{"key":"e_1_3_2_95_2","doi-asserted-by":"crossref","first-page":"5218","DOI":"10.18653\/v1\/2020.acl-main.466","volume-title":"Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics","author":"Zhong Haoxi","year":"2020","unstructured":"Haoxi Zhong, Chaojun Xiao, Cunchao Tu, Tianyang Zhang, Zhiyuan Liu, and Maosong Sun. 2020. How does NLP benefit legal system: A summary of legal artificial intelligence. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, 5218\u20135230. DOI:10.18653\/v1\/2020.acl-main.466"},{"key":"e_1_3_2_96_2","doi-asserted-by":"publisher","DOI":"10.1038\/d41586-018-05707-8"},{"key":"e_1_3_2_97_2","doi-asserted-by":"publisher","DOI":"10.1017\/S1351324922000304"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3628602","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3628602","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T00:03:44Z","timestamp":1750291424000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3628602"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,13]]},"references-count":96,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2024,5,31]]}},"alternative-id":["10.1145\/3628602"],"URL":"https:\/\/doi.org\/10.1145\/3628602","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,13]]},"assertion":[{"value":"2022-12-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-12","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-02-13","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}