{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T22:33:44Z","timestamp":1768343624855,"version":"3.49.0"},"publisher-location":"New York, NY, USA","reference-count":26,"publisher":"ACM","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2025,6,16]]},"DOI":"10.1145\/3769126.3769128","type":"proceedings-article","created":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T14:50:37Z","timestamp":1768315837000},"page":"404-408","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Identifying Legal Holdings with LLMs: A Systematic Study of Performance, Scale, and Memorization"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-9077-2185","authenticated-orcid":false,"given":"Chuck","family":"Arvin","sequence":"first","affiliation":[{"name":"Gould School of Law, University of Southern California, Los Angeles, California, USA"}]}],"member":"320","published-online":{"date-parts":[[2026,1,13]]},"reference":[{"key":"e_1_3_3_2_2_2","unstructured":"Tom\u00a0B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger Tom Henighan Rewon Child Aditya Ramesh Daniel\u00a0M. Ziegler Jeffrey Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language Models are Few-Shot Learners. arxiv:https:\/\/arXiv.org\/abs\/2005.14165\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2005.14165"},{"key":"e_1_3_3_2_3_2","doi-asserted-by":"crossref","unstructured":"Ilias Chalkidis Abhik Jana Dirk Hartung Michael Bommarito Ion Androutsopoulos Daniel\u00a0Martin Katz and Nikolaos Aletras. 2022. LexGLUE: A Benchmark Dataset for Legal Language Understanding in English. arxiv:https:\/\/arXiv.org\/abs\/2110.00976\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2110.00976","DOI":"10.2139\/ssrn.3936759"},{"key":"e_1_3_3_2_4_2","unstructured":"Zhaopeng Feng Yan Zhang Hao Li Bei Wu Jiayu Liao Wenqiang Liu Jun Lang Yang Feng Jian Wu and Zuozhu Liu. 2024. TEaR: Improving LLM-based Machine Translation with Systematic Self-Refinement. arxiv:https:\/\/arXiv.org\/abs\/2402.16379\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2402.16379"},{"key":"e_1_3_3_2_5_2","doi-asserted-by":"crossref","unstructured":"Neel Guha Julian Nyarko Daniel Ho Christopher R\u00e9 Adam Chilton Alex Chohlas-Wood Austin Peters Brandon Waldon Daniel Rockmore Diego Zambrano et\u00a0al. 2024. Legalbench: A collaboratively built benchmark for measuring legal reasoning in large language models. Advances in Neural Information Processing Systems 36 (2024).","DOI":"10.2139\/ssrn.4583531"},{"key":"e_1_3_3_2_6_2","unstructured":"Valentin Hartmann Anshuman Suri Vincent Bindschaedler David Evans Shruti Tople and Robert West. 2023. SoK: Memorization in General-Purpose Large Language Models. arxiv:https:\/\/arXiv.org\/abs\/2310.18362\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2310.18362"},{"key":"e_1_3_3_2_7_2","unstructured":"Peter Henderson Mark\u00a0S. Krass Lucia Zheng Neel Guha Christopher\u00a0D. Manning Dan Jurafsky and Daniel\u00a0E. Ho. 2022. Pile of Law: Learning Responsible Data Filtering from the Law and a 256GB Open-Source Legal Dataset. arxiv:https:\/\/arXiv.org\/abs\/2207.00220\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2207.00220"},{"key":"e_1_3_3_2_8_2","unstructured":"Jared Kaplan Sam McCandlish Tom Henighan Tom\u00a0B. Brown Benjamin Chess Rewon Child Scott Gray Alec Radford Jeffrey Wu and Dario Amodei. 2020. Scaling Laws for Neural Language Models. arxiv:https:\/\/arXiv.org\/abs\/2001.08361\u00a0[cs.LG] https:\/\/arxiv.org\/abs\/2001.08361"},{"key":"e_1_3_3_2_9_2","unstructured":"Daniel\u00a0Martin Katz Dirk Hartung Lauritz Gerlach Abhik Jana and Michael J.\u00a0Bommarito II. 2023. Natural Language Processing in the Legal Domain. arxiv:https:\/\/arXiv.org\/abs\/2302.12039\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2302.12039"},{"key":"e_1_3_3_2_10_2","doi-asserted-by":"publisher","DOI":"10.1145\/3643795.3648384"},{"key":"e_1_3_3_2_11_2","unstructured":"Weixin Liang Yaohui Zhang Zhengxuan Wu Haley Lepp Wenlong Ji Xuandong Zhao Hancheng Cao Sheng Liu Siyu He Zhi Huang Diyi Yang Christopher Potts Christopher\u00a0D Manning and James\u00a0Y. Zou. 2024. Mapping the Increasing Use of LLMs in Scientific Papers. arxiv:https:\/\/arXiv.org\/abs\/2404.01268\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2404.01268"},{"key":"e_1_3_3_2_12_2","volume-title":"The Language of the Law","author":"Mellinkoff D.","year":"2004","unstructured":"D. Mellinkoff. 2004. The Language of the Law. Resource Publications. https:\/\/books.google.com\/books?id=YBRLAwAAQBAJ"},{"key":"e_1_3_3_2_13_2","unstructured":"Meta. 2024. Llama 3.2: Revolutionizing edge AI and vision with open customizable models. (2024). https:\/\/ai.meta.com\/blog\/llama-3-2-connect-2024-vision-edge-mobile-devices\/"},{"key":"e_1_3_3_2_14_2","doi-asserted-by":"crossref","unstructured":"Joel Niklaus Veton Matoshi Matthias Sturmer Ilias Chalkidis and Daniel\u00a0E. Ho. 2024. MultiLegalPile: A 689GB Multilingual Legal Corpus. arxiv:https:\/\/arXiv.org\/abs\/2306.02069\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2306.02069","DOI":"10.18653\/v1\/2024.acl-long.805"},{"key":"e_1_3_3_2_15_2","unstructured":"The State\u00a0Bar of California. 2024. PRACTICAL GUIDANCE FOR THE USE OF GENERATIVE ARTIFICIAL INTELLIGENCE IN THE PRACTICE OF LAW. https:\/\/www.calbar.ca.gov\/Portals\/0\/documents\/ethics\/Generative-AI-Practical-Guidance.pdf"},{"key":"e_1_3_3_2_16_2","unstructured":"OpenAI. 2024. GPT-4o System Card. arxiv:https:\/\/arXiv.org\/abs\/2410.21276\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2410.21276"},{"key":"e_1_3_3_2_17_2","unstructured":"Abel Salinas and Fred Morstatter. 2024. The Butterfly Effect of Altering Prompts: How Small Changes and Jailbreaks Affect Large Language Model Performance. arxiv:https:\/\/arXiv.org\/abs\/2401.03729\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2401.03729"},{"key":"e_1_3_3_2_18_2","unstructured":"Amazon\u00a0Web Services. 2024. Introducing Amazon Nova foundation models: Frontier intelligence and industry leading price performance. (2024). https:\/\/aws.amazon.com\/blogs\/aws\/introducing-amazon-nova-frontier-intelligence-and-industry-leading-price-performance\/"},{"key":"e_1_3_3_2_19_2","unstructured":"Zejiang Shen Kyle Lo Lauren Yu Nathan Dahlberg Margo Schlanger and Doug Downey. 2022. Multi-LexSum: Real-World Summaries of Civil Rights Lawsuits at Multiple Granularities. arxiv:https:\/\/arXiv.org\/abs\/2206.10883\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2206.10883"},{"key":"e_1_3_3_2_20_2","unstructured":"Maja Stahl Leon Biermann Andreas Nehring and Henning Wachsmuth. 2024. Exploring LLM Prompting Strategies for Joint Essay Scoring and Feedback Generation. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2404.15845 (2024)."},{"key":"e_1_3_3_2_21_2","unstructured":"The New\u00a0York Times. 2024. The New York Times vs Microsoft et al.https:\/\/nytco-assets.nytimes.com\/2023\/12\/NYT_Complaint_Dec2023.pdf"},{"key":"e_1_3_3_2_22_2","unstructured":"John Tredennick and Dr.\u00a0William Webber. 2024. An Introduction to Large Language Models for Ediscovery Professionals. MIT Computational Law Report (2024). https:\/\/law.mit.edu\/pub\/anintroductiontolargelanguagemodelsforediscoveryprofessionals\/release\/1"},{"key":"e_1_3_3_2_23_2","unstructured":"John Villasenor. 2024. Generative Artificial Intelligence and the Practice of Law: Impact Opportunities and Risks. Minnesota Journal of Law Science and Technology (2024). https:\/\/scholarship.law.umn.edu\/cgi\/viewcontent.cgi?article=1563&context=mjlst"},{"key":"e_1_3_3_2_24_2","unstructured":"Evan Wang Federico Cassano Catherine Wu Yunfeng Bai Will Song Vaskar Nath Ziwen Han Sean Hendryx Summer Yue and Hugh Zhang. 2024. Planning In Natural Language Improves LLM Search For Code Generation. arxiv:https:\/\/arXiv.org\/abs\/2409.03733\u00a0[cs.LG] https:\/\/arxiv.org\/abs\/2409.03733"},{"key":"e_1_3_3_2_25_2","unstructured":"Jason Wei Xuezhi Wang Dale Schuurmans Maarten Bosma Brian Ichter Fei Xia Ed Chi Quoc Le and Denny Zhou. 2023. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arxiv:https:\/\/arXiv.org\/abs\/2201.11903\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2201.11903"},{"key":"e_1_3_3_2_26_2","doi-asserted-by":"crossref","unstructured":"Lucia Zheng Neel Guha Brandon\u00a0R. Anderson Peter Henderson and Daniel\u00a0E. Ho. 2021. When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Dataset. arxiv:https:\/\/arXiv.org\/abs\/2104.08671\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2104.08671","DOI":"10.1145\/3462757.3466088"},{"key":"e_1_3_3_2_27_2","unstructured":"Wenhao Zhu Hongyi Liu Qingxiu Dong Jingjing Xu Shujian Huang Lingpeng Kong Jiajun Chen and Lei Li. 2024. Multilingual Machine Translation with Large Language Models: Empirical Results and Analysis. arxiv:https:\/\/arXiv.org\/abs\/2304.04675\u00a0[cs.CL] https:\/\/arxiv.org\/abs\/2304.04675"}],"event":{"name":"ICAIL 2025: 20th International Conference on Artificial Intelligence and Law","location":"Chicago , IL , USA","acronym":"ICAIL 2025"},"container-title":["Proceedings of the Twentieth International Conference on Artificial Intelligence and Law"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3769126.3769128","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T15:50:02Z","timestamp":1768319402000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3769126.3769128"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,6,16]]},"references-count":26,"alternative-id":["10.1145\/3769126.3769128","10.1145\/3769126"],"URL":"https:\/\/doi.org\/10.1145\/3769126.3769128","relation":{},"subject":[],"published":{"date-parts":[[2025,6,16]]},"assertion":[{"value":"2026-01-13","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}