{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,21]],"date-time":"2026-11-21T11:14:25Z","timestamp":1795259665062,"version":"4.10.3"},"reference-count":83,"publisher":"SAGE Publications","license":[{"start":{"date-parts":[[2026,11,21]],"date-time":"2026-11-21T00:00:00Z","timestamp":1795219200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"},{"start":{"date-parts":[[2026,11,21]],"date-time":"2026-11-21T00:00:00Z","timestamp":1795219200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Journal of Law &amp; Empirical Analysis"],"abstract":"<jats:p>This paper presents ELARA (Empirical Legal Analysis with Research Agents), a framework for using large language models (LLMs) as agents to conduct empirical legal research. ELARA introduces a new method to generate quantitative data from text (like judicial opinions, contracts, or statutes) using sub-agents. I show how ELARA produces results similar to conventional API calls on a variety of representative tasks, at approximately one eighteenth the cost. In addition, ELARA assists with every step of a real project: framing a research question, gathering data, conducting analysis, validating LLM work, writing up results, and building a replication package. For each of these steps, the appendix includes copyable prompt templates and a companion ELARA repository for use with Claude Code and Codex.<\/jats:p>","DOI":"10.1177\/2755323x261498461","type":"journal-article","created":{"date-parts":[[2026,11,21]],"date-time":"2026-11-21T10:22:48Z","timestamp":1795256568000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["ELARA: A Framework for Empirical Legal Research With AI Agents"],"prefix":"10.1177","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-1647-7463","authenticated-orcid":false,"given":"Jonathan H.","family":"Choi","sequence":"first","affiliation":[{"name":"School of Law, Washington University, St. Louis, Missouri, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2026,11,21]]},"reference":[{"key":"e_1_3_7_2_1","doi-asserted-by":"crossref","unstructured":"Abrams D. S. Choi J. H. (2025). Prose and cons: Evaluating the legality of police stops with large language models. Working Paper.","DOI":"10.2139\/ssrn.6210681"},{"key":"e_1_3_7_3_1","unstructured":"Adelson P. Nyarko J. (2025). The material contracts corpus. Working paper arXiv:2504.02864."},{"key":"e_1_3_7_4_1","doi-asserted-by":"publisher","DOI":"10.3386\/w35188"},{"key":"e_1_3_7_5_1","doi-asserted-by":"publisher","DOI":"10.52202\/079017-2447"},{"key":"e_1_3_7_6_1","unstructured":"Alizadeh M. Gilardi F. Mosleh M. Kasneci E. (2026a). AI coding agents in social science: Methodologically diverse empirically consistent interpretively vulnerable. arXiv:2606.11456."},{"key":"e_1_3_7_7_1","unstructured":"Alizadeh M. Mosleh M. Gilardi F. Kasirzadeh A. Tucker J. (2026b). AI coding agents can reproduce social science findings. arXiv:2606.11447."},{"key":"e_1_3_7_8_1","doi-asserted-by":"publisher","DOI":"10.1198\/016214508000000841"},{"key":"e_1_3_7_9_1","doi-asserted-by":"publisher","DOI":"10.3386\/w23544"},{"key":"e_1_3_7_10_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.adi6000"},{"key":"e_1_3_7_11_1","unstructured":"Anthropic. (2025). Prompt engineering overview. https:\/\/docs.anthropic.com\/en\/docs\/build-with-claude\/prompt-engineering\/overview (Accessed 29 August 2025)."},{"key":"e_1_3_7_12_1","unstructured":"Anthropic. (2026). Prompting best practices. https:\/\/platform.claude.com\/docs\/en\/build-with-claude\/prompt-engineering\/claude-prompting-best-practices (Accessed 15 October 2026)."},{"issue":"2","key":"e_1_3_7_13_1","first-page":"451","article-title":"Generative interpretation","volume":"99","author":"Arbel Y.","year":"2024","unstructured":"Arbel Y., Hoffman D. A. (2024). Generative interpretation. New York University Law Review, 99(2), 451\u2013514.","journal-title":"New York University Law Review"},{"key":"e_1_3_7_14_1","doi-asserted-by":"publisher","DOI":"10.1017\/pan.2023.2"},{"key":"e_1_3_7_15_1","doi-asserted-by":"publisher","DOI":"10.3386\/w34834"},{"key":"e_1_3_7_16_1","unstructured":"Barrie C. Palaiologou E. T\u00f6rnberg P. (2024a). Prompt stability scoring for text annotation with large language models. arXiv:2407.02039."},{"key":"e_1_3_7_17_1","volume-title":"Replication for language models: Problems, principles, and best practice for political science","author":"Barrie C.","year":"2024","unstructured":"Barrie C., Palmer A., Spirling A. (2024b). Replication for language models: Problems, principles, and best practice for political science. New York University. Working paper."},{"key":"e_1_3_7_18_1","doi-asserted-by":"crossref","unstructured":"Battaglia L. Christensen T. Hansen S. Sacher S. (2025). Inference for regression with variables generated by ai or machine learning. arXiv preprint.","DOI":"10.2139\/ssrn.4842086"},{"key":"e_1_3_7_19_1","unstructured":"Baumann J. R\u00f6ttger P. Urman A. Wendsj\u00f6 A. Plaza\u2010del\u2010Arco F. M. Gruber J. B. Hovy D. (2025). Large language model hacking: Quantifying the hidden risks of using LLMs for text annotation. arXiv:2509.08825."},{"key":"e_1_3_7_20_1","doi-asserted-by":"crossref","unstructured":"Blair-Stanek A. Van Durme B. (2025). LLMs provide unstable answers to legal questions. arXiv:2502.05196.","DOI":"10.1145\/3769126.3769245"},{"key":"e_1_3_7_21_1","doi-asserted-by":"publisher","DOI":"10.1257\/aer.20190687"},{"key":"e_1_3_7_22_1","doi-asserted-by":"crossref","unstructured":"Calderon N. Reichart R. Dror R. (2025). The alternative annotator test for LLM-as-a-judge: How to statistically justify replacing human annotators with LLMs. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 16051\u201316081). Association for Computational Linguistics. https:\/\/doi.org\/10.18653\/v1\/2025.acl-long.782","DOI":"10.18653\/v1\/2025.acl-long.782"},{"key":"e_1_3_7_23_1","unstructured":"Carlson J. Dell M. (2025). A unifying framework for robust and efficient inference with unstructured data. arXiv:2505.00282."},{"key":"e_1_3_7_24_1","doi-asserted-by":"publisher","DOI":"10.52202\/079017-4320"},{"key":"e_1_3_7_25_1","doi-asserted-by":"publisher","DOI":"10.1628\/jite-2024-0006"},{"key":"e_1_3_7_26_1","unstructured":"Choi J. H. Mendelson N. A. (2027). Canons and the court. Yale Law Journal 136 (forthcoming)."},{"key":"e_1_3_7_27_1","doi-asserted-by":"crossref","unstructured":"Choi J. H. Connell P. (2024). Estimating and correcting for misclassification error in empirical textual research. Available at SSRN: https:\/\/ssrn.com\/abstract=4913179 or https:\/\/doi.org\/10.2139\/ssrn.4913179","DOI":"10.2139\/ssrn.4913179"},{"key":"e_1_3_7_28_1","doi-asserted-by":"publisher","DOI":"10.1257\/jel.20171350"},{"key":"e_1_3_7_29_1","doi-asserted-by":"publisher","DOI":"10.1177\/1536867X20976314"},{"key":"e_1_3_7_30_1","doi-asserted-by":"crossref","unstructured":"Dahl C. M. Johansen T. Vedel C. (2026). Breaking the HISCO barrier: Automatic occupational standardization with OccCANINE. arXiv:2402.13604v3.","DOI":"10.2139\/ssrn.6479788"},{"key":"e_1_3_7_31_1","doi-asserted-by":"publisher","DOI":"10.1093\/jla\/laae003"},{"key":"e_1_3_7_32_1","volume-title":"International conference on learning representations (ICLR)","author":"Dominguez-Olmedo R.","year":"2025","unstructured":"Dominguez-Olmedo R., Nanda V., Abebe R., Bechtold S., Engel C., Frankenreiter J., Gummadi K., Hardt M., Livermore M. A. (2025). Lawma: The power of specialization for legal annotation. In International conference on learning representations (ICLR)."},{"key":"e_1_3_7_33_1","doi-asserted-by":"publisher","DOI":"10.52202\/075280-3000"},{"issue":"2","key":"e_1_3_7_34_1","first-page":"235","article-title":"Asking GPT for the ordinary meaning of statutory terms","volume":"2024","author":"Engel C.","year":"2024","unstructured":"Engel C., McAdams R. H. (2024). Asking GPT for the ordinary meaning of statutory terms. University of Illinois Journal of Law, Technology & Policy, 2024(2), 235\u2013296.","journal-title":"University of Illinois Journal of Law, Technology & Policy"},{"key":"e_1_3_7_35_1","doi-asserted-by":"publisher","DOI":"10.1017\/pan.2017.22"},{"key":"e_1_3_7_36_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00511"},{"key":"e_1_3_7_37_1","doi-asserted-by":"publisher","DOI":"10.3386\/w35374"},{"key":"e_1_3_7_38_1","doi-asserted-by":"crossref","unstructured":"Gao R. Xiao S. C. (2026). Nonstandard errors in AI agents. arXiv:2603.16744.","DOI":"10.2139\/ssrn.6427518"},{"key":"e_1_3_7_39_1","unstructured":"Gelman A. Loken E. (2013). The garden of forking paths: Why multiple comparisons can be a problem even when there is no \u201cfishing expedition\u201d or \u201cp-hacking\u201d and the research hypothesis was posited ahead of time. Working paper. Department of Statistics Columbia University."},{"key":"e_1_3_7_40_1","series-title":"Long Papers)","first-page":"3514","volume-title":"Human Language Technologies","author":"Gligori\u0107 K.","year":"2025","unstructured":"Gligori\u0107 K., Zrnic T., Lee C., Cand\u00e8s E. J., Jurafsky D. (2025). Can unconfident LLM annotations be used for confident conclusions? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1:Long Papers) (pp. 3514\u20133533). Association for Computational Linguistics."},{"key":"e_1_3_7_41_1","volume-title":"International conference on learning representations (ICLR)","author":"Guo Q.","year":"2024","unstructured":"Guo Q., Wang R., Guo J., Li B., Song K., Tan X., Liu G., Bian J., Yang Y. (2024). Connecting large language models with evolutionary algorithms yields powerful prompt optimizers. In International conference on learning representations (ICLR)."},{"key":"e_1_3_7_42_1","doi-asserted-by":"publisher","DOI":"10.1017\/pan.2025.10017"},{"key":"e_1_3_7_43_1","unstructured":"Han D. Han M. Unsloth team. (2023). Unsloth. https:\/\/github.com\/unslothai\/unsloth (accessed 30 July 2025)."},{"key":"e_1_3_7_44_1","unstructured":"He H. (2025). Defeating nondeterminism in LLM inference. Thinking Machines Lab blog. https:\/\/thinkingmachines.ai\/blog\/defeating-nondeterminism-in-llm-inference\/"},{"key":"e_1_3_7_45_1","doi-asserted-by":"publisher","DOI":"10.3386\/w31122"},{"key":"e_1_3_7_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.lindif.2023.102274"},{"key":"e_1_3_7_47_1","article-title":"RCT rejection sampling for causal estimation evaluation","author":"Keith K. A.","year":"2023","unstructured":"Keith K. A., Feldman S., Jurgens D., Bragg J., Bhattacharya R. (2023). RCT rejection sampling for causal estimation evaluation. Transactions on Machine Learning Research.","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_3_7_48_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.474"},{"key":"e_1_3_7_49_1","unstructured":"Kmieliauskas L. (2026). This is how much your ChatGPT and Claude usage costs OpenAI and Anthropic. Cybernews. Reporting SemiAnalysis estimates. June 15 2026. https:\/\/cybernews.com\/ai-news\/chatgpt-claude-usage-costs-openai-anthropic\/ (accessed 8 August 2026)."},{"key":"e_1_3_7_50_1","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.5399463"},{"key":"e_1_3_7_51_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.92"},{"key":"e_1_3_7_52_1","unstructured":"Lin J. Diesendruck M. Du L. Abraham R. (2024). BatchPrompt: Accomplish more with less. International Conference on Learning Representations (ICLR 2024)."},{"key":"e_1_3_7_53_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10683-018-09597-5"},{"key":"e_1_3_7_54_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00638"},{"key":"e_1_3_7_55_1","doi-asserted-by":"publisher","DOI":"10.1145\/3560815"},{"key":"e_1_3_7_56_1","doi-asserted-by":"publisher","DOI":"10.1628\/jite-2024-0009"},{"key":"e_1_3_7_57_1","doi-asserted-by":"publisher","DOI":"10.1146\/annurev-economics-120925-105620"},{"key":"e_1_3_7_58_1","doi-asserted-by":"publisher","DOI":"10.1073\/pnas.1708274114"},{"key":"e_1_3_7_59_1","doi-asserted-by":"publisher","DOI":"10.1257\/jep.29.3.61"},{"key":"e_1_3_7_60_1","doi-asserted-by":"publisher","DOI":"10.1038\/s42256-023-00783-6"},{"key":"e_1_3_7_61_1","unstructured":"OpenAI. (2025a). Prompt engineering. https:\/\/platform.openai.com\/docs\/guides\/prompt-engineering (Accessed 29 August 2025)."},{"key":"e_1_3_7_62_1","unstructured":"OpenAI. (2025b). Prompt engineering: Six strategies for getting better results. https:\/\/platform.openai.com\/docs\/guides\/prompt-engineering\/six-strategies-for-getting-better-results (Accessed 29 August 2025)."},{"key":"e_1_3_7_63_1","volume-title":"\u201cUpdate API and model parameters\u201d section","author":"OpenAI","year":"2026","unstructured":"OpenAI. (2026). Using GPT-6 Astra. \u201cUpdate API and model parameters\u201d section. https:\/\/developers.openai.com\/api\/docs\/guides\/latest-model (Accessed 10 September 2026)."},{"key":"e_1_3_7_64_1","unstructured":"Pangakis N. Wolken S. Fasching N. (2023). Automated annotation with generative AI requires validation. arXiv:2306.00176."},{"key":"e_1_3_7_65_1","unstructured":"Parthasarathy V. B. Zafar A. Khan A. Shahid A. (2024). The ultimate guide to fine-tuning LLMs from basics to breakthroughs: An exhaustive review of technologies research best practices applied research challenges and opportunities. arXiv preprint arXiv:2408.13296."},{"key":"e_1_3_7_66_1","unstructured":"Pathak R. Cheung C. OpenAI. (2025). Gpt-5 prompt migration and improvement using the new optimizer. OpenAI Cookbook. https:\/\/cookbook.openai.com\/examples\/gpt-5\/prompt-optimization-cookbook (Accessed 29 August 2025)."},{"key":"e_1_3_7_67_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.findings-naacl.130"},{"key":"e_1_3_7_68_1","doi-asserted-by":"crossref","unstructured":"Reiss M. V. (2023). Testing the reliability of ChatGPT for text annotation and classification: A cautionary remark. arXiv:2304.11085.","DOI":"10.31219\/osf.io\/rvy5p"},{"key":"e_1_3_7_69_1","unstructured":"Runpod. (2026). Runpod documentation: Gpu cloud computing. https:\/\/docs.runpod.io (Accessed 1 July 2026)."},{"issue":"1","key":"e_1_3_7_70_1","first-page":"1","article-title":"AI tools for lawyers: A practical guide","volume":"108","author":"Schwarcz D.","year":"2023","unstructured":"Schwarcz D., Choi J. H. (2023). AI tools for lawyers: A practical guide. Minnesota Law Review Headnotes, 108(1), 1\u201339. Essay.","journal-title":"Minnesota Law Review Headnotes"},{"key":"e_1_3_7_71_1","volume-title":"Proceedings of the international conference on learning representations (ICLR)","author":"Sclar M.","year":"2024","unstructured":"Sclar M., Choi Y., Tsvetkov Y., Suhr A. (2024). Quantifying language models' sensitivity to spurious features in prompt design or: How I learned to start worrying about prompt formatting. In Proceedings of the international conference on learning representations (ICLR)."},{"key":"e_1_3_7_72_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-emnlp.490"},{"key":"e_1_3_7_73_1","unstructured":"Si C. Yang D. Hashimoto T. (2025). Can LLMs generate novel research ideas? A large-scale human study with 100+ NLP researchers. In International Conference on Learning Representations (ICLR 2025)."},{"key":"e_1_3_7_74_1","doi-asserted-by":"publisher","DOI":"10.1177\/0956797611417632"},{"key":"e_1_3_7_75_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41562-020-0912-z"},{"key":"e_1_3_7_76_1","doi-asserted-by":"publisher","DOI":"10.1177\/1745691616658637"},{"key":"e_1_3_7_77_1","unstructured":"Together AI (2025). Together AI fine-tuning documentation. https:\/\/docs.together.ai\/docs\/fine-tuning-overview (accessed 30 July 2025)."},{"issue":"2","key":"e_1_3_7_78_1","first-page":"67","article-title":"Best practices for text annotation with large language models","volume":"18","author":"T\u00f6rnberg P.","year":"2024","unstructured":"T\u00f6rnberg P. (2024). Best practices for text annotation with large language models. Sociologica, 18(2), 67\u201385. https:\/\/doi.org\/10.6092\/issn.1971-8853\/19461","journal-title":"Sociologica"},{"key":"e_1_3_7_79_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2024.naacl-short.71"},{"key":"e_1_3_7_80_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-023-41032-5"},{"key":"e_1_3_7_81_1","unstructured":"Zhang B. Liu Z. Cherry C. Firat O. (2024). When scaling meets LLM finetuning: The effect of data model and finetuning method. In Proceedings of the Twelfth International Conference on Learning Representations. https:\/\/proceedings.iclr.cc\/paper_files\/paper\/2024\/hash\/c2ad28981782bb62f025d2893791b629-Abstract-Conference.html"},{"key":"e_1_3_7_82_1","article-title":"LIMA: Less is more for alignment","author":"Zhou C.","year":"2023","unstructured":"Zhou C., Liu P., Xu P., Iyer S., Sun J., Mao Y., Ma X., Efrat A., Yu P., Yu L., Zhang S., Ghosh G., Lewis M., Zettlemoyer L., Levy O. (2023). LIMA: Less is more for alignment. In Advances in Neural Information Processing Systems (36, pp. 55006-55021). Curran Associates, Inc.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_7_83_1","article-title":"Large language models are human-level prompt engineers","author":"Zhou Y.","year":"2023","unstructured":"Zhou Y., Muresanu A. I., Han Z., Paster K., Pitis S., Chan H., Ba J. (2023). Large language models are human-level prompt engineers. International Conference on Learning Representations (ICLR 2023).","journal-title":"International Conference on Learning Representations (ICLR 2023)"},{"key":"e_1_3_7_84_1","doi-asserted-by":"publisher","DOI":"10.1162\/coli_a_00502"}],"container-title":["Journal of Law &amp; Empirical Analysis"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/2755323X261498461","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/2755323X261498461","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/2755323X261498461","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,11,21]],"date-time":"2026-11-21T10:22:55Z","timestamp":1795256575000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/2755323X261498461"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,11,21]]},"references-count":83,"alternative-id":["10.1177\/2755323X261498461"],"URL":"https:\/\/doi.org\/10.1177\/2755323x261498461","relation":{},"ISSN":["2755-323X","2755-323X"],"issn-type":[{"value":"2755-323X","type":"print"},{"value":"2755-323X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,11,21]]},"article-number":"2755323X261498461"}}