{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,22]],"date-time":"2026-09-22T14:48:16Z","timestamp":1790088496065,"version":"4.0.1"},"reference-count":125,"publisher":"Wiley","issue":"2","license":[{"start":{"date-parts":[[2025,4,23]],"date-time":"2025-04-23T00:00:00Z","timestamp":1745366400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"},{"start":{"date-parts":[[2025,4,23]],"date-time":"2025-04-23T00:00:00Z","timestamp":1745366400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["J Empirical Legal Studies"],"published-print":{"date-parts":[[2025,6]]},"abstract":"<jats:title>ABSTRACT<\/jats:title>\n                  <jats:p>Legal practice has witnessed a sharp rise in products incorporating artificial intelligence (AI). Such tools are designed to assist with a wide range of core legal tasks, from search and summarization of caselaw to document drafting. However, the large language models used in these tools are prone to \u201challucinate,\u201d or make up false information, making their use risky in high\u2010stakes domains. Recently, certain legal research providers have touted methods such as retrieval\u2010augmented generation (RAG) as \u201celiminating\u201d or \u201cavoid[ing]\u201d hallucinations, or guaranteeing \u201challucination\u2010free\u201d legal citations. Because of the closed nature of these systems, systematically assessing these claims is challenging. In this article, we design and report on the first preregistered empirical evaluation of AI\u2010driven legal research tools. We demonstrate that the providers' claims are overstated. While hallucinations are reduced relative to general\u2010purpose chatbots (GPT\u20104), we find that the AI research tools made by LexisNexis (Lexis+ AI) and Thomson Reuters (Westlaw AI\u2010Assisted Research and Ask Practical Law AI) each hallucinate between 17% and 33% of the time. We also document substantial differences between systems in responsiveness and accuracy. Our article makes four key contributions. It is the first to assess and report the performance of RAG\u2010based proprietary legal AI tools. Second, it introduces a comprehensive, preregistered dataset for identifying and understanding vulnerabilities in these systems. Third, it proposes a clear typology for differentiating between hallucinations and accurate legal responses. Last, it provides evidence to inform the responsibilities of legal professionals in supervising and verifying AI outputs, which remains a central open question for the responsible integration of AI into law.<\/jats:p>","DOI":"10.1111\/jels.12413","type":"journal-article","created":{"date-parts":[[2025,4,23]],"date-time":"2025-04-23T19:25:22Z","timestamp":1745436322000},"page":"216-242","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":156,"title":["Hallucination\u2010Free? Assessing the Reliability of Leading\n                    <scp>AI<\/scp>\n                    Legal Research Tools"],"prefix":"10.1111","volume":"22","author":[{"given":"Varun","family":"Magesh","sequence":"first","affiliation":[{"name":"Regulation, Evaluation, and Governance Lab Stanford University  Stanford California USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Faiz","family":"Surani","sequence":"additional","affiliation":[{"name":"Regulation, Evaluation, and Governance Lab Stanford University  Stanford California USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Matthew","family":"Dahl","sequence":"additional","affiliation":[{"name":"Yale University  New Haven Connecticut USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mirac","family":"Suzgun","sequence":"additional","affiliation":[{"name":"Department of Computer Science, Stanford Law School, Regulation, Evaluation, and Governance Lab Stanford University  Stanford California USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Christopher D.","family":"Manning","sequence":"additional","affiliation":[{"name":"Department of Linguistics, Department of Computer Science, Stanford Artificial Intelligence Laboratory, Stanford Institute for Human\u2010Centered AI, Regulation, Evaluation, and Governance Lab Stanford University  Stanford California USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel E.","family":"Ho","sequence":"additional","affiliation":[{"name":"Stanford Law School, Department of Political Science, Department of Computer Science, Stanford Institute for Human\u2010Centered AI, Stanford Institute for Economic Policy Research, Regulation, Evaluation, and Governance Lab Stanford University  Stanford California USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2025,4,23]]},"reference":[{"key":"e_1_2_13_2_1","doi-asserted-by":"crossref","unstructured":"Agrawal A. M.Suzgun L.Mackey andA. T.Kalai.2023.\u201cDo Language Models Know When They're Hallucinating References?\u201dhttps:\/\/doi.org\/10.48550\/arXiv.2305.18248.","DOI":"10.18653\/v1\/2024.findings-eacl.62"},{"key":"e_1_2_13_3_1","unstructured":"Ambrogi B.2023.\u201cMajor Thomson Reuters News: Westlaw Gets Generative AI Research Plus Integration With Casetext CoCounsel.\u201dGen AI Coming Soon to Practical Law.https:\/\/www.lawnext.com\/2023\/11\/major\u2010thomson\u2010reuters\u2010news\u2010westlaw\u2010gets\u2010generative\u2010ai\u2010research\u2010plus\u2010casetext\u2010integration\u2010gen\u2010ai\u2010coming\u2010soon\u2010to\u2010practical\u2010law.html."},{"key":"e_1_2_13_4_1","unstructured":"Ambrogi B.2024.\u201cLawNext: Thomson Reuters' AI Strategy for Legal With Mike Dahn Head of Westlaw and Joel Hron.\u201dHead of AI.https:\/\/www.lawnext.com\/2024\/02\/lawnext\u2010thomson\u2010reuters\u2010ai\u2010strategy\u2010for\u2010legal\u2010with\u2010mike\u2010dahn\u2010head\u2010of\u2010westlaw\u2010and\u2010joel\u2010hron\u2010head\u2010of\u2010ai.html."},{"key":"e_1_2_13_5_1","unstructured":"American Bar Association.2018.\u201cAlphabetical List of Jurisdictions Adopting Model Rules.\u201dhttps:\/\/www.americanbar.org\/groups\/professional%5C_responsibility\/publications\/model%5C_rules%5C_of%5C_professional%5C_conduct\/alpha%5C_list%5C_state%5C_adopting%5C_model%5C_rules\/."},{"issue":"4","key":"e_1_2_13_6_1","first-page":"797","article-title":"Open Access in a Closed Universe: Lexis, Westlaw, Law Schools, and the Legal Information Market","volume":"10","author":"Arewa O. B.","year":"2006","journal-title":"Lewis and Clark Law Review"},{"key":"e_1_2_13_7_1","doi-asserted-by":"crossref","unstructured":"Asai A. T.Schick P.Lewis et\u00a0al.2022.\u201cTask\u2010Aware Retrieval With Instructions.\u201darXiv preprint arXiv:2211.09260.","DOI":"10.18653\/v1\/2023.findings-acl.225"},{"key":"e_1_2_13_8_1","unstructured":"Atleson M.2023.\u201cKeep Your AI Claims in Check.\u201dhttps:\/\/www.ftc.gov\/business\u2010guidance\/blog\/2023\/02\/keep\u2010your\u2010ai\u2010claims\u2010check."},{"issue":"1","key":"e_1_2_13_9_1","first-page":"64","article-title":"ChatGPT, Esq.: Recasting Unauthorized Practice of Law in the Era of Generative AI","volume":"26","author":"Avery J. J.","year":"2023","journal-title":"Yale Journal of Law and Technology"},{"key":"e_1_2_13_10_1","unstructured":"BARBRI Inc.2013.Multistate Testing Practice Questions."},{"key":"e_1_2_13_11_1","doi-asserted-by":"crossref","unstructured":"Barnett S. S.Kurniawan S.Thudumu Z.Brannelly andM.Abdelrazek.2024.\u201cSeven Failure Points When Engineering a Retrieval Augmented Generation System.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.2401.05856.","DOI":"10.1145\/3644815.3644945"},{"key":"e_1_2_13_12_1","doi-asserted-by":"publisher","DOI":"10.1111\/jels.12224"},{"key":"e_1_2_13_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1394251.1394261"},{"key":"e_1_2_13_14_1","unstructured":"Berkeley Law School.2024.\u201cGenerative AI Resources for Berkeley Law Faculty & Staff.\u201dhttps:\/\/www.law.berkeley.edu\/library\/legal\u2010research\/chatgpt\/."},{"issue":"1","key":"e_1_2_13_15_1","first-page":"27","article-title":"Full\u2010Text Databases and Legal Research: Backing Into the Future","volume":"1","author":"Berring R. C.","year":"1986","journal-title":"High Technology Law Journal"},{"key":"e_1_2_13_16_1","article-title":"Beyond the Imitation Game: Quantifying and Extrapolating the Capabilities of Language Models","author":"BIG\u2010bench Authors","year":"2023","journal-title":"Transactions on Machine Learning Research"},{"key":"e_1_2_13_17_1","doi-asserted-by":"publisher","DOI":"10.1111\/jels.12012"},{"key":"e_1_2_13_18_1","unstructured":"Bommasani R. D. A.Hudson E.Adeli et\u00a0al.2022.\u201cOn the Opportunities and Risks of Foundation Models.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.2108.07258."},{"issue":"2","key":"e_1_2_13_19_1","first-page":"389","article-title":"Bots Behaving Badly: A Products Liability Approach to Chatbot\u2010Generated Defamation","volume":"3","author":"Brown N.","year":"2023","journal-title":"Journal of Free Speech Law"},{"key":"e_1_2_13_20_1","unstructured":"Campbell E.2024.\u201cResources for Exploring the Benefits and Drawbacks of AI.\u201dhttps:\/\/library.law.unc.edu\/2024\/02\/resources\u2010for\u2010exploring\u2010the\u2010benefits\u2010and\u2010drawbacks\u2010of\u2010ai\/."},{"key":"e_1_2_13_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2071389.2071390"},{"key":"e_1_2_13_22_1","unstructured":"Casetext.2023.\u201cGPT\u20104 Alone Is Not a Reliable Legal Solution\u2014but It Does Enable One: CoCounsel Harnesses GPT\u20104's Power to Deliver Results That Legal Professionals Can Rely On.\u201dhttps:\/\/casetext.com\/blog\/cocounsel\u2010harnesses\u2010gpt\u20104s\u2010power\u2010to\u2010deliver\u2010results\u2010that\u2010legal\u2010professionals\u2010can\u2010rely\u2010on\/."},{"key":"e_1_2_13_23_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i16.29728"},{"key":"e_1_2_13_24_1","doi-asserted-by":"crossref","unstructured":"Chen L. M.Zaharia andJ.Zou.2023.\u201cHow Is Chatgpt's Behavior Changing Over Time?\u201darXiv preprint arXiv:2307.09009.","DOI":"10.1162\/99608f92.5317da47"},{"key":"e_1_2_13_25_1","first-page":"903","article-title":"Generative AI and Legal Aid: Results From a Field Study and 100 Use Cases to Bridge the Access to Justice Gap","volume":"57","author":"Chien C. V.","year":"2024","journal-title":"Loyola of Los Angeles Law Review"},{"key":"e_1_2_13_26_1","volume-title":"How Generative AI Can Help Address the Access to Justice Gap Through the Courts","author":"Chien C. V.","year":"2024"},{"key":"e_1_2_13_27_1","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.4626276"},{"key":"e_1_2_13_28_1","doi-asserted-by":"publisher","DOI":"10.1177\/001316446002000104"},{"key":"e_1_2_13_29_1","unstructured":"Collens J. R.Reimer G.Schifman andP.Wilkinson.2024.\u201cAI Survey: Where Artificial Intelligence Stands in the Legal Industry.\u201dhttps:\/\/assets.law360news.com\/1826000\/1826128\/law360_pulse\u2010ai_survey.pdf."},{"key":"e_1_2_13_30_1","doi-asserted-by":"crossref","unstructured":"Cuconasu F. G.Trappolini F.Siciliano et\u00a0al.2024.\u201cThe Power of Noise: Redefining Retrieval for RAG Systems.\u201dhttps:\/\/doi.org\/10.1145\/3626772.3657834.","DOI":"10.1145\/3626772.3657834"},{"issue":"1","key":"e_1_2_13_31_1","first-page":"401","article-title":"A Human Being Wrote This Law Review Article: GPT\u20103 and the Practice of Law","volume":"55","author":"Cyphert A. B.","year":"2021","journal-title":"UC Davis Law Review"},{"key":"e_1_2_13_32_1","doi-asserted-by":"publisher","DOI":"10.1093\/jla\/laae003"},{"key":"e_1_2_13_33_1","doi-asserted-by":"crossref","unstructured":"Dell'Acqua F. E.McFowland E. R.Mollick et\u00a0al.2023.\u201cNavigating the Jagged Technological Frontier: Field Experimental Evidence of the Effects of AI on Knowledge Worker Productivity and Quality.\u201dhttps:\/\/doi.org\/10.2139\/ssrn.4573321.","DOI":"10.2139\/ssrn.4573321"},{"key":"e_1_2_13_34_1","doi-asserted-by":"crossref","unstructured":"Deng C. Y.Zhao X.Tang M.Gerstein andA.Cohan.2023.\u201cInvestigating Data Contamination in Modern Benchmarks for Large Language Models.\u201darXiv preprint arXiv:2311.09783.","DOI":"10.18653\/v1\/2024.naacl-long.482"},{"issue":"1","key":"e_1_2_13_35_1","first-page":"317","article-title":"Vicarious Liability for AI","volume":"99","author":"Diamantis M. E.","year":"2023","journal-title":"Indiana Law Journal"},{"key":"e_1_2_13_36_1","volume-title":"Law's Empire","author":"Dworkin R.","year":"1986"},{"key":"e_1_2_13_37_1","doi-asserted-by":"publisher","DOI":"10.1093\/pan\/mpm011"},{"key":"e_1_2_13_38_1","unstructured":"Free Law Project.2024.\u201cCourtlistener.\u201dcourtlistener.com."},{"key":"e_1_2_13_39_1","unstructured":"Gao Y. Y.Xiong X.Gao et\u00a0al.2024.\u201cRetrieval\u2010Augmented Generation for Large Language Models: A Survey.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.2312.10997."},{"key":"e_1_2_13_40_1","doi-asserted-by":"publisher","DOI":"10.1136\/amiajnl\u20102011\u2010000089"},{"key":"e_1_2_13_41_1","unstructured":"Google.2024.\u201cGoogle Search Results for \u2018Practical Law\u2019.\u201dAccessed May 05 2024.https:\/\/web.archive.org\/web\/20240522191224\/https:\/\/www.google.com\/search?q=practical+law."},{"key":"e_1_2_13_42_1","volume-title":"Paul Weiss Assessing Value of AI","author":"Gottlieb I.","year":"2024"},{"key":"e_1_2_13_43_1","unstructured":"Greenhill S.2024.\u201cLawyers Cross Into the New Era of Generative AI.\u201dhttps:\/\/www.lexisnexis.co.uk\/insights\/lawyers\u2010cross\u2010into\u2010the\u2010new\u2010era\u2010of\u2010generative\u2010ai\/index.html."},{"key":"e_1_2_13_44_1","unstructured":"Grewal G. S.2024.\u201cRemarks at Program on Corporate Compliance and Enforcement Spring Conference 2024.\u201dhttps:\/\/www.sec.gov\/news\/speech\/gurbir\u2010remarks\u2010pcce\u2010041524."},{"key":"e_1_2_13_45_1","unstructured":"Guha N. J.Nyarko D. E.Ho et\u00a0al.2023.\u201cLegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Models.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.2308.11462."},{"key":"e_1_2_13_46_1","first-page":"3929","volume-title":"International Conference on Machine Learning","author":"Guu K.","year":"2020"},{"key":"e_1_2_13_47_1","first-page":"3929","volume-title":"Proceedings of the 37th International Conference on Machine Learning","author":"Guu K.","year":"2020"},{"key":"e_1_2_13_48_1","unstructured":"Haimes J. C.Wenner K.Thaman et\u00a0al.2024.\u201cBenchmark Inflation: Revealing Llm Performance Gaps Using Retro\u2010Holdouts.\u201darXiv:2410.09247.https:\/\/doi.org\/10.48550\/arXiv.2410.09247."},{"key":"e_1_2_13_49_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1169"},{"key":"e_1_2_13_50_1","unstructured":"Henderson P. M. S.Krass L.Zheng et\u00a0al.2022.\u201cPile of Law: Learning Responsible Data Filtering From the Law and a 256GB.\u201dOpen\u2010Source Legal Dataset.https:\/\/doi.org\/10.48550\/arXiv.2207.00220."},{"key":"e_1_2_13_51_1","volume-title":"International Conference on Learning Representations","author":"Hendrycks D.","year":"2020"},{"key":"e_1_2_13_52_1","unstructured":"Henry J.2024.\u201cWe Asked Every am Law 100 Law Firm How They're Using Gen AI. Here's What we Learned.\u201dThe American Lawyer.https:\/\/www.law.com\/americanlawyer\/2024\/01\/29\/we\u2010asked\u2010every\u2010am\u2010law\u2010100\u2010firm\u2010how\u2010theyre\u2010using\u2010gen\u2010ai\u2010heres\u2010what\u2010we\u2010learned\/."},{"key":"e_1_2_13_53_1","unstructured":"Izacard G. P.Lewis M.Lomeli et\u00a0al.2022.\u201cFew\u2010Shot Learning With Retrieval Augmented Language Models.\u201darXiv preprint arXiv:2208.03299 2 (3)."},{"key":"e_1_2_13_54_1","doi-asserted-by":"publisher","DOI":"10.1145\/3571730"},{"key":"e_1_2_13_55_1","doi-asserted-by":"crossref","unstructured":"Kalai A. T. andS. S.Vempala.2023.\u201cCalibrated Language Models Must Hallucinate.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.2311.14648.","DOI":"10.1145\/3618260.3649777"},{"issue":"2","key":"e_1_2_13_56_1","article-title":"Promises and Pitfalls of Artificial Intelligence for Legal Applications","volume":"2","author":"Kapoor S.","year":"2024","journal-title":"Journal of Cross\u2010Disciplinary Research in Computational Law"},{"key":"e_1_2_13_57_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.emnlp-main.550"},{"key":"e_1_2_13_58_1","volume-title":"2023 Artificial Intelligence (AI) TechReport (Tech. Rep.)","author":"Kite\u2010Jackson D. W.","year":"2023"},{"key":"e_1_2_13_59_1","unstructured":"Krieger S. H. andK. F.Kuh.2014.\u201cAccessing Law: An Empirical Study Exploring the Influence of Legal Research Medium.\u201d(2456679).https:\/\/papers.ssrn.com\/abstract=2456679."},{"key":"e_1_2_13_60_1","unstructured":"Law360.2024.\u201cTracking Federal Judge Orders on Artificial Intelligence.\u201dhttps:\/\/www.law360.com\/pulse\/ai\u2010tracker."},{"key":"e_1_2_13_61_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-33220-4_20"},{"key":"e_1_2_13_62_1","first-page":"9459","article-title":"Retrieval\u2010Augmented Generation for Knowledge\u2010Intensive NLP Tasks","volume":"33","author":"Lewis P.","year":"2020","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_13_63_1","unstructured":"LexisNexis.2023a.\u201cGeneral Terms and Conditions.\u201dhttps:\/\/www.lexisnexis.com\/en\u2010us\/terms\/general\/default.page."},{"key":"e_1_2_13_64_1","unstructured":"LexisNexis.2023b.\u201cLexisNexis Launches Lexis+ AI a Generative AI Solution With Linked Hallucination\u2010Free Legal Citations.\u201dhttps:\/\/www.lexisnexis.com\/community\/pressroom\/b\/news\/posts\/lexisnexis\u2010launches\u2010lexis\u2010ai\u2010a\u2010generative\u2010ai\u2010solution\u2010with\u2010hallucination\u2010free\u2010linked\u2010legal\u2010citations."},{"key":"e_1_2_13_65_1","unstructured":"LexisNexis.2024.\u201cLexisNexis Launches Second\u2010Generation Legal AI Assistant on Lexis+ AI.\u201dhttps:\/\/www.lexisnexis.com\/community\/pressroom\/b\/news\/posts\/lexisnexis\u2010launches\u2010second\u2010generation\u2010legal\u2010ai\u2010assistant\u2010on\u2010lexis\u2010ai."},{"key":"e_1_2_13_66_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v38i16.29808"},{"key":"e_1_2_13_67_1","doi-asserted-by":"crossref","unstructured":"Liang P. R.Bommasani T.Lee et\u00a0al.2023.\u201cHolistic Evaluation of Language Models.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.2211.09110.","DOI":"10.1111\/nyas.15007"},{"key":"e_1_2_13_68_1","unstructured":"Lissner M.2022.\u201cImportant Opinions on Courtlistener Are Now Summarized by the Top Experts\u2014Judges.\u201dhttps:\/\/free.law\/2022\/03\/17\/summarizing\u2010important\u2010cases."},{"key":"e_1_2_13_69_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/D16-1230"},{"key":"e_1_2_13_70_1","unstructured":"Ma M. A.Sinha A.Tandon andJ.Richards.2024.\u201cGenerative AI Legal Landscape 2024.\u201d(Tech. Rep.).https:\/\/law.stanford.edu\/publications\/generative\u2010ai\u2010legal\u2010landscape\u20102024\u20102\/."},{"issue":"2","key":"e_1_2_13_71_1","first-page":"300","article-title":"The Legal Tech Bro Blues: Generative AI, Legal Indeterminacy, and the Future of Legal Research and Writing","volume":"8","author":"Magnanelli N.","year":"2024","journal-title":"Georgetown Law Technology Review"},{"key":"e_1_2_13_72_1","doi-asserted-by":"crossref","unstructured":"Manakul P. A.Liusie andM. J. F.Gales.2023.\u201cSelfCheckGPT: Zero\u2010Resource Black\u2010Box Hallucination Detection for Generative Large Language Models.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.2303.08896.","DOI":"10.18653\/v1\/2023.emnlp-main.557"},{"key":"e_1_2_13_73_1","doi-asserted-by":"publisher","DOI":"10.1007\/s43681\u2010024\u201000461\u20102"},{"key":"e_1_2_13_74_1","unstructured":"Mart S.2018.\u201cResults May Vary.\u201dABA Journal.https:\/\/scholar.law.colorado.edu\/faculty\u2010articles\/964."},{"key":"e_1_2_13_75_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10506\u2010024\u201009396\u20109"},{"key":"e_1_2_13_76_1","unstructured":"McGreel P.2024.\u201cI Asked Lexas+ AI [Sic] a Simple Question: \u201cWhat Cases Have Applied Students for Fair Admissions Inc. v. Harvard College to the Use of Race in Government Decisionmaking?\u201d This Screenshot Has the Answer I Received. Here Are Some of the (Serious) Problems With This Answer.\u201d[Twitter].https:\/\/x.com\/conlawgeek\/status\/1762561111725899859."},{"key":"e_1_2_13_77_1","unstructured":"McIntosh T. R. T.Susnjak T.Liu P.Watters andM. N.Halgamuge.2024.Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligence."},{"key":"e_1_2_13_78_1","doi-asserted-by":"publisher","DOI":"10.1155\/2023\/6660983"},{"key":"e_1_2_13_79_1","unstructured":"Mik E.2024.\u201cCaveat Lector: Large Language Models in Legal Practice.\u201dhttps:\/\/papers.ssrn.com\/abstract=4757452."},{"key":"e_1_2_13_80_1","volume-title":"Remarks: Clerks of Judge Luther A","author":"Miner R. J.","year":"1989"},{"key":"e_1_2_13_81_1","unstructured":"M\u00fcndler N. J.He S.Jenko andM.Vechev.2023.\u201cSelf\u2010Contradictory Hallucinations of Lage Language Models: Evaluation.\u201dDetection and Mitigation.https:\/\/doi.org\/10.48550\/arXiv.2305.15852."},{"key":"e_1_2_13_82_1","unstructured":"NIST.2020.\u201cFace Recognition Vendor Test (FRVT).\u201dhttps:\/\/www.nist.gov\/programs\u2010projects\/face\u2010recognition\u2010vendor\u2010test\u2010frvt."},{"key":"e_1_2_13_83_1","volume-title":"The Twelfth International Conference on Learning Representations","author":"Oren Y.","year":"2023"},{"key":"e_1_2_13_84_1","doi-asserted-by":"crossref","unstructured":"\u00d6stling A. H.Sargeant H.Xie et\u00a0al.2024.\u201cThe Cambridge Law Corpus: A Dataset for Legal AI Research.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.2309.12269.","DOI":"10.2139\/ssrn.4763429"},{"key":"e_1_2_13_85_1","doi-asserted-by":"crossref","unstructured":"Perlman A.2023.\u201cThe Implications of ChatGPT for Legal Services and Society.\u201dThe Practice (March\/April).","DOI":"10.2139\/ssrn.4294197"},{"key":"e_1_2_13_86_1","unstructured":"Raji I. D. E.Denton E. M.Bender A.Hanna andA.Paullada.2021.\u201cAI and the Everything in the Whole Wide World Benchmark.\u201dThirty\u2010Fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2)."},{"key":"e_1_2_13_87_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533158"},{"key":"e_1_2_13_88_1","doi-asserted-by":"crossref","unstructured":"Raji I. D. P.Xu C.Honigsberg andD. E.Ho.2022b.\u201cOutsider Oversight: Designing a Third Party Audit Ecosystem for Ai Governance.\u201darXiv:2206.04737[cs].http:\/\/arxiv.org\/abs\/2206.04737.","DOI":"10.1145\/3514094.3534181"},{"key":"e_1_2_13_89_1","doi-asserted-by":"crossref","unstructured":"Rajpurkar P. J.Zhang K.Lopyrev andP.Liang.2016.\u201cSQuAD: 100 000+ Questions for Machine Comprehension of Text.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.1606.05250.","DOI":"10.18653\/v1\/D16-1264"},{"key":"e_1_2_13_90_1","unstructured":"Reuters T.2019.\u201cWestlaw Tip of the Week: Checking Cases With Keycite.\u201dhttps:\/\/legal.thomsonreuters.com\/blog\/westlaw\u2010tip\u2010of\u2010the\u2010week\u2010checking\u2010cases\u2010with\u2010keycite\/."},{"key":"e_1_2_13_91_1","unstructured":"Rivers C. C.2024.\u201cMoffatt v. Air Canada.\u201dhttps:\/\/www.canlii.org\/en\/bc\/bccrt\/doc\/2024\/2024bccrt149\/2024bccrt149.html."},{"key":"e_1_2_13_92_1","unstructured":"Roberts J. G.2023.\u201c2023 Year\u2010End Report on the Federal Judiciary.\u201d(tech. rep.).https:\/\/www.uscourts.gov\/news\/2023\/12\/31\/chief\u2010justice\u2010roberts\u2010issues\u20102023\u2010year\u2010end\u2010report."},{"key":"e_1_2_13_93_1","doi-asserted-by":"publisher","DOI":"10.2139\/ssrn.4687558"},{"key":"e_1_2_13_94_1","first-page":"1","article-title":"AI Tools for Lawyers: A Practical Guide","volume":"108","author":"Schwarcz D.","year":"2023","journal-title":"Minnesota Law Review Headnotes"},{"key":"e_1_2_13_95_1","unstructured":"Securities and Exchange Commission.2024.\u201cSec Charges Two Investment Advisers With Making False and Misleading Statements About Their Use of Artificial Intelligence.\u201dhttps:\/\/www.sec.gov\/news\/press\u2010release\/2024\u201036."},{"key":"e_1_2_13_96_1","unstructured":"Sharma M. M.Tong T.Korbak et\u00a0al.2023.\u201cTowards Understanding Sycophancy in Language Models.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.2310.13548."},{"key":"e_1_2_13_97_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00530"},{"key":"e_1_2_13_98_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.nlp4convai-1.8"},{"key":"e_1_2_13_99_1","unstructured":"Suffolk University.2023.\u201cLaw Faculty Guide to Artificial Intelligence: Practical Law AI (Westlaw).\u201dhttps:\/\/lawguides.suffolk.edu\/AIFac\/PracticalLaw."},{"key":"e_1_2_13_100_1","unstructured":"Surani F. M.Dahl andV.Magesh.2024.\u201cLegal RAG Hallucinations.\u201dhttps:\/\/osf.io\/etzp2."},{"key":"e_1_2_13_101_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.findings-acl.824"},{"key":"e_1_2_13_102_1","unstructured":"Tan J. H.Westermann andK.Benyekhlef.2023.\u201cChatGPT as an Artificial Lawyer?\u201dProceedings of the ICAIL 2023 Workshop on Artificial Intelligence for Access to Justice."},{"key":"e_1_2_13_103_1","unstructured":"Task Force on Artificial Intelligence.2024.\u201cReport and Recommendations of the New York State Bar Association Task Force on Artificial Intelligence\u201d (tech. rep. New York State Bar Association).https:\/\/nysba.org\/app\/uploads\/2024\/04\/Task\u2010Force\u2010on\u2010AI\u2010Report\u2010draft\u20102024\u201004\u201002\u2010FINAL.pdf."},{"key":"e_1_2_13_104_1","unstructured":"The Florida Bar.2024.\u201cFlorida Bar Ethics Opinion \u201d (tech. rep. No. 24\u20131. The Florida Bar).https:\/\/www.floridabar.org\/etopinions\/opinion\u201024\u20101\/."},{"key":"e_1_2_13_105_1","unstructured":"The Library Innovation Lab.2024.Transitions for the Caselaw Access Project."},{"key":"e_1_2_13_106_1","unstructured":"The State Bar of California.2023.\u201cPractical Guidance for the Use of Generative Artificial Intelligence in the Practice of Law\u201d (Tech. Rep. The State Bar of California).https:\/\/www.calbar.ca.gov\/Portals\/0\/documents\/ethics\/Generative\u2010AI\u2010Practical\u2010Guidance.pdf."},{"key":"e_1_2_13_107_1","unstructured":"Thomson Reuters.2023.\u201cIntroducing AI\u2010Assisted Research: Legal Research Meets Generative AI.\u201dhttps:\/\/legal.thomsonreuters.com\/blog\/legal\u2010research\u2010meets\u2010generative\u2010ai\/."},{"key":"e_1_2_13_108_1","unstructured":"Thomson Reuters.2024a.\u201cAccelerate How You Find Answers so You Can Practice With Confidence.\u201dhttps:\/\/legal.thomsonreuters.com\/en\/products\/practical\u2010law."},{"key":"e_1_2_13_109_1","unstructured":"Thomson Reuters.2024b.\u201cIntroducing Ask Practical Law AI on Practical Law: Generative AI Meets Legal How\u2010To.\u201dhttps:\/\/legal.thomsonreuters.com\/blog\/legal\u2010how\u2010to\u2010meets\u2010generative\u2010ai\/."},{"key":"e_1_2_13_110_1","unstructured":"University of Washington.2024.\u201cArtificial Intelligence.\u201dUniversity of Washington Gallagher law Library.https:\/\/lib.law.uw.edu\/c.php?g=1387996&p=10265405."},{"key":"e_1_2_13_111_1","unstructured":"van derMerwe M. K.Ramakrishnan andM.Anderljung.2024.\u201cTort Law and Frontier AI Governance\u201d (tech. rep. Lawfare)."},{"issue":"2","key":"e_1_2_13_112_1","first-page":"489","article-title":"Large Libel Models? Liability for AI Output","volume":"3","author":"Volokh E.","year":"2023","journal-title":"Journal of Free Speech Law"},{"key":"e_1_2_13_113_1","doi-asserted-by":"publisher","DOI":"10.1023\/A:1014513930336"},{"issue":"4","key":"e_1_2_13_114_1","first-page":"1073","article-title":"The Model Rules of Autonomous Conduct: Ethical Responsibilities of Lawyers and Artificial Intelligence","volume":"35","author":"Walters E.","year":"2019","journal-title":"Georgia State University Law Review"},{"key":"e_1_2_13_115_1","article-title":"Here's What Happens When Your Lawyer Uses ChatGPT","author":"Weiser B.","year":"2023","journal-title":"New York Times"},{"key":"e_1_2_13_116_1","unstructured":"Weiser B. andJ. E.Bromwich.2023.\u201cMichael Cohen Used Artificial Intelligence in Feeding Lawyer Bogus Cases.\u201dThe New York Times.https:\/\/www.nytimes.com\/2023\/12\/29\/nyregion\/michael\u2010cohen\u2010ai\u2010fake\u2010cases.html."},{"key":"e_1_2_13_117_1","unstructured":"Wellen S.2024a.\u201cTech Innovation With LLMs Producing More Secure and Reliable Gen AI Results.\u201dhttps:\/\/www.lexisnexis.com\/community\/insights\/legal\/b\/thought\u2010leadership\/posts\/tech\u2010innovation\u2010with\u2010llms\u2010producing\u2010more\u2010secure\u2010and\u2010reliable\u2010gen\u2010ai\u2010results."},{"key":"e_1_2_13_118_1","unstructured":"Wellen S.2024b.\u201cHow Lexis+ AI Delivers Hallucination\u2010Free Linked Legal Citations.\u201dhttps:\/\/www.lexisnexis.com\/community\/insights\/legal\/b\/product\u2010features\/posts\/how\u2010lexis\u2010ai\u2010delivers\u2010hallucination\u2010free\u2010linked\u2010legal\u2010citations."},{"key":"e_1_2_13_119_1","unstructured":"Wills P.2025. \u201cCare for Chatbots.\u201dUBC Law Review.https:\/\/www.ubclawreview.ca\/special\u2010issue\u2010ai\u2010the\u2010legal\u2010profession."},{"key":"e_1_2_13_120_1","unstructured":"Yale Law School.2024.\u201cLexis and Westlaw Generative AI Products.\u201dYale Law Library Lillian Goldman Law Library.https:\/\/library.law.yale.edu\/news\/lexis\u2010and\u2010westlaw\u2010generative\u2010ai\u2010products."},{"issue":"3","key":"e_1_2_13_121_1","first-page":"877","article-title":"Artificial Intelligence in the Legal Field and the Indispensable Human Element Legal Ethics Demands","volume":"33","author":"Yamane N.","year":"2020","journal-title":"Georgetown Journal of Legal Ethics"},{"key":"e_1_2_13_122_1","doi-asserted-by":"crossref","unstructured":"Yang Z. P.Qi S.Zhang et\u00a0al.2018.\u201cHotpotQA: A Dataset for Diverse Explainable Multi\u2010Hop Question Answering.\u201dhttps:\/\/doi.org\/10.48550\/arXiv.1809.09600.","DOI":"10.18653\/v1\/D18-1259"},{"key":"e_1_2_13_123_1","unstructured":"Zheng L. W.\u2010L.Chiang Y.Sheng et\u00a0al.2023.\u201cJudging LLM\u2010As\u2010a\u2010Judge With MT\u2010Bench and Chatbot Arena.\u201dThirty\u2010Seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track."},{"key":"e_1_2_13_124_1","doi-asserted-by":"publisher","DOI":"10.1145\/3462757.3466088"},{"key":"e_1_2_13_125_1","unstructured":"Zou A. L.Phan S.Chen et\u00a0al.2023a.Representation Engineering: A Top\u2010Down Approach to AI Transparency."},{"key":"e_1_2_13_126_1","unstructured":"Zou A. Z.Wang J. Z.Kolter andM.Fredrikson.2023b.\u201cUniversal and Transferable Adversarial Attacks on Aligned Language Models.\u201darXiv preprint arXiv:2307.15043."}],"container-title":["Journal of Empirical Legal Studies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/jels.12413","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/jels.12413","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/jels.12413","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,16]],"date-time":"2026-03-16T07:28:47Z","timestamp":1773646127000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/jels.12413"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,4,23]]},"references-count":125,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2025,6]]}},"alternative-id":["10.1111\/jels.12413"],"URL":"https:\/\/doi.org\/10.1111\/jels.12413","archive":["Portico"],"relation":{},"ISSN":["1740-1453","1740-1461"],"issn-type":[{"value":"1740-1453","type":"print"},{"value":"1740-1461","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,4,23]]},"assertion":[{"value":"2025-03-14","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-04-23","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}