{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T12:18:24Z","timestamp":1776773904348,"version":"3.51.2"},"reference-count":38,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T00:00:00Z","timestamp":1772582400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T00:00:00Z","timestamp":1772582400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Rev Socionetwork Strat"],"published-print":{"date-parts":[[2026,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Legal reasoning is complex and multi-faceted, requiring a broad set of skills. By employing domain knowledge from legal experts, we design five elements that can be included in prompts for large language models that could aid in legal reasoning tasks. We use additional legal guidelines, 1-shot prompting, dictionary definitions, knowledge representations of legal articles, and IRAC-style prompting. We investigate the effect of each prompt element on the model\u2019s performance on a legal entailment task. Certain prompt elements can improve performance, depending on the context and the model. For the smaller models, increasing the number of prompt elements improves performance on average. For any particular combination of model and sub-task, only using a subset of the prompt elements seems to work best. For the most advanced reasoning model we evaluate, using a selection of prompt elements increases average performance across all evaluated sub-tasks. Results indicate that the problem space of the legal entailment task may be too large for a single model and prompt. In future research, we therefore aim to investigate the capabilities of an ensemble of specialized models.<\/jats:p>","DOI":"10.1007\/s12626-026-00202-3","type":"journal-article","created":{"date-parts":[[2026,3,4]],"date-time":"2026-03-04T04:32:53Z","timestamp":1772598773000},"page":"255-279","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Investigating Expert-Based Prompt Engineering for Legal Entailment Tasks"],"prefix":"10.1007","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-6887-1687","authenticated-orcid":false,"given":"Cor","family":"Steging","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3165-4376","authenticated-orcid":false,"given":"Ludi","family":"van Leeuwen","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9052-6978","authenticated-orcid":false,"given":"Tadeusz","family":"Zbiegie\u0144","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2026,3,4]]},"reference":[{"key":"202_CR1","first-page":"257","volume-title":"BM25","author":"G Amati","year":"2009","unstructured":"Amati, G. (2009). BM25 (pp. 257\u2013260). Springer."},{"key":"202_CR2","unstructured":"Araszkiewicz, M., & \u0160avelka, J. (2011). Two methods for representing judicial reasoning in the framework of coherence as constraint satisfaction. JURIX: The Twenty-Fourth Annual Conference, Frontiers in Artificial Intelligence and Applications. In K. Atkinson (Ed.), Legal Knowledge and Information Systems (Vol. 235, pp. 165\u2013166). IOS Press."},{"key":"202_CR3","first-page":"673","volume-title":"Handbook in legal reasoning and argumentation","author":"K Ashley","year":"2011","unstructured":"Ashley, K. (2011). Precedent and legal analogy. In G. Bongiovanni, D. Postema, A. Rotolo, G. Sartor, C. Valentini, & D. Walton (Eds.), Handbook in legal reasoning and argumentation (pp. 673\u2013710). Springer."},{"issue":"1","key":"202_CR4","doi-asserted-by":"publisher","first-page":"66","DOI":"10.1111\/raju.12268","volume":"33","author":"B Askeland","year":"2020","unstructured":"Askeland, B. (2020). The potential of abductive legal reasoning. Ratio Juris, 33(1), 66\u201381.","journal-title":"Ratio Juris"},{"key":"202_CR5","doi-asserted-by":"crossref","unstructured":"Atkinson, K., & Bench-Capon, T. (2023). ANGELIC II: An improved methodology for representing legal domain knowledge. In ICAIL \u201923: Proceedings of the Nineteenth International Conference for Artificial Intelligence and Law. ACM, Braga, Portugal.","DOI":"10.1145\/3594536.3595137"},{"key":"202_CR6","unstructured":"Black, H. (1910). Black\u2019s Law Dictionary (2nd Edition). West Publishing Company. Retrieved from https:\/\/gist.github.com\/medelman17\/55bf480caafbfcc6e9f9d22c273cf2c4. Extracted in JSON."},{"key":"202_CR7","unstructured":"Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., & Amodei, D. (2020). Language models are few-shot learners. In: H. Larochelle, M. Ranzato, R. Hadsell, M. Balcan, H. Lin (eds.) Advances in Neural Information Processing Systems, (vol. 33, pp. 1877\u20131901). Curran Associates, Inc."},{"issue":"2","key":"202_CR8","doi-asserted-by":"publisher","first-page":"57","DOI":"10.5204\/jld.v10i2.229","volume":"10","author":"K Burton","year":"2017","unstructured":"Burton, K. (2017). \u2018Think like a lawyer\u2019 using a legal reasoning grid and criterion-referenced assessment rubric on IRAC (issue, rule, application, conclusion). Journal of Learning Design, 10(2), 57\u201368.","journal-title":"Journal of Learning Design"},{"key":"202_CR9","volume-title":"Legal reasoning","author":"P Ellsworth","year":"2005","unstructured":"Ellsworth, P. (2005). Legal reasoning. Cambridge University Press."},{"key":"202_CR10","doi-asserted-by":"crossref","unstructured":"Goebel, R., Kano, Y., Kim, M., Rabelo, J., Satoh, K., & Yoshioka, M. (2023). Summary of the competition on legal information, extraction\/entailment (COLIEE) 2023. In Proceedings of the nineteenth international conference on artificial intelligence and law (pp. 472\u2013480). ACM, Braga Portugal.","DOI":"10.1145\/3594536.3595176"},{"key":"202_CR11","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1007\/978-981-97-3076-6_8","volume-title":"New frontiers in artificial intelligence","author":"R Goebel","year":"2024","unstructured":"Goebel, R., Kano, Y., Kim, M., Rabelo, J., Satoh, K., & Yoshioka, M. (2024). Overview of benchmark datasets and methods for the legal information extraction\/entailment competition (COLIEE) 2024. In T. Suzumura & M. Bono (Eds.), New frontiers in artificial intelligence (pp. 109\u2013124). Springer."},{"key":"202_CR12","doi-asserted-by":"publisher","first-page":"481","DOI":"10.1007\/s10506-022-09329-4","volume":"30","author":"G Governatori","year":"2022","unstructured":"Governatori, G., Bench-Capon, T., & Verheij, B. (2022). Thirty years of artificial intelligence and law: The first decade. Artificial Intelligence and Law, 30, 481\u2013519.","journal-title":"Artificial Intelligence and Law"},{"key":"202_CR13","unstructured":"Guha, N., Nyarko, J., Ho, D., R\u00e9, C., Chilton, A., Narayana, A., Chohlas-Wood, A., Peters, A., Waldon, B., Rockmore, D., Zambrano, D., Talisman, D., Hoque, E., Surani, F., Fagan, F., Sarfaty, G., Dickinson, G., Porat, H., Hegland, J., Wu, J., Nudell, J., Niklaus, J., Nay, J., Choi, J., Tobia, K., Hagan, M., Ma, M., Livermore, M., Rasumov-Rahe, N., Holzenberger, N., Kolt, N., Henderson, P., Rehaag, S., Goel, S., Gao, S., Williams, S., Gandhi, S., Zur, T., Iyer, V., & Li, Z. (2024). LegalBench: a collaboratively built benchmark for measuring legal reasoning in large language models. In Proceedings of the 37th international conference on neural information processing systems, NIPS \u201923. Curran Associates, Red Hook, NY, USA."},{"issue":"4","key":"202_CR14","doi-asserted-by":"publisher","first-page":"593","DOI":"10.2307\/1338225","volume":"71","author":"HLA Hart","year":"1958","unstructured":"Hart, H. L. A. (1958). Positivism and the separation of law and morals. Harvard Law Review, 71(4), 593\u2013629.","journal-title":"Harvard Law Review"},{"key":"202_CR15","first-page":"547","volume":"37","author":"P Jaccard","year":"1901","unstructured":"Jaccard, P. (1901). Etude de la distribution florale dans une portion des Alpeset du Jura. Bulletin de la Societe Vaudoise des Sciences Naturelles, 37, 547\u2013579.","journal-title":"Bulletin de la Societe Vaudoise des Sciences Naturelles"},{"key":"202_CR16","doi-asserted-by":"crossref","unstructured":"Jiang, C., & Yang, X. (2023). Legal syllogism prompting: Teaching large language models for legal judgment prediction. In Proceedings of the nineteenth international conference on artificial intelligence and law, ICAIL \u201923 (pp. 417\u2013421). Association for Computing Machinery, New York, NY, USA.","DOI":"10.1145\/3594536.3595170"},{"issue":"2270","key":"202_CR17","doi-asserted-by":"publisher","first-page":"20230254","DOI":"10.1098\/rsta.2023.0254","volume":"382","author":"D Katz","year":"2024","unstructured":"Katz, D., Bommarito, M., Gao, S., & Arredondo, P. (2024). Gpt-4 passes the bar exam. Philosophical Transactions of the Royal Society A, 382(2270), 20230254.","journal-title":"Philosophical Transactions of the Royal Society A"},{"key":"202_CR18","first-page":"387","volume-title":"Handbook in legal reasoning and argumentation","author":"L Kornhauser","year":"2011","unstructured":"Kornhauser, L. (2011). Choosing ends and choosing means: Teleological reasoning in law. In G. Bongiovanni, D. Postema, A. Rotolo, G. Sartor, C. Valentini, & D. Walton (Eds.), Handbook in legal reasoning and argumentation (pp. 387\u2013412). Springer."},{"key":"202_CR19","doi-asserted-by":"publisher","first-page":"501","DOI":"10.2307\/1597535","volume":"15","author":"E Levi","year":"1948","unstructured":"Levi, E. (1948). An introduction to legal reasoning. University of Chicago Law Review, 15, 501\u2013574.","journal-title":"University of Chicago Law Review"},{"key":"202_CR20","doi-asserted-by":"crossref","unstructured":"Mart\u00ednez, E. (2024). Re-evaluating gpt-4\u2019s bar exam performance. Artificial Intelligence and Law.","DOI":"10.2139\/ssrn.4441311"},{"key":"202_CR21","first-page":"501","volume":"80","author":"J Metzler","year":"2003","unstructured":"Metzler, J. (2003). The importance of IRAC and legal writing. University of Detroit Mercy Law Review, 80, 501\u2013503.","journal-title":"University of Detroit Mercy Law Review"},{"key":"202_CR22","doi-asserted-by":"publisher","first-page":"125","DOI":"10.1007\/978-981-97-3076-6_9","volume-title":"New frontiers in artificial intelligence","author":"P Nguyen","year":"2024","unstructured":"Nguyen, P., Nguyen, C., Nguyen, H., Nguyen, M., Trieu, A., Nguyen, D., & Nguyen, L. (2024). CAPTAIN at COLIEE 2024: Large language model for legal text retrieval and entailment. In T. Suzumura & M. Bono (Eds.), New frontiers in artificial intelligence (pp. 125\u2013139). Springer Nature."},{"key":"202_CR23","doi-asserted-by":"publisher","first-page":"433","DOI":"10.2307\/1599942","volume":"59","author":"RA Posner","year":"1992","unstructured":"Posner, R. A. (1992). Legal reasoning from the top down and from the bottom up: The question of unenumerated constitutional rights. University of Chicago Law Review, 59, 433\u2013450.","journal-title":"University of Chicago Law Review"},{"issue":"1","key":"202_CR24","doi-asserted-by":"publisher","first-page":"118","DOI":"10.1111\/j.0952-1917.2004.00259.x","volume":"17","author":"H Prakken","year":"2004","unstructured":"Prakken, H., & Sartor, G. (2004). The three faces of defeasibility in the law. Ratio Juris, 17(1), 118\u2013139.","journal-title":"Ratio Juris"},{"issue":"1","key":"202_CR25","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1007\/s12626-022-00105-z","volume":"16","author":"J Rabelo","year":"2022","unstructured":"Rabelo, J., Goebel, R., Kim, M., Kano, Y., Yoshioka, M., & Satoh, K. (2022). Overview and discussion of the competition on legal information extraction\/entailment (COLIEE) 2021. The Review of Socionetwork Strategies, 16(1), 111\u2013133.","journal-title":"The Review of Socionetwork Strategies"},{"key":"202_CR26","first-page":"315","volume-title":"Handbook in legal reasoning and argumentation","author":"G Sartor","year":"2011","unstructured":"Sartor, G. (2011). Defeasibility in law. In G. Bongiovanni, D. Postema, A. Rotolo, G. Sartor, C. Valentini, & D. Walton (Eds.), Handbook in legal reasoning and argumentation (pp. 315\u2013364). Springer."},{"key":"202_CR27","doi-asserted-by":"publisher","first-page":"521","DOI":"10.1007\/s10506-022-09326-7","volume":"30","author":"G Sartor","year":"2022","unstructured":"Sartor, G., Araszkiewicz, M., Atkinson, K., et al. (2022). Thirty years of artificial intelligence and law: The second decade. Artificial Intelligence and Law, 30, 521\u2013557.","journal-title":"Artificial Intelligence and Law"},{"key":"202_CR28","volume-title":"Reading law: The interpretation of legal texts","author":"A Scalia","year":"2012","unstructured":"Scalia, A., & Garner, B. (2012). Reading law: The interpretation of legal texts. Thomson\/West."},{"issue":"4","key":"202_CR29","doi-asserted-by":"publisher","first-page":"509","DOI":"10.2307\/796369","volume":"97","author":"F Schauer","year":"1988","unstructured":"Schauer, F. (1988). Formalism. Yale Law Journal, 97(4), 509\u2013548.","journal-title":"Yale Law Journal"},{"issue":"5","key":"202_CR30","doi-asserted-by":"publisher","first-page":"370","DOI":"10.1145\/5689.5920","volume":"29","author":"MJ Sergot","year":"1986","unstructured":"Sergot, M. J., Sadri, F., Kowalski, R. A., Kriwaczek, F., Hammond, P., & Cory, H. T. (1986). The British nationality act as a logic program. Communications of the ACM, 29(5), 370\u2013386.","journal-title":"Communications of the ACM"},{"key":"202_CR31","doi-asserted-by":"crossref","unstructured":"Steging, C., Renooij, S., & Verheij, B. (2021). Discovering the rationale of decisions: Towards a method for aligning learning and reasoning. In: J. Maranh\u00e3o, A. Wyner (eds.) ICAIL \u201921: Proceedings of the eighteenth international conference for artificial intelligence and law (pp. 235\u2013239). ACM, S\u00e3o Paolo, Brazil.","DOI":"10.1145\/3462757.3466059"},{"key":"202_CR32","unstructured":"Steging, C., Renooij, S., & Verheij, B. (2023). Taking the law more seriously by investigating design choices in machine learning prediction research. In ASAIL 2023. Automated semantic analysis of information in legal text. Proceedings of the 6th workshop on automated semantic analysis of information in legal text co-located with the 19th international conference on artificial intelligence and law (ICAIL 2023) (pp. 49\u201359). CEUR-WS, Braga, Portugal."},{"key":"202_CR33","unstructured":"Steging, C., & van Leeuwen, L. (2024). A hybrid approach to legal textual entailment. In Eighteenth international workshop on juris-informatics (JURISIN 2024). Hamamatsu, Japan."},{"key":"202_CR34","doi-asserted-by":"publisher","first-page":"741","DOI":"10.2307\/1341662","volume":"106","author":"C Sunstein","year":"1992","unstructured":"Sunstein, C. (1992). On analogical reasoning commentary. Harvard Law Review, 106, 741\u2013791.","journal-title":"Harvard Law Review"},{"key":"202_CR35","first-page":"165","volume":"18","author":"T Turner","year":"2012","unstructured":"Turner, T. (2012). Finding consensus in legal writing discourse regarding organizational structure: A review and proposal. Legal Writing: Journal of the Legal Writing Institute, 18, 165\u2013206.","journal-title":"Legal Writing: Journal of the Legal Writing Institute"},{"issue":"4","key":"202_CR36","doi-asserted-by":"publisher","first-page":"561","DOI":"10.1007\/s10506-022-09327-6","volume":"30","author":"S Villata","year":"2022","unstructured":"Villata, S., Araszkiewicz, M., Ashley, K., Bench-Capon, T., Branting, L., Conrad, J. G., & Wyner, A. (2022). Thirty years of artificial intelligence and law: the third decade. Artificial Intelligence and Law, 30(4), 561\u2013591.","journal-title":"Artificial Intelligence and Law"},{"key":"202_CR37","unstructured":"Walton, D. (2011). Introduction. In: G. Bongiovanni, D. Postema, A. Rotolo, G. Sartor, C. Valentini, D. Walton (eds.) Handbook in legal reasoning and argumentation (pp. ix-xxiii). Springer."},{"key":"202_CR38","unstructured":"Yu, F., Quartey, L., & Schilder, F. (2022). Legal prompting: Teaching a language model to think like a Lawyer."}],"container-title":["The Review of Socionetwork Strategies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12626-026-00202-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12626-026-00202-3","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12626-026-00202-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T02:00:03Z","timestamp":1776736803000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12626-026-00202-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,4]]},"references-count":38,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,4]]}},"alternative-id":["202"],"URL":"https:\/\/doi.org\/10.1007\/s12626-026-00202-3","relation":{},"ISSN":["2523-3173","1867-3236"],"issn-type":[{"value":"2523-3173","type":"print"},{"value":"1867-3236","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,4]]},"assertion":[{"value":"31 October 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"5 February 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 March 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}