{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T02:40:22Z","timestamp":1776739222681,"version":"3.51.2"},"reference-count":19,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T00:00:00Z","timestamp":1771545600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T00:00:00Z","timestamp":1771545600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada.","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100000038","name":"Natural Sciences and Engineering Research Council of Canada.","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/100013373","name":"Alberta Machine Intelligence Institute","doi-asserted-by":"publisher","id":[{"id":"10.13039\/100013373","id-type":"DOI","asserted-by":"publisher"}]},{"name":"JST PRESTO"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Rev Socionetwork Strat"],"published-print":{"date-parts":[[2026,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>We summarize the 12th Competition on Legal Information Extraction and Entailment. In this edition, the competition included two tasks on case law, two tasks on statute law, plus a new pilot task on Tort law. The two case law components include an information retrieval task (Task 1), and the confirmation of an entailment relation between an existing case and an unseen case (Task 2). The statute law components include an information retrieval task (Task 3), and an entailment\/question-answering task based on retrieved civil code statutes (Task 4). The new pilot task involves tort prediction and rationale extraction. Participation was open to any group using any approach. Eight teams submitted a total of 21 runs for Task 1, achieving a top F1 score of 0.3604, with dominant approaches featuring multi-stage retrieval pipelines combining traditional IR with neural re-ranking methods. We summarize the variety of approaches, provide our official evaluation, and give a summary analysis of our data and submission results. In Task 2, the NOWJ team, which used BM25 for retrieval and DeepSeek-V3 and Qwen\/QwQ-32B for reranking, achieved the best score of 0.3195. Eight teams submitted a total of 22 runs for Task 3. The best-performing system employs a multi-stage retrieval approach: it first retrieves a limited number of candidate articles in the initial stage, then applies an LLM-based cross-encoder for re-ranking, and finally determines the relevant articles using multiple LLMs. This system achieves nearly perfect retrieval performance for questions with a single relevant article; however, it still faces challenges in retrieving all relevant articles for questions that have multiple relevant answers. 11 teams submitted a total of 29 runs for Task 4, achieving a top accuracy score of 0.9041, where the solution uses an LLM coupled with a prompt engineering approach. Most teams used LLMs but their approaches and the models used were quite different. The new pilot task received 10 runs from four teams. All the teams employed LLMs. The best performing runs are JAIST-LJPJT25 (acc.=0.765) by the CAPTAIN team for the tort prediction task and KIS5 (F1=0.712) by the KIS team for the rationale extraction task. Finally, based on the strong performance observed in Tasks 3 and 4 this year, we propose introducing a new task for the next COLIEE, focusing on statute retrieval.<\/jats:p>","DOI":"10.1007\/s12626-026-00199-9","type":"journal-article","created":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T05:14:44Z","timestamp":1771564484000},"page":"183-213","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["The COLIEE 2025 Competition on Legal Information Extraction and Entailment: Overview, Discussion, and Dataset Expansion"],"prefix":"10.1007","volume":"20","author":[{"given":"Randy","family":"Goebel","sequence":"first","affiliation":[]},{"given":"Yoshinobu","family":"Kano","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0009-0006-4486-9738","authenticated-orcid":false,"given":"Mi-Young","family":"Kim","sequence":"additional","affiliation":[]},{"given":"Calum","family":"Kwan","sequence":"additional","affiliation":[]},{"given":"Juliano","family":"Rabelo","sequence":"additional","affiliation":[]},{"given":"Ken","family":"Satoh","sequence":"additional","affiliation":[]},{"given":"Hiroaki","family":"Yamada","sequence":"additional","affiliation":[]},{"given":"Masaharu","family":"Yoshioka","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2026,2,20]]},"reference":[{"key":"199_CR1","unstructured":"Baek, E., Dai, J., Hasan, H.M.Q., Kim, Y., Babiker, H., Kim, M.Y., & Goebel, R. (2025). From tf-idf to instruction-tuned llms: Hybrid legal reasoning systems for coliee 2025. In: Workshop of the Twelfth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR2","unstructured":"Chu, Q.H., Chu, Q.N.H., Luu, T.S., Vo, T.T., Nguyen, D., Nguyen, M.P., & Nguyen, L.M. (2025). Captain at coliee 2025: Enhancing legal text processing and structural analysis with large language models. In: Workshop of the Tenth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR3","series-title":"Workshops Lecture Notes in Computer Science","doi-asserted-by":"crossref","first-page":"207","DOI":"10.1007\/978-3-319-98654-8","volume-title":"JSAI-isAI","author":"R Hoshino","year":"2018","unstructured":"Hoshino, R., Taniguchi, R., Kiyota, N., & Kano, Y. (2018). Question answering system for legal bar examination using predicate argument structure. Workshops Lecture Notes in Computer ScienceIn K. Kojima, M. Sakamoto, K. Mineshima, & K. Satoh (Eds.), JSAI-isAI (Vol. 11717, pp. 207\u2013220). Springer."},{"key":"199_CR4","unstructured":"Jain, B., Harde, P., Sadikov, T., Kujur, E.N., & Jain, S. (2025). Sil@coliee 2025: A cascading framework for finding relevant case laws. In: Workshop of the Twelfth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR5","unstructured":"Kadowaki, K., & Kano, Y. (2025). A modernbert-based system by team kis for the coliee 2025 pilot task: Toward robust evaluation in legal judgment prediction. In: Workshop of the Twelfth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR6","unstructured":"Leburu-Dingalo, T.K., Thuma, E., Mosweunyane, G., Motlogelwa, N.P., & Mudongo, M.N. (2025). Leveraging rhetorical role-based summarization for legal case retrieval. In: Workshop of the Twelfth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR7","unstructured":"Miyao, Y., Ishida, S., Okamoto, T., Han, N., Mousterou, A., Fourrier, C., Hayashi, T., & Tachibana, Y. (2024). Open Japanese llm leaderboard. https:\/\/huggingface.co\/spaces\/llm-jp\/open-japanese-llm-leaderboard."},{"key":"199_CR8","unstructured":"Mizuno, T., & Kano, Y. (2025). Hierarchical and referential structure-aware retrieval for statutory articles using graph neural networks. In: Workshop of the Tenth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR9","unstructured":"Nguyen, H.T., Vuong, T.H.Y., Nguyen, T.M., Le, X.B., Le, T.K., Nguyen, K.H., Nguyen, H.T., & Nguyen, L.M. (2025). Nowj@coliee2025: A multi-stage framework integrating embedding models and large language models for legal retrieval and entailment. In: Workshop of the Twelfth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR10","unstructured":"Nguyen, T.H., Nguyen, K.V.H., Pham, N.A.T., Nguyen, N.M., Trieu, H.A., Le, N.K., Do, D.T., & Nguyen, L.M. (2025). Jnlp at coliee 2025: Hybrid large language model-based framework for legal information retrieval and entailment. In: Workshop of the Twelfth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR11","doi-asserted-by":"crossref","unstructured":"Onaga, T., & Kano, Y. (2025). Kis: Coliee 2025 task 4 solver using Japanese llm. In: Workshop of the Tenth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL).","DOI":"10.1007\/s12626-026-00209-w"},{"key":"199_CR12","doi-asserted-by":"publisher","first-page":"111","DOI":"10.1007\/s12626-022-00105-z","volume":"16","author":"J Rabelo","year":"2022","unstructured":"Rabelo, J., Goebel, R., Kim, M. Y., Kano, Y., Yoshioka, M., & Satoh, K. (2022). Overview and discussion of the competition on legal information extraction\/entailment (COLIEE) 2021. The Review of Socionetwork Strategies, 16, 111\u2013133. https:\/\/doi.org\/10.1007\/s12626-022-00105-z","journal-title":"The Review of Socionetwork Strategies"},{"key":"199_CR13","doi-asserted-by":"crossref","unstructured":"Steging, C., van Leeuwen, L., Zbiegie\u0144, T., Wedda, D., & Liu, J. (2025). Investigating expert-based prompt engineering for legal entailment tasks. In: Workshop of the Tenth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL).","DOI":"10.1007\/s12626-026-00202-3"},{"key":"199_CR14","unstructured":"Tang, Y., Qiu, R., & Huang, Z. (2025). Advancing legal case retrieval with large language models and graph neural networks. In: Workshop of the Twelfth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR15","unstructured":"Tjandra, B., Negara, M.S.N., & Wicaksono, A.F. (2025). Irnlpui at coliee 2025: Utilization of llms for statute law retrieval and legal entailment task. In: Workshop of the Tenth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR16","unstructured":"Wehnert, S., Valappil, B.B.C., & Luca, E.W.D. (2025). Llms, knowledge graphs, and hybrid search: Task-specific approaches to legal ai in coliee. In: Workshop of the Twelfth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR17","unstructured":"Wu, D., Lawrence, S., & Mansouri, B. (2025). Aiir lab at coliee 2025: Exploring applications of large language models for legal text retrieval and entailmen. In: Workshop of the Twelfth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192025) in the 21th International Conference on Artificial Intelligence and Law (ICAIL)."},{"key":"199_CR18","doi-asserted-by":"publisher","DOI":"10.1007\/s10506-024-09402-0","author":"H Yamada","year":"2024","unstructured":"Yamada, H., Tokunaga, T., Ohara, R., Tokutsu, A., Takeshita, K., & Sumida, M. (2024). Japanese tort-case dataset for rationale-supported legal judgment prediction. Artificial Intelligence and Law. https:\/\/doi.org\/10.1007\/s10506-024-09402-0","journal-title":"Artificial Intelligence and Law"},{"key":"199_CR19","unstructured":"Yoshioka, M., & Aoki, Y. (2023). Hukb at coliee 2023 statute law task. In: Workshop of the Tenth Competition on Legal Information Extraction\/Entailment (COLIEE\u20192023) in the 19th International Conference on Artificial Intelligence and Law (ICAIL)."}],"container-title":["The Review of Socionetwork Strategies"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12626-026-00199-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12626-026-00199-9","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12626-026-00199-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,21]],"date-time":"2026-04-21T02:00:08Z","timestamp":1776736808000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12626-026-00199-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,20]]},"references-count":19,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,4]]}},"alternative-id":["199"],"URL":"https:\/\/doi.org\/10.1007\/s12626-026-00199-9","relation":{},"ISSN":["2523-3173","1867-3236"],"issn-type":[{"value":"2523-3173","type":"print"},{"value":"1867-3236","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,20]]},"assertion":[{"value":"31 October 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 January 2026","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"20 February 2026","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"On behalf of all authors, the corresponding author states that there is no conflict of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Conflict of interest"}}]}}