{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,24]],"date-time":"2026-03-24T16:22:48Z","timestamp":1774369368659,"version":"3.50.1"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643686387","type":"electronic"}],"license":[{"start":{"date-parts":[[2025,12,2]],"date-time":"2025-12-02T00:00:00Z","timestamp":1764633600000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2025,12,2]]},"abstract":"<jats:p>Deploying Large Language Models (LLMs) for legal question-answering requires ensuring factual accuracy and logical coherence. Current evaluation metrics inadequately capture legal reasoning complexity, while expert assessments lack scalability. We propose a two-dimensional framework that independently measures Truthfulness and Reasoning Soundness in model outputs, applied to Italian asylum proceedings requiring evidence-based analysis. This dual-axis approach reveals critical issues\u2014such as legally correct answers derived through unsound or hallucinatory reasoning\u2014that standard metrics fail to detect. To enable large-scale application, we implement an automated LLM-as-a-Judge system with bias-mitigation techniques. Experimental results demonstrate strong correspondence between automated judgments and expert evaluations, confirming framework reliability. This work advances diagnostic methodology for assessing LLMs in legal domains, offering both theoretical insight and practical applicability toward more trustworthy and accountable legal AI systems.<\/jats:p>","DOI":"10.3233\/faia251579","type":"book-chapter","created":{"date-parts":[[2025,12,4]],"date-time":"2025-12-04T12:04:36Z","timestamp":1764849876000},"source":"Crossref","is-referenced-by-count":1,"title":["A Two-Dimensional Evaluation Framework for Factual and Reasoning Assessment of LLMs in Legal Question Answering"],"prefix":"10.3233","author":[{"given":"Sinan","family":"Gultekin","sequence":"first","affiliation":[{"name":"CIRSFID Alma-AI, Faculty of Law, University of Bologna, Italy"}]},{"given":"Matteo","family":"Rossi Reich","sequence":"additional","affiliation":[{"name":"CIRSFID Alma-AI, Faculty of Law, University of Bologna, Italy"}]},{"given":"Francesca","family":"Galloni","sequence":"additional","affiliation":[{"name":"CIRSFID Alma-AI, Faculty of Law, University of Bologna, Italy"}]},{"given":"Francesca","family":"Lagioia","sequence":"additional","affiliation":[{"name":"CIRSFID Alma-AI, Faculty of Law, University of Bologna, Italy"},{"name":"European University Institute, Law Department, Italy"}]},{"given":"Elena","family":"Consiglio","sequence":"additional","affiliation":[{"name":"University of Palermo, Italy"}]},{"given":"Giovanni","family":"Sartor","sequence":"additional","affiliation":[{"name":"CIRSFID Alma-AI, Faculty of Law, University of Bologna, Italy"},{"name":"European University Institute, Law Department, Italy"}]},{"given":"Sara","family":"Bagnato","sequence":"additional","affiliation":[{"name":"University of Roma LUMSA, Italy"}]}],"member":"7437","container-title":["Frontiers in Artificial Intelligence and Applications","Legal Knowledge and Information Systems"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/FAIA251579","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,12,4]],"date-time":"2025-12-04T12:04:37Z","timestamp":1764849877000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/FAIA251579"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,12,2]]},"ISBN":["9781643686387"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/faia251579","relation":{},"ISSN":["0922-6389","1879-8314"],"issn-type":[{"value":"0922-6389","type":"print"},{"value":"1879-8314","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,12,2]]}}}