{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,15]],"date-time":"2026-11-15T16:57:20Z","timestamp":1794761840196,"version":"4.9.1"},"reference-count":0,"publisher":"IOS Press","isbn-type":[{"value":"9781643684727","type":"print"},{"value":"9781643684734","type":"electronic"}],"license":[{"start":{"date-parts":[[2023,12,7]],"date-time":"2023-12-07T00:00:00Z","timestamp":1701907200000},"content-version":"unspecified","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2023,12,7]]},"abstract":"<jats:p>Large Language Models (LLMs) face limitations in logical reasoning, which restrict their applicability in critical domains such as law. Current evaluation methods often lead to inaccurate assessments of LLMs\u2019 capabilities due to their simplicity. This paper presents a refined evaluation method for assessing LLMs\u2019 capability to answer legal questions by eliminating the possibility of obtaining correct responses by chance. Furthermore, we introduce the LogiLaw dataset, which aims to enhance the models\u2019 logical reasoning capacities in general and legal reasoning specifically. By leveraging the refined evaluation technique, the LogiLaw dataset, and the proposed Reinforcement Learning from Logical Feedback (RLLF) approach, our work aims to open new avenues for research to bolster LLMs\u2019 performance in law and other logic-intensive disciplines while addressing the shortcomings of conventional evaluation approaches.<\/jats:p>","DOI":"10.3233\/faia230967","type":"book-chapter","created":{"date-parts":[[2023,12,11]],"date-time":"2023-12-11T04:37:22Z","timestamp":1702269442000},"source":"Crossref","is-referenced-by-count":2,"title":["LogiLaw Dataset Towards Reinforcement Learning from Logical Feedback (RLLF)"],"prefix":"10.3233","author":[{"given":"Ha-Thanh","family":"Nguyen","sequence":"first","affiliation":[{"name":"National Institute of Informatics, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Wachara","family":"Fungwacharakorn","sequence":"additional","affiliation":[{"name":"National Institute of Informatics, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ken","family":"Satoh","sequence":"additional","affiliation":[{"name":"National Institute of Informatics, Japan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"7437","container-title":["Frontiers in Artificial Intelligence and Applications","Legal Knowledge and Information Systems"],"original-title":[],"link":[{"URL":"https:\/\/ebooks.iospress.nl\/pdf\/doi\/10.3233\/FAIA230967","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,11]],"date-time":"2023-12-11T04:37:23Z","timestamp":1702269443000},"score":1,"resource":{"primary":{"URL":"https:\/\/ebooks.iospress.nl\/doi\/10.3233\/FAIA230967"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,12,7]]},"ISBN":["9781643684727","9781643684734"],"references-count":0,"URL":"https:\/\/doi.org\/10.3233\/faia230967","relation":{},"ISSN":["0922-6389","1879-8314"],"issn-type":[{"value":"0922-6389","type":"print"},{"value":"1879-8314","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,12,7]]}}}