{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T17:10:27Z","timestamp":1793121027675,"version":"4.5.0"},"reference-count":54,"publisher":"MDPI AG","issue":"13","license":[{"start":{"date-parts":[[2026,7,6]],"date-time":"2026-07-06T00:00:00Z","timestamp":1783296000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Committee of Science of the Ministry of Science and Higher Education of the Republic of Kazakhstan","award":["BR27101389"],"award-info":[{"award-number":["BR27101389"]}]},{"award":["BR27101389"],"award-info":[{"award-number":["BR27101389"]}],"id":[{"id":"https:\/\/ror.org\/05tyne317","id-type":"ROR","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Applied Sciences"],"abstract":"<jats:p>This paper reports a systematic study of low-rank adaptation (LoRA)-based fine-tuning applied to Qwen3 language models (4B, 8B, and 14B parameters) for the task of legal question answering within the jurisdiction of the Republic of Kazakhstan. The bilingual dataset comprises 63,114 question\u2013answer pairs (76.2% Russian, 23.8% Kazakh) covering 11 legal domains. Models are evaluated through both automated metrics (BERTScore, citation accuracy, and hallucination rate) and blind expert assessment by a panel of two practising legal experts. Key findings: (1) all fine-tuned models reach BERTScore F1 close to 90% (89.6\u201390.2%) versus 82.2\u201383.1% for untuned base models; (2) fine-tuned models outperform GPT-4o (87.2%) and GPT-4o-mini (86.7%) on semantic similarity while exhibiting far lower hallucination rates (27\u201329% vs. 83\u201390%); (3) blind expert assessment confirms the advantage of fine-tuned models, with panel mean completeness scores of 4.28\/5 versus 1.95\/5 for base models (quadratically weighted Cohen\u2019s \u03ba = 0.80\u20130.95 across rating dimensions, indicating substantial to almost perfect inter-rater agreement); and (4) we identify a practical scaling plateau: paired Wilcoxon tests (n = 500) detect statistically significant but practically small differences across the 4B, 8B, and 14B fine-tuned variants (largest mean gap 0.67 pp on BERTScore F1; Cohen\u2019s |d_z| \u2264 0.34), gains too small to justify the 3.5\u00d7 parameter increase. These findings show that parameter-efficient adaptation of compact open-source models can match or exceed commercial LLMs for specialised legal QA in a low-resource bilingual context. We note one scope restriction: three domains (administrative, criminal, and housing law) are represented only in Russian, so the model is not validated for Kazakh language queries in these areas.<\/jats:p>","DOI":"10.3390\/app16136777","type":"journal-article","created":{"date-parts":[[2026,7,6]],"date-time":"2026-07-06T14:01:08Z","timestamp":1783346468000},"page":"6777","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["Fine-Tuning Qwen3 Models for the Legal Domain of Kazakhstan: A Comparative Study of LoRA-Adapted Models for Bilingual Legal Question Answering"],"prefix":"10.3390","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0425-6527","authenticated-orcid":false,"given":"Arman","family":"Yeleussinov","sequence":"first","affiliation":[{"name":"Faculty of Information Technology, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Zholdas","family":"Buribayev","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nurbol","family":"Beisov","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-5776-0971","authenticated-orcid":false,"given":"Nurlykhan","family":"Kalzhanov","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4621-6646","authenticated-orcid":false,"given":"Maxatbek","family":"Satymbekov","sequence":"additional","affiliation":[{"name":"School of STEM Education, Shakarim University, Semey 071412, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ualikhan","family":"Akhatov","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7782-409X","authenticated-orcid":false,"given":"Yerbol","family":"Alimkulov","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, Al-Farabi Kazakh National University, Almaty 050040, Kazakhstan"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2026,7,6]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1016\/j.aiopen.2024.09.002","article-title":"Large language models in law: A survey","volume":"5","author":"Lai","year":"2024","journal-title":"AI Open"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Janatian, S., Westermann, H., Tan, J., Savelka, J., and Benyekhlef, K. (2023). From text to structure: Using large language models to support the development of legal expert systems. arXiv.","DOI":"10.3233\/FAIA230962"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1977","DOI":"10.1057\/s41599-025-05924-3","article-title":"Large language models in legal systems: A survey","volume":"12","author":"Dehghani","year":"2025","journal-title":"Humanit. Soc. Sci. Commun."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3777009","article-title":"Natural language processing for the legal domain: A survey of tasks, datasets, models and challenges","volume":"58","author":"Ariai","year":"2025","journal-title":"ACM Comput. Surv."},{"key":"ref_5","unstructured":"Li, H., Chen, J., Yang, J., Ai, Q., Jia, W., Liu, Y., Lin, K., Wu, Y., Yuan, G., and Hu, Y. (2025). Large language models meet legal artificial intelligence: A survey. arXiv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"64","DOI":"10.1093\/jla\/laae003","article-title":"Large legal fictions: Profiling legal hallucinations in large language models","volume":"16","author":"Dahl","year":"2024","journal-title":"J. Legal Anal."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Krasadakis, P., Sakkopoulos, E., and Verykios, V.S. (2024). A survey on challenges and advances in natural language processing with a focus on legal informatics and low-resource languages. Electronics, 13.","DOI":"10.3390\/electronics13030648"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"21582440251329663","DOI":"10.1177\/21582440251329663","article-title":"Legal judgment prediction using NLP and ML methods: A systematic literature review","volume":"15","author":"Dina","year":"2025","journal-title":"SAGE Open"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Joshi, P., Santy, S., Budhiraja, A., Bali, K., and Choudhury, M. (2020, January 5\u201310). State and fate of linguistic diversity and inclusion in the NLP world. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Online.","DOI":"10.18653\/v1\/2020.acl-main.560"},{"key":"ref_10","unstructured":"Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., and Chen, W. (2022, January 25). LoRA: Low-rank adaptation of large language models. Proceedings of the International Conference on Learning Representations ICLR, Virtual."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1007\/s10462-025-11236-4","article-title":"Parameter-efficient fine-tuning in large language models: A survey of methodologies","volume":"58","author":"Wang","year":"2025","journal-title":"Artif. Intell. Rev."},{"key":"ref_12","unstructured":"Xu, L., Xie, H., Qin, S.-Z.J., Tao, X., and Wang, F.L. (2024). A survey on parameter-efficient fine-tuning methods for large language models. arXiv."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Dettmers, T., Pagnoni, A., Holtzman, A., and Zettlemoyer, L. (2023, January 10\u201316). QLoRA: Efficient finetuning of quantized LLMs. Proceedings of the Thirty-Seventh Conference on Neural Information Processing Systems NeurIPS, 2023, New Orleans, LA, USA.","DOI":"10.52202\/075280-0441"},{"key":"ref_14","unstructured":"Kaplan, J., McCandlish, S., Henighan, T., Brown, T.B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. (2020). Scaling laws for neural language models. arXiv."},{"key":"ref_15","unstructured":"Yue, S., Chen, W., Wang, S., Li, B., Shen, C., Liu, S., Zhou, Y., Xiao, Y., Yun, S., and Huang, X. (2023). DISC-LawLLM: Fine-tuning large language models for intelligent legal services. arXiv."},{"key":"ref_16","unstructured":"Cui, J., Ning, M., Li, Z., Chen, B., Yan, Y., Li, H., Ling, B., Tian, Y., and Yuan, L. (2023). ChatLaw: A multi-agent collaborative framework for automatic Chinese legal analysis. arXiv."},{"key":"ref_17","unstructured":"Colombo, P., Pires, T.P., Boudiaf, M., Culver, D., Melo, R., Corro, C., Martins, A.F.T., Esposito, F., Raposo, V.L., and Morgado, S. (2024). SaulLM-7B: A pioneering large language model for law. arXiv."},{"key":"ref_18","unstructured":"Zhong, H., Fei, Z., Shen, X., Zhu, D., Wang, X., Ge, J., Cao, S., Wang, M., Zhou, F., and Cao, J. (2025, January 19\u201324). InternLM-Law: An open source Chinese legal large language model. Proceedings of the COLING, 2025, Abu Dhabi, United Arab Emirates."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Kim, Y., Choi, Y., Choi, E., Choi, J., Park, H.J., and Hwang, W. (2024, January 12\u201316). Developing a pragmatic benchmark for assessing Korean legal language understanding in large language models. Proceedings of the EMNLP Findings, 2024, Miami, FL, USA.","DOI":"10.18653\/v1\/2024.findings-emnlp.319"},{"key":"ref_20","unstructured":"Zhou, J., Song, P.-X., Yang, X.-W., Jin, Y.-X., Guo, L.-Z., and Li, Y.-F. (2024). LawGPT: A Chinese legal knowledge-enhanced large language model. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Chalkidis, I., Fergadiotis, M., Malakasiotis, P., Aletras, N., and Androutsopoulos, I. (2020, January 16\u201320). LEGAL-BERT: The muppets straight out of law school. Proceedings of the EMNLP Findings, 2020, Online.","DOI":"10.18653\/v1\/2020.findings-emnlp.261"},{"key":"ref_22","unstructured":"Guha, N., Nyarko, J., Ho, D.E., R\u00e9, C., Chilton, A., Narayana, A., Chohlas-Wood, A., Peters, A., Waldon, B., and Rockmore, D.N. (2023, January 10\u201316). LegalBench: A collaboratively built benchmark for measuring legal reasoning in large language models. Proceedings of the Thirty-Seventh Conference on Neural Information Processing Systems NeurIPS, 2023, New Orleans, LA, USA."},{"key":"ref_23","unstructured":"Han, Z., Gao, C., Liu, J., Zhang, J., and Zhang, S.Q. (2024). Parameter-efficient fine-tuning for large models: A comprehensive survey. arXiv."},{"key":"ref_24","unstructured":"Houlsby, N., Giurgiu, A., Jastrzebski, S., Morrone, B., de Laroussilhe, Q., Gesmundo, A., Attariyan, M., and Gelly, S. (2019, January 9\u201315). Parameter-efficient transfer learning for NLP. Proceedings of the 36th International Conference on Machine Learning ICML, 2019, Long Beach, CA, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Li, X.L., and Liang, P. (2021, January 1\u20136). Prefix-tuning: Optimizing continuous prompts for generation. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics ACL-IJCNLP, 2021, Online.","DOI":"10.18653\/v1\/2021.acl-long.353"},{"key":"ref_26","unstructured":"Huang, Q., Tao, M., Zhang, C., An, Z., Jiang, C., Chen, Z., Wu, Z., and Feng, Y. (2023). Lawyer LLaMA technical report. arXiv."},{"key":"ref_27","unstructured":"Liu, H., Liao, Y., Meng, Y., and Wang, Y. (2026, May 12). LawGPT_zh: A Chinese Legal Large Language Model Based on ChatGLM. GitHub 2023. Available online: https:\/\/github.com\/LiuHC0428\/LAW_GPT."},{"key":"ref_28","unstructured":"Shu, D., Zhao, H., Liu, X., Demeter, D., Du, M., and Zhang, Y. (2024, January 21\u201325). LawLLM: Law large language model for the US legal system. Proceedings of the CIKM, 2024, Boise, ID, USA."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de Las Casas, D., Hendricks, L.A., Welbl, J., and Clark, A. (December, January 28). Training compute-optimal large language models. Proceedings of the Thirty-Sixth Conference on Neural Information Processing Systems NeurIPS, 2022, New Orleans, LA, USA.","DOI":"10.52202\/068431-2176"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1109\/JPROC.2020.3004555","article-title":"A comprehensive survey on transfer learning","volume":"109","author":"Zhuang","year":"2021","journal-title":"Proc. IEEE"},{"key":"ref_31","unstructured":"Zhang, S., Li, Y., Cui, L., Cai, D., Liu, L., Fu, T., Huang, X., Zhao, E., Zhang, Y., and Chen, Y. (2023). Siren\u2019s song in the AI ocean: A survey on hallucination in large language models. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1111\/jels.12413","article-title":"Hallucination-free? Assessing the reliability of leading AI legal research tools","volume":"22","author":"Magesh","year":"2025","journal-title":"J. Empir. Legal Stud."},{"key":"ref_33","unstructured":"Hu, Y., Gan, L., Xiao, W., Kuang, K., and Wu, F. (2025). Fine-tuning large language models for improving factuality in legal question answering. arXiv."},{"key":"ref_34","unstructured":"Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., and Artzi, Y. (2020, January 30). BERTScore: Evaluating text generation with BERT. Proceedings of the International Conference on Learning Representations ICLR, 2020, Addis Ababa, Ethiopia."},{"key":"ref_35","unstructured":"Lin, C.-Y. (2004, January 21\u201326). ROUGE: A package for automatic evaluation of summaries. Proceedings of the ACL Workshop, 2004, Barcelona, Spain."},{"key":"ref_36","unstructured":"Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., and Xing, E.P. (2024, January 10\u201315). Judging LLM-as-a-judge with MT-Bench and Chatbot Arena. Proceedings of the 38th Annual Conference on Neural Information Processing Systems NeurIPS, 2024, Vancouver, BC, Canada."},{"key":"ref_37","unstructured":"Martin, L., Whitehouse, N., Yiu, S., Catterson, L., and Perera, R. (2024). Better call GPT: Comparing large language models against lawyers. arXiv."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Yeshpanov, R., Efimov, P., Boytsov, L., Shalkarbayuli, A., and Braslavski, P. (2024, January 20\u201325). KazQAD: Kazakh open-domain question answering dataset. Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation LREC-COLING, Torino, Italy.","DOI":"10.63317\/3j6y9kcsnpp9"},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Togmanov, M., Mukhituly, N., Turmakhan, D., Mansurov, J., Goloburda, M., Sakip, A., Xie, Z., Wang, Y., Syzdykov, B., and Laiyk, N. (2025). KazMMLU: Evaluating language models on Kazakh, Russian, and regional knowledge of Kazakhstan. arXiv.","DOI":"10.18653\/v1\/2025.acl-long.701"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Rakhimova, D., Turarbek, A., Karyukin, V., Sarsenbayeva, A., and Alieyev, R. (2025). Legal AI in low-resource languages: Building and evaluating QA systems for the Kazakh legislation. Computers, 14.","DOI":"10.3390\/computers14090354"},{"key":"ref_41","unstructured":"Faisal, F., and Yousaf, U. (2024). LEGAL-UQA: A low-resource Urdu-English dataset for legal question answering. arXiv."},{"key":"ref_42","unstructured":"Bahaj, A., and Ghogho, M. (2025). MizanQA: Benchmarking large language models on Moroccan legal question answering. arXiv."},{"key":"ref_43","unstructured":"Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., and Lv, C. (2025). Qwen3 technical report. arXiv."},{"key":"ref_44","unstructured":"Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., and Wei, H. (2024). Qwen2.5 technical report. arXiv."},{"key":"ref_45","unstructured":"Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, \u0141., and Polosukhin, I. (2017, January 4\u20139). Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems NeurIPS, 2017, Long Beach, CA, USA."},{"key":"ref_46","unstructured":"Han, D., and Han, M. (2026, May 12). Unsloth: Efficient LLM Fine-Tuning Framework. GitHub 2023. Available online: https:\/\/github.com\/unslothai\/unsloth."},{"key":"ref_47","unstructured":"Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozi\u00e8re, B., Goyal, N., Hambro, E., and Azhar, F. (2023). LLaMA: Open and efficient foundation language models. arXiv."},{"key":"ref_48","unstructured":"Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., and Askell, A. (2020, January 6\u201312). Language models are few-shot learners. Proceedings of the Thirty-Fourth Annual Conference on Neural Information Processing Systems NeurIPS, 2020, Virtual."},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C.L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., and Ray, A. (December, January 28). Training language models to follow instructions with human feedback. Proceedings of the Thirty-Sixth Conference on Neural Information Processing Systems NeurIPS, 2022, New Orleans, LA, USA.","DOI":"10.52202\/068431-2011"},{"key":"ref_50","unstructured":"OpenAI (2026, May 12). GPT-4o Technical Report. Available online: https:\/\/openai.com\/index\/hello-gpt-4o\/."},{"key":"ref_51","unstructured":"Cohen, J. (1988). Statistical Power Analysis for the Behavioral Sciences, Lawrence Erlbaum Associates. [2nd ed.]."},{"key":"ref_52","unstructured":"Wei, J., Bosma, M., Zhao, V.Y., Guu, K., Yu, A.W., Lester, B., Du, N., Dai, A.M., and Le, Q.V. (2022, January 25). Finetuned language models are zero-shot learners. Proceedings of the International Conference on Learning Representations ICLR, Virtual."},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"159","DOI":"10.2307\/2529310","article-title":"The measurement of observer agreement for categorical data","volume":"33","author":"Landis","year":"1977","journal-title":"Biometrics"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Krippendorff, K. (2018). Content Analysis: An Introduction to Its Methodology, SAGE Publications. [4th ed.].","DOI":"10.4135\/9781071878781"}],"container-title":["Applied Sciences"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2076-3417\/16\/13\/6777\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,6]],"date-time":"2026-07-06T16:09:55Z","timestamp":1783354195000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2076-3417\/16\/13\/6777"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,7,6]]},"references-count":54,"journal-issue":{"issue":"13","published-online":{"date-parts":[[2026,7]]}},"alternative-id":["app16136777"],"URL":"https:\/\/doi.org\/10.3390\/app16136777","relation":{},"ISSN":["2076-3417"],"issn-type":[{"value":"2076-3417","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,7,6]]}}}