{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,12]],"date-time":"2026-10-12T19:05:42Z","timestamp":1791831942621,"version":"4.3.4"},"reference-count":36,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2025,8,1]],"date-time":"2025-08-01T00:00:00Z","timestamp":1754006400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Wuhan East Lake High-Tech Development Zone (also known as the Optics Valley of China, or OVC) National Comprehensive Experimental Base for Governance of Intelligent Society","award":["2022YFC3301900"],"award-info":[{"award-number":["2022YFC3301900"]}]},{"name":"Ministry of Science and Technology of the People\u2019s Republic of China","award":["2022YFC3301900"],"award-info":[{"award-number":["2022YFC3301900"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Systems"],"abstract":"<jats:p>The justice system has increasingly applied AI techniques for legal judgment to enhance efficiency. However, most AI techniques focus on decision-making outcomes, failing to capture the deliberative nature of the real-world judicial process. To address these challenges, we propose a large language model-based multi-agent framework named AgentsBench. Our approach leverages multiple LLM-driven agents that simulate the discussion process of the Chinese judicial bench, which is often composed of professional and lay judge agents. We conducted experiments on a legal judgment prediction task, and the results show that our framework outperforms existing LLM-based methods in terms of performance and decision quality. By incorporating these elements, our framework reflects real-world judicial processes more closely, enhancing accuracy, fairness, and societal consideration. While the simulation is based on China\u2019s lay judge system, our framework is generalizable and can be adapted to various legal scenarios and other legal systems involving collective decision-making processes.<\/jats:p>","DOI":"10.3390\/systems13080641","type":"journal-article","created":{"date-parts":[[2025,8,5]],"date-time":"2025-08-05T08:46:55Z","timestamp":1754383615000},"page":"641","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["AgentsBench: A Multi-Agent LLM Simulation Framework for Legal Judgment Prediction"],"prefix":"10.3390","volume":"13","author":[{"given":"Cong","family":"Jiang","sequence":"first","affiliation":[{"name":"Peking University Law School, Peking University, Beijing 100871, China"},{"name":"Institute for AI, Peking University, Beijing 100871, China"},{"name":"PKU-WUHAN Institute for Artiffcial Intelligence, Wuhan 430072, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaolei","family":"Yang","sequence":"additional","affiliation":[{"name":"Peking University Law School, Peking University, Beijing 100871, China"},{"name":"Institute for AI, Peking University, Beijing 100871, China"},{"name":"PKU-WUHAN Institute for Artiffcial Intelligence, Wuhan 430072, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2025,8,1]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Zhong, H., Xiao, C., Tu, C., Zhang, T., Liu, Z., and Sun, M. (2020). How does nlp benefit legal system: A summary of legal artificial intelligence. arXiv.","DOI":"10.18653\/v1\/2020.acl-main.466"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"105904","DOI":"10.1016\/j.clsr.2023.105904","article-title":"An entity-centric approach to manage court judgments based on natural language processing","volume":"52","author":"Bellandi","year":"2024","journal-title":"Comput. Law Secur. Rev."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Xu, S., Santosh, T., Ichim, O., Plank, B., and Grabmair, M. (2024). Through the lens of split vote: Exploring disagreement, difficulty and calibration in legal case outcome classification. arXiv.","DOI":"10.18653\/v1\/2024.acl-long.13"},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"102050","DOI":"10.1109\/ACCESS.2023.3317083","article-title":"A survey on legal judgment prediction: Datasets, metrics, models and challenges","volume":"11","author":"Cui","year":"2023","journal-title":"IEEE Access"},{"key":"ref_5","first-page":"179","article-title":"A tale of two juries: Lay participation comes to Japanese and Chinese courts","volume":"25","author":"Landsman","year":"2007","journal-title":"UCLA Pac. Basin Law J."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Dawson, J.P. (1960). A History of Lay Judges, Harvard University Press.","DOI":"10.4159\/harvard.9780674331372"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Heppenstall, A., Malleson, N., and Crooks, A. (2016). \u201cSpace, the Final Frontier\u201d: How good are agent-based models at simulating individuals and space in cities?. Systems, 4.","DOI":"10.3390\/systems4010009"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Benthall, S., and Strandburg, K.J. (2021). Agent-based modeling as a legal theory tool. Front. Phys., 9.","DOI":"10.3389\/fphy.2021.666386"},{"key":"ref_9","unstructured":"Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., Jiang, L., Zhang, X., Zhang, S., and Liu, J. (2023). Autogen: Enabling next-gen LLM applications via multi-agent conversation. arXiv."},{"key":"ref_10","unstructured":"Hong, S., Zheng, X., Chen, J., Cheng, Y., Wang, J., Zhang, C., Wang, Z., Yau, S.K.S., Lin, Z., and Zhou, L. (2023). Metagpt: Meta programming for multi-agent collaborative framework. arXiv."},{"key":"ref_11","unstructured":"Abdelnabi, S., Gomaa, A., Sivaprasad, S., Sch\u00f6nherr, L., and Fritz, M. (2024, January 7\u201311). Llm-deliberation: Evaluating llms with interactive multi-agent negotiation games. Proceedings of the ICLR 2024, Vienna, Austria."},{"key":"ref_12","unstructured":"Park, J.S., O\u2019Brien, J., Cai, C.J., Morris, M.R., Liang, P., and Bernstein, M.S. (November, January 29). Generative agents: Interactive simulacra of human behavior. Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, San Francisco, CA, USA."},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"He, Z., Cao, P., Chen, Y., Liu, K., Li, R., Sun, M., and Zhao, J. (2023, January 6\u201310). Lego: A multi-agent collaborative framework with role-playing and iterative feedback for causality explanation generation. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore.","DOI":"10.18653\/v1\/2023.findings-emnlp.613"},{"key":"ref_14","unstructured":"Han, S., Zhang, Q., Yao, Y., Jin, W., and Xu, Z. (2024). LLM multi-agent systems: Challenges and open problems. arXiv."},{"key":"ref_15","unstructured":"Li, J., Wang, S., Zhang, M., Li, W., Lai, Y., Kang, X., Ma, W., and Liu, Y. (2024). Agent hospital: A simulacrum of hospital with evolvable medical agents. arXiv."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Qian, C., Liu, W., Liu, H., Chen, N., Dang, Y., Li, J., Yang, C., Chen, W., Su, Y., and Cong, X. (2024, January 11\u201316). Chatdev: Communicative agents for software development. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Bangkok, Thailand.","DOI":"10.18653\/v1\/2024.acl-long.810"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.2307\/1951767","article-title":"Predicting supreme court decisions mathematically: A quantitative analysis of the \u201cright to counsel\u201d cases","volume":"51","author":"Kort","year":"1957","journal-title":"Am. Political Sci. Rev."},{"key":"ref_18","unstructured":"Xiao, C., Zhong, H., Guo, Z., Tu, C., Liu, Z., Sun, M., Feng, Y., Han, X., Hu, Z., and Wang, H. (2018). Cail2018: A large-scale legal dataset for judgment prediction. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Chalkidis, I., Androutsopoulos, I., and Aletras, N. (2019). Neural legal judgment prediction in English. arXiv.","DOI":"10.18653\/v1\/P19-1424"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Luo, B., Feng, Y., Xu, J., Zhang, X., and Zhao, D. (2017, January 7\u201311). Learning to predict charges for criminal cases with legal basis. Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Denmark.","DOI":"10.18653\/v1\/D17-1289"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"105863","DOI":"10.1016\/j.clsr.2023.105863","article-title":"Improving colloquial case legal judgment prediction via abstractive text summarization","volume":"51","author":"Hong","year":"2023","journal-title":"Comput. Law Secur. Rev."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"105902","DOI":"10.1016\/j.clsr.2023.105902","article-title":"Legal knowledge management for prosecutors based on judgment prediction and error analysis from indictments","volume":"52","author":"Chien","year":"2024","journal-title":"Comput. Law Secur. Rev."},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"An, Z., Huang, Q., Jiang, C., Feng, Y., and Zhao, D. (2022, January 7\u201311). Do charge prediction models learn legal theory?. Proceedings of the Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates.","DOI":"10.18653\/v1\/2022.findings-emnlp.275"},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Medvedeva, M., and Mcbride, P. (2023, January 7). Legal judgment prediction: If you are going to do it, do it right. Proceedings of the Natural Legal Language Processing Workshop 2023, Singapore.","DOI":"10.18653\/v1\/2023.nllp-1.9"},{"key":"ref_25","unstructured":"Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F.L., Almeida, D., Altenschmidt, J., Altman, S., and Anadkat, S. (2023). GPT-4 technical report. arXiv."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"105864","DOI":"10.1016\/j.clsr.2023.105864","article-title":"Fine-tuning gpt-3 for legal rule classification","volume":"51","author":"Liga","year":"2023","journal-title":"Comput. Law Secur. Rev."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Jiang, H., Zhang, X., Mahari, R., Kessler, D., Ma, E., August, T., Li, I., Pentland, A., Kim, Y., and Kabbara, J. (2024). Leveraging large language models for learning complex legal concepts through storytelling. arXiv.","DOI":"10.18653\/v1\/2024.acl-long.388"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Deroy, A., Ghosh, K., and Ghosh, S. (2024). Applicability of large language models and generative models for legal case judgement summarization. Artif. Intell. Law.","DOI":"10.1007\/s10506-024-09411-z"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Jiang, C., and Yang, X. (2023, January 19\u201323). Legal syllogism prompting: Teaching large language models for legal judgment prediction. Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, Braga, Portugal.","DOI":"10.1145\/3594536.3595170"},{"key":"ref_30","unstructured":"Licari, D., and Comand\u00e8, G. (2022, January 26). Italian-legal-bert: A pre-trained transformer language model for Italian law. Proceedings of the Knowledge Management for Law Workshop (KM4LAW), Bozen-Bolzano, Italy."},{"key":"ref_31","unstructured":"Huang, Q., Tao, M., Zhang, C., An, Z., Jiang, C., Chen, Z., Wu, Z., and Feng, Y. (2023). Lawyer llama technical report. arXiv."},{"key":"ref_32","unstructured":"Hamilton, S. (2023). Blind judgement: Agent-based supreme court modelling with gpt. arXiv."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Fei, Z., Shen, X., Zhu, D., Zhou, F., Han, Z., Zhang, S., Chen, K., Shen, Z., and Ge, J. (2023). Lawbench: Benchmarking legal knowledge of large language models. arXiv.","DOI":"10.18653\/v1\/2024.emnlp-main.452"},{"key":"ref_34","unstructured":"Bai, J., Bai, S., Chu, Y., Cui, Z., Dang, K., Deng, X., Fan, Y., Ge, W., Han, Y., and Huang, F. (2023). Qwen technical report. arXiv."},{"key":"ref_35","first-page":"22199","article-title":"Large language models are zero-shot reasoners","volume":"35","author":"Kojima","year":"2022","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Blair-Stanek, A., Holzenberger, N., and Van Durme, B. (2023, January 19\u201323). Can gpt-3 perform statutory reasoning?. Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, Braga, Portugal.","DOI":"10.1145\/3594536.3595163"}],"container-title":["Systems"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-8954\/13\/8\/641\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:20:32Z","timestamp":1760034032000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-8954\/13\/8\/641"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,1]]},"references-count":36,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,8]]}},"alternative-id":["systems13080641"],"URL":"https:\/\/doi.org\/10.3390\/systems13080641","relation":{},"ISSN":["2079-8954"],"issn-type":[{"value":"2079-8954","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,1]]}}}