{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T18:40:03Z","timestamp":1776883203213,"version":"3.51.2"},"reference-count":41,"publisher":"Wiley","issue":"2","license":[{"start":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T00:00:00Z","timestamp":1773014400000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"},{"start":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T00:00:00Z","timestamp":1773014400000},"content-version":"tdm","delay-in-days":0,"URL":"http:\/\/doi.wiley.com\/10.1002\/tdm_license_1.1"}],"content-domain":{"domain":["onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Topics in Cognitive Science"],"published-print":{"date-parts":[[2026,4]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Large Language Models (LLMs) with hundreds of billions of parameters have exhibited human\u2010like intelligence by learning from vast amounts of internet\u2010scale data. However, the uninterpretability of large\u2010scale neural networks raises concerns about the reliability of LLM. Studies have attempted to assess the psychometric properties of LLMs by borrowing concepts from human psychology to enhance their interpretability, but they fail to account for the fundamental differences between LLMs and humans. This results in high rejection rates when human scales are reused directly. Furthermore, these scales do not support the measurement of LLM psychological property variations in different languages. This paper introduces AIPsychoBench, a specialized benchmark tailored to assess the psychological properties of LLM. It uses a lightweight role\u2010playing prompt to bypass LLM alignment, improving the average effective response rate from 70.12% to 90.40%. Meanwhile, the average biases are only 3.3% (positive) and 2.1% (negative), which are significantly lower than the biases of 9.8% and 6.9%, respectively, caused by traditional jailbreak prompts. Furthermore, among the total of 112 psychometric subcategories, the score deviations for seven languages compared to English ranged from 5% to 20.2% in 43 subcategories, providing the first comprehensive evidence of the linguistic impact on the psychometrics of\u00a0LLM.<\/jats:p>","DOI":"10.1111\/tops.70041","type":"journal-article","created":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T19:33:54Z","timestamp":1773084834000},"update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["AIPsychoBench: Understanding the Psychometric Differences Between LLMs and Humans"],"prefix":"10.1111","volume":"18","author":[{"given":"Wei","family":"Xie","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology National University of Defense Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhenhua","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology National University of Defense Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shuoyoucheng","family":"Ma","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology National University of Defense Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Xiaobing","family":"Sun","sequence":"additional","affiliation":[{"name":"Institute of High Performance Computing, Agency for Science, Technology and Research (A*STAR)"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Kai","family":"Chen","sequence":"additional","affiliation":[{"name":"Institute of Information Engineering, Chinese Academy of Sciences"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Enze","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology National University of Defense Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Wei","family":"Liu","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology National University of Defense Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hanying","family":"Tong","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology National University of Defense Technology"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"311","published-online":{"date-parts":[[2026,3,9]]},"reference":[{"key":"e_1_2_9_2_1","unstructured":"0xk1h0. (2024).Chatgpt_dan.https:\/\/github.com\/0xk1h0\/ChatGPT_DANAccessed: 2025\u201005\u201010."},{"key":"e_1_2_9_3_1","unstructured":"Anthropic. (2024).Claude 3.5 Sonnet.https:\/\/www.anthropic.com\/claude\/sonnetAccessed: 2025\u201005\u201010."},{"key":"e_1_2_9_4_1","doi-asserted-by":"crossref","unstructured":"Bousetouane F.(2025).Physical AI agents: Integrating cognitive intelligence with real\u2010world action.http:\/\/arxiv.org\/abs\/2501.08944arXiv:2501.08944 [cs].","DOI":"10.32388\/HA0F5Z"},{"key":"e_1_2_9_5_1","unstructured":"Bubeck S. Chandrasekaran V. Eldan R. Gehrke J. Horvitz E. Kamar E. Lee P. Lee Y. T. Li Y. Lundberg S. Nori H. Palangi H. Ribeiro M. T. &Zhang Y.(2023).Sparks of artificial general intelligence: Early experiments with GPT\u20104.http:\/\/arxiv.org\/abs\/2303.12712arXiv:2303.12712 [cs]."},{"key":"e_1_2_9_6_1","unstructured":"Coda\u2010Forno J. Binz M. Wang J. X. &Schulz E.(2024).CogBench: A large language model walks into a psychology lab.http:\/\/arxiv.org\/abs\/2402.18225arXiv:2402.18225 [cs]."},{"key":"e_1_2_9_7_1","unstructured":"Coda\u2010Forno J. Witte K. Jagadish A. K. Binz M. Akata Z. &Schulz E.(2024).Inducing anxiety in large language models can induce bias.http:\/\/arxiv.org\/abs\/2304.11111arXiv:2304.11111 [cs]."},{"key":"e_1_2_9_8_1","unstructured":"Diaz\u2010Garcia J. A. &Carvalho J. P.(2025).A survey of textual cyber abuse detection using cutting\u2010edge language models and large language models.http:\/\/arxiv.org\/abs\/2501.05443arXiv:2501.05443 [cs]."},{"key":"e_1_2_9_9_1","doi-asserted-by":"crossref","unstructured":"Du Z. Qian Y. Liu X. Ding M. Qiu J. Yang Z. &Tang J.(2024).GLM: General language model pretraining with autoregressive blank infilling. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL 2024)(pp. 320\u2013335).","DOI":"10.18653\/v1\/2022.acl-long.26"},{"key":"e_1_2_9_10_1","unstructured":"Gemini Team. (2024).Gemini: A family of highly capable multimodal models.http:\/\/arxiv.org\/abs\/2312.11805arXiv:2312.11805 [cs]."},{"key":"e_1_2_9_11_1","doi-asserted-by":"publisher","DOI":"10.1111\/jopy.12674"},{"key":"e_1_2_9_12_1","doi-asserted-by":"crossref","unstructured":"Guo D. Yang D. Zhang H. Song J. Zhang R. Xu R. Zhu Q. Ma S. Wang P. Bi X. Zhang X. Yu X. Wu Y. Wu Z.~F. Gou Z. Shao Z. Li Z. Gao Z. Liu A. Xue B. Wang B. Wu B. Feng B. Lu C. Zhao C. Deng C. Zhang C. Ruan C. Dai D. Chen D. Ji D. Li E. Lin F. Dai F. Luo F. Hao G. Chen G. Li G. Zhang H. Bao H. Xu H. Wang H. Ding H. Xin H. Gao H. Qu H. Li H. Guo J. Li J. Wang J. Chen J. Yuan J. Qiu J. Li J. Cai J. L. Ni J. Liang J. Chen J. Dong K. Hu K. Gao K. Guan K. Huang K. Yu K. Wang L. Zhang L. Zhao L. Wang L. Zhang L. Xu L. Xia L. Zhang M. Zhang M. Tang M. Li M. Wang M. Li M. Tian N. Huang P. Zhang P. Wang Q. Chen Q. Du Q. Ge R. Zhang R. Pan R. Wang R. Chen R. J. Jin R. L. Chen R. Lu S. Zhou S. Chen S. Ye S. Wang S. Yu S. Zhou S. Pan S. Li S. S. Zhou S. Wu S. Ye S. Yun T. Pei T. Sun T. Wang T. Zeng W. Zhao W. Liu W. Liang W. Gao W. Yu W. Zhang W. Xiao W. L. An W. Liu X. Wang X. Chen X. Nie X. Cheng X. Liu X. Xie X. Liu X. Yang X. Li X. Su X. Lin X. Li X. Q. Jin X. Shen X. Chen X. Sun X. Wang X. Song X. Zhou X. Wang X. Shan X. Li Y. K. Wang Y. Q. Wei Y. X. Zhang Y. Xu Y. Li Y. Zhao Y. Sun Y. Wang Y. Yu Y. Zhang Y. Shi Y. Xiong Y. He Y. Piao Y. Wang Y. Tan Y. Ma Y. Liu Y. Guo Y. Ou Y. Wang Y. Gong Y. Zou Y. He Y. Xiong Y. Luo Y. You Y. Liu Y. Zhou Y. Zhu Y. X. Xu Y. Huang Y. Li Y. Zheng Y. Zhu Y. Ma Y. Tang Y. Zha Y. Yan Y. Ren Z. Z. Ren Z. Sha Z. Fu Z. Xu Z. Xie Z. Zhang Z. Hao Z. Ma Z. Yan Z. Wu Z. Gu Z. Zhu Z. Liu Z. Li Z. Xie Z. Song Z. Pan Z. Huang Z. Xu Z. Zhang Z. &Zhang Z.(2025).DeepSeek\u2010R1: Incentivizing reasoning capability in LLMs via reinforcement learning.https:\/\/arxiv.org\/abs\/2501.12948.","DOI":"10.1038\/s41586-025-09422-z"},{"key":"e_1_2_9_13_1","unstructured":"Guo J. Guo Y. Li M. &Tan S.(2025).FLAME: Financial Large\u2010Language Model Assessment and Metrics Evaluation.http:\/\/arxiv.org\/abs\/2501.06211arXiv:2501.06211 [cs]."},{"key":"e_1_2_9_14_1","doi-asserted-by":"crossref","unstructured":"Guo J. Luo Y. Wei D. Zhang L. Li Z. Shang H. Rao Z. Li S. Yang J. Wu Z. &Yang H.(2025).Doc\u2010guided sent2sent++: A sent2sent++ agent with doc\u2010guided memory for document\u2010level machine translation.http:\/\/arxiv.org\/abs\/2501.08523arXiv:2501.08523 [cs].","DOI":"10.1007\/978-981-95-3349-7_18"},{"key":"e_1_2_9_15_1","unstructured":"Hagendorff T. Dasgupta I. Binz M. Chan S. C. Y. Lampinen A. Wang J. X. Akata Z. &Schulz E.(2024).Machine psychology. arXiv:2303.13988 [cs]."},{"key":"e_1_2_9_16_1","doi-asserted-by":"publisher","DOI":"10.1038\/s43588-023-00527-x"},{"key":"e_1_2_9_17_1","doi-asserted-by":"crossref","unstructured":"Hegde D. Yasarla R. Cai H. Han S. Bhattacharyya A. Mahajan S. Liu L. Garrepalli R. Patel V. M. &Porikli F.(2025).Distilling multi\u2010modal large language models for autonomous driving.http:\/\/arxiv.org\/abs\/2501.09757arXiv:2501.09757 [cs].","DOI":"10.1109\/CVPR52734.2025.02568"},{"key":"e_1_2_9_18_1","unstructured":"Hendrycks D. Carlini N. Schulman J. &Steinhardt J.(2022).Unsolved problems in ML safety.http:\/\/arxiv.org\/abs\/2109.13916arXiv:2109.13916 [cs]."},{"key":"e_1_2_9_19_1","doi-asserted-by":"crossref","unstructured":"Huang K. Mo F. Zhang X. Li H. Li Y. Zhang Y. Yi W. Mao Y. Liu J. Xu Y. Xu J. Nie J.\u2010Y. &Liu Y.(2025).A survey on large language models with multilingualism: Recent advances and new frontiers.http:\/\/arxiv.org\/abs\/2405.10936arXiv:2405.10936 [cs].","DOI":"10.1007\/s10462-026-11534-5"},{"key":"e_1_2_9_20_1","unstructured":"Jiang G. Han W. Xu M. Zhang C. Zhu S.\u2010C. &Zhu Y.(2023).Evaluating and inducing personality in pre\u2010trained language models. InThe 37th Conference on Neural Information Processing Systems(NeurIPS 2023). NeurIPS."},{"key":"e_1_2_9_21_1","doi-asserted-by":"crossref","unstructured":"Jiang H. Zhang X. Cao X. Breazeal C. Roy D. &Kabbara J.(2024).PersonaLLM: Investigating the ability of large language models to express personality traits. arXiv:2305.02547 [cs].","DOI":"10.18653\/v1\/2024.findings-naacl.229"},{"key":"e_1_2_9_22_1","unstructured":"Kostina A. Dikaiakos M. D. Stefanidis D. &Pallis G.(2025).Large language models for text classification: Case study and comprehensive review.http:\/\/arxiv.org\/abs\/2501.08457arXiv:2501.08457 [cs]."},{"key":"e_1_2_9_23_1","doi-asserted-by":"crossref","unstructured":"Lai V. D. Ngo N. T. Veyseh A. P. B. Man H. Dernoncourt F. Bui T. &Nguyen T. H.(2023).ChatGPT beyond English: Towards a comprehensive evaluation of large language models in multilingual learning.ArXiv abs\/2304.05613.","DOI":"10.18653\/v1\/2023.findings-emnlp.878"},{"key":"e_1_2_9_24_1","unstructured":"Li X. Li Y. Qiu L. Joty S. &Bing L.(2024).Evaluating psychological safety of large language models.http:\/\/arxiv.org\/abs\/2212.10529arXiv:2212.10529 [cs]."},{"key":"e_1_2_9_25_1","unstructured":"Liu J. Liu C. Zhou P. Lv R. Zhou K. &Zhang Y.(2023).Is ChatGPT a good recommender? A preliminary study.http:\/\/arxiv.org\/abs\/2304.10149arXiv:2304.10149 [cs]."},{"key":"e_1_2_9_26_1","doi-asserted-by":"crossref","unstructured":"Ma S. Xie W. Wang Z. Sun X. Chen K. Wang E. Liu W. &Tong H.(2025).Do large language models truly grasp mathematics? An empirical exploration from cognitive psychology. InProceedings of the Annual Meeting of the Cognitive Science Society (CogSci) San Francisco CA USA.","DOI":"10.21203\/rs.3.rs-5273334\/v1"},{"key":"e_1_2_9_27_1","unstructured":"Naous T. &Xu W.(2025).On the origin of cultural biases in language models: From pre\u2010training data to linguistic phenomena.https:\/\/arxiv.org\/abs\/2501.04662arXiv:2501.04662 [cs]."},{"key":"e_1_2_9_28_1","unstructured":"OpenAI. (2023).GPT\u20104 Technical Report.http:\/\/arxiv.org\/abs\/2303.08774arXiv:2303.08774 [cs]."},{"key":"e_1_2_9_29_1","unstructured":"Pan K. &Zeng Y.(2023).Do LLMs possess a personality? Making the MBTI test an amazing evaluation for large language models.ArXiv abs\/2307.16180."},{"key":"e_1_2_9_30_1","unstructured":"Shen T. Jin R. Huang Y. Liu C. Dong W. Guo Z. Wu X. Liu Y. &Xiong D.(2023).Large language model alignment: A survey.http:\/\/arxiv.org\/abs\/2309.15025arXiv:2309.15025 [cs]."},{"key":"e_1_2_9_31_1","unstructured":"Tang T. Luo W. Huang H. Zhang D. Wang X. Zhao X. Wei F. &Wen J.\u2010R.(2024).Language\u2010specific neurons: The key to multilingual capabilities in large language models.http:\/\/arxiv.org\/abs\/2402.16438arXiv:2402.16438 [cs]."},{"key":"e_1_2_9_32_1","doi-asserted-by":"publisher","DOI":"10.1093\/pnasnexus\/pgae346"},{"key":"e_1_2_9_33_1","doi-asserted-by":"crossref","unstructured":"tse Huang J. Jiao W. Lam M. H. Li E. J. Wang W. &Lyu M. R.(2024).On the reliability of psychological scales on large language models.","DOI":"10.18653\/v1\/2024.emnlp-main.354"},{"key":"e_1_2_9_34_1","unstructured":"tse Huang J. Wang W. Li E. J. Lam M. H. Ren S. Yuan Y. Jiao W. Tu Z. &Lyu M. R.(2024).Who is ChatGPT? Benchmarking LLMs' psychological portrayal using PsychoBench.https:\/\/arxiv.org\/abs\/2310.01386"},{"key":"e_1_2_9_35_1","unstructured":"W3techs. (2025).Usage statistics of content languages for websites.https:\/\/w3techs.com\/technologies\/overview\/content_languageAccessed: 2025\u201002\u201001."},{"key":"e_1_2_9_36_1","doi-asserted-by":"crossref","unstructured":"Wang W. Tu Z. Chen C. Yuan Y. Huang J.\u2010T. Jiao W. &Lyu M.(2024).All languages matter: On the multilingual safety of LLMs.https:\/\/aclanthology.org\/2024.findings\u2010acl.349","DOI":"10.18653\/v1\/2024.findings-acl.349"},{"key":"e_1_2_9_37_1","unstructured":"Wang Y. Cai Y. Chen M. Liang Y. &Hooi B.(2023).Primacy effect of ChatGPT.http:\/\/arxiv.org\/abs\/2310.13206arXiv:2310.13206 [cs]."},{"key":"e_1_2_9_38_1","unstructured":"Wang Z. Bi B. Pentyala S. K. Ramnath K. Chaudhuri S. Mehrotra S. Zixu Zhu Mao X.\u2010B. Asur S. &Na C.(2024).A comprehensive survey of llm alignment techniques: RLHF RLAIF PPO DPO and more.http:\/\/arxiv.org\/abs\/2407.16216arXiv:2407.16216 [cs]."},{"key":"e_1_2_9_39_1","unstructured":"Wang Z. Xie W. Ma S. Sun X. Wen Z. Chen K. Wang E. &Wang B.(2025).Self\u2010persuasion: A novel cognitive approach to effective LLM jailbreaking. InProceedings of the Annual Meeting of the Cognitive Science Society (CogSci) San Francisco CA USA."},{"key":"e_1_2_9_40_1","doi-asserted-by":"crossref","unstructured":"Yang Q. Wang Z. M. Chen H. Wang S. Pu Y. Gao X. Huang W. Song S. &Huang G.(2024).PsychoGAT: A novel psychological measurement paradigm through interactive fiction games with LLM agents. InAnnual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics.","DOI":"10.18653\/v1\/2024.acl-long.779"},{"key":"e_1_2_9_41_1","unstructured":"Zhang Y. Bai R. H. Zhang R. Gu J. Zhai S. Susskind J. M. &Jaitly N.(2024).How far are we from intelligent visual deductive reasoning?ArXiv abs\/2403.04732."},{"key":"e_1_2_9_42_1","unstructured":"Zou A. Phan L. Chen S. Campbell J. Guo P. Ren R. Pan A. Yin X. Mazeika M. Dombrowski A.\u2010K. Goel S. Li N. Byun M. J. Wang Z. Mallen A. Basart S. Koyejo S. Song D. Fredrikson M. Kolter J. Z. &Hendrycks D.(2023).Representation engineering: A top\u2010down approach to AI transparency.http:\/\/arxiv.org\/abs\/2310.01405arXiv:2310.01405 [cs]."}],"container-title":["Topics in Cognitive Science"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/tops.70041","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/full-xml\/10.1111\/tops.70041","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/pdf\/10.1111\/tops.70041","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,22]],"date-time":"2026-04-22T17:45:45Z","timestamp":1776879945000},"score":1,"resource":{"primary":{"URL":"https:\/\/onlinelibrary.wiley.com\/doi\/10.1111\/tops.70041"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,3,9]]},"references-count":41,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2026,4]]}},"alternative-id":["10.1111\/tops.70041"],"URL":"https:\/\/doi.org\/10.1111\/tops.70041","archive":["Portico"],"relation":{},"ISSN":["1756-8757","1756-8765"],"issn-type":[{"value":"1756-8757","type":"print"},{"value":"1756-8765","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,3,9]]},"assertion":[{"value":"2025-09-24","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-21","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-09","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e70041"}}