{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,10]],"date-time":"2026-11-10T17:06:12Z","timestamp":1794330372370,"version":"4.9.0"},"reference-count":37,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,10,24]],"date-time":"2025-10-24T00:00:00Z","timestamp":1761264000000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,10,24]],"date-time":"2025-10-24T00:00:00Z","timestamp":1761264000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"SNFS","award":["200557"],"award-info":[{"award-number":["200557"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Sci Rep"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Psychological questionnaires are typically designed to measure latent constructs by asking respondents a series of semantically related questions. But what if these semantic relationships, rather than reflecting only the underlying construct, also impose their own structure on the data we collect? In other words, to what extent is what we \u201cmeasure\u201d in questionnaires shaped a priori by item semantics rather than revealed solely a posteriori through empirical correlations? To examine this epistemological question, we propose LLMs Psychometrics, a novel paradigm that harness LLMs to investigate how the semantic structure of questionnaire items influences psychometric outcomes. We hypothesize that the correlations among items partly mirror their linguistic similarity, such that LLMs can predict these correlations-even in the absence of empirical data. To test this, we compared actual correlation matrices from established instruments\u2014the Big 5 Personality (Big 5) and Depression Anxiety Stress Scale (DASS-42)\u2014with the semantic similarity structures computed by LLMs. Among the top 3 semantically similar items, the empirically most correlated item was found in 95% of DASS cases and 82% of Big 5 cases. Building on this, we developed PsychoLLM, a neural proof-of-concept architecture, which uses item semantics to predict responses to new items\u2013demonstrated with the Generalized Anxiety Disorder-7 (GAD-7) and Patient Health Questionnaire-9 (PHQ-9). PsychoLLM achieved \u00a070% accuracy when predicting one scale\u2019s responses from the other, enabling new analyses based on semantic relationships. This work underscores an important epistemological implication for psychometrics: item semantics may influence measurement outcomes to varying degrees, more extensively than previously assumed. By leveraging LLMs to expose this a priori semantic structure, researchers can refine questionnaire design, assess data quality, and expand interpretive possibilities, ultimately inviting a reexamination of \u201c\n                    <jats:italic>what<\/jats:italic>\n                    \u201d and \u201c\n                    <jats:italic>how<\/jats:italic>\n                    \u201d we truly measure in psychology.\n                  <\/jats:p>","DOI":"10.1038\/s41598-025-21289-8","type":"journal-article","created":{"date-parts":[[2025,10,24]],"date-time":"2025-10-24T09:28:35Z","timestamp":1761298115000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Rethinking psychometrics through LLMs: how item semantics shape measurement and prediction in psychological questionnaires"],"prefix":"10.1038","volume":"15","author":[{"given":"Federico","family":"Ravenda","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antonio","family":"Preti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michele","family":"Poletti","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Antonietta","family":"Mira","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andrea","family":"Raballo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2025,10,24]]},"reference":[{"issue":"2684","key":"21289_CR1","doi-asserted-by":"publisher","first-page":"677","DOI":"10.1126\/science.103.2684.677","volume":"103","author":"SS Stevens","year":"1946","unstructured":"Stevens, S. S. On the theory of scales of measurement. Science 103(2684), 677\u2013680 (1946).","journal-title":"Science"},{"issue":"2","key":"21289_CR2","doi-asserted-by":"publisher","first-page":"93","DOI":"10.1037\/0003-066X.54.2.93","volume":"54","author":"N Schwarz","year":"1999","unstructured":"Schwarz, N. Self-reports: How the questions shape the answers. Am. Psychol. 54(2), 93 (1999).","journal-title":"Am. Psychol."},{"issue":"2007","key":"21289_CR3","first-page":"224","volume":"1","author":"DL Paulhus","year":"2007","unstructured":"Paulhus, D. L. et al. The self-report method. Handb. Res. Methods Pers. Psychol. 1(2007), 224\u2013239 (2007).","journal-title":"Handb. Res. Methods Pers. Psychol."},{"issue":"5","key":"21289_CR4","doi-asserted-by":"publisher","first-page":"879","DOI":"10.1037\/0021-9010.88.5.879","volume":"88","author":"PM Podsakoff","year":"2003","unstructured":"Podsakoff, P. M., MacKenzie, S. B., Lee, J.-Y. & Podsakoff, N. P. Common method biases in behavioral research: A critical review of the literature and recommended remedies. J. Appl. Psychol. 88(5), 879 (2003).","journal-title":"J. Appl. Psychol."},{"issue":"5","key":"21289_CR5","doi-asserted-by":"publisher","first-page":"859","DOI":"10.1037\/0033-2909.133.5.859","volume":"133","author":"R Tourangeau","year":"2007","unstructured":"Tourangeau, R. & Yan, T. Sensitive questions in surveys. Psychol. Bull. 133(5), 859 (2007).","journal-title":"Psychol. Bull."},{"issue":"1","key":"21289_CR6","doi-asserted-by":"publisher","first-page":"3","DOI":"10.1037\/0033-2909.126.1.3","volume":"126","author":"BW Roberts","year":"2000","unstructured":"Roberts, B. W. & DelVecchio, W. F. The rank-order consistency of personality traits from childhood to old age: A quantitative review of longitudinal studies. Psychol. Bull. 126(1), 3 (2000).","journal-title":"Psychol. Bull."},{"key":"21289_CR7","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1146\/annurev.psych.55.042902.130709","volume":"55","author":"W Mischel","year":"2004","unstructured":"Mischel, W. Toward an integrative science of the person. Annu. Rev. Psychol. 55, 1\u201322 (2004).","journal-title":"Annu. Rev. Psychol."},{"issue":"8","key":"21289_CR8","doi-asserted-by":"publisher","first-page":"1264","DOI":"10.1037\/xge0000383","volume":"147","author":"Y Bar-Anan","year":"2018","unstructured":"Bar-Anan, Y. & Vianello, M. A multi-method multi-trait test of the dual-attitude perspective. J. Exp. Psychol. Gen. 147(8), 1264 (2018).","journal-title":"J. Exp. Psychol. Gen."},{"issue":"4","key":"21289_CR9","doi-asserted-by":"publisher","first-page":"557","DOI":"10.1037\/pas0000648","volume":"31","author":"LJ Simms","year":"2019","unstructured":"Simms, L. J., Zelazny, K., Williams, T. F. & Bernstein, L. Does the number of response options matter? Psychometric perspectives using personality questionnaire data. Psychol. Assess. 31(4), 557 (2019).","journal-title":"Psychol. Assess."},{"issue":"6","key":"21289_CR10","doi-asserted-by":"publisher","first-page":"1268","DOI":"10.1080\/02699931.2016.1204989","volume":"31","author":"R Pekrun","year":"2017","unstructured":"Pekrun, R., Vogl, E., Muis, K. R. & Sinatra, G. M. Measuring emotions during epistemic activities: The epistemically-related emotion scales. Cogn. Emot. 31(6), 1268\u20131276 (2017).","journal-title":"Cogn. Emot."},{"key":"21289_CR11","doi-asserted-by":"crossref","unstructured":"Furr, M. Scale construction and psychometrics for social and personality psychology. Scale Construction and Psychometrics for Social and Personality Psychology 1\u2013160 (2011).","DOI":"10.4135\/9781446287866"},{"issue":"2","key":"21289_CR12","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1037\/1082-989X.5.2.155","volume":"5","author":"JR Edwards","year":"2000","unstructured":"Edwards, J. R. & Bagozzi, R. P. On the nature and direction of relationships between constructs and measures. Psychol. Methods 5(2), 155 (2000).","journal-title":"Psychol. Methods"},{"key":"21289_CR13","unstructured":"Clark, L. A. & Watson, D. Constructing Validity: Basic Issues in Objective Scale Development (2016)."},{"issue":"5","key":"21289_CR14","doi-asserted-by":"publisher","first-page":"406","DOI":"10.1037\/h0069792","volume":"38","author":"LL Thurstone","year":"1931","unstructured":"Thurstone, L. L. Multiple factor analysis. Psychol. Rev. 38(5), 406 (1931).","journal-title":"Psychol. Rev."},{"key":"21289_CR15","doi-asserted-by":"publisher","unstructured":"Bollen, K. A. Structural Equations with Latent Variables (Wiley, 1989). https:\/\/doi.org\/10.1002\/9781118619179.fmatter.","DOI":"10.1002\/9781118619179.fmatter"},{"key":"21289_CR16","unstructured":"Lord, F. Applications of Item Response Theory to Practical Testing Problems (Hillsdale, 1980)."},{"issue":"3","key":"21289_CR17","doi-asserted-by":"publisher","first-page":"380","DOI":"10.1037\/met0000244","volume":"25","author":"H Rosenbusch","year":"2020","unstructured":"Rosenbusch, H., Wanders, F. & Pit, I. L. The semantic scale network: An online tool to detect semantic overlap of psychological scales and prevent scale redundancies. Psychol. Methods 25(3), 380 (2020).","journal-title":"Psychol. Methods"},{"issue":"9","key":"21289_CR18","doi-asserted-by":"publisher","first-page":"73791","DOI":"10.1371\/journal.pone.0073791","volume":"8","author":"HA Schwartz","year":"2013","unstructured":"Schwartz, H. A. et al. Personality, gender, and age in the language of social media: The open-vocabulary approach. PLoS ONE 8(9), 73791 (2013).","journal-title":"PLoS ONE"},{"key":"21289_CR19","doi-asserted-by":"crossref","unstructured":"Yang, F., Yang, T., Quan, X. & Su, Q. Learning to answer psychological questionnaire for personality detection. In Findings of the Association for Computational Linguistics: EMNLP 2021 1131\u20131142 (2021).","DOI":"10.18653\/v1\/2021.findings-emnlp.98"},{"key":"21289_CR20","doi-asserted-by":"crossref","unstructured":"Vu, H., Abdurahman, S., Bhatia, S. & Ungar, L. Predicting responses to psychological questionnaires from participants\u2019 social media posts and question text embeddings. In Findings of the Association for Computational Linguistics: EMNLP 2020 1512\u20131524 (2020).","DOI":"10.18653\/v1\/2020.findings-emnlp.137"},{"key":"21289_CR21","doi-asserted-by":"publisher","unstructured":"Ravenda, F., Bahrainian, S. A., Raballo, A., Mira, A. & Kando, N. Are {LLM}s effective psychological assessors? Leveraging adaptive {RAG} for interpretable mental health screening through psychometric practice. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Vienna, Austria. 8975\u20138991. https:\/\/doi.org\/10.18653\/v1\/2025.acl-long.440 (2025).","DOI":"10.18653\/v1\/2025.acl-long.440"},{"key":"21289_CR22","doi-asserted-by":"crossref","unstructured":"De Grandi, A., Ravenda, F., Raballo, A. & Crestani, F. The emotional spectrum of llms: Leveraging empathy and emotion-based markers for mental health support. In Proceedings of the 10th Workshop on Computational Linguistics and Clinical Psychology (CLPsych 2025) 26\u201343 (2025).","DOI":"10.18653\/v1\/2025.clpsych-1.3"},{"key":"21289_CR23","unstructured":"Raballo, A., Ravenda, F. & Mira, A. Diagnosing schizophrenia spectrum disorders: Large language models (llms) vs. leading international psychiatrists (lips). Psychiatry and Clinical Neurosciences"},{"key":"21289_CR24","unstructured":"Coombs, C. H. A Theory of Data (1964)."},{"issue":"1","key":"21289_CR25","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1037\/1040-3590.4.1.26","volume":"4","author":"LR Goldberg","year":"1992","unstructured":"Goldberg, L. R. The development of markers for the big-five factor structure. Psychol. Assess. 4(1), 26 (1992).","journal-title":"Psychol. Assess."},{"issue":"1","key":"21289_CR26","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1016\/S0005-7967(96)00068-X","volume":"35","author":"TA Brown","year":"1997","unstructured":"Brown, T. A., Chorpita, B. F., Korotitsch, W. & Barlow, D. H. Psychometric properties of the depression anxiety stress scales (dass) in clinical samples. Behav. Res. Ther. 35(1), 79\u201389 (1997).","journal-title":"Behav. Res. Ther."},{"issue":"2","key":"21289_CR27","doi-asserted-by":"publisher","first-page":"1949","DOI":"10.1002\/mpr.1949","volume":"32","author":"O McBride","year":"2023","unstructured":"McBride, O. et al. An 18-month follow-up of the covid-19 psychology research consortium study panel: Survey design and fieldwork procedures for wave 6. Int. J. Methods Psychiatr. Res. 32(2), 1949 (2023).","journal-title":"Int. J. Methods Psychiatr. Res."},{"key":"21289_CR28","unstructured":"Devlin, J., Chang, M.-W., Lee, K. & Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805 (2018)."},{"key":"21289_CR29","unstructured":"Liu, Y. Roberta: A robustly optimized bert pretraining approach. arXiv:1907.11692 (2019)."},{"key":"21289_CR30","unstructured":"Sanh, V. Distilbert, a distilled version of bert: Smaller, faster, cheaper and lighter. arXiv:1910.01108 (2019)."},{"key":"21289_CR31","doi-asserted-by":"crossref","unstructured":"Reimers, N. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv:1908.10084 (2019).","DOI":"10.18653\/v1\/D19-1410"},{"key":"21289_CR32","unstructured":"Brown, T. B. Language models are few-shot learners. arXiv:2005.14165 (2020)."},{"issue":"140","key":"21289_CR33","first-page":"1","volume":"21","author":"C Raffel","year":"2020","unstructured":"Raffel, C. et al. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res. 21(140), 1\u201367 (2020).","journal-title":"J. Mach. Learn. Res."},{"issue":"11","key":"21289_CR34","first-page":"2579","volume":"9","author":"L Maaten","year":"2008","unstructured":"Maaten, L. & Hinton, G. Visualizing data using t-sne. J. Mach. Learn. Res. 9(11), 2579\u20132605 (2008).","journal-title":"J. Mach. Learn. Res."},{"key":"21289_CR35","doi-asserted-by":"publisher","first-page":"1101","DOI":"10.1001\/jamapsychiatry.2024.2047","volume":"81","author":"TR Spiller","year":"2024","unstructured":"Spiller, T. R. et al. Unveiling the structure in mental disorder presentations. JAMA Psychiatry 81, 1101\u20131107 (2024).","journal-title":"JAMA Psychiatry"},{"key":"21289_CR36","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1023\/A:1010933404324","volume":"45","author":"L Breiman","year":"2001","unstructured":"Breiman, L. Random forests. Mach. Learn. 45, 5\u201332 (2001).","journal-title":"Mach. Learn."},{"key":"21289_CR37","doi-asserted-by":"publisher","first-page":"1189","DOI":"10.1214\/aos\/1013203451","volume":"29","author":"JH Friedman","year":"2001","unstructured":"Friedman, J. H. Greedy function approximation: A gradient boosting machine. Ann. Stat. 29, 1189\u20131232 (2001).","journal-title":"Ann. Stat."}],"container-title":["Scientific Reports"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-21289-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-21289-8","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-21289-8.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,24]],"date-time":"2025-10-24T09:28:41Z","timestamp":1761298121000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41598-025-21289-8"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,10,24]]},"references-count":37,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["21289"],"URL":"https:\/\/doi.org\/10.1038\/s41598-025-21289-8","relation":{},"ISSN":["2045-2322"],"issn-type":[{"value":"2045-2322","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,10,24]]},"assertion":[{"value":"17 March 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 September 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 October 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare no competing interests.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"37313"}}