{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T18:05:11Z","timestamp":1790186711955,"version":"4.1.0"},"reference-count":60,"publisher":"Public Library of Science (PLoS)","issue":"9","license":[{"start":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T00:00:00Z","timestamp":1790121600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.plosmentalhealth.org"],"crossmark-restriction":false},"short-container-title":["PLOS Ment Health"],"abstract":"<jats:p>Generative artificial intelligence (GenAI), including large language models and large multimodal models, is increasingly used to support psychological assessment and digital mental health measurement. This review proposes a lifecycle framework for evaluating these uses without weakening psychometric standards. The framework covers construct definition; item, prompt, indicator, or signal generation; evaluation of content, response processes, usability, and data quality; piloting and calibration; validation; fairness and measurement invariance; scoring and interpretation; documentation; and post-deployment monitoring. We argue that digital traces from smartphones, chatbots, ecological momentary assessment, wearables, social media, clinical notes, and multimodal systems should not be treated as psychometric measures by default. They should remain raw data, candidate indicators, or algorithmic scores until their construct interpretation and intended use are theoretically specified, technically verified, empirically calibrated, and validated with human data. GenAI may help generate candidate items, refine wording, classify open-text responses, extract structured information, support multimodal integration, and assist scoring under explicit rules. These uses may improve efficiency and scale, but they do not establish validity, objectivity, fairness, or clinical meaning. We distinguish two complementary facets: GenAI for psychometrics, in which models support measurement development and scoring, and psychometrics for GenAI, in which psychometric methods evaluate model behavior when LLMs are part of measurement workflows. Key risks include construct drift, face validity without structural validity, compressed variability in synthetic respondents, algorithmic bias, lack of invariance, prompt sensitivity, model drift, automation bias, data-security and privacy failures, and loss of subjectivity and disagreement. Responsible use requires human oversight, transparent reporting, validation with human data, fairness evaluation, secure data governance, and continuous monitoring. GenAI should augment, not replace, psychometric science.<\/jats:p>","DOI":"10.1371\/journal.pmen.0000702","type":"journal-article","created":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T17:27:47Z","timestamp":1790184467000},"page":"e0000702","update-policy":"https:\/\/doi.org\/10.1371\/journal.pmen.corrections_policy","source":"Crossref","is-referenced-by-count":0,"title":["Psychometric applications of generative artificial intelligence: Lifecycle, risks, and research agenda"],"prefix":"10.1371","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2222-4764","authenticated-orcid":true,"given":"David","family":"Villarreal-Zegarra","sequence":"first","affiliation":[{"name":"Universidad Continental","place":["Lima, Peru"],"department":["Escuela de Posgrado"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yscenia","family":"Paredes-Gonzales","sequence":"additional","affiliation":[{"name":"Digital Health S.A.","place":["Lima, Peru"],"department":["Digital Health Research Center"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jackeline","family":"Garc\u00eda-Serna","sequence":"additional","affiliation":[{"name":"Digital Health S.A.","place":["Lima, Peru"],"department":["Digital Health Research Center"]},{"name":"Instituto Peruano de Orientaci\u00f3n Psicol\u00f3gica","place":["Lima, Peru"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2026,9,23]]},"reference":[{"key":"pmen.0000702.ref001","doi-asserted-by":"crossref","DOI":"10.2196\/59505","article-title":"Multimodal Large Language Models in Health Care: Applications, Challenges, and Future Outlook","volume":"26","author":"R AlSaad","year":"2024","journal-title":"J Med Internet Res"},{"issue":"1","key":"pmen.0000702.ref002","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1001\/jamapsychiatry.2025.3258","article-title":"Generative psychometrics-an emerging frontier in mental health measurement","volume":"83","author":"IR Galatzer-Levy","year":"2026","journal-title":"JAMA Psychiatry"},{"key":"pmen.0000702.ref003","doi-asserted-by":"crossref","first-page":"115667","DOI":"10.1016\/j.psychres.2023.115667","article-title":"Beyond rating scales: With targeted evaluation, large language models are poised for psychological assessment","volume":"333","author":"ONE Kjell","year":"2024","journal-title":"Psychiatry Res"},{"issue":"4","key":"pmen.0000702.ref004","doi-asserted-by":"crossref","first-page":"319","DOI":"10.1001\/jama.2024.21700","article-title":"Testing and evaluation of health care applications of large language models: A systematic review","volume":"333","author":"S Bedi","year":"2025","journal-title":"JAMA"},{"key":"pmen.0000702.ref005","volume-title":"Standards for educational and psychological testing","author":"American Educational Research Association, American Psychological Association, National Council on Measurement in Education","year":"2014"},{"issue":"4","key":"pmen.0000702.ref006","doi-asserted-by":"crossref","first-page":"539","DOI":"10.1007\/s11136-010-9606-8","article-title":"The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: an international Delphi study","volume":"19","author":"LB Mokkink","year":"2010","journal-title":"Qual Life Res"},{"issue":"5","key":"pmen.0000702.ref007","doi-asserted-by":"crossref","first-page":"1147","DOI":"10.1007\/s11136-018-1798-3","article-title":"COSMIN guideline for systematic reviews of patient-reported outcome measures","volume":"27","author":"CAC Prinsen","year":"2018","journal-title":"Qual Life Res"},{"issue":"10","key":"pmen.0000702.ref008","doi-asserted-by":"crossref","DOI":"10.2196\/37968","article-title":"A digital mental health intervention (inuka) for common mental health disorders in zimbabwean adults in response to the COVID-19 pandemic: Feasibility and acceptability pilot study","volume":"9","author":"J Dambi","year":"2022","journal-title":"JMIR Ment Health"},{"key":"pmen.0000702.ref009","doi-asserted-by":"crossref","first-page":"1495999","DOI":"10.3389\/fdgth.2024.1495999","article-title":"A systematic survey on the application of federated learning in mental state detection and human activity recognition","volume":"6","author":"A Grataloup","year":"2024","journal-title":"Front Digit Health"},{"issue":"2","key":"pmen.0000702.ref010","doi-asserted-by":"crossref","first-page":"1453","DOI":"10.3390\/ijerph20021453","article-title":"Digital mental health interventions for young people in rural South Africa: Prospects and challenges for implementation","volume":"20","author":"T Mindu","year":"2023","journal-title":"Int J Environ Res Public Health"},{"issue":"1","key":"pmen.0000702.ref011","doi-asserted-by":"crossref","DOI":"10.1136\/bmjment-2023-300670","article-title":"Digital mental health: Challenges and next steps","volume":"26","author":"KA Smith","year":"2023","journal-title":"BMJ Ment Health"},{"key":"pmen.0000702.ref012","doi-asserted-by":"crossref","first-page":"645260","DOI":"10.3389\/fpubh.2021.645260","article-title":"Digital data sources and their impact on people\u2019s health: A systematic review of systematic reviews","volume":"9","author":"L Li","year":"2021","journal-title":"Front Public Health"},{"key":"pmen.0000702.ref013","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1038\/s41746-020-0260-4","article-title":"Verification, analytical validation, and clinical validation (V3): the foundation of determining fit-for-purpose for Biometric Monitoring Technologies (BioMeTs)","volume":"3","author":"JC Goldsack","year":"2020","journal-title":"NPJ Digit Med"},{"key":"pmen.0000702.ref014","doi-asserted-by":"crossref","first-page":"1691","DOI":"10.1038\/npp.2016.7","article-title":"Harnessing smartphone-based digital phenotyping to enhance behavioral and mental health","volume":"41","author":"J-P Onnela","year":"2016","journal-title":"Neuropsychopharmacol Off Publ Am Coll Neuropsychopharmacol"},{"key":"pmen.0000702.ref015","doi-asserted-by":"crossref","first-page":"715","DOI":"10.3102\/10769986241248771","article-title":"Artificial intelligence and educational measurement: Opportunities and threats","volume":"49","author":"AD Ho","year":"2024","journal-title":"J Educ Behav Stat"},{"key":"pmen.0000702.ref016","doi-asserted-by":"crossref","first-page":"267","DOI":"10.1111\/jedm.12331","article-title":"Validity arguments meet artificial intelligence in innovative educational assessment","volume":"59","author":"DW Dorsey","year":"2022","journal-title":"J Educ Meas"},{"key":"pmen.0000702.ref017","doi-asserted-by":"crossref","first-page":"169","DOI":"10.1027\/1015-5759\/a000904","article-title":"Ten guidelines for scoring psychological assessments using artificial intelligence","volume":"41","author":"D Dumas","year":"2025","journal-title":"Eur J Psychol Assess"},{"key":"pmen.0000702.ref018","article-title":"Responsible AI for test equity and quality: The Duolingo english test as a case study","author":"J Burstein","year":"2024","journal-title":"arXiv"},{"key":"pmen.0000702.ref019","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1177\/14707853251384769","article-title":"Using generative AI to enhance psychometric scale development in market research","volume":"68","author":"D Keane","year":"2026","journal-title":"Int J Mark Res"},{"key":"pmen.0000702.ref020","doi-asserted-by":"crossref","first-page":"1426209","DOI":"10.3389\/fpsyg.2024.1426209","article-title":"ChatGPT\u2019s quality: Reliability and validity of concept inventory items","volume":"15","author":"S K\u00fcchemann","year":"2024","journal-title":"Front Psychol"},{"key":"pmen.0000702.ref021","author":"P Wang","year":"2024","journal-title":"Not yet: Large language models cannot replace human respondents for psychometric research. OSF Preprints"},{"key":"pmen.0000702.ref022","doi-asserted-by":"crossref","DOI":"10.2196\/75347","article-title":"Psychometric evaluation of large language model embeddings for personality trait prediction","volume":"27","author":"J Maharjan","year":"2025","journal-title":"J Med Internet Res"},{"issue":"2","key":"pmen.0000702.ref023","doi-asserted-by":"crossref","first-page":"749","DOI":"10.1007\/s11336-021-09823-9","article-title":"Transformer-based deep neural language modeling for construct-specific automatic item generation","volume":"87","author":"BE Hommel","year":"2022","journal-title":"Psychometrika"},{"issue":"3","key":"pmen.0000702.ref024","doi-asserted-by":"crossref","first-page":"494","DOI":"10.1037\/met0000540","article-title":"Let the algorithm speak: How to use neural networks for automatic item generation in psychological scale development","volume":"29","author":"FM G\u00f6tz","year":"2024","journal-title":"Psychol Methods"},{"key":"pmen.0000702.ref025","doi-asserted-by":"crossref","first-page":"1011","DOI":"10.1111\/peps.12543","article-title":"The AI-IP: Minimizing the guesswork of personality scale item development through artificial intelligence","volume":"76","author":"I Hernandez","year":"2023","journal-title":"Pers Psychol"},{"issue":"4","key":"pmen.0000702.ref026","doi-asserted-by":"crossref","DOI":"10.1177\/25152459251377093","article-title":"Language models accurately infer correlations between psychological items and scales from text alone","volume":"8","author":"BE Hommel","year":"2025","journal-title":"Advances in Methods and Practices in Psychological Science"},{"key":"pmen.0000702.ref027","doi-asserted-by":"crossref","first-page":"163","DOI":"10.1007\/s10869-022-09864-6","article-title":"A paradigm shift from \u201chuman writing\u201d to \u201cmachine generation\u201d in personality test development: An application of state-of-the-art natural language processing","volume":"38","author":"P Lee","year":"2023","journal-title":"J Bus Psychol"},{"key":"pmen.0000702.ref028","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1027\/2698-1866\/a000095","article-title":"Comparing psychometric properties of expert-developed and ai-generated personality scales","volume":"6","author":"L Oeljeklaus","year":"2025","journal-title":"Psychological Test Adaptation and Development"},{"issue":"4","key":"pmen.0000702.ref029","doi-asserted-by":"crossref","first-page":"847","DOI":"10.1007\/s11336-018-9608-y","article-title":"Automated item generation with recurrent neural networks","volume":"83","author":"M von Davier","year":"2018","journal-title":"Psychometrika"},{"key":"pmen.0000702.ref030","doi-asserted-by":"crossref","first-page":"101742","DOI":"10.1016\/j.jacadv.2025.101742","article-title":"Disparities in artificial intelligence-based tools among diverse minority populations: biases, barriers, and solutions","volume":"4","author":"M Ahluwalia","year":"2025","journal-title":"JACC Adv"},{"key":"pmen.0000702.ref031","doi-asserted-by":"crossref","first-page":"1097","DOI":"10.1162\/coli_a_00524","article-title":"Bias and fairness in large language models: A survey","volume":"50","author":"IO Gallegos","year":"2024","journal-title":"Comput Linguist"},{"key":"pmen.0000702.ref032","doi-asserted-by":"crossref","first-page":"1824","DOI":"10.3390\/electronics15091824","article-title":"Bias in large language models: Origin, evaluation, and mitigation","volume":"15","author":"Y Guo","year":"2026","journal-title":"Electronics"},{"key":"pmen.0000702.ref033","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.dr.2016.06.004","article-title":"Measurement Invariance Conventions and Reporting: The State of the Art and Future Directions for Psychological Research","volume":"41","author":"DL Putnick","year":"2016","journal-title":"Dev Rev"},{"issue":"1","key":"pmen.0000702.ref034","article-title":"A Conceptual Framework for Investigating and Mitigating Machine-Learning Measurement Bias (MLMB) in Psychological Assessment","volume":"5","author":"L Tay","year":"2022","journal-title":"Advances in Methods and Practices in Psychological Science"},{"issue":"1","key":"pmen.0000702.ref035","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1038\/s41591-024-03425-5","article-title":"The TRIPOD-LLM reporting guideline for studies using large language models","volume":"31","author":"J Gallifant","year":"2025","journal-title":"Nat Med"},{"key":"pmen.0000702.ref036","doi-asserted-by":"crossref","DOI":"10.1136\/bmj-2023-078378","article-title":"TRIPOD+AI statement: Updated guidance for reporting clinical prediction models that use regression or machine learning methods","volume":"385","author":"GS Collins","year":"2024","journal-title":"BMJ"},{"key":"pmen.0000702.ref037","doi-asserted-by":"crossref","first-page":"6","DOI":"10.1038\/s44401-024-00011-2","article-title":"Current and future state of evaluation of large language models for medical summarization tasks","volume":"2","author":"E Croxford","year":"2025","journal-title":"Npj Health Syst"},{"issue":"2","key":"pmen.0000702.ref038","article-title":"The use of a large language model to create plain language summaries of evidence reviews in healthcare: A feasibility study","volume":"2","author":"C Ovelman","year":"2024","journal-title":"Cochrane Evid Synth Methods"},{"key":"pmen.0000702.ref039","author":"Food and Drug Administration","year":"2023"},{"issue":"12","key":"pmen.0000702.ref040","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pdig.0000685","article-title":"Evaluating Large Language Models in extracting cognitive exam dates and scores","volume":"3","author":"H Zhang","year":"2024","journal-title":"PLOS Digit Health"},{"key":"pmen.0000702.ref041","article-title":"Leveraging LLM respondents for item evaluation: A psychometric analysis","author":"Y Liu","year":"2025","journal-title":"Br J Educ Technol"},{"key":"pmen.0000702.ref042","article-title":"Generation of synthetic data in health surveys using large language models","author":"D Villarreal-Zegarra","year":"2026","journal-title":"medRxiv"},{"issue":"5","key":"pmen.0000702.ref043","doi-asserted-by":"crossref","first-page":"944","DOI":"10.1038\/s41562-024-02089-y","article-title":"Semantic embeddings reveal and address taxonomic incommensurability in psychological measurement","volume":"9","author":"DU Wulff","year":"2025","journal-title":"Nat Hum Behav"},{"key":"pmen.0000702.ref044","article-title":"The ultimate tutorial for AI-driven scale development in generative psychometrics: releasing AIGENIE from its bottle","author":"L Russell-Lasalandra","year":"2026","journal-title":"arXiv.org"},{"key":"pmen.0000702.ref045","doi-asserted-by":"crossref","first-page":"5683","DOI":"10.3390\/app15105683","article-title":"Large language model-powered automated assessment: A systematic review","volume":"15","author":"E Emirtekin","year":"2025","journal-title":"Appl Sci"},{"key":"pmen.0000702.ref046","first-page":"8","article-title":"Is GPT-4 a reliable rater? Evaluating consistency in GPT-4\u2019s text ratings","author":"V Hackl","year":"2023","journal-title":"Front Educ"},{"issue":"5","key":"pmen.0000702.ref047","doi-asserted-by":"crossref","first-page":"924","DOI":"10.1038\/s41591-022-01772-9","article-title":"Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI","volume":"28","author":"B Vasey","year":"2022","journal-title":"Nat Med"},{"issue":"9","key":"pmen.0000702.ref048","doi-asserted-by":"crossref","first-page":"1364","DOI":"10.1038\/s41591-020-1034-x","article-title":"Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension","volume":"26","author":"X Liu","year":"2020","journal-title":"Nature Medicine"},{"key":"pmen.0000702.ref049","doi-asserted-by":"crossref","first-page":"101060","DOI":"10.1016\/j.chbr.2026.101060","article-title":"Exploring the future of psychometrics from a Large Language Model perspective: A case study analysis","volume":"22","author":"W Mieleszczenko-Kowszewicz","year":"2026","journal-title":"Comput Hum Behav Rep"},{"key":"pmen.0000702.ref050","doi-asserted-by":"crossref","DOI":"10.2196\/55988","article-title":"Assessing the Alignment of Large Language Models With Human Values for Mental Health Integration: Cross-Sectional Study Using Schwartz\u2019s Theory of Basic Values","volume":"11","author":"D Hadar-Shoval","year":"2024","journal-title":"JMIR Ment Health"},{"key":"pmen.0000702.ref051","article-title":"Psychometric Alignment: Capturing human knowledge distributions via language models","author":"J He-Yueya","year":"2024","journal-title":"arXiv.org"},{"issue":"12","key":"pmen.0000702.ref052","doi-asserted-by":"crossref","first-page":"1954","DOI":"10.1038\/s42256-025-01115-6","article-title":"A psychometric framework for evaluating and shaping personality traits in large language models","volume":"7","author":"G Serapio-Garc\u00eda","year":"2025","journal-title":"Nat Mach Intell"},{"key":"pmen.0000702.ref053","doi-asserted-by":"crossref","first-page":"108769","DOI":"10.1016\/j.chb.2025.108769","article-title":"Delving into the psychology of Machines: Exploring the structure of self-regulated learning via LLM-generated survey responses","volume":"173","author":"LVDE Vogelsmeier","year":"2025","journal-title":"Comput Hum Behav"},{"key":"pmen.0000702.ref054","article-title":"An LLM-native psychometric instrument reveals a self-report--behavior gap across 25 models","author":"JM Contreras","year":"2026","journal-title":"arXiv"},{"issue":"25","key":"pmen.0000702.ref055","doi-asserted-by":"crossref","first-page":"26400","DOI":"10.1609\/aaai.v39i25.34839","article-title":"Measuring human and AI values based on generative psychometrics with large language models","volume":"39","author":"H Ye","year":"2025","journal-title":"AAAI"},{"key":"pmen.0000702.ref056","doi-asserted-by":"crossref","unstructured":"Basile V. It\u2019s the end of the gold standard as we know it: Leveraging non-aggregated data for better evaluation and explanation of subjective tasks. AIxIA 2020 \u2013 Advances in Artificial Intelligence: XIXth International Conference of the Italian Association for Artificial Intelligence, Virtual Event, November 25\u201327, 2020, Revised Selected Papers. Berlin, Heidelberg: Springer-Verlag; 2020. pp. 441\u2013453. https:\/\/doi.org\/10.1007\/978-3-030-77091-4_26","DOI":"10.1007\/978-3-030-77091-4_26"},{"key":"pmen.0000702.ref057","unstructured":"Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA: Curran Associates Inc.; 2017. pp. 6000\u201310. https:\/\/dl.acm.org\/doi\/10.5555\/3295222.3295349"},{"key":"pmen.0000702.ref058","article-title":"Calibrate before use: Improving few-shot performance of language models","author":"TZ Zhao","year":"2021","journal-title":"arXiv"},{"key":"pmen.0000702.ref059","article-title":"On the opportunities and risks of foundation models","author":"R Bommasani","year":"2022","journal-title":"arXiv"},{"issue":"5","key":"pmen.0000702.ref060","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1006\/ijhc.1999.0252","article-title":"Does automation bias decision-making?","volume":"51","author":"LJ Skitka","year":"1999","journal-title":"International Journal of Human-Computer Studies"}],"container-title":["PLOS Mental Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pmen.0000702","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T17:27:54Z","timestamp":1790184474000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pmen.0000702"}},"subtitle":[],"editor":[{"given":"Karli","family":"Montague-Cardoso","sequence":"first","affiliation":[{"name":"PLOS: Public Library of Science","place":["UNITED KINGDOM OF GREAT BRITAIN AND NORTHERN IRELAND"]}],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2026,9,23]]},"references-count":60,"journal-issue":{"issue":"9","published-online":{"date-parts":[[2026,9,23]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pmen.0000702","relation":{},"ISSN":["2837-8156"],"issn-type":[{"value":"2837-8156","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,9,23]]}}}