{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,26]],"date-time":"2026-10-26T03:00:11Z","timestamp":1792983611101,"version":"4.5.0"},"reference-count":46,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2021,6,4]],"date-time":"2021-06-04T00:00:00Z","timestamp":1622764800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Psychol."],"abstract":"<jats:sec><jats:title>Background<\/jats:title><jats:p>Question-based computational language assessments (QCLA) of mental health, based on self-reported and freely generated word responses and analyzed with artificial intelligence, is a potential complement to rating scales for identifying mental health issues. This study aimed to examine to what extent this method captures items related to the primary and secondary symptoms associated with Major Depressive Disorder (MDD) and Generalized Anxiety Disorder (GAD) described in the Diagnostic and Statistical Manual of Mental Disorders (DSM-5). We investigated whether the word responses that participants generated contained information of all, or some, of the criteria that define MDD and GAD using symptom-based rating scales that are commonly used in clinical research and practices.<\/jats:p><\/jats:sec><jats:sec><jats:title>Method<\/jats:title><jats:p>Participants (<jats:italic>N<\/jats:italic> = 411) described their mental health with freely generated words and rating scales relating to depression and worry\/anxiety. Word responses were quantified and analyzed using natural language processing and machine learning.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The QCLA correlated significantly with the individual items connected to the DSM-5 diagnostic criteria of MDD (PHQ-9; Pearson\u2019s <jats:italic>r<\/jats:italic> = 0.30\u20130.60, <jats:italic>p<\/jats:italic> &amp;lt; 0.001) and GAD (GAD-7; Pearson\u2019s <jats:italic>r<\/jats:italic> = 0.41\u20130.52, <jats:italic>p<\/jats:italic> &amp;lt; 0.001; PSWQ-8; Spearman\u2019s <jats:italic>r<\/jats:italic> = 0.52\u20130.63, <jats:italic>p<\/jats:italic> &amp;lt; 0.001) for respective rating scales. Items measuring primary criteria (cognitive and emotional aspects) yielded higher predictability than secondary criteria (behavioral aspects).<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusion<\/jats:title><jats:p>Together these results suggest that QCLA may be able to complement rating scales in measuring mental health in clinical settings. The approach carries the potential to personalize assessments and contributes to the ongoing discussion regarding the diagnostic heterogeneity of depression.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fpsyg.2021.602581","type":"journal-article","created":{"date-parts":[[2021,6,4]],"date-time":"2021-06-04T04:43:18Z","timestamp":1622781798000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":18,"title":["Freely Generated Word Responses Analyzed With Artificial Intelligence Predict Self-Reported Symptoms of Depression, Anxiety, and Worry"],"prefix":"10.3389","volume":"12","author":[{"given":"Katarina","family":"Kjell","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Per","family":"Johnsson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sverker","family":"Sikstr\u00f6m","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2021,6,4]]},"reference":[{"key":"B1","year":"2013","journal-title":"Diagnostic and Statistical Manual of Mental Disorders (DSM-5)."},{"key":"B2","doi-asserted-by":"crossref","first-page":"684","DOI":"10.1037\/pas0000217","article-title":"The importance of assessing clinical phenomena in Mechanical Turk research.","volume":"28","author":"Arditte","year":"2016","journal-title":"Psychol. Assessm."},{"key":"B3","author":"Bradley","year":"1999","journal-title":"Affective Norms for English Words (ANEW): INSTRUCTION MANUAL and Affective Ratings."},{"key":"B4","doi-asserted-by":"publisher","first-page":"90","DOI":"10.1186\/1741-7015-9-90","article-title":"Cross-national epidemiology of DSM-IV major depressive episode.","volume":"9","author":"Bromet","year":"2011","journal-title":"BMC Med."},{"key":"B5","doi-asserted-by":"publisher","first-page":"53","DOI":"10.1146\/annurev-clinpsy-021815-093623","article-title":"Conducting clinical research using crowdsourced convenience samples.","volume":"12","author":"Chandler","year":"2016","journal-title":"Annu. Rev. Clin. Psychol."},{"key":"B6","doi-asserted-by":"crossref","first-page":"692","DOI":"10.1016\/S2215-0366(20)30191-7","article-title":"Identifying outcomes for depression that matter to patients, informal caregivers, and health-care professionals: qualitative content analysis of a large international online survey.","volume":"7","author":"Chevance","year":"2020","journal-title":"Lancet Psychiatry"},{"key":"B7","author":"Cohen","year":"1988","journal-title":"Statistical Power Analysis for the Behavioralbehavioural Sciences."},{"key":"B8","doi-asserted-by":"crossref","first-page":"1036","DOI":"10.1016\/j.janxdis.2005.11.006","article-title":"Assessing worry in older and younger adults: psychometric properties of an abbreviated Penn State Worry Questionnaire (PSWQ-A).","volume":"20","author":"Crittendon","year":"2006","journal-title":"J. Anxiety Disord."},{"key":"B9","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1002\/cpp.696","article-title":"Routine outcome monitoring in the Netherlands: practical experiences with a web- based strategy for the assessment of treatment outcome in clinical practice.","volume":"18","author":"de Beurs","year":"2011","journal-title":"Clin. Psychol. Psychother."},{"key":"B10","article-title":"Predicting depression via social media","author":"De Choudhury","year":"2013","journal-title":"Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media ICWSM-13"},{"key":"B11","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1080\/16506073.2011.582138","article-title":"Psychometric comparison of the generalized anxiety disorder scale-7 and the Penn State Worry Questionnaire for measuring response during treatment of generalised anxiety disorder.","volume":"40","author":"Dear","year":"2011","journal-title":"Cogn. Behav. Therapy"},{"key":"B12","article-title":"Bert: pre-training of deep bidirectional transformers for language understanding.","author":"Devlin","year":"2019","journal-title":"arXiv"},{"key":"B13","doi-asserted-by":"crossref","first-page":"11203","DOI":"10.1073\/pnas.1802331115","article-title":"Facebook language predicts depression in medical records.","volume":"115","author":"Eichstaedt","year":"2018","journal-title":"Proc. Natl. Acad. Sci. U.S.A."},{"key":"B14","doi-asserted-by":"crossref","first-page":"191","DOI":"10.1016\/j.jad.2016.10.019","article-title":"The 52 symptoms of major depression: lack of content overlap among seven common depression scales.","volume":"208","author":"Fried","year":"2017","journal-title":"J. Affect. Disord."},{"key":"B15","author":"Harrell","year":"2017","journal-title":"Hmisc: Harrell Miscellaneous. R Package Version 4.0-3."},{"key":"B16","doi-asserted-by":"crossref","first-page":"55","DOI":"10.1080\/00401706.1970.10488634","article-title":"Ridge regression: biased estimation for nonorthogonal problems.","volume":"12","author":"Hoerl","year":"1970","journal-title":"Technometrics"},{"key":"B17","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1037\/1040-3590.15.2.173","article-title":"Assessing worry in older adults: confirmatory factor analysis of the Penn state worry questionnaire and psychometric properties of an abbreviated model.","volume":"15","author":"Hopko","year":"2003","journal-title":"Psychol. Assess."},{"key":"B18","doi-asserted-by":"crossref","first-page":"1146","DOI":"10.1037\/a0037251","article-title":"Psychometric properties of abbreviated and ultra-brief versions of the Penn State Worry Questionnaire.","volume":"26","author":"Kertz","year":"2014","journal-title":"Psychol. Assess."},{"key":"B19","doi-asserted-by":"publisher","first-page":"117","DOI":"10.1111\/j.1745-6916.2008.00069.x","article-title":"In pursuit of happiness empirical answers to philosophical questions.","volume":"3","author":"Kesebir","year":"2008","journal-title":"Perspect. Psychol. Sci."},{"key":"B20","doi-asserted-by":"crossref","first-page":"183","DOI":"10.1080\/00223891.2020.1737093","article-title":"Abbreviated three-item versions of the satisfaction with life scale and the harmony in life scale yield as strong psychometric properties as the original scales.","volume":"103","author":"Kjell","year":"2020","journal-title":"J. Pers. Assess."},{"key":"B21","doi-asserted-by":"publisher","DOI":"10.3389\/fpsyg.2021.601679","article-title":"Computational language assessments of harmony in life -not satisfaction with life nor rating scales - correlate with cooperative behaviors.","author":"Kjell","year":"","journal-title":"Front. Spec. Issue Seman. Algor. Assess. Attitud. Personal."},{"key":"B22","doi-asserted-by":"crossref","first-page":"92","DOI":"10.1037\/met0000191","article-title":"Semantic measures: using natural language processing to measure, differentiate, and describe psychological constructs.","volume":"24","author":"Kjell","year":"2019","journal-title":"Psychol. Methods"},{"key":"B23","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-37250-7_4","article-title":"Semantic similarity scales: using semantic similarity scales to measure depression and worry","author":"Kjell","year":"2020","journal-title":"Statistical Semantics"},{"key":"B24","author":"Kjell","year":"","journal-title":"Text: An R-Package for Analyzing and Visualizing Human Language Using Natural Language Processing and Deep Learning"},{"key":"B25","author":"Kjell","year":"","journal-title":"Question Based Text Responses Analysed with AI-Based Transformer Approach the Upper Limits of Rating Scale Reliability."},{"key":"B26","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1046\/j.1525-1497.2001.016009606.x","article-title":"The PHQ-9: validity of a brief depression severity measure [Electronic version].","volume":"16","author":"Kroenke","year":"2001","journal-title":"J. Gen. Intern. Med."},{"key":"B27","doi-asserted-by":"publisher","first-page":"211","DOI":"10.1037\/0033-295x.104.2.211","article-title":"A solution to Plato\u2019s problem: the latent semantic analysis theory of acquisition, induction, and representation of knowledge.","volume":"104","author":"Landauer","year":"1997","journal-title":"Psychol. Rev."},{"key":"B28","doi-asserted-by":"crossref","first-page":"266","DOI":"10.1097\/MLR.0b013e318160d093","article-title":"Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population.","volume":"46","author":"L\u00f6we","year":"2008","journal-title":"Med. Care"},{"key":"B29","doi-asserted-by":"crossref","first-page":"1194","DOI":"10.1097\/00005650-200412000-00006","article-title":"Monitoring depression treatment outcomes with the patient health questionnaire-9.","volume":"42","author":"L\u00f6we","year":"2004","journal-title":"Med. Care"},{"key":"B30","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.genhosppsych.2005.07.003","article-title":"Validity of the brief patient health questionnaire mood scale (PHQ-9) in the general population.","volume":"28","author":"Martin","year":"2006","journal-title":"Gen. Hosp. Psychiatry"},{"key":"B31","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1016\/0005-7967(90)90135-6","article-title":"Development and validation of the penn state worry questionnaire.","volume":"28","author":"Meyer","year":"1990","journal-title":"Behav. Res. Therapy"},{"key":"B32","doi-asserted-by":"crossref","first-page":"221","DOI":"10.1177\/1745691612441215","article-title":"The smartphone psychology manifesto.","volume":"7","author":"Miller","year":"2012","journal-title":"Perspect. Psychol. Sci."},{"key":"B33","doi-asserted-by":"publisher","first-page":"6561","DOI":"10.5210\/ojphi.v8i1.6561","article-title":"Identifying depression-related tweets from twitter for public health monitoring.","volume":"8","author":"Mowery","year":"2016","journal-title":"Online J. Public Health Inform."},{"key":"B34","doi-asserted-by":"crossref","first-page":"867","DOI":"10.1016\/j.jesp.2009.03.009","article-title":"Instructional manipulation checks: detecting satisficing to increase statistical power.","volume":"45","author":"Oppenheimer","year":"2009","journal-title":"J. Exp. Soc. Psychol."},{"key":"B35","year":"2020","journal-title":"R: A Language and Environment for Statistical Computing."},{"key":"B36","doi-asserted-by":"publisher","first-page":"13006","DOI":"10.1038\/s41598-017-12961-9","article-title":"Forecasting the onset and course of mental illness with Twitter data.","volume":"7","author":"Reece","year":"2017","journal-title":"Sci. Rep."},{"key":"B37","author":"Revelle","year":"2017","journal-title":"Psych: Procedures for Personality and Psychological Research."},{"key":"B38","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1177\/2167702612469015","article-title":"Using mechanical turk to study clinical populations.","volume":"1","author":"Shapiro","year":"2013","journal-title":"Clin. Psychol. Sci."},{"key":"B39","author":"Sikstr\u00f6m","year":"2018","journal-title":"Semantic Excel: An Introduction to a User-Friendly Online Software Application for Statistical Analyses of Text Data"},{"key":"B40","doi-asserted-by":"publisher","first-page":"1092","DOI":"10.1001\/archinte.166.10.1092","article-title":"A brief measure for assessing generalized anxiety disorder: the gad-7.","volume":"166","author":"Spitzer","year":"2006","journal-title":"Archiv. Intern. Med."},{"key":"B41","doi-asserted-by":"publisher","DOI":"10.1177\/1073191120976863","article-title":"On dimensionality, measurement invariance, and suitability of sum scores for the PHQ-9 and the GAD-7.","author":"Stochl","year":"2020","journal-title":"Assessment"},{"key":"B42","doi-asserted-by":"publisher","first-page":"156","DOI":"10.1186\/1741-7015-10-156","article-title":"Data-driven subtypes of major depressive disorder: a systematic review.","volume":"10","author":"Van Loo","year":"2012","journal-title":"BMC Med."},{"key":"B43","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1177\/1073191114534883","article-title":"Multidimensional assessment of beliefs about emotion: development and validation of the emotion and regulation beliefs scale.","volume":"22","author":"Veilleux","year":"2015","journal-title":"Assessment"},{"key":"B44","doi-asserted-by":"crossref","first-page":"e31","DOI":"10.1056\/NEJMp1109407","article-title":"The patient-centered outcomes research institute\u2013promoting better information, decisions, and health.","volume":"365","author":"Washington","year":"2011","journal-title":"N. Engl. J. Med."},{"key":"B45","doi-asserted-by":"crossref","first-page":"1686","DOI":"10.21105\/joss.01686","article-title":"Welcome to the tidyverse.","volume":"4","author":"Wickham","year":"2019","journal-title":"J. Open Source Softw."},{"key":"B46","doi-asserted-by":"publisher","first-page":"657","DOI":"10.1016\/j.janxdis.2014.07.005","article-title":"Comparison of the Penn State Worry Questionnaire (PSWQ) and abbreviated version (PSWQ-A) in a clinical and non-clinical population of older adults.","volume":"28","author":"Wuthrich","year":"2014","journal-title":"J. Anx. Disord."}],"container-title":["Frontiers in Psychology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpsyg.2021.602581\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,4]],"date-time":"2021-06-04T04:43:47Z","timestamp":1622781827000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpsyg.2021.602581\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,6,4]]},"references-count":46,"alternative-id":["10.3389\/fpsyg.2021.602581"],"URL":"https:\/\/doi.org\/10.3389\/fpsyg.2021.602581","relation":{},"ISSN":["1664-1078"],"issn-type":[{"value":"1664-1078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,6,4]]},"article-number":"602581"}}