{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T13:44:05Z","timestamp":1790862245179,"version":"4.1.0"},"reference-count":52,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2024,6,5]],"date-time":"2024-06-05T00:00:00Z","timestamp":1717545600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Psychol."],"abstract":"<jats:sec><jats:title>Introduction<\/jats:title><jats:p>This study explores the stability of scores on psychometrically validated trait questionnaires over time. We illustrate potential pitfalls through a larger study that used the Ruminative Response Scale (RRS) to categorize participants prior to study inclusion into two groups based on their habitual tendency to ruminate. Surprisingly, when we re-administered the RRS at the start of an experimental session, significant score changes occurred, resulting in participants shifting between the two groups.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>To address this, we modified our recruitment process, aiming to reduce careless responses, including an online RRS assessment a week before the lab appointment. We analyzed the different samples prior to and after changing the recruitment procedure, as well as the total sample regarding the psychometric properties of the RRS. We also explored various indices to identify and predict score changes due to careless responding; however, only a subgroup of participants was successfully identified.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>Our findings suggest that Mahalanobis distances are effective for identifying substantial score changes, with baseline state rumination emerging as a marginally significant predictor.<\/jats:p><\/jats:sec><jats:sec><jats:title>Discussion<\/jats:title><jats:p>We discuss the importance of conducting manipulation checks and offer practical implications for research involving psychometrically validated trait questionnaires.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fpsyg.2024.1368390","type":"journal-article","created":{"date-parts":[[2024,6,11]],"date-time":"2024-06-11T19:32:29Z","timestamp":1718134349000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["On assessing trait rumination using the Ruminative Response Scale"],"prefix":"10.3389","volume":"15","author":[{"given":"Isabell","family":"Int-Veen","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ann-Christine","family":"Ehlis","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Andreas J.","family":"Fallgatter","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David","family":"Rosenbaum","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2024,6,5]]},"reference":[{"key":"ref1","author":"Allaire","year":"2017"},{"key":"ref2","year":""},{"key":"ref3","author":"Arnold","year":"2021"},{"key":"ref4","author":"Attali","year":"2022"},{"key":"ref5","doi-asserted-by":"publisher","first-page":"4","DOI":"10.1016\/j.jesp.2015.07.006","article-title":"Methods for the detection of carelessly invalid responses in survey data","volume":"66","author":"Curran","year":"2016","journal-title":"J. Exp. Soc. Psychol."},{"key":"ref6","doi-asserted-by":"publisher","first-page":"240","DOI":"10.1026\/1616-3443.38.4.240","article-title":"Einfluss von Achtsamkeits\u00fcbung und Dezentrierung auf Rumination und Spezifit\u00e4t autobiographischer Erinnerungen","volume":"38","author":"de Jong-Meyer","year":"2009","journal-title":"Z. Klin. Psychol. Psychother."},{"key":"ref7","doi-asserted-by":"publisher","first-page":"559","DOI":"10.1007\/s10869-017-9514-9","article-title":"Dirty data: the effects of screening respondents who provide low-quality data in survey research","volume":"33","author":"DeSimone","year":"2018","journal-title":"J. Bus. Psychol."},{"key":"ref8","doi-asserted-by":"publisher","first-page":"S50","DOI":"10.1097\/01.mlr.0000245426.10853.30","article-title":"Classical test theory","volume":"44","author":"DeVellis","year":"2006","journal-title":"Med. Care"},{"key":"ref9","doi-asserted-by":"publisher","first-page":"446","DOI":"10.3389\/fnhum.2013.00446","article-title":"The Amsterdam resting-state questionnaire reveals multiple phenotypes of resting-state cognition","volume":"7","author":"Diaz","year":"2013","journal-title":"Front. Hum. Neurosci."},{"key":"ref10","doi-asserted-by":"publisher","first-page":"225","DOI":"10.1016\/j.jbtep.2010.12.003","article-title":"The perseverative thinking questionnaire (PTQ): validation of a content-independent measure of repetitive negative thinking","volume":"42","author":"Ehring","year":"2011","journal-title":"J. Behav. Ther. Exp. Psychiatry"},{"key":"ref11","doi-asserted-by":"publisher","first-page":"854","DOI":"10.1108\/IntR-03-2018-0089","article-title":"The value of online surveys: a look back and a look ahead","volume":"28","author":"Evans","year":"2018","journal-title":"Internet Res."},{"key":"ref12","doi-asserted-by":"publisher","first-page":"219","DOI":"10.1027\/1015-5759\/a000413","article-title":"States and traits: theories, models, and assessment","volume":"33","author":"Geiser","year":"2017","journal-title":"Eur. J. Psychol. Assess."},{"key":"ref13","doi-asserted-by":"publisher","first-page":"716","DOI":"10.2466\/02.08.PR0.112.3.716-726","article-title":"Translation and initial validation of the Japanese version of the ruminative responses scale","volume":"112","author":"Hasegawa","year":"2013","journal-title":"Psychol. Rep."},{"key":"ref14","first-page":"115","article-title":"Personality assessment via Internet: Comparing online and paper-and-pencil questionnaires","author":"Hertel","year":"2002","journal-title":"Online social sciences"},{"key":"ref15","doi-asserted-by":"publisher","first-page":"559","DOI":"10.1177\/0146621616664046","article-title":"Comparison of classical test theory and item response theory in individual change assessment","volume":"40","author":"Jabrayilov","year":"2016","journal-title":"Appl. Psychol. Meas."},{"key":"ref16","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1016\/j.jrp.2004.09.009","article-title":"Ascertaining the validity of individual protocols from web-based personality inventories","volume":"39","author":"Johnson","year":"2005","journal-title":"J. Res. Pers."},{"key":"ref17","doi-asserted-by":"publisher","first-page":"381","DOI":"10.1037\/pha0000546","article-title":"Careless responding in crowdsourced alcohol research: a systematic review and meta-analysis of practices and prevalence","volume":"30","author":"Jones","year":"2022","journal-title":"Exp. Clin. Psychopharmacol."},{"key":"ref18","doi-asserted-by":"publisher","first-page":"221","DOI":"10.1037\/0021-843X.106.2.221","article-title":"The response styles theory of depression: tests and an extension of the theory","volume":"106","author":"Just","year":"1997","journal-title":"J. Abnorm. Psychol."},{"key":"ref19","doi-asserted-by":"publisher","first-page":"375","DOI":"10.1037\/1040-3590.13.3.375","article-title":"A construct validation study of the response styles questionnaire rumination scale in participants with a recent-onset major depressive episode","volume":"13","author":"Kasch","year":"2001","journal-title":"Psychol. Assess."},{"key":"ref20","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1159\/000119004","article-title":"The \u2018Trier social stress test\u2019 \u2013 a tool for investigating psychobiological stress responses in a laboratory setting","volume":"28","author":"Kirschbaum","year":"1993","journal-title":"Neuropsychobiology"},{"key":"ref21","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting Intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","year":"2016","journal-title":"J. Chiropr. Med."},{"key":"ref22","author":"Kruijt","year":"2023"},{"key":"ref23","doi-asserted-by":"publisher","first-page":"103525","DOI":"10.1016\/j.nicl.2023.103525","article-title":"In situ fNIRS measurements during cognitive behavioral emotion regulation training in rumination-focused therapy: a randomized-controlled trial","volume":"40","author":"Laicher","year":"2023","journal-title":"NeuroImage Clin."},{"key":"ref24","doi-asserted-by":"publisher","first-page":"829","DOI":"10.1177\/1073191119888582","article-title":"Do state and trait measures measure states and traits? The case of community-dwelling caregivers of older adults","volume":"28","author":"Lance","year":"2021","journal-title":"Assessment"},{"key":"ref25","doi-asserted-by":"publisher","first-page":"229","DOI":"10.18148\/SRM\/2019.V13I3.7403","article-title":"Too fast, too straight, too weird: non-reactive indicators for meaningless data in internet surveys","author":"Leiner","year":"2019","journal-title":"Survey Res. Methods"},{"key":"ref26","doi-asserted-by":"publisher","first-page":"319","DOI":"10.1177\/0049124104268664","article-title":"Meta-analysis of randomized response research: thirty-five years of validation","volume":"33","author":"Lensvelt-Mulders","year":"2005","journal-title":"Sociol. Methods Res."},{"key":"ref27","doi-asserted-by":"publisher","first-page":"e0219854","DOI":"10.1371\/journal.pone.0219854","article-title":"Intraclass correlation \u2013 a discussion and demonstration of basic features","volume":"14","author":"Liljequist","year":"2019","journal-title":"PLoS One"},{"key":"ref28","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1016\/j.jrp.2013.09.008","article-title":"Caring about carelessness: participant inattention and its effects on research","volume":"48","author":"Maniaci","year":"2014","journal-title":"J. Res. Pers."},{"key":"ref9001","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1037\/a0028085","article-title":"Identifying careless responses in survey data","volume":"17","author":"Meade","year":"2012","journal-title":"Psychol. Methods"},{"key":"ref29","doi-asserted-by":"publisher","first-page":"1034","DOI":"10.1016\/j.brat.2008.06.004","article-title":"Ruminative self-focus, negative life events, and negative affect","volume":"46","author":"Moberly","year":"2008","journal-title":"Behav. Res. Ther."},{"key":"ref30","doi-asserted-by":"publisher","first-page":"e1000097","DOI":"10.1371\/journal.pmed.1000097","article-title":"Preferred reporting items for systematic reviews and Meta-analyses: the PRISMA statement","volume":"6","author":"Moher","year":"2009","journal-title":"PLoS Med."},{"key":"ref31","doi-asserted-by":"publisher","first-page":"569","DOI":"10.1037\/0021-843X.100.4.569","article-title":"Responses to depression and their effects on the duration of depressive episodes","volume":"100","author":"Nolen-Hoeksema","year":"1991","journal-title":"J. Abnorm. Psychol."},{"key":"ref32","doi-asserted-by":"publisher","first-page":"1061","DOI":"10.1037\/\/0022-3514.77.5.1061","article-title":"Explaining the gender difference in depressive symptoms","volume":"77","author":"Nolen-Hoeksema","year":"1999","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref33","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1037\/0022-3514.61.1.115","article-title":"A prospective study of depression and posttraumatic stress symptoms after a natural disaster: the 1989 Loma Prieta earthquake","volume":"61","author":"Nolen-Hoeksema","year":"1991","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref34","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1037\/\/0022-3514.67.1.92","article-title":"Ruminative coping with depressed mood following loss","volume":"67","author":"Nolen-Hoeksema","year":"1994","journal-title":"J. Pers. Soc. Psychol."},{"key":"ref35","doi-asserted-by":"publisher","first-page":"400","DOI":"10.1111\/j.1745-6924.2008.00088.x","article-title":"Rethinking rumination","volume":"3","author":"Nolen-Hoeksema","year":"2008","journal-title":"Perspect. Psychol. Sci."},{"key":"ref36","author":"Papageorgiou","year":"2004"},{"key":"ref37","volume-title":"R: A language and environment for statistical computing","year":"2023"},{"key":"ref38","volume-title":"Psych: Procedures for psychological, psychometric, and personality research","author":"Revelle","year":""},{"key":"ref39","volume-title":"psychTools: Tools to accompany the \u201cpsych\u201d package for psychological research","author":"Revelle","year":""},{"key":"ref01","doi-asserted-by":"publisher","first-page":"107","DOI":"10.1016\/j.neuroimage.2017.12.061","article-title":"Cortical hemodynamic changes during the Trier Social Stress Test: An fNIRS study","volume":"171","author":"Rosenbaum","year":"","journal-title":"NeuroImage"},{"key":"ref02","doi-asserted-by":"publisher","first-page":"15588","DOI":"10.1038\/s41598-018-33777-1","article-title":"Disrupted prefrontal functional connectivity during post-stress adaption in high ruminators","volume":"8","author":"Rosenbaum","year":"","journal-title":"Sci. Rep"},{"key":"ref40","doi-asserted-by":"publisher","first-page":"100344","DOI":"10.1016\/j.ynstr.2021.100344","article-title":"Insights from a laboratory and naturalistic investigation on stress, rumination and frontal brain functioning in MDD: an fNIRS study","volume":"15","author":"Rosenbaum","year":"2021","journal-title":"Neurobiology Stress"},{"key":"ref41","volume-title":"RStudio: Integrated development environment for R (version 2023.03.1)","year":"2023"},{"key":"ref42","doi-asserted-by":"publisher","first-page":"609","DOI":"10.1007\/s10862-010-9182-9","article-title":"Brooding and reflection as subtypes of rumination: evidence from confirmatory factor analysis in nonclinical samples using the Dutch ruminative response scale","volume":"32","author":"Schoofs","year":"2010","journal-title":"J. Psychopathol. Behav. Assess."},{"key":"ref43","doi-asserted-by":"publisher","first-page":"116","DOI":"10.1016\/j.cpr.2008.10.003","article-title":"A roadmap to rumination: a review of the definition, assessment, and conceptualization of this multifaceted construct","volume":"29","author":"Smith","year":"2009","journal-title":"Clin. Psychol. Rev."},{"key":"ref44","first-page":"79","article-title":"States and traits in psychological assessment","volume":"8","author":"Steyer","year":"1992","journal-title":"Eur. J. Psychol. Assess."},{"key":"ref45","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1023\/A:1023910315561","article-title":"Rumination reconsidered: a psychometric analysis","volume":"27","author":"Treynor","year":"2003","journal-title":"Cogn. Ther. Res."},{"key":"ref46","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1080\/10615806.2010.482985","article-title":"Relations between a ruminative thinking style and obsessive-compulsive symptoms in non-clinical samples","volume":"24","author":"Wahl","year":"2011","journal-title":"Anxiety Stress Coping"},{"key":"ref9002","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1111\/apps.12118","article-title":"Applying social psychology to prevent careless responding during online surveys","volume":"67","author":"Ward","year":"2018","journal-title":"Appl. Psychol."},{"key":"ref47","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1146\/annurev-psych-040422-045007","article-title":"Dealing with careless responding in survey data: prevention, identification, and recommended best practices","volume":"74","author":"Ward","year":"2023","journal-title":"Annu. Rev. Psychol."},{"key":"ref48","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-319-24277-4","volume-title":"ggplot2: Elegant graphics for data analysis","author":"Wickham","year":"2016"}],"container-title":["Frontiers in Psychology"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpsyg.2024.1368390\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,6,11]],"date-time":"2024-06-11T19:32:37Z","timestamp":1718134357000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fpsyg.2024.1368390\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,6,5]]},"references-count":52,"alternative-id":["10.3389\/fpsyg.2024.1368390"],"URL":"https:\/\/doi.org\/10.3389\/fpsyg.2024.1368390","relation":{},"ISSN":["1664-1078"],"issn-type":[{"value":"1664-1078","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,6,5]]},"article-number":"1368390"}}