{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T23:54:12Z","timestamp":1781826852695,"version":"3.54.5"},"reference-count":57,"publisher":"Oxford University Press (OUP)","issue":"5","license":[{"start":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T00:00:00Z","timestamp":1781740800000},"content-version":"vor","delay-in-days":20,"URL":"https:\/\/creativecommons.org\/licenses\/by-nc\/4.0\/"}],"funder":[{"name":"Social Sciences and Humanities Research","award":["1037949"],"award-info":[{"award-number":["1037949"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2026,5,29]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>Performance invalidity is well-documented in clinical and forensic neuropsychology, yet its potential influence on data quality in psychological research has received little attention. This raises concerns for reproducibility, particularly given widespread use of undergraduate students as research participants. The present study examined whether incorporating Performance Validity Tests (PVTs) into a research protocol provides a practical method for improving data quality and interpretation.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Method<\/jats:title>\n                    <jats:p>As part of a larger replication initiative, PVTs were embedded into a direct replication of a 2008 prescribed optimism study. 87 undergraduate students were classified as credible or non-credible based on PVT performance, and analyses were repeated within subgroups.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>The effect replicated successfully across the sample (d\u2009=\u20090.98), consistent with the original study (d\u2009=\u20090.93). 17.2% of participants failed one or more PVTs. The credible subgroup yielded a larger effect (d\u2009=\u20091.02) than the full sample, whereas the non-credible subgroup produced a smaller effect (d\u2009=\u20090.87), and a directional reversal in the secondary analysis \u2013 a pattern inconsistent with prior literature and suggestive of systematic noise that disengaged responding may introduce into data. Further, credible and non-credible subgroups differed significantly on the primary outcome despite being demographically and experimentally equivalent.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Performance invalidity occurs at a non-trivial rate in undergraduate research samples and can influence study outcomes. Incorporating PVTs into research protocols offers a practical, scalable safeguard to improving data quality and transparency, highlighting how neuropsychology\u2019s established tools could contribute solutions to reproducibility challenges in psychological science.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1093\/arclin\/acag044","type":"journal-article","created":{"date-parts":[[2026,5,21]],"date-time":"2026-05-21T11:44:13Z","timestamp":1779363853000},"source":"Crossref","is-referenced-by-count":0,"title":["Does performance invalidity undermine replication? Evidence from an undergraduate student participant sample"],"prefix":"10.1093","volume":"41","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1631-105X","authenticated-orcid":false,"given":"Melanie A","family":"Mascarenhas","sequence":"first","affiliation":[{"name":"Graduate Department of Psychological Clinical Science, University of Toronto Scarborough , Toronto, Ontario ,","place":["Canada"]}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Konstantine K","family":"Zakzanis","sequence":"additional","affiliation":[{"name":"Graduate Department of Psychological Clinical Science, University of Toronto Scarborough , Toronto, Ontario ,","place":["Canada"]},{"name":"Department of Psychology, University of Toronto Scarborough , Toronto, Ontario ,","place":["Canada"]}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"286","published-online":{"date-parts":[[2026,6,18]]},"reference":[{"issue":"8","key":"2026061819371550400_ref1","doi-asserted-by":"publisher","first-page":"849","DOI":"10.1093\/arclin\/acs085","article-title":"Conducting research with non-clinical healthy undergraduates: Does effort play a role in neuropsychological test performance?","volume":"27","author":"An","year":"2012","journal-title":"Archives of Clinical Neuropsychology"},{"issue":"1","key":"2026061819371550400_ref2","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1037\/met0000051","article-title":"There\u2019s more than one way to conduct a replication study: Beyond statistical significance","volume":"21","author":"Anderson","year":"2016","journal-title":"Psychological Methods"},{"issue":"4","key":"2026061819371550400_ref3","doi-asserted-by":"publisher","first-page":"329","DOI":"10.1111\/j.1467-9280.2008.02089.x","article-title":"Prescribed optimism: Is it right to be wrong about the future?","volume":"19","author":"Armor","year":"2008","journal-title":"Psychological Science"},{"issue":"1","key":"2026061819371550400_ref4","doi-asserted-by":"publisher","first-page":"86","DOI":"10.1177\/0146167202238374","article-title":"The effects of mindset on behavior: Self-regulation in deliberative and implemental frames of mind","volume":"29","author":"Armor","year":"2003","journal-title":"Personality and Social Psychology Bulletin"},{"issue":"7","key":"2026061819371550400_ref5","doi-asserted-by":"publisher","first-page":"602","DOI":"10.1037\/0003-066x.63.7.602","article-title":"The neglected 95%: Why American psychology needs to become less American","volume":"63","author":"Arnett","year":"2008","journal-title":"American Psychologist"},{"key":"2026061819371550400_ref6","volume-title":"The dot counting test manual","author":"Boone","year":"2002"},{"issue":"5","key":"2026061819371550400_ref7","doi-asserted-by":"publisher","first-page":"765","DOI":"10.1037\/a0031751","article-title":"Do the disadvantaged legitimize the social system? A large-scale test of the status\u2013legitimacy hypothesis","volume":"104","author":"Brandt","year":"2013","journal-title":"Journal of Personality and Social Psychology"},{"key":"2026061819371550400_ref8","doi-asserted-by":"publisher","first-page":"217","DOI":"10.1016\/j.jesp.2013.10.005","article-title":"The replication recipe: What makes for a convincing replication?","volume":"50","author":"Brandt","year":"2014","journal-title":"Journal of Experimental Social Psychology"},{"issue":"11","key":"2026061819371550400_ref9","doi-asserted-by":"publisher","first-page":"1314","DOI":"10.1177\/0956797611435528","article-title":"Most reported genetic associations with general intelligence are probably false positives","volume":"23","author":"Chabris","year":"2012","journal-title":"Psychological Science"},{"key":"2026061819371550400_ref10","doi-asserted-by":"publisher","DOI":"10.4324\/9780203771587","article-title":"Statistical power analysis for the Behavioral sciences","author":"Cohen","year":"1988","journal-title":"Routledge."},{"issue":"3","key":"2026061819371550400_ref11","doi-asserted-by":"publisher","first-page":"314","DOI":"10.1017\/S1355617719001139","article-title":"Performance and symptom validity assessment in patients with apathy and cognitive impairment","volume":"26","author":"Dandachi-FitzGerald","year":"2020","journal-title":"Journal of the International Neuropsychological Society"},{"issue":"1","key":"2026061819371550400_ref12","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1080\/13854046.2014.989267","article-title":"I just want my research credit: Frequency of suboptimal effort in a non-clinical healthy undergraduate sample","volume":"29","author":"DeRight","year":"2014","journal-title":"The Clinical Neuropsychologist"},{"issue":"4","key":"2026061819371550400_ref13","doi-asserted-by":"publisher","first-page":"806","DOI":"10.1111\/ene.15703","article-title":"Performance validity tests in NONLITIGANT patients with functional motor disorder","volume":"30","author":"Di Vico","year":"2023","journal-title":"European Journal of Neurology"},{"issue":"2","key":"2026061819371550400_ref14","doi-asserted-by":"publisher","first-page":"175","DOI":"10.3758\/BF03193146","article-title":"G*power 3: A flexible statistical power analysis program for the social, behavioral, and biomedical sciences","volume":"39","author":"Faul","year":"2007","journal-title":"Behavior Research Methods"},{"issue":"6","key":"2026061819371550400_ref15","doi-asserted-by":"publisher","first-page":"555","DOI":"10.1177\/1745691612459059","article-title":"A vast graveyard of undead theories","volume":"7","author":"Ferguson","year":"2012","journal-title":"Perspectives on Psychological Science"},{"issue":"3","key":"2026061819371550400_ref16","doi-asserted-by":"publisher","first-page":"587","DOI":"10.1080\/13854046.2016.1277786","article-title":"Specificity rates for non-clinical, bilingual, Mexican Americans on three popular performance validity measures","volume":"31","author":"Gasquoine","year":"2017","journal-title":"The Clinical Neuropsychologist"},{"issue":"11","key":"2026061819371550400_ref17","doi-asserted-by":"publisher","first-page":"1074","DOI":"10.1037\/pas0001166","article-title":"Limitations of performance validity tests in dementia evaluations: The role of base rates","volume":"34","author":"Gaudet","year":"2022","journal-title":"Psychological Assessment"},{"issue":"1","key":"2026061819371550400_ref18","doi-asserted-by":"publisher","first-page":"43","DOI":"10.1016\/j.pmr.2006.11.002","article-title":"The pervasive influence of effort on neuropsychological tests","volume":"18","author":"Green","year":"2007","journal-title":"Physical Medicine and Rehabilitation Clinics of North America"},{"issue":"12","key":"2026061819371550400_ref19","doi-asserted-by":"publisher","first-page":"1045","DOI":"10.1080\/02699050110088254","article-title":"Effort has a greater effect on test scores than severe brain injury in compensation claimants","volume":"15","author":"Green","year":"2001","journal-title":"Brain Injury"},{"issue":"3","key":"2026061819371550400_ref20","doi-asserted-by":"publisher","first-page":"218","DOI":"10.1037\/1040-3590.6.3.218","article-title":"Validation of malingered amnesia measures with a large clinical sample","volume":"6","author":"Greiffenstein","year":"1994","journal-title":"Psychological Assessment"},{"issue":"2","key":"2026061819371550400_ref21","doi-asserted-by":"publisher","first-page":"402","DOI":"10.1080\/13854046.2022.2054863","article-title":"Validation of alternative dot counting test E-score cutoffs based on degree of cognitive impairment in veteran and civilian clinical samples","volume":"37","author":"Hansen","year":"2023","journal-title":"The Clinical Neuropsychologist"},{"issue":"2\u20133","key":"2026061819371550400_ref22","doi-asserted-by":"publisher","first-page":"61","DOI":"10.1017\/S0140525X0999152X","article-title":"The weirdest people in the world?","volume":"33","author":"Henrich","year":"2010","journal-title":"Behavioral and Brain Sciences"},{"issue":"2","key":"2026061819371550400_ref23","doi-asserted-by":"publisher","first-page":"210","DOI":"10.1006\/obhd.1994.1012","article-title":"The effects of choice and incentives on the overestimation of future performance","volume":"57","author":"Henry","year":"1994","journal-title":"Organizational Behavior and Human Decision Processes"},{"key":"2026061819371550400_ref24","volume-title":"IBM SPSS statistics for mac, version 28.0","author":"IBM Corp","year":"2021"},{"issue":"8","key":"2026061819371550400_ref25","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pmed.0020124","article-title":"Why most published research findings are false","volume":"2","author":"Ioannidis","year":"2005","journal-title":"PLoS Medicine"},{"issue":"6060","key":"2026061819371550400_ref26","doi-asserted-by":"publisher","first-page":"1225","DOI":"10.1126\/science.334.6060.1225","article-title":"Again, and again, and again \u2026","volume":"334","author":"Jasny","year":"2011","journal-title":"Science"},{"issue":"4","key":"2026061819371550400_ref27","doi-asserted-by":"publisher","first-page":"437","DOI":"10.1080\/0887044022000004920","article-title":"Perceived control and the optimistic bias: A meta-analytic review","volume":"17","author":"Klein","year":"2002","journal-title":"Psychology & Health"},{"issue":"3","key":"2026061819371550400_ref28","doi-asserted-by":"publisher","first-page":"142","DOI":"10.1027\/1864-9335\/a000178","article-title":"Investigating variation in replicability: A \u201cmany labs\u201d replication project","volume":"45","author":"Klein","year":"2014","journal-title":"Social Psychology"},{"key":"2026061819371550400_ref29","volume-title":"Massey & Sackett (2008, PS, study 1). Open Science Framework","author":"Lassetter","year":"2013"},{"key":"2026061819371550400_ref30","volume-title":"Neuropsychological assessment","author":"Lezak","year":"2012"},{"key":"2026061819371550400_ref30a","doi-asserted-by":"publisher","volume-title":"Archives of Clinical Neuropsychology","author":"","DOI":"10.1093\/arclin\/acy027"},{"issue":"6","key":"2026061819371550400_ref31","doi-asserted-by":"publisher","first-page":"537","DOI":"10.1177\/1745691612460688","article-title":"Replications in psychology research","volume":"7","author":"Makel","year":"2012","journal-title":"Perspectives on Psychological Science"},{"issue":"2","key":"2026061819371550400_ref32","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1177\/10892680221077997","article-title":"Introduction: Replication of crises - interdisciplinary reflections on the phenomenon of the replication crisis in psychology","volume":"26","author":"Malich","year":"2022","journal-title":"Review of General Psychology"},{"issue":"1","key":"2026061819371550400_ref33","doi-asserted-by":"publisher","first-page":"88","DOI":"10.1080\/13854046.2019.1637027","article-title":"A systematic review and meta-analysis of the test of memory malingering in adults: Two decades of deception detection","volume":"34","author":"Martin","year":"2020","journal-title":"The Clinical Neuropsychologist"},{"issue":"6","key":"2026061819371550400_ref34","doi-asserted-by":"publisher","first-page":"1054","DOI":"10.1080\/13854046.2018.1425481","article-title":"Cross-validation of the dot counting test in a large sample of credible and non-credible patients referred for neuropsychological testing","volume":"32","author":"McCaul","year":"2018","journal-title":"The Clinical Neuropsychologist"},{"issue":"9","key":"2026061819371550400_ref35","doi-asserted-by":"publisher","first-page":"945","DOI":"10.1136\/jnnp-2020-323776","article-title":"Performance validity test failure in clinical populations\u2014A systematic review","volume":"91","author":"McWhirter","year":"2020","journal-title":"Journal of Neurology, Neurosurgery & Psychiatry"},{"issue":"8","key":"2026061819371550400_ref36","doi-asserted-by":"publisher","first-page":"1094","DOI":"10.1076\/jcen.24.8.1094.8379","article-title":"Base rates of malingering and symptom exeggeration","volume":"24","author":"Mittenberg","year":"2002","journal-title":"Journal of Clinical and Experimental Neuropsychology"},{"issue":"6251","key":"2026061819371550400_ref37","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1126\/science.aac4716","article-title":"Estimating the reproducibility of psychological science","volume":"349","author":"Open Science Collaboration","year":"2015","journal-title":"Science"},{"issue":"4","key":"2026061819371550400_ref38","doi-asserted-by":"publisher","first-page":"867","DOI":"10.1016\/j.jesp.2009.03.009","article-title":"Instructional manipulation checks: Detecting satisficing to increase statistical power","volume":"45","author":"Oppenheimer","year":"2009","journal-title":"Journal of Experimental Social Psychology"},{"key":"2026061819371550400_ref39","article-title":"RStudio: Integrated development environment for R (version 2026.01.2) [computer software]. Posit software, PBC","author":"Posit Team","year":"2024"},{"key":"2026061819371550400_ref40","volume-title":"R: A language and environment for statistical computing (version 4.5.3)","author":"R Core Team","year":"2025"},{"issue":"1","key":"2026061819371550400_ref41","doi-asserted-by":"publisher","first-page":"97","DOI":"10.1093\/arclin\/acv062","article-title":"Performance Invalidity Base rates among healthy undergraduate research Participants","volume":"31","author":"Ross","year":"2016","journal-title":"Archives of Clinical Neuropsychology"},{"issue":"7","key":"2026061819371550400_ref42","doi-asserted-by":"publisher","first-page":"609","DOI":"10.1093\/arclin\/acu039","article-title":"Effort in college undergraduates is sufficient on the word memory test","volume":"29","author":"Santos","year":"2014","journal-title":"Archives of Clinical Neuropsychology"},{"issue":"6","key":"2026061819371550400_ref43","doi-asserted-by":"publisher","first-page":"1063","DOI":"10.1037\/0022-3514.67.6.1063","article-title":"Distinguishing optimism from neuroticism (and trait anxiety, self-mastery, and self-esteem): A reevaluation of the life orientation test","volume":"67","author":"Scheier","year":"1994","journal-title":"Journal of Personality and Social Psychology"},{"key":"2026061819371550400_ref44","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1016\/j.jesp.2015.10.001","article-title":"The pipeline project: Pre-publication independent replications of a single laboratory's research pipeline","volume":"66","author":"Schweinsberg","year":"2016","journal-title":"Journal of Experimental Social Psychology"},{"issue":"3","key":"2026061819371550400_ref45","doi-asserted-by":"publisher","first-page":"515","DOI":"10.1037\/0022-3514.51.3.515","article-title":"College sophomores in the laboratory: Influences of a narrow data base on social psychology's view of human nature","volume":"51","author":"Sears","year":"1986","journal-title":"Journal of Personality and Social Psychology"},{"issue":"6","key":"2026061819371550400_ref46","doi-asserted-by":"publisher","first-page":"735","DOI":"10.1093\/arclin\/acaa019","article-title":"Multidimensional malingering criteria for neuropsychological assessment: A 20-year update of the malingered neuropsychological dysfunction criteria","volume":"35","author":"Sherman","year":"2020","journal-title":"Archives of Clinical Neuropsychology"},{"issue":"5","key":"2026061819371550400_ref47","doi-asserted-by":"publisher","first-page":"415","DOI":"10.1093\/arclin\/acu028","article-title":"Base rate of performance invalidity among non-clinical undergraduate research participants","volume":"29","author":"Silk-Eglit","year":"2014","journal-title":"Archives of Clinical Neuropsychology"},{"key":"2026061819371550400_ref48","doi-asserted-by":"publisher","DOI":"10.1037\/e519702015-014","article-title":"False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant","author":"Simmons","year":"2011"},{"issue":"2","key":"2026061819371550400_ref49","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1037\/h0083096","article-title":"Subject selection bias in psychological research. Canadian psychologist \/","volume":"7a","author":"Smart","year":"1966","journal-title":"Psychologie Canadienne"},{"issue":"8","key":"2026061819371550400_ref50","doi-asserted-by":"publisher","first-page":"1362","DOI":"10.1080\/13854046.2013.845248","article-title":"WAIS-IV reliable digit span is no more accurate than age corrected scaled score as an indicator of invalid performance in a veteran sample undergoing evaluation for Mtbi","volume":"27","author":"Spencer","year":"2013","journal-title":"The Clinical Neuropsychologist"},{"issue":"6","key":"2026061819371550400_ref51","doi-asserted-by":"publisher","first-page":"1053","DOI":"10.1080\/13854046.2021.1896036","article-title":"American Academy of clinical neuropsychology (AACN) 2021 consensus statement on validity assessment: Update of the 2009 AACN consensus conference statement on neuropsychological assessment of effort, response bias, and malingering","volume":"35","author":"Sweet","year":"2021","journal-title":"The Clinical Neuropsychologist"},{"issue":"3","key":"2026061819371550400_ref52","doi-asserted-by":"publisher","first-page":"455","DOI":"10.1016\/S0887-6177(03)00078-7","article-title":"The test of memory malingering (TOMM): Normative data from cognitively intact, cognitively impaired, and elderly patients with dementia","volume":"19","author":"Teichner","year":"2004","journal-title":"Archives of Clinical Neuropsychology"},{"key":"2026061819371550400_ref53","volume-title":"Test of memory malingering (TOMM)","author":"Tombaugh","year":"1996"},{"issue":"8","key":"2026061819371550400_ref54","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1080\/13854046.2013.835447","article-title":"Assessing effort: Differentiating performance and symptom validity","volume":"27","author":"Van Dyke","year":"2013","journal-title":"The Clinical Neuropsychologist"},{"key":"2026061819371550400_ref54a","doi-asserted-by":"publisher","DOI":"10.1037\/t15169-000","volume-title":"Wechsler Adult Intelligence Scale--Fourth Edition [Jeu de donn\u00e9es]","author":"Wechsler","year":"2008"},{"issue":"3","key":"2026061819371550400_ref55","doi-asserted-by":"publisher","first-page":"216","DOI":"10.1037\/h0086893","article-title":"Psychologists' response to criticisms about research based on undergraduate participants: A developmental perspective","volume":"42","author":"Wintre","year":"2001","journal-title":"Canadian Psychology \/ Psychologie Canadienne"}],"container-title":["Archives of Clinical Neuropsychology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/academic.oup.com\/acn\/article-pdf\/41\/5\/acag044\/68555374\/acag044.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/academic.oup.com\/acn\/article-pdf\/41\/5\/acag044\/68555374\/acag044.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,18]],"date-time":"2026-06-18T23:37:22Z","timestamp":1781825842000},"score":1,"resource":{"primary":{"URL":"https:\/\/academic.oup.com\/acn\/article\/doi\/10.1093\/arclin\/acag044\/8711297"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,29]]},"references-count":57,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,5,29]]}},"URL":"https:\/\/doi.org\/10.1093\/arclin\/acag044","relation":{},"ISSN":["1873-5843"],"issn-type":[{"value":"1873-5843","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026,8]]},"published":{"date-parts":[[2026,5,29]]},"article-number":"acag044"}}