{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,28]],"date-time":"2026-07-28T18:59:52Z","timestamp":1785265192400,"version":"3.55.0"},"reference-count":32,"publisher":"MIT Press","license":[{"start":{"date-parts":[[2026,7,28]],"date-time":"2026-07-28T00:00:00Z","timestamp":1785196800000},"content-version":"vor","delay-in-days":208,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/100023581","name":"NSF GRFP","doi-asserted-by":"crossref","award":["2140743"],"award-info":[{"award-number":["2140743"]}],"id":[{"id":"10.13039\/100023581","id-type":"DOI","asserted-by":"crossref"}]},{"name":"National Defense Science and Engineering Graduate Fellowship Program"},{"DOI":"10.13039\/100000181","name":"Air Force Office of Scientific Research","doi-asserted-by":"crossref","award":["FA9550-23-F-0014"],"award-info":[{"award-number":["FA9550-23-F-0014"]}],"id":[{"id":"10.13039\/100000181","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,7,15]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>Experimenter demand effects occur when participants alter their behavior to align with perceived study hypotheses, threatening internal validity. Concern about demand effects is pervasive in psychology. Experimenter demand may be especially acute in studies relying on experienced participants recruited online (e.g., via Prolific), who may readily guess hypotheses, or when using common paradigms (e.g., vignette studies and interventions) where study goals are transparent. We conducted three preregistered experiments (N = 2,252) examining whether explicit demand cues influence online participants\u2019 behavior across three paradigms commonly used in psychology: a dictator game, replicating prior work on demand effects (Experiment 1); a moral dilemma vignette (Experiment 2); and an intervention on group attitudes (Experiment 3). We randomly assigned participants on Prolific to receive information about the study\u2019s hypothesis or to a no-information control. As expected, we find that receiving such information significantly shifts participants\u2019 beliefs about the study\u2019s hypothesis, creating the potential for experimenter demand effects. Yet we find no evidence that learning any study\u2019s hypothesis alters participants\u2019 behavior, judgments, or attitudes, suggesting that demand effects may be elusive in online samples. Bayesian region-of-practical-equivalence (ROPE) analyses indicate that experimenter demand effects in these paradigms are typically confined to small magnitudes (|d| &amp;lt; 0.20), suggesting that effects larger than Cohen\u2019s d = 0.20 are unlikely to be fully explained by experimenter demand alone. These findings offer important insights for the design and interpretation of modern online psychology experiments.<\/jats:p>","DOI":"10.1162\/opmi.a.367","type":"journal-article","created":{"date-parts":[[2026,7,28]],"date-time":"2026-07-28T18:45:40Z","timestamp":1785264340000},"page":"998-1016","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":0,"title":["No Evidence of Experimenter Demand Effects in Three Online Psychology Experiments"],"prefix":"10.1162","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-6821-7443","authenticated-orcid":true,"given":"Lucas","family":"Woodley","sequence":"first","affiliation":[{"name":"Department of Psychology, Harvard University, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5006-2873","authenticated-orcid":true,"given":"Xavier","family":"Roberts-Gaal","sequence":"additional","affiliation":[{"name":"Department of Psychology, Harvard University, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-4708-8754","authenticated-orcid":true,"given":"Rachel","family":"Calcott","sequence":"additional","affiliation":[{"name":"Department of Psychology, Harvard University, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6929-9982","authenticated-orcid":true,"given":"Fiery","family":"Cushman","sequence":"additional","affiliation":[{"name":"Department of Psychology, Harvard University, Cambridge, MA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"281","published-online":{"date-parts":[[2026,7,15]]},"reference":[{"issue":"6","key":"2026072814453816400_bib1","doi-asserted-by":"publisher","first-page":"842","DOI":"10.1177\/0146167218798821","article-title":"The MTurkification of social and personality psychology","volume":"45","author":"Anderson","year":"2019","journal-title":"Personality & Social Psychology Bulletin"},{"issue":"3","key":"2026072814453816400_bib2","doi-asserted-by":"publisher","first-page":"739","DOI":"10.1111\/ajps.12081","article-title":"Separating the shirkers from the workers? Making sure respondents pay attention on self-administered surveys","volume":"58","author":"Berinsky","year":"2014","journal-title":"American Journal of Political Science"},{"key":"2026072814453816400_bib3","volume-title":"Heroku experiment template","author":"Callaway","year":"2025"},{"issue":"22","key":"2026072814453816400_bib4","doi-asserted-by":"publisher","first-page":"3127","DOI":"10.1002\/1097-0258(20001130)19:22%3C3127::aid-sim784%3E3.0.co;2-m","article-title":"A simple method for converting an odds ratio to effect size for use in meta-analysis","volume":"19","author":"Chinn","year":"2000","journal-title":"Statistics in Medicine"},{"key":"2026072814453816400_bib5","volume-title":"Statistical power analysis for the behavioral sciences","author":"Cohen","year":"1988","edition":"2nd ed."},{"issue":"1","key":"2026072814453816400_bib6","doi-asserted-by":"publisher","first-page":"143005","DOI":"10.1525\/collabra.143005","article-title":"A meta-analysis of the impact and heterogeneity of explicit demand characteristics","volume":"11","author":"Coles","year":"2025","journal-title":"Collabra: Psychology"},{"issue":"1","key":"2026072814453816400_bib7","doi-asserted-by":"publisher","first-page":"83","DOI":"10.1177\/10888683221104368","article-title":"Sixty years after Orne\u2019s American Psychologist article: A conceptual framework for subjective experiences elicited by demand characteristics","volume":"27","author":"Corneille","year":"2023","journal-title":"Personality and Social Psychology Review"},{"issue":"11","key":"2026072814453816400_bib8","doi-asserted-by":"publisher","first-page":"3266","DOI":"10.1257\/aer.20171330","article-title":"Measuring and bounding experimenter demand","volume":"108","author":"de Quidt","year":"2018","journal-title":"American Economic Review"},{"key":"2026072814453816400_bib9","doi-asserted-by":"publisher","first-page":"384","DOI":"10.4337\/9781788110563.00031","article-title":"Experimenter demand effects","volume-title":"Handbook of research methods and applications in experimental economics","author":"de Quidt","year":"2019"},{"issue":"5","key":"2026072814453816400_bib10","doi-asserted-by":"publisher","first-page":"964","DOI":"10.3758\/PBR.16.5.964","article-title":"Who is being deceived? The experimental demands of wearing a backpack","volume":"16","author":"Durgin","year":"2009","journal-title":"Psychonomic Bulletin & Review"},{"key":"2026072814453816400_bib11","doi-asserted-by":"publisher","first-page":"e229","DOI":"10.1017\/s0140525x15000965","article-title":"Cognition does not affect perception: Evaluating the evidence for \u201ctop-down\u201d effects","volume":"39","author":"Firestone","year":"2016","journal-title":"Behavioral and Brain Sciences"},{"key":"2026072814453816400_bib12","article-title":"How to improve data quality in online studies","author":"Gordon","year":"2025","journal-title":"Prolific"},{"issue":"3","key":"2026072814453816400_bib13","doi-asserted-by":"publisher","first-page":"247","DOI":"10.1006\/obhd.1996.0077","article-title":"The evaluability hypothesis: An explanation for preference reversals between joint and separate evaluations of alternatives","volume":"67","author":"Hsee","year":"1996","journal-title":"Organizational Behavior and Human Decision Processes"},{"issue":"1","key":"2026072814453816400_bib14","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1086\/297701","article-title":"Response effects in polite cultures: A test of acquiescence in Kazakhstan","volume":"63","author":"Javeline","year":"1999","journal-title":"Public Opinion Quarterly"},{"issue":"2","key":"2026072814453816400_bib15","doi-asserted-by":"publisher","first-page":"264","DOI":"10.1177\/0022022104272905","article-title":"The relation between culture and response styles: Evidence from 19 countries","volume":"36","author":"Johnson","year":"2005","journal-title":"Journal of Cross-Cultural Psychology"},{"issue":"4","key":"2026072814453816400_bib16","doi-asserted-by":"publisher","first-page":"101496","DOI":"10.1016\/j.leaqua.2021.101496","article-title":"The problem of demand effects in power studies: Moving beyond power priming","volume":"32","author":"Khademi","year":"2021","journal-title":"Leadership Quarterly"},{"issue":"10","key":"2026072814453816400_bib17","doi-asserted-by":"publisher","first-page":"1511","DOI":"10.1177\/01461672221103058","article-title":"Deontology and utilitarianism in real life: A set of moral dilemmas based on historic events","volume":"49","author":"K\u00f6rner","year":"2023","journal-title":"Personality & Social Psychology Bulletin"},{"issue":"4","key":"2026072814453816400_bib18","doi-asserted-by":"publisher","first-page":"355","DOI":"10.1177\/1948550617697177","article-title":"Equivalence tests: A practical primer for t tests, correlations, and meta-analyses","volume":"8","author":"Lakens","year":"2017","journal-title":"Social Psychological and Personality Science"},{"issue":"2","key":"2026072814453816400_bib19","doi-asserted-by":"publisher","first-page":"268","DOI":"10.1038\/s41562-025-02271-w","article-title":"Learning from outcomes shapes reliance on moral rules versus cost\u2013benefit reasoning","volume":"10","author":"Maier","year":"2026","journal-title":"Nature Human Behaviour"},{"issue":"2","key":"2026072814453816400_bib20","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1017\/S0003055418000837","article-title":"Demand effects in survey experiments: An empirical assessment","volume":"113","author":"Mummolo","year":"2019","journal-title":"American Political Science Review"},{"key":"2026072814453816400_bib21","doi-asserted-by":"publisher","DOI":"10.1017\/9781009010054","volume-title":"The Cambridge handbook of research methods and statistics for the social and behavioral sciences: Volume 1: Building a program of research","author":"Nichols","year":"2023"},{"issue":"11","key":"2026072814453816400_bib22","doi-asserted-by":"publisher","first-page":"776","DOI":"10.1037\/h0043424","article-title":"On the social psychology of the psychology experiment: With particular reference to demand characteristics and their implications","volume":"17","author":"Orne","year":"1962","journal-title":"American Psychologist"},{"key":"2026072814453816400_bib23","doi-asserted-by":"publisher","first-page":"110","DOI":"10.1093\/acprof:oso\/9780195385540.003.0005","article-title":"Demand characteristics and the concept of quasi-controls","volume-title":"Artifacts in behavioral research: Robert Rosenthal and Ralph L. Rosnow\u2019s classic books","author":"Orne","year":"2009"},{"key":"2026072814453816400_bib24","volume-title":"R: A language and environment for statistical computing","author":"R Core Team","year":"2025"},{"issue":"4","key":"2026072814453816400_bib25","doi-asserted-by":"publisher","first-page":"281","DOI":"10.1037\/mot0000091","article-title":"A 50-year review of psychological reactance theory: Do not read this article","volume":"4","author":"Rosenberg","year":"2018","journal-title":"Motivation Science"},{"key":"2026072814453816400_bib26","volume-title":"Introducing psychology","author":"Schacter","year":"2021","edition":"5th ed."},{"issue":"8","key":"2026072814453816400_bib27","doi-asserted-by":"publisher","first-page":"943","DOI":"10.1177\/19485506241273193","article-title":"The game within the game: The potential influence of demand characteristics and participant beliefs in violent video game studies","volume":"15","author":"Seetahul","year":"2024","journal-title":"Social Psychological and Personality Science"},{"issue":"3","key":"2026072814453816400_bib28","doi-asserted-by":"publisher","first-page":"393","DOI":"10.1037\/cns0000435","article-title":"Ignorance is bliss: A meta-analysis of the fear-reducing effects of very brief exposure","volume":"12","author":"Siegel","year":"2025","journal-title":"Psychology of Consciousness: Theory, Research, and Practice"},{"issue":"1","key":"2026072814453816400_bib29","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/0022-1031(70)90072-7","article-title":"The cooperative subject: Myth or reality?","volume":"6","author":"Sigall","year":"1970","journal-title":"Journal of Experimental Social Psychology"},{"issue":"7","key":"2026072814453816400_bib30","doi-asserted-by":"publisher","first-page":"1753","DOI":"10.1037\/emo0001368","article-title":"Examining demand effects on direct and indirect affect measures in affect induction procedures","volume":"24","author":"Wenzel","year":"2024","journal-title":"Emotion"},{"key":"2026072814453816400_bib31","volume-title":"The effect of experimenter demand on inference","author":"Winichakul","year":"2024"},{"issue":"1","key":"2026072814453816400_bib32","doi-asserted-by":"publisher","first-page":"75","DOI":"10.1007\/s10683-009-9230-z","article-title":"Experimenter demand effects in economic experiments","volume":"13","author":"Zizzo","year":"2010","journal-title":"Experimental Economics"}],"container-title":["Open Mind"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/opmi\/article-pdf\/doi\/10.1162\/OPMI.a.367\/2617749\/opmi.a.367.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/opmi\/article-pdf\/doi\/10.1162\/OPMI.a.367\/2617749\/opmi.a.367.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,28]],"date-time":"2026-07-28T18:45:53Z","timestamp":1785264353000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/opmi\/article\/doi\/10.1162\/OPMI.a.367\/137947\/No-Evidence-of-Experimenter-Demand-Effects-in"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026]]},"references-count":32,"URL":"https:\/\/doi.org\/10.1162\/opmi.a.367","relation":{},"ISSN":["2470-2986"],"issn-type":[{"value":"2470-2986","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2026]]},"published":{"date-parts":[[2026]]}}}