{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T04:19:26Z","timestamp":1777349966090,"version":"3.51.4"},"reference-count":80,"publisher":"Springer Science and Business Media LLC","issue":"S23","license":[{"start":{"date-parts":[[2019,11,19]],"date-time":"2019-11-19T00:00:00Z","timestamp":1574121600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2019,11,19]],"date-time":"2019-11-19T00:00:00Z","timestamp":1574121600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"NWO","award":["016.Vici.170.083"],"award-info":[{"award-number":["016.Vici.170.083"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Synthese"],"published-print":{"date-parts":[[2021,10]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>The replicability of findings in experimental psychology can be improved by distinguishing sharply between hypothesis-generating research and hypothesis-testing research. This distinction can be achieved by preregistration, a method that has recently attracted widespread attention. Although preregistration is fair in the sense that it inoculates researchers against hindsight bias and confirmation bias, preregistration does not allow researchers to analyze the data flexibly without the analysis being demoted to exploratory. To alleviate this concern we discuss how researchers may conduct blinded analyses (MacCoun and Perlmutter in Nature 526:187\u2013189, 2015). As with preregistration, blinded analyses break the feedback loop between the analysis plan and analysis outcome, thereby preventing cherry-picking and significance seeking.  However, blinded analyses retain the flexibility to account for unexpected peculiarities in the data.  We discuss different methods of blinding, offer recommendations for blinding of popular experimental designs, and introduce the design for an online blinding protocol.<\/jats:p>","DOI":"10.1007\/s11229-019-02456-7","type":"journal-article","created":{"date-parts":[[2019,11,19]],"date-time":"2019-11-19T12:03:33Z","timestamp":1574165013000},"page":"5745-5772","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Flexible yet fair: blinding analyses in experimental psychology"],"prefix":"10.1007","volume":"198","author":[{"given":"Gilles","family":"Dutilh","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0031-685X","authenticated-orcid":false,"given":"Alexandra","family":"Sarafoglou","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1596-1034","authenticated-orcid":false,"given":"Eric-Jan","family":"Wagenmakers","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2019,11,19]]},"reference":[{"key":"2456_CR1","doi-asserted-by":"publisher","first-page":"e21253","DOI":"10.7554\/eLife.21253","volume":"6","author":"F Aird","year":"2017","unstructured":"Aird, F., Kandela, I., Mantis, C., et al. (2017). Replication study: BET bromodomain inhibition as a therapeutic strategy to target c-myc. Elife, 6, e21253.","journal-title":"Elife"},{"key":"2456_CR2","doi-asserted-by":"publisher","first-page":"021303","DOI":"10.1103\/PhysRevLett.118.021303","volume":"118","author":"DS Akerib","year":"2017","unstructured":"Akerib, D. S., Alsum, S., Ara\u00fajo, H. M., Bai, X., Bailey, A. J., Balajthy, J., et al. (2017). Results from a search for dark matter in the complete lux exposure. Physical Review Letters, 118, 021303.","journal-title":"Physical Review Letters"},{"key":"2456_CR3","doi-asserted-by":"publisher","first-page":"e3000246","DOI":"10.1371\/journal.pbio.3000246","volume":"17","author":"C Allen","year":"2019","unstructured":"Allen, C., & Mehler, D. M. A. (2019). Open science challenges, benefits and tips in early career and beyond. PLOS Biology, 17, e3000246.","journal-title":"PLOS Biology"},{"key":"2456_CR4","doi-asserted-by":"publisher","first-page":"543","DOI":"10.1177\/1745691612459060","volume":"7","author":"M Bakker","year":"2012","unstructured":"Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7, 543\u2013554.","journal-title":"Perspectives on Psychological Science"},{"key":"2456_CR5","doi-asserted-by":"publisher","first-page":"666","DOI":"10.3758\/s13428-011-0089-5","volume":"43","author":"M Bakker","year":"2011","unstructured":"Bakker, M., & Wicherts, J. M. (2011). The (mis)reporting of statistical results in psychology journals. Behavior Research Methods, 43, 666\u2013678.","journal-title":"Behavior Research Methods"},{"key":"2456_CR6","volume-title":"Pitfalls in human research: Ten pivotal points","author":"TX Barber","year":"1976","unstructured":"Barber, T. X. (1976). Pitfalls in human research: Ten pivotal points. New York: Pergamon Press Inc."},{"issue":"9","key":"2456_CR7","doi-asserted-by":"publisher","first-page":"973","DOI":"10.1016\/j.jclinepi.2014.04.008","volume":"67","author":"S Bello","year":"2014","unstructured":"Bello, S., Krogsb\u00f8ll, L. T., Gruber, J., Zhao, Z. J., Fischer, D., & Hr\u00f3bjartsson, A. (2014). Lack of blinding of outcome assessors in animal model experiments implies risk of bias. Journal of Clinical Epidemiology, 67(9), 973\u2013983.","journal-title":"Journal of Clinical Epidemiology"},{"key":"2456_CR8","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1038\/s41562-017-0189-z","volume":"2","author":"DJ Benjamin","year":"2018","unstructured":"Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., et al. (2018). Redefine statistical significance. Nature Human Behaviour, 2, 6\u201310.","journal-title":"Nature Human Behaviour"},{"key":"2456_CR9","unstructured":"Bohannon, J. (2015). I fooled millions into thinking chocolate helps weight loss. Here\u2019s how. (Blog No. May 27). http:\/\/io9.com\/i-fooled-millions-into-thinking-chocolate-helps-weight-1707251800."},{"key":"2456_CR10","doi-asserted-by":"publisher","first-page":"108","DOI":"10.1006\/jmps.1999.1279","volume":"44","author":"M Browne","year":"2000","unstructured":"Browne, M. (2000). Cross-validation methods. Journal of Mathematical Psychology, 44, 108\u2013132.","journal-title":"Journal of Mathematical Psychology"},{"key":"2456_CR11","volume-title":"Model selection and multimodel inference: A practical information-theoretic approach","author":"KP Burnham","year":"2002","unstructured":"Burnham, K. P., & Anderson, D. R. (2002). Model selection and multimodel inference: A practical information-theoretic approach (2nd ed.). New York: Springer.","edition":"2"},{"key":"2456_CR12","doi-asserted-by":"publisher","first-page":"637","DOI":"10.1038\/s41562-018-0399-z","volume":"2","author":"CF Camerer","year":"2018","unstructured":"Camerer, C. F., Dreber, A., Holzmeister, F., Ho, T.-H., Huber, J., Johannesson, M., et al. (2018). Evaluating replicability of social science experiments in nature and science. Nature Human Behaviour, 2, 637\u2013644.","journal-title":"Nature Human Behaviour"},{"key":"2456_CR13","doi-asserted-by":"publisher","first-page":"149","DOI":"10.3389\/fnins.2012.00149","volume":"6","author":"J Carp","year":"2012","unstructured":"Carp, J. (2012). On the plurality of (methodological) worlds: Estimating the analytic flexibility of fMRI experiments. Frontiers in Neuroscience, 6, 149.","journal-title":"Frontiers in Neuroscience"},{"key":"2456_CR14","doi-asserted-by":"publisher","first-page":"609","DOI":"10.1016\/j.cortex.2012.12.016","volume":"49","author":"CD Chambers","year":"2013","unstructured":"Chambers, C. D. (2013). Registered reports: A new publishing initiative at cortex. Cortex, 49, 609\u2013610.","journal-title":"Cortex"},{"key":"2456_CR15","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1111\/add.12728","volume":"110","author":"CD Chambers","year":"2015","unstructured":"Chambers, C. D. (2015). Ten reasons why journals must review manuscripts before results are known. Addiction, 110, 10\u201311.","journal-title":"Addiction"},{"key":"2456_CR16","doi-asserted-by":"publisher","DOI":"10.1515\/9781400884940","volume-title":"The seven deadly sins of psychology: A manifesto for reforming the culture of scientific practice","author":"CD Chambers","year":"2017","unstructured":"Chambers, C. D. (2017). The seven deadly sins of psychology: A manifesto for reforming the culture of scientific practice. Princeton: Princeton University Press."},{"key":"2456_CR17","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1086\/503533","volume":"644","author":"A Conley","year":"2006","unstructured":"Conley, A., Goldhaber, G., Wan, L., Aldering, G., Amanullah, R., & Commins, E. D. (2006). The Supernova Cosmology Project. Measurement of $$\\omega $$m, $$\\omega \\lambda $$ from a blind analysis of type Ia supernovae with CMAGIC: Using color information to verify the acceleration of the universe. The Astrophysical Journal, 644, 1\u201320.","journal-title":"The Astrophysical Journal"},{"key":"2456_CR18","doi-asserted-by":"publisher","first-page":"640","DOI":"10.3758\/s13423-015-0913-5","volume":"23","author":"AOJ Cramer","year":"2016","unstructured":"Cramer, A. O. J., van Ravenzwaaij, D., Matzke, D., Steingroever, H., Wetzels, R., Grasman, R. P. P. P., et al. (2016). Hidden multiplicity in multiway ANOVA: Prevalence, consequences, and remedies. Psychonomic Bulletin and Review, 23, 640\u2013647.","journal-title":"Psychonomic Bulletin and Review"},{"key":"2456_CR19","doi-asserted-by":"publisher","first-page":"188","DOI":"10.1016\/j.actpsy.2014.02.001","volume":"148","author":"AD De Groot","year":"2014","unstructured":"De Groot, A. D. (2014). The meaning of \u201csignificance\u201d for different types of research. Translated and annotated by Eric-Jan Wagenmakers, Denny Borsboom, Josine Verhagen, Rogier Kievit, Marjan Bakker, Angelique Cramer, Dora Matzke, Don Mellenbergh, and Han L. J. van der Maas. Acta Psychologica, 148, 188\u2013194.","journal-title":"Acta Psychologica"},{"key":"2456_CR20","doi-asserted-by":"publisher","DOI":"10.1515\/9783112313121","volume-title":"Methodology: Foundations of inference and research in the behavioral sciences","author":"AD De Groot","year":"1969","unstructured":"De Groot, A. D. (1969). Methodology: Foundations of inference and research in the behavioral sciences. The Hague: Mouton."},{"key":"2456_CR21","doi-asserted-by":"publisher","first-page":"e13","DOI":"10.1037\/xge0000124","volume":"145","author":"L de Moli\u00e8re","year":"2016","unstructured":"de Moli\u00e8re, L., & Harris, A. J. L. (2016). Conceptual and direct replications fail to support the stake-likelihood hypothesis as an explanation for the interdependence of utility and likelihood judgments. Journal of Experimental Psychology: General, 145, e13.","journal-title":"Journal of Experimental Psychology: General"},{"key":"2456_CR22","doi-asserted-by":"publisher","first-page":"627","DOI":"10.1037\/0033-2909.125.6.627","volume":"125","author":"EL Deci","year":"1999","unstructured":"Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin, 125, 627\u2013668.","journal-title":"Psychological Bulletin"},{"key":"2456_CR23","doi-asserted-by":"publisher","first-page":"475","DOI":"10.1103\/PhysRev.52.475","volume":"52","author":"FG Dunnington","year":"1937","unstructured":"Dunnington, F. G. (1937). A determination of e\/m for an electron by a new deflection method. II. Physical Review, 52, 475\u2013501.","journal-title":"Physical Review"},{"key":"2456_CR24","doi-asserted-by":"publisher","first-page":"713","DOI":"10.3758\/s13414-017-1304-y","volume":"79","author":"G Dutilh","year":"2017","unstructured":"Dutilh, G., Vandekerckhove, J., Ly, A., Matzke, D., Pedroni, A., Frey, R., et al. (2017). A test of the diffusion model explanation for the worst performance rule using preregistration and blinding. Attention, Perception, & Psychophysics, 79, 713\u2013725.","journal-title":"Attention, Perception, & Psychophysics"},{"issue":"1","key":"2456_CR25","doi-asserted-by":"publisher","first-page":"158","DOI":"10.1177\/1745691615605826","volume":"11","author":"A Eerland","year":"2016","unstructured":"Eerland, A., Sherrill, A. M., Magliano, J. P., & Zwaan, R. A. (2016). Registered replication report: Hart & Albarrac\u00edn (2011). Perspectives on Psychological Science, 11(1), 158\u2013171.","journal-title":"Perspectives on Psychological Science"},{"key":"2456_CR26","doi-asserted-by":"publisher","first-page":"e0149794","DOI":"10.1371\/journal.pone.0149794","volume":"11","author":"A Etz","year":"2016","unstructured":"Etz, A., & Vandekerckhove, J. (2016). A Bayesian perspective on the reproducibility project: Psychology. PLoS One, 11, e0149794.","journal-title":"PLoS One"},{"key":"2456_CR27","volume-title":"The meaning of it all: Thoughts of a citizen-scientist","author":"R Feynman","year":"1998","unstructured":"Feynman, R. (1998). The meaning of it all: Thoughts of a citizen-scientist. New York: Perseus Books, Reading, MA."},{"key":"2456_CR28","doi-asserted-by":"publisher","first-page":"17538","DOI":"10.1073\/pnas.0805903105","volume":"105","author":"BU Forstmann","year":"2008","unstructured":"Forstmann, B. U., Dutilh, G., Brown, S. D., Neumann, J., von Cramon, D. Y., Ridderinkhof, K. R., et al. (2008). Striatum and pre-SMA facilitate decision-making under time pressure. Proceedings of the National Academy of Sciences, 105, 17538\u201317542.","journal-title":"Proceedings of the National Academy of Sciences"},{"key":"2456_CR29","doi-asserted-by":"publisher","first-page":"460","DOI":"10.1511\/2014.111.460","volume":"102","author":"A Gelman","year":"2014","unstructured":"Gelman, A., & Loken, E. (2014). The statistical crisis in science. American Scientist, 102, 460\u2013465.","journal-title":"American Scientist"},{"key":"2456_CR30","volume-title":"Bad science","author":"B Goldacre","year":"2009","unstructured":"Goldacre, B. (2009). Bad science. London: Fourth Estate."},{"key":"2456_CR31","doi-asserted-by":"publisher","first-page":"285","DOI":"10.1016\/0197-2456(95)00263-4","volume":"17","author":"PC G\u00f8tzsche","year":"1996","unstructured":"G\u00f8tzsche, P. C. (1996). Blinding during data analysis and writing of manuscripts. Controlled Clinical Trials, 17, 285\u2013290.","journal-title":"Controlled Clinical Trials"},{"key":"2456_CR32","doi-asserted-by":"publisher","first-page":"e72467","DOI":"10.1371\/journal.pone.0072467","volume":"8","author":"CR Harris","year":"2013","unstructured":"Harris, C. R., Coburn, N., Rohrer, D., & Pashler, H. (2013). Two failures to replicate high-performance-goal priming effects. PLoS One, 8, e72467.","journal-title":"PLoS One"},{"key":"2456_CR33","doi-asserted-by":"publisher","first-page":"e1002106","DOI":"10.1371\/journal.pbio.1002106","volume":"13","author":"ML Head","year":"2015","unstructured":"Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015). The extent and consequences of p-hacking in science. PLoS Biology, 13, e1002106.","journal-title":"PLoS Biology"},{"key":"2456_CR34","unstructured":"Heinrich, J. G. (2003). Benefits of blind analysis techniques. Unpublished manuscript. Retrieved November 14, 2019 from https:\/\/www-cdf.fnal.gov\/physics\/statistics\/notes\/cdf6576_blind.pdf."},{"key":"2456_CR35","doi-asserted-by":"publisher","first-page":"382","DOI":"10.1214\/ss\/1009212519","volume":"14","author":"JA Hoeting","year":"1999","unstructured":"Hoeting, J. A., Madigan, D., Raftery, A. E., & Volinsky, C. T. (1999). Bayesian model averaging: A tutorial. Statistical Science, 14, 382\u2013417.","journal-title":"Statistical Science"},{"key":"2456_CR36","doi-asserted-by":"publisher","first-page":"e1002190","DOI":"10.1371\/journal.pbio.1002190","volume":"13","author":"L Holman","year":"2015","unstructured":"Holman, L., Head, M. L., Lanfear, R., & Jennions, M. D. (2015). Evidence of experimental bias in the life sciences: We need blind data recording. PLOS Biology, 13, e1002190.","journal-title":"PLOS Biology"},{"key":"2456_CR37","doi-asserted-by":"publisher","first-page":"e21634","DOI":"10.7554\/eLife.21634","volume":"6","author":"SK Horrigan","year":"2017","unstructured":"Horrigan, S. K., Courville, P., Sampey, D., Zhou, F., Cai, S., et al. (2017). Replication study: Melanoma genome sequencing reveals frequent prex2 mutations. Elife, 6, e21634.","journal-title":"Elife"},{"key":"2456_CR38","doi-asserted-by":"publisher","first-page":"e1119","DOI":"10.1136\/bmj.e1119","volume":"344","author":"A Hr\u00f3bjartsson","year":"2012","unstructured":"Hr\u00f3bjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Hilden, J., Boutron, I., et al. (2012). Observer bias in randomised clinical trials with binary outcomes: Systematic review of trials with both blinded and non-blinded outcome assessors. BMJ, 344, e1119.","journal-title":"BMJ"},{"key":"2456_CR39","doi-asserted-by":"publisher","first-page":"937","DOI":"10.1093\/ije\/dyt270","volume":"43","author":"A Hr\u00f3bjartsson","year":"2014","unstructured":"Hr\u00f3bjartsson, A., Thomsen, A. S. S., Emanuelsson, F., Tendal, B., Rasmussen, J. V., Hilden, J., et al. (2014). Observer bias in randomized clinical trials with time-to-event outcomes: Systematic review of trials with both blinded and non-blinded outcome assessors. International Journal of Epidemiology, 43, 937\u2013948.","journal-title":"International Journal of Epidemiology"},{"key":"2456_CR40","first-page":"696","volume":"2","author":"JPA Ioannidis","year":"2005","unstructured":"Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2, 696\u2013701.","journal-title":"PLoS Medicine"},{"key":"2456_CR41","doi-asserted-by":"publisher","first-page":"524","DOI":"10.1177\/0956797611430953","volume":"23","author":"LK John","year":"2012","unstructured":"John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23, 524\u2013532.","journal-title":"Psychological Science"},{"key":"2456_CR42","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1146\/annurev.nucl.55.090704.151521","volume":"55","author":"JR Klein","year":"2005","unstructured":"Klein, J. R., & Roodman, A. (2005). Blind analysis in nuclear and particle physics. Annual Review of Nuclear and Particle Science, 55, 141\u2013163.","journal-title":"Annual Review of Nuclear and Particle Science"},{"key":"2456_CR43","doi-asserted-by":"publisher","first-page":"443","DOI":"10.1177\/2515245918810225","volume":"1","author":"R Klein","year":"2018","unstructured":"Klein, R., Vianello, M., Hasselman, F., Adams, B., Adams, R., Alper, S., et al. (2018). Many labs 2: Investigating variation in replicability across sample and setting. Advances in Methods and Practices in Psychological Science, 1, 443\u2013490.","journal-title":"Advances in Methods and Practices in Psychological Science"},{"key":"2456_CR44","doi-asserted-by":"publisher","first-page":"1827","DOI":"10.1177\/0956797615616374","volume":"26","author":"DS Lindsay","year":"2015","unstructured":"Lindsay, D. S. (2015). Replication in psychological science. Psychological Science, 26, 1827\u20131832.","journal-title":"Psychological Science"},{"issue":"10","key":"2456_CR45","first-page":"14","volume":"29","author":"DS Lindsay","year":"2016","unstructured":"Lindsay, D. S., Simons, D. J., & Lilienfeld, S. O. (2016). Research preregistration 101. APS Observer, 29(10), 14\u201316.","journal-title":"APS Observer"},{"key":"2456_CR46","doi-asserted-by":"publisher","first-page":"187","DOI":"10.1038\/526187a","volume":"526","author":"R MacCoun","year":"2015","unstructured":"MacCoun, R., & Perlmutter, S. (2015). Hide results to seek the truth. Nature, 526, 187\u2013189.","journal-title":"Nature"},{"key":"2456_CR47","first-page":"297","volume-title":"Psychological science under scrutiny: Recent challenges and proposed solutions","author":"R MacCoun","year":"2017","unstructured":"MacCoun, R., & Perlmutter, S. (2017). Blind analysis as a correction for confirmatory bias in physics and in psychology. In S. O. Lilienfeld & I. Waldman (Eds.), Psychological science under scrutiny: Recent challenges and proposed solutions (pp. 297\u2013321). Hoboken: Wiley."},{"key":"2456_CR48","doi-asserted-by":"publisher","first-page":"160426","DOI":"10.1098\/rsos.160426","volume":"4","author":"M Marsman","year":"2017","unstructured":"Marsman, M., Sch\u00f6nbrodt, F., Morey, R. D., Yao, Y., Gelman, A., & Wagenmakers, E.-J. (2017). A Bayesian bird\u2019s eye view of \u201creplications of important results in social psychology\u201d. Royal Society Open Science, 4, 160426.","journal-title":"Royal Society Open Science"},{"key":"2456_CR49","doi-asserted-by":"publisher","first-page":"e1","DOI":"10.1037\/xge0000038","volume":"144","author":"D Matzke","year":"2015","unstructured":"Matzke, D., Nieuwenhuis, S., van Rijn, H., Slagter, H. A., van der Molen, M. W., & Wagenmakers, E.-J. (2015). The effect of horizontal eye movements on free recall: A preregistered adversarial collaboration. Journal of Experimental Psychology: General, 144, e1\u2013e15.","journal-title":"Journal of Experimental Psychology: General"},{"issue":"2","key":"2456_CR50","doi-asserted-by":"publisher","first-page":"e16","DOI":"10.1037\/xge0000049","volume":"144","author":"A Meyer","year":"2015","unstructured":"Meyer, A., Frederick, S., Burnham, T. C., Guevara Pinto, J. D., Boyer, T. W., Ball, L. J., et al. (2015). Disfluent fonts don\u2019t help people solve math problems. Journal of Experimental Psychology: General, 144(2), e16.","journal-title":"Journal of Experimental Psychology: General"},{"key":"2456_CR51","doi-asserted-by":"publisher","first-page":"240","DOI":"10.1016\/j.cct.2010.11.004","volume":"32","author":"LE Miller","year":"2011","unstructured":"Miller, L. E., & Stewart, M. E. (2011). The blind leading the blind: Use and misuse of blinding in randomized controlled trials. Contemporary Clinical Trials, 32, 240\u2013243.","journal-title":"Contemporary Clinical Trials"},{"key":"2456_CR52","doi-asserted-by":"publisher","first-page":"e1","DOI":"10.1016\/j.jclinepi.2010.03.004","volume":"63","author":"D Moher","year":"2010","unstructured":"Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gtzsche, P. C., Devereaux, P. J., et al. (2010). CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trials. Journal of Clinical Epidemiology, 63, e1\u2013e37.","journal-title":"Journal of Clinical Epidemiology"},{"key":"2456_CR53","doi-asserted-by":"publisher","first-page":"315","DOI":"10.1177\/0956797613511257","volume":"25","author":"J Moher","year":"2014","unstructured":"Moher, J., Lakshmanan, B. M., Egeth, H. E., & Ewen, J. B. (2014). Inhibition drives early feature-based attention. Psychological Science, 25, 315\u2013324.","journal-title":"Psychological Science"},{"key":"2456_CR54","doi-asserted-by":"publisher","first-page":"0021","DOI":"10.1038\/s41562-016-0021","volume":"1","author":"MR Munaf\u00f2","year":"2017","unstructured":"Munaf\u00f2, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., et al. (2017). A manifesto for reproducible science. Nature Human Behaviour, 1, 0021.","journal-title":"Nature Human Behaviour"},{"key":"2456_CR55","doi-asserted-by":"publisher","first-page":"1422","DOI":"10.1126\/science.aab2374","volume":"348","author":"BA Nosek","year":"2015","unstructured":"Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., et al. (2015). Promoting an open research culture. Science, 348, 1422\u20131425.","journal-title":"Science"},{"key":"2456_CR56","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1027\/1864-9335\/a000192","volume":"45","author":"BA Nosek","year":"2014","unstructured":"Nosek, B. A., & Lakens, D. (2014). A method to increase the credibility of published results. Social Psychology, 45, 137\u2013141.","journal-title":"Social Psychology"},{"key":"2456_CR57","doi-asserted-by":"crossref","unstructured":"Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349, p. aac4716.","DOI":"10.1126\/science.aac4716"},{"key":"2456_CR58","doi-asserted-by":"publisher","first-page":"776","DOI":"10.1037\/h0043424","volume":"17","author":"MT Orne","year":"1962","unstructured":"Orne, M. T. (1962). On the social psychology of the psychological experiment: With particular reference to demand characteristics and their implications. American Psychologist, 17, 776\u2013783.","journal-title":"American Psychologist"},{"key":"2456_CR59","first-page":"470","volume":"13","author":"CS Peirce","year":"1878","unstructured":"Peirce, C. S. (1878). Deduction, induction, and hypothesis. Popular Science Monthly, 13, 470\u2013482.","journal-title":"Popular Science Monthly"},{"key":"2456_CR60","first-page":"126","volume-title":"Studies in logic","author":"CS Peirce","year":"1883","unstructured":"Peirce, C. S. (1883). A theory of probable inference. In C. S. Peirce (Ed.), Studies in logic (pp. 126\u2013181). Boston: Little and Brown."},{"key":"2456_CR61","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1038\/nrn.2016.167","volume":"18","author":"RA Poldrack","year":"2017","unstructured":"Poldrack, R. A., Baker, C. I., Durnez, J., Gorgolewski, K. J., Matthews, P. M., Munaf\u00f2, M. R., et al. (2017). Scanning the horizon: Towards transparent and reproducible neuroimag-ing research. Nature Reviews Neuroscience, 18, 115\u2013126.","journal-title":"Nature Reviews Neuroscience"},{"key":"2456_CR62","first-page":"226","volume-title":"Blinding as a solution to bias: Strengthening biomedical science, forensic science, and law","author":"J Resnik","year":"2016","unstructured":"Resnik, J., & Curtis, D. (2016). Why eyes? Cautionary tales from law\u2019s blindfolded justice. In C. T. Robertson & A. S. Kesselheim (Eds.), Blinding as a solution to bias: Strengthening biomedical science, forensic science, and law (pp. 226\u2013247). Amsterdam: Academic Press."},{"key":"2456_CR63","volume-title":"Blinding as a solution to bias: Strengthening biomedical science, forensic science, and law","author":"CT Robertson","year":"2016","unstructured":"Robertson, C. T., & Kesselheim, A. S. (2016). Blinding as a solution to bias: Strengthening biomedical science, forensic science, and law. Amsterdam: Academic Press."},{"key":"2456_CR64","doi-asserted-by":"crossref","unstructured":"Rosenthal, R. (1966). Experimenter effects in behavioral research (pp. 7, 62). Appleton-Century-Crofts.","DOI":"10.2307\/2065929"},{"key":"2456_CR65","doi-asserted-by":"publisher","first-page":"301","DOI":"10.3758\/s13423-014-0595-4","volume":"21","author":"JN Rouder","year":"2014","unstructured":"Rouder, J. N. (2014). Optional stopping: No problem for Bayesians. Psychonomic Bulletin & Review, 21, 301\u2013308.","journal-title":"Psychonomic Bulletin & Review"},{"key":"2456_CR66","doi-asserted-by":"publisher","first-page":"10","DOI":"10.1007\/BF01568704","volume":"31","author":"A Sainz","year":"1957","unstructured":"Sainz, A., Bigelow, N., & Barwise, C. (1957). On a methodology for the clinical evaluation of phrenopraxic drugs. The Psychiatric Quarterly, 31, 10\u201316.","journal-title":"The Psychiatric Quarterly"},{"key":"2456_CR67","doi-asserted-by":"publisher","first-page":"696","DOI":"10.1016\/S0140-6736(02)07816-9","volume":"359","author":"KF Schulz","year":"2002","unstructured":"Schulz, K. F., & Grimes, D. A. (2002). Blinding in randomised trials: Hiding who got what. The Lancet, 359, 696\u2013700.","journal-title":"The Lancet"},{"key":"2456_CR68","doi-asserted-by":"publisher","first-page":"e56515","DOI":"10.1371\/journal.pone.0056515","volume":"8","author":"DR Shanks","year":"2013","unstructured":"Shanks, D. R., Newell, B. R., Lee, E. H., Balakrishnan, D., Ekelund, L., Cenac, Z., et al. (2013). Priming intelligent behavior: An elusive phenomenon. PLoS One, 8, e56515.","journal-title":"PLoS One"},{"key":"2456_CR69","doi-asserted-by":"publisher","first-page":"803","DOI":"10.1111\/j.1467-9280.2007.01983.x","volume":"18","author":"AF Shariff","year":"2007","unstructured":"Shariff, A. F., & Norenzayan, A. (2007). God is watching you: Priming God concepts increases prosocial behavior in an anonymous economic game. Psychological Science, 18, 803\u2013809.","journal-title":"Psychological Science"},{"key":"2456_CR70","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1177\/2515245917747646","volume":"1","author":"R Silberzahn","year":"2018","unstructured":"Silberzahn, R., Uhlmann, E. L., Martin, D. P., Anselmi, P., Aust, F., Awtrey, E., et al. (2018). Many analysts, one data set: Making transparent how variations in analytic choices affect results. Advances in Methods and Practices in Psychological Science, 1, 337\u2013356.","journal-title":"Advances in Methods and Practices in Psychological Science"},{"key":"2456_CR71","doi-asserted-by":"publisher","first-page":"1359","DOI":"10.1177\/0956797611417632","volume":"22","author":"JP Simmons","year":"2011","unstructured":"Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22, 1359\u20131366.","journal-title":"Psychological Science"},{"key":"2456_CR72","doi-asserted-by":"publisher","first-page":"702","DOI":"10.1177\/1745691616658637","volume":"11","author":"S Steegen","year":"2016","unstructured":"Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science, 11, 702\u2013712.","journal-title":"Perspectives on Psychological Science"},{"key":"2456_CR73","doi-asserted-by":"publisher","first-page":"1042","DOI":"10.3758\/s13428-018-01189-8","volume":"51","author":"AM Stefan","year":"2019","unstructured":"Stefan, A. M., Gronau, Q. F., Schonbrodt, F. D., & Wagenmakers, E.-J. (2019). A tutorial on Bayes factor design analysis using an informed prior. Behavior Research Methods, 51, 1042\u20131058.","journal-title":"Behavior Research Methods"},{"key":"2456_CR74","doi-asserted-by":"publisher","first-page":"759","DOI":"10.1177\/0956797615570367","volume":"26","author":"N Unsworth","year":"2015","unstructured":"Unsworth, N., Redick, T. S., McMillan, B. D., Hambrick, D. Z., Kane, M. J., & Engle, R. W. (2015). Is playing video games related to cognitive abilities? Psychological Science, 26, 759\u2013774.","journal-title":"Psychological Science"},{"key":"2456_CR75","doi-asserted-by":"publisher","first-page":"821","DOI":"10.4088\/JCP.12m08321","volume":"74","author":"M van Dongen-Boomsma","year":"2013","unstructured":"van Dongen-Boomsma, M., Vollebregt, M. A., Slaats-Willemse, D., & Buitelaar, J. K. (2013). A randomized placebo-controlled trial of electroencephalographic (EEG) neurofeedback in children with attention-deficit\/hyperactivity disorder. The Journal of Clinical Psychiatry, 74, 821\u2013827.","journal-title":"The Journal of Clinical Psychiatry"},{"key":"2456_CR76","doi-asserted-by":"publisher","first-page":"2","DOI":"10.1016\/j.jesp.2016.03.004","volume":"67","author":"AE van \u2019t Veer","year":"2016","unstructured":"van \u2019t Veer, A. E., & Giner-Sorolla, R. (2016). Pre-registration in social psychology\u2014a discussion and suggested template. Journal of Experimental Social Psychology, 67, 2\u201312.","journal-title":"Journal of Experimental Social Psychology"},{"key":"2456_CR77","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1177\/1745691616674458","volume":"11","author":"E-J Wagenmakers","year":"2016","unstructured":"Wagenmakers, E.-J., Beek, T., Dijkhoff, L., Gronau, Q. F., Acosta, A., Adams, R., et al. (2016). Registered replication report: Strack, Martin, & Stepper (1988). Perspectives on Psychological Science, 11, 917\u2013928.","journal-title":"Perspectives on Psychological Science"},{"key":"2456_CR78","doi-asserted-by":"publisher","first-page":"830","DOI":"10.1037\/0033-295X.114.3.830","volume":"114","author":"E-J Wagenmakers","year":"2007","unstructured":"Wagenmakers, E.-J., & Brown, S. D. (2007). On the linear relation between the mean and the standard deviation of a response time distribution. Psychological Review, 114, 830\u2013841.","journal-title":"Psychological Review"},{"key":"2456_CR79","doi-asserted-by":"publisher","first-page":"627","DOI":"10.1177\/1745691612463078","volume":"7","author":"E-J Wagenmakers","year":"2012","unstructured":"Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012). An agenda for purely confirmatory research. Perspectives on Psychological Science, 7, 627\u2013633.","journal-title":"Perspectives on Psychological Science"},{"key":"2456_CR80","doi-asserted-by":"publisher","first-page":"1100","DOI":"10.1177\/1745691617693393","volume":"12","author":"T Yarkoni","year":"2017","unstructured":"Yarkoni, T., & Westfall, J. (2017). Choosing prediction over explanation in psychology: Lessons from machine learning. Perspectives on Psychological Science, 12, 1100\u20131122.","journal-title":"Perspectives on Psychological Science"}],"container-title":["Synthese"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11229-019-02456-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s11229-019-02456-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s11229-019-02456-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,11,3]],"date-time":"2021-11-03T19:04:14Z","timestamp":1635966254000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s11229-019-02456-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,11,19]]},"references-count":80,"journal-issue":{"issue":"S23","published-print":{"date-parts":[[2021,10]]}},"alternative-id":["2456"],"URL":"https:\/\/doi.org\/10.1007\/s11229-019-02456-7","relation":{},"ISSN":["0039-7857","1573-0964"],"issn-type":[{"value":"0039-7857","type":"print"},{"value":"1573-0964","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,11,19]]},"assertion":[{"value":"10 December 2018","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 October 2019","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"19 November 2019","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}]}}