{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T22:39:06Z","timestamp":1777415946809,"version":"3.51.4"},"reference-count":20,"publisher":"MIT Press","issue":"2","license":[{"start":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T00:00:00Z","timestamp":1701043200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"European Union's Horizon 2020 Research and Innovation Program","award":["101023805"],"award-info":[{"award-number":["101023805"]}]},{"name":"Cambridge-NTU Centre for Lifelong Learning and Individualised Cognition"},{"DOI":"10.13039\/501100001659","name":"Deutsche Forschungsgemeinschaft","doi-asserted-by":"publisher","award":["META-REP: A Meta-scientific Programme to Analyse and Optimise Replicability in the Behavioural, Social, and Cognitive Sciences"],"award-info":[{"award-number":["META-REP: A Meta-scientific Programme to Analyse and Optimise Replicability in the Behavioural, Social, and Cognitive Sciences"]}],"id":[{"id":"10.13039\/501100001659","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004472","name":"Riksbankens Jubileumsfond","doi-asserted-by":"crossref","award":["P21-0384"],"award-info":[{"award-number":["P21-0384"]}],"id":[{"id":"10.13039\/501100004472","id-type":"DOI","asserted-by":"crossref"}]},{"name":"Lundbeck Foundation","award":["R336-2020-1035"],"award-info":[{"award-number":["R336-2020-1035"]}]}],"content-domain":{"domain":["direct.mit.edu"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2024,2,1]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>The ongoing reproducibility crisis in psychology and cognitive neuroscience has sparked increasing calls to re-evaluate and reshape scientific culture and practices. Heeding those calls, we have recently launched the EEGManyPipelines project as a means to assess the robustness of EEG research in naturalistic conditions and experiment with an alternative model of conducting scientific research. One hundred sixty-eight analyst teams, encompassing 396 individual researchers from 37 countries, independently analyzed the same unpublished, representative EEG data set to test the same set of predefined hypotheses and then provided their analysis pipelines and reported outcomes. Here, we lay out how large-scale scientific projects can be set up in a grassroots, community-driven manner without a central organizing laboratory. We explain our recruitment strategy, our guidance for analysts, the eventual outputs of this project, and how it might have a lasting impact on the field.<\/jats:p>","DOI":"10.1162\/jocn_a_02087","type":"journal-article","created":{"date-parts":[[2023,11,27]],"date-time":"2023-11-27T10:38:19Z","timestamp":1701081499000},"page":"217-224","update-policy":"https:\/\/doi.org\/10.1162\/mitpressjournals.corrections.policy","source":"Crossref","is-referenced-by-count":30,"title":["EEGManyPipelines: A Large-scale, Grassroots Multi-analyst Study of Electroencephalography Analysis Practices in the Wild"],"prefix":"10.1162","volume":"36","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-7977-1366","authenticated-orcid":true,"given":"Darinka","family":"Tr\u00fcbutschek","sequence":"first","affiliation":[{"name":"Research Group Neural Circuits, Consciousness and Cognition, Max Planck Institute for Empirical Aesthetics, Frankfurt\/Main, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9089-6020","authenticated-orcid":true,"given":"Yu-Fang","family":"Yang","sequence":"additional","affiliation":[{"name":"Division of Experimental Psychology and Neuropsychology, Department of Education and Psychology, Freie Universit\u00e4t Berlin, Berlin, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3970-9055","authenticated-orcid":true,"given":"Claudia","family":"Gianelli","sequence":"additional","affiliation":[{"name":"Department of Clinical and Experimental Medicine, University of Messina, Messina, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5477-6670","authenticated-orcid":true,"given":"Elena","family":"Cesnaite","sequence":"additional","affiliation":[{"name":"Institute of Psychology, University of M\u00fcnster, M\u00fcnster, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1383-1917","authenticated-orcid":true,"given":"Nastassja L.","family":"Fischer","sequence":"additional","affiliation":[{"name":"Centre for Research and Development in Learning (CRADLE), Nanyang Technological University, Singapore, Singapore"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7954-2886","authenticated-orcid":true,"given":"Mikkel C.","family":"Vinding","sequence":"additional","affiliation":[{"name":"Copenhagen University Hospital - Amager and Hvidovre, Copenhagen, Denmark"},{"name":"Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6628-0218","authenticated-orcid":true,"given":"Tom R.","family":"Marshall","sequence":"additional","affiliation":[{"name":"Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1694-7069","authenticated-orcid":true,"given":"Johannes","family":"Algermissen","sequence":"additional","affiliation":[{"name":"Radboud University, Donders Institute for Brain, Cognition, and Behaviour, Nijmegen, The Netherlands"},{"name":"Department of Experimental Psychology, University of Oxford, Oxford, United Kingdom"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8795-0815","authenticated-orcid":true,"given":"Annalisa","family":"Pascarella","sequence":"additional","affiliation":[{"name":"Institute of Applied Mathematics \u201cM. Picone\u201d, National Council of Research, Rome, Italy"}]},{"given":"Tuomas","family":"Puoliv\u00e4li","sequence":"additional","affiliation":[{"name":"Faculty of Information Technology, University of Jyv\u00e4skyl\u00e4, Finland"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0388-4932","authenticated-orcid":true,"given":"Andrea","family":"Vitale","sequence":"additional","affiliation":[{"name":"Laboratory for Autism and Neurodevelopmental Disorders, Center for Neuroscience and Cognitive Systems @UniTn, Istituto Italiano di Tecnologia, Rovereto, Italy"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4837-0345","authenticated-orcid":true,"given":"Niko A.","family":"Busch","sequence":"additional","affiliation":[{"name":"Institute of Psychology, University of M\u00fcnster, M\u00fcnster, Germany"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5273-0150","authenticated-orcid":true,"given":"Gustav","family":"Nilsonne","sequence":"additional","affiliation":[{"name":"Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden"},{"name":"Department of Psychology, Stockholm University, Stockholm, Sweden"}]}],"member":"281","published-online":{"date-parts":[[2024,2,1]]},"reference":[{"key":"2024011014314091400_bib1","doi-asserted-by":"publisher","first-page":"e72185","DOI":"10.7554\/eLife.72185","article-title":"Consensus-based guidance for conducting and reporting multi-analyst studies","volume":"10","author":"Aczel","year":"2021","journal-title":"eLife"},{"key":"2024011014314091400_bib2","doi-asserted-by":"publisher","DOI":"10.17605\/OSF.IO\/42K5H","volume-title":"EEGManyPipelines: Robustness of EEG results across analysis pipelines","author":"Algermissen","year":"2022"},{"key":"2024011014314091400_bib3","doi-asserted-by":"publisher","first-page":"230235","DOI":"10.1098\/rsos.230235","article-title":"How to build up big team science: A practical guide for large-scale collaborations","volume":"10","author":"Baumgartner","year":"2023","journal-title":"Royal Society Open Science"},{"key":"2024011014314091400_bib4","doi-asserted-by":"publisher","first-page":"20","DOI":"10.1016\/j.brs.2022.12.004","article-title":"T4TE: Team for TMS-EEG to improve reproducibility through an open collaborative initiative","volume":"16","author":"Bortoletto","year":"2023","journal-title":"Brain Stimulation"},{"key":"2024011014314091400_bib5","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1038\/s41586-020-2314-9","article-title":"Variability in the analysis of a single neuroimaging dataset by many teams","volume":"582","author":"Botvinik-Nezer","year":"2020","journal-title":"Nature"},{"key":"2024011014314091400_bib6","doi-asserted-by":"publisher","first-page":"596","DOI":"10.1111\/1469-8986.3750596","article-title":"Short duration power changes in the EEG during recognition memory for words and faces","volume":"37","author":"Burgess","year":"2000","journal-title":"Psychophysiology"},{"key":"2024011014314091400_bib7","doi-asserted-by":"publisher","first-page":"118712","DOI":"10.1016\/j.neuroimage.2021.118712","article-title":"The data-processing multiverse of event-related potentials (ERPs): A roadmap for the optimization and standardization of ERP processing and reduction pipelines","volume":"245","author":"Clayson","year":"2021","journal-title":"Neuroimage"},{"key":"2024011014314091400_bib8","doi-asserted-by":"publisher","first-page":"505","DOI":"10.1038\/d41586-022-00150-2","article-title":"Build up big-team science","volume":"601","author":"Coles","year":"2022","journal-title":"Nature"},{"key":"2024011014314091400_bib9","doi-asserted-by":"publisher","first-page":"136","DOI":"10.1111\/j.1469-8986.1990.tb00365.x","article-title":"Cognitive event-related potential components during continuous recognition memory for pictures","volume":"27","author":"Friedman","year":"1990","journal-title":"Psychophysiology"},{"key":"2024011014314091400_bib19","doi-asserted-by":"publisher","first-page":"1213","DOI":"10.1016\/j.neuron.2017.12.013","article-title":"An international laboratory for systems and computational neuroscience","volume":"96","author":"International Brain Laboratory","year":"2017","journal-title":"Neuron"},{"key":"2024011014314091400_bib10","doi-asserted-by":"publisher","first-page":"501","DOI":"10.1177\/2515245918797607","article-title":"The Psychological Science Accelerator: Advancing psychology through a distributed collaborative network","volume":"1","author":"Moshontz","year":"2018","journal-title":"Advances in Methods and Practices in Psychological Science"},{"key":"2024011014314091400_bib11","doi-asserted-by":"publisher","first-page":"aac4716","DOI":"10.1126\/science.aac4716","article-title":"Estimating the reproducibility of psychological science","volume":"349","author":"Open Science Collaboration","year":"2015","journal-title":"Science"},{"key":"2024011014314091400_bib12","doi-asserted-by":"publisher","first-page":"1","DOI":"10.5964\/ps.7177","article-title":"The methodology and dataset of the coscience EEG-personality project\u2014A large-scale, multi-laboratory project grounded in cooperative forking paths analysis","volume":"3","author":"Paul","year":"2022","journal-title":"Personality Science"},{"key":"2024011014314091400_bib13","doi-asserted-by":"publisher","first-page":"213","DOI":"10.1016\/j.cortex.2021.03.013","article-title":"#EEGManyLabs: Investigating the replicability of influential EEG experiments","volume":"144","author":"Pavlov","year":"2021","journal-title":"Cortex"},{"key":"2024011014314091400_bib14","doi-asserted-by":"publisher","first-page":"103","DOI":"10.1038\/s41597-019-0104-8","article-title":"EEG-BIDS, an extension to the brain imaging data structure for electroencephalography","volume":"6","author":"Pernet","year":"2019","journal-title":"Scientific Data"},{"key":"2024011014314091400_bib16","doi-asserted-by":"publisher","first-page":"577","DOI":"10.1007\/s11065-021-09513-4","article-title":"How to do better N400 studies: Reproducibility, consistency and adherence to research standards in the existing literature","volume":"32","author":"\u0160o\u0161ki\u0107","year":"2022","journal-title":"Neuropsychology Review"},{"key":"2024011014314091400_bib17","doi-asserted-by":"publisher","first-page":"702","DOI":"10.1177\/1745691616658637","article-title":"Increasing transparency through a multiverse analysis","volume":"11","author":"Steegen","year":"2016","journal-title":"Perspectives on Psychological Science"},{"key":"2024011014314091400_bib18","doi-asserted-by":"publisher","first-page":"e2000797","DOI":"10.1371\/journal.pbio.2000797","article-title":"Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature","volume":"15","author":"Szucs","year":"2017","journal-title":"PLoS Biology"},{"key":"2024011014314091400_bib20","doi-asserted-by":"publisher","first-page":"952","DOI":"10.1016\/j.neuroimage.2005.03.003","article-title":"Changes in brain electrical activity during extended continuous word recognition","volume":"26","author":"Van Strien","year":"2005","journal-title":"Neuroimage"},{"key":"2024011014314091400_bib21","doi-asserted-by":"publisher","first-page":"423","DOI":"10.1038\/d41586-022-01332-8","article-title":"One statistical analysis must not rule them all","volume":"605","author":"Wagenmakers","year":"2022","journal-title":"Nature"}],"container-title":["Journal of Cognitive Neuroscience"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/36\/2\/217\/2208612\/jocn_a_02087.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/direct.mit.edu\/jocn\/article-pdf\/36\/2\/217\/2208612\/jocn_a_02087.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,1,10]],"date-time":"2024-01-10T09:31:56Z","timestamp":1704879116000},"score":1,"resource":{"primary":{"URL":"https:\/\/direct.mit.edu\/jocn\/article\/36\/2\/217\/118308\/EEGManyPipelines-A-Large-scale-Grassroots-Multi"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024]]},"references-count":20,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,2,1]]},"published-print":{"date-parts":[[2024,2,1]]}},"URL":"https:\/\/doi.org\/10.1162\/jocn_a_02087","relation":{"has-preprint":[{"id-type":"doi","id":"10.31222\/osf.io\/jq342","asserted-by":"object"}]},"ISSN":["0898-929X","1530-8898"],"issn-type":[{"value":"0898-929X","type":"print"},{"value":"1530-8898","type":"electronic"}],"subject":[],"published-other":{"date-parts":[[2024]]},"published":{"date-parts":[[2024]]}}}