{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,15]],"date-time":"2026-01-15T12:38:36Z","timestamp":1768480716195,"version":"3.49.0"},"update-to":[{"DOI":"10.1371\/journal.pcbi.1008286","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2020,12,7]],"date-time":"2020-12-07T00:00:00Z","timestamp":1607299200000}}],"reference-count":27,"publisher":"Public Library of Science (PLoS)","issue":"11","license":[{"start":{"date-parts":[[2020,11,23]],"date-time":"2020-11-23T00:00:00Z","timestamp":1606089600000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["www.ploscompbiol.org"],"crossmark-restriction":false},"short-container-title":["PLoS Comput Biol"],"abstract":"<jats:p>There has been considerable debate and concern as to whether there is a replication crisis in the scientific literature. A likely cause of poor replication is the multiple comparisons problem. An important way in which this problem can manifest in the M\/EEG context is through post hoc tailoring of analysis windows (a.k.a. regions-of-interest, ROIs) to landmarks in the collected data. Post hoc tailoring of ROIs is used because it allows researchers to adapt to inter-experiment variability and discover novel differences that fall outside of windows defined by prior precedent, thereby reducing Type II errors. However, this approach can dramatically inflate Type I error rates. One way to avoid this problem is to tailor windows according to a contrast that is orthogonal (strictly parametrically orthogonal) to the contrast being tested. A key approach of this kind is to identify windows on a <jats:italic>fully flattened<\/jats:italic> average. On the basis of simulations, this approach has been argued to be safe for post hoc tailoring of analysis windows under many conditions. Here, we present further simulations and mathematical proofs to show exactly why the Fully Flattened Average approach is unbiased, providing a formal grounding to the approach, clarifying the limits of its applicability and resolving published misconceptions about the method. We also provide a statistical power analysis, which shows that, in specific contexts, the fully flattened average approach provides higher statistical power than Fieldtrip cluster inference. This suggests that the Fully Flattened Average approach will enable researchers to identify more effects from their data without incurring an inflation of the false positive rate.<\/jats:p>","DOI":"10.1371\/journal.pcbi.1008286","type":"journal-article","created":{"date-parts":[[2020,11,23]],"date-time":"2020-11-23T19:53:10Z","timestamp":1606161190000},"page":"e1008286","update-policy":"https:\/\/doi.org\/10.1371\/journal.pcbi.corrections_policy","source":"Crossref","is-referenced-by-count":22,"title":["Breaking the circularity in circular analyses: Simulations and formal treatment of the flattened average approach"],"prefix":"10.1371","volume":"16","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-4736-1869","authenticated-orcid":true,"given":"Howard","family":"Bowman","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5364-3611","authenticated-orcid":true,"given":"Joseph L.","family":"Brooks","sequence":"additional","affiliation":[]},{"given":"Omid","family":"Hajilou","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0810-0319","authenticated-orcid":true,"given":"Alexia","family":"Zoumpoulaki","sequence":"additional","affiliation":[]},{"given":"Vladimir","family":"Litvak","sequence":"additional","affiliation":[]}],"member":"340","published-online":{"date-parts":[[2020,11,23]]},"reference":[{"issue":"9","key":"pcbi.1008286.ref001","doi-asserted-by":"crossref","first-page":"1105","DOI":"10.1038\/nn.2886","article-title":"Erroneous analyses of interactions in neuroscience: a problem of significance","volume":"14","author":"S Nieuwenhuis","year":"2011","journal-title":"Nature neuroscience"},{"issue":"3","key":"pcbi.1008286.ref002","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1111\/j.1745-6924.2009.01125.x","article-title":"Puzzlingly high correlations in fMRI studies of emotion, personality, and social cognition","volume":"4","author":"E Vul","year":"2009","journal-title":"Perspectives on psychological science"},{"issue":"Suppl 1","key":"pcbi.1008286.ref003","doi-asserted-by":"crossref","first-page":"S125","DOI":"10.1016\/S1053-8119(09)71202-9","article-title":"Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: An argument for multiple comparisons correction","volume":"47","author":"CM Bennett","year":"2009","journal-title":"Neuroimage"},{"issue":"6251","key":"pcbi.1008286.ref004","doi-asserted-by":"crossref","first-page":"aac4716","DOI":"10.1126\/science.aac4716","article-title":"Estimating the reproducibility of psychological science","volume":"349","author":"Open Science Collaboration","year":"2015","journal-title":"Science"},{"issue":"5","key":"pcbi.1008286.ref005","doi-asserted-by":"crossref","first-page":"535","DOI":"10.1038\/nn.2303","article-title":"Circular analysis in systems neuroscience: the dangers of double dipping","volume":"12","author":"N Kriegeskorte","year":"2009","journal-title":"Nature neuroscience"},{"key":"pcbi.1008286.ref006","first-page":"201602413","article-title":"Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates","author":"A Eklund","year":"2016","journal-title":"Proceedings of the National Academy of Sciences"},{"issue":"1","key":"pcbi.1008286.ref007","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1111\/psyp.12639","article-title":"How to get statistically significant effects in any ERP experiment (and why you shouldn't)","volume":"54","author":"SJ Luck","year":"2017","journal-title":"Psychophysiology"},{"issue":"1","key":"pcbi.1008286.ref008","doi-asserted-by":"crossref","first-page":"100","DOI":"10.1111\/psyp.12682","article-title":"Data-driven region-of-interest selection without inflating Type I error rate","volume":"54","author":"JL Brooks","year":"2017","journal-title":"Psychophysiology"},{"key":"pcbi.1008286.ref009","article-title":"I tried a bunch of things: The dangers of unexpected overfitting in classification of brain data","author":"M Hosseini","year":"2020","journal-title":"Neuroscience & Biobehavioral Reviews"},{"key":"pcbi.1008286.ref010","doi-asserted-by":"crossref","first-page":"101","DOI":"10.1016\/j.neuropsychologia.2018.03.014","article-title":"The impact of sample size on the reproducibility of voxel-based lesion-deficit mappings","volume":"115","author":"DL Lorca-Puls","year":"2018","journal-title":"Neuropsychologia"},{"issue":"1","key":"pcbi.1008286.ref011","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0054258","article-title":"Subliminal salience search illustrated: EEG identity and deception detection on the fringe of awareness","volume":"8","author":"H Bowman","year":"2013","journal-title":"PLoS One"},{"issue":"1","key":"pcbi.1008286.ref012","doi-asserted-by":"crossref","first-page":"4","DOI":"10.3934\/Neuroscience.2014.1.4","article-title":"Instead of \u201cplaying the game\" it is time to change the rules: Registered Reports at AIMS Neuroscience and beyond","volume":"1","author":"CD Chambers","year":"2014","journal-title":"AIMS Neuroscience"},{"issue":"3","key":"pcbi.1008286.ref013","doi-asserted-by":"crossref","DOI":"10.1371\/journal.pone.0090595","article-title":"Countering countermeasures: detecting identity lies by detecting conscious breakthrough","volume":"9","author":"H Bowman","year":"2014","journal-title":"PloS one"},{"issue":"4","key":"pcbi.1008286.ref014","doi-asserted-by":"crossref","first-page":"1077","DOI":"10.1016\/j.neuroimage.2005.08.012","article-title":"A critique of functional localisers","volume":"30","author":"KJ Friston","year":"2006","journal-title":"Neuroimage"},{"key":"pcbi.1008286.ref015","author":"WD Penny","year":"2011"},{"issue":"1","key":"pcbi.1008286.ref016","doi-asserted-by":"crossref","first-page":"177","DOI":"10.1016\/j.jneumeth.2007.03.024","article-title":"Nonparametric statistical testing of EEG-and MEG-data","volume":"164","author":"E Maris","year":"2007","journal-title":"Journal of neuroscience methods"},{"key":"pcbi.1008286.ref017","first-page":"2011","article-title":"FieldTrip: open source software for advanced analysis of MEG, EEG, and invasive electrophysiological data","author":"R Oostenveld","year":"2011","journal-title":"Computational intelligence and neuroscience"},{"issue":"27","key":"pcbi.1008286.ref018","doi-asserted-by":"crossref","first-page":"11194","DOI":"10.1523\/JNEUROSCI.0114-13.2013","article-title":"Kochen, S, Ib\u00e1\u00f1ez, A, Bekinschtein TA. Expectation and attention in hierarchical auditory prediction","volume":"33","author":"S Chennu","year":"2013","journal-title":"Journal of Neuroscience"},{"key":"pcbi.1008286.ref019","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1155\/2011\/831409","article-title":"LIMO EEG: a toolbox for hierarchical LInear MOdeling of ElectroEncephaloGraphic data","volume":"2011","author":"CR Pernet","year":"2011","journal-title":"Computational intelligence and neuroscience"},{"key":"pcbi.1008286.ref020","author":"SJ Luck","year":"2014"},{"issue":"6","key":"pcbi.1008286.ref021","doi-asserted-by":"crossref","first-page":"822","DOI":"10.1111\/j.1469-8986.2004.00239.x","article-title":"Detection of synchronized oscillations in the electroencephalogram: an evaluation of methods","volume":"41","author":"N Yeung","year":"2004","journal-title":"Psychophysiology"},{"issue":"12","key":"pcbi.1008286.ref022","doi-asserted-by":"crossref","first-page":"1559","DOI":"10.1111\/psyp.12521","article-title":"Latency as a region contrast: Measuring ERP latency differences with dynamic time warping","volume":"52","author":"A Zoumpoulaki","year":"2015","journal-title":"Psychophysiology"},{"key":"pcbi.1008286.ref023","first-page":"1","article-title":"Parameter orthogonality and approximate conditional inference","author":"DR Cox","year":"1987","journal-title":"Journal of the Royal Statistical Society"},{"key":"pcbi.1008286.ref024","article-title":"Circularity Revisited: Valid Same-Data Selection and Analysis","volume":"2010","author":"GR Ridgway","year":"2010","journal-title":"Human Brain Mapping"},{"issue":"2","key":"pcbi.1008286.ref025","first-page":"74","article-title":"Dependence of F ratios sharing a common denominator mean square","volume":"30","author":"RT Hurlburt","year":"1976","journal-title":"The American Statistician"},{"key":"pcbi.1008286.ref026","first-page":"455030","article-title":"Countering Cross-Individual Variance in Event Related Potentials with Functional Profiling","author":"A Alsufyani","year":"2018","journal-title":"bioRxiv"},{"issue":"4","key":"pcbi.1008286.ref027","doi-asserted-by":"crossref","first-page":"1088","DOI":"10.1016\/j.neuroimage.2005.12.062","article-title":"Divide and conquer: a defense of functional localizers","volume":"30","author":"R Saxe","year":"2006","journal-title":"Neuroimage"}],"updated-by":[{"DOI":"10.1371\/journal.pcbi.1008286","type":"new_version","label":"New version","source":"publisher","updated":{"date-parts":[[2020,12,7]],"date-time":"2020-12-07T00:00:00Z","timestamp":1607299200000}}],"container-title":["PLOS Computational Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1008286","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2020,12,7]],"date-time":"2020-12-07T23:33:06Z","timestamp":1607383986000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pcbi.1008286"}},"subtitle":[],"editor":[{"given":"Leyla","family":"Isik","sequence":"first","affiliation":[]}],"short-title":[],"issued":{"date-parts":[[2020,11,23]]},"references-count":27,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2020,11,23]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pcbi.1008286","relation":{"new_version":[{"id-type":"doi","id":"10.1371\/journal.pcbi.1008286","asserted-by":"object"}]},"ISSN":["1553-7358"],"issn-type":[{"value":"1553-7358","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,11,23]]}}}