Breaking the circularity in circular analyses: Simulations and formal treatment of the flattened average approach
| Autor: | Alexia Zoumpoulaki, Howard Bowman, Omid Hajilou, Joseph L. Brooks, Vladimir Litvak |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2020 |
| Předmět: |
Computer science
Physiology Inference Event-Related Potentials 0302 clinical medicine Medicine and Health Sciences Power Distribution Biology (General) Statistical Data Clinical Neurophysiology Brain Mapping Ecology Simulation and Modeling Experimental Design 05 social sciences Statistics Contrast (statistics) Electroencephalography Research Assessment Electrophysiology Bioassays and Physiological Analysis Computational Theory and Mathematics Brain Electrophysiology Research Design Modeling and Simulation Physical Sciences Engineering and Technology Anatomy Algorithm Type I and type II errors Research Article QA75 Power Grids QH301-705.5 Imaging Techniques BF Neurophysiology Context (language use) Neuroimaging Mathematical proof Research and Analysis Methods 050105 experimental psychology Statistical power 03 medical and health sciences Cellular and Molecular Neuroscience Genetics 0501 psychology and cognitive sciences Molecular Biology Ecology Evolution Behavior and Systematics Research Errors Replication crisis Scalp Electrophysiological Techniques Biology and Life Sciences Computational Biology Reproducibility of Results Models Theoretical Energy and Power Multiple comparisons problem RC0321 Clinical Medicine Head 030217 neurology & neurosurgery Mathematics Neuroscience |
| Zdroj: | PLoS Computational Biology PLoS Computational Biology, Vol 16, Iss 11, p e1008286 (2020) |
| ISSN: | 1553-7358 1553-734X |
| Popis: | There has been considerable debate and concern as to whether there is a replication crisis in the scientific literature. A likely cause of poor replication is the multiple comparisons problem. An important way in which this problem can manifest in the M/EEG context is through post hoc tailoring of analysis windows (a.k.a. regions-of-interest, ROIs) to landmarks in the collected data. Post hoc tailoring of ROIs is used because it allows researchers to adapt to inter-experiment variability and discover novel differences that fall outside of windows defined by prior precedent, thereby reducing Type II errors. However, this approach can dramatically inflate Type I error rates. One way to avoid this problem is to tailor windows according to a contrast that is orthogonal (strictly parametrically orthogonal) to the contrast being tested. A key approach of this kind is to identify windows on a fully flattened average. On the basis of simulations, this approach has been argued to be safe for post hoc tailoring of analysis windows under many conditions. Here, we present further simulations and mathematical proofs to show exactly why the Fully Flattened Average approach is unbiased, providing a formal grounding to the approach, clarifying the limits of its applicability and resolving published misconceptions about the method. We also provide a statistical power analysis, which shows that, in specific contexts, the fully flattened average approach provides higher statistical power than Fieldtrip cluster inference. This suggests that the Fully Flattened Average approach will enable researchers to identify more effects from their data without incurring an inflation of the false positive rate. Author summary It is clear from recent replicability studies that the replication rate in psychology and cognitive neuroscience is not high. One reason for this is that the noise in high dimensional neuroimaging data sets can “look-like” signal. A classic manifestation would be selecting a region in the data volume where an effect is biggest and then specifically reporting results on that region. There is a key trade-off in the selection of such regions of interest: liberal selection will inflate false positive rates, but conservative selection (e.g. strictly on the basis of prior precedent in the literature) can reduce statistical power, causing real effects to be missed. We propose a means to reconcile these two possibilities, by which regions of interest can be tailored to the pattern in the collected data, while not inflating false-positive rates. This is based upon generating what we call the Flattened Average. Critically, we validate the correctness of this method both in (ground-truth) simulations and with formal mathematical proofs. Given the replication “crisis”, there may be no more important issue in psychology and cognitive neuroscience than improving the application of methods. This paper makes a valuable contribution to this improvement. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
| Nepřihlášeným uživatelům se plný text nezobrazuje | K zobrazení výsledku je třeba se přihlásit. |