Variability in the analysis of a single neuroimaging dataset by many teams.

Autor: Botvinik-Nezer R; Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel.; Department of Neurobiology, The George S. Wise Faculty of Life Sciences, Tel Aviv University, Tel Aviv, Israel.; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA., Holzmeister F; Department of Banking and Finance, University of Innsbruck, Innsbruck, Austria., Camerer CF; HSS and CNS, California Institute of Technology, Pasadena, CA, USA., Dreber A; Department of Economics, Stockholm School of Economics, Stockholm, Sweden.; Department of Economics, University of Innsbruck, Innsbruck, Austria., Huber J; Department of Banking and Finance, University of Innsbruck, Innsbruck, Austria., Johannesson M; Department of Economics, Stockholm School of Economics, Stockholm, Sweden., Kirchler M; Department of Banking and Finance, University of Innsbruck, Innsbruck, Austria., Iwanir R; Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel.; Department of Neurobiology, The George S. Wise Faculty of Life Sciences, Tel Aviv University, Tel Aviv, Israel., Mumford JA; Center for Healthy Minds, University of Wisconsin-Madison, Madison, WI, USA., Adcock RA; Center for Cognitive Neuroscience, Duke University, Durham, NC, USA.; Department of Psychiatry and Behavioral Sciences, Duke University, Durham, NC, USA., Avesani P; Neuroinformatics Laboratory, Fondazione Bruno Kessler, Trento, Italy.; Center for Mind/Brain Sciences - CIMeC, University of Trento, Rovereto, Italy., Baczkowski BM; Department of Neurology, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany., Bajracharya A; Department of Otolaryngology, Washington University in St. Louis, St. Louis, MO, USA., Bakst L; Department of Psychological and Brain Sciences, Boston University, Boston, MA, USA.; Center for Systems Neuroscience, Boston University, Boston, MA, USA., Ball S; Department of Economics, Virginia Tech, Blacksburg, VA, USA.; School of Neuroscience, Virginia Tech, Blacksburg, VA, USA., Barilari M; Crossmodal Perception and Plasticity Laboratory, Institutes for Research in Psychology (IPSY) and Neurosciences (IoNS), UCLouvain, Louvain-la-Neuve, Belgium., Bault N; School of Psychology, University of Plymouth, Plymouth, UK., Beaton D; Rotman Research Institute, Baycrest Health Sciences Centre, Toronto, Ontario, Canada., Beitner J; Department of Psychology, University of Amsterdam, Amsterdam, The Netherlands.; Department of Psychology, Goethe University, Frankfurt am Main, Germany., Benoit RG; Max Planck Research Group: Adaptive Memory, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany., Berkers RMWJ; Max Planck Research Group: Adaptive Memory, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany., Bhanji JP; Department of Psychology, Rutgers University-Newark, Newark, NJ, USA., Biswal BB; Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ, USA.; School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China., Bobadilla-Suarez S; Department of Experimental Psychology, University College London, London, UK., Bortolini T; D'Or Institute for Research and Education (IDOR), Rio de Janeiro, Brazil., Bottenhorn KL; Department of Psychology, Florida International University, Miami, FL, USA., Bowring A; Oxford Big Data Institute, Li Ka Shing Centre for Health Information and Discovery, Nuffield Department of Population Health, University of Oxford, Oxford, UK., Braem S; Department of Experimental Psychology, Ghent University, Ghent, Belgium.; Department of Psychology, Vrije Universiteit Brussel, Brussels, Belgium., Brooks HR; Department of Psychology, University of Denver, Denver, CO, USA., Brudner EG; Department of Psychology, Rutgers University-Newark, Newark, NJ, USA., Calderon CB; Department of Experimental Psychology, Ghent University, Ghent, Belgium., Camilleri JA; Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany., Castrellon JJ; Center for Cognitive Neuroscience, Duke University, Durham, NC, USA.; Department of Psychology and Neuroscience, Duke University, Durham, NC, USA., Cecchetti L; MoMiLab Research Unit, IMT School for Advanced Studies Lucca, Lucca, Italy., Cieslik EC; Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany., Cole ZJ; Department of Psychology, University of Nebraska-Lincoln, Lincoln, NE, USA., Collignon O; Center for Mind/Brain Sciences - CIMeC, University of Trento, Rovereto, Italy.; Crossmodal Perception and Plasticity Laboratory, Institutes for Research in Psychology (IPSY) and Neurosciences (IoNS), UCLouvain, Louvain-la-Neuve, Belgium., Cox RW; National Institute of Mental Health (NIMH), National Institutes of Health, Bethesda, MD, USA., Cunningham WA; Department of Psychology, University of Toronto, Toronto, Ontario, Canada., Czoschke S; Institute of Medical Psychology, Goethe University, Frankfurt am Main, Germany., Dadi K; Inria, CEA, Université Paris-Saclay, Palaiseau, France., Davis CP; Department of Psychological Sciences, University of Connecticut, Storrs, CT, USA.; Brain Imaging Research Center, University of Connecticut, Storrs, CT, USA.; Connecticut Institute for the Brain and Cognitive Sciences, University of Connecticut, Storrs, CT, USA., Luca A; PROVIDI Lab, Image Sciences Institute, University Medical Center Utrecht, Utrecht, The Netherlands., Delgado MR; Department of Psychology, Rutgers University-Newark, Newark, NJ, USA., Demetriou L; Section of Endocrinology and Investigative Medicine, Faculty of Medicine, Imperial College London, London, UK.; Nuffield Department of Women's and Reproductive Health, University of Oxford, Oxford, UK., Dennison JB; Department of Psychology, Temple University, Philadelphia, PA, USA., Di X; Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ, USA.; School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China., Dickie EW; Krembil Centre for Neuroinformatics, Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Ontario, Canada.; Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada., Dobryakova E; Center for Traumatic Brain Injury Research, Kessler Foundation, East Hanover, NJ, USA., Donnat CL; Department of Statistics, Stanford University, Stanford, CA, USA., Dukart J; Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany., Duncan NW; Graduate Institute of Mind, Brain and Consciousness, Taipei Medical University, Taipei, Taiwan.; Brain and Consciousness Research Centre, TMU-ShuangHo Hospital, New Taipei City, Taiwan., Durnez J; Department of Psychology and Stanford Center for Reproducible Neuroscience, Stanford University, Stanford, CA, USA., Eed A; Instituto de Neurociencias, CSIC-UMH, Alicante, Spain., Eickhoff SB; Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany., Erhart A; Department of Psychology, University of Denver, Denver, CO, USA., Fontanesi L; Faculty of Psychology, University of Basel, Basel, Switzerland., Fricke GM; Computer Science Department, University of New Mexico, Albuquerque, NM, USA., Fu S; School of Management, Zhejiang University of Technology, Hangzhou, China.; Institute of Neuromanagement, Zhejiang University of Technology, Hangzhou, China., Galván A; Department of Psychology, University of California Los Angeles, Los Angeles, CA, USA., Gau R; Crossmodal Perception and Plasticity Laboratory, Institutes for Research in Psychology (IPSY) and Neurosciences (IoNS), UCLouvain, Louvain-la-Neuve, Belgium., Genon S; Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany., Glatard T; Department of Computer Science and Software Engineering, Concordia University, Montreal, Quebec, Canada., Glerean E; Department of Neuroscience and Biomedical Engineering, Aalto University, Espoo, Finland., Goeman JJ; Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden, The Netherlands., Golowin SAE; Graduate Institute of Mind, Brain and Consciousness, Taipei Medical University, Taipei, Taiwan., González-García C; Department of Experimental Psychology, Ghent University, Ghent, Belgium., Gorgolewski KJ; Department of Psychology, Stanford University, Stanford, CA, USA., Grady CL; Rotman Research Institute, Baycrest Health Sciences Centre, Toronto, Ontario, Canada., Green MA; Center for Cognitive Neuroscience, Duke University, Durham, NC, USA.; Department of Psychology and Neuroscience, Duke University, Durham, NC, USA., Guassi Moreira JF; Department of Psychology, University of California Los Angeles, Los Angeles, CA, USA., Guest O; Department of Experimental Psychology, University College London, London, UK.; Research Centre on Interactive Media, Smart Systems and Emerging Technologies - RISE, Nicosia, Cyprus., Hakimi S; Center for Cognitive Neuroscience, Duke University, Durham, NC, USA., Hamilton JP; Center for Social and Affective Neuroscience, Department of Biomedical and Clinical Sciences, Linköping University, Linköping, Sweden., Hancock R; Brain Imaging Research Center, University of Connecticut, Storrs, CT, USA.; Connecticut Institute for the Brain and Cognitive Sciences, University of Connecticut, Storrs, CT, USA., Handjaras G; MoMiLab Research Unit, IMT School for Advanced Studies Lucca, Lucca, Italy., Harry BB; The MARCS Institute for Brain, Behaviour and Development, Western Sydney University, Sydney, New South Wales, Australia., Hawco C; Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Ontario, Canada., Herholz P; McConnell Brain Imaging Centre, The Neuro (Montreal Neurological Institute-Hospital), Faculty of Medicine, McGill University, Montreal, Quebec, Canada., Herman G; Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Ontario, Canada., Heunis S; Department of Electrical Engineering, Eindhoven University of Technology, Eindhoven, The Netherlands.; Department of Research and Development, Epilepsy Centre Kempenhaeghe, Heeze, The Netherlands., Hoffstaedter F; Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany., Hogeveen J; Department of Psychology, University of New Mexico, Albuquerque, NM, USA.; Psychology Clinical Neuroscience Center, University of New Mexico, Albuquerque, NM, USA., Holmes S; Department of Statistics, Stanford University, Stanford, CA, USA., Hu CP; Leibniz-Institut für Resilienzforschung (LIR), Mainz, Germany., Huettel SA; Department of Psychology and Neuroscience, Duke University, Durham, NC, USA., Hughes ME; School of Health Sciences, Swinburne University of Technology, Hawthorn, Victoria, Australia., Iacovella V; Center for Mind/Brain Sciences - CIMeC, University of Trento, Rovereto, Italy., Iordan AD; Department of Psychology, University of Michigan, Ann Arbor, MI, USA., Isager PM; Department of Industrial Engineering and Innovation Sciences, Eindhoven University of Technology, Eindhoven, The Netherlands., Isik AI; Department of Neuroscience, Max Planck Institute for Empirical Aesthetics, Frankfurt am Main, Germany., Jahn A; fMRI Laboratory, University of Michigan, Ann Arbor, MI, USA., Johnson MR; Department of Psychology, University of Nebraska-Lincoln, Lincoln, NE, USA.; Center for Brain, Biology and Behavior, University of Nebraska-Lincoln, Lincoln, NE, USA., Johnstone T; School of Health Sciences, Swinburne University of Technology, Hawthorn, Victoria, Australia., Joseph MJE; Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, Ontario, Canada., Juliano AC; Center for Neuropsychology and Neuroscience Research, Kessler Foundation, East Hanover, NJ, USA., Kable JW; Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA.; MindCORE, University of Pennsylvania, Philadelphia, PA, USA., Kassinopoulos M; Graduate Program in Biological and Biomedical Engineering, McGill University, Montreal, Quebec, Canada., Koba C; MoMiLab Research Unit, IMT School for Advanced Studies Lucca, Lucca, Italy., Kong XZ; Language and Genetics Department, Max Planck Institute for Psycholinguistics, Nijmegen, The Netherlands., Koscik TR; Department of Psychiatry, University of Iowa Carver College of Medicine, Iowa City, IA, USA., Kucukboyaci NE; Center for Traumatic Brain Injury Research, Kessler Foundation, East Hanover, NJ, USA.; Department of Physical Medicine and Rehabilitation, Rutgers New Jersey Medical School, Newark, NJ, USA., Kuhl BA; Department of Psychology, University of Oregon, Eugene, OR, USA., Kupek S; Faculty of Economics and Statistics, University of Innsbruck, Innsbruck, Austria., Laird AR; Department of Physics, Florida International University, Miami, Florida, USA., Lamm C; Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria.; Vienna Cognitive Science Hub, University of Vienna, Vienna, Austria., Langner R; Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany., Lauharatanahirun N; US CCDC Army Research Laboratory, Human Research and Engineering Directorate, Aberdeen Proving Ground, MD, USA.; Annenberg School for Communication, University of Pennsylvania, Philadelphia, PA, USA., Lee H; Department of Psychological and Brain Sciences, Johns Hopkins University, Baltimore, MD, USA., Lee S; Department of Psychology, University of Pennsylvania, Philadelphia, PA, USA., Leemans A; PROVIDI Lab, Image Sciences Institute, University Medical Center Utrecht, Utrecht, The Netherlands., Leo A; MoMiLab Research Unit, IMT School for Advanced Studies Lucca, Lucca, Italy., Lesage E; Department of Experimental Psychology, Ghent University, Ghent, Belgium., Li F; Fralin Biomedical Research Institute, Roanoke, VA, USA.; Economics Experimental Lab, Nanjing Audit University, Nanjing, China., Li MYC; Department of Psychological Sciences, University of Connecticut, Storrs, CT, USA.; Brain Imaging Research Center, University of Connecticut, Storrs, CT, USA.; Connecticut Institute for the Brain and Cognitive Sciences, University of Connecticut, Storrs, CT, USA.; Haskins Laboratories, New Haven, CT, USA., Lim PC; Department of Psychology, University of Nebraska-Lincoln, Lincoln, NE, USA.; Center for Brain, Biology and Behavior, University of Nebraska-Lincoln, Lincoln, NE, USA., Lintz EN; Department of Psychology, University of Nebraska-Lincoln, Lincoln, NE, USA., Liphardt SW; Biology Department, University of New Mexico, Albuquerque, NM, USA., Losecaat Vermeer AB; Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria., Love BC; Department of Experimental Psychology, University College London, London, UK.; The Alan Turing Institute, London, UK., Mack ML; Department of Psychology, University of Toronto, Toronto, Ontario, Canada., Malpica N; Laboratorio de Análisis de Imagen Médica y Biometría (LAIMBIO), Universidad Rey Juan Carlos, Madrid, Spain., Marins T; D'Or Institute for Research and Education (IDOR), Rio de Janeiro, Brazil., Maumet C; Inria, Univ Rennes, CNRS, Inserm, IRISA UMR 6074, Empenn ERL U 1228, Rennes, France., McDonald K; Department of Psychology and Neuroscience, Duke University, Durham, NC, USA., McGuire JT; Department of Psychological and Brain Sciences, Boston University, Boston, MA, USA.; Center for Systems Neuroscience, Boston University, Boston, MA, USA., Melero H; Laboratorio de Análisis de Imagen Médica y Biometría (LAIMBIO), Universidad Rey Juan Carlos, Madrid, Spain.; Departamento de Psicobiología, División de Psicología, CES Cardenal Cisneros, Madrid, Spain.; Northeastern University Biomedical Imaging Center, Northeastern University, Boston, MA, USA., Méndez Leal AS; Department of Psychology, University of California Los Angeles, Los Angeles, CA, USA., Meyer B; Leibniz-Institut für Resilienzforschung (LIR), Mainz, Germany.; Neuroimaging Center (NIC), Focus Program Translational Neurosciences (FTN), Johannes Gutenberg University Medical Center Mainz, Mainz, Germany., Meyer KN; Department of Psychology and Neuroscience, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA., Mihai G; Max Planck Research Group: Neural Mechanisms of Human Communication, Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.; Chair of Cognitive and Clinical Neuroscience, Faculty of Psychology, Technische Universität Dresden, Dresden, Germany., Mitsis GD; Department of Bioengineering, McGill University, Montreal, Quebec, Canada., Moll J; D'Or Institute for Research and Education (IDOR), Rio de Janeiro, Brazil.; Department of Psychology, Stanford University, Stanford, CA, USA., Nielson DM; Data Science and Sharing Team, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, USA., Nilsonne G; Department of Clinical Neuroscience, Karolinska Institutet, Stockholm, Sweden.; Department of Psychology, Stockholm University, Stockholm, Sweden., Notter MP; The Laboratory for Investigative Neurophysiology (The LINE), Department of Radiology, University Hospital Center and University of Lausanne, Lausanne, Switzerland., Olivetti E; Neuroinformatics Laboratory, Fondazione Bruno Kessler, Trento, Italy.; Center for Mind/Brain Sciences - CIMeC, University of Trento, Rovereto, Italy., Onicas AI; MoMiLab Research Unit, IMT School for Advanced Studies Lucca, Lucca, Italy., Papale P; MoMiLab Research Unit, IMT School for Advanced Studies Lucca, Lucca, Italy.; Department of Vision and Cognition, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands., Patil KR; Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany., Peelle JE; Department of Otolaryngology, Washington University in St. Louis, St. Louis, MO, USA., Pérez A; McConnell Brain Imaging Centre, The Neuro (Montreal Neurological Institute-Hospital), Faculty of Medicine, McGill University, Montreal, Quebec, Canada., Pischedda D; Bernstein Center for Computational Neuroscience and Berlin Center for Advanced Neuroimaging and Clinic for Neurology, Charité Universitätsmedizin, corporate member of Freie Universität Berlin, Humboldt Universität zu Berlin, and Berlin Institute of Health, Berlin, Germany.; Cluster of Excellence Science of Intelligence, Technische Universität Berlin and Humboldt Universität zu Berlin, Berlin, Germany.; NeuroMI - Milan Center for Neuroscience, Milan, Italy., Poline JB; McConnell Brain Imaging Centre, The Neuro (Montreal Neurological Institute-Hospital), Faculty of Medicine, McGill University, Montreal, Quebec, Canada.; Henry H. Wheeler, Jr. Brain Imaging Center, Helen Wills Neuroscience Institute, University of California Berkeley, Berkeley, CA, USA., Prystauka Y; Department of Psychological Sciences, University of Connecticut, Storrs, CT, USA.; Brain Imaging Research Center, University of Connecticut, Storrs, CT, USA.; Connecticut Institute for the Brain and Cognitive Sciences, University of Connecticut, Storrs, CT, USA., Ray S; Department of Biomedical Engineering, New Jersey Institute of Technology, Newark, NJ, USA., Reuter-Lorenz PA; Department of Psychology, University of Michigan, Ann Arbor, MI, USA., Reynolds RC; Scientific and Statistical Computing Core, National Institute of Mental Health, National Institutes of Health, Bethesda, MD, USA., Ricciardi E; MoMiLab Research Unit, IMT School for Advanced Studies Lucca, Lucca, Italy., Rieck JR; Rotman Research Institute, Baycrest Health Sciences Centre, Toronto, Ontario, Canada., Rodriguez-Thompson AM; Department of Psychology and Neuroscience, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA., Romyn A; Department of Psychology, University of Toronto, Toronto, Ontario, Canada., Salo T; Department of Psychology, Florida International University, Miami, FL, USA., Samanez-Larkin GR; Center for Cognitive Neuroscience, Duke University, Durham, NC, USA.; Department of Psychology and Neuroscience, Duke University, Durham, NC, USA., Sanz-Morales E; Laboratorio de Análisis de Imagen Médica y Biometría (LAIMBIO), Universidad Rey Juan Carlos, Madrid, Spain., Schlichting ML; Department of Psychology, University of Toronto, Toronto, Ontario, Canada., Schultz DH; Department of Psychology, University of Nebraska-Lincoln, Lincoln, NE, USA.; Center for Brain, Biology and Behavior, University of Nebraska-Lincoln, Lincoln, NE, USA., Shen Q; School of Management, Zhejiang University of Technology, Hangzhou, China.; Institute of Neuromanagement, Zhejiang University of Technology, Hangzhou, China., Sheridan MA; Department of Psychology and Neuroscience, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA., Silvers JA; Department of Psychology, University of California Los Angeles, Los Angeles, CA, USA., Skagerlund K; Department of Behavioural Sciences and Learning, Linköping University, Linköping, Sweden.; Center for Social and Affective Neuroscience, Department of Clinical and Experimental Medicine, Linköping University, Linköping, Sweden., Smith A; Department of Economics, Virginia Tech, Blacksburg, VA, USA.; School of Neuroscience, Virginia Tech, Blacksburg, VA, USA., Smith DV; Department of Psychology, Temple University, Philadelphia, PA, USA., Sokol-Hessner P; Department of Psychology, University of Denver, Denver, CO, USA., Steinkamp SR; Institute of Neuroscience and Medicine, Cognitive Neuroscience (INM-3), Research Centre Jülich, Jülich, Germany., Tashjian SM; Department of Psychology, University of California Los Angeles, Los Angeles, CA, USA., Thirion B; Inria, CEA, Université Paris-Saclay, Palaiseau, France., Thorp JN; Department of Psychology, Columbia University, New York, NY, USA., Tinghög G; Department of Management and Engineering, Linköping University, Linköping, Sweden.; Department of Health, Medicine and Caring Sciences, Linköping University, Linköping, Sweden., Tisdall L; Department of Psychology, Stanford University, Stanford, CA, USA.; Center for Cognitive and Decision Sciences, University of Basel, Basel, Switzerland., Tompson SH; US CCDC Army Research Laboratory, Human Research and Engineering Directorate, Aberdeen Proving Ground, MD, USA., Toro-Serey C; Department of Psychological and Brain Sciences, Boston University, Boston, MA, USA.; Center for Systems Neuroscience, Boston University, Boston, MA, USA., Torre Tresols JJ; Inria, CEA, Université Paris-Saclay, Palaiseau, France., Tozzi L; Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, USA., Truong V; Graduate Institute of Mind, Brain and Consciousness, Taipei Medical University, Taipei, Taiwan.; Brain and Consciousness Research Centre, TMU-ShuangHo Hospital, New Taipei City, Taiwan., Turella L; Center for Mind/Brain Sciences - CIMeC, University of Trento, Rovereto, Italy., van 't Veer AE; Methodology and Statistics Unit, Institute of Psychology, Leiden University, Leiden, The Netherlands., Verguts T; Department of Experimental Psychology, Ghent University, Ghent, Belgium., Vettel JM; US Combat Capabilities Development Command Army Research Laboratory, Aberdeen, MD, USA.; University of California Santa Barbara, Santa Barbara, CA, USA.; University of Pennsylvania, Philadelphia, PA, USA., Vijayarajah S; Department of Psychology, University of Toronto, Toronto, Ontario, Canada., Vo K; Center for Cognitive Neuroscience, Duke University, Durham, NC, USA.; Department of Psychology and Neuroscience, Duke University, Durham, NC, USA., Wall MB; Invicro, London, UK.; Faculty of Medicine, Imperial College London, London, UK.; Clinical Psychopharmacology Unit, University College London, London, UK., Weeda WD; Methodology and Statistics Unit, Institute of Psychology, Leiden University, Leiden, The Netherlands., Weis S; Institute of Neuroscience and Medicine, Brain and Behaviour (INM-7), Research Centre Jülich, Jülich, Germany.; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany., White DJ; Centre for Human Psychopharmacology, Swinburne University, Hawthorn, Victoria, Australia., Wisniewski D; Department of Experimental Psychology, Ghent University, Ghent, Belgium., Xifra-Porxas A; Graduate Program in Biological and Biomedical Engineering, McGill University, Montreal, Quebec, Canada., Yearling EA; Department of Psychological Sciences, University of Connecticut, Storrs, CT, USA.; Brain Imaging Research Center, University of Connecticut, Storrs, CT, USA.; Connecticut Institute for the Brain and Cognitive Sciences, University of Connecticut, Storrs, CT, USA., Yoon S; Department of Management and Marketing, School of Business, University of Dayton, Dayton, OH, USA., Yuan R; Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, USA., Yuen KSL; Leibniz-Institut für Resilienzforschung (LIR), Mainz, Germany.; Neuroimaging Center (NIC), Focus Program Translational Neurosciences (FTN), Johannes Gutenberg University Medical Center Mainz, Mainz, Germany., Zhang L; Department of Cognition, Emotion, and Methods in Psychology, Faculty of Psychology, University of Vienna, Vienna, Austria., Zhang X; Brain Imaging Research Center, University of Connecticut, Storrs, CT, USA.; Connecticut Institute for the Brain and Cognitive Sciences, University of Connecticut, Storrs, CT, USA.; Biomedical Engineering Department, University of Connecticut, Storrs, CT, USA., Zosky JE; Department of Psychology, University of Nebraska-Lincoln, Lincoln, NE, USA.; Center for Brain, Biology and Behavior, University of Nebraska-Lincoln, Lincoln, NE, USA., Nichols TE; Oxford Big Data Institute, Li Ka Shing Centre for Health Information and Discovery, Nuffield Department of Population Health, University of Oxford, Oxford, UK. thomas.nichols@bdi.ox.ac.uk., Poldrack RA; Department of Psychology, Stanford University, Stanford, CA, USA. poldrack@stanford.edu., Schonberg T; Sagol School of Neuroscience, Tel Aviv University, Tel Aviv, Israel. schonberg@tauex.tau.ac.il.; Department of Neurobiology, The George S. Wise Faculty of Life Sciences, Tel Aviv University, Tel Aviv, Israel. schonberg@tauex.tau.ac.il.
Jazyk: angličtina
Zdroj: Nature [Nature] 2020 Jun; Vol. 582 (7810), pp. 84-88. Date of Electronic Publication: 2020 May 20.
DOI: 10.1038/s41586-020-2314-9
Abstrakt: Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses 1 . The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset 2-5 . Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed.
Databáze: MEDLINE