Reproducibility in the absence of selective reporting : An illustration from large-scale brain asymmetry research

Autor: Kong, Xiang-Zhen, Francks, Clyde, Allen, Nicholas B., Heslenfeld, Dirk, Hester, Robert, Hibar, Derrek Paul, Ho, Beng-Choon, Ho, Tiffany C., Hoekstra, Pieter J., Holst, Ruth J., Hoogman, Martine, Høvik, Marie F., Howells, Fleur M., Ames, David, Hugdahl, Kenneth, Huyser, Chaim, Ingvar, Martin, Ishikawa, Akari, James, Anthony, Jahanshad, Neda, Jernigan, Terry L., Jönsson, Erik G, Kaleda, Vasily, Kelly, Clare, Andreassen, Ole A., Kerich, Michael, Keshavan, Matcheri S., Khadka, Sabin, Kircher, Tilo, Kohls, Gregor, Konrad, Kerstin, Korucuoglu, Ozlem, Krämer, Bernd, Krug, Axel, Kuntsi, Jonna, Vasquez, Alejandro Arias, Kwon, Jun Soo, Lambregts-Rommelse, Nanda, Landén, Mikael, Lázaro, Luisa, Lebedeva, Irina, Lenroot, Rhoshel, Lesch, Klaus-Peter, Li, Qinqin, Lim, Kelvin O., Liu, Jia, Armstrong, Nicola J., Lochner, Christine, London, Edythe D., Lorenzetti, Valentina, Luciano, Michelle, Luijten, Maartje, Lundervold, Astri J., Mackey, Scott, MacMaster, Frank P., Maingault, Sophie, Malpas, Charles B., Asherson, Phil, Malt, Ulrik F., Mataix-Cols, David, Martin-Santos, Rocio, Mayer, Andrew R., McCarthy, Hazel, Medland, Sarah, Metha, Mitul, Mitchell, Philip B., Mueller, Bryon A., Maniega, Susana Muñoz, Bergo, Felipe, Mazoyer, Bernard, McDonald, Colm, McLellan, Quinn, McMahon, Katie L., McPhilemy, Genevieve, Momenan, Reza, Morales, Angelica M., Narayanaswamy, Janardhanan C., Moreira, José Carlos Vasques, Nerland, Stener, Bastin, Mark E., Nestor, Liam, Newman, Erik, Nigg, Joel T., Nordvik, Jan Egil, Novotny, Stephanie, Weiss, Eileen Oberwelland, O'Gorman, Ruth L., Oosterlaan, Jaap, Oranje, Bob, Orr, Catherine, Batalla, Albert, Overs, Bronwyn, Paloyelis, Yannis, Pauli, Paul, Paulus, Martin, Plessen, Kerstin Jessica, Polier, Georg G., Pomarol-Clotet, Edith, Portella, Maria J., Qiu, Jiang, Radua, Joaquim, Bauer, Jochen, Ramos-Quiroga, Josep Antoni, Reddy, Y. C. Janardhan, Reif, Andreas, Roberts, Gloria, Rosa, Pedro, Rubia, Katya, Sacchet, Matthew D., Sachdev, Perminder S., Salvador, Raymond, Schmaal, Lianne, Baune, Bernhard T, Schulte-Rüther, Martin, Schweren, Lizanne, Seitz, Jochen, Serpa, Mauricio Henriques, Shaw, Philip, Shumskaya, Elena, Silk, Timothy J., Simmons, Alan N., Simulionyte, Egle, Sinha, Rajita, Baur-Streubel, Ramona, Sjoerds, Zsuzsika, Smelror, Runar Elle, Soliva, Joan Carlos, Solowij, Nadia, Souza-Duran, Fabio Luisde, Sponheim, Scott R., Stein, Dan J., Stein, Elliot A., Stevens, Michael, Strike, Lachlan T., Biederman, Joseph, Sudre, Gustavo, Sui, Jing, Tamm, Leanne, Temmingh, Hendrik S., Thoma, Robert J., Tomyshev, Alexander, Tronchin, Giulia, Turner, Jessica, Uhlmann, Anne, Erp, Theo G. M., Blaine, Sara K., Heuvel, Odile A., Meer, Dennis, Eijk, Liza, Vance, Alasdair, Veer, Ilya M., Veltman, Dick J., Venkatasubramanian, Ganesan, Vilarroya, Oscar, Vives-Gilabert, Yolanda, Voineskos, Aristotle N, Boedhoe, Premika, Völzke, Henry, Vuletic, Daniella, Walitza, Susanne, Walter, Henrik, Walton, Esther, Wardlaw, Joanna M., Wen, Wei, Westlye, Lars T., Whelan, Christopher D., White, Tonya, Bøen, Erlend, Wiers, Reinout W., Wright, Margaret J., Wittfeld, Katharina, Yang, Tony T., Yasuda, Clarissa L., Yoncheva, Yuliya, Yücel, Murat, Yun, Je-Yeon, Zanetti, Marcus Vinicius, Zhen, Zonglei, Bose, Anushree, Zhu, Xing-xing, Ziegler, Georg C., Zubicaray, Greig I., Zwiers, Marcel, Project, Karolinska Schizophrenia, Glahn, David C., Crivello, Fabrice, Fisher, Simon E., Thompson, Paul M., Bralten, Janita, Farde, Lars, Flyckt, Lena, Engberg, Göran, Erhardt, Sophie, Fatouros-Bergman, Helena, Cervenka, Simon, Schwieler, Lilly, Piehl, Fredrik, Agartz, Ingrid, Collste, Karin, Brandeis, Daniel, Victorsson, Pauliina, Malmqvist, Anna, Hedberg, Mikael, Orhan, Funda, Sellgren, Carl, Brem, Silvia, Mathias, Samuel R., Brodaty, Henry, Yüksel, Dilara, Brooks, Samantha J., Buitelaar, Jan, Bürger, Christian, Bülow, Robin, Calhoun, Vince, Calvo, Anna, Canales-Rodríguez, Erick Jorge, Cannon, Dara M., Guadalupe, Tulio, Caparelli, Elisabeth C., Castellanos, Francisco X., Cendes, Fernando, Chaim-Avancini, Tiffany Moukbel, Chantiluke, Kaylita, Chen, Qun-lin, Chen, Xiayu, Cheng, Yuqi, Christakou, Anastasia, Clark, Vincent P., Abé, Christoph, Coghill, David, Connolly, Colm G., Conzelmann, Annette, Córdova-Palomera, Aldo, Cousijn, Janna, Crow, Tim, Cubillo, Ana, Dannlowski, Udo, Bruttopilo, Sara Ambrosino, Zeeuw, Patrick, Deary, Ian J., Demeter, Damion V., Di Martino, Adriana, Dickie, Erin W, Dietsche, Bruno, Doan, Nhat Trung, Doherty, Colin P., Doyle, Alysa, Durston, Sarah, Earl, Eric, Akudjedu, Theophilus N., Ehrlich, Stefan, Ekman, Carl Johan, Elvsåshagen, Torbjørn, Epstein, Jeffery N., Fair, Damien A., Faraone, Stephen V., Fernández, Guillén, Flint, Claas, Filho, Geraldo Busatto, Förster, Katharina, Aleman, Andre, Fouche, Jean-Paul, Foxe, John J., Frodl, Thomas, Fuentes-Claramonte, Paola, Fullerton, Janice M., Garavan, Hugh, Santos Garcia, Danielle, Gotlib, Ian H., Goudriaan, Anna E., Grabe, Hans Jörgen, Alhusaini, Saud, Groenewold, Nynke A., Grotegerd, Dominik, Gruber, Oliver, Gurholt, Tiril, Haavik, Jan, Hahn, Tim, Hansell, Narelle K., Harris, Mathew A., Hartman, Catharina A., Carmen Valdés Hernández, Maria
Přispěvatelé: Alhusaini, Saud, Del Carmen Valdés Hernández, Maria, Heslenfeld, Dirk, Hester, Robert, Hibar, Derrek Paul, Ho, Beng-Choon, Ho, Tiffany C., Hoekstra, Pieter J., van Holst, Ruth J., Hoogman, Martine, Høvik, Marie F., Allen, Nicholas B., Howells, Fleur M., Hugdahl, Kenneth, Huyser, Chaim, Ingvar, Martin, Ishikawa, Akari, James, Anthony, Jahanshad, Neda, Jernigan, Terry L., Jönsson, Erik G., Kaleda, Vasily, Ames, David, Kelly, Clare, Kerich, Michael, Keshavan, Matcheri S., Khadka, Sabin, Kircher, Tilo, Kohls, Gregor, Konrad, Kerstin, Korucuoglu, Ozlem, Krämer, Bernd, Krug, Axel, Andreassen, Ole A., Kuntsi, Jonna, Kwon, Jun Soo, Lambregts-Rommelse, Nanda, Landén, Mikael, Lázaro, Luisa, Lebedeva, Irina, Lenroot, Rhoshel, Lesch, Klaus-Peter, Li, Qinqin, Lim, Kelvin O., Vasquez, Alejandro Arias, Liu, Jia, Lochner, Christine, London, Edythe D., Lorenzetti, Valentina, Luciano, Michelle, Luijten, Maartje, Lundervold, Astri J., Mackey, Scott, MacMaster, Frank P., Maingault, Sophie, Armstrong, Nicola J., Malpas, Charles B., Malt, Ulrik F., Mataix-Cols, David, Martin-Santos, Rocio, Mayer, Andrew R., McCarthy, Hazel, Medland, Sarah, Metha, Mitul, Mitchell, Philip B., Mueller, Bryon A., Asherson, Phil, Maniega, Susana Muñoz, Mazoyer, Bernard, McDonald, Colm, McLellan, Quinn, McMahon, Katie L., McPhilemy, Genevieve, Momenan, Reza, Morales, Angelica M., Narayanaswamy, Janardhanan C., Moreira, José Carlos Vasques, Bergo, Felipe, Nerland, Stener, Nestor, Liam, Newman, Erik, Nigg, Joel T., Nordvik, Jan Egil, Novotny, Stephanie, Weiss, Eileen Oberwelland, O'Gorman, Ruth L., Oosterlaan, Jaap, Oranje, Bob, Bastin, Mark E., Orr, Catherine, Overs, Bronwyn, Paloyelis, Yannis, Pauli, Paul, Paulus, Martin, Plessen, Kerstin Jessica, von Polier, Georg G., Pomarol-Clotet, Edith, Portella, Maria J., Qiu, Jiang, Batalla, Albert, Radua, Joaquim, Ramos-Quiroga, Josep Antoni, Reddy, Y. C. Janardhan, Reif, Andreas, Roberts, Gloria, Rosa, Pedro, Rubia, Katya, Sacchet, Matthew D., Sachdev, Perminder S., Salvador, Raymond, Bauer, Jochen, Schmaal, Lianne, Schulte-Rüther, Martin, Schweren, Lizanne, Seitz, Jochen, Serpa, Mauricio Henriques, Shaw, Philip, Shumskaya, Elena, Silk, Timothy J., Simmons, Alan N., Simulionyte, Egle, Baune, Bernhard T., Sinha, Rajita, Sjoerds, Zsuzsika, Smelror, Runar Elle, Soliva, Joan Carlos, Solowij, Nadia, Souza-Duran, Fabio Luisde, Sponheim, Scott R., Stein, Dan J., Stein, Elliot A., Stevens, Michael, Baur-Streubel, Ramona, Strike, Lachlan T., Sudre, Gustavo, Sui, Jing, Tamm, Leanne, Temmingh, Hendrik S., Thoma, Robert J., Tomyshev, Alexander, Tronchin, Giulia, Turner, Jessica, Uhlmann, Anne, Biederman, Joseph, van Erp, Theo G. M., van den Heuvel, Odile A., van der Meer, Dennis, van Eijk, Liza, Vance, Alasdair, Veer, Ilya M., Veltman, Dick J., Venkatasubramanian, Ganesan, Vilarroya, Oscar, Vives-Gilabert, Yolanda, Blaine, Sara K., Voineskos, Aristotle N., Völzke, Henry, Vuletic, Daniella, Walitza, Susanne, Walter, Henrik, Walton, Esther, Wardlaw, Joanna M., Wen, Wei, Westlye, Lars T., Whelan, Christopher D., Boedhoe, Premika, White, Tonya, Wiers, Reinout W., Wright, Margaret J., Wittfeld, Katharina, Yang, Tony T., Yasuda, Clarissa L., Yoncheva, Yuliya, Yücel, Murat, Yun, Je-Yeon, Zanetti, Marcus Vinicius, Bøen, Erlend, Zhen, Zonglei, Zhu, Xing-Xing, Ziegler, Georg C., de Zubicaray, Greig I., Zwiers, Marcel, Project, Karolinska Schizophrenia, Glahn, David C., Crivello, Fabrice, Fisher, Simon E., Thompson, Paul M., Bose, Anushree, Francks, Clyde, Farde, Lars, Flyckt, Lena, Engberg, Göran, Erhardt, Sophie, Fatouros-Bergman, Helena, Cervenka, Simon, Schwieler, Lilly, Piehl, Fredrik, Agartz, Ingrid, Bralten, Janita, Collste, Karin, Victorsson, Pauliina, Malmqvist, Anna, Hedberg, Mikael, Orhan, Funda, Sellgren, Carl, Brandeis, Daniel, Kong, Xiang-Zhen, Brem, Silvia, Brodaty, Henry, Yüksel, Dilara, Brooks, Samantha J., Buitelaar, Jan, Bürger, Christian, Bülow, Robin, Calhoun, Vince, Calvo, Anna, Canales-Rodríguez, Erick Jorge, Mathias, Samuel R., Cannon, Dara M., Caparelli, Elisabeth C., Castellanos, Francisco X., Cendes, Fernando, Chaim-Avancini, Tiffany Moukbel, Chantiluke, Kaylita, Chen, Qun-Lin, Chen, Xiayu, Cheng, Yuqi, Christakou, Anastasia, Guadalupe, Tulio, Clark, Vincent P., Coghill, David, Connolly, Colm G., Conzelmann, Annette, Córdova-Palomera, Aldo, Cousijn, Janna, Crow, Tim, Cubillo, Ana, Dannlowski, Udo, de Bruttopilo, Sara Ambrosino, Abé, Christoph, de Zeeuw, Patrick, Deary, Ian J., Demeter, Damion V., Di Martino, Adriana, Dickie, Erin W., Dietsche, Bruno, Doan, Nhat Trung, Doherty, Colin P., Doyle, Alysa, Durston, Sarah, Earl, Eric, Ehrlich, Stefan, Ekman, Carl Johan, Elvsåshagen, Torbjørn, Epstein, Jeffery N., Fair, Damien A., Faraone, Stephen V., Fernández, Guillén, Flint, Claas, Filho, Geraldo Busatto, Akudjedu, Theophilus N., Förster, Katharina, Fouche, Jean-Paul, Foxe, John J., Frodl, Thomas, Fuentes-Claramonte, Paola, Fullerton, Janice M., Garavan, Hugh, do Santos Garcia, Danielle, Gotlib, Ian H., Goudriaan, Anna E., Aleman, Andre, Grabe, Hans Jörgen, Groenewold, Nynke A., Grotegerd, Dominik, Gruber, Oliver, Gurholt, Tiril, Haavik, Jan, Hahn, Tim, Hansell, Narelle K., Harris, Mathew A., Hartman, Catharina A., Ontwikkelingspsychologie (Psychologie, FMG), Adult Psychiatry, APH - Mental Health, ANS - Compulsivity, Impulsivity & Attention, Child and Adolescent Psychiatry & Psychosocial Care, Child Psychiatry, ANS - Cellular & Molecular Mechanisms, Laboratory Genetic Metabolic Diseases, Paediatrics, General Paediatrics, ARD - Amsterdam Reproduction and Development, Paediatric Pulmonology, Graduate School, APH - Digital Health
Jazyk: angličtina
Rok vydání: 2022
Předmět:
P‐hacking
Datasets as Topic
Publication bias
0302 clinical medicine
130 000 Cognitive Neurology & Memory
Statistics
team science
Brain asymmetry
Multicenter Studies as Topic
Cervell
Research Articles
Cerebral Cortex
Radiological and Ultrasound Technology
P-hacking
05 social sciences
Brain
Cerebral cortex
Middle Aged
Magnetic Resonance Imaging
Discoveries in science
Escorça cerebral
Neurology
Biaix de publicació
Anatomy
Psychology
Research Article
Neuroinformatics
Adult
Adolescent
Neuroimaging
Descobriments científics
050105 experimental psychology
Statistical power
03 medical and health sciences
Young Adult
Magnetic resonance imaging
Imatges per ressonància magnètica
multisite collaboration
Humans
0501 psychology and cognitive sciences
Radiology
Nuclear Medicine and imaging
ddc:610
reproducibility
Aged
publication bias
Reproducibility
Neurodevelopmental disorders Donders Center for Medical Neuroscience [Radboudumc 7]
Perspective (graphical)
Reproducibility of Results
Brain Cortical Thickness
Research data
Sample size determination
Dades de recerca
Neurology (clinical)
Scale (map)
Developmental Psychopathology
030217 neurology & neurosurgery
Zdroj: Human Brain Mapping, 43, 244-254
Human Brain Mapping
Human brain mapping 43(1), 244-254 (2022). doi:10.1002/hbm.25154
Human brain mapping 43(1), 244-254 (2022). doi:10.1002/hbm.25154 special issue: "Special Issue: The ENIGMA Consortium: the first 10 years / Issue Edited by: P.M. Thompson, N. Jahanshad, L. Schmaal, J.A. Turner, A. Winkler, S.I. Thomopoulos, G.F. Egan, P. Kochunov"
Human Brain Mapping, 43, 1, pp. 244-254
Human Brain Mapping, 43(1), 244-254. Wiley-Liss Inc.
Human brain mapping, 43(1), 244-254. Wiley-Liss Inc.
ISSN: 1065-9471
Popis: The problem of poor reproducibility of scientific findings has received much attention over recent years, in a variety of fields including psychology and neuroscience. The problem has been partly attributed to publication bias and unwanted practices such as p‐hacking. Low statistical power in individual studies is also understood to be an important factor. In a recent multisite collaborative study, we mapped brain anatomical left–right asymmetries for regional measures of surface area and cortical thickness, in 99 MRI datasets from around the world, for a total of over 17,000 participants. In the present study, we revisited these hemispheric effects from the perspective of reproducibility. Within each dataset, we considered that an effect had been reproduced when it matched the meta‐analytic effect from the 98 other datasets, in terms of effect direction and significance threshold. In this sense, the results within each dataset were viewed as coming from separate studies in an “ideal publishing environment,” that is, free from selective reporting and p hacking. We found an average reproducibility rate of 63.2% (SD = 22.9%, min = 22.2%, max = 97.0%). As expected, reproducibility was higher for larger effects and in larger datasets. Reproducibility was not obviously related to the age of participants, scanner field strength, FreeSurfer software version, cortical regional measurement reliability, or regional size. These findings constitute an empirical illustration of reproducibility in the absence of publication bias or p hacking, when assessing realistic biological effects in heterogeneous neuroscience data, and given typically‐used sample sizes.
Region‐wise effect sizes and reproducibility rates of hemispheric asymmetry effects. In general, effects with higher effect sizes showed higher reproducibility, given the same conditions (e.g., sample size and data heterogeneity).
Databáze: OpenAIRE
Externí odkaz: