Raising the value of research studies in psychological science by increasing the credibility of research reports: The Transparent Psi Project - Preprint

Autor: Zoltan Kekecs, Bence Palfi, Barnabas Szaszi, Peter Szecsi, Mark Zrubka, Marton Kovacs, Bence Endre Bakos, Denis Cousineau, Patrizio Tressoldi, Kathleen Schmidt, Massimo Grassi, Thomas Rhys Evans, Yuki Yamada, Jeremy K. Miller, Huanxu Liu, Fumiya Yonemitsu, Dmitrii Dubrov, Jan Philipp Röer, Marvin Becker, Roxane Schnepper, Atsunori Ariga, Patrícia Arriaga, Raquel Oliveira, Nele Põldver, Kairi Kreegipuu, Braeden Forrester Hall, Sera Wiechert, Bruno Verschuere, Kyra Giran, Balazs Aczel
Rok vydání: 2019
Předmět:
Popis: Those wishing to join as collaborating labs to data collection can do so via: https://t.co/W0fv5VwPi2?amp=1. Those interested in signing up as auditors for the project should send an email to the first author. ABSTRACT: The low reproducibility rate in social sciences lead researchers to hesitate to accept published findings at their face value. It became apparent that the field is lacking the tools necessary to verify credibility of research reports. In the present paper, we describe methodologies that let researchers craft highly credible research, and allow their peers to verify this credibility. We demonstrate the application of these methods in a fully transparent multi-lab replication of Bem’s Experiment 1 (2011), which was co-designed by a consensus panel including both proponents and opponents of Bem’s original hypothesis. In the main study, we applied direct data deposition, in combination with born-open data and real-time research report to extend transparency to protocol delivery and data collection. We also used piloting, checklists, laboratory logs and video documented trial sessions to ascertain as-intended protocol delivery by the experimenters, and external research auditors to monitor research integrity. We found X% successful guesses, while Bem reported 53.07% success rate. The effect reported by Bem was not/was replicated in our study/This study outcome did not reach the pre-specified criteria for supporting or contradicting Bem’s findings. [Conclusions about the feasibility of the credibility-enhancing methodologies will be discussed here.]. Plain word summary: [In case of a negative result:]This project aimed to demonstrate the use of research methods that could improve the reliability of scientific findings in psychological science. Using rigorous methodology, we could not replicate the positive findings of Bem’s 2011 Experiment 1. This finding does not confirm, nor contradict the existence of ESP in general, and this was not the point of our study. Instead, the results tell us that (1) it is likely that the original experiment was biased by methodological flaws, and (2) it is improbable that the paradigm used in the original study would be useful in detecting ESP effects if they exist. [In case of a positive result:]This project aimed to demonstrate the use of research methods that could improve the reliability of scientific findings in psychological science. Using rigorous methodology we could replicate the positive findings of Bem’s 2011 Experiment 1. This finding does not confirm, nor contradict the existence of ESP in general, and this was not the point of our study. Instead, the results tell us that (1) it is unlikely that the positive findings of the original experiment can be explained only by the currently known methodological biases, and (2) more studies are warranted to investigate the causes for the positive effect. We do not know yet what these causes are, but it is important to note, that neither our study, nor the original study provide any evidence that these causes would be “paranormal”. Thus, it is still safe to assume that the effects at play are within the boundaries of known physics, psychology, and research methodology.
Databáze: OpenAIRE
načítá se...