The sceptical Bayes factor for the assessment of replication success
| Autor: | Pawel, Samuel, Held, Leonhard |
|---|---|
| Přispěvatelé: | University of Zurich, Pawel, Samuel |
| Jazyk: | angličtina |
| Rok vydání: | 2022 |
| Předmět: |
FOS: Computer and information sciences
Statistics and Probability 05 social sciences Statistics 610 Medicine & health 10060 Epidemiology Biostatistics and Prevention Institute (EBPI) 01 natural sciences 050105 experimental psychology Methodology (stat.ME) 010104 statistics & probability 0501 psychology and cognitive sciences Probability and Uncertainty 1804 Statistics Probability and Uncertainty 0101 mathematics Statistics Probability and Uncertainty 2613 Statistics and Probability Statistics - Methodology |
| Popis: | Replication studies are increasingly conducted but there is no established statistical criterion for replication success. We propose a novel approach combining reverse-Bayes analysis with Bayesian hypothesis testing: a sceptical prior is determined for the effect size such that the original finding is no longer convincing in terms of a Bayes factor. This prior is then contrasted to an advocacy prior (the reference posterior of the effect size based on the original study), and replication success is declared if the replication data favour the advocacy over the sceptical prior at a higher level than the original data favoured the sceptical prior over the null hypothesis. The sceptical Bayes factor is the highest level where replication success can be declared. A comparison to existing methods reveals that the sceptical Bayes factor combines several notions of replicability: it ensures that both studies show sufficient evidence against the null and penalises incompatibility of their effect estimates. Analysis of asymptotic properties and error rates, as well as case studies from the Social Sciences Replication Project show the advantages of the method for the assessment of replicability. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|