Shrinkage Estimators in Online Experiments
| Autor: | Drew Dimmery, Eytan Bakshy, Jasjeet S. Sekhon |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2019 |
| Předmět: |
FOS: Computer and information sciences
Mean squared error Computer science Causal effect Estimator 02 engineering and technology Treatment and control groups Methodology (stat.ME) Bayes' theorem Frequentist inference 020204 information systems Covariate 0202 electrical engineering electronic engineering information engineering Econometrics 020201 artificial intelligence & image processing Statistics - Methodology Shrinkage |
| Zdroj: | KDD |
| Popis: | We develop and analyze empirical Bayes Stein-type estimators for use in the estimation of causal effects in large-scale online experiments. While online experiments are generally thought to be distinguished by their large sample size, we focus on the multiplicity of treatment groups. The typical analysis practice is to use simple differences-in-means (perhaps with covariate adjustment) as if all treatment arms were independent. In this work we develop consistent, small bias, shrinkage estimators for this setting. In addition to achieving lower mean squared error these estimators retain important frequentist properties such as coverage under most reasonable scenarios. Modern sequential methods of experimentation and optimization such as multi-armed bandit optimization (where treatment allocations adapt over time to prior responses) benefit from the use of our shrinkage estimators. Exploration under empirical Bayes focuses more efficiently on near-optimal arms, improving the resulting decisions made under uncertainty. We demonstrate these properties by examining seventeen large-scale experiments conducted on Facebook from April to June 2017. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|