Testing Independence of Exchangeable Random Variables

Autor:	Hutter, Marcus
Rok vydání:	2022
Předmět:	Mathematics - Statistics Theory Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Given well-shuffled data, can we determine whether the data items are statistically (in)dependent? Formally, we consider the problem of testing whether a set of exchangeable random variables are independent. We will show that this is possible and develop tests that can confidently reject the null hypothesis that data is independent and identically distributed and have high power for (some) exchangeable distributions. We will make no structural assumptions on the underlying sample space. One potential application is in Deep Learning, where data is often scraped from the whole internet, with duplications abound, which can render data non-iid and test-set evaluation prone to give wrong answers. Comment: 42 pages, 3 figures
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2210.12392 Zobrazit plný text záznamu View this record from Arxiv