A Geometric Characterization of Fisher Information from Quantized Samples with Applications to Distributed Statistical Estimation
| Autor: | Yanjun Han, Ayfer Ozgur, Leighton Pate Barnes |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Trace (linear algebra)
Score 020206 networking & telecommunications 02 engineering and technology 010501 environmental sciences 01 natural sciences Upper and lower bounds Combinatorics symbols.namesake Distribution (mathematics) Unit vector 0202 electrical engineering electronic engineering information engineering symbols Probability distribution Fisher information Random variable 0105 earth and related environmental sciences Mathematics |
| Zdroj: | Allerton |
| Popis: | Consider the Fisher information for estimating a vector $\theta \in \mathbb {R}^{d}$ from the quantized version of a statistical sample $X \sim f(x|\theta)$. Let M be a k-bit quantization of $X.$ We provide a geometric characterization of the trace of the Fisher information matrix $I_{M}(\theta)$ in terms of the score function $S_{\theta }(X)$. When $k=1$, we exactly solve the extremal problem of maximizing this geometric quantity for the Gaussian location model, which allows us to conclude that in this model, a half-space quantization is the one-bit quantization that maximizes $Tr(I_{M}(\theta))$. Under assumptions on the tail of the distribution of $S_{\theta }(X)$ projected onto any unit vector in $\mathbb {R}^{d}$, we give upper bounds demonstrating how $Tr(I_{M}(\theta))$ can scale with k. We apply these results to find lower bounds on the minimax risk of estimating $\theta $ from multiple quantized samples of X, for example in a distributed setting where the samples are distributed across multiple nodes and each node has a total budget of k-bits to communicate its sample to a centralized estimator. Our bounds apply in a unified way to many common statistical models including the Gaussian location model and discrete distribution estimation, and they recover and generalize existing results in the literature with simpler and more transparent proofs. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|