An oracle property of the Nadaraya–Watson kernel estimator for high‐dimensional nonparametric regression
| Autor: | Gang Li, Daniel Conn |
|---|---|
| Rok vydání: | 2018 |
| Předmět: |
Statistics and Probability
Statistics::Theory Rank (linear algebra) 05 social sciences Kernel density estimation Estimator 01 natural sciences Oracle Nonparametric regression 010104 statistics & probability Rate of convergence Kernel (statistics) 0502 economics and business Applied mathematics 0101 mathematics Statistics Probability and Uncertainty 050205 econometrics Mathematics Curse of dimensionality |
| Zdroj: | Scandinavian Journal of Statistics. 46:735-764 |
| ISSN: | 1467-9469 0303-6898 |
| Popis: | The celebrated Nadaraya-Watson kernel estimator is among the most studied method for nonparametric regression. A classical result is that its rate of convergence depends on the number of covariates and deteriorates quickly as the dimension grows, which underscores the "curse of dimensionality" and has limited its use in high dimensional settings. In this article, we show that when the true regression function is single or multi-index, the effects of the curse of dimensionality may be mitigated for the Nadaraya-Watson kernel estimator. Specifically, we prove that with $K$-fold cross-validation, the Nadaraya-Watson kernel estimator indexed by a positive semidefinite bandwidth matrix has an oracle property that its rate of convergence depends on the number of indices of the regression function rather than the number of covariates. Intuitively, this oracle property is a consequence of allowing the bandwidths to diverge to infinity as opposed to restricting them all to converge to zero at certain rates as done in previous theoretical studies. Our result provides a theoretical perspective for the use of kernel estimation in high dimensional nonparametric regression and other applications such as metric learning when a low rank structure is anticipated. Numerical illustrations are given through simulations and real data examples. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Plný text