Estimating multi-index models with response-conditional least squares
| Autor: | Timo Klock, Stefano Vigogna, Alessandro Lanteri |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2020 |
| Předmět: |
Statistics and Probability
sufficient dimension reduction Mathematics - Statistics Theory Statistics Theory (math.ST) 01 natural sciences 010104 statistics & probability 62G08 0502 economics and business FOS: Mathematics Applied mathematics 62G05 0101 mathematics Multi-index model 050205 econometrics Mathematics Polynomial regression 05 social sciences Estimator 62G05 62G08 62H99 Regression analysis finite sample bounds Nonparametric regression nonparametric regression Ordinary least squares Piecewise Statistics Probability and Uncertainty Simple linear regression 62H99 Curse of dimensionality |
| Zdroj: | Electron. J. Statist. 15, no. 1 (2021), 589-629 |
| Popis: | The multi-index model is a simple yet powerful high-dimensional regression model which circumvents the curse of dimensionality assuming $ \mathbb{E} [ Y | X ] = g(A^\top X) $ for some unknown index space $A$ and link function $g$. In this paper we introduce a method for the estimation of the index space, and study the propagation error of an index space estimate in the regression of the link function. The proposed method approximates the index space by the span of linear regression slope coefficients computed over level sets of the data. Being based on ordinary least squares, our approach is easy to implement and computationally efficient. We prove a tight concentration bound that shows $N^{-1/2}$-convergence, but also faithfully describes the dependence on the chosen partition of level sets, hence giving indications on the hyperparameter tuning. The estimator's competitiveness is confirmed by extensive comparisons with state-of-the-art methods, both on synthetic and real data sets. As a second contribution, we establish minimax optimal generalization bounds for k-nearest neighbors and piecewise polynomial regression when trained on samples projected onto any $N^{-1/2}$-consistent estimate of the index space, thus providing complete and provable estimation of the multi-index model. 30 pages, 13 figures, 1 table |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|