A Confidence Region Approach to Tuning for Variable Selection
| Autor: | Funda Gunes, Howard D. Bondell |
|---|---|
| Rok vydání: | 2013 |
| Předmět: |
Statistics and Probability
Statistics::Theory Computer science Model selection Estimator Feature selection Confidence interval Article Statistics::Machine Learning Lasso (statistics) Bayesian information criterion Statistics Discrete Mathematics and Combinatorics Statistics::Methodology Statistics Probability and Uncertainty Akaike information criterion Algorithm Confidence region |
| Zdroj: | Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America. 21(2) |
| ISSN: | 1061-8600 |
| Popis: | We develop an approach to tuning of penalized regression variable selection methods by calculating the sparsest estimator contained in a confidence region of a specified level. Because confidence intervals/regions are generally understood, tuning penalized regression methods in this way is intuitive and more easily understood by scientists and practitioners. More importantly, our work shows that tuning to a fixed confidence level often performs better than tuning via the common methods based on AIC, BIC, or cross-validation (CV) over a wide range of sample sizes and levels of sparsity. Additionally, we prove that by tuning with a sequence of confidence levels converging to one, asymptotic selection consistency is obtained; and with a simple two-stage procedure, an oracle property is achieved. The confidence region based tuning parameter is easily calculated using output from existing penalized regression computer packages.Our work also shows how to map any penalty parameter to a corresponding confidence coefficient. This mapping facilitates comparisons of tuning parameter selection methods such as AIC, BIC and CV, and reveals that the resulting tuning parameters correspond to confidence levels that are extremely low, and can vary greatly across data sets. Supplemental materials for the article are available online. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|