A Problem in Model Selection of LASSO and Introduction of Scaling
| Autor: | Katsuyuki Hagiwara |
|---|---|
| Rok vydání: | 2016 |
| Předmět: |
Mathematical optimization
Model selection Value (computer science) Estimator 02 engineering and technology Sparse approximation 01 natural sciences Thresholding 010104 statistics & probability Risk Estimate Lasso (statistics) Statistics 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing 0101 mathematics Scaling Mathematics |
| Zdroj: | Neural Information Processing ISBN: 9783319466712 ICONIP (2) |
| DOI: | 10.1007/978-3-319-46672-9_3 |
| Popis: | In this article, we considered to assign a single scaling parameter to LASSO estimators for investigating and improving a problem of excessive shrinkage at a sparse representation. This problem is important because it directly affects a quality of model selection in LASSO. We derived a prediction risk for LASSO with scaling and obtained an optimal scaling parameter value that minimizes the risk. We then showed the risk is improved by assigning the optimal scaling value. In a numerical example, we found that an estimate of the optimal scaling value is larger than one especially at a sparse representation; i.e. excessive shrinkage is relaxed by expansion via scaling. Additionally, we observed that a risk for LASSO is high at a sparse representation and it is minimized at a relatively large model while this is improved by the introduction of an estimate of the optimal scaling value. We here constructed a fully empirical risk estimate that approximates the actual risk well. We then observed that, by applying the risk estimate as a model selection criterion, LASSO with scaling tends to obtain a model with low risk and high sparsity compared to LASSO without scaling. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|