GAP Safe screening rules for sparse multi-task and multi-class models
| Autor: | Ndiaye, E., Fercoq, O., Alexandre Gramfort, Salmon, J. |
|---|---|
| Přispěvatelé: | Signal, Statistique et Apprentissage (S2A), Laboratoire Traitement et Communication de l'Information (LTCI), Institut Mines-Télécom [Paris] (IMT)-Télécom Paris-Institut Mines-Télécom [Paris] (IMT)-Télécom Paris, Département Traitement du Signal et des Images (TSI), Télécom ParisTech-Centre National de la Recherche Scientifique (CNRS), HAL, TelecomParis |
| Rok vydání: | 2015 |
| Předmět: |
FOS: Computer and information sciences
[MATH.MATH-OC] Mathematics [math]/Optimization and Control [math.OC] Machine Learning (stat.ML) [INFO.INFO-LG] Computer Science [cs]/Machine Learning [cs.LG] Statistics - Computation Machine Learning (cs.LG) Computer Science - Learning Statistics::Machine Learning [INFO.INFO-LG]Computer Science [cs]/Machine Learning [cs.LG] Statistics - Machine Learning Optimization and Control (math.OC) FOS: Mathematics 68Uxx 49N15 62Jxx 68Q32 62-04 [MATH.MATH-OC]Mathematics [math]/Optimization and Control [math.OC] Mathematics - Optimization and Control Computation (stat.CO) |
| Zdroj: | Conference on Neural Information Processing Systems Conference on Neural Information Processing Systems, Dec 2015, Montréal, Canada Scopus-Elsevier |
| DOI: | 10.48550/arxiv.1506.03736 |
| Popis: | High dimensional regression benefits from sparsity promoting regularizations. Screening rules leverage the known sparsity of the solution by ignoring some variables in the optimization, hence speeding up solvers. When the procedure is proven not to discard features wrongly the rules are said to be \emph{safe}. In this paper we derive new safe rules for generalized linear models regularized with $\ell_1$ and $\ell_1/\ell_2$ norms. The rules are based on duality gap computations and spherical safe regions whose diameters converge to zero. This allows to discard safely more variables, in particular for low regularization parameters. The GAP Safe rule can cope with any iterative solver and we illustrate its performance on coordinate descent for multi-task Lasso, binary and multinomial logistic regression, demonstrating significant speed ups on all tested datasets with respect to previous safe rules. Comment: in Proceedings of the 29-th Conference on Neural Information Processing Systems (NIPS), 2015 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|