Margin-adaptive model selection in statistical learning

Autor: Peter L. Bartlett, Sylvain Arlot
Přispěvatelé: Laboratoire d'informatique de l'école normale supérieure (LIENS), Département d'informatique - ENS Paris (DI-ENS), Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche en Informatique et en Automatique (Inria)-École normale supérieure - Paris (ENS Paris), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Centre National de la Recherche Scientifique (CNRS)-Institut National de Recherche en Informatique et en Automatique (Inria)-École normale supérieure - Paris (ENS Paris), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL), Models of visual object recognition and scene understanding (WILLOW), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Inria Paris-Rocquencourt, Institut National de Recherche en Informatique et en Automatique (Inria), Computer Science Division [Berkeley], University of California [Berkeley], University of California-University of California, Department of Statistics [Berkeley], École normale supérieure - Paris (ENS-PSL), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-École normale supérieure - Paris (ENS-PSL), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-Inria Paris-Rocquencourt, University of California [Berkeley] (UC Berkeley), University of California (UC)-University of California (UC), École normale supérieure - Paris (ENS Paris), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Centre National de la Recherche Scientifique (CNRS), Département d'informatique de l'École normale supérieure (DI-ENS), Université Paris sciences et lettres (PSL)-Université Paris sciences et lettres (PSL)-Institut National de Recherche en Informatique et en Automatique (Inria)-Centre National de la Recherche Scientifique (CNRS)-École normale supérieure - Paris (ENS Paris)
Jazyk: angličtina
Rok vydání: 2008
Předmět:
Statistics and Probability
FOS: Computer and information sciences
Mathematical optimization
model selection
empirical minimization
Context (language use)
Machine Learning (stat.ML)
Mathematics - Statistics Theory
02 engineering and technology
Statistics Theory (math.ST)
[STAT.OT]Statistics [stat]/Other Statistics [stat.ML]
01 natural sciences
adaptivity
010104 statistics & probability
[STAT.ML]Statistics [stat]/Machine Learning [stat.ML]
Margin (machine learning)
[MATH.MATH-ST]Mathematics [math]/Statistics [math.ST]
Statistics - Machine Learning
0202 electrical engineering
electronic engineering
information engineering
FOS: Mathematics
oracle inequalities
Empirical risk minimization
0101 mathematics
Oracle inequality
local Rademacher complexity
Mathematics
Statistical learning
Model selection
empirical risk minimization
020206 networking & telecommunications
[STAT.TH]Statistics [stat]/Statistics Theory [stat.TH]
AMS primary 68T05
62H30
secondary 68Q32
62G08
statistical learning
classification
margin condition
Zdroj: Bernoulli 17, no. 2 (2011), 687-713
Bernoulli
Bernoulli, Bernoulli Society for Mathematical Statistics and Probability, 2011, 17 (2), pp.687-713. ⟨10.3150/10-BEJ288⟩
Bernoulli, 2011, 17 (2), pp.687-713. ⟨10.3150/10-BEJ288⟩
ISSN: 1350-7265
DOI: 10.3150/10-BEJ288⟩
Popis: A classical condition for fast learning rates is the margin condition, first introduced by Mammen and Tsybakov. We tackle in this paper the problem of adaptivity to this condition in the context of model selection, in a general learning framework. Actually, we consider a weaker version of this condition that allows one to take into account that learning within a small model can be much easier than within a large one. Requiring this "strong margin adaptivity" makes the model selection problem more challenging. We first prove, in a general framework, that some penalization procedures (including local Rademacher complexities) exhibit this adaptivity when the models are nested. Contrary to previous results, this holds with penalties that only depend on the data. Our second main result is that strong margin adaptivity is not always possible when the models are not nested: for every model selection procedure (even a randomized one), there is a problem for which it does not demonstrate strong margin adaptivity.
Published in at http://dx.doi.org/10.3150/10-BEJ288 the Bernoulli (http://isi.cbs.nl/bernoulli/) by the International Statistical Institute/Bernoulli Society (http://isi.cbs.nl/BS/bshome.htm)
Databáze: OpenAIRE
Externí odkaz: