On the Distribution of Penalized Maximum Likelihood Estimators: The LASSO, SCAD, and Thresholding

Autor: Benedikt M. Poetscher, Hannes Leeb
Jazyk: angličtina
Rok vydání: 2007
Předmět:
FOS: Computer and information sciences
Statistics and Probability
SCAD
Uniform consistency
Penalized maximum likelihood
LASSO
thresholding
post-model-selection estimator
finite-sample distribution
asymptotic distribution
estimation of distribution
uniform consistency
Asymptotic distribution
Machine Learning (stat.ML)
Feature selection
Mathematics - Statistics Theory
Statistics Theory (math.ST)
Methodology (stat.ME)
Finite-sample distribution
Lasso (statistics)
Statistics - Machine Learning
primary
Estimation of distribution
Statistics
FOS: Mathematics
Applied mathematics
jel:C2
Limit (mathematics)
Post-model-selection estimator
Statistics - Methodology
Mathematics
Numerical Analysis
Model selection
Mathematical statistics
Estimator
jel:C13
M-estimator
Thresholding
Distribution (mathematics)
Statistics
Probability and Uncertainty
Oracle property
Scad
62J07
62F12
62E15
Popis: We study the distributions of the LASSO, SCAD, and thresholding estimators, in finite samples and in the large-sample limit. The asymptotic distributions are derived for both the case where the estimators are tuned to perform consistent model selection and for the case where the estimators are tuned to perform conservative model selection. Our findings complement those of Knight and Fu [K. Knight, W. Fu, Asymptotics for lasso-type estimators, Annals of Statistics 28 (2000) 1356–1378] and Fan and Li [J. Fan, R. Li, Variable selection via non-concave penalized likelihood and its oracle properties, Journal of the American Statistical Association 96 (2001) 1348–1360]. We show that the distributions are typically highly non-normal regardless of how the estimator is tuned, and that this property persists in large samples. The uniform convergence rate of these estimators is also obtained, and is shown to be slower than n−1/2 in case the estimator is tuned to perform consistent model selection. An impossibility result regarding estimation of the estimators’ distribution function is also provided.
Databáze: OpenAIRE
Externí odkaz: