Theoretical Analysis of Leave-one-out Cross Validation for Non-differentiable Penalties under High-dimensional Settings

Autor:	Zou, Haolin, Auddy, Arnab, Rad, Kamiar Rahnama, Maleki, Arian
Rok vydání:	2024
Předmět:	Mathematics - Statistics Theory Statistics - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Despite a large and significant body of recent work focused on estimating the out-of-sample risk of regularized models in the high dimensional regime, a theoretical understanding of this problem for non-differentiable penalties such as generalized LASSO and nuclear norm is missing. In this paper we resolve this challenge. We study this problem in the proportional high dimensional regime where both the sample size n and number of features p are large, and n/p and the signal-to-noise ratio (per observation) remain finite. We provide finite sample upper bounds on the expected squared error of leave-one-out cross-validation (LO) in estimating the out-of-sample risk. The theoretical framework presented here provides a solid foundation for elucidating empirical findings that show the accuracy of LO. Comment: 30 pages
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2402.08543 Zobrazit plný text záznamu View this record from Arxiv