Zobrazeno 1 - 10
of 43
pro vyhledávání: '"Paquette, Courtney"'
Autor:
Collins-Woodfin, Elizabeth, Seroussi, Inbar, Malaxechebarría, Begoña García, Mackenzie, Andrew W., Paquette, Elliot, Paquette, Courtney
We develop a framework for analyzing the training and learning rate dynamics on a large class of high-dimensional optimization problems, which we call the high line, trained using one-pass stochastic gradient descent (SGD) with adaptive learning rate
Externí odkaz:
http://arxiv.org/abs/2405.19585
We consider the three parameter solvable neural scaling model introduced by Maloney, Roberts, and Sully. The model has three parameters: data complexity, target complexity, and model-parameter-count. We use this neural scaling model to derive new pre
Externí odkaz:
http://arxiv.org/abs/2405.15074
We study the problem of differentially-private (DP) stochastic (convex-concave) saddle-points in the polyhedral setting. We propose $(\varepsilon, \delta)$-DP algorithms based on stochastic mirror descent that attain nearly dimension-independent conv
Externí odkaz:
http://arxiv.org/abs/2403.02912
Autor:
Marion, Pierre, Korba, Anna, Bartlett, Peter, Blondel, Mathieu, De Bortoli, Valentin, Doucet, Arnaud, Llinares-López, Felipe, Paquette, Courtney, Berthet, Quentin
We present a new algorithm to optimize distributions defined implicitly by parameterized stochastic diffusions. Doing so allows us to modify the outcome distribution of sampling processes by optimizing over their parameters. We introduce a general fr
Externí odkaz:
http://arxiv.org/abs/2402.05468
We analyze the dynamics of streaming stochastic gradient descent (SGD) in the high-dimensional limit when applied to generalized linear models and multi-index models (e.g. logistic regression, phase retrieval) with general data-covariance. In particu
Externí odkaz:
http://arxiv.org/abs/2308.08977
The recently developed average-case analysis of optimization methods allows a more fine-grained and representative convergence analysis than usual worst-case results. In exchange, this analysis requires a more precise hypothesis over the data generat
Externí odkaz:
http://arxiv.org/abs/2206.09901
Stochastic gradient descent (SGD) is a pillar of modern machine learning, serving as the go-to optimization algorithm for a diverse array of problems. While the empirical success of SGD is often attributed to its computational efficiency and favorabl
Externí odkaz:
http://arxiv.org/abs/2206.07252
We analyze the dynamics of large batch stochastic gradient descent with momentum (SGD+M) on the least squares problem when both the number of samples and dimensions are large. In this setting, we show that the dynamics of SGD+M converge to a determin
Externí odkaz:
http://arxiv.org/abs/2206.01029
We develop a stochastic differential equation, called homogenized SGD, for analyzing the dynamics of stochastic gradient descent (SGD) on a high-dimensional random least squares problem with $\ell^2$-regularization. We show that homogenized SGD is th
Externí odkaz:
http://arxiv.org/abs/2205.07069
Autor:
Paquette, Courtney, Paquette, Elliot
We analyze a class of stochastic gradient algorithms with momentum on a high-dimensional random least squares problem. Our framework, inspired by random matrix theory, provides an exact (deterministic) characterization for the sequence of loss values
Externí odkaz:
http://arxiv.org/abs/2106.03696