Zobrazeno 1 - 10
of 262
pro vyhledávání: '"Shamir, Ohad"'
Autor:
Kornowski, Guy, Shamir, Ohad
Recent results show that vanilla gradient descent can be accelerated for smooth convex objectives, merely by changing the stepsize sequence. We show that this can lead to surprisingly large errors indefinitely, and therefore ask: Is there any stepsiz
Externí odkaz:
http://arxiv.org/abs/2406.13888
We study depth separation in infinite-width neural networks, where complexity is controlled by the overall squared $\ell_2$-norm of the weights (sum of squares of all weights in the network). Whereas previous depth separation results focused on separ
Externí odkaz:
http://arxiv.org/abs/2402.08808
Autor:
Barzilai, Daniel, Shamir, Ohad
It is by now well-established that modern over-parameterized models seem to elude the bias-variance tradeoff and generalize well despite overfitting noise. Many recent works attempt to analyze this phenomenon in the relatively tractable setting of ke
Externí odkaz:
http://arxiv.org/abs/2312.15995
Autor:
Kornilov, Nikita, Shamir, Ohad, Lobanov, Aleksandr, Dvinskikh, Darina, Gasnikov, Alexander, Shibaev, Innokentiy, Gorbunov, Eduard, Horváth, Samuel
In this paper, we consider non-smooth stochastic convex optimization with two function evaluations per round under infinite noise variance. In the classical setting when noise has finite variance, an optimal algorithm, built upon the batched accelera
Externí odkaz:
http://arxiv.org/abs/2310.18763
Autor:
Kornowski, Guy, Shamir, Ohad
We study the complexity of producing $(\delta,\epsilon)$-stationary points of Lipschitz objectives which are possibly neither smooth nor convex, using only noisy function evaluations. Recent works proposed several stochastic zero-order algorithms tha
Externí odkaz:
http://arxiv.org/abs/2307.04504
Autor:
Magen, Roey, Shamir, Ohad
We provide several new results on the sample complexity of vector-valued linear predictors (parameterized by a matrix), and more generally neural networks. Focusing on size-independent bounds, where only the Frobenius norm distance of the parameters
Externí odkaz:
http://arxiv.org/abs/2305.16475
Overparameterized neural networks (NNs) are observed to generalize well even when trained to perfectly fit noisy data. This phenomenon motivated a large body of work on "benign overfitting", where interpolating predictors achieve near-optimal perform
Externí odkaz:
http://arxiv.org/abs/2305.15141
We study the complexity of optimizing nonsmooth nonconvex Lipschitz functions by producing $(\delta,\epsilon)$-stationary points. Several recent works have presented randomized algorithms that produce such points using $\tilde O(\delta^{-1}\epsilon^{
Externí odkaz:
http://arxiv.org/abs/2302.08300
Autor:
Kornowski, Guy, Shamir, Ohad
We study the oracle complexity of producing $(\delta,\epsilon)$-stationary points of Lipschitz functions, in the sense proposed by Zhang et al. [2020]. While there exist dimension-free randomized algorithms for producing such points within $\widetild
Externí odkaz:
http://arxiv.org/abs/2209.10346
Understanding to what extent neural networks memorize training data is an intriguing question with practical and theoretical implications. In this paper we show that in some cases a significant fraction of the training data can in fact be reconstruct
Externí odkaz:
http://arxiv.org/abs/2206.07758