Zobrazeno 1 - 10
of 90
pro vyhledávání: '"Roulet, Vincent"'
Autor:
Roulet, Vincent, Agarwala, Atish, Grill, Jean-Bastien, Swirszcz, Grzegorz, Blondel, Mathieu, Pedregosa, Fabian
Curvature information -- particularly, the largest eigenvalue of the loss Hessian, known as the sharpness -- often forms the basis for learning rate tuners. However, recent work has shown that the curvature information undergoes complex dynamics duri
Externí odkaz:
http://arxiv.org/abs/2407.06183
Autor:
Blondel, Mathieu, Roulet, Vincent
Artificial intelligence has recently experienced remarkable advances, fueled by large models, vast datasets, accelerated hardware, and, last but not least, the transformative power of differentiable programming. This new programming paradigm enables
Externí odkaz:
http://arxiv.org/abs/2403.14606
Recent empirical work has revealed an intriguing property of deep learning models by which the sharpness (largest eigenvalue of the Hessian) increases throughout optimization until it stabilizes around a critical value at which the optimizer operates
Externí odkaz:
http://arxiv.org/abs/2312.00209
We consider the distributionally robust optimization (DRO) problem with spectral risk-based uncertainty set and $f$-divergence penalty. This formulation includes common risk-sensitive learning objectives such as regularized condition value-at-risk (C
Externí odkaz:
http://arxiv.org/abs/2310.13863
Autor:
Roulet, Vincent, Blondel, Mathieu
Inspired by Gauss-Newton-like methods, we study the benefit of leveraging the structure of deep learning objectives, namely, the composition of a convex loss function and of a nonlinear network, in order to derive better direction oracles than stocha
Externí odkaz:
http://arxiv.org/abs/2308.08886
Gauss-Newton methods and their stochastic version have been widely used in machine learning and signal processing. Their nonsmooth counterparts, modified Gauss-Newton or prox-linear algorithms, can lead to contrasting outcomes when compared to gradie
Externí odkaz:
http://arxiv.org/abs/2305.10634
Spectral risk objectives - also called $L$-risks - allow for learning systems to interpolate between optimizing average-case performance (as in empirical risk minimization) and worst-case performance on a task. We develop stochastic algorithms to opt
Externí odkaz:
http://arxiv.org/abs/2212.05149
We present the implementation of nonlinear control algorithms based on linear and quadratic approximations of the objective from a functional viewpoint. We present a gradient descent, a Gauss-Newton method, a Newton method, differential dynamic progr
Externí odkaz:
http://arxiv.org/abs/2207.06362
A classical approach for solving discrete time nonlinear control on a finite horizon consists in repeatedly minimizing linear quadratic approximations of the original problem around current candidate solutions. While widely popular in many domains, s
Externí odkaz:
http://arxiv.org/abs/2204.02322
Autor:
Roulet, Vincent, Harchaoui, Zaid
Target Propagation (TP) algorithms compute targets instead of gradients along neural networks and propagate them backward in a way that is similar yet different than gradient back-propagation (BP). The idea was first presented as a perturbative alter
Externí odkaz:
http://arxiv.org/abs/2112.01453