Výsledky vyhledávání - "Shustin, Boris"

Report

AdaRankGrad: Adaptive Gradient-Rank and Moments for Memory-Efficient LLMs Training and Fine-Tuning

Autor: Refael, Yehonathan, Svirsky, Jonathan, Shustin, Boris, Huleihel, Wasim, Lindenbaum, Ofir

Training and fine-tuning large language models (LLMs) come with challenges related to memory and computational requirements due to the increasing size of the model weights and the optimizer states. Various techniques have been developed to tackle the

Externí odkaz: http://arxiv.org/abs/2410.17881

Zobrazit plný text záznamu

Report

Manifold Free Riemannian Optimization

Autor: Shustin, Boris, Avron, Haim, Sober, Barak

Riemannian optimization is a principled framework for solving optimization problems where the desired optimum is constrained to a smooth manifold $\mathcal{M}$. Algorithms designed in this framework usually require some geometrical description of the

Externí odkaz: http://arxiv.org/abs/2209.03269

Zobrazit plný text záznamu

Report

Faster Randomized Methods for Orthogonality Constrained Problems

Autor: Shustin, Boris, Avron, Haim

Recent literature has advocated the use of randomized methods for accelerating the solution of various matrix problems arising throughout data science and computational science. One popular strategy for leveraging randomization is to use it as a way

Externí odkaz: http://arxiv.org/abs/2106.12060

Zobrazit plný text záznamu

Report

Solving Trust Region Subproblems Using Riemannian Optimization

Autor: Mor, Uria, Shustin, Boris, Avron, Haim

The Trust Region Subproblem is a fundamental optimization problem that takes a pivotal role in Trust Region Methods. However, the problem, and variants of it, also arise in quite a few other applications. In this article, we present a family of itera

Externí odkaz: http://arxiv.org/abs/2010.07547

Zobrazit plný text záznamu

Report

Riemannian optimization with a preconditioning scheme on the generalized Stiefel manifold

Autor: Shustin, Boris, Avron, Haim

Optimization problems on the generalized Stiefel manifold (and products of it) are prevalent across science and engineering. For example, in computational science they arise in symmetric (generalized) eigenvalue problems, in nonlinear eigenvalue prob

Externí odkaz: http://arxiv.org/abs/1902.01635

Zobrazit plný text záznamu