Convergence of continuous-time stochastic gradient descent with applications to linear deep neural networks

Autor:	Lugosi, Gabor, Nualart, Eulalia
Rok vydání:	2024
Předmět:	Computer Science - Machine Learning Mathematics - Optimization and Control Statistics - Machine Learning
Druh dokumentu:	Working Paper
Popis:	We study a continuous-time approximation of the stochastic gradient descent process for minimizing the expected loss in learning problems. The main results establish general sufficient conditions for the convergence, extending the results of Chatterjee (2022) established for (nonstochastic) gradient descent. We show how the main result can be applied to the case of overparametrized linear neural network training.
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2409.07401 Zobrazit plný text záznamu View this record from Arxiv