Zobrazeno 1 - 8
of 8
pro vyhledávání: '"Wolinski, Pierre"'
Autor:
Wolinski, Pierre
We consider a gradient-based optimization method applied to a function $\mathcal{L}$ of a vector of variables $\boldsymbol{\theta}$, in the case where $\boldsymbol{\theta}$ is represented as a tuple of tensors $(\mathbf{T}_1, \cdots, \mathbf{T}_S)$.
Externí odkaz:
http://arxiv.org/abs/2312.03885
The field of Tiny Machine Learning (TinyML) has gained significant attention due to its potential to enable intelligent applications on resource-constrained devices. This review provides an in-depth analysis of the advancements in efficient neural ne
Externí odkaz:
http://arxiv.org/abs/2311.11883
This work studies the global convergence and implicit bias of Gauss Newton's (GN) when optimizing over-parameterized one-hidden layer networks in the mean-field regime. We first establish a global convergence result for GN in the continuous-time limi
Externí odkaz:
http://arxiv.org/abs/2302.02904
Autor:
Wolinski, Pierre, Arbel, Julyan
The study of feature propagation at initialization in neural networks lies at the root of numerous initialization designs. An assumption very commonly made in the field states that the pre-activations are Gaussian. Although this convenient Gaussian h
Externí odkaz:
http://arxiv.org/abs/2205.12379
In machine learning, it is common to optimize the parameters of a probabilistic model, modulated by an ad hoc regularization term that penalizes some values of the parameters. Regularization terms appear naturally in Variational Inference, a tractabl
Externí odkaz:
http://arxiv.org/abs/2002.00178
Hyperparameter tuning is a bothersome step in the training of deep learning models. One of the most sensitive hyperparameters is the learning rate of the gradient descent. We present the 'All Learning Rates At Once' (Alrao) optimization method for ne
Externí odkaz:
http://arxiv.org/abs/1810.01322
Autor:
Wolinski, Pierre, Arbel, Julyan
Publikováno v:
JDS 2022-53es Journées de Statistique de la Société Française de Statistiques (SFdS)
JDS 2022-53es Journées de Statistique de la Société Française de Statistiques (SFdS), Jun 2022, Lyon, France
JDS 2022-53es Journées de Statistique de la Société Française de Statistiques (SFdS), Jun 2022, Lyon, France
International audience; The goal of the present work is to propose a way to modify both the initialization distribution of the weights of a neural network and its activation function, such that all pre-activations are Gaussian. We propose a family of
Externí odkaz:
https://explore.openaire.eu/search/publication?articleId=od_______165::e4a4ef3015af09b124a7424cf0ffe86b
https://hal.science/hal-03853790
https://hal.science/hal-03853790
24 pages, including 2 pages of references and 10 pages of appendix; In machine learning, it is common to optimize the parameters of a probabilistic model, modulated by a somewhat ad hoc regularization term that penalizes some values of the parameters
Externí odkaz:
https://explore.openaire.eu/search/publication?articleId=dedup_wf_001::ce8511993fb0365500bdb02a62540b00
https://hal.science/hal-02466702
https://hal.science/hal-02466702