Zobrazeno 1 - 10
of 2 192
pro vyhledávání: '"Dandi SO"'
A key property of neural networks is their capacity of adapting to data during training. Yet, our current mathematical understanding of feature learning and its relationship to generalization remain limited. In this work, we provide a random matrix a
Externí odkaz:
http://arxiv.org/abs/2410.18938
Autor:
Arnaboldi, Luca, Dandi, Yatin, Krzakala, Florent, Loureiro, Bruno, Pesce, Luca, Stephan, Ludovic
Publikováno v:
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:1730-1762, 2024
We study the impact of the batch size $n_b$ on the iteration time $T$ of training two-layer neural networks with one-pass stochastic gradient descent (SGD) on multi-index target functions of isotropic covariates. We characterize the optimal batch siz
Externí odkaz:
http://arxiv.org/abs/2406.02157
Autor:
Troiani, Emanuele, Dandi, Yatin, Defilippis, Leonardo, Zdeborová, Lenka, Loureiro, Bruno, Krzakala, Florent
Multi-index models - functions which only depend on the covariates through a non-linear transformation of their projection on a subspace - are a useful benchmark for investigating feature learning with neural nets. This paper examines the theoretical
Externí odkaz:
http://arxiv.org/abs/2405.15480
Neural networks can identify low-dimensional relevant structures within high-dimensional noisy data, yet our mathematical understanding of how they do so remains scarce. Here, we investigate the training dynamics of two-layer shallow neural networks
Externí odkaz:
http://arxiv.org/abs/2405.15459
Autor:
Cui, Hugo, Pesce, Luca, Dandi, Yatin, Krzakala, Florent, Lu, Yue M., Zdeborová, Lenka, Loureiro, Bruno
Publikováno v:
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:9662-9695, 2024
In this manuscript, we investigate the problem of how two-layer neural networks learn features from data, and improve over the kernel regime, after being trained with a single gradient descent step. Leveraging the insight from (Ba et al., 2022), we m
Externí odkaz:
http://arxiv.org/abs/2402.04980
Autor:
Dandi, Yatin, Troiani, Emanuele, Arnaboldi, Luca, Pesce, Luca, Zdeborová, Lenka, Krzakala, Florent
Publikováno v:
Proceedings of the 41st International Conference on Machine Learning, PMLR 235:9991-10016, 2024
We investigate the training dynamics of two-layer neural networks when learning multi-index target functions. We focus on multi-pass gradient descent (GD) that reuses the batches multiple times and show that it significantly changes the conclusion ab
Externí odkaz:
http://arxiv.org/abs/2402.03220
Publikováno v:
Journal of the Mechanical Behavior of Materials, Vol 33, Iss 1, Pp 9438-54 (2024)
Natural sources used in industry, such as environmental waste fibers for plants, waste paper, and others, can lessen waste-throwing problems and reduce environmental pollution to save lives on the earth’s crust. The natural composites of natural fi
Externí odkaz:
https://doaj.org/article/04e07ed9b1cd410ca26600f9d0f8b75c
Publikováno v:
Advances in Aerodynamics, Vol 6, Iss 1, Pp 1-18 (2024)
Abstract The accurate computation of different turbulent statistics poses different requirements on numerical methods. In this paper, we investigate the capabilities of two representative numerical schemes in predicting mean velocities, Reynolds stre
Externí odkaz:
https://doaj.org/article/55afdc7bfa194b889790d125fc2d0c0d
Publikováno v:
Ecological Processes, Vol 13, Iss 1, Pp 1-16 (2024)
Abstract Background Bacteria, Archaea, and Microeukaryotes comprise taxonomic domains that interact in mediating biogeochemical cycles in coastal waters. Many studies have revealed contrasting biogeographic patterns of community structure and assembl
Externí odkaz:
https://doaj.org/article/14d3f36dcaae46adbad00cca5e835f1b
The rapid progress in machine learning in recent years has been based on a highly productive connection to gradient-based optimization. Further progress hinges in part on a shift in focus from pattern recognition to decision-making and multi-agent pr
Externí odkaz:
http://arxiv.org/abs/2309.04877