Optimal Approximation Complexity of High-Dimensional Functions with Neural Networks

Autor:	Goverse, Vincent P. H., Hamdan, Jad, Tanner, Jared
Rok vydání:	2023
Předmět:	Computer Science - Machine Learning Computer Science - Neural and Evolutionary Computing 41A10
Druh dokumentu:	Working Paper
Popis:	We investigate properties of neural networks that use both ReLU and $x^2$ as activation functions and build upon previous results to show that both analytic functions and functions in Sobolev spaces can be approximated by such networks of constant depth to arbitrary accuracy, demonstrating optimal order approximation rates across all nonlinear approximators, including standard ReLU networks. We then show how to leverage low local dimensionality in some contexts to overcome the curse of dimensionality, obtaining approximation rates that are optimal for unknown lower-dimensional subspaces. Comment: 10 pages, 1 figure
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2301.13091 Zobrazit plný text záznamu View this record from Arxiv