From Activation to Initialization: Scaling Insights for Optimizing Neural Fields

Autor:	Saratchandran, Hemanth, Ramasinghe, Sameera, Lucey, Simon
Rok vydání:	2024
Předmět:	Computer Science - Computer Vision and Pattern Recognition Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	In the realm of computer vision, Neural Fields have gained prominence as a contemporary tool harnessing neural networks for signal representation. Despite the remarkable progress in adapting these networks to solve a variety of problems, the field still lacks a comprehensive theoretical framework. This article aims to address this gap by delving into the intricate interplay between initialization and activation, providing a foundational basis for the robust optimization of Neural Fields. Our theoretical insights reveal a deep-seated connection among network initialization, architectural choices, and the optimization process, emphasizing the need for a holistic approach when designing cutting-edge Neural Fields. Comment: CVPR 2024
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2403.19205 Zobrazit plný text záznamu View this record from Arxiv