Achieving High Accuracy with PINNs via Energy Natural Gradients

Autor:	Müller, Johannes, Zeinhofer, Marius
Rok vydání:	2023
Předmět:	Computer Science - Machine Learning Mathematics - Numerical Analysis
Druh dokumentu:	Working Paper
Popis:	We propose energy natural gradient descent, a natural gradient method with respect to a Hessian-induced Riemannian metric as an optimization algorithm for physics-informed neural networks (PINNs) and the deep Ritz method. As a main motivation we show that the update direction in function space resulting from the energy natural gradient corresponds to the Newton direction modulo an orthogonal projection onto the model's tangent space. We demonstrate experimentally that energy natural gradient descent yields highly accurate solutions with errors several orders of magnitude smaller than what is obtained when training PINNs with standard optimizers like gradient descent or Adam, even when those are allowed significantly more computation time. Comment: Published version
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2302.13163 Zobrazit plný text záznamu View this record from Arxiv