Statistical Mechanics of Min-Max Problems

Autor:	Ichikawa, Yuma, Hukushima, Koji
Rok vydání:	2024
Předmět:	Computer Science - Machine Learning Condensed Matter - Disordered Systems and Neural Networks Mathematics - Statistics Theory Statistics - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Min-max optimization problems, also known as saddle point problems, have attracted significant attention due to their applications in various fields, such as fair beamforming, generative adversarial networks (GANs), and adversarial learning. However, understanding the properties of these min-max problems has remained a substantial challenge. This study introduces a statistical mechanical formalism for analyzing the equilibrium values of min-max problems in the high-dimensional limit, while appropriately addressing the order of operations for min and max. As a first step, we apply this formalism to bilinear min-max games and simple GANs, deriving the relationship between the amount of training data and generalization error and indicating the optimal ratio of fake to real data for effective learning. This formalism provides a groundwork for a deeper theoretical analysis of the equilibrium properties in various machine learning methods based on min-max problems and encourages the development of new algorithms and architectures. Comment: 16 pages, 1 figures
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2409.06053 Zobrazit plný text záznamu View this record from Arxiv