Newton Optimization on Helmholtz Decomposition for Continuous Games

Autor:	Ramponi, G., Restelli, M.
Rok vydání:	2021
Předmět:	FOS: Computer and information sciences Computer Science - Machine Learning Statistics - Machine Learning Machine Learning (stat.ML) General Medicine Machine Learning (cs.LG)
Zdroj:	Proceedings of the AAAI Conference on Artificial Intelligence. 35:11325-11333
ISSN:	2374-3468 2159-5399
DOI:	10.1609/aaai.v35i13.17350
Popis:	Many learning problems involve multiple agents optimizing different interactive functions. In these problems, the standard policy gradient algorithms fail due to the non-stationarity of the setting and the different interests of each agent. In fact, algorithms must take into account the complex dynamics of these systems to guarantee rapid convergence towards a (local) Nash equilibrium. In this paper, we propose NOHD (Newton Optimization on Helmholtz Decomposition), a Newton-like algorithm for multi-agent learning problems based on the decomposition of the dynamics of the system in its irrotational (Potential) and solenoidal (Hamiltonian) component. This method ensures quadratic convergence in purely irrotational systems and pure solenoidal systems. Furthermore, we show that NOHD is attracted to stable fixed points in general multi-agent systems and repelled by strict saddle ones. Finally, we empirically compare the NOHD's performance with that of state-of-the-art algorithms on some bimatrix games and in a continuous Gridworld environment. In 35th AAAI Conference on Artificial Intelligence (AAAI 2021)
Databáze:	OpenAIRE
Externí odkaz:	https://explore.openaire.eu/search/publication?articleId=doi_dedup___::35b514fc273b0d540aa0b0e9c56720a4 https://doi.org/10.1609/aaai.v35i13.17350 Zobrazit plný text záznamu