A distributed adaptive policy gradient method based on momentum for multi-agent reinforcement learning
Autor: | Junru Shi, Xin Wang, Mingchuan Zhang, Muhua Liu, Junlong Zhu, Qingtao Wu |
---|---|
Jazyk: | angličtina |
Rok vydání: | 2024 |
Předmět: | |
Zdroj: | Complex & Intelligent Systems, Vol 10, Iss 5, Pp 7297-7310 (2024) |
Druh dokumentu: | article |
ISSN: | 2199-4536 2198-6053 |
DOI: | 10.1007/s40747-024-01529-6 |
Popis: | Abstract Policy Gradient (PG) method is one of the most popular algorithms in Reinforcement Learning (RL). However, distributed adaptive variants of PG are rarely studied in multi-agent. For this reason, this paper proposes a distributed adaptive policy gradient algorithm (IS-DAPGM) incorporated with Adam-type updates and importance sampling technique. Furthermore, we also establish the theoretical convergence rate of $$\mathcal {O}(1/\sqrt{T})$$ O ( 1 / T ) , where T represents the number of iterations, it can match the convergence rate of the state-of-the-art centralized policy gradient methods. In addition, many experiments are conducted in a multi-agent environment, which is a modification on the basis of Particle world environment. By comparing with some other distributed PG methods and changing the number of agents, we verify the performance of IS-DAPGM is more efficient than the existing methods. |
Databáze: | Directory of Open Access Journals |
Externí odkaz: |