CGAR: Critic Guided Action Redistribution in Reinforcement Leaning

Autor:	Tairan Huang, Xu Li, Hao Li, Mingming Sun, Ping Li
Rok vydání:	2022
Předmět:	FOS: Computer and information sciences Computer Science - Machine Learning Machine Learning (cs.LG)
DOI:	10.48550/arxiv.2206.11494
Popis:	Training a game-playing reinforcement learning agent requires multiple interactions with the environment. Ignorant random exploration may cause a waste of time and resources. It's essential to alleviate such waste. As discussed in this paper, under the settings of the off-policy actor critic algorithms, we demonstrate that the critic can bring more expected discounted rewards than or at least equal to the actor. Thus, the Q value predicted by the critic is a better signal to redistribute the action originally sampled from the policy distribution predicted by the actor. This paper introduces the novel Critic Guided Action Redistribution (CGAR) algorithm and tests it on the OpenAI MuJoCo tasks. The experimental results demonstrate that our method improves the sample efficiency and achieves state-of-the-art performance. Our code can be found at https://github.com/tairanhuang/CGAR. Comment: IEEE Conference on Games (CoG), 2022
Databáze:	OpenAIRE
Externí odkaz:	https://explore.openaire.eu/search/publication?articleId=doi_dedup___::a2a3d5ed629e7aacdbebfedd8b0eea48 Zobrazit plný text záznamu