Bias Correction in Deterministic Policy Gradient Using Robust MPC

Autor:	Kordabad, Arash Bahari, Esfahani, Hossein Nejatbakhsh, Gros, Sebastien
Rok vydání:	2021
Předmět:	Electrical Engineering and Systems Science - Systems and Control
Druh dokumentu:	Working Paper
Popis:	In this paper, we discuss the deterministic policy gradient using the Actor-Critic methods based on the linear compatible advantage function approximator, where the input spaces are continuous. When the policy is restricted by hard constraints, the exploration may not be Centred or Isotropic (non-CI). As a result, the policy gradient estimation can be biased. We focus on constrained policies based on Model Predictive Control (MPC) schemes and to address the bias issue, we propose an approximate Robust MPC approach accounting for the exploration. The RMPC-based policy ensures that a Centered and Isotropic (CI) exploration is approximately feasible. A posterior projection is used to ensure its exact feasibility, we formally prove that this approach does not bias the gradient estimation. Comment: This paper has been accepted to ECC 2021. 6 pages
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2104.02413 Zobrazit plný text záznamu View this record from Arxiv