Estimating Risk and Uncertainty in Deep Reinforcement Learning

Autor:	Clements, William R., Van Delft, Bastien, Robaglia, Benoît-Marie, Slaoui, Reda Bahi, Toth, Sébastien
Rok vydání:	2019
Předmět:	Computer Science - Machine Learning Computer Science - Artificial Intelligence Statistics - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Reinforcement learning agents are faced with two types of uncertainty. Epistemic uncertainty stems from limited data and is useful for exploration, whereas aleatoric uncertainty arises from stochastic environments and must be accounted for in risk-sensitive applications. We highlight the challenges involved in simultaneously estimating both of them, and propose a framework for disentangling and estimating these uncertainties on learned Q-values. We derive unbiased estimators of these uncertainties and introduce an uncertainty-aware DQN algorithm, which we show exhibits safe learning behavior and outperforms other DQN variants on the MinAtar testbed. Comment: Work presented at the ICML 2020 Workshop on Uncertainty and Robustness in Deep Learning
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/1905.09638 Zobrazit plný text záznamu View this record from Arxiv