Safe Reinforcement Learning Using Black-Box Reachability Analysis

Autor:	Selim, Mahmoud, Alanwar, Amr, Kousik, Shreyas, Gao, Grace, Pavone, Marco, Johansson, Karl H.
Rok vydání:	2022
Předmět:	Computer Science - Robotics Computer Science - Machine Learning Electrical Engineering and Systems Science - Systems and Control
Druh dokumentu:	Working Paper
DOI:	10.1109/LRA.2022.3192205
Popis:	Reinforcement learning (RL) is capable of sophisticated motion planning and control for robots in uncertain environments. However, state-of-the-art deep RL approaches typically lack safety guarantees, especially when the robot and environment models are unknown. To justify widespread deployment, robots must respect safety constraints without sacrificing performance. Thus, we propose a Black-box Reachability-based Safety Layer (BRSL) with three main components: (1) data-driven reachability analysis for a black-box robot model, (2) a trajectory rollout planner that predicts future actions and observations using an ensemble of neural networks trained online, and (3) a differentiable polytope collision check between the reachable set and obstacles that enables correcting unsafe actions. In simulation, BRSL outperforms other state-of-the-art safe RL methods on a Turtlebot 3, a quadrotor, a trajectory-tracking point mass, and a hexarotor in wind with an unsafe set adjacent to the area of highest reward. Comment: This paper is accepted at IEEE Robotics and Automation Letters and International Conference on Robotics and Automation (ICRA)
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2204.07417 Zobrazit plný text záznamu View this record from Arxiv