Initial State Interventions for Deconfounded Imitation Learning

Autor:	Pfrommer, Samuel, Bai, Yatong, Lee, Hyunin, Sojoudi, Somayeh
Rok vydání:	2023
Předmět:	Computer Science - Machine Learning Electrical Engineering and Systems Science - Systems and Control
Druh dokumentu:	Working Paper
Popis:	Imitation learning suffers from causal confusion. This phenomenon occurs when learned policies attend to features that do not causally influence the expert actions but are instead spuriously correlated. Causally confused agents produce low open-loop supervised loss but poor closed-loop performance upon deployment. We consider the problem of masking observed confounders in a disentangled representation of the observation space. Our novel masking algorithm leverages the usual ability to intervene in the initial system state, avoiding any requirement involving expert querying, expert reward functions, or causal graph specification. Under certain assumptions, we theoretically prove that this algorithm is conservative in the sense that it does not incorrectly mask observations that causally influence the expert; furthermore, intervening on the initial state serves to strictly reduce excess conservatism. The masking algorithm is applied to behavior cloning for two illustrative control systems: CartPole and Reacher. Comment: 62nd IEEE Conference on Decision and Control
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2307.15980 Zobrazit plný text záznamu View this record from Arxiv