Scalable regret for learning to control network-coupled subsystems with unknown dynamics

Autor:	Sudhakara, Sagar, Mahajan, Aditya, Nayyar, Ashutosh, Ouyang, Yi
Rok vydání:	2021
Předmět:	Electrical Engineering and Systems Science - Systems and Control Computer Science - Artificial Intelligence Mathematics - Optimization and Control
Druh dokumentu:	Working Paper
Popis:	We consider the problem of controlling an unknown linear quadratic Gaussian (LQG) system consisting of multiple subsystems connected over a network. Our goal is to minimize and quantify the regret (i.e. loss in performance) of our strategy with respect to an oracle who knows the system model. Viewing the interconnected subsystems globally and directly using existing LQG learning algorithms for the global system results in a regret that increases super-linearly with the number of subsystems. Instead, we propose a new Thompson sampling based learning algorithm which exploits the structure of the underlying network. We show that the expected regret of the proposed algorithm is bounded by $\tilde{\mathcal{O}} \big( n \sqrt{T} \big)$ where $n$ is the number of subsystems, $T$ is the time horizon and the $\tilde{\mathcal{O}}(\cdot)$ notation hides logarithmic terms in $n$ and $T$. Thus, the regret scales linearly with the number of subsystems. We present numerical experiments to illustrate the salient features of the proposed algorithm. Comment: 12 pages
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2108.07970 Zobrazit plný text záznamu View this record from Arxiv