Finite-horizon Approximations and Episodic Equilibrium for Stochastic Games

Autor:	Sayin, Muhammed O.
Rok vydání:	2023
Předmět:	Computer Science - Computer Science and Game Theory Mathematics - Optimization and Control
Druh dokumentu:	Working Paper
Popis:	This paper proposes a finite-horizon approximation scheme and introduces episodic equilibrium as a solution concept for stochastic games (SGs), where agents strategize based on the current state and episode stage. The paper also establishes an upper bound on the approximation error that decays with the episode length for both discounted and time-averaged utilities. This approach bridges the gap in the analysis of finite and infinite-horizon SGs, and provides a unifying framework to address time-averaged and discounted utilities. To show the effectiveness of the scheme, the paper presents episodic, decentralized (i.e., payoff-based), and model-free learning dynamics proven to reach (near) episodic equilibrium in broad classes of SGs, including zero-sum, identical-interest and specific general-sum SGs with switching controllers for both time-averaged and discounted utilities. Comment: The working draft has gone through a major revision with a new formulation highlighting the finite-horizon approximation scheme and episodic equilibrium
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2310.07256 Zobrazit plný text záznamu View this record from Arxiv