Optimal Causal Representations and the Causal Information Bottleneck

Autor:	Simoes, Francisco N. F. Q., Dastani, Mehdi, van Ommen, Thijs
Rok vydání:	2024
Předmět:	Computer Science - Machine Learning Computer Science - Artificial Intelligence Computer Science - Information Theory Statistics - Machine Learning
Druh dokumentu:	Working Paper
Popis:	To effectively study complex causal systems, it is often useful to construct representations that simplify parts of the system by discarding irrelevant details while preserving key features. The Information Bottleneck (IB) method is a widely used approach in representation learning that compresses random variables while retaining information about a target variable. Traditional methods like IB are purely statistical and ignore underlying causal structures, making them ill-suited for causal tasks. We propose the Causal Information Bottleneck (CIB), a causal extension of the IB, which compresses a set of chosen variables while maintaining causal control over a target variable. This method produces representations which are causally interpretable, and which can be used when reasoning about interventions. We present experimental results demonstrating that the learned representations accurately capture causality as intended. Comment: Submitted to ICLR 2025. Code available at github.com/francisco-simoes/cib-optimization-psagd
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2410.00535 Zobrazit plný text záznamu View this record from Arxiv