Score-based generative models learn manifold-like structures with constrained mixing

Autor:	Wenliang, Li Kevin, Moran, Ben
Rok vydání:	2023
Předmět:	Statistics - Machine Learning Computer Science - Computer Vision and Pattern Recognition Computer Science - Machine Learning
Druh dokumentu:	Working Paper
Popis:	How do score-based generative models (SBMs) learn the data distribution supported on a low-dimensional manifold? We investigate the score model of a trained SBM through its linear approximations and subspaces spanned by local feature vectors. During diffusion as the noise decreases, the local dimensionality increases and becomes more varied between different sample sequences. Importantly, we find that the learned vector field mixes samples by a non-conservative field within the manifold, although it denoises with normal projections as if there is an energy function in off-manifold directions. At each noise level, the subspace spanned by the local features overlap with an effective density function. These observations suggest that SBMs can flexibly mix samples with the learned score field while carefully maintaining a manifold-like structure of the data distribution. Comment: NeurIPS 2022 Workshop on Score-Based Methods
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2311.09952 Zobrazit plný text záznamu View this record from Arxiv