Uniform Transformation: Refining Latent Representation in Variational Autoencoders

Autor:	Shi, Ye, Lee, C. S. George
Rok vydání:	2024
Předmět:	Computer Science - Machine Learning Electrical Engineering and Systems Science - Image and Video Processing Mathematics - Optimization and Control Statistics - Machine Learning
Druh dokumentu:	Working Paper
Popis:	Irregular distribution in latent space causes posterior collapse, misalignment between posterior and prior, and ill-sampling problem in Variational Autoencoders (VAEs). In this paper, we introduce a novel adaptable three-stage Uniform Transformation (UT) module -- Gaussian Kernel Density Estimation (G-KDE) clustering, non-parametric Gaussian Mixture (GM) Modeling, and Probability Integral Transform (PIT) -- to address irregular latent distributions. By reconfiguring irregular distributions into a uniform distribution in the latent space, our approach significantly enhances the disentanglement and interpretability of latent representations, overcoming the limitation of traditional VAE models in capturing complex data structures. Empirical evaluations demonstrated the efficacy of our proposed UT module in improving disentanglement metrics across benchmark datasets -- dSprites and MNIST. Our findings suggest a promising direction for advancing representation learning techniques, with implication for future research in extending this framework to more sophisticated datasets and downstream tasks. Comment: Accepted by 2024 IEEE 20th International Conference on Automation Science and Engineering
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2407.02681 Zobrazit plný text záznamu View this record from Arxiv