RAQ-VAE: Rate-Adaptive Vector-Quantized Variational Autoencoder

Autor:	Seo, Jiwan, Kang, Joonhyuk
Rok vydání:	2024
Předmět:	Computer Science - Machine Learning Computer Science - Computer Vision and Pattern Recognition Electrical Engineering and Systems Science - Image and Video Processing
Druh dokumentu:	Working Paper
Popis:	Vector Quantized Variational AutoEncoder (VQ-VAE) is an established technique in machine learning for learning discrete representations across various modalities. However, its scalability and applicability are limited by the need to retrain the model to adjust the codebook for different data or model scales. We introduce the Rate-Adaptive VQ-VAE (RAQ-VAE) framework, which addresses this challenge with two novel codebook representation methods: a model-based approach using a clustering-based technique on an existing well-trained VQ-VAE model, and a data-driven approach utilizing a sequence-to-sequence (Seq2Seq) model for variable-rate codebook generation. Our experiments demonstrate that RAQ-VAE achieves effective reconstruction performance across multiple rates, often outperforming conventional fixed-rate VQ-VAE models. This work enhances the adaptability and performance of VQ-VAEs, with broad applications in data reconstruction, generation, and computer vision tasks. Comment: Under review
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2405.14222 Zobrazit plný text záznamu View this record from Arxiv