Personalized Lightweight Text-to-Speech: Voice Cloning with Adaptive Structured Pruning

Autor:	Huang, Sung-Feng, Chen, Chia-ping, Chen, Zhi-Sheng, Tsai, Yu-Pao, Lee, Hung-yi
Rok vydání:	2023
Předmět:	Computer Science - Sound Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	Personalized TTS is an exciting and highly desired application that allows users to train their TTS voice using only a few recordings. However, TTS training typically requires many hours of recording and a large model, making it unsuitable for deployment on mobile devices. To overcome this limitation, related works typically require fine-tuning a pre-trained TTS model to preserve its ability to generate high-quality audio samples while adapting to the target speaker's voice. This process is commonly referred to as ``voice cloning.'' Although related works have achieved significant success in changing the TTS model's voice, they are still required to fine-tune from a large pre-trained model, resulting in a significant size for the voice-cloned model. In this paper, we propose applying trainable structured pruning to voice cloning. By training the structured pruning masks with voice-cloning data, we can produce a unique pruned model for each target speaker. Our experiments demonstrate that using learnable structured pruning, we can compress the model size to 7 times smaller while achieving comparable voice-cloning performance. Comment: ICASSP 2023
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2303.11816 Zobrazit plný text záznamu View this record from Arxiv