Boosting Multi-Speaker Expressive Speech Synthesis with Semi-supervised Contrastive Learning

Autor:	Zhu, Xinfa, Li, Yuke, Lei, Yi, Jiang, Ning, Zhao, Guoqing, Xie, Lei
Rok vydání:	2023
Předmět:	Electrical Engineering and Systems Science - Audio and Speech Processing Computer Science - Sound
Druh dokumentu:	Working Paper
Popis:	This paper aims to build a multi-speaker expressive TTS system, synthesizing a target speaker's speech with multiple styles and emotions. To this end, we propose a novel contrastive learning-based TTS approach to transfer style and emotion across speakers. Specifically, contrastive learning from different levels, i.e. utterance and category level, is leveraged to extract the disentangled style, emotion, and speaker representations from speech for style and emotion transfer. Furthermore, a semi-supervised training strategy is introduced to improve the data utilization efficiency by involving multi-domain data, including style-labeled data, emotion-labeled data, and abundant unlabeled data. To achieve expressive speech with diverse styles and emotions for a target speaker, the learned disentangled representations are integrated into an improved VITS model. Experiments on multi-domain data demonstrate the effectiveness of the proposed method. Comment: 6 pages, 4 figures; Accepted by ICME 2024
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2310.17101 Zobrazit plný text záznamu View this record from Arxiv