On the Effectiveness of Acoustic BPE in Decoder-Only TTS

Autor:	Li, Bohan, Shen, Feiyu, Guo, Yiwei, Wang, Shuai, Chen, Xie, Yu, Kai
Rok vydání:	2024
Předmět:	Computer Science - Sound Computer Science - Artificial Intelligence Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	Discretizing speech into tokens and generating them by a decoder-only model have been a promising direction for text-to-speech (TTS) and spoken language modeling (SLM). To shorten the sequence length of speech tokens, acoustic byte-pair encoding (BPE) has emerged in SLM that treats speech tokens from self-supervised semantic representations as characters to further compress the token sequence. But the gain in TTS has not been fully investigated, and the proper choice of acoustic BPE remains unclear. In this work, we conduct a comprehensive study on various settings of acoustic BPE to explore its effectiveness in decoder-only TTS models with semantic speech tokens. Experiments on LibriTTS verify that acoustic BPE uniformly increases the intelligibility and diversity of synthesized speech, while showing different features across BPE settings. Hence, acoustic BPE is a favorable tool for decoder-only TTS. Comment: 5 pages, 3 tables, 1 figures. accepted to Interspeech 2024
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2407.03892 Zobrazit plný text záznamu View this record from Arxiv