Efficient Incremental Text-to-Speech on GPUs

Autor:	Du, Muyang, Liu, Chuan, Qi, Jiaxing, Lai, Junjie
Rok vydání:	2022
Předmět:	Computer Science - Sound Computer Science - Machine Learning Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	Incremental text-to-speech, also known as streaming TTS, has been increasingly applied to online speech applications that require ultra-low response latency to provide an optimal user experience. However, most of the existing speech synthesis pipelines deployed on GPU are still non-incremental, which uncovers limitations in high-concurrency scenarios, especially when the pipeline is built with end-to-end neural network models. To address this issue, we present a highly efficient approach to perform real-time incremental TTS on GPUs with Instant Request Pooling and Module-wise Dynamic Batching. Experimental results demonstrate that the proposed method is capable of producing high-quality speech with a first-chunk latency lower than 80ms under 100 QPS on a single NVIDIA A10 GPU and significantly outperforms the non-incremental twin in both concurrency and latency. Our work reveals the effectiveness of high-performance incremental TTS on GPUs. Comment: 5 pages, 4 figures
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2211.13939 Zobrazit plný text záznamu View this record from Arxiv