Improving Multi-Speaker TTS Prosody Variance with a Residual Encoder and Normalizing Flows
| Autor: | Roberto Barra-Chicote, Jasha Droppo, Grzegorz Beringer, Julian Roth, Iván Vallés-Pérez |
|---|---|
| Rok vydání: | 2021 |
| Předmět: |
FOS: Computer and information sciences
Sound (cs.SD) Computer Science - Computation and Language Computer science Speech recognition Inference Intelligibility (communication) Computer Science - Sound Variable (computer science) Naturalness Audio and Speech Processing (eess.AS) FOS: Electrical engineering electronic engineering information engineering Optimal distinctiveness theory Prosody Set (psychology) Computation and Language (cs.CL) Encoder Electrical Engineering and Systems Science - Audio and Speech Processing |
| Zdroj: | Interspeech 2021. |
| DOI: | 10.21437/interspeech.2021-562 |
| Popis: | Text-to-speech systems recently achieved almost indistinguishable quality from human speech. However, the prosody of those systems is generally flatter than natural speech, producing samples with low expressiveness. Disentanglement of speaker id and prosody is crucial in text-to-speech systems to improve on naturalness and produce more variable syntheses. This paper proposes a new neural text-to-speech model that approaches the disentanglement problem by conditioning a Tacotron2-like architecture on flow-normalized speaker embeddings, and by substituting the reference encoder with a new learned latent distribution responsible for modeling the intra-sentence variability due to the prosody. By removing the reference encoder dependency, the speaker-leakage problem typically happening in this kind of systems disappears, producing more distinctive syntheses at inference time. The new model achieves significantly higher prosody variance than the baseline in a set of quantitative prosody features, as well as higher speaker distinctiveness, without decreasing the speaker intelligibility. Finally, we observe that the normalized speaker embeddings enable much richer speaker interpolations, substantially improving the distinctiveness of the new interpolated speakers. in Proceedings of Interspeech 2021 conference |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|