Improving Isochronous Machine Translation with Target Factors and Auxiliary Counters

Autor:	Pal, Proyag, Thompson, Brian, Virkar, Yogesh, Mathur, Prashant, Chronopoulou, Alexandra, Federico, Marcello
Rok vydání:	2023
Předmět:	Computer Science - Computation and Language Computer Science - Sound Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	To translate speech for automatic dubbing, machine translation needs to be isochronous, i.e. translated speech needs to be aligned with the source in terms of speech durations. We introduce target factors in a transformer model to predict durations jointly with target language phoneme sequences. We also introduce auxiliary counters to help the decoder to keep track of the timing information while generating target phonemes. We show that our model improves translation quality and isochrony compared to previous work where the translation model is instead trained to predict interleaved sequences of phonemes and durations. Comment: Accepted at INTERSPEECH 2023
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2305.13204 Zobrazit plný text záznamu View this record from Arxiv