Optimizing Bilingual Neural Transducer with Synthetic Code-switching Text Generation

Autor:	Nguyen, Thien, Tran, Nathalie, Deng, Liuhui, da Silva, Thiago Fraga, Radzihovsky, Matthew, Hsiao, Roger, Mason, Henry, Braun, Stefan, McDermott, Erik, Can, Dogan, Swietojanski, Pawel, Verwimp, Lyan, Oyman, Sibel, Arvizo, Tresi, Silovsky, Honza, Ghoshal, Arnab, Martel, Mathieu, Ambati, Bharat Ram, Ali, Mohamed
Rok vydání:	2022
Předmět:	Computer Science - Sound Computer Science - Computation and Language Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	Code-switching describes the practice of using more than one language in the same sentence. In this study, we investigate how to optimize a neural transducer based bilingual automatic speech recognition (ASR) model for code-switching speech. Focusing on the scenario where the ASR model is trained without supervised code-switching data, we found that semi-supervised training and synthetic code-switched data can improve the bilingual ASR system on code-switching speech. We analyze how each of the neural transducer's encoders contributes towards code-switching performance by measuring encoder-specific recall values, and evaluate our English/Mandarin system on the ASCEND data set. Our final system achieves 25% mixed error rate (MER) on the ASCEND English/Mandarin code-switching test set -- reducing the MER by 2.1% absolute compared to the previous literature -- while maintaining good accuracy on the monolingual test sets. Comment: 5 pages, 1 figure, submitted to ICASSP 2023, *: equal contributions
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2210.12214 Zobrazit plný text záznamu View this record from Arxiv