Serialized Output Training by Learned Dominance

Autor:	Shi, Ying, Li, Lantian, Yin, Shi, Wang, Dong, Han, Jiqing
Rok vydání:	2024
Předmět:	Computer Science - Sound Computer Science - Artificial Intelligence Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	Serialized Output Training (SOT) has showcased state-of-the-art performance in multi-talker speech recognition by sequentially decoding the speech of individual speakers. To address the challenging label-permutation issue, prior methods have relied on either the Permutation Invariant Training (PIT) or the time-based First-In-First-Out (FIFO) rule. This study presents a model-based serialization strategy that incorporates an auxiliary module into the Attention Encoder-Decoder architecture, autonomously identifying the crucial factors to order the output sequence of the speech components in multi-talker speech. Experiments conducted on the LibriSpeech and LibriMix databases reveal that our approach significantly outperforms the PIT and FIFO baselines in both 2-mix and 3-mix scenarios. Further analysis shows that the serialization module identifies dominant speech components in a mixture by factors including loudness and gender, and orders speech components based on the dominance score. Comment: accepted by INTERSPEECH 2024
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2407.03966 Zobrazit plný text záznamu View this record from Arxiv