Fine-Tuned Self-Supervised Speech Representations for Language Diarization in Multilingual Code-Switched Speech

Autor:	Frost, Geoffrey, Morris, Emily, van Vüren, Joshua Jansen, Niesler, Thomas
Rok vydání:	2023
Předmět:	Electrical Engineering and Systems Science - Audio and Speech Processing Computer Science - Artificial Intelligence Computer Science - Sound
Druh dokumentu:	Working Paper
DOI:	10.1007/978-3-031-22321-1_17
Popis:	Annotating a multilingual code-switched corpus is a painstaking process requiring specialist linguistic expertise. This is partly due to the large number of language combinations that may appear within and across utterances, which might require several annotators with different linguistic expertise to consider an utterance sequentially. This is time-consuming and costly. It would be useful if the spoken languages in an utterance and the boundaries thereof were known before annotation commences, to allow segments to be assigned to the relevant language experts in parallel. To address this, we investigate the development of a continuous multilingual language diarizer using fine-tuned speech representations extracted from a large pre-trained self-supervised architecture (WavLM). We experiment with a code-switched corpus consisting of five South African languages (isiZulu, isiXhosa, Setswana, Sesotho and English) and show substantial diarization error rate improvements for language families, language groups, and individual languages over baseline systems. Comment: Presented at SACAIR 2022
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2312.09645 Zobrazit plný text záznamu View this record from Arxiv