Speaker-Independent Acoustic-to-Articulatory Inversion through Multi-Channel Attention Discriminator

Autor:	Chung, Woo-Jin, Kang, Hong-Goo
Rok vydání:	2024
Předmět:	Electrical Engineering and Systems Science - Signal Processing Computer Science - Sound Electrical Engineering and Systems Science - Audio and Speech Processing Physics - Biological Physics
Druh dokumentu:	Working Paper
Popis:	We present a novel speaker-independent acoustic-to-articulatory inversion (AAI) model, overcoming the limitations observed in conventional AAI models that rely on acoustic features derived from restricted datasets. To address these challenges, we leverage representations from a pre-trained self-supervised learning (SSL) model to more effectively estimate the global, local, and kinematic pattern information in Electromagnetic Articulography (EMA) signals during the AAI process. We train our model using an adversarial approach and introduce an attention-based Multi-duration phoneme discriminator (MDPD) designed to fully capture the intricate relationship among multi-channel articulatory signals. Our method achieves a Pearson correlation coefficient of 0.847, marking state-of-the-art performance in speaker-independent AAI models. The implementation details and code can be found online. Comment: Accepted to INTERSPEECH 2024
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2406.17329 Zobrazit plný text záznamu View this record from Arxiv