On the Effectiveness of ASR Representations in Real-world Noisy Speech Emotion Recognition

Autor:	Shi, Xiaohan, He, Jiajun, Li, Xingfeng, Toda, Tomoki
Rok vydání:	2023
Předmět:	Computer Science - Sound Computer Science - Computation and Language Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	This paper proposes an efficient attempt to noisy speech emotion recognition (NSER). Conventional NSER approaches have proven effective in mitigating the impact of artificial noise sources, such as white Gaussian noise, but are limited to non-stationary noises in real-world environments due to their complexity and uncertainty. To overcome this limitation, we introduce a new method for NSER by adopting the automatic speech recognition (ASR) model as a noise-robust feature extractor to eliminate non-vocal information in noisy speech. We first obtain intermediate layer information from the ASR model as a feature representation for emotional speech and then apply this representation for the downstream NSER task. Our experimental results show that 1) the proposed method achieves better NSER performance compared with the conventional noise reduction method, 2) outperforms self-supervised learning approaches, and 3) even outperforms text-based approaches using ASR transcription or the ground truth transcription of noisy speech. Comment: Submitted to ICASSP 2024
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2311.07093 Zobrazit plný text záznamu View this record from Arxiv