Speech Emotion Recognition using Self-Supervised Features

Autor:	Morais, Edmilson, Hoory, Ron, Zhu, Weizhong, Gat, Itai, Damasceno, Matheus, Aronowitz, Hagai
Rok vydání:	2022
Předmět:	Computer Science - Sound Computer Science - Artificial Intelligence Computer Science - Machine Learning Electrical Engineering and Systems Science - Audio and Speech Processing
Druh dokumentu:	Working Paper
Popis:	Self-supervised pre-trained features have consistently delivered state-of-art results in the field of natural language processing (NLP); however, their merits in the field of speech emotion recognition (SER) still need further investigation. In this paper we introduce a modular End-to- End (E2E) SER system based on an Upstream + Downstream architecture paradigm, which allows easy use/integration of a large variety of self-supervised features. Several SER experiments for predicting categorical emotion classes from the IEMOCAP dataset are performed. These experiments investigate interactions among fine-tuning of self-supervised feature models, aggregation of frame-level features into utterance-level features and back-end classification networks. The proposed monomodal speechonly based system not only achieves SOTA results, but also brings light to the possibility of powerful and well finetuned self-supervised acoustic features that reach results similar to the results achieved by SOTA multimodal systems using both Speech and Text modalities. Comment: 5 pages, 4 figures, 2 tables, ICASSP 2022
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2202.03896 Zobrazit plný text záznamu View this record from Arxiv