| Autor: |
Vazquez-Rodriguez, Juan, Lefebvre, Grégoire, Cumin, Julien, Crowley, James L. |
| Rok vydání: |
2023 |
| Předmět: |
|
| Zdroj: |
Affective Computing and Intelligent Interaction (ACII), Sep 2023, Cambridge (MA), United States |
| Druh dokumentu: |
Working Paper |
| Popis: |
Decades of research indicate that emotion recognition is more effective when drawing information from multiple modalities. But what if some modalities are sometimes missing? To address this problem, we propose a novel Transformer-based architecture for recognizing valence and arousal in a time-continuous manner even with missing input modalities. We use a coupling of cross-attention and self-attention mechanisms to emphasize relationships between modalities during time and enhance the learning process on weak salient inputs. Experimental results on the Ulm-TSST dataset show that our model exhibits an improvement of the concordance correlation coefficient evaluation of 37% when predicting arousal values and 30% when predicting valence values, compared to a late-fusion baseline approach. |
| Databáze: |
arXiv |
| Externí odkaz: |
|
