| Autor: |
Syed Aun Muhammad Zaidi, Siddique Latif, Junaid Qadir |
| Jazyk: |
angličtina |
| Rok vydání: |
2024 |
| Předmět: |
|
| Zdroj: |
IEEE Open Journal of the Computer Society, Vol 5, Pp 684-693 (2024) |
| Druh dokumentu: |
article |
| ISSN: |
2644-1268 |
| DOI: |
10.1109/OJCS.2024.3486904 |
| Popis: |
Despite the recent progress in emotion recognition, state-of-the-art systems are unable to achieve improved performance in cross-language settings. In this article we propose a Multimodal Dual Attention Transformer (MDAT) model to improve cross-language multimodal emotion recognition. Our model utilises pre-trained models for multimodal feature extraction and is equipped with dual attention mechanisms including graph attention and co-attention to capture complex dependencies across different modalities and languages to achieve improved cross-language multimodal emotion recognition. In addition, our model also exploits a transformer encoder layer for high-level feature representation to improve emotion classification accuracy. This novel construct preserves modality-specific emotional information while enhancing cross-modality and cross-language feature generalisation, resulting in improved performance with minimal target language data. We assess our model's performance on four publicly available emotion recognition datasets and establish its superior effectiveness compared to recent approaches and baseline models. |
| Databáze: |
Directory of Open Access Journals |
| Externí odkaz: |
|
