Modeling Hierarchical Uncertainty for Multimodal Emotion Recognition in Conversation

Autor:	Feiyu Chen, Jie Shao, Anjie Zhu, Deqiang Ouyang, Xueliang Liu, Heng Tao Shen
Rok vydání:	2022
Předmět:	Human-Computer Interaction Control and Systems Engineering Electrical and Electronic Engineering Software Computer Science Applications Information Systems
Zdroj:	IEEE Transactions on Cybernetics. :1-12
ISSN:	2168-2275 2168-2267
DOI:	10.1109/tcyb.2022.3185119
Popis:	Approximating the uncertainty of an emotional AI agent is crucial for improving the reliability of such agents and facilitating human-in-the-loop solutions, especially in critical scenarios. However, none of the existing systems for emotion recognition in conversation (ERC) has attempted to estimate the uncertainty of their predictions. In this article, we present HU-Dialogue, which models hierarchical uncertainty for the ERC task. We perturb contextual attention weight values with source-adaptive noises within each modality, as a regularization scheme to model context-level uncertainty and adapt the Bayesian deep learning method to the capsule-based prediction layer to model modality-level uncertainty. Furthermore, a weight-sharing triplet structure with conditional layer normalization is introduced to detect both invariance and equivariance among modalities for ERC. We provide a detailed empirical analysis for extensive experiments, which shows that our model outperforms previous state-of-the-art methods on three popular multimodal ERC datasets.
Databáze:	OpenAIRE
Externí odkaz:	https://explore.openaire.eu/search/publication?articleId=doi_dedup___::69b566cb6bba591f4a98846d27054787 https://doi.org/10.1109/tcyb.2022.3185119 Zobrazit plný text záznamu