Recognition of facial expressions based on CNN features

Autor: Luis Pellegrin, Hugo Jair Escalante, Sonia M. González-Lozoya, Ma. Auxilio Medina, Jorge de la Calleja, Antonio Benitez-Ruiz
Rok vydání: 2020
Předmět:
Zdroj: Multimedia Tools and Applications. 79:13987-14007
ISSN: 1573-7721
1380-7501
DOI: 10.1007/s11042-020-08681-4
Popis: Facial expressions are a natural way to communicate emotional states and intentions. In recent years, automatic facial expression recognition (FER) has been studied due to its practical importance in many human-behavior analysis tasks such as interviews, autonomous-driving, medical treatment, among others. In this paper we propose a method for facial expression recognition based on features extracted with convolutional neural networks (CNN), taking advantage of a pre-trained model in similar tasks. Unlike other approaches, the proposed FER method learns from mixed instances taken from different databases with the goal of improving generalization, a major issue in machine learning. Experimental results show that the FER method is able to recognize the six universal expressions with an accuracy above 92% considering five of the widely used databases. In addition, we have extended our method to deal with micro-expressions recognition (MER). In this regard, we propose three strategies to create a temporal-aggregated feature vector: mean, standard deviation and early fusion. In this case, the best result is 78.80% accuracy. Furthermore, we present a prototype system that implements the two proposed methods for FER and MER as a tool that allows to analyze videos.
Databáze: OpenAIRE