Phase Space Reconstruction Driven Spatio-Temporal Feature Learning for Dynamic Facial Expression Recognition
| Autor: | shan wang, Qingshan Liu, Hui Shuai |
|---|---|
| Rok vydání: | 2022 |
| Předmět: |
Facial expression
business.industry Computer science Frame (networking) ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION 020207 software engineering Pattern recognition 02 engineering and technology Convolutional neural network Visualization Human-Computer Interaction Phase space 0202 electrical engineering electronic engineering information engineering Trajectory 020201 artificial intelligence & image processing Artificial intelligence Dynamical system (definition) business Feature learning Software |
| Zdroj: | IEEE Transactions on Affective Computing. 13:1466-1476 |
| ISSN: | 2371-9850 |
| Popis: | Automatic Dynamic Facial Expression Recognition (DFER) is challenging since how to effectively capture facial temporal dynamics is still an open problem. As the variations of facial expressions is a dynamic system that satisfies underlying rules, it is essential to explore the fundamental temporal properties for recognizing dynamic expressions. Inspired by the phase space reconstruction method for time series analysis, we propose a Phase Space Reconstruction Network (PSRNet) for learning spatio-temporal facial features. First, 3D convolutional neural networks are used to extract spatial and short-term features, which indicate each frame's state termed as observations in the phase space. All the observations compose the trajectory of the dynamical system. Then, a data-driven across-correlation matrix is inferred to reveal the relationship of the observations. With this matrix, the phase space reconstruction module reconstructs the trajectory by aggregating the observations adaptively. Reconstructed observations represent the gradual process of dynamic facial expressions, which is beneficial to recognize these expressions. The experiment results on Oulu, MMI, and CK+ demonstrate that PSRNet can extract more informative and representative spatio-temporal features for DFER. Moreover, the visualization reveals that the reconstructed features have global consistency in facial regions and find the underlying evolutionary pattern of dynamic facial expression. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|