Emotion Dependent Facial Animation from Affective Speech

Autor: Sasan Asadiabadi, Engin Erzin, Rizwan Sadiq
Přispěvatelé: Sadiq, Rizwan, Asadiabadi, Sasan, Erzin, Engin (ORCID 0000-0002-2715-2368 & YÖK ID 34503), Graduate School of Sciences and Engineering, College of Engineering, Department of Electrical and Electronics Engineering, Department of Computer Engineering
Rok vydání: 2020
Předmět:
FOS: Computer and information sciences
Sound (cs.SD)
Mean squared error
Speech recognition
ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION
02 engineering and technology
Universal model
Computer Science - Sound
Audio and Speech Processing (eess.AS)
FOS: Electrical engineering
electronic engineering
information engineering
0202 electrical engineering
electronic engineering
information engineering
Computer facial animation
ComputingMethodologies_COMPUTERGRAPHICS
Landmark
business.industry
Deep learning
Estimator
020206 networking & telecommunications
020207 software engineering
Animation
Multimedia (cs.MM)
Multimedia signal processing
Training
Visualization
Shape
Training data
Facial animation
Speech processing
Artificial intelligence
business
Psychology
Computer Science - Multimedia
Electrical Engineering and Systems Science - Audio and Speech Processing
Zdroj: MMSP
2020 IEEE 22nd International Workshop on Multimedia Signal Processing (MMSP)
DOI: 10.1109/mmsp48831.2020.9287086
Popis: In human-to-computer interaction, facial animation in synchrony with affective speech can deliver more naturalistic conversational agents. In this paper, we present a two-stage deep learning approach for affective speech driven facial shape animation. In the first stage, we classify affective speech into seven emotion categories. In the second stage, we train separate deep estimators within each emotion category to synthesize facial shape from the affective speech. Objective and subjective evaluations are performed over the SAVEE dataset. The proposed emotion dependent facial shape model performs better in terms of the Mean Squared Error (MSE) loss and in generating the landmark animations, as compared to training a universal model regardless of the emotion.
Scientific and Technological Research Council of Turkey (TÜBİTAK)
Databáze: OpenAIRE
Externí odkaz: