3D Visual Speech Animation Using 2D Videos
| Autor: | Steve Maddock, Yoshihiko Gotoh, Rabab Algadhy |
|---|---|
| Rok vydání: | 2019 |
| Předmět: |
Computer science
Head (linguistics) Speech recognition Foreign language 0202 electrical engineering electronic engineering information engineering 020207 software engineering 020201 artificial intelligence & image processing 02 engineering and technology Animation Motion (physics) Speech animation Visualization |
| Zdroj: | ICASSP |
| DOI: | 10.1109/icassp.2019.8682455 |
| Popis: | In visual speech animation, lip motion accuracy is of paramount importance for speech intelligibility, especially for the hard of hearing or foreign language learners. We present an approach for visual speech animation that uses tracked lip motion in front-view 2D videos of a real speaker to drive the lip motion of a synthetic 3D head. This makes use of a 3D morphable model (3DMM), built using 3D synthetic head poses, with corresponding landmarks identified in the 2D videos and the 3DMM. We show that using a wider range of synthetic head poses for different phoneme intensities to create a 3DMM, as well as a combination of front and side photographs of the real speakers rather than just front photographs to produce initial neutral 3D synthetic head poses, gives better animation results when compared to ground truth data consisting of front-view 2D videos of real speakers. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|