GANimation: One-Shot Anatomically Consistent Facial Animation
| Autor: | Francesc Moreno-Noguer, Aleix M. Martinez, Alberto Sanfeliu, Antonio Agudo, Albert Pumarola |
|---|---|
| Přispěvatelé: | Agencia Estatal de Investigación (España), Amazon, Ministerio de Economía y Competitividad (España), Ministerio de Ciencia, Innovación y Universidades (España), European Commission, National Institutes of Health (US), Universitat Politècnica de Catalunya. Doctorat en Automàtica, Robòtica i Visió, Institut de Robòtica i Informàtica Industrial, Universitat Politècnica de Catalunya. Departament d'Enginyeria de Sistemes, Automàtica i Informàtica Industrial, Universitat Politècnica de Catalunya. VIS - Visió Artificial i Sistemes Intel·ligents, Universitat Politècnica de Catalunya. ROBiri - Grup de Robòtica de l'IRI |
| Rok vydání: | 2019 |
| Předmět: |
Facial expression
Informàtica::Automàtica i control [Àrees temàtiques de la UPC] business.industry Computer science Pattern recognition Face animation 02 engineering and technology Action-unit condition Expression (mathematics) Pattern recognition [Classificació INSPEC] Domain (software engineering) GAN Task (computing) Artificial Intelligence Pattern recognition (psychology) 0202 electrical engineering electronic engineering information engineering Code (cryptography) 020201 artificial intelligence & image processing Computer Vision and Pattern Recognition Artificial intelligence business Set (psychology) Software Computer facial animation |
| Zdroj: | UPCommons. Portal del coneixement obert de la UPC Universitat Politècnica de Catalunya (UPC) Digital.CSIC. Repositorio Institucional del CSIC instname |
| ISSN: | 1573-1405 0920-5691 |
| Popis: | Recent advances in generative adversarial networks (GANs) have shown impressive results for the task of facial expression synthesis. The most successful architecture is StarGAN (Choi et al. in CVPR, 2018), that conditions GANs’ generation process with images of a specific domain, namely a set of images of people sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content and granularity of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on action units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combining several of them. Additionally, we propose a weakly supervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit a novel self-learned attention mechanism that makes our network robust to changing backgrounds, lighting conditions and occlusions. Extensive evaluation shows that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild. The code of this work is publicly available at https://github.com/albertpumarola/GANimation. This work is partially supported by an Amazon Research Award, by the Spanish Ministry of Economy and Competitiveness under Projects HuMoUR TIN2017-90086-R, ColRobTransp DPI2016-78957 and María de Maeztu Seal of Excellence MDM-2016-0656; by the EU Project AEROARMS ICT-2014-1-644271; and by the Grant R01-DC- 014498 of the National Institute of Health. We also thank Nvidia for hardware donation under the GPU Grant Program. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full text from SpringerLink