Semantically consistent text to fashion image synthesis with an enhanced attentional generative adversarial network
| Autor: | Ashraf A. Kassim, Joo-Hwee Lim, Kenan E. Ak, Jo Yew Tham |
|---|---|
| Rok vydání: | 2020 |
| Předmět: |
Normalization (statistics)
Computer science business.industry Stability (learning theory) Normalization (image processing) Context (language use) Pattern recognition 02 engineering and technology 01 natural sciences Artificial Intelligence Feature (computer vision) 0103 physical sciences Signal Processing Similarity (psychology) 0202 electrical engineering electronic engineering information engineering 020201 artificial intelligence & image processing Computer Vision and Pattern Recognition Artificial intelligence Noise (video) 010306 general physics business Software Natural language Similarity learning Sentence |
| Zdroj: | Pattern Recognition Letters. 135:22-29 |
| ISSN: | 0167-8655 |
| DOI: | 10.1016/j.patrec.2020.02.030 |
| Popis: | Recent advancements in Generative Adversarial Networks (GANs) have led to significant improvements in various image generation tasks including image synthesis based on text descriptions. In this paper, we present an enhanced Attentional Generative Adversarial Network (e-AttnGAN) with improved training stability for text-to-image synthesis. e-AttnGAN’s integrated attention module utilizes both sentence and word context features and performs feature-wise linear modulation (FiLM) to fuse visual and natural language representations. In addition to multimodal similarity learning for text and image features of AttnGAN [1], similarity and feature matching losses between real and generated images are included while employing classification losses for “significant attributes”. In order to improve the stability of the training and solve the mode collapse issue, spectral normalization and two-time scale update rule are used for the discriminator together with instance noise. Our experiments show that e-AttnGAN outperforms state-of-the-art methods using the FashionGen and DeepFashion-Synthesis datasets in terms of inception score, R-precision and classification accuracy. A detailed ablation study has been conducted to observe the effect of each component. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|
Full Text from ScienceDirect