GiGAN: Gate in GAN, could gate mechanism filter the features in image-to-image translation?

Autor: Haoxuan Ding, Edward K. Wong, Xuan Nie, Jianchao Jia
Rok vydání: 2021
Předmět:
Zdroj: Neurocomputing. 462:376-388
ISSN: 0925-2312
DOI: 10.1016/j.neucom.2021.07.085
Popis: Image-to-image translation techniques have been used in many different fields and have obtained remarkable performance in recent years. However, in many image-to-image translation tasks, only certain parts of the image need to be converted instead of the whole image. Traditional GAN-based methods often reconstruct the entire image, which may lead to artifacts and low-quality results. To address this issue, we propose a novel model, named GiGAN: Gate in GAN, which utilizes special Residual Blocks embedded with Gate Cells to filter and extract the features for facial attribute transfer and facial expression synthesis tasks. Specifically, we treat the intermediate feature from source domain to target domain as a sequence, and introduce the gate mechanism into this sequential task. To achieve this, we introduce the convolutional layers into gate cell and modify the stream in traditional gate cell to suit for image-to-image translation task. We designed two types of methods based on reusing parameters in the residual blocks or not, namely GiGAN-reuse and GiGAN-non-reuse. Experimental results and quantitative evaluations show that our model has superior performance against state-of-the-arts. And ablation studies demonstrate the effectiveness of our method. Furthermore, visualization of the features in Gate Cells shows that Gate Mechanism can filter the features in image-to-image translation effectively.
Databáze: OpenAIRE
Externí odkaz: