Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation
| Autor: | Xingang Pan, Xiaohang Zhan, Bo Dai, Dahua Lin, Chen Change Loy, Ping Luo |
|---|---|
| Rok vydání: | 2022 |
| Předmět: |
FOS: Computer and information sciences
Computer Vision and Pattern Recognition (cs.CV) Applied Mathematics Image and Video Processing (eess.IV) ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION Computer Science - Computer Vision and Pattern Recognition Electrical Engineering and Systems Science - Image and Video Processing Computational Theory and Mathematics Artificial Intelligence FOS: Electrical engineering electronic engineering information engineering Image Processing Computer-Assisted Computer Vision and Pattern Recognition Algorithms Software |
| Zdroj: | IEEE Transactions on Pattern Analysis and Machine Intelligence. 44:7474-7489 |
| ISSN: | 1939-3539 0162-8828 |
| DOI: | 10.1109/tpami.2021.3115428 |
| Popis: | Learning a good image prior is a long-term goal for image restoration and manipulation. While existing methods like deep image prior (DIP) capture low-level image statistics, there are still gaps toward an image prior that captures rich image semantics including color, spatial coherence, textures, and high-level concepts. This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images. As shown in Fig.1, the deep generative prior (DGP) provides compelling results to restore missing semantics, e.g., color, patch, resolution, of various degraded images. It also enables diverse image manipulation including random jittering, image morphing, and category transfer. Such highly flexible restoration and manipulation are made possible through relaxing the assumption of existing GAN-inversion methods, which tend to fix the generator. Notably, we allow the generator to be fine-tuned on-the-fly in a progressive manner regularized by feature distance obtained by the discriminator in GAN. We show that these easy-to-implement and practical changes help preserve the reconstruction to remain in the manifold of nature image, and thus lead to more precise and faithful reconstruction for real images. Code is available at https://github.com/XingangPan/deep-generative-prior. Accepted to ECCV2020 as oral. 1) Precise GAN-inversion by discriminator-guided generator finetuning. 2) A versatile way for high-quality image restoration and manipulation. Code: https://github.com/XingangPan/deep-generative-prior |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|