Designing CNNs for Multimodal Image Super-Resolution via the Method of Multipliers
| Autor: | Bruno Cornelis, Iman Marivani, Evaggelia Tsiligianni, Nikos Deligiannis |
|---|---|
| Přispěvatelé: | Electronics and Informatics, Faculty of Engineering |
| Rok vydání: | 2021 |
| Předmět: |
Artificial neural network
Computer science business.industry Deep learning ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION 020206 networking & telecommunications Pattern recognition 02 engineering and technology Iterative reconstruction Convolutional neural network Superresolution Computer Science::Computer Vision and Pattern Recognition 0202 electrical engineering electronic engineering information engineering RGB color model 020201 artificial intelligence & image processing Artificial intelligence business Neural coding MM algorithm |
| Zdroj: | EUSIPCO Vrije Universiteit Brussel |
| DOI: | 10.23919/eusipco47968.2020.9287361 |
| Popis: | Multimodal alias, guided, image super-resolution (SR) refers to the reconstruction of a high-resolution (HR) version of a low-resolution (LR) image with the aid of an HR image from another image modality. Common approaches for the SR problem include analytical methods which are computationally expensive. Deep learning methods are capable of learning a nonlinear mapping between LR and HR images from data, delivering high reconstruction accuracy at a low-computational cost during inference; however, these methods do not incorporate any prior knowledge about the problem, with the neural network model behaving like a black box. In this paper, we formulate multimodal image SR as a coupled convolutional sparse coding problem. To solve the corresponding minimization problem, we adopt the Method of Multipliers (MM). We then design a convolutional neural network (CNN) that unfolds the obtained MM algorithm. The proposed CNN accepts as input the LR image from the main modality and the HR image from the guidance modality to reconstruct the desired HR image. Unlike existing deep learning methods, our CNN provides an efficient and structured way to fuse information at different stages of the network and achieves high reconstruction accuracy. We evaluate the performance of the proposed model for the super-resolution of multi-spectral images guided by their high resolution RGB counterparts. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|