SwinFuSR: an image fusion-inspired model for RGB-guided thermal image super-resolution

Autor:	Arnold, Cyprien, Jouvet, Philippe, Seoud, Lama
Rok vydání:	2024
Předmět:	Electrical Engineering and Systems Science - Image and Video Processing Computer Science - Computer Vision and Pattern Recognition
Druh dokumentu:	Working Paper
Popis:	Thermal imaging plays a crucial role in various applications, but the inherent low resolution of commonly available infrared (IR) cameras limits its effectiveness. Conventional super-resolution (SR) methods often struggle with thermal images due to their lack of high-frequency details. Guided SR leverages information from a high-resolution image, typically in the visible spectrum, to enhance the reconstruction of a high-res IR image from the low-res input. Inspired by SwinFusion, we propose SwinFuSR, a guided SR architecture based on Swin transformers. In real world scenarios, however, the guiding modality (e.g. RBG image) may be missing, so we propose a training method that improves the robustness of the model in this case. Our method has few parameters and outperforms state of the art models in terms of Peak Signal to Noise Ratio (PSNR) and Structural SIMilarity (SSIM). In Track 2 of the PBVS 2024 Thermal Image Super-Resolution Challenge, it achieves 3rd place in the PSNR metric. Our code and pretained weights are available at https://github.com/VisionICLab/SwinFuSR. Comment: Accepted at 20th IEEE Workshop on Perception Beyond the Visible Spectrum, CVPR 2024
Databáze:	arXiv
Externí odkaz:	http://arxiv.org/abs/2404.14533 Zobrazit plný text záznamu View this record from Arxiv