GLFuse: A Global and Local Four-Branch Feature Extraction Network for Infrared and Visible Image Fusion

Autor: Genping Zhao, Zhuyong Hu, Silu Feng, Zhuowei Wang, Heng Wu
Jazyk: angličtina
Rok vydání: 2024
Předmět:
Zdroj: Remote Sensing, Vol 16, Iss 17, p 3246 (2024)
Druh dokumentu: article
ISSN: 2072-4292
DOI: 10.3390/rs16173246
Popis: Infrared and visible image fusion integrates complementary information from different modalities into a single image, providing sufficient imaging information for scene interpretation and downstream target recognition tasks. However, existing fusion methods often focus only on highlighting salient targets or preserving scene details, failing to effectively combine entire features from different modalities during the fusion process, resulting in underutilized features and poor overall fusion effects. To address these challenges, a global and local four-branch feature extraction image fusion network (GLFuse) is proposed. On one hand, the Super Token Transformer (STT) block, which is capable of rapidly sampling and predicting super tokens, is utilized to capture global features in the scene. On the other hand, a Detail Extraction Block (DEB) is developed to extract local features in the scene. Additionally, two feature fusion modules, namely the Attention-based Feature Selection Fusion Module (ASFM) and the Dual Attention Fusion Module (DAFM), are designed to facilitate selective fusion of features from different modalities. Of more importance, the various perceptual information of feature maps learned from different modality images at the different layers of a network is investigated to design a perceptual loss function to better restore scene detail information and highlight salient targets by treating the perceptual information separately. Extensive experiments confirm that GLFuse exhibits excellent performance in both subjective and objective evaluations. It deserves note that GLFuse effectively improves downstream target detection performance on a unified benchmark.
Databáze: Directory of Open Access Journals
Nepřihlášeným uživatelům se plný text nezobrazuje