Practical Techniques for Vision-Language Segmentation Model in Remote Sensing
| Autor: | Y. Lin, K. Suzuki, S. Sogo |
|---|---|
| Jazyk: | angličtina |
| Rok vydání: | 2024 |
| Předmět: | |
| Zdroj: | The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol XLVIII-2-2024, Pp 203-210 (2024) |
| Druh dokumentu: | article |
| ISSN: | 1682-1750 2194-9034 |
| DOI: | 10.5194/isprs-archives-XLVIII-2-2024-203-2024 |
| Popis: | Traditional semantic segmentation models often struggle with poor generalizability in zero-shot scenarios such as recognizing attributes unseen in the training labels. On the other hands, language-vision models (VLMs) have shown promise in improving performance on zero-shot tasks by leveraging semantic information from textual inputs and fusing this information with visual features. However, existing VLM-based methods do not perform as effectively on remote sensing data due to the lack of such data in their training datasets. In this paper, we introduce a two-stage fine-tuning approach for a VLM-based segmentation model using a large remote sensing image-caption dataset, which we created using an existing image-caption model. Additionally, we propose a modified decoder and a visual prompt technique using a saliency map to enhance segmentation results. Through these methods, we achieve superior segmentation performance on remote sensing data, demonstrating the effectiveness of our approach. |
| Databáze: | Directory of Open Access Journals |
| Externí odkaz: |
|